Compare commits

...
39 Commits
Author SHA1 Message Date
enricobuehlerandClaude Fable 5 c90343c22f feat(android): shared clipboard (text) — device↔host sync while streaming
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m13s
apple / swift (push) Successful in 1m21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
decky / build-publish (push) Successful in 33s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 8m18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m49s
deb / build-publish (push) Successful in 10m26s
release / apple (push) Successful in 9m22s
docker / deploy-docs (push) Successful in 28s
deb / build-publish-host (push) Successful in 12m44s
arch / build-publish (push) Successful in 15m51s
windows-host / package (push) Successful in 16m14s
apple / screenshots (push) Successful in 6m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m6s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m18s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
ci / rust (push) Successful in 24m48s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m21s
flatpak / build-publish (push) In progress
android / android (push) Failing after 40s
The desktop clients' clipboard protocol, adopted on Android (text-only v1):
opt-in ClipControl at stream start when the host advertises HOST_CAP_CLIPBOARD
and the new "Shared clipboard" setting (default on) allows.

Device → host: local copies (primary-clip listener + a probe at start) are
announced as lazy format-list offers; the text crosses only when the host
actually pastes (FetchRequest → served from the live clipboard). Host →
device: a host copy's offer is fetched eagerly and lands in the system
clipboard (Android has no practical lazy-paste provider), with an echo guard
so the resulting clip-changed callback doesn't bounce it back as a new offer.

Native side: session/clipboard.rs JNI shims over NativeClient's clip_*
surface; events cross to Kotlin as compact strings from a blocking
nativeNextClip poll on a dedicated thread (the nativeNextRumble pattern),
joined before the session handle is freed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 19:27:19 +02:00
enricobuehlerandClaude Fable 5 abe6228b42 feat(core+host+android): committed-text input plane (IME path) — InputKind::TextInput
The VK key-event vocabulary cannot express text an input method COMMITS
(autocorrect, gesture typing, non-Latin scripts, emoji). Add a first-class
text event and negotiate it:

- punktfunk-core: InputKind::TextInput (= 15) carries one Unicode scalar per
  event in `code`; HOST_CAP_TEXT_INPUT (0x04) in Welcome::host_caps.
- Host advertises the cap only where the session's inject backend can type
  text: Windows SendInput (KEYEVENTF_UNICODE, surrogate-pair aware) and the
  Linux wlroots backend — a dedicated second zwp_virtual_keyboard whose xkb
  keymap grows Unicode keysyms on demand (the wtype model), so keymap
  re-uploads never disturb the main device's layout/modifier state. The
  KWin-fake-input/libei/gamescope backends can only press layout keycodes, so
  those sessions don't set the bit and clients keep the VK fallback.
- GameStream plane: Moonlight's UTF-8 text packet (MAGIC_UTF8, previously
  recognized-and-dropped) now decodes to the same TextInput events.
- Android: KeyCaptureView picks a real editable InputConnection when the host
  has the cap — the IME runs its full machinery, mirrored to the host live via
  common-prefix diffs of the composition (backspaces + new suffix), with
  setComposingRegion adopting committed text so autocorrect-revert flows diff
  instead of retyping; newline→Enter, deleteSurroundingText→Backspace/Delete.
  Older hosts keep the TYPE_NULL raw-key path unchanged.
- keymap: media VKs (0xB0-0xB3) → evdev so the Android media keys land on
  Linux hosts too.

Verified: punktfunk-core + host gamestream + pf-inject tests green on Linux
(Ubuntu box), clippy clean; Android app+native builds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 19:19:11 +02:00
enricobuehlerandClaude Fable 5 ddb72edcba feat(android): physical mouse + TV remote-as-pointer + keyboard hardening
Phase A — hardware mouse: new MouseForwarder routes every SOURCE_MOUSE event
while streaming. Uncaptured (default): hover/drag positions forward as absolute
cursor moves, wheel with fractional accumulation (high-res wheels), all five
buttons incl. back/forward as X1/X2; the local cursor is hidden over the stream
(TYPE_NULL pointer icon) so the host's composited cursor is the visible one.
Captured ("Capture pointer for games" setting, or Ctrl+Alt+Shift+Q live): the
OS pointer is grabbed via requestPointerCapture on the (repurposed) key-capture
view and raw relative deltas forward as mouse-look; held buttons are tracked
and flushed on capture loss/teardown, and the engaging click is swallowed
(desktop parity). Mouse clicks are intercepted before Compose so they can never
be misread as trackpad taps; a mouse back-button's FALLBACK BACK is dropped so
it can't yank the user out of the stream.

Phase B — keyboard: media/consumer keys (play/pause/next/prev/stop) now map to
their VKs instead of falling to the system; TV-remote SELECT maps to Enter.

Phase D — Android TV: RemotePointer turns a D-pad-only remote into a pointer
(Siri-remote analogue): hold SELECT ~0.8s toggles, D-pad glides the host cursor
with ramped acceleration (Choreographer-paced, diagonal-normalized), SELECT
taps click, play/pause right-clicks, holding it toggles the on-screen keyboard,
BACK leaves pointer mode; an overlay hint teaches the vocabulary.

Settings: "Capture pointer for games" + "Invert scroll direction" (applies
to the wheel and two-finger touch scrolling).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 18:56:38 +02:00
enricobuehlerandClaude Fable 5 5e088af7ed fix(pf-win-display): unpin mode indexes on paths the CCD isolate deactivates
ci / web (push) Successful in 49s
apple / swift (push) Successful in 1m24s
ci / docs-site (push) Successful in 55s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
apple / screenshots (push) Successful in 6m33s
ci / bench (push) Successful in 6m41s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m23s
android / android (push) Successful in 12m50s
docker / deploy-docs (push) Successful in 25s
deb / build-publish-host (push) Successful in 10m10s
arch / build-publish (push) Successful in 17m18s
deb / build-publish (push) Successful in 12m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m3s
ci / rust (push) Successful in 27m3s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m32s
Field report: with Exclusive Topology, the physical monitor sometimes stayed
lit — every isolate attempt re-committed and failed SetDisplayConfig with
0x57 ERROR_INVALID_PARAMETER. Clearing DISPLAYCONFIG_PATH_ACTIVE alone leaves
the deactivated path's source/target modeInfoIdx pinned to the queried mode
entries, and per the SetDisplayConfig contract a path being turned off must
have BOTH mode indexes set to DISPLAYCONFIG_PATH_MODE_IDX_INVALID; some
driver/topology combinations (notably when the doomed path shares a source
with the kept virtual path) reject the supplied config outright, so the
retry loop could never converge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 12:26:44 +02:00
enricobuehlerandClaude Fable 5 b7a00137eb feat(host): hold a logind sleep/idle inhibitor while clients stream
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 1m0s
android / android (push) Failing after 1m6s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 48s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 52s
ci / bench (push) Successful in 8m13s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m40s
deb / build-publish (push) Successful in 11m0s
release / apple (push) Successful in 9m50s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10m19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m57s
docker / deploy-docs (push) Successful in 35s
deb / build-publish-host (push) Successful in 14m3s
apple / screenshots (push) Successful in 6m27s
arch / build-publish (push) Successful in 17m49s
windows-host / package (push) Successful in 18m15s
flatpak / build-publish (push) Failing after 8m25s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m24s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m37s
ci / rust (push) Successful in 25m51s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m48s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m52s
Remote INPUT resets the compositor's idle timers, but a passive
(video-only) viewer sends none — observed live: a SteamOS Game-Mode
host s2idled mid-day and dropped off the network (in a VM with GPU
passthrough it never woke again). Refcounted across planes: the native
LiveSessionGuard and the GameStream video worker each hold a share;
first hold takes the logind sleep:idle block inhibitor on a dedicated
plain thread (zbus blocking must not run on a tokio worker), last drop
releases. Best-effort: no logind → warn once and stream on. No-op off
Linux (macOS/Windows manage their own assertions).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 12:09:47 +02:00
enricobuehlerandClaude Fable 5 acce43ebbf feat(steamdeck): self-healing reliability — post-OS-update rebuild check + script/docs polish
- rebuild-check.sh + punktfunk-rebuild-check.service (enabled, ordered
  Before=punktfunk-host): ldd-probes the host binary at session start —
  milliseconds when healthy, a full update.sh rebuild only when a
  SteamOS update actually broke its library links. update.sh restarts
  go --no-block so the check → update.sh → restart chain can't deadlock
  against the unit ordering. update.sh retrofits the unit.
- installer summary: web console is https (the unit serves TLS).
- docs: steamos-host.md (runner in the build step, keep-list + auto
  rebuild = updates survive hands-free), gamescope.md (Gaming Mode
  touch = single-finger pointer, exact taps, no multi-touch),
  plugins.mdx (SteamOS runner note), scripts/steamdeck/README.md
  (rebuild-check, runner payload, keep list, honest packaging
  trade-off + the CI-prebuilt future direction).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 11:56:09 +02:00
enricobuehlerandClaude Fable 5 35923080fb fix(host/linux): scale degenerate-region absolute coords by the session's real output size
The raw-client-pixels fallback is exact only while the managed session
runs at the client's mode. When they diverge — foreign-gamescope
attach at its own resolution, supersample/under-render, transitions —
raw pixels drift or land out of range. The EIS relay file now carries
the session's current output size as a second "WxH" line (from
current_gamescope_output_size()); the injector scales normalized
client positions into it, keeping raw pixels only as the last resort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:53:14 +02:00
enricobuehlerandClaude Fable 5 0c9461242c fix(host/linux): don't normalize absolute coords into gamescope's degenerate INT32_MAX EIS region
The previous commit's no-region fallback never fired: gamescope DOES
advertise a region — (0,0,INT32_MAX,INT32_MAX), meaning "coordinates
are raw". Mapping normalized positions into it explodes a center tap
to x≈1e9, clamped by gamescope to the far corner — the cursor pinned
at (1279,799) and every degraded-touch tap landing there. Treat a
region as an output geometry only when plausibly sized (≤16384 px);
otherwise emit raw client pixels.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:40:02 +02:00
enricobuehlerandClaude Fable 5 cc6b37fef0 fix(host/linux): emit absolute pointer/touch on region-less EIS devices + abs input-test probe
gamescope's "Gamescope Virtual Input" advertises pointer_abs but no
region, so every MouseMoveAbs (and the degraded-touch moves built on
it) was silently dropped (emitted=false) — clicks then landed at a
stale cursor position. With no region, the managed session runs at the
client's mode, so client pixels are output pixels: emit them raw.
Also: log region count/dims at device-resume, and add
PUNKTFUNK_INPUT_TEST_ABS=WxH to `input-test` (corners + center, 1s
apart) so the degraded-touch path is verifiable with xdotool alone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:36:05 +02:00
enricobuehlerandClaude Fable 5 1927077cbe fix(host/linux): degrade touch to absolute pointer when the EIS has no touchscreen device
gamescope's EIS ("Gamescope Virtual Input") advertises pointer/
pointer_abs/button but never a touchscreen — so in Game Mode every
remote TouchDown/Move/Up was dropped on the floor (Desktop Mode works
via KwinFakeInput, which does touch). Headless KWin's libei has the
same gap. Instead of dropping: degrade to a single-finger absolute
pointer — down = abs-move + left press, move = abs-move, up = release —
synthesized through the normal Mouse* inject paths so region mapping,
held-state tracking, and release_all apply. First finger drives the
pointer; later fingers are ignored (a pinch degrades to a drag). A
touchscreen device appearing later takes over on the next touch —
checked per event, never latched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:24:28 +02:00
enricobuehlerandClaude Fable 5 068da456f5 fix(steamdeck): survive SteamOS A/B updates — register system tuning on the atomic-update keep list
Verified live on a SteamOS 3.8.14→3.8.16 update: the A/B partset switch
rebuilds /etc and drops everything not on Valve's keep list — the udev
rule (uhid access: virtual pads silently degrade to Xbox 360), the
vhci-hcd autoload (native Deck pad transport), and the UDP buffer sysctl
(back to 208 KB → stutter). Valve's sanctioned extension point is a
drop-in under /etc/atomic-update.conf.d/ — itself on the stock keep
list, so it self-preserves. install.sh §4 and update.sh now register
scripts/punktfunk-atomic-keep.conf there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:07:19 +02:00
enricobuehlerandClaude Fable 5 256aa5f7f2 fix(steamdeck): install the plugin runner + harden Deck pad typing + repair update.sh
- scripts/steamdeck/install.sh (§2b) + update.sh: build and install the
  scripting runner user-scoped (~/.local wrapper + pinned bun + bundle) —
  SteamOS's read-only /usr can't take the .deb layout, so the console's
  plugin store reported "the plugin runner isn't installed" on every
  SteamOS host. update.sh retrofits it onto existing installs.
- plugins.rs runner discovery: check the ~/.local layout after the /usr
  ones; mention the SteamOS path in the not-installed error.
- update.sh: define warn() — it was used but never defined, so under
  `set -e` the first warn aborted the update before services restarted.
- pf-client-core gamepad: a Steam Input virtual pad forwarded on a real
  Deck (the only-pad case) now declares the DECK kind in its per-pad
  arrival instead of the wrapper's Xbox 360 identity — the session-level
  auto_pref already resolved SteamDeck, but the arrival overrides it on
  current hosts.
- reject tests: cover SetupFailed; move the foreign-code probe off 0x68.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:01:54 +02:00
enricobuehlerandClaude Fable 5 2562663fc6 fix(host/linux): survive and self-heal dead gamescope sessions on headless SteamOS boxes
Three fixes for the "most connects simply fail" trap on a SteamOS host
with no physical display (VM testbox, panel-less mini PC):

- restore path: skip the restore-to-physical-panel when no DRM connector
  reports a connected display — removing the headless drop-in and
  restarting gamescope-session.target on such a box just crash-loops
  gamescope and strands every later connect on "no usable compositor".
  Keep the headless session (and the takeover state) instead.
- connect path: when no live graphical session exists but the MANAGED
  gamescope infra is present (SteamOS gamescope-session / Bazzite
  session-plus), route to the gamescope takeover — which rebuilds the
  session at the client's mode — instead of failing the connect.
- error delivery: a session-setup failure used to just drop the QUIC
  connection, reaching the client as "control stream finished mid-frame"
  (indistinguishable from transport trouble). Close with a typed
  setup-failed code (0x68, RejectedSetupFailed = -29) carrying the error
  text, and give the client handshake/pairing error paths a short grace
  for the CONNECTION_CLOSE to beat the stream error it caused.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 09:50:30 +02:00
enricobuehler 87e7c82cbc chore(release): bump workspace version to 0.17.2
audit / bun-audit (push) Failing after 18s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
apple / swift (push) Successful in 1m23s
audit / cargo-audit (push) Successful in 2m44s
ci / bench (push) Successful in 6m26s
apple / screenshots (push) Successful in 6m38s
ci / rust (push) Successful in 24m53s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m10s
decky / build-publish (push) Successful in 22s
android-screenshots / screenshots (push) Successful in 3m7s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 44s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
deb / build-publish (push) Successful in 10m38s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9m3s
release / apple (push) Successful in 10m50s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m41s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m4s
deb / build-publish-host (push) Successful in 12m44s
android / android (push) Successful in 14m35s
docker / deploy-docs (push) Successful in 23s
arch / build-publish (push) Successful in 15m35s
web-screenshots / screenshots (push) Successful in 3m1s
linux-client-screenshots / screenshots (push) Successful in 7m31s
windows-host / package (push) Successful in 13m4s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m24s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m32s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m39s
flatpak / build-publish (push) Successful in 6m40s
2026-07-21 23:33:53 +02:00
enricobuehlerandClaude Opus 4.8 b2e3d1b540 fix(steamdeck/install): add cmake dep + acquire sudo in preflight
ci / web (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m52s
apple / swift (push) Successful in 1m22s
android / android (push) Successful in 13m4s
ci / bench (push) Successful in 8m2s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
deb / build-publish (push) Successful in 9m14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m14s
apple / screenshots (push) Successful in 6m51s
arch / build-publish (push) Successful in 18m54s
docker / deploy-docs (push) Successful in 27s
ci / rust (push) Successful in 19m17s
deb / build-publish-host (push) Successful in 9m17s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m33s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m29s
pyrowave-sys's build.rs runs cmake to build the vendored PyroWave C library,
but cmake wasn't in the distrobox apt list — so `cargo build -p punktfunk-host`
dies on every SteamOS host with "is cmake not installed?". Add it.

Also move sudo acquisition from step 4 (after the ~15-min build) into preflight
and keep the timestamp warm across the build. Before, a non-interactive run
(`ssh host 'bash install.sh'`) silently skipped the UDP-buffer/udev/vhci-hcd/
input-group tuning AND enable-linger — a "successful" host with no gamepad
passthrough, lossy streaming, and no headless persistence (dies on logout).
Now sudo is prompted once up front (authorize and walk away), and a no-TTY /
no-password run fails loudly before the build instead of silently at the end.

Validated end-to-end on SteamOS 3.8 in a VM with Radeon 780M passthrough:
full host + web build, all planes listening, VAAPI render node, headless
auto-start via linger.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:50:00 +02:00
enricobuehlerandClaude Fable 5 d36bec6e9d feat(core+host): negotiate ChaCha20-Poly1305 as the session cipher for soft-AES clients
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
arch / build-publish (push) Has been cancelled
audit / bun-audit (push) Failing after 13s
audit / cargo-audit (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Successful in 10m1s
windows-host / package (push) Successful in 11m20s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m27s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m28s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m39s
Lifts the ~100 Mbps decrypt ceiling on clients without hardware AES — the
armv7 soft-AES targets (webOS TVs), where AES-128-GCM resolves to fixsliced
software AES + software GHASH (~50-100 cpb) while ChaCha20-Poly1305's ARX
construction runs ~10-17 cpb portable, a 4-7x lift that PyroWave-on-TV needs
(design/chacha20-session-cipher.md).

Phase 1 (core crypto, no wire change): SessionKey merges cipher choice and
key material (invalid combinations unrepresentable, zeroize + redacted-Debug
discipline kept); SessionCrypto dispatches both aead-0.5 ciphers per call —
the salt||seq nonce scheme, per-direction salts, seq-as-AAD and replay
window carry over verbatim (same 96-bit nonce / 16-byte tag, const-asserted).
Config.key becomes SessionKey; validate's zero-key rejection follows the
active variant. The C ABI keeps its fixed 16-byte key mapped to AES — no
ABI_VERSION bump.

Phase 2 (negotiation): VIDEO_CAP_CHACHA20 (0x40) — support-plus-request in
one bit, the VIDEO_CAP_444 precedent. Welcome grows cipher@68 +
key_chacha@69..101, emitted only when non-zero so an AES session's Welcome
stays byte-identical to the pre-cipher form; decode is fail-closed (short
key or unknown id -> Err, never a silent AES fallback). No WIRE_VERSION
bump; downgrade resistance inherited from the pinned-TLS control channel.

Phase 3 (host): grant only when the client advertised the bit and the
PUNKTFUNK_CHACHA20 kill-switch (default on, documented) allows; fresh
32-byte per-session key from the same RNG discipline, legacy key field
stays independently random; resolved cipher logged at session start.

Verification: seal/open suites parameterized over both ciphers + a
cross-cipher tamper case; Welcome roundtrip/truncation/fail-closed tests;
ChaCha lossy-loopback soak (loss/replay is cipher-independent); bench
gains _chacha20 series (AES ids unchanged for CI history) — host-side
sealing line-rate-trivial on both x86 (~640 MiB/s) and Apple Silicon
(~535 MiB/s). punktfunk-probe drives the interop matrix via
PUNKTFUNK_CLIENT_CHACHA20=1 and logs the negotiated cipher.

Phase 4 (pf-webos pin bump + unconditional cap bit) follows the next
core release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:50:00 +02:00
enricobuehlerandClaude Fable 5 abc54a7d13 fix(decky): actually ship shortcut artwork + Steam Input layout in the published plugin
apple / swift (push) Successful in 1m23s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 56s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m58s
The CI workflow assembled the store zip by hand and silently drifted from
scripts/package.sh when 60d46530 added assets/ and controller_config/ — every
published build (canary and stable through 0.17.1) was missing both, so the
Steam shortcut a fresh install creates had no images and no controller layout
(shortcut_art() found nothing to send; apply_controller_config failed with
template-missing). CI now stages through scripts/package.sh — the plugin file
list lives in one place — and a backstop verifies the runtime-loaded pieces are
in the zip before publishing.

Frontend hardening for installs poisoned by those builds: applyArtwork stamped
its per-appId ART_VERSION marker even when zero assets were applied, so
shipping the files alone would never heal existing shortcuts — only mark done
when something actually landed, and bump ART_VERSION 2→3 to force one re-apply.
Same guard for the controller-config marker (ok with zero account configset
dirs is not done), plus one deferred artwork retry for the fresh-AddShortcut
registration race setShortcutHidden already defers around.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:36:55 +02:00
enricobuehlerandClaude Fable 5 309e37f1e1 fix(host/linux): scope the GPU clock pin to live client sessions
The PUNKTFUNK_PIN_CLOCKS clock pin (AMD power_dpm_force_performance_level=high / NVIDIA nvmlDeviceSetGpuLockedClocks) was armed once at host start and released only at host exit, so the box's clocks stayed pinned the whole time the host ran — even with no client connected.

Move the pin behind a box-wide refcount (gpuclocks::session_pin()) shared across both streaming planes (native + GameStream): it arms on the first live client and releases on the last disconnect. on_host_start() now only installs the process-scoped NVIDIA P2-cap application profile. No behavior change when PUNKTFUNK_PIN_CLOCKS is unset.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:36:55 +02:00
enricobuehlerandClaude Fable 5 56d21f9445 fix(client/wol): retry the QUIC dial across the connect budget so wake-from-sleep launches survive
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m3s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 40s
ci / bench (push) Successful in 6m51s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m6s
deb / build-publish (push) Successful in 9m54s
release / apple (push) Successful in 9m5s
deb / build-publish-host (push) Successful in 9m35s
flatpak / build-publish (push) Failing after 8m19s
android / android (push) Successful in 16m57s
apple / screenshots (push) Successful in 6m29s
arch / build-publish (push) Successful in 17m35s
windows-host / package (push) Successful in 18m46s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
docker / deploy-docs (push) Successful in 26s
ci / rust (push) Successful in 25m24s
Launching from the Decky plugin against a sleeping host fired the
Wake-on-LAN packet and dialed immediately — but the dial was a single
quinn attempt that gave up after the transport's 8 s idle window, far
shorter than a suspend-to-RAM resume. The host woke to find the client
already gone (its retransmitted Initials show up host-side as
"QUIC accept failed error=timed out"), the stream shortcut exited, and
the launch looked cancelled.

- punktfunk-core: dial in a loop until the embedder's connect budget is
  spent — short 3 s attempts keep the Initial cadence dense, so the
  connect lands within ~a second of the host's network returning. Only
  silence is retried: pin mismatches, typed rejections, and any real
  answer surface immediately, and the shutdown flag stops the loop.
- decky: stretch the budget to 75 s when a magic packet actually went
  out (PF_CONNECT_TIMEOUT → --connect-timeout via the wrapper), so the
  retry window covers the resume; an awake host still connects in <1 s.
- host: a clean close before the control handshake is a reachability
  probe (hosts-page online pips), not a failed session — log it at
  debug instead of WARN "session ended with error", which buried the
  real failures in the reporter's log.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:26:10 +02:00
enricobuehlerandClaude Opus 4.8 f36d13e371 fix(steamdeck/install): prompt for sudo instead of requiring passwordless
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
ci / bench (push) Successful in 6m5s
apple / screenshots (push) Successful in 6m34s
deb / build-publish (push) Successful in 9m43s
docker / deploy-docs (push) Successful in 48s
deb / build-publish-host (push) Successful in 10m9s
android / android (push) Successful in 18m18s
arch / build-publish (push) Successful in 19m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m16s
ci / rust (push) Successful in 28m27s
The system-tuning block (UDP buffers, gamepad udev rule, vhci-hcd autoload,
input group) was gated on `sudo -n true` — passwordless sudo — so on a stock
SteamOS box (the deck account needs a password) the whole block was silently
skipped: /dev/uhid stayed root-only so the gamepad degraded to an Xbox 360 pad,
and the UDP buffers stayed at the 416 KB default. Video was unaffected because
Game Mode/gamescope needs none of it, which masked the skip on-glass.

- install.sh + update.sh: acquire sudo interactively (`sudo -v` prompt on a TTY)
  instead of requiring passwordless; skip only when there's no TTY or auth fails,
  and then print the exact manual block plus a `passwd` hint — a stock SteamOS
  'deck' account has no password, so sudo can't work until one is set.
- update.sh: also retrofit the UDP-buffer sysctl for older/skipped installs, and
  loudly nag to reboot when the input group was just added (a --user restart does
  not pick up the new group; only a fresh login does).
- steamos-host.md: note this step prompts for the sudo password.

Root is unavoidable: the udev rule, input group, and vhci-hcd module are all
kernel operations with no user-space alternative.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:56:20 +02:00
enricobuehlerandClaude Opus 4.8 aacd610b68 fix(steamdeck/install): set up controller passthrough + surface the required reboot
ci / docs-site (push) Successful in 1m12s
ci / web (push) Successful in 1m13s
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
ci / bench (push) Successful in 6m35s
apple / screenshots (push) Successful in 6m13s
deb / build-publish (push) Successful in 10m21s
docker / deploy-docs (push) Successful in 34s
deb / build-publish-host (push) Successful in 9m58s
android / android (push) Successful in 17m7s
arch / build-publish (push) Successful in 17m47s
ci / rust (push) Successful in 19m28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m2s
A fresh SteamOS host install failed on-glass two ways: KWin denied Desktop-mode
capture (zkde_screencast_unstable_v1) so every session died in the pipeline build,
and the native Steam Deck pad degraded to an Xbox 360 controller. Both stem from
setup that only a fresh login applies — the host user-service keeps the
pre-install group set and KWin reads its .desktop grant at session start — plus a
missing vhci-hcd autoload the pad's USB/IP transport needs (the deb/rpm/arch
packages ship it; the Deck installer did not).

- install.sh: install punktfunk-modules.conf (+ modprobe vhci-hcd now) for the
  native Deck controller transport, seed the KDE RemoteDesktop grant
  (kde-authorized) for Desktop-mode input, and print a loud "reboot before
  streaming" notice when a relogin is actually required (input group added or the
  .desktop grant first installed).
- update.sh: retrofit the udev rule, vhci-hcd autoload, input group, and grant so
  existing installs pick them up on the next update without a reinstall.
- steamos-host.md: document the first-install reboot and the native controller
  passthrough requirements (input group + vhci-hcd + client-side Steam Input Off).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:22:08 +02:00
enricobuehlerandClaude Opus 4.8 081ff64087 docs: explain how to actually run the VirtualHere passthrough example
apple / swift (push) Successful in 1m22s
ci / web (push) Successful in 1m11s
ci / docs-site (push) Successful in 1m11s
ci / bench (push) Successful in 5m39s
apple / screenshots (push) Successful in 6m38s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 57s
deb / build-publish (push) Successful in 9m0s
deb / build-publish-host (push) Successful in 9m20s
windows-host / package (push) Successful in 9m46s
android / android (push) Successful in 18m37s
arch / build-publish (push) Successful in 18m48s
docker / deploy-docs (push) Successful in 28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m44s
ci / rust (push) Successful in 26m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m2s
The VirtualHere DualSense recipe linked the `virtualhere-dualsense.ts` SDK
example as a deployable recipe but skipped everything needed to run it: that
VirtualHere is a server (couch) + client (host) pair, and that the example —
like every example — imports `../src/index.js`, which only resolves inside the
SDK repo. A user copying it out had no way to know they must
`bun add @punktfunk/host` and swap that import.

- automation.md: rewrite the recipe into a full walkthrough — the two-sided
  VirtualHere setup, the `-t` verbs, the zero-code hooks version, and a new
  scripted section with exact deploy steps, env vars, and running it as a
  service (its own SIGTERM handler makes `systemctl stop` release the pad).
- sdk/README.md: say how to run an example in-repo vs deployed on a host.
- virtualhere-dualsense.ts: header note on the import swap + service setup,
  since that file is where the doc link lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 17:28:23 +02:00
enricobuehlerandClaude Opus 4.8 675fd24cce chore(release): bump workspace version to 0.17.1
audit / cargo-audit (push) Successful in 3m22s
audit / bun-audit (push) Failing after 13s
apple / swift (push) Successful in 1m23s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m11s
ci / bench (push) Successful in 5m12s
apple / screenshots (push) Successful in 6m55s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m55s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m39s
ci / rust (push) Successful in 24m5s
decky / build-publish (push) Successful in 30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
android-screenshots / screenshots (push) Successful in 3m34s
linux-client-screenshots / screenshots (push) Successful in 8m32s
release / apple (push) Successful in 11m16s
deb / build-publish-host (push) Successful in 12m51s
deb / build-publish (push) Successful in 12m57s
docker / deploy-docs (push) Successful in 27s
web-screenshots / screenshots (push) Successful in 3m13s
windows-host / package (push) Successful in 14m6s
android / android (push) Successful in 15m33s
arch / build-publish (push) Successful in 16m3s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m43s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m40s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m10s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 23m16s
flatpak / build-publish (push) Successful in 7m9s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 16:37:32 +02:00
enricobuehlerandClaude Fable 5 0e60e58cab perf(apple): windowed macOS presents — off-main transactional commit, surface prototype, safe-present setting
ci / docs-site (push) Successful in 52s
ci / web (push) Successful in 56s
decky / build-publish (push) Successful in 21s
apple / swift (push) Successful in 1m24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / bench (push) Successful in 5m53s
release / apple (push) Successful in 9m19s
deb / build-publish (push) Successful in 12m27s
arch / build-publish (push) Successful in 12m52s
docker / deploy-docs (push) Successful in 26s
deb / build-publish-host (push) Successful in 13m18s
apple / screenshots (push) Successful in 6m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m51s
android / android (push) Successful in 18m40s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m15s
ci / rust (push) Successful in 27m38s
Task 1 (latency): the transactional present now commits from the RENDER
thread inside an explicit CATransaction + flush() instead of hopping to
main. The present harness (2026-07-21, 240 Hz Studio, full-size window)
measured the main hop batching presents at runloop-iteration rate when
main is busy — an active implicit transaction NESTS the explicit one —
which is the field's presents=55 @ fps=240 / display_p50 18.6 ms.
Off-main commits measured immune to main churn: glass p50 ~10 ms,
cadence a clean 4.17 ms at 240. flush() is load-bearing: without it the
render thread's own implicit transaction (drawableSize/colour writes)
swallows the explicit commit and NOTHING reaches glass — the harness
reproduced the exact freeze (every present dropped). Legacy main-hop
kept as PUNKTFUNK_TXN_PRESENT=main for field A/B. New pf-windowed
Console line (subsystem io.unom.punktfunk, category presenter)
decomposes the issue side per second.

Task 1 lever D: the f407f418 IOSurface contents-swap path is
resurrected format-aware as WindowedPresentMode.surface — bgra8 SDR /
rgba16Float+PQ-tagged HDR pool, swap committed off-main from the
completion handler; EDR anchored by the metal layer underneath.
Env-only prototype (PUNKTFUNK_WINDOWED_PRESENT=surface) until the HDR
composite is eyeballed on glass.

Task 2 (setting): punktfunk.windowedSafePresent (default ON =
transactional mitigation; OFF = the fast async path whose panic the
saga is about — the caption says so in plain words). Rows in the touch
Settings (Presentation) and gamepad settings, macOS-only.
PUNKTFUNK_WINDOWED_PRESENT=async|transaction|surface overrides for dev
A/B. maximumDrawableCount stays 3 — the harness measured 2 starving the
render loop (172/s) and worsening glass p50.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:58:47 +02:00
enricobuehlerandClaude Fable 5 71faf38a85 fix(apple): stats os_log used %d for 64-bit Swift Int — macOS 26 drops the line
Swift `Int` is 64-bit (Foundation infers %lld); the stats mirror's format
string used %d (32-bit C int) for fps/presents/lost. macOS 26's strict
String(format:) validator rejects the mismatch and drops the whole line
(the error also cascades onto the float args, mis-blaming floor_p50). Use
%lld for the three Int fields. Pre-existing, unrelated to the DCP work;
surfaced while reading presenter logs during the panic fix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:14:41 +02:00
enricobuehlerandClaude Fable 5 f49d38826b fix(apple): windowed macOS DCP panic via presentsWithTransaction — keep real HDR, drop the SDR IOSurface path
The windowed-macOS "mismatched swapID's" @UnifiedPipeline.cpp kernel
panic is the CAMetalLayer ASYNC image queue (commandBuffer.present,
mandatory with displaySyncEnabled=false) diverging from WindowServer's
compositor on a high-refresh composited session — it survived glass
pacing (PyroWave, 2026-07-18) and hit windowed HEVC on the same 240 Hz
Mac Studio (2026-07-21), so it's the async queue itself, at any codec
or pacing.

f407f418's mitigation routed windowed PyroWave through a BGRA8 IOSurface
layer.contents pool, which cannot carry HDR — extending it to HEVC would
have tone-mapped windowed HDR down to SDR. Instead, present the drawable
through a Core Animation transaction (CAMetalLayer.presentsWithTransaction)
for every windowed session: commit, waitUntilScheduled, then drawable.present()
inside a CATransaction, so the swap commits with the layer tree and stays
in lockstep with the compositor instead of racing it. The full
rgba16Float / PQ / EDR render path is untouched — windowed HDR is real
HDR again. Fullscreen keeps the async path (direct scanout, lowest latency,
no panic there).

Removes the IOSurface surface pool, its surfaceLayer sibling, and the
macOS windowed PQ->SDR tone-map (tvOS keeps its no-headroom tone-map).
Net -150 lines. swift build + 169 tests green. Needs on-glass soak on
the 240 Hz Mac Studio (kernel race — only real hardware confirms).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:45:11 +02:00
enricobuehlerandClaude Fable 5 42198eb1b6 feat(linux/vulkan-encode): default-on native NV12 capture + RGB true-extent
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m20s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 25s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 42s
ci / bench (push) Successful in 5m59s
apple / screenshots (push) Successful in 6m17s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m29s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m16s
deb / build-publish (push) Successful in 12m58s
deb / build-publish-host (push) Successful in 13m40s
windows-host / package (push) Successful in 10m46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m52s
android / android (push) Successful in 16m18s
arch / build-publish (push) Successful in 16m20s
docker / deploy-docs (push) Successful in 30s
ci / rust (push) Successful in 19m25s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m19s
Flip both zero-copy levers from opt-in to default:

- The capture negotiation now PREFERS gamescope's producer-side NV12 pod by
  default (PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation).
  Codec-aware gating rides a new OutputFormat::nv12_native ->
  ZeroCopyPolicy::native_nv12_session edge resolved by the host facade from
  the session plan (pf_encode::linux_native_nv12_ok): only H265/AV1 sessions
  whose backend can open the raw Vulkan Video encoder ever see the NV12 pod
  -- an H264/Moonlight session (libav VAAPI, which would misread the
  two-plane buffer) keeps today's BGRx negotiation, as do the
  GameStream-resolve and portal-mirror paths, PyroWave, and NVENC prefs.
- RGB-direct's unaligned modes default to the true-extent direct import
  (PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 restores the padded-copy staging).
  Guarded-tested on Van Gogh with the kernel journal watched: clean, and at
  5.38 ms p50 the fastest 1080p encode path measured on that hardware. The
  EFC only exists on Mesa >= 26, where the codedExtent-driven session_init
  padding is guaranteed (verified back to Mesa 24.2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehlerandClaude Fable 5 b4fbc94e36 feat(linux/vulkan-encode): PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT bring-up gate
Opt-in alternative to RGB-direct's padded-copy staging at unaligned modes:
direct-import the visible-size capture and declare the TRUE-SIZE source
codedExtent. RADV has derived the VCN session_init firmware padding from
srcPictureResource.codedExtent since Mesa 24.2, so the EFC is told the source
lacks the alignment rows and the hardware edge-extends them internally --
which also reframes the 2026-07-20 field GPU reset: that crash passed the
ALIGNED extent as the source codedExtent (firmware padding zero) over a
visible-size buffer. Off by default until a guarded live test proves the EFC
front-end honors the padding like the YUV fetch path does; the padded-copy
staging remains the shipped behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehlerandClaude Fable 5 004f0cacb8 perf(linux/vulkan-encode): true-size headers make native NV12 zero-copy at every mode
Drop the padded-copy staging for producer-native NV12 and direct-import the
visible-size buffer at unaligned modes (1080p) too. Safe by the driver's own
contract rather than by allocation luck: native sessions now author the H265
SPS / AV1 sequence header at the RENDER size and pass the matching codedExtent
on every picture resource. RADV rounds the bitstream SPS up itself (with a
conformance window -- radv_video_patch_encode_session_parameters, per the
VK_KHR_video_encode_h265 proposal's "implementations may override" clause) and
programs the VCN session_init with the true extent plus nonzero firmware
padding, so the hardware edge-extends the alignment rows internally and never
fetches past the source's real extent -- the exact mechanism VAAPI/radeonsi has
always used to encode 1080p. The driver-emitted header NALs already carry the
patched SPS, so the wire format is unchanged.

The CSC and RGB-direct paths keep the app-aligned convention (their sources
genuinely cover the aligned extent); RGB padded-copy staging is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehlerandClaude Fable 5 49c6dd00c9 fix(presenter): CPU frames never take the HDR10 passthrough surface
audit / bun-audit (push) Failing after 15s
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 58s
apple / swift (push) Successful in 1m22s
audit / cargo-audit (push) Successful in 2m20s
decky / build-publish (push) Successful in 32s
ci / bench (push) Successful in 7m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 13s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m18s
release / apple (push) Successful in 9m28s
deb / build-publish-host (push) Successful in 10m32s
deb / build-publish (push) Successful in 11m48s
docker / deploy-docs (push) Successful in 23s
android / android (push) Successful in 14m36s
arch / build-publish (push) Successful in 15m4s
windows-host / package (push) Successful in 16m11s
flatpak / build-publish (push) Failing after 8m19s
apple / screenshots (push) Successful in 6m40s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m11s
ci / rust (push) Successful in 19m38s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m39s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m36s
Software decode uploads swscale RGBA with no CSC/tonemap pass. On the new
desktop-compositor HDR10 surfaces (GNOME 48 / Plasma 6 + Mesa >= 25.1 offer
ST2084 even on SDR desktops) that sRGB-encoded content was composed as PQ —
the field-reported psychedelic picture (Fedora clients decode HEVC in
software because stock Mesa strips it; reproduced + fix live-validated on
GNOME Wayland: mode-0 soup -> HDR→SDR washed-out-but-correct).

The CPU lane keeps the SDR swapchain and its known untonemapped-PQ gap; a
real PQ→sRGB pass for CPU frames is the follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehlerandClaude Fable 5 e7c6c96e5f fix(presenter): set the Wayland app_id so the session window gets its icon
SDL_APP_ID = io.unom.Punktfunk — compositors match it against the shipped
.desktop for the window icon; without it KWin/mutter show the generic Wayland
icon (field-noticed on the Deck). Linux analog of win32 AppUserModelID.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehlerandClaude Fable 5 da134ad045 feat(presenter): PUNKTFUNK_HDR10=0 refuses the HDR10 swapchain (SDR tonemap pin)
GNOME 48 / Plasma 6 with Mesa >= 25.1 now offer HDR10_ST2084 surfaces even on
SDR desktops, silently flipping PQ streams into the mode-0 passthrough lane —
previously unreachable on desktop compositors and never validated there. The
hatch pins the shader tonemap for diagnosis and as the field escape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehlerandClaude Fable 5 428bd2519c test(qsv): converter→ring-profile-P010→encoder live e2e (the RTV-written seam)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehlerandClaude Fable 5 87d7eceabf fix(encode/qsv): write the colour VUI unconditionally + Intel-lane colour diagnostics
The native QSV encoder only attached mfxExtVideoSignalInfo on HDR sessions —
an SDR stream carried no colour description at all (NVENC writes 709-limited
unconditionally; qsv.rs now mirrors it).

Diagnostics for the field-reported blue/magenta Intel-host colours:
- hdr-p010-selftest takes an optional WxH + GPU vendor (intel|nvidia|amd) and
  prints which adapter it tested — 64x64 on the default GPU proved nothing on
  dual-GPU boxes, and the field heights (1080/1400) are not 16-aligned.
- pf-capture: ignored live test running the converter at 1920x1080 pinned to
  the Intel adapter.
- pf-encode: qsv_live_p010_1080_colorbars_dump — known P010 bars through the
  UNALIGNED-height ingest copy (1080 src -> align16 1088 pool, the seam no
  640x480 test exercises) to Main10 HEVC, dumped for off-box decode checks.
- dxgi.rs: drop the stale 'falls back to the R10 path' claim (no such path).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehlerandClaude Fable 5 1d4795666e feat(linux/vulkan-encode): opt-in gamescope producer-native NV12 encode source
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
decky / build-publish (push) Successful in 19s
apple / swift (push) Successful in 1m21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m53s
apple / screenshots (push) Successful in 6m16s
windows-host / package (push) Successful in 9m38s
deb / build-publish (push) Successful in 9m55s
docker / deploy-docs (push) Successful in 26s
arch / build-publish (push) Successful in 12m52s
android / android (push) Successful in 12m58s
deb / build-publish-host (push) Successful in 14m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m0s
ci / rust (push) Successful in 28m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m41s
With PUNKTFUNK_PIPEWIRE_NV12=1 (bring-up gate), the PipeWire negotiation
offers an NV12 LINEAR DMA-BUF pod (BT.709 limited pinned MANDATORY) ahead
of the BGRx one, and gamescope's producer-side RGB->YUV pass replaces the
host CSC entirely: the encoder imports the two-plane buffer as a profiled
VIDEO_ENCODE_SRC image and the VCN encodes it directly. Contributed
measurements: encode p99 2.9 ms (from ~4-4.5 ms via EFC RGB-direct),
60 fps capture, 0 send drops.

Hardening on top of the contributed patch:
- unaligned modes (1080p!) stage through a padded aligned NV12 copy (edge
  rows/columns duplicated, transfer-only) instead of direct-importing the
  visible-size buffer -- a direct import would make the VCN read past the
  producer allocation, the exact OOB class behind the 2026-07-20 field GPU
  reset; the encode extents return to the aligned coded extent everywhere
- the UV plane layout honors the producer's plane-1 chunk (offset/stride)
  when the SPA buffer carries one (same-BO verified by inode), with the
  contiguous-plane contract as fallback
- PyroWave sessions are excluded from the gate (their Vulkan compute CSC
  ingests packed RGB), and a native-NV12 session that resolves to libav
  VAAPI (H264 codec, PUNKTFUNK_VULKAN_ENCODE=0, feature off, or a failed
  Vulkan open) refuses at open instead of streaming garbage chroma
- pad staging images carry TRANSFER_SRC (the width-padding pass self-copies
  the staging image -- previously missing on 1366-wide modes)
- metadata-cursor one-shot warn (parity with RGB-direct) and a padded-NV12
  PUNKTFUNK_PERF split label; the padded RGB copy gets its timestamps too

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 11:57:42 +02:00
enricobuehlerandClaude Opus 4.8 58b27acbd5 fix(vdisplay/session): scrub the dead desktop's WAYLAND_DISPLAY from the user-manager env
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m14s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 27s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 19s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / bench (push) Successful in 5m44s
apple / screenshots (push) Successful in 6m18s
android / android (push) Successful in 13m0s
docker / deploy-docs (push) Successful in 27s
deb / build-publish (push) Successful in 13m36s
deb / build-publish-host (push) Successful in 14m44s
windows-host / package (push) Successful in 16m32s
arch / build-publish (push) Successful in 17m49s
ci / rust (push) Successful in 19m58s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m42s
settle_desktop_portal pushes the live desktop's WAYLAND_DISPLAY into the
systemd --user manager (import-environment) so a re-activated portal inherits
the right session — but the import PERSISTS after that desktop dies. Every
later user unit then inherits the stale socket, including
gamescope-session.target: gamescope sees WAYLAND_DISPLAY, runs NESTED, and
aborts with "Failed to connect to wayland socket: wayland-0" — observed live
on a Deck, where Game Mode could not start at all (autologin crash-looped)
until the var was unset by hand.

Two-layer fix, mirroring the protection launch_session already gives its
transient unit:
- observe_session_instance: when a desktop compositor instance goes away,
  unset WAYLAND_DISPLAY/DISPLAY from the manager env (a desktop bounce is
  harmless — the next portal settle re-imports).
- write_steamos_dropin: UnsetEnvironment=DISPLAY WAYLAND_DISPLAY on the
  takeover drop-in, unit-scoped belt-and-suspenders for host-initiated
  restarts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:06:55 +02:00
enricobuehlerandClaude Opus 4.8 333c8979e9 fix(vdisplay/gamescope): capture-loss rebuild probes must not steal the box session
Observed live on a Deck host: the user switched Game Mode → Desktop, KDE came
up in 0.8 s — and the capture-loss rebuild's FIRST detection ran inside that
gap, still said Gaming, and its gamescope re-acquire restarted
gamescope-session.target. On SteamOS that steals the seat: the user was
yanked out of the KDE session they had just chosen, the two session managers
fought (job canceled / dependency failed), no gamescope node appeared within
30 s, and the stream died at the 40 s rebuild budget.

A rebuild probe acting on possibly-stale detection must never stop, relaunch,
or take over box sessions. New pf_vdisplay::rebuild_probe_scope (RAII,
counted): while held, the gamescope managed/takeover create paths only attach
to a live node and fail fast otherwise — no stop_autologin_sessions, no
session-plus relaunch, no gamescope-session.target restart. The capture-loss
loop holds it for the first 4 s after a loss (the detection-ambiguity
window); after that, destructive rebuilds are allowed again, so a genuine
switch INTO Game Mode still gets its headless takeover. Probe failures are
instant, so the loop now paces itself at ~2 Hz instead of spinning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:00:02 +02:00
enricobuehlerandClaude Opus 4.8 46d09ed973 fix(host/linux): stop burning retry budget on doomed pipeline builds
Two session-transition stalls found live on a SteamOS Deck host, one cause:
the pipeline retry loop couldn't tell a wait-it-out failure from a
retry-now-and-it-works one.

- Gamescope first-frame race: a PipeWire stream connected while gamescope
  re-initializes its headless takeover negotiates a format, reaches
  Streaming, and never receives a buffer — while a fresh connect delivers
  within ~0.5 s. Every gamescope bring-up ate the full 10 s first-frame
  budget on attempt 1 (17 s bring-ups; KWin: 1.2 s). The retry loop's FIRST
  attempt now waits 2.5 s (Capturer::next_frame_within), so the reconnect
  that fixes the race happens at ~3 s. Later attempts keep the patient 10 s
  — the documented 30-60 s Big Picture cold start still fits the budget.

- Capture-loss rebuild vs session switch: the rebuild loop re-detects the
  active session between build_pipeline_with_retry calls, but each call
  burned 8 attempts (~13 s) against a compositor that no longer exists (a
  Desktop→Gaming switch spent 13 of its 27 s retrying gone-KWin). The
  capture-loss path now passes max_attempts=2, turning the outer loop into
  ~1 s detect-and-retry cycles that follow the box to the new session.

The resize path's direct build_pipeline call keeps the default budget (no
retry wrapper there to absorb an early bail).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 10:34:49 +02:00
enricobuehlerandClaude Opus 4.8 946f1b3d69 fix(steamdeck): build the host with vulkan-encode, and wire up what it needs to actually engage
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
apple / swift (push) Successful in 1m20s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m43s
apple / screenshots (push) Successful in 6m32s
deb / build-publish (push) Successful in 9m57s
docker / deploy-docs (push) Successful in 26s
arch / build-publish (push) Successful in 12m46s
android / android (push) Successful in 13m40s
deb / build-publish-host (push) Successful in 13m3s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m42s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m59s
ci / rust (push) Successful in 26m41s
The SteamOS scripts built the host featureless, so PlatformAuto could never
pick the Vulkan Video backend (default-on since 84329205) and every session
silently ran libav VAAPI — unlike the deb/arch/rpm/sysext builds, which all
pass the feature. Three gaps, found live on a Deck host:

- install.sh/update.sh: cargo build with --features punktfunk-host/vulkan-encode
  (matches the packaged builds; pure-Rust ash, no new system dep).
- host.env: RADV_PERFTEST=video_encode — Van Gogh RADV still gates
  VK_KHR_video_encode_* behind it, so without it the backend can't open and
  falls back to VAAPI. Harmless where encode is exposed by default.
- io.unom.Punktfunk.Host.desktop (Exec rewritten to this install's binary):
  KWin only grants zkde_screencast_unstable_v1/org_kde_kwin_fake_input to an
  exe a .desktop authorizes — without it Desktop-Mode capture fails outright
  and a mid-stream Game↔Desktop switch strands the stream on the old backend.

update.sh retrofits the last two idempotently so existing installs get them
without a reinstall.

Validated on a SteamOS 3.8.14 Deck (Van Gogh): Vulkan HEVC opened on both the
KWin and gamescope paths at 5120x1440 — in-place ABR reconfigure (no IDR, no
encoder rebuild) vs the VAAPI path's full teardown per ABR step, and Desktop
capture came up without a re-login.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 10:14:31 +02:00
101 changed files with 5409 additions and 867 deletions
+24 -30
View File
@@ -6,17 +6,12 @@
#
# The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT
# need the Decky CLI (which requires Docker + rust-nightly only to compile native backends).
# We build the frontend with pnpm and assemble the store-layout zip by hand:
#
# punktfunk.zip
# punktfunk/ <- single top-level dir == plugin.json "name"
# plugin.json [required]
# package.json [required; CI stamps "version" — Decky reads the installed version here]
# main.py [required: python backend]
# dist/index.js [required: rollup output]
# update.json [CI-baked {channel, manifest}: where the plugin's self-update check polls]
# README.md (recommended)
# LICENSE [required by the plugin store]
# We build the frontend with pnpm and stage the store-layout tree with the SAME script local
# builds use (clients/decky/scripts/package.sh) — the plugin's file list lives in exactly ONE
# place, so a file added there (bin/, assets/, controller_config/, …) can never be silently
# missing from the published build. (Hand-assembling the zip here is how the shipped plugin
# lost the shortcut artwork + Steam Input layout for a while.) CI only adds `update.json` on
# top: the {channel, manifest} pointer the plugin's self-update check polls.
#
# SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel
# `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed
@@ -90,28 +85,27 @@ jobs:
- name: Assemble store-layout zip
working-directory: ${{ gitea.workspace }}
run: |
apt-get update && apt-get install -y --no-install-recommends zip >/dev/null
STAGE="$RUNNER_TEMP/decky"
DEST="$STAGE/$PLUGIN"
rm -rf "$STAGE"; mkdir -p "$DEST/dist" "$DEST/bin"
cp clients/decky/plugin.json "$DEST/"
cp clients/decky/package.json "$DEST/"
cp clients/decky/main.py "$DEST/"
cp clients/decky/dist/index.js "$DEST/dist/"
cp clients/decky/README.md "$DEST/"
# The stream-launch wrapper (target of the Steam shortcut); keep it executable
# (runner_info() also re-chmods at runtime in case the zip/extract drops the bit).
cp clients/decky/bin/punktfunkrun.sh "$DEST/bin/"
chmod 0755 "$DEST/bin/punktfunkrun.sh"
# Store requires a LICENSE in the plugin root; the project is MIT OR Apache-2.0.
cp LICENSE-MIT "$DEST/LICENSE"
# Self-update channel pointer the backend reads (main.py check_update). It points at
# THIS channel's manifest.json (published below); that manifest in turn points at the
# immutable per-version zip, so its sha256 stays valid across future alias re-uploads.
# node:22-bookworm ships python3 (a package.sh dep) but not zip; install both anyway
# so an image change can't silently break the build.
apt-get update && apt-get install -y --no-install-recommends zip python3 >/dev/null
# Stage the canonical plugin tree (dist/, main.py, bin/, assets/, controller_config/,
# LICENSE, …) with the same script local/sideload builds use — see the header comment.
# Runs AFTER the version stamp, so the staged package.json carries $VERSION.
bash clients/decky/scripts/package.sh
DEST="clients/decky/out/$PLUGIN"
# CI-only addition: the self-update channel pointer the backend reads (main.py
# check_update). It points at THIS channel's manifest.json (published below); that
# manifest in turn points at the immutable per-version zip, so its sha256 stays valid
# across future alias re-uploads.
printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json"
( cd "$STAGE" && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
( cd clients/decky/out && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
ls -lh "$RUNNER_TEMP/punktfunk.zip"
unzip -l "$RUNNER_TEMP/punktfunk.zip"
# Backstop against packaging drift: the runtime-loaded pieces MUST be in the zip.
for f in main.py dist/index.js bin/punktfunkrun.sh assets/grid.png \
controller_config/punktfunk.vdf update.json; do
unzip -l "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN/$f" >/dev/null || { echo "MISSING $f" >&2; exit 1; }
done
# The update manifest the plugin polls: the immutable per-version artifact + its
# sha256 (Decky's installer verifies the download against this hash, aborting on
# mismatch — so it MUST be the per-version URL, never the mutable alias).
Generated
+65 -27
View File
@@ -656,6 +656,30 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chacha20"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3613f74bd2eac03dad61bd53dbe620703d4371614fe0bc3b9f04dd36fe4e818"
dependencies = [
"cfg-if",
"cipher",
"cpufeatures",
]
[[package]]
name = "chacha20poly1305"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10cd79432192d1c0f4e1a0fef9527696cc039165d729fb41b3f4f4f354c2dc35"
dependencies = [
"aead",
"chacha20",
"cipher",
"poly1305",
"zeroize",
]
[[package]]
name = "ciborium"
version = "0.2.2"
@@ -691,6 +715,7 @@ checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
dependencies = [
"crypto-common",
"inout",
"zeroize",
]
[[package]]
@@ -2159,7 +2184,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.17.0"
version = "0.17.2"
[[package]]
name = "lazy_static"
@@ -2264,7 +2289,7 @@ dependencies = [
[[package]]
name = "libvpl-sys"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"bindgen",
"cmake",
@@ -2299,7 +2324,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"punktfunk-core",
]
@@ -2788,7 +2813,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"ashpd",
@@ -2808,7 +2833,7 @@ dependencies = [
[[package]]
name = "pf-client-core"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"ash",
@@ -2832,7 +2857,7 @@ dependencies = [
[[package]]
name = "pf-clipboard"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"ashpd",
@@ -2850,7 +2875,7 @@ dependencies = [
[[package]]
name = "pf-console-ui"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"ash",
@@ -2871,7 +2896,7 @@ dependencies = [
[[package]]
name = "pf-encode"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"ash",
@@ -2881,6 +2906,7 @@ dependencies = [
"libvpl-sys",
"nvidia-video-codec-sdk",
"openh264",
"pf-capture",
"pf-frame",
"pf-gpu",
"pf-host-config",
@@ -2894,7 +2920,7 @@ dependencies = [
[[package]]
name = "pf-ffvk"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"ash",
"bindgen",
@@ -2903,7 +2929,7 @@ dependencies = [
[[package]]
name = "pf-frame"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"libc",
@@ -2915,7 +2941,7 @@ dependencies = [
[[package]]
name = "pf-gpu"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"pf-host-config",
@@ -2929,11 +2955,11 @@ dependencies = [
[[package]]
name = "pf-host-config"
version = "0.17.0"
version = "0.17.2"
[[package]]
name = "pf-inject"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"ashpd",
@@ -2961,14 +2987,14 @@ dependencies = [
[[package]]
name = "pf-paths"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"ash",
@@ -2983,7 +3009,7 @@ dependencies = [
[[package]]
name = "pf-vdisplay"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"ashpd",
@@ -3013,7 +3039,7 @@ dependencies = [
[[package]]
name = "pf-win-display"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"pf-paths",
@@ -3025,7 +3051,7 @@ dependencies = [
[[package]]
name = "pf-zerocopy"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"ash",
@@ -3136,6 +3162,17 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "poly1305"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8159bd90725d2df49889a078b54f4f79e87f1f8a8444194cdca81d38f5393abf"
dependencies = [
"cpufeatures",
"opaque-debug",
"universal-hash",
]
[[package]]
name = "polyval"
version = "0.6.2"
@@ -3221,7 +3258,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"android_logger",
"jni",
@@ -3237,7 +3274,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"async-channel",
@@ -3253,7 +3290,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3268,7 +3305,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"async-channel",
"ffmpeg-next",
@@ -3287,11 +3324,12 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"aes-gcm",
"bytes",
"cbindgen",
"chacha20poly1305",
"criterion",
"fec-rs",
"hmac",
@@ -3318,7 +3356,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"aes",
"aes-gcm",
@@ -3402,7 +3440,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3416,7 +3454,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"anyhow",
"ksni",
@@ -3439,7 +3477,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]]
name = "pyrowave-sys"
version = "0.17.0"
version = "0.17.2"
dependencies = [
"bindgen",
"cmake",
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.17.0"
version = "0.17.2"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+1 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0"
},
"version": "0.17.0"
"version": "0.17.2"
},
"paths": {
"/api/v1/clients": {
@@ -0,0 +1,107 @@
package io.unom.punktfunk
import android.content.ClipData
import android.content.ClipboardManager
import android.content.Context
import android.os.Handler
import android.os.Looper
import io.unom.punktfunk.kit.NativeBridge
/**
* Text clipboard sync for the active session (the desktop-client model, text-only v1):
* * **Device → host**: a local copy (the primary-clip listener, plus one probe at start) is
* announced as a lazy offer — the text crosses only when the host actually pastes (a
* `fetch:` event, answered with the clipboard's current content).
* * **Host → device**: a host copy arrives as an `offer:` event and is fetched eagerly into
* the system clipboard (Android apps can't lazily materialize a paste from the network
* without a content-provider round-trip that isn't worth it here).
*
* Loop guard: text set from a host fetch is remembered ([lastFromHost]) so the resulting
* primary-clip-changed callback doesn't bounce it straight back as a new offer. Clipboard reads
* happen while the stream is foreground (Android only allows focused-app reads). The native
* events are drained on a dedicated thread and applied on the main thread; [stop] joins it.
*/
class ClipboardSync(
private val context: Context,
private val handle: Long,
) {
private val main = Handler(Looper.getMainLooper())
private val cm = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
@Volatile private var running = true
private var seq = 0
private var lastOffered: String? = null
private var lastFromHost: String? = null
private var pendingFetch = -1
private var thread: Thread? = null
private val clipListener = ClipboardManager.OnPrimaryClipChangedListener { offerLocal() }
fun start() {
NativeBridge.nativeClipControl(handle, true)
cm.addPrimaryClipChangedListener(clipListener)
thread = Thread({ pollLoop() }, "pf-clipboard").also { it.start() }
offerLocal() // whatever is already on the clipboard is pasteable host-side right away
}
fun stop() {
running = false
cm.removePrimaryClipChangedListener(clipListener)
thread?.join(600) // one poll timeout (250 ms) + slack
thread = null
}
/** Announce the current local text (if it's new and not an echo of a host copy). */
private fun offerLocal() {
if (!running) return
val text = currentClipText() ?: return
if (text == lastOffered || text == lastFromHost) return
lastOffered = text
seq += 1
NativeBridge.nativeClipOfferText(handle, seq)
}
private fun currentClipText(): String? = runCatching {
cm.primaryClip?.takeIf { it.itemCount > 0 }?.getItemAt(0)
?.coerceToText(context)?.toString()?.takeIf { it.isNotEmpty() }
}.getOrNull()
private fun pollLoop() {
while (running) {
val ev = NativeBridge.nativeNextClip(handle) ?: continue
if (ev == "closed") return
main.post { handleEvent(ev) }
}
}
private fun handleEvent(ev: String) {
if (!running) return
val parts = ev.split(":", limit = 3)
when (parts[0]) {
"offer" -> {
val offerSeq = parts.getOrNull(1)?.toIntOrNull() ?: return
if (parts.getOrNull(2) == "1") {
pendingFetch = NativeBridge.nativeClipFetchText(handle, offerSeq)
}
}
"fetch" -> {
val req = parts.getOrNull(1)?.toIntOrNull() ?: return
val text = currentClipText()
if (text != null) {
NativeBridge.nativeClipServeText(handle, req, text)
} else {
NativeBridge.nativeClipCancel(handle, req)
}
}
"data" -> {
val xfer = parts.getOrNull(1)?.toIntOrNull() ?: return
if (xfer != pendingFetch) return // stale/unknown transfer
pendingFetch = -1
val text = parts.getOrNull(2)?.takeIf { it.isNotEmpty() } ?: return
lastFromHost = text
runCatching { cm.setPrimaryClip(ClipData.newPlainText("Punktfunk", text)) }
}
// "state"/"cancel"/"error": nothing to drive in the text-only v1.
}
}
}
@@ -54,6 +54,21 @@ class MainActivity : ComponentActivity() {
var padKeyProbe: ((KeyEvent) -> Boolean)? = null
var padMotionProbe: ((MotionEvent) -> Boolean)? = null
/**
* Physical-mouse forwarder for the active session (built/released by StreamScreen, like
* [gamepadRouter]): uncaptured hover/click/wheel forwards as absolute cursor input, captured
* ([android.view.View.requestPointerCapture]) raw deltas as relative mouse-look. The dispatch
* overrides below route every SOURCE_MOUSE event here while streaming. Null while not streaming.
*/
var mouseForwarder: MouseForwarder? = null
/**
* TV remote-as-pointer for the active session (StreamScreen builds it on TV devices only):
* hold SELECT to toggle, then the D-pad glides the host cursor. Consulted first for
* non-gamepad keys while streaming. Null while not streaming or not a TV.
*/
var remotePointer: RemotePointer? = null
/**
* Set by [StreamScreen] to its disconnect action. The emergency-exit chord (below) invokes it so a
* couch user with no keyboard/Back can always leave a stream.
@@ -324,9 +339,29 @@ class MainActivity : ComponentActivity() {
return true // consumed
}
}
// TV remote-as-pointer sees non-gamepad keys first (SELECT long-press toggles it;
// while active it owns the D-pad/SELECT/PLAY-PAUSE/BACK).
if (!event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
remotePointer?.let { if (it.onKey(event)) return true }
}
// Ctrl+Alt+Shift+Q — the cross-client pointer-capture toggle chord. Swallow both
// edges of the Q (the modifiers already went over the wire, exactly like desktop).
if (event.keyCode == KeyEvent.KEYCODE_Q &&
event.isCtrlPressed && event.isAltPressed && event.isShiftPressed
) {
if (event.action == KeyEvent.ACTION_DOWN && event.repeatCount == 0) {
mouseForwarder?.toggleCapture()
}
return true
}
when (event.keyCode) {
// A mouse back/forward button whose BUTTON_* press went unconsumed makes the
// framework synthesize a FALLBACK BACK — the button already went over the wire
// as X1/X2, and it must never yank the user out of the stream.
KeyEvent.KEYCODE_BACK ->
if (event.flags and KeyEvent.FLAG_FALLBACK != 0) return true
// Leave these to the system even while streaming.
KeyEvent.KEYCODE_BACK, // → BackHandler leaves the stream
// (BACK above → BackHandler leaves the stream.)
KeyEvent.KEYCODE_VOLUME_UP,
KeyEvent.KEYCODE_VOLUME_DOWN,
KeyEvent.KEYCODE_VOLUME_MUTE,
@@ -394,6 +429,10 @@ class MainActivity : ComponentActivity() {
override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean {
if (streamHandle != 0L) {
if (gamepadRouter?.onMotion(event) == true) return true
// Physical mouse (uncaptured): hover motion, wheel, button edges.
if (event.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onGenericMotion(event)) return true }
}
return super.dispatchGenericMotionEvent(event)
}
// The Controllers debug screen sees pad motion before the stick→D-pad synthesis below.
@@ -431,6 +470,24 @@ class MainActivity : ComponentActivity() {
return super.dispatchGenericMotionEvent(event)
}
/**
* Mouse clicks/drags ride the TOUCH stream (the pointer is "down"). While streaming they
* belong to the mouse forwarder, never to the Compose touch-gesture layer — a physical
* mouse click must be a real click at the cursor, not a synthesized trackpad tap.
*/
override fun dispatchTouchEvent(ev: MotionEvent): Boolean {
if (streamHandle != 0L && ev.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onTouchEvent(ev)) return true }
}
return super.dispatchTouchEvent(ev)
}
/** The OS is the source of truth for pointer capture (it releases on focus loss). */
override fun onPointerCaptureChanged(hasCapture: Boolean) {
super.onPointerCaptureChanged(hasCapture)
mouseForwarder?.onCaptureChanged(hasCapture)
}
/** Keys that drive the console UI — D-pad + face buttons; used to classify the last input source. */
private fun isConsoleNavKey(kc: Int): Boolean = when (kc) {
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN, KeyEvent.KEYCODE_DPAD_LEFT,
@@ -0,0 +1,206 @@
package io.unom.punktfunk
import android.view.InputDevice
import android.view.MotionEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.roundToInt
/** True when any connected input device is a pointer (USB/BT mouse, or a touchpad driving one). */
fun hasPhysicalMouse(): Boolean = InputDevice.getDeviceIds().any { id ->
InputDevice.getDevice(id)?.supportsSource(InputDevice.SOURCE_MOUSE) == true
}
/**
* Physical mouse → wire, in two modes (the iPadOS/desktop model):
* * **uncaptured** (default): hover/drag positions forward as absolute cursor moves
* (`MouseMoveAbs`, host-normalized against the window size) — desktop-style pointing. The
* local cursor is hidden over the stream (StreamScreen sets a TYPE_NULL pointer icon); the
* host's own cursor, composited into the video, is the one you see.
* * **captured**: the OS pointer is grabbed ([android.view.View.requestPointerCapture]) and raw
* relative deltas forward as `MouseMove` — FPS mouse-look. Engaged at stream start / by
* clicking into the stream when the "Capture pointer for games" setting is on, and toggled
* any time by Ctrl+Alt+Shift+Q (the cross-client chord). Focus loss releases it (the OS
* guarantees that); a click re-engages.
*
* Buttons ride [MotionEvent.ACTION_BUTTON_PRESS]/RELEASE edges (left/middle/right/back/forward →
* wire 1/2/3/4/5), the wheel rides [MotionEvent.ACTION_SCROLL] with fractional accumulation so
* high-resolution wheels don't lose sub-notch travel. Held buttons are tracked and flushed on
* capture loss / stream exit so nothing sticks on the host. Events reach this class from
* MainActivity's dispatch overrides (uncaptured) and the capture view's captured-pointer listener.
*/
class MouseForwarder(
private val handle: Long,
private val invertScroll: Boolean,
private val captureWanted: Boolean,
private val surfaceSize: () -> Pair<Int, Int>,
) {
/** Capture plumbing, owned by StreamScreen (the focusable capture view). */
var onRequestCapture: (() -> Unit)? = null
var onReleaseCapture: (() -> Unit)? = null
/** Live capture state, updated from [android.app.Activity.onPointerCaptureChanged]. */
var captured = false
private set
/** Chord-released: no auto re-engage (start / click) until the user opts back in. */
private var userReleased = false
private val heldButtons = mutableSetOf<Int>()
private var scrollAccV = 0f
private var scrollAccH = 0f
private var moveAccX = 0f
private var moveAccY = 0f
/** Uncaptured mouse events on the TOUCH stream (position while a button is down). */
fun onTouchEvent(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_DOWN -> {
if (captureWanted && !captured && !userReleased) {
// The engaging click: grab the pointer and swallow the click (desktop
// parity — the click that captures never reaches the host). The paired
// BUTTON_RELEASE is dropped by the held-set guard in [button].
onRequestCapture?.invoke()
return true
}
sendAbs(ev)
}
MotionEvent.ACTION_MOVE -> sendAbs(ev)
// Button edges are documented on the generic stream, but be robust to either.
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
}
return true
}
/** Uncaptured mouse events on the GENERIC stream (hover motion, wheel, button edges). */
fun onGenericMotion(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_HOVER_MOVE -> sendAbs(ev)
MotionEvent.ACTION_SCROLL -> wheel(ev)
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_EXIT -> {}
else -> return false
}
return true
}
/**
* Captured-pointer events (the view holds [android.view.View.requestPointerCapture]): x/y ARE
* the relative deltas ([InputDevice.SOURCE_MOUSE_RELATIVE]), batched samples included. A
* captured touchpad reports absolute finger coordinates instead — not handled (the touch
* gesture layer is the touchpad story); returning false leaves those to the framework.
*/
fun onCapturedPointer(ev: MotionEvent): Boolean {
if (!ev.isFromSource(InputDevice.SOURCE_MOUSE_RELATIVE)) return false
when (ev.actionMasked) {
MotionEvent.ACTION_MOVE -> {
var dx = 0f
var dy = 0f
for (i in 0 until ev.historySize) {
dx += ev.getHistoricalX(i)
dy += ev.getHistoricalY(i)
}
dx += ev.x
dy += ev.y
moveAccX += dx
moveAccY += dy
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept w/ sign
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_SCROLL -> wheel(ev)
}
return true
}
/** Ctrl+Alt+Shift+Q: release the grab, or (re-)engage it — works even when auto-capture is off. */
fun toggleCapture() {
if (captured) {
userReleased = true
onReleaseCapture?.invoke()
} else {
userReleased = false
onRequestCapture?.invoke()
}
}
/** Auto-engage at stream start (setting on + a mouse actually present). */
fun engageFromStart() {
if (captureWanted && !captured && !userReleased && hasPhysicalMouse()) {
onRequestCapture?.invoke()
}
}
/** From [android.app.Activity.onPointerCaptureChanged] — the OS is the source of truth. */
fun onCaptureChanged(has: Boolean) {
captured = has
// Losing the grab (focus loss, chord) must not leave buttons held on the host.
if (!has) flushButtons()
}
/** Stream teardown: lift anything held and let the grab go. */
fun release() {
flushButtons()
if (captured) onReleaseCapture?.invoke()
}
private fun sendAbs(ev: MotionEvent) {
val (w, h) = surfaceSize()
if (w <= 0 || h <= 0) return
NativeBridge.nativeSendPointerAbs(
handle,
ev.x.roundToInt().coerceIn(0, w - 1),
ev.y.roundToInt().coerceIn(0, h - 1),
w,
h,
)
}
private fun wheel(ev: MotionEvent) {
val dir = if (invertScroll) -1f else 1f
// Android: AXIS_VSCROLL + = up/away, AXIS_HSCROLL + = right — the wire's convention too.
scrollAccV += ev.getAxisValue(MotionEvent.AXIS_VSCROLL) * 120f * dir
scrollAccH += ev.getAxisValue(MotionEvent.AXIS_HSCROLL) * 120f * dir
val v = scrollAccV.toInt()
if (v != 0) {
NativeBridge.nativeSendScroll(handle, 0, v)
scrollAccV -= v
}
val h = scrollAccH.toInt()
if (h != 0) {
NativeBridge.nativeSendScroll(handle, 1, h)
scrollAccH -= h
}
}
private fun button(actionButton: Int, down: Boolean) {
val b = when (actionButton) {
MotionEvent.BUTTON_PRIMARY -> 1
MotionEvent.BUTTON_TERTIARY -> 2
MotionEvent.BUTTON_SECONDARY -> 3
MotionEvent.BUTTON_BACK -> 4
MotionEvent.BUTTON_FORWARD -> 5
else -> return
}
if (down) {
heldButtons.add(b)
NativeBridge.nativeSendPointerButton(handle, b, true)
} else if (heldButtons.remove(b)) {
// Only release what we pressed — drops the release of a swallowed engaging click
// and anything that raced a capture transition.
NativeBridge.nativeSendPointerButton(handle, b, false)
}
}
private fun flushButtons() {
heldButtons.forEach { NativeBridge.nativeSendPointerButton(handle, it, false) }
heldButtons.clear()
}
}
@@ -0,0 +1,193 @@
package io.unom.punktfunk
import android.os.Handler
import android.os.Looper
import android.view.Choreographer
import android.view.KeyEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.hypot
// Hold this long on SELECT (pointer-mode toggle) / PLAY-PAUSE (keyboard toggle) for the long-press
// action instead of the tap action.
private const val LONG_PRESS_MS = 800L
// D-pad glide ballistics, in screen-widths per second: start slow enough to hit a close button,
// ramp over RAMP_S seconds of continuous hold so crossing the desktop doesn't take all day.
private const val SPEED_MIN = 0.14f
private const val SPEED_MAX = 0.70f
private const val RAMP_S = 1.2f
/**
* Android TV remote as a pointer — the Android analogue of the Apple client's Siri-remote pointer,
* adapted for D-pad-only remotes (most Android TV remotes have no touch surface). For the
* "TV as a desktop client" use case, where a plain remote is often the only thing in hand.
*
* While streaming on a TV, **hold SELECT ≈ 0.8 s** to toggle pointer mode. While active:
* * D-pad (held) glides the host cursor with ramping acceleration (relative `MouseMove`,
* Choreographer-paced, diagonal-normalized);
* * SELECT tap = left click; PLAY/PAUSE tap = right click (Siri-remote parity);
* * PLAY/PAUSE held = toggle the on-screen keyboard; BACK = leave pointer mode
* (a second BACK then leaves the stream as usual).
* While inactive, everything except the SELECT long-press passes through untouched (D-pad =
* arrow keys, SELECT tap = Enter — synthesized on release, since the down was held back to
* disambiguate the long-press).
*
* Only consulted for non-gamepad key events on TV devices (MainActivity gates the calls); all
* state lives on the main thread.
*/
class RemotePointer(
private val handle: Long,
private val surfaceWidth: () -> Int,
private val onActiveChanged: (Boolean) -> Unit,
private val onKeyboardToggle: () -> Unit,
) {
var active = false
private set
private val handler = Handler(Looper.getMainLooper())
private val held = mutableSetOf<Int>() // D-pad keycodes currently down
private var moveAccX = 0f
private var moveAccY = 0f
private var lastFrameNs = 0L
private var rampSec = 0f
private var tickerRunning = false
private var centerLongFired = false
private var playLongFired = false
private val centerLong = Runnable {
centerLongFired = true
toggle()
}
private val playLong = Runnable {
playLongFired = true
onKeyboardToggle()
}
private val frame = object : Choreographer.FrameCallback {
override fun doFrame(nowNs: Long) {
if (!tickerRunning) return
if (held.isEmpty() || !active) {
tickerRunning = false
return
}
val dt = if (lastFrameNs == 0L) {
1f / 60f
} else {
((nowNs - lastFrameNs) / 1e9f).coerceIn(0.001f, 0.1f)
}
lastFrameNs = nowNs
rampSec += dt
var vx = 0f
var vy = 0f
if (KeyEvent.KEYCODE_DPAD_LEFT in held) vx -= 1f
if (KeyEvent.KEYCODE_DPAD_RIGHT in held) vx += 1f
if (KeyEvent.KEYCODE_DPAD_UP in held) vy -= 1f
if (KeyEvent.KEYCODE_DPAD_DOWN in held) vy += 1f
val mag = hypot(vx, vy)
if (mag > 0f) {
val w = surfaceWidth().coerceAtLeast(640)
val speed = w * (SPEED_MIN + (SPEED_MAX - SPEED_MIN) * (rampSec / RAMP_S).coerceAtMost(1f))
moveAccX += vx / mag * speed * dt
moveAccY += vy / mag * speed * dt
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
Choreographer.getInstance().postFrameCallback(this)
}
}
/** One remote key event; true = consumed. Ignore key repeats — the ticker owns motion. */
fun onKey(event: KeyEvent): Boolean {
val down = event.action == KeyEvent.ACTION_DOWN
when (event.keyCode) {
KeyEvent.KEYCODE_DPAD_CENTER -> {
if (down) {
if (event.repeatCount == 0) {
centerLongFired = false
handler.postDelayed(centerLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(centerLong)
if (!centerLongFired) {
if (active) {
click(1)
} else {
// The down was held back to disambiguate the long-press, so the
// normal path never saw it — synthesize the Enter here instead.
NativeBridge.nativeSendKey(handle, 0x0D, true, 0)
NativeBridge.nativeSendKey(handle, 0x0D, false, 0)
}
}
}
return true
}
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN,
KeyEvent.KEYCODE_DPAD_LEFT, KeyEvent.KEYCODE_DPAD_RIGHT,
-> {
if (!active) return false
if (down) {
if (held.add(event.keyCode) && held.size == 1) startTicker()
} else {
held.remove(event.keyCode)
}
return true
}
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE -> {
if (!active) return false // inactive: the media-key VK path owns it
if (down) {
if (event.repeatCount == 0) {
playLongFired = false
handler.postDelayed(playLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(playLong)
if (!playLongFired) click(3)
}
return true
}
KeyEvent.KEYCODE_BACK -> {
if (!active) return false
if (!down) toggle() // leave pointer mode; the next BACK leaves the stream
return true
}
else -> return false
}
}
/** Stream teardown: stop timers/ticker; nothing wire-held to flush (clicks are edges). */
fun release() {
handler.removeCallbacks(centerLong)
handler.removeCallbacks(playLong)
active = false
held.clear()
tickerRunning = false
}
private fun toggle() {
active = !active
if (!active) {
held.clear()
tickerRunning = false
}
onActiveChanged(active)
}
private fun startTicker() {
rampSec = 0f
lastFrameNs = 0L
if (!tickerRunning) {
tickerRunning = true
Choreographer.getInstance().postFrameCallback(frame)
}
}
private fun click(button: Int) {
NativeBridge.nativeSendPointerButton(handle, button, true)
NativeBridge.nativeSendPointerButton(handle, button, false)
}
}
@@ -109,6 +109,27 @@ data class Settings(
* setup where the OS-level pad (lizard mode) is preferred.
*/
val sc2Capture: Boolean = true,
/**
* Lock a physical mouse to the stream ([android.view.View.requestPointerCapture]) and forward
* raw relative motion — FPS mouse-look, the iPad "Capture pointer for games" twin. Engages at
* stream start and on a click into the stream; Ctrl+Alt+Shift+Q toggles it live (the chord
* works even with this off). Off (default): a mouse points absolutely, desktop-style.
*/
val pointerCapture: Boolean = false,
/**
* Flip scroll direction — the mouse wheel and the two-finger touch scroll both. Parity with
* the Apple/GTK clients' "Invert scroll direction".
*/
val invertScroll: Boolean = false,
/**
* Sync text copied on this device to the host and vice versa while streaming (the desktop
* clients' shared clipboard, text-only here). Only effective when the host advertises the
* clipboard capability; the protocol is opt-in per session either way.
*/
val clipboardSync: Boolean = true,
)
/** [Settings.touchMode] values; persisted by name. */
@@ -172,6 +193,9 @@ class SettingsStore(context: Context) {
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false),
sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true),
pointerCapture = prefs.getBoolean(K_POINTER_CAPTURE, false),
invertScroll = prefs.getBoolean(K_INVERT_SCROLL, false),
clipboardSync = prefs.getBoolean(K_CLIPBOARD_SYNC, true),
)
fun save(s: Settings) {
@@ -195,6 +219,9 @@ class SettingsStore(context: Context) {
.putBoolean(K_AUTO_WAKE, s.autoWakeEnabled)
.putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone)
.putBoolean(K_SC2_CAPTURE, s.sc2Capture)
.putBoolean(K_POINTER_CAPTURE, s.pointerCapture)
.putBoolean(K_INVERT_SCROLL, s.invertScroll)
.putBoolean(K_CLIPBOARD_SYNC, s.clipboardSync)
.apply()
}
@@ -233,6 +260,9 @@ class SettingsStore(context: Context) {
const val K_AUTO_WAKE = "auto_wake_enabled"
const val K_RUMBLE_ON_PHONE = "rumble_on_phone"
const val K_SC2_CAPTURE = "sc2_capture"
const val K_POINTER_CAPTURE = "pointer_capture"
const val K_INVERT_SCROLL = "invert_scroll"
const val K_CLIPBOARD_SYNC = "clipboard_sync"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */
const val K_TRACKPAD = "trackpad_mode"
@@ -412,6 +412,27 @@ private fun ControlsSettings(s: Settings, update: (Settings) -> Unit, onOpenCont
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
ToggleRow(
title = "Capture pointer for games",
subtitle = "Lock a connected mouse to the stream and send raw relative motion " +
"(mouse-look). Ctrl+Alt+Shift+Q toggles it live; click the stream to re-capture. " +
"Off: the mouse points at the desktop directly",
checked = s.pointerCapture,
onCheckedChange = { on -> update(s.copy(pointerCapture = on)) },
)
ToggleRow(
title = "Invert scroll direction",
subtitle = "Flip the mouse wheel and two-finger touch scrolling",
checked = s.invertScroll,
onCheckedChange = { on -> update(s.copy(invertScroll = on)) },
)
ToggleRow(
title = "Shared clipboard",
subtitle = "Text copied here pastes on the host and vice versa (hosts with " +
"clipboard sharing enabled)",
checked = s.clipboardSync,
onCheckedChange = { on -> update(s.copy(clipboardSync = on)) },
)
}
SettingsCard {
SettingDropdown(
@@ -176,6 +176,14 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// "hold to quit" hint overlay. Set from the router's onExitArmed (main thread).
var exitArming by remember { mutableStateOf(false) }
// True while the TV remote is acting as a pointer (hold SELECT toggles) — drives the mode hint.
var remotePointerOn by remember { mutableStateOf(false) }
// Focus anchor the soft keyboard is summoned onto AND the pointer-capture grab target (a grab
// needs a focusable view; captured-pointer events land on it). Declared before the effect
// below so the capture callbacks can reach the view once it exists.
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) }
DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
wifiLocks.forEach { lock ->
@@ -221,6 +229,54 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Show a "hold to quit" hint the moment the chord completes (the router debounces the actual
// exit); it clears when the buttons release early or the hold elapses. Runs on the main thread.
router.onExitArmed = { armed -> exitArming = armed }
// Physical mouse: uncaptured hover/click/wheel forwards as absolute pointing; captured
// (setting or the Ctrl+Alt+Shift+Q chord) raw deltas forward as relative mouse-look.
// The local cursor is hidden over the stream — the host's own cursor, composited into
// the video, is the one the user sees (twin of the desktop clients' hidden cursor).
val decor = window?.decorView
val priorPointerIcon = decor?.pointerIcon
decor?.pointerIcon = android.view.PointerIcon.getSystemIcon(
context,
android.view.PointerIcon.TYPE_NULL,
)
val mouse = MouseForwarder(
handle,
invertScroll = initialSettings.invertScroll,
captureWanted = initialSettings.pointerCapture,
surfaceSize = { (decor?.width ?: 0) to (decor?.height ?: 0) },
)
mouse.onRequestCapture = {
// The grab needs the (focusable) capture view: focus it, then ask. Posted so a
// request racing view attach/focus settles on the next frame.
keyCapture?.let { v ->
v.post {
v.requestFocus()
v.requestPointerCapture()
}
}
}
mouse.onReleaseCapture = { keyCapture?.releasePointerCapture() }
activity?.mouseForwarder = mouse
// TV remote-as-pointer: hold SELECT ≈ 0.8 s to toggle; the D-pad then glides the host
// cursor (see RemotePointer). TV only — a phone's remote-less keys stay on the VK path.
val remote = if (isTv) {
RemotePointer(
handle,
surfaceWidth = { decor?.width ?: 1920 },
onActiveChanged = { on -> remotePointerOn = on },
onKeyboardToggle = { keyCapture?.let { it.setImeVisible(!it.imeShown) } },
)
} else {
null
}
activity?.remotePointer = remote
// Shared clipboard (text v1): only when the user setting is on AND the host has a
// working clipboard service. Protocol-level opt-in + the poll thread live in the sync.
val clip = if (initialSettings.clipboardSync && NativeBridge.nativeClipSupported(handle)) {
ClipboardSync(context, handle).also { it.start() }
} else {
null
}
activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate
// Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad
// index via the router; poll threads stopped + joined before the router is released and the
@@ -286,6 +342,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
}
onDispose {
closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it
clip?.stop() // stop + join the clipboard poll thread BEFORE the handle is freed
feedback.onHidRaw = null
feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed
sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
@@ -293,6 +350,12 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down
router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener
activity?.gamepadRouter = null
// Mouse/remote-pointer teardown: lift held buttons, drop the grab, restore the cursor.
mouse.release()
activity?.mouseForwarder = null
remote?.release()
activity?.remotePointer = null
decor?.pointerIcon = priorPointerIcon
activity?.streamHandle = 0L
activity?.requestStreamExit = null
// Back in the menus: the SC2 (if present) resumes driving the console UI.
@@ -320,8 +383,12 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
// Focus anchor the three-finger keyboard swipe summons the IME onto (see KeyCaptureView).
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) }
// Auto-engage pointer capture at stream start (setting on + a mouse actually present).
// Delayed a beat: the grab needs window focus and the capture view attached.
LaunchedEffect(handle) {
delay(400)
activity?.mouseForwarder?.engageFromStart()
}
Box(modifier = Modifier.fillMaxSize()) {
AndroidView(
@@ -379,11 +446,26 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
if (exitArming) {
ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
}
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe
// up in the mouse modes) — it never draws or takes touches, it just owns IME focus.
// Remote-pointer mode hint — the remote's keys are remapped while it's on, so say so.
if (remotePointerOn) {
RemotePointerHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
}
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe up
// in the mouse modes) AND the pointer-capture grab target — it never draws or takes
// touches, it just owns IME focus and receives captured-pointer events.
AndroidView(
modifier = Modifier.size(1.dp),
factory = { ctx -> KeyCaptureView(ctx).also { keyCapture = it } },
factory = { ctx ->
KeyCaptureView(ctx).also { v ->
keyCapture = v
// Real IME text path when the host types committed text (see KeyCaptureView).
v.textHandle =
if (NativeBridge.nativeTextInputSupported(handle)) handle else 0L
v.setOnCapturedPointerListener { _, ev ->
(ctx as? MainActivity)?.mouseForwarder?.onCapturedPointer(ev) ?: false
}
}
},
)
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no
@@ -396,6 +478,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
else -> streamTouchInput(
handle,
trackpad = touchMode == TouchMode.TRACKPAD,
invertScroll = initialSettings.invertScroll,
onCycleStats = { statsVerbosity = statsVerbosity.next() },
onKeyboard = { show -> keyCapture?.setImeVisible(show) },
)
@@ -423,14 +506,35 @@ private fun ExitChordHint(modifier: Modifier = Modifier) {
)
}
/**
* The remote-pointer mode cue: while active the remote's keys are remapped (D-pad glides the host
* cursor, SELECT clicks), so the overlay both confirms the toggle and teaches the vocabulary.
*/
@Composable
private fun RemotePointerHint(modifier: Modifier = Modifier) {
Text(
"Remote pointer — SELECT click · play/pause right-click · hold SELECT to exit",
modifier = modifier
.background(Color.Black.copy(alpha = 0.55f), RoundedCornerShape(8.dp))
.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White,
fontSize = 15.sp,
)
}
/**
* Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME
* onto this view. `TYPE_NULL` puts the IME in "dumb keyboard" mode — it delivers raw [KeyEvent]s
* (no composing text, no autocorrect), which flow through `MainActivity.dispatchKeyEvent` →
* `Keymap.toVk` → the host, the exact path a hardware keyboard takes. Text an IME insists on
* committing instead still arrives: the non-editable [BaseInputConnection] synthesizes KeyEvents
* for it via `KeyCharacterMap` (with Shift carried as meta state — see the IME-shift wrap in
* `MainActivity.dispatchKeyEvent`).
* onto this view. Two IME models, picked by the host's capabilities:
* * **Text path** ([textHandle] set — the host advertised `HOST_CAP_TEXT_INPUT`): a real
* editable [HostTextConnection], so the IME gives autocorrect, gesture typing, non-Latin
* composition and emoji, all mirrored to the host as committed text + diffs.
* * **Fallback** (older host): `TYPE_NULL` puts the IME in "dumb keyboard" mode — raw
* [KeyEvent]s flow through `MainActivity.dispatchKeyEvent` → `Keymap.toVk` → the host, the
* exact path a hardware keyboard takes (with the IME-shift wrap documented there).
*
* Doubles as the pointer-capture grab target: a grab needs a focusable view, and captured-pointer
* events are delivered to it (routed to [MouseForwarder.onCapturedPointer] via the listener the
* stream screen installs).
*/
private class KeyCaptureView(context: Context) : View(context) {
init {
@@ -438,17 +542,32 @@ private class KeyCaptureView(context: Context) : View(context) {
isFocusableInTouchMode = true
}
/** The session handle when the host types committed text; `0` = VK-only fallback. */
var textHandle: Long = 0L
/** Whether [setImeVisible] last showed the IME — for toggle-style callers (remote pointer). */
var imeShown = false
private set
override fun onCheckIsTextEditor(): Boolean = true
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection {
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or
EditorInfo.IME_FLAG_NO_FULLSCREEN or EditorInfo.IME_FLAG_NO_ENTER_ACTION
return if (textHandle != 0L) {
outAttrs.inputType = InputType.TYPE_CLASS_TEXT or
InputType.TYPE_TEXT_FLAG_AUTO_CORRECT or InputType.TYPE_TEXT_FLAG_MULTI_LINE
HostTextConnection(this, textHandle)
} else {
outAttrs.inputType = InputType.TYPE_NULL
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or EditorInfo.IME_FLAG_NO_FULLSCREEN
return BaseInputConnection(this, false)
BaseInputConnection(this, false)
}
}
fun setImeVisible(show: Boolean) {
val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager
?: return
imeShown = show
if (show) {
requestFocus()
imm.showSoftInput(this, 0)
@@ -457,3 +576,113 @@ private class KeyCaptureView(context: Context) : View(context) {
}
}
}
/**
* IME → host text bridge (the `HOST_CAP_TEXT_INPUT` path): a real **editable** connection, so
* the IME runs its full machinery (autocorrect, gesture typing, non-Latin composition), mirrored
* to the host as it happens. The one piece of host-side state tracked is *what the host currently
* shows of the active composition* ([sentComposition]): composing updates send a common-prefix
* diff (backspaces + the new suffix) so corrections materialize live on the host; a commit
* settles it. [setComposingRegion] adopts already-committed text as the active composition
* (autocorrect-revert / backspace-into-word flows), so the next update diffs against it instead
* of retyping. Newlines become Enter taps; [deleteSurroundingText] becomes Backspace/Delete taps.
*
* Known approximation: diff lengths are counted in Unicode scalars, assuming one host Backspace
* deletes one scalar — true for the composition text IMEs actually produce (emoji and other
* multi-unit graphemes commit directly rather than composing).
*/
private class HostTextConnection(
view: KeyCaptureView,
private val handle: Long,
) : BaseInputConnection(view, true) {
/** What the host currently shows of the active composition ("" = none). */
private var sentComposition = ""
override fun commitText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
sentComposition = ""
val ok = super.commitText(text, newCursorPosition)
trimEditable()
return ok
}
override fun setComposingText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
return super.setComposingText(text, newCursorPosition)
}
override fun finishComposingText(): Boolean {
// The composition text stands as committed — the host already shows it verbatim.
sentComposition = ""
return super.finishComposingText()
}
override fun setComposingRegion(start: Int, end: Int): Boolean {
val e = editable
if (e != null) {
val a = start.coerceIn(0, e.length)
val b = end.coerceIn(0, e.length)
sentComposition = e.subSequence(minOf(a, b), maxOf(a, b)).toString()
}
return super.setComposingRegion(start, end)
}
override fun deleteSurroundingText(beforeLength: Int, afterLength: Int): Boolean {
repeat(beforeLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_BACK) }
repeat(afterLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_DELETE) }
return super.deleteSurroundingText(beforeLength, afterLength)
}
override fun performEditorAction(actionCode: Int): Boolean {
tapVk(VK_RETURN)
return true
}
/** Replace the host's view of the composition with [text] via a common-prefix diff. */
private fun retype(text: String) {
var common = sentComposition.commonPrefixWith(text)
// Never split a surrogate pair mid-diff — back off to the pair boundary.
if (common.isNotEmpty() && common.last().isHighSurrogate()) {
common = common.dropLast(1)
}
val stale = sentComposition.substring(common.length)
repeat(stale.codePointCount(0, stale.length).coerceAtMost(MAX_TAPS)) { tapVk(VK_BACK) }
sendText(text.substring(common.length))
sentComposition = text
}
/** Forward literal text, turning newlines into Enter taps (control chars never ride text). */
private fun sendText(s: String) {
var chunk = StringBuilder()
for (ch in s) {
if (ch == '\n') {
if (chunk.isNotEmpty()) {
NativeBridge.nativeSendText(handle, chunk.toString())
chunk = StringBuilder()
}
tapVk(VK_RETURN)
} else {
chunk.append(ch)
}
}
if (chunk.isNotEmpty()) NativeBridge.nativeSendText(handle, chunk.toString())
}
private fun tapVk(vk: Int) {
NativeBridge.nativeSendKey(handle, vk, true, 0)
NativeBridge.nativeSendKey(handle, vk, false, 0)
}
/** Bound the mirror buffer: once nothing is composing, old text serves no purpose. */
private fun trimEditable() {
val e = editable ?: return
if (getComposingSpanStart(e) == -1 && e.length > 4000) e.clear()
}
private companion object {
const val VK_BACK = 0x08
const val VK_RETURN = 0x0D
const val VK_DELETE = 0x2E
const val MAX_TAPS = 256
}
}
@@ -99,9 +99,11 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
internal suspend fun PointerInputScope.streamTouchInput(
handle: Long,
trackpad: Boolean,
invertScroll: Boolean,
onCycleStats: () -> Unit,
onKeyboard: (show: Boolean) -> Unit,
) {
val scrollDir = if (invertScroll) -1 else 1
var lastTapUp = 0L
var lastTapX = 0f
var lastTapY = 0f
@@ -184,12 +186,12 @@ internal suspend fun PointerInputScope.streamTouchInput(
val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up
val sx = ((cx - prevCx) / SCROLL_DIV).toInt()
if (sy != 0) {
NativeBridge.nativeSendScroll(handle, 0, sy * 120)
NativeBridge.nativeSendScroll(handle, 0, sy * 120 * scrollDir)
prevCy = cy
moved = true
}
if (sx != 0) {
NativeBridge.nativeSendScroll(handle, 1, sx * 120)
NativeBridge.nativeSendScroll(handle, 1, sx * 120 * scrollDir)
prevCx = cx
moved = true
}
@@ -106,6 +106,17 @@ object Keymap {
KeyEvent.KEYCODE_DPAD_UP -> 0x26
KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27
KeyEvent.KEYCODE_DPAD_DOWN -> 0x28
// TV-remote SELECT = Enter (a gamepad's press routes via SOURCE_GAMEPAD before this).
KeyEvent.KEYCODE_DPAD_CENTER -> 0x0D
// Consumer/media keys — forwarded to the host while streaming (volume stays local:
// MainActivity's pass-through list wins before the map is consulted).
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE,
KeyEvent.KEYCODE_MEDIA_PLAY,
KeyEvent.KEYCODE_MEDIA_PAUSE -> 0xB3 // VK_MEDIA_PLAY_PAUSE
KeyEvent.KEYCODE_MEDIA_NEXT -> 0xB0 // VK_MEDIA_NEXT_TRACK
KeyEvent.KEYCODE_MEDIA_PREVIOUS -> 0xB1 // VK_MEDIA_PREV_TRACK
KeyEvent.KEYCODE_MEDIA_STOP -> 0xB2 // VK_MEDIA_STOP
// Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant)
KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0
@@ -287,6 +287,50 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
/**
* Whether the host advertised committed-text injection (`HOST_CAP_TEXT_INPUT`) — its inject
* backend can type Unicode text directly. Picks the real IME `InputConnection` (autocorrect,
* gesture typing, non-Latin scripts) over the TYPE_NULL raw-key fallback. False on `0`.
*/
external fun nativeTextInputSupported(handle: Long): Boolean
/**
* Committed IME text → one `TextInput` wire event per Unicode scalar, in order. Control
* characters are skipped natively (Enter/Backspace ride [nativeSendKey]). Only meaningful
* when [nativeTextInputSupported] returned true — older hosts ignore the events.
*/
external fun nativeSendText(handle: Long, text: String)
// ---- Shared clipboard (text v1): Kotlin drives ClipboardManager, Rust the protocol ----
// Opt-in per session (nativeClipControl). Local copies are announced as lazy offers; bytes
// cross only when the host pastes (a "fetch:" event answered by nativeClipServeText). Host
// copies arrive as "offer:" events, fetched eagerly into the system clipboard.
/** Whether the host advertised a working shared-clipboard service (HOST_CAP_CLIPBOARD). */
external fun nativeClipSupported(handle: Long): Boolean
/** Session-level clipboard opt-in/out; nothing happens until enabled=true crosses. */
external fun nativeClipControl(handle: Long, enabled: Boolean)
/** Announce "this device's clipboard now holds text". [seq]: monotonic, newest wins. */
external fun nativeClipOfferText(handle: Long, seq: Int)
/** Pull the text of the host's offer [seq] → transfer id echoed on "data:"/"error:", or -1. */
external fun nativeClipFetchText(handle: Long, seq: Int): Int
/** Answer a "fetch:" event with the clipboard's current text (the host is pasting). */
external fun nativeClipServeText(handle: Long, reqId: Int, text: String)
/** Abort a clipboard transfer by id (either direction). */
external fun nativeClipCancel(handle: Long, id: Int)
/**
* Block ≤250 ms for the next clipboard event, as a compact string: `state:<0|1>` ·
* `offer:<seq>:<hasText>` · `fetch:<reqId>` · `data:<xferId>:<text>` · `cancel:<id>` ·
* `error:<id>:<code>` · `closed` (session gone) — null on timeout. Dedicated poll thread.
*/
external fun nativeNextClip(handle: Long): String?
// ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ----
// The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0,
// so its wire is byte-identical to the old single-pad path. The core folds the per-transition
@@ -0,0 +1,182 @@
//! Shared-clipboard plane (text-only v1): Kotlin drives the Android `ClipboardManager`, these
//! shims drive [`punktfunk_core::client::NativeClient`]'s clipboard surface.
//!
//! Model (mirrors the desktop clients): opt-in via `nativeClipControl(true)`; local copies are
//! announced lazily as format-list offers (`nativeClipOfferText`) and the bytes cross only when
//! the host pastes (a `fetch` event answered by `nativeClipServeText`); a host copy arrives as an
//! `offer` event, which the Kotlin side fetches eagerly (Android's clipboard has no lazy provider
//! path worth the complexity) and lands in the system clipboard on the `data` event.
//!
//! Events cross to Kotlin as compact strings from the blocking `nativeNextClip` poll (drained on
//! a dedicated thread, same pattern as `nativeNextRumble`):
//! `state:<0|1>` · `offer:<seq>:<has_text 0|1>` · `fetch:<req_id>` · `data:<xfer_id>:<text>` ·
//! `cancel:<id>` · `error:<id>:<code>` · `closed` — null on a poll timeout. Non-text fetch
//! requests are cancelled natively (only text is ever offered, so they shouldn't occur).
use std::time::Duration;
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jint, jlong, jstring};
use jni::JNIEnv;
use punktfunk_core::clipboard::ClipEventCore;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::quic::{ClipKind, CLIP_FILE_INDEX_NONE, HOST_CAP_CLIPBOARD};
use super::SessionHandle;
/// The portable wire MIME both ends map to their platform text type.
const TEXT_MIME: &str = "text/plain;charset=utf-8";
/// Deref the opaque handle (`0` → `None`).
///
/// SAFETY: live handle per the nativeConnect/nativeClose contract; every method used is `&self`
/// on the `Sync` connector.
fn client(handle: jlong) -> Option<&'static SessionHandle> {
if handle == 0 {
return None;
}
// SAFETY: see the function docs — the Kotlin side guarantees the handle outlives the call.
Some(unsafe { &*(handle as *const SessionHandle) })
}
/// `NativeBridge.nativeClipSupported(handle)` — the host advertised `HOST_CAP_CLIPBOARD`.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
client(handle).map_or(0, |h| {
u8::from(h.client.host_caps() & HOST_CAP_CLIPBOARD != 0)
})
}
/// `NativeBridge.nativeClipControl(handle, enabled)` — session-level opt-in/out. Nothing
/// clipboard-related happens on either side until an `enabled: true` crosses.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipControl(
_env: JNIEnv,
_this: JObject,
handle: jlong,
enabled: jboolean,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_control(enabled != 0, 0);
}
}
/// `NativeBridge.nativeClipOfferText(handle, seq)` — announce "the Android clipboard now holds
/// text" (format list only; bytes cross when the host fetches). `seq` is Kotlin's monotonic
/// counter, newest wins.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipOfferText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_offer(
seq as u32,
vec![ClipKind {
mime: TEXT_MIME.into(),
size_hint: 0,
}],
);
}
}
/// `NativeBridge.nativeClipFetchText(handle, seq)` — pull the text of the host's offer `seq`.
/// Returns the transfer id echoed on the matching `data:`/`error:` event, or 1.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipFetchText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) -> jint {
client(handle)
.and_then(|h| {
h.client
.clip_fetch(seq as u32, TEXT_MIME.into(), CLIP_FILE_INDEX_NONE)
.ok()
})
.map_or(-1, |xfer| xfer as jint)
}
/// `NativeBridge.nativeClipServeText(handle, reqId, text)` — answer a `fetch:` event with the
/// clipboard's current text (the host is pasting our offer).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipServeText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
req_id: jint,
text: JString,
) {
let Some(h) = client(handle) else { return };
let Ok(s) = env.get_string(&text) else {
let _ = h.client.clip_cancel(req_id as u32);
return;
};
let _ = h
.client
.clip_serve(req_id as u32, String::from(s).into_bytes(), true);
}
/// `NativeBridge.nativeClipCancel(handle, id)` — abort a transfer (either direction).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipCancel(
_env: JNIEnv,
_this: JObject,
handle: jlong,
id: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_cancel(id as u32);
}
}
/// `NativeBridge.nativeNextClip(handle)` — block ≤250 ms for the next clipboard event, encoded
/// as a compact string (module docs); null on timeout, `"closed"` once the session is gone.
/// Call from a dedicated poll thread.
///
/// Text payloads ride `data:<xfer_id>:<text>` decoded lossily — safe because the phase-0
/// clipboard task delivers a whole payload in ONE event (`last = true`), so a chunk boundary
/// can never split a UTF-8 sequence.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextClip(
env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jstring {
let Some(h) = client(handle) else {
return std::ptr::null_mut();
};
let msg = match h.client.next_clip(Duration::from_millis(250)) {
Ok(ClipEventCore::State { enabled, .. }) => format!("state:{}", u8::from(enabled)),
Ok(ClipEventCore::RemoteOffer { seq, kinds }) => {
let has_text = kinds.iter().any(|k| k.mime.starts_with("text/plain"));
format!("offer:{seq}:{}", u8::from(has_text))
}
Ok(ClipEventCore::FetchRequest { req_id, mime, .. }) => {
if mime.starts_with("text/plain") {
format!("fetch:{req_id}")
} else {
// We only ever offer text; cancel anything else rather than stall the host.
let _ = h.client.clip_cancel(req_id);
return std::ptr::null_mut();
}
}
Ok(ClipEventCore::Data { xfer_id, bytes, .. }) => {
format!("data:{xfer_id}:{}", String::from_utf8_lossy(&bytes))
}
Ok(ClipEventCore::Cancelled { id }) => format!("cancel:{id}"),
Ok(ClipEventCore::Error { id, code }) => format!("error:{id}:{code}"),
Err(PunktfunkError::NoFrame) => return std::ptr::null_mut(),
Err(_) => "closed".into(),
};
env.new_string(msg)
.map(|s| s.into_raw())
.unwrap_or(std::ptr::null_mut())
}
+41 -2
View File
@@ -6,11 +6,11 @@
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::{JByteBuffer, JObject};
use jni::objects::{JByteBuffer, JObject, JString};
use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX, HOST_CAP_TEXT_INPUT};
use super::SessionHandle;
@@ -145,6 +145,45 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
send_event(handle, kind, vk as u32, 0, 0, mods as u32);
}
/// `NativeBridge.nativeTextInputSupported(handle)` — whether the host advertised
/// `HOST_CAP_TEXT_INPUT` (its inject backend types committed text), so the Kotlin side can pick
/// the real IME `InputConnection` over the TYPE_NULL raw-key fallback. `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_TEXT_INPUT != 0)
}
/// `NativeBridge.nativeSendText(handle, text)` — committed IME text, one `TextInput` event per
/// Unicode scalar (`code` = the scalar; multi-char commits are consecutive events in order).
/// Control characters are skipped — Enter/Backspace/Tab ride the VK key path. Call only when
/// [`Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported`] returned true.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
text: JString,
) {
if handle == 0 {
return;
}
let Ok(s) = env.get_string(&text) else {
return;
};
for ch in String::from(s).chars().filter(|c| !c.is_control()) {
send_event(handle, InputKind::TextInput, ch as u32, 0, 0, 0);
}
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried
// in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a
@@ -17,6 +17,7 @@
//! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode
//! renegotiation. Port the remaining orchestration from `clients/linux`.
mod clipboard;
mod connect;
mod input;
mod planes;
@@ -705,8 +705,11 @@ final class SessionModel: ObservableObject {
// captured before the 2026-07 floor policy); the appended trio carries the
// measured OS present floor and the floor-shaved values the HUD displays.
let line = String(
format: "fps=%d presents=%d e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
+ "decode_p50=%.1f display_p50=%.1f lost=%d "
// Swift Int is 64-bit %lld, NOT %d (which is a 32-bit C int); macOS 26's
// strict String(format:) validator rejects the %d/Int mismatch and drops
// the whole line (a cascade error that also mis-blames the float args).
format: "fps=%lld presents=%lld e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
+ "decode_p50=%.1f display_p50=%.1f lost=%lld "
+ "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f",
frames,
displayWindow?.count ?? 0,
@@ -43,6 +43,9 @@ struct GamepadSettingsView: View {
@AppStorage(DefaultsKey.presentPriority) private var presentPriority =
SettingsOptions.presentPriorityDefault
@AppStorage(DefaultsKey.smoothBuffer) private var smoothBuffer = 0
#if os(macOS)
@AppStorage(DefaultsKey.windowedSafePresent) private var windowedSafePresent = true
#endif
#if os(iOS)
@AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false
#endif
@@ -345,6 +348,22 @@ struct GamepadSettingsView: View {
detail: "Turn off to use the touch interface even with a controller connected.",
value: $gamepadUIEnabled),
]
#if os(macOS)
// The windowed safe-present toggle slots in after "Smoothness buffer" (staying inside
// the Video group) macOS only, mirroring the touch SettingsView's Presentation row
// (the DCP swapID-panic mitigation; see DefaultsKey.windowedSafePresent).
if let at = list.firstIndex(where: { $0.id == "smoothBuffer" }) {
list.insert(
toggleRow(
id: "windowedSafePresent", icon: "macwindow.badge.plus",
label: "Safe windowed presentation",
detail: "Windowed streams present in step with the compositor — avoids a "
+ "macOS display-driver crash on high-refresh displays, at a small "
+ "latency cost. Fullscreen always uses the fastest path.",
value: $windowedSafePresent),
at: at + 1)
}
#endif
#if os(iOS)
// The device-rumble mirror slots in after "Controller type" (staying inside the
// Controller group the next row carries the "Interface" header). iPhone only in
@@ -300,6 +300,18 @@ extension SettingsView {
+ "of added latency. Off shows frames as soon as they're ready.") {
Toggle("V-Sync", isOn: $vsync)
}
// The DCP swapID-panic mitigation's user handle (see DefaultsKey.windowedSafePresent
// for the saga). Default ON: turning it off re-arms a WHOLE-MACHINE kernel panic on
// affected setups, so the caption says so in plain words.
described(windowedSafePresent
? "Windowed streams present in step with the system compositor — avoids a macOS "
+ "display-driver crash seen on high-refresh displays, at a small latency "
+ "cost. Fullscreen always uses the fastest path."
: "Windowed streams use the fastest present path. On some high-refresh setups "
+ "this can crash macOS itself (kernel panic) — turn back on if your Mac "
+ "restarts during windowed streaming.") {
Toggle("Safe windowed presentation", isOn: $windowedSafePresent)
}
#endif
}
}
@@ -40,6 +40,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.smoothBuffer) var smoothBuffer = 0
#if os(macOS)
@AppStorage(DefaultsKey.vsync) var vsync = false
@AppStorage(DefaultsKey.windowedSafePresent) var windowedSafePresent = true
#endif
#if !os(tvOS)
@AppStorage(DefaultsKey.allowVRR) var allowVRR = true
@@ -23,6 +23,34 @@ import os
private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter")
#if os(macOS)
/// HOW a windowed (composited) macOS session pushes finished frames to glass the DCP
/// "mismatched swapID's" kernel-panic saga's mechanism picker. Fullscreen always presents
/// `async` (direct-scanout promotion, lowest latency, no panic reports there); the windowed
/// mechanism is resolved per session by SessionPresenter (user setting +
/// PUNKTFUNK_WINDOWED_PRESENT env override) and routed here via `setWindowedPresent`.
///
/// - `async`: the CAMetalLayer image queue (`commandBuffer.present`) the fastest composited
/// path and the PANIC TRIGGER on high-refresh displays (the out-of-band swaps race
/// WindowServer's compositor; it survived glass pacing and every codec).
/// - `transaction`: `CAMetalLayer.presentsWithTransaction` the swap commits WITH the layer
/// tree, in lockstep with the compositor (Apple's documented remedy; validated no-panic on
/// the 240 Hz repro machine). The present is committed from the RENDER thread inside an
/// explicit CATransaction + flush see `encodePresent` for why that beats the original
/// main-thread hop.
/// - `surface`: no image queue at all render into a pooled IOSurface and swap it into a plain
/// CALayer's `contents` (the f407f418 PyroWave mitigation, resurrected format-aware:
/// rgba16Float + PQ tagging keeps HDR). WindowServer treats it as ordinary layer damage on
/// its own composite cadence. PROTOTYPE: whether the compositor honors PQ/EDR for plain-layer
/// IOSurface contents still needs an on-glass eyeball the metal layer stays underneath with
/// `wantsExtendedDynamicRangeContent` as the EDR anchor.
enum WindowedPresentMode: String, Sendable {
case async
case transaction
case surface
}
#endif
/// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the
/// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors
/// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the
@@ -198,8 +226,8 @@ fragment float4 pf_frag_hdr(VOut in [[stage_in]],
// in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.)
// The shared PQdisplay-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084
// EOTF 203-nit-anchored scene light BT.2020709 primaries extended-Reinhard rolloff
// BT.709 OETF. Used by the tvOS biplanar tone-map and the planar (PyroWave) tone-map the
// latter also on macOS windowed sessions, whose IOSurface present path is BGRA8-only.
// BT.709 OETF. Used by the tvOS biplanar tone-map and the tvOS planar (PyroWave) tone-map (the
// no-HDR-headroom fallback). macOS keeps real HDR windowed now see `WindowedPresentMode`.
static inline float3 pqToSdr(float3 pq) {
const float m1 = 2610.0/16384.0;
const float m2 = 78.84375;
@@ -230,8 +258,7 @@ fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
// PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows
// fold in depth-10 MSB packing) PQ RGB the shared SDR tail. Used when a PQ pyrowave
// stream must land on an 8-bit surface: tvOS without HDR headroom, and macOS WINDOWED sessions
// (the IOSurface present path the DCP-panic mitigation is BGRA8). The passthrough planar
// stream must land on an 8-bit surface: tvOS without HDR headroom. The passthrough planar
// HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math the
// layer's itur_2100_PQ colour space + EDR metadata do the interpretation).
fragment float4 pf_frag_planar_tm(VOut in [[stage_in]],
@@ -259,21 +286,51 @@ public final class MetalVideoPresenter {
public let layer: CAMetalLayer
#if os(macOS)
/// The WINDOWED-mode PyroWave present target: a plain CALayer sized like `layer` (installed
/// as a sibling ABOVE it), fed IOSurfaces via `contents` inside ordinary CATransactions.
/// WINDOWED-mode present coordination the macOS DCP KERNEL PANIC mitigation.
///
/// Why this exists the macOS DCP KERNEL PANIC ("mismatched swapID's" @UnifiedPipeline.cpp,
/// WindowServer dies, machine reboots): out-of-band CAMetalLayer image-queue swaps into a
/// COMPOSITED (windowed) session race WindowServer's own swap submissions on high-refresh
/// displays, and the race survives glass pacing a fully serialized one-in-flight present
/// stream still panicked a 240 Hz Mac Studio (2026-07-18, twice). So in windowed mode we stop
/// using the image queue entirely and present the way video players do: render the planar CSC
/// into an IOSurface pool and swap `contents` on main WindowServer treats it as ordinary
/// damage on its own composite cadence, coalescing faster-than-refresh updates instead of
/// latching queue swaps mid-cycle. Fullscreen keeps the CAMetalLayer path (direct-scanout
/// promotion, no compositing, no panic reports). Contents updates are transparent to the
/// layer below when nil, so flipping modes just covers/uncovers the metal layer.
public let surfaceLayer: CALayer = {
/// The panic ("mismatched swapID's" @UnifiedPipeline.cpp, WindowServer dies, machine reboots):
/// the CAMetalLayer's ASYNCHRONOUS image queue (`commandBuffer.present(drawable)` an
/// out-of-band flip, mandatory with `displaySyncEnabled=false`) diverges from WindowServer's
/// compositor on a high-refresh COMPOSITED (windowed) session the compositor's notion of the
/// current swap and the layer's queued swap disagree, and the DCP asserts. It survived glass
/// pacing: a fully serialized one-in-flight present stream still panicked a 240 Hz Mac Studio
/// (2026-07-18, PyroWave), and a windowed HEVC session panicked the same machine 2026-07-21
/// so it is the async image queue itself, at any pacing or codec, not a present rate.
///
/// The fix keeps the full render path (rgba16Float / PQ / EDR real HDR is preserved) and
/// only changes HOW the drawable is presented: `CAMetalLayer.presentsWithTransaction`. With it
/// set, we don't hand the drawable to the command buffer; we commit, wait until scheduled, then
/// call `drawable.present()` INSIDE a CATransaction the present is enrolled in Core
/// Animation's transaction and committed together with the layer tree, so the swap stays in
/// lockstep with the compositor instead of racing it (Apple's documented remedy for Metal
/// presentation drifting out of sync with CA). Fullscreen keeps the async path (direct-scanout
/// promotion, lowest latency, no compositor and no panic reports there).
///
/// 2026-07-21 latency rework: the mitigation MECHANISM is now a three-way pick
/// (`WindowedPresentMode`) and the transactional present commits from the RENDER thread
/// see `encodePresent`. Staged under `stagingLock` (main pushes it via
/// `setComposited``setWindowedPresent`); the render thread drains it and toggles the layer
/// property + present style. `Active` is the render-thread copy so the layer property flips
/// exactly once per mode change.
private var windowedPresentStaged: WindowedPresentMode = .async
private var windowedPresentActive: WindowedPresentMode = .async
/// PUNKTFUNK_TXN_PRESENT=main the ORIGINAL transactional present (commit
/// waitUntilScheduled hop to the MAIN thread and present inside its CATransaction), kept
/// as a field A/B lever. The default is the render-thread commit: the present harness
/// (2026-07-21, this saga) measured the main hop landing a runloop turn late on a busy main
/// thread, and an ACTIVE implicit transaction there NESTS the explicit one presents batch
/// at runloop-iteration rate (the field's presents=55 @ fps=240, display_p50 18.6 ms).
/// Off-main commits measured immune to main-thread churn (~10 ms glass p50 at 240 Hz
/// full-size vs 14+ ms under a churned main hop).
private let txnPresentOnMain =
ProcessInfo.processInfo.environment["PUNKTFUNK_TXN_PRESENT"] == "main"
/// The WINDOWED-mode `surface` present target: a plain CALayer sized like `layer` (installed
/// as a sibling ABOVE it by SessionPresenter), fed IOSurfaces via `contents` inside explicit
/// CATransactions. Transparent (nil contents) whenever surface mode is off, so the metal
/// layer below shows through. See `WindowedPresentMode.surface`.
let surfaceLayer: CALayer = {
let l = CALayer()
l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout
l.isOpaque = true
@@ -281,8 +338,8 @@ public final class MetalVideoPresenter {
return l
}()
/// One IOSurface-backed render target of the windowed present pool. All pool state is
/// RENDER-THREAD confined; only the immutable surface refs cross to main (contents swap).
/// One IOSurface-backed render target of the windowed surface-present pool. All pool state
/// is RENDER-THREAD confined; only the immutable surface refs cross threads (contents swap).
private struct SurfaceSlot {
let surface: IOSurfaceRef
let texture: MTLTexture
@@ -292,15 +349,52 @@ public final class MetalVideoPresenter {
private var surfacePool: [SurfaceSlot] = []
private var surfacePoolSize: CGSize = .zero
private var surfacePoolHDR = false
private var surfaceSeq: UInt64 = 0
/// Index of the slot most recently handed to the layer never rewritten next, even if its
/// use count already dropped (the compositor may still be scanning out the previous frame).
private var lastHandedOff: Int?
/// Staged (under `stagingLock`, like every cross-thread input): the hosting view's windowed
/// vs fullscreen state, pushed from main via `setSurfacePresents`. Drained in `renderPlanar`.
private var surfacePresentsStaged = false
/// Render-thread copy, so pool teardown happens exactly once on a mode flip.
private var surfacePresentsActive = false
/// Once-per-second decomposition of the ACTIVE windowed present path (the field-diagnosis
/// half of the DCP-latency work): scheduled/completed wait + commit/flush cost per present,
/// and how many presents/swaps were issued. The pf-present line shows the GLASS side
/// (latchMs / dropped); this shows the ISSUE side. Logged via `presenterLog` only while a
/// windowed mechanism is active (zero cost fullscreen). Lock-guarded: transaction mode
/// records from the render thread, surface mode from Metal completion threads.
private final class WindowedPresentDiag: @unchecked Sendable {
private let lock = NSLock()
private var presents = 0
private var schedMs: [Double] = []
private var commitMs: [Double] = []
private var last = CACurrentMediaTime()
func record(schedMs sched: Double, commitMs commit: Double, mode: WindowedPresentMode) {
lock.lock()
presents += 1
schedMs.append(sched)
commitMs.append(commit)
let now = CACurrentMediaTime()
guard now - last >= 1 else {
lock.unlock()
return
}
last = now
let sSched = schedMs.sorted()
let sCommit = commitMs.sorted()
let line = String(
format: "pf-windowed mode=%@ presents=%d schedMs p50=%.2f max=%.2f "
+ "commitMs p50=%.2f max=%.2f",
mode.rawValue, presents, sSched[sSched.count / 2], sSched.last ?? 0,
sCommit[sCommit.count / 2], sCommit.last ?? 0)
presents = 0
schedMs.removeAll(keepingCapacity: true)
commitMs.removeAll(keepingCapacity: true)
lock.unlock()
presenterLog.info("\(line, privacy: .public)")
}
}
private let windowedDiag = WindowedPresentDiag()
#endif
private let device: MTLDevice
@@ -316,7 +410,7 @@ public final class MetalVideoPresenter {
private let pipelinePlanar: MTLRenderPipelineState
/// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour
/// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm
/// bgra8; tvOS without headroom + macOS windowed IOSurface presents).
/// bgra8; tvOS without HDR headroom).
private let pipelinePlanarHDR: MTLRenderPipelineState
private let pipelinePlanarToneMap: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache?
@@ -591,13 +685,14 @@ public final class MetalVideoPresenter {
}
#if os(macOS)
/// Park the windowed-vs-fullscreen present routing (MAIN thread the hosting view pushes its
/// window state on every layout). true = PyroWave frames present via `surfaceLayer` contents
/// (the DCP swapID-panic mitigation see `surfaceLayer`); false = the CAMetalLayer path.
/// Park the windowed present mechanism (MAIN thread the hosting view pushes its window
/// state on every layout; SessionPresenter resolves the mechanism per session). `.async` =
/// FULLSCREEN (or the user opted out of the mitigation): the image queue. `.transaction` /
/// `.surface` = COMPOSITED (windowed) mitigation mechanisms see `WindowedPresentMode`.
/// Applied by the render thread on the next frame, like every other staged value here.
public func setSurfacePresents(_ on: Bool) {
func setWindowedPresent(_ mode: WindowedPresentMode) {
stagingLock.lock()
surfacePresentsStaged = on
windowedPresentStaged = mode
stagingLock.unlock()
}
#endif
@@ -734,36 +829,12 @@ public final class MetalVideoPresenter {
) -> Bool {
stagingLock.lock()
let targetFromLayout = drawableTarget
#if os(macOS)
let surfaceMode = surfacePresentsStaged
#endif
stagingLock.unlock()
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path;
// macOS WINDOWED sessions stay on the SDR layer (the IOSurface path tone-maps in-shader).
#if os(macOS)
configure(hdr: planes.pq && !surfaceMode)
#else
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path
// including macOS windowed sessions, which keep real HDR (the DCP mitigation is the
// transactional present in `encodePresent`, not a colour downgrade).
configure(hdr: planes.pq)
#endif
var csc = planes.csc
#if os(macOS)
if surfaceMode != surfacePresentsActive {
surfacePresentsActive = surfaceMode
presenterLog.info(
"stage2: windowed surface presents \(surfaceMode ? "ON" : "OFF", privacy: .public) (PyroWave DCP-panic mitigation)")
if !surfaceMode {
// Back to the metal path (fullscreen): drop the pool at 5K it holds >100 MB,
// and re-entering windowed mode rebuilds it in one frame.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
}
if surfaceMode {
return renderPlanarToSurface(
planes, targetFromLayout: targetFromLayout, csc: &csc, onPresented: onPresented)
}
#endif
// PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still)
// 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps
// in-shader instead (the pipeline must match the drawable's pixel format).
@@ -792,118 +863,6 @@ public final class MetalVideoPresenter {
}
}
#if os(macOS)
/// The windowed-mode present tail (see `surfaceLayer` for why this path exists): render the
/// planar CSC into a pooled IOSurface and hand it to `surfaceLayer.contents` on MAIN inside a
/// plain CATransaction an ordinary damaged-layer update on WindowServer's own composite
/// cadence, no CAMetalLayer image-queue swap anywhere. `presentAtMediaTime` doesn't apply
/// (the compositor paces); `onPresented` fires after the contents swap is committed, stamped
/// with CLOCK_REALTIME then the closest observable analogue of "reached glass" here (the
/// composite follows within a refresh, so the meters' display stage reads slightly optimistic).
private func renderPlanarToSurface(
_ planes: WaveletPlanes, targetFromLayout: CGSize, csc: inout CscUniform,
onPresented: ((Int64?) -> Void)?
) -> Bool {
let decodedSize = CGSize(width: planes.width, height: planes.height)
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
ensureSurfacePool(size: targetSize)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(planes.pq ? pipelinePlanarToneMap : pipelinePlanar)
encoder.setFragmentTexture(planes.y, index: 0)
encoder.setFragmentTexture(planes.cb, index: 1)
encoder.setFragmentTexture(planes.cr, index: 2)
encoder.setFragmentBytes(&csc, length: MemoryLayout<CscUniform>.stride, index: 0)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let keepAlive: [Any] = [planes.y, planes.cb, planes.cr]
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // ring textures pinned until the GPU finished sampling
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
onPresented?(
Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size` 4 BGRA8 IOSurface render targets (one on glass, one queued
/// in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the pool empty;
/// the caller returns false and the ring's putBack + display-link retry take over.
private func ensureSurfacePool(size: CGSize) {
guard size != surfacePoolSize else { return }
surfacePool.removeAll()
surfacePoolSize = size
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// 256-byte row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerRow = ((w * 4) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: 4,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one
/// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule
/// the present and the on-glass callback.
@@ -936,6 +895,36 @@ public final class MetalVideoPresenter {
#if DEBUG
logSizeIfChanged(decoded: decodedSize, drawable: targetSize)
#endif
#if os(macOS)
// Windowed (composited) the DCP swapID-panic mitigation mechanism (see
// `WindowedPresentMode`). Toggle the layer property BEFORE vending a drawable so the
// vend matches how it will be presented; drained here on the render thread, flipped
// exactly once per mode change.
stagingLock.lock()
let windowedMode = windowedPresentStaged
stagingLock.unlock()
if windowedMode != windowedPresentActive {
windowedPresentActive = windowedMode
layer.presentsWithTransaction = windowedMode == .transaction
if windowedMode != .surface, !surfacePool.isEmpty {
// Leaving surface mode (fullscreen entry / mechanism A/B): drop the pool at 5K
// it holds >100 MB, and re-entering rebuilds it in one frame. SessionPresenter
// clears the surface layer's contents on main.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
presenterLog.info(
"stage2: windowed present mode \(windowedMode.rawValue, privacy: .public) (DCP swapID-panic mitigation)")
}
if windowedMode == .surface {
// No image queue at all: render into a pooled IOSurface and swap it into the
// sibling layer's contents. The drawable/queue tail below never runs.
return encodeToSurface(
targetSize: targetSize, pipeline: pipeline, onPresented: onPresented,
keepAlive: keepAlive, bind: bind)
}
#endif
if let providedDrawable,
providedDrawable.texture.pixelFormat != layer.pixelFormat {
return false // config outran the vend (HDR flip) next vend has the new format
@@ -974,6 +963,61 @@ public final class MetalVideoPresenter {
}
#endif
}
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
#if os(macOS)
if windowedPresentActive == .transaction {
// Windowed DCP mitigation: present the drawable THROUGH a Core Animation transaction
// (`presentsWithTransaction`, set above) instead of the async image queue, so the swap
// commits with the layer tree and stays in lockstep with the compositor (no out-of-band
// flip to race WindowServer's swaps). Wait until the GPU work is scheduled (contents
// will be ready p50 ~0.1 ms), then present inside an EXPLICIT CATransaction ON THIS
// RENDER THREAD and `flush()`. `presentAtMediaTime` does not apply the transaction
// paces.
//
// Threading history, because BOTH failure modes shipped or nearly shipped:
// A bare `present()` from this thread (no transaction) never flushes nothing
// commits a runloop-less thread's implicit transaction, so drawables are never
// released; after maximumDrawableCount vends `nextDrawable()` blocks forever and
// the stream FREEZES (the fullscreenwindowed switch did exactly this).
// The explicit begin/commit alone is NOT enough either: this thread has an ACTIVE
// implicit transaction (the layer mutations above drawableSize/colour created
// it), so the explicit transaction NESTS inside it and its commit defers to the
// implicit one that never comes. The harness reproduced the exact freeze: every
// present reported presentedTime=0, nothing reached glass. `CATransaction.flush()`
// pushes the implicit transaction (present included) to the render server NOW.
// The original fix hopped to MAIN and presented there correct, but slow in the
// field (presents=55 @ fps=240, display_p50 18.6 ms on the 240 Hz Studio): each
// present lands a runloop turn late, and main's own implicit transaction batches
// enrolled presents at runloop-iteration rate. Kept as PUNKTFUNK_TXN_PRESENT=main.
// The off-main commit measured immune to main-thread churn in the harness
// (2026-07-21: glass p50 ~10 ms at 240 Hz full-size, cadence a clean 4.17 ms).
commandBuffer.commit()
let schedStart = CACurrentMediaTime()
commandBuffer.waitUntilScheduled()
let schedMs = (CACurrentMediaTime() - schedStart) * 1000
let commitStart = CACurrentMediaTime()
if txnPresentOnMain {
let presentedDrawable = drawable
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
presentedDrawable.present()
CATransaction.commit()
}
} else {
CATransaction.begin()
CATransaction.setDisableActions(true)
drawable.present()
CATransaction.commit()
CATransaction.flush()
}
windowedDiag.record(
schedMs: schedMs, commitMs: (CACurrentMediaTime() - commitStart) * 1000,
mode: .transaction)
return true
}
#endif
// Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment);
// immediate otherwise. A target already in the past presents immediately same thing.
if let presentAtMediaTime {
@@ -981,12 +1025,146 @@ public final class MetalVideoPresenter {
} else {
commandBuffer.present(drawable)
}
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
commandBuffer.commit()
return true
}
#if os(macOS)
/// The WINDOWED `surface` present tail (see `WindowedPresentMode.surface`): render with the
/// same per-frame pipeline into a pooled IOSurface and hand it to `surfaceLayer.contents`
/// from the command buffer's COMPLETION handler, inside an explicit CATransaction + flush
/// (the same off-main commit discipline as the transactional present an ordinary
/// damaged-layer update on WindowServer's own composite cadence, no image queue anywhere).
/// RENDER THREAD. `onPresented` is stamped right after the contents swap commits the
/// closest observable analogue of "reached glass" here (the composite follows within a
/// refresh, so the display-stage meters read slightly OPTIMISTIC in this mode).
///
/// The pool tracks `hdrActive`: bgra8 for SDR, rgba16Float tagged BT.2100 PQ for HDR
/// `configure` already ran, so the caller's `pipeline` attachment format always matches.
/// HDR OPEN RISK (why this whole mode is a prototype): whether the compositor honors the
/// PQ tag + EDR for plain-CALayer IOSurface contents needs an on-glass eyeball; the metal
/// layer underneath keeps `wantsExtendedDynamicRangeContent` as the EDR anchor (the harness
/// measured the display's EDR headroom engaging with this arrangement).
private func encodeToSurface(
targetSize: CGSize, pipeline: MTLRenderPipelineState,
onPresented: ((Int64?) -> Void)?,
keepAlive: [Any], bind: (MTLRenderCommandEncoder) -> Void
) -> Bool {
ensureSurfacePool(size: targetSize, hdr: hdrActive)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(pipeline)
bind(encoder)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let diag = windowedDiag
let commitStamp = CACurrentMediaTime()
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // sources pinned until the GPU finished sampling
let completedAt = CACurrentMediaTime()
// Swap on THIS Metal completion thread: explicit transaction + flush, so the commit
// reaches the render server now, independent of main (completion handlers for one
// queue fire in execution order, so swaps can't reorder).
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
CATransaction.flush()
diag.record(
schedMs: (completedAt - commitStamp) * 1000,
commitMs: (CACurrentMediaTime() - completedAt) * 1000, mode: .surface)
onPresented?(Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size`/`hdr` 4 IOSurface render targets (one on glass, one
/// committed in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the
/// pool empty; the caller returns false and the ring's putBack + display-link retry take
/// over.
private func ensureSurfacePool(size: CGSize, hdr: Bool) {
guard size != surfacePoolSize || hdr != surfacePoolHDR else { return }
surfacePool.removeAll()
surfacePoolSize = size
surfacePoolHDR = hdr
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// rgba16Float (8 B/px) carries the PQ-encoded HDR samples; bgra8 the SDR ones. 256-byte
// row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerElement = hdr ? 8 : 4
let bytesPerRow = ((w * bytesPerElement) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: bytesPerElement,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: hdr
? kCVPixelFormatType_64RGBAHalf : kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: hdr ? .rgba16Float : .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
if hdr, let name = CGColorSpace(name: CGColorSpace.itur_2100_PQ)?.name {
// Tag the surface BT.2100 PQ so the compositor interprets the half-float
// samples as PQ-encoded HDR (the CALayer-contents analogue of the metal
// layer's colorspace).
IOSurfaceSetValue(surface, "IOSurfaceColorSpace" as CFString, name)
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
// The EDR request rides the SURFACE layer too (its contents are what composite); the
// metal layer underneath keeps its own from configureColor as the anchor. Layer flags
// are committed by the next swap's transaction flush.
surfaceLayer.wantsExtendedDynamicRangeContent = hdr
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the
/// draw the MTLTexture is only valid while its CVMetalTexture is retained.
private func makeTexture(
@@ -142,20 +142,17 @@ enum PresentPriority: Equatable {
final class SessionPresenter {
/// Present pacing for this session. Stage-3 always means glass gating; under the stage-2
/// default, macOS PyroWave sessions ALSO get glass gating a kernel-panic mitigation, not a
/// latency tweak. macOS's DCP panics ("mismatched swapID's" @UnifiedPipeline.cpp, the whole
/// machine dies) when WindowServer's swap submissions race, and the reliable trigger is
/// out-of-band CAMetalLayer presents (displaySyncEnabled=false mandatory for us, see
/// MetalVideoPresenter's init) arriving faster than the compositor latches them in a
/// COMPOSITED (windowed) session. Arrival pacing does exactly that with PyroWave: the wavelet
/// decode is near-instant Metal compute, so a network clump of frames presents within the
/// same millisecond, and PyroWave is the codec that sustains stream rates above the panel's
/// refresh. The glass gate admits one presented-but-undisplayed swap at a time (serialized on
/// the on-glass callback, 100 ms stale backstop), which removes the racing pattern outright;
/// frames the panel couldn't have shown anyway coalesce in the newest-wins ring. An explicit
/// stage-2 pick (setting/env) still forces arrival pacing that A/B lever must stay honest.
/// VideoToolbox codecs keep arrival pacing: decode latency spaces their presents, and years
/// of stage-2 defaults there predate any panic report.
/// default, macOS PyroWave sessions ALSO get glass gating for SMOOTHNESS, not as the panic
/// fix (that is the windowed transactional present see `setComposited`). PyroWave's wavelet
/// decode is near-instant Metal compute, so a network clump presents within the same
/// millisecond, and it is the codec that sustains stream rates above the panel's refresh; the
/// glass gate admits one presented-but-undisplayed swap at a time (serialized on the on-glass
/// callback, 100 ms stale backstop) so those bursts coalesce in the newest-wins ring instead
/// of flooding the queue. (Glass pacing was ALSO the original DCP-panic mitigation attempt
/// disproven: a fully serialized stream still panicked, which is why the real fix moved to the
/// present mechanism.) An explicit stage-2 pick (setting/env) still forces arrival pacing
/// that A/B lever must stay honest. VideoToolbox codecs keep arrival pacing: decode latency
/// spaces their presents.
static func pacing(
for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec
) -> PresentPacing {
@@ -178,10 +175,25 @@ final class SessionPresenter {
/// that doesn't exist after the first Wi-Fi clump. Sub-refresh display latency needs pacing
/// that can't queue at all that's stage-4 (`PresentPacing.deadline`), not a deeper gate.
///
#if os(macOS)
/// Resolve the windowed (composited) present MECHANISM for this session the DCP
/// swapID-panic mitigation picker (see `WindowedPresentMode`). The
/// `PUNKTFUNK_WINDOWED_PRESENT=async|transaction|surface` env lever wins (dev A/B);
/// otherwise the user's safe-present setting: ON/unset `.transaction` (the validated
/// mitigation), OFF `.async` (the fast pre-mitigation path the panic returns on
/// affected high-refresh setups; the Settings caption says so). `.surface` is currently
/// env-only (prototype HDR-composite verification owed). Fullscreen always presents
/// async regardless (`setComposited`). Internal (not private) for unit tests.
static func windowedPresentMode(setting: Bool?, env: String?) -> WindowedPresentMode {
if let env, let mode = WindowedPresentMode(rawValue: env) { return mode }
return (setting ?? true) ? .transaction : .async
}
#endif
/// `PUNKTFUNK_GATE_DEPTH` (13) still overrides on iOS/tvOS so the standing-queue ladder
/// stays reproducible on-device; macOS is pinned to 1, env ignored glass pacing exists
/// there as the DCP swapID kernel-panic mitigation (see `pacing`), and STRICT present
/// serialization is its point. Internal (not private) for unit tests.
/// stays reproducible on-device; macOS is pinned to 1, env ignored a deeper gate only builds
/// a standing queue (see above), and macOS glass pacing exists for PyroWave smoothness
/// (see `pacing`), where depth 1 is the point. Internal (not private) for unit tests.
static func gateDepth(env: String?) -> Int {
#if os(macOS)
return 1
@@ -196,10 +208,16 @@ final class SessionPresenter {
private var stage2Link: CADisplayLink?
private var metalLayer: CAMetalLayer?
#if os(macOS)
/// The windowed-mode PyroWave present target (sibling above `metalLayer`) and the last
/// routing pushed to the pipeline see `setComposited`. Main-thread only, like all of this.
/// The windowed present MECHANISM this session runs while composited (resolved once per
/// session in `start` the user's safe-present setting + the PUNKTFUNK_WINDOWED_PRESENT
/// dev override) and the routing last pushed to the pipeline see `setComposited` (the DCP
/// swapID-panic mitigation). Main-thread only, like all of this.
private var windowedMode: WindowedPresentMode = .transaction
private var windowedPresentApplied: WindowedPresentMode = .async
/// The windowed `surface` present target (sibling above `metalLayer`, transparent while
/// unused) installed whenever stage-2 runs so a mechanism flip never has to mutate the
/// layer tree mid-session.
private var surfaceLayer: CALayer?
private var surfacePresentsActive = false
#endif
private var connection: PunktfunkConnection?
/// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's
@@ -283,11 +301,17 @@ final class SessionPresenter {
baseLayer.addSublayer(metal)
metalLayer = metal
#if os(macOS)
// The windowed-PyroWave present target sits ABOVE the metal layer: transparent (nil
// contents) while the metal path presents, covering it while surface presents run.
windowedPresentApplied = .async
// Resolve THIS session's windowed mechanism once (setting + dev env lever)
// `setComposited` routes between it and fullscreen-async from every layout.
windowedMode = Self.windowedPresentMode(
setting: UserDefaults.standard.object(
forKey: DefaultsKey.windowedSafePresent) as? Bool,
env: ProcessInfo.processInfo.environment["PUNKTFUNK_WINDOWED_PRESENT"])
// The surface present target sits ABOVE the metal layer: transparent (nil contents)
// unless the surface mechanism actually presents, covering it while it does.
baseLayer.addSublayer(pipeline.surfaceLayer)
surfaceLayer = pipeline.surfaceLayer
surfacePresentsActive = false
#endif
stage2 = pipeline
// The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger
@@ -432,19 +456,23 @@ final class SessionPresenter {
#if os(macOS)
/// Route presents for the window's composited state (MAIN thread the view pushes it on
/// every layout, which fullscreen transitions always trigger). PyroWave sessions in a
/// COMPOSITED (windowed) session present via `surfaceLayer` contents instead of the
/// CAMetalLayer image queue the DCP "mismatched swapID's" kernel-panic mitigation (see
/// `MetalVideoPresenter.surfaceLayer`; the metal-swap race survives glass pacing, so pacing
/// alone was not enough). VT codecs keep the metal path: no panic reports there, and their
/// HDR/EDR presentation has no surface-contents equivalent wired.
/// every layout, which fullscreen transitions always trigger). A COMPOSITED (windowed)
/// session presents through this session's resolved mitigation mechanism (`windowedMode`
/// transactional by default, see `windowedPresentMode`) instead of the async image queue
/// the DCP "mismatched swapID's" kernel-panic mitigation (see `MetalVideoPresenter`; the
/// async-swap race survives glass pacing, so pacing alone was not enough). ALL codecs:
/// PyroWave hit it 2026-07-18 and windowed HEVC hit the same 240 Hz Mac Studio 2026-07-21
/// it is the async image queue itself, not any codec or present rate. Fullscreen keeps the
/// async path (direct scanout, lowest latency, no panic there). The full HDR/EDR render
/// path is preserved in every mechanism.
func setComposited(_ composited: Bool) {
guard let stage2, let connection else { return }
let wantsSurface = composited && connection.videoCodec == .pyrowave
guard wantsSurface != surfacePresentsActive else { return }
surfacePresentsActive = wantsSurface
stage2.setSurfacePresents(wantsSurface)
if !wantsSurface {
guard let stage2 else { return }
let mode: WindowedPresentMode = composited ? windowedMode : .async
guard mode != windowedPresentApplied else { return }
let wasSurface = windowedPresentApplied == .surface
windowedPresentApplied = mode
stage2.setWindowedPresent(mode)
if wasSurface {
// Uncover the metal layer NOW (its last drawable is still attached, so fullscreen
// entry shows the previous frame until the next present no black flash).
CATransaction.begin()
@@ -471,7 +499,7 @@ final class SessionPresenter {
#if os(macOS)
surfaceLayer?.removeFromSuperlayer()
surfaceLayer = nil
surfacePresentsActive = false
windowedPresentApplied = .async
#endif
connection = nil
}
@@ -1114,15 +1114,15 @@ public final class Stage2Pipeline {
}
#if os(macOS)
/// The windowed-mode PyroWave present target (see `MetalVideoPresenter.surfaceLayer` the
/// DCP swapID-panic mitigation). The hosting view installs it as a sibling above `layer`.
public var surfaceLayer: CALayer { presenter.surfaceLayer }
/// Forward the windowed-vs-fullscreen present routing (MAIN thread see
/// `MetalVideoPresenter.setSurfacePresents`).
public func setSurfacePresents(_ on: Bool) {
presenter.setSurfacePresents(on)
/// Forward the windowed present mechanism (MAIN thread see
/// `MetalVideoPresenter.setWindowedPresent`, the DCP swapID-panic mitigation).
func setWindowedPresent(_ mode: WindowedPresentMode) {
presenter.setWindowedPresent(mode)
}
/// The windowed `surface` present target the hosting SessionPresenter installs as a sibling
/// ABOVE `layer` (transparent while unused see `MetalVideoPresenter.surfaceLayer`).
var surfaceLayer: CALayer { presenter.surfaceLayer }
#endif
/// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen`
@@ -700,9 +700,9 @@ public final class StreamLayerView: NSView {
private func layoutPresenter() {
presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1)
// Present routing tracks the window's composited state (fullscreen transitions always
// re-layout, so this stays current): windowed PyroWave presents via surface contents
// the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited). A view
// not yet in a window counts as composited (the safe default).
// re-layout, so this stays current): a windowed session presents through a Core Animation
// transaction the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited).
// A view not yet in a window counts as composited (the safe default).
presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false))
// Feed the follower only once in a window (backing scale is real then) and with real
// bounds a pre-window layout would report point-sized dimensions.
@@ -70,6 +70,16 @@ public enum DefaultsKey {
/// (lowest latency the default, OFF). Resolved once per session;
/// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header.
public static let vsync = "punktfunk.vsync"
/// macOS: present WINDOWED sessions in lockstep with the system compositor (the DCP
/// "mismatched swapID's" kernel-panic mitigation see SessionPresenter.windowedPresentMode
/// and the MetalVideoPresenter saga notes). ON/unset (the default): windowed presents ride
/// a Core Animation transaction validated panic-free on the 240 Hz repro machine, at a
/// small display-latency cost vs the raw path. OFF: windowed sessions keep the fast async
/// image queue ON AFFECTED SETUPS (high-refresh displays) THAT PATH KERNEL-PANICS THE
/// WHOLE MAC, which is why the default is ON. Fullscreen always presents async (fast path)
/// regardless. Resolved once per session; PUNKTFUNK_WINDOWED_PRESENT=async|transaction|
/// surface overrides it for dev A/B.
public static let windowedSafePresent = "punktfunk.windowedSafePresent"
/// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor,
/// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical
/// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off,
@@ -316,6 +316,41 @@ final class PresentPacingTests: XCTestCase {
SessionPresenter.pacing(for: .stage4, explicit: .stage4, codec: .pyrowave), .deadline)
}
// MARK: - Windowed present mechanism (the macOS DCP swapID-panic mitigation picker)
#if os(macOS)
/// The safe-present setting: ON/unset the validated transactional mitigation; an explicit
/// OFF the fast async path (the user accepted the affected-setup panic risk). The
/// PUNKTFUNK_WINDOWED_PRESENT env lever overrides both ways, `surface` is env-only (the
/// prototype mechanism), and garbage/empty env values are "unset", not an override.
func testWindowedPresentModeResolution() {
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: nil), .transaction,
"unset defaults to the panic mitigation")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: nil), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: nil), .async,
"an explicit opt-out gets the fast async path")
// The dev env lever wins over the setting, both directions.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "async"), .async)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "transaction"),
.transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "surface"), .surface,
"the surface prototype is reachable via env only")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "surface"), .surface)
// Garbage/empty env = unset.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: "garbage"), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: ""), .async)
}
#endif
// MARK: - Glass-gate depth
/// The in-flight present budget is 1 EVERYWHERE: any deeper gate keeps a standing queue
+11 -2
View File
@@ -16,6 +16,8 @@
# PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games)
# PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect)
# PF_MGMT management-API port for --browse (optional; client defaults to 47990)
# PF_CONNECT_TIMEOUT connect budget in seconds (optional; the plugin stretches it after
# firing Wake-on-LAN so the connect survives the host's resume)
# PF_APPID flatpak app id (default io.unom.Punktfunk)
# PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH)
#
@@ -61,10 +63,17 @@ if [ -z "${PF_HOST:-}" ]; then
echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2
exit 2
fi
# Trailing args shared by both streaming execs. A stretched connect budget rides along when the
# plugin set one (it just fired Wake-on-LAN, so the host may still be resuming); an older flatpak
# without --connect-timeout ignores the flag harmlessly (hand-scanned argv).
set -- --fullscreen
if [ -n "${PF_CONNECT_TIMEOUT:-}" ]; then
set -- --connect-timeout "$PF_CONNECT_TIMEOUT" "$@"
fi
if [ -n "${PF_LAUNCH:-}" ]; then
# A pinned game: the id rides the session Hello and the host launches that title.
echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" --fullscreen
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" "$@"
fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" "$@"
+32 -7
View File
@@ -70,7 +70,9 @@ function setShortcutHidden(appId: number, hidden: boolean): void {
};
// Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId).
const ART_VERSION = 2;
// v3: CI zips through 0.17.1 shipped no assets/ at all, yet v2 was still recorded as applied
// on those installs — the bump makes them re-apply once on the first build that has the files.
const ART_VERSION = 3;
function artKey(appId: number): string {
return `punktfunk:shortcutArt:${appId}`;
}
@@ -79,7 +81,7 @@ function artKey(appId: number): string {
* Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per
* appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call.
*/
async function applyArtwork(appId: number): Promise<void> {
async function applyArtwork(appId: number, isRetry = false): Promise<void> {
try {
if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) {
return;
@@ -91,16 +93,29 @@ async function applyArtwork(appId: number): Promise<void> {
[art.logo, 2],
[art.gridwide, 3],
];
let applied = false;
for (const [data, assetType] of assets) {
if (data) {
await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType);
applied = true;
}
}
if (art.icon_path) {
SteamClient.Apps.SetShortcutIcon(appId, art.icon_path);
applied = true;
}
// Only record "done" when something actually landed — a plugin build whose assets/ is
// missing/empty must keep retrying on later mounts instead of poisoning the marker.
if (applied) {
localStorage.setItem(artKey(appId), `${ART_VERSION}`);
}
} catch (e) {
// A shortcut fresh out of AddShortcut may not be registered yet (the same race
// setShortcutHidden defers around) — one deferred second attempt, then leave it to
// the next mount.
if (!isRetry) {
setTimeout(() => void applyArtwork(appId, true), 2500);
}
console.warn("punktfunk: shortcut artwork not applied", e);
}
}
@@ -157,7 +172,9 @@ async function ensureControllerConfig(): Promise<void> {
return;
}
const r = await applyControllerConfig(SHORTCUT_NAME);
if (r?.ok) {
// `ok` alone isn't done: with zero account configset dirs (fresh Steam) the backend
// succeeds without pointing any account at the template — keep retrying until one lands.
if (r?.ok && (r.applied ?? []).some((a) => a.startsWith("configset:"))) {
localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`);
} else {
console.warn("punktfunk: controller config not fully applied", r);
@@ -283,13 +300,21 @@ export async function launchStream(
opts: LaunchOpts = {},
): Promise<void> {
// Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now
// so it races with the shortcut setup (near-zero added latency), and awaited just before RunGame.
// so it races with the shortcut setup (near-zero added latency); its outcome is needed below
// (the connect budget), and RunGame follows the await either way, so nothing is slower for it.
// Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is
// known), and the connect that follows has its own retry window, so a failure never blocks launch.
const waking = wake(host, port).catch(() => ({ ok: false }));
const { appId, runner } = await ensureStreamShortcut();
const [{ appId, runner }, woke] = await Promise.all([ensureStreamShortcut(), waking]);
const target = port && port !== 9777 ? `${host}:${port}` : host;
const env = [`PF_HOST=${target}`];
// A magic packet actually went out (a MAC was known), so the host may be mid-resume from
// suspend — that takes far longer than the client's default 15 s connect budget. Stretch the
// budget so the client's wake-tolerant dial keeps retrying across the resume; against an
// already-awake host the connect still lands in under a second, so this costs nothing.
if (woke.ok) {
env.push("PF_CONNECT_TIMEOUT=75");
}
if (opts.browse) {
env.push("PF_BROWSE=1");
if (opts.mgmt) {
@@ -303,9 +328,9 @@ export async function launchStream(
env.push(`PF_LAUNCH=${opts.launchId}`);
}
// KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper
// script rides behind it as an argument and reads PF_* from the environment.
// script rides behind it as an argument and reads PF_* from the environment. The wake was
// awaited above, so the magic packet is out before the connect attempt.
SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`);
await waking; // ensure the magic packet is out before the connect attempt
SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100);
}
+12
View File
@@ -498,6 +498,13 @@ async fn session(args: Args) -> Result<()> {
if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_444;
}
// PUNKTFUNK_CLIENT_CHACHA20=1 advertises VIDEO_CAP_CHACHA20 — drives the
// host's ChaCha20-Poly1305 session-cipher resolution (the soft-AES armv7
// negotiation, design/chacha20-session-cipher.md §7) without a webOS build;
// the negotiated cipher is reported in the welcome log line below.
if std::env::var_os("PUNKTFUNK_CLIENT_CHACHA20").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_CHACHA20;
}
caps
},
// `--audio-channels` (default stereo); the probe multistream-decodes + validates the
@@ -535,6 +542,11 @@ async fn session(args: Args) -> Result<()> {
chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444,
chroma_format_idc = welcome.chroma_format,
codec = codec_ext(welcome.codec),
cipher = if welcome.cipher == punktfunk_core::quic::CIPHER_CHACHA20_POLY1305 {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
"session offer"
);
+16
View File
@@ -24,6 +24,16 @@ use pf_frame::DmabufFrame;
pub trait Capturer: Send {
fn next_frame(&mut self) -> Result<CapturedFrame>;
/// [`next_frame`](Self::next_frame) with a caller-chosen first-frame budget instead of the
/// backend's default. The pipeline retry loop shortens its FIRST attempt's wait: a PipeWire
/// stream connected while gamescope re-inits its headless takeover can negotiate a format,
/// reach `Streaming`, and still never receive a buffer — a fresh connect then delivers within
/// ~0.5 s, so waiting out the full default budget on a doomed stream just delays the retry
/// that fixes it. Backends without an internal wait budget ignore it (the default delegates).
fn next_frame_within(&mut self, _budget: std::time::Duration) -> Result<CapturedFrame> {
self.next_frame()
}
/// Non-blocking: the freshest frame available since the last call, or `None` if none has
/// arrived (the caller reuses its last frame to hold a steady output rate). The default
/// just produces a frame each call — fine for instant synthetic sources; the portal
@@ -249,6 +259,12 @@ pub struct ZeroCopyPolicy {
/// passthrough (like the VAAPI backend) instead of the EGL→CUDA import whose payloads only
/// NVENC can consume. Per-session (the codec is negotiated), unlike `backend_is_vaapi`.
pub pyrowave_session: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (the Linux raw Vulkan
/// Video backend on an H265/AV1 session — resolved by the host facade via
/// `pf_encode::linux_native_nv12_ok`). Gates whether the negotiation PREFERS gamescope's
/// producer-side NV12 pod: libav VAAPI (H264's backend) would misread the two-plane buffer,
/// so H264/GameStream/PyroWave sessions must never see NV12 frames.
pub native_nv12_session: bool,
/// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc,
/// resolved when the session encodes PyroWave (the passthrough advertises them so Mutter+NVIDIA,
/// which allocates tiled-only, still negotiates zero-copy). Empty otherwise.
+155 -49
View File
@@ -299,29 +299,11 @@ fn spawn_pipewire(
impl Capturer for PortalCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
// First frame can lag behind format negotiation; later frames arrive at ~fps. Wait in
// short slices so a GPU-import poison (worker death) fails the capture within ~0.5 s
// instead of sitting out the full first-frame budget.
let deadline = std::time::Instant::now() + Duration::from_secs(10);
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
}
if let Some(f) = self.pending.take() {
return Ok(f); // a wait_arrival stash outranks the channel (it's older)
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e),
}
self.frame_within(Duration::from_secs(10))
}
fn next_frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
self.frame_within(budget)
}
fn supports_arrival_wait(&self) -> bool {
@@ -417,9 +399,41 @@ impl Capturer for PortalCapturer {
}
impl PortalCapturer {
/// The [`Capturer::next_frame`] budget expired (or the thread ended) — turn it into the
/// diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged.
fn next_frame_timed_out(&self, err: RecvTimeoutError) -> Result<CapturedFrame> {
/// The blocking first-frame wait behind [`Capturer::next_frame`] /
/// [`Capturer::next_frame_within`]. First frame can lag behind format negotiation; later
/// frames arrive at ~fps. Wait in short slices so a GPU-import poison (worker death) fails
/// the capture within ~0.5 s instead of sitting out the full first-frame budget.
fn frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
let deadline = std::time::Instant::now() + budget;
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
}
if let Some(f) = self.pending.take() {
return Ok(f); // a wait_arrival stash outranks the channel (it's older)
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e, budget),
}
}
}
/// The [`frame_within`](Self::frame_within) budget expired (or the thread ended) — turn it
/// into the diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged.
fn next_frame_timed_out(
&self,
err: RecvTimeoutError,
budget: Duration,
) -> Result<CapturedFrame> {
let within = budget.as_secs_f32();
match err {
RecvTimeoutError::Timeout => {
// Split the two black-screen root causes apart so the operator gets a cause, not
@@ -427,9 +441,10 @@ impl PortalCapturer {
// not (no acceptable format / node never emitted a param)?
if self.negotiated.load(Ordering::Relaxed) {
Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiated but no buffers \
arrived the compositor produced no frames (virtual output idle/unmapped, \
or capture never started)",
"no PipeWire frame within {within}s (node {}): format negotiated but no \
buffers arrived the compositor produced no frames (virtual output \
idle/unmapped, capture never started, or a stream bound during a \
compositor (re)start that will never deliver a reconnect fixes that)",
self.node_id
))
} else if self.hdr_offer {
@@ -440,10 +455,10 @@ impl PortalCapturer {
// auto-reconnects) negotiates SDR instead of re-running this same timeout.
super::note_hdr_capture_failed();
Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \
the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored monitor in \
HDR mode on GNOME 50+? Downgrading this host to SDR capture; reconnect \
to stream SDR",
"no PipeWire frame within {within}s (node {}): the compositor never \
accepted the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored \
monitor in HDR mode on GNOME 50+? Downgrading this host to SDR capture; \
reconnect to stream SDR",
self.node_id
))
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
@@ -452,14 +467,15 @@ impl PortalCapturer {
// retries on the CPU offer instead of failing this same negotiation forever.
pf_zerocopy::note_vaapi_dmabuf_failed();
Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \
the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this host to the \
CPU capture path; the pipeline rebuild will renegotiate without dmabuf",
"no PipeWire frame within {within}s (node {}): the compositor never \
accepted the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this \
host to the CPU capture path; the pipeline rebuild will renegotiate \
without dmabuf",
self.node_id
))
} else {
Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiation never \
"no PipeWire frame within {within}s (node {}): format negotiation never \
completed the compositor offered no format this consumer accepts \
(pixel-format/modifier mismatch) or the node never emitted a Format param",
self.node_id
@@ -824,6 +840,7 @@ mod pipewire {
VideoFormat::RGBA => PixelFormat::Rgba,
VideoFormat::RGB => PixelFormat::Rgb,
VideoFormat::BGR => PixelFormat::Bgr,
VideoFormat::NV12 => PixelFormat::Nv12,
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by
// the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020).
VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10,
@@ -989,10 +1006,10 @@ mod pipewire {
.into_inner())
}
/// Build a BGRx dmabuf `EnumFormat` pod advertising the EGL-importable `modifiers` as a
/// mandatory enum Choice; the compositor fixates to one of them that it can allocate, which
/// we read back in `param_changed`.
/// Build a LINEAR/modifier DMA-BUF `EnumFormat` pod. Packed BGRx is the existing import path;
/// NV12 is gamescope's producer-side RGB→YUV path (opt-in during bring-up).
fn build_dmabuf_format(
format: VideoFormat,
modifiers: &[u64],
preferred: Option<(u32, u32, u32)>,
) -> Result<Vec<u8>> {
@@ -1003,7 +1020,7 @@ mod pipewire {
pw::spa::param::ParamType::EnumFormat,
pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video),
pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw),
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, VideoFormat::BGRx),
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, format),
pw::spa::pod::property!(
FormatProperties::VideoSize,
Choice,
@@ -1032,6 +1049,22 @@ mod pipewire {
pw::spa::utils::Fraction { num: 240, denom: 1 }
),
);
if format == VideoFormat::NV12 {
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorMatrix,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_MATRIX_BT709,
)),
});
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorRange,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_RANGE_16_235,
)),
});
}
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
@@ -1604,8 +1637,8 @@ mod pipewire {
}
}
// VAAPI zero-copy passthrough: hand the raw dmabuf straight to the encoder, which imports
// it into a VA surface and does RGB→NV12 on the GPU video engine. No CUDA importer here.
// Raw DMA-BUF passthrough: packed RGB is imported for GPU CSC; producer-native NV12 can
// be consumed by the Vulkan Video encoder without another color conversion.
if ud.vaapi_passthrough {
if let Some(fmt) = ud.format {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
@@ -1613,9 +1646,41 @@ mod pipewire {
let chunk = datas[0].chunk();
let offset = chunk.offset();
let stride = chunk.stride().max(0) as u32;
// Native NV12 usually arrives as a two-plane SPA buffer over ONE buffer
// object; plane 1's chunk carries the REAL UV offset/stride (compositors
// may align the Y plane before UV). Pass it through instead of assuming
// contiguity. Each spa_data holds its own (dup'd) fd, so BO identity is
// by inode, not fd number; a genuinely two-BO frame cannot travel through
// the single-fd import — drop it with a diagnosis instead of streaming
// garbage chroma.
let plane1 =
if fmt == PixelFormat::Nv12 && datas.len() >= 2 && datas[1].fd() > 0 {
// SAFETY: zeroed `libc::stat` is a valid POD initializer; both fds are
// owned by the live PipeWire buffer for this callback, and `fstat`
// only writes the out-param structs, whose fields are read only after
// the `== 0` success checks.
let same_bo = unsafe {
let mut s0: libc::stat = std::mem::zeroed();
let mut s1: libc::stat = std::mem::zeroed();
libc::fstat(datas[0].fd() as i32, &mut s0) == 0
&& libc::fstat(datas[1].fd() as i32, &mut s1) == 0
&& (s0.st_dev, s0.st_ino) == (s1.st_dev, s1.st_ino)
};
if !same_bo {
warn_once(
"NV12 planes live in different buffer objects — frames \
dropped (single-fd import only)",
);
return;
}
let c1 = datas[1].chunk();
Some((c1.offset(), c1.stride().max(0) as u32))
} else {
None
};
// dup the fd so it survives the SPA buffer recycle — the encode thread
// imports it. (Content stability across the brief map+CSC window relies on
// the compositor's buffer-pool depth, like any zero-copy capture.)
// imports it. Content stability across the brief import/encode window relies
// on the compositor's buffer-pool depth, like any zero-copy capture.
// SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid
// for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd,
// touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1).
@@ -1642,9 +1707,10 @@ mod pipewire {
modifier: ud.modifier,
offset,
stride,
plane1,
}),
// Cursor-as-metadata: the encoder blends this into its owned VA
// surface (raw dmabuf never touched).
// Cursor-as-metadata is blended only by RGB→NV12 backends. Gamescope
// embeds its pointer in the produced pixels, so native NV12 has none.
cursor: ud.cursor.overlay(),
});
static ONCE: std::sync::atomic::AtomicBool =
@@ -1655,7 +1721,12 @@ mod pipewire {
h,
modifier = ud.modifier,
fourcc = format_args!("{:#010x}", fourcc),
"zero-copy: handing the raw dmabuf to the encoder (GPU import + CSC)"
source = if fmt == PixelFormat::Nv12 {
"producer-native NV12"
} else {
"packed RGB (encoder GPU CSC)"
},
"zero-copy: handing the raw DMA-BUF to the encoder"
);
}
return;
@@ -2015,6 +2086,28 @@ mod pipewire {
// PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw
// dmabuf straight to the encoder.
let vaapi_passthrough = zerocopy && !force_shm && importer.is_none() && raw_passthrough;
// Producer-side NV12 (default-on; PUNKTFUNK_PIPEWIRE_NV12=0 escapes): gamescope offers a
// one-fd LINEAR NV12 image when the consumer asks — its compositor pass does the RGB→YUV,
// and the Vulkan Video encoder imports the buffer as its encode source directly (no host
// CSC at all). `native_nv12_session` restricts this to sessions whose encoder can ingest
// it (Linux vulkan-encode H265/AV1 — never H264/libav-VAAPI, GameStream-resolve, or
// PyroWave, whose Vulkan compute CSC ingests packed RGB only). Raw passthrough is
// required because the CUDA importer expects packed RGB, and 4:4:4/HDR must not be
// silently subsampled/downconverted. Non-NV12 compositors (KWin/GNOME) simply match the
// packed-RGB fallback pod.
let prefer_native_nv12 = std::env::var("PUNKTFUNK_PIPEWIRE_NV12").as_deref() != Ok("0")
&& policy.native_nv12_session
&& backend_is_vaapi
&& vaapi_passthrough
&& !policy.pyrowave_session
&& !want_444
&& !want_hdr;
if prefer_native_nv12 {
tracing::info!(
"zero-copy: preferring gamescope producer-side NV12 LINEAR DMA-BUF (no host \
RGB CSC; PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation)"
);
}
// Modifiers our import stack handles for BGRx: the EGL-importable (tiled) set, plus LINEAR
// (0) — NVIDIA's EGL won't list it, but LINEAR dmabufs (gamescope's only offer) import via
// CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only:
@@ -2054,7 +2147,9 @@ mod pipewire {
tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path");
} else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() {
tracing::info!(
"zero-copy: advertising LINEAR dmabuf for direct VAAPI import (GPU CSC)"
native_nv12_preferred = prefer_native_nv12,
"zero-copy: advertising LINEAR DMA-BUF for encoder import (native NV12 first \
when enabled, packed RGB fallback)"
);
} else if want_dmabuf && !vaapi_passthrough {
tracing::info!(
@@ -2394,7 +2489,18 @@ mod pipewire {
build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?,
]
} else if want_dmabuf {
vec![build_dmabuf_format(&modifiers, preferred)?]
let mut pods = Vec::with_capacity(if prefer_native_nv12 { 2 } else { 1 });
if prefer_native_nv12 {
// First compatible consumer pod wins. Gamescope advertises NV12 and BGRx; pinning
// BT.709 limited here selects its RGB→NV12 shader with our bitstream colorimetry.
pods.push(build_dmabuf_format(VideoFormat::NV12, &[0], preferred)?);
}
pods.push(build_dmabuf_format(
VideoFormat::BGRx,
&modifiers,
preferred,
)?);
pods
} else {
vec![serialize_pod(obj)?]
};
+209 -12
View File
@@ -308,8 +308,9 @@ float2 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
/// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV
/// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This
/// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`]
/// surfaces a clear error if `CreateRenderTargetView` rejects the plane format so the caller can fall
/// back to the existing R10 path.
/// surfaces a clear error if `CreateRenderTargetView` rejects the plane format. (There is no runtime
/// fallback — the error propagates through `try_consume` and ends the session; the "R10 path" the
/// original design referenced was never kept.)
pub(crate) struct HdrP010Converter {
vs: ID3D11VertexShader,
ps_y: ID3D11PixelShader,
@@ -737,14 +738,157 @@ fn p010_reference(r: f64, g: f64, b: f64) -> (f64, f64, f64) {
/// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL.
#[cfg(target_os = "windows")]
pub fn hdr_p010_selftest() -> Result<()> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_HARDWARE;
use windows::Win32::Graphics::Dxgi::IDXGIAdapter;
hdr_p010_selftest_at(64, 64, None)
}
// 64x64, even dims. A 4x4 grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform →
// exact chroma comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus
// a couple of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only).
const W: u32 = 64;
const H: u32 = 64;
/// [`hdr_p010_selftest`] at an arbitrary even size and (optionally) on a specific GPU vendor
/// (PCI vendor id, e.g. `0x8086` Intel / `0x10de` NVIDIA / `0x1002` AMD). The size matters on
/// top of the 64×64 default because the field sessions run at capture resolutions whose height
/// is NOT 16-aligned (1080 → the encoder's align16 pool seam) and a driver may treat the planar
/// RTVs differently at real sizes; the vendor pin matters on dual-GPU boxes where the default
/// adapter is not the one the session encodes on.
/// Test support (used by pf-encode's live e2e): the 8 sRGB colour bars (white/yellow/cyan/green/
/// magenta/red/blue/black, sRGB 1.0 = scRGB 1.0 = 80 nits) as a w×h FP16 scRGB texture on the
/// adapter with `luid`, converted through the REAL [`HdrP010Converter`] into a P010 texture with
/// **`BIND_RENDER_TARGET` only, `MiscFlags` 0 — the exact bind profile of the IDD out-ring** (the
/// CPU-upload encoder tests can't use that profile, so only this path exercises "RTV-written P010
/// → encoder ingest copy"). Returns `(device, p010)`; expected decoded codes per bar are the
/// bars_pq2020 fixture's: (490,512,512) (478,423,518) (464,525,473) (450,432,476) (350,584,585)
/// (325,448,598) (226,650,535) (64,512,512).
#[cfg(target_os = "windows")]
#[doc(hidden)]
pub fn hdr_p010_convert_bars_on_luid(
luid: [u8; 8],
w: u32,
h: u32,
) -> Result<(ID3D11Device, ID3D11Texture2D)> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("bars pattern needs even non-zero dimensions, got {w}x{h}");
}
// sRGB primaries at full/zero channels: sRGB EOTF(1.0)=1.0, (0)=0 → the scRGB pattern is
// pure 0/1 floats and the PQ/BT.2020 reference codes above are exact.
const BARS: [(f32, f32, f32); 8] = [
(1.0, 1.0, 1.0),
(1.0, 1.0, 0.0),
(0.0, 1.0, 1.0),
(0.0, 1.0, 0.0),
(1.0, 0.0, 1.0),
(1.0, 0.0, 0.0),
(0.0, 0.0, 1.0),
(0.0, 0.0, 0.0),
];
let bar_w = (w / 8).max(1) as usize;
let mut fp16 = vec![0u16; (w * h * 4) as usize];
for y in 0..h as usize {
for x in 0..w as usize {
let (r, g, b) = BARS[(x / bar_w).min(7)];
let i = (y * w as usize + x) * 4;
fp16[i] = f32_to_f16(r);
fp16[i + 1] = f32_to_f16(g);
fp16[i + 2] = f32_to_f16(b);
fp16[i + 3] = f32_to_f16(1.0);
}
}
// SAFETY: same single-device/single-thread contract as `hdr_p010_selftest_at`; the FP16
// initial-data Vec outlives the synchronous CreateTexture2D; the returned COM handles own
// their references.
unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().context("dxgi factory")?;
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).context("adapter by luid")?;
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("D3D11CreateDevice(luid) for bars convert")?;
let device = device.context("null device")?;
let context = context.context("null context")?;
let src_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R16G16B16A16_FLOAT,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: fp16.as_ptr() as *const c_void,
SysMemPitch: w * 8,
SysMemSlicePitch: 0,
};
let mut src_tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&src_desc, Some(&init), Some(&mut src_tex))
.context("CreateTexture2D(fp16 bars)")?;
let src_tex = src_tex.context("null src tex")?;
let mut src_srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&src_tex, None, Some(&mut src_srv))
.context("CreateShaderResourceView(fp16 bars)")?;
let src_srv = src_srv.context("null src srv")?;
// The IDD out-ring's exact profile: P010, RENDER_TARGET only, MiscFlags 0.
let p010_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_RENDER_TARGET.0 as u32,
..Default::default()
};
let mut p010: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&p010_desc, None, Some(&mut p010))
.context("CreateTexture2D(P010 bars dst)")?;
let p010 = p010.context("null p010 tex")?;
let conv = HdrP010Converter::new(&device)?;
conv.convert(&device, &context, &src_srv, &p010, w, h)?;
Ok((device, p010))
}
}
#[cfg(target_os = "windows")]
pub fn hdr_p010_selftest_at(w: u32, h: u32, vendor: Option<u32>) -> Result<()> {
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_UNKNOWN};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter, IDXGIFactory1};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("hdr-p010-selftest needs even non-zero dimensions, got {w}x{h}");
}
// A grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform → exact chroma
// comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus a couple
// of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only).
#[allow(non_snake_case)]
let (W, H) = (w, h);
const BLK: u32 = 16;
// (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values.
let named: [(&str, f32, f32, f32); 8] = [
@@ -797,12 +941,36 @@ pub fn hdr_p010_selftest() -> Result<()> {
// `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are
// proven individually at the `read_u16` closure below.
unsafe {
// Hardware D3D11 device (no adapter pin — the default GPU is fine for the self-test).
// Device on the requested vendor's adapter (dual-GPU boxes encode on a specific one), else
// the default hardware GPU. Always says which adapter ran — a PASS is only meaningful for
// the GPU it actually tested.
let adapter: Option<IDXGIAdapter> = match vendor {
None => None,
Some(want) => {
let factory: IDXGIFactory1 = CreateDXGIFactory1().context("dxgi factory")?;
let mut found = None;
for i in 0.. {
let Ok(a) = factory.EnumAdapters(i) else {
break;
};
let desc = a.GetDesc().context("adapter desc")?;
if desc.VendorId == want {
found = Some(a);
break;
}
}
Some(found.with_context(|| format!("no adapter with vendor id {want:#x}"))?)
}
};
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
None::<&IDXGIAdapter>,
D3D_DRIVER_TYPE_HARDWARE,
adapter.as_ref(),
if adapter.is_some() {
D3D_DRIVER_TYPE_UNKNOWN
} else {
D3D_DRIVER_TYPE_HARDWARE
},
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
@@ -814,6 +982,22 @@ pub fn hdr_p010_selftest() -> Result<()> {
.context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?;
let device = device.context("null device")?;
let context = context.context("null context")?;
{
let dxgi: windows::Win32::Graphics::Dxgi::IDXGIDevice =
device.cast().context("device -> IDXGIDevice")?;
let desc = dxgi.GetAdapter().context("GetAdapter")?.GetDesc()?;
let name = String::from_utf16_lossy(
&desc.Description[..desc
.Description
.iter()
.position(|&c| c == 0)
.unwrap_or(desc.Description.len())],
);
println!(
"adapter: {name} (vendor {:#06x}, luid {:08x}:{:08x})",
desc.VendorId, desc.AdapterLuid.HighPart, desc.AdapterLuid.LowPart
);
}
// Source FP16 texture (initialized) + SRV.
let src_desc = D3D11_TEXTURE2D_DESC {
@@ -1175,3 +1359,16 @@ impl VideoConverter {
blt.context("VideoProcessorBlt")
}
}
#[cfg(test)]
mod hdr_selftests {
/// LIVE (needs the GPU): [`super::hdr_p010_selftest_at`] at the field capture size — 1080 is
/// NOT 16-aligned, and the planar-RTV write path is driver-specific per vendor. Pinned to the
/// Intel adapter (`0x8086`), so it runs on the Intel validation boxes and errors out cleanly
/// ("no adapter") elsewhere. `cargo test -p pf-capture -- --ignored hdr_p010 --nocapture`.
#[test]
#[ignore]
fn hdr_p010_selftest_intel_1080_live() {
super::hdr_p010_selftest_at(1920, 1080, Some(0x8086)).expect("hdr p010 selftest @1080");
}
}
+10 -4
View File
@@ -876,10 +876,16 @@ impl Worker {
);
return;
};
let pref = self
.pad_info(id)
.map(|p| p.pref)
.unwrap_or(GamepadPref::Xbox360);
let pref = match self.pad_info(id) {
// Steam Input's virtual pad standing in front of the Deck's built-in controls (the
// only-pad-forwarded case, [`Self::forwarded_ids`]): declare the DECK kind, not the
// wrapper's Xbox 360 identity. [`Self::auto_pref`] already resolves the SESSION
// default this way, but a current host honors the per-pad arrival over the session
// default — so without this the host builds an X-Box 360 pad on a real Deck.
Some(p) if p.steam_virtual && is_steam_deck() => GamepadPref::SteamDeck,
Some(p) => p.pref,
None => GamepadPref::Xbox360,
};
match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) {
Ok(pad) => {
let mut slot = Slot::new(id, index, pref, pad);
+5
View File
@@ -325,6 +325,11 @@ pub fn connect_reject_message(reason: punktfunk_core::reject::RejectReason) -> S
"Client and host versions don't match — update both to the same release.".into()
}
R::Busy => "The host is busy with another session.".into(),
R::SetupFailed => {
"The host accepted the connection but couldn't start the stream — the host's log \
(web console Log) has the cause."
.into()
}
}
}
+5
View File
@@ -25,6 +25,11 @@ tracing = "0.1"
# A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`).
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
[target.'cfg(target_os = "windows")'.dev-dependencies]
# The QSV live e2e drives the REAL HdrP010Converter output (an RTV-written, ring-profile P010
# texture) into the encoder — the one seam the CPU-upload tests can't reach.
pf-capture = { path = "../pf-capture" }
[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
# Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms.
openh264 = "0.9"
+24 -6
View File
@@ -37,9 +37,11 @@ pub(crate) fn fourcc_to_vk(fourcc: u32) -> Option<vk::Format> {
const AR24: u32 = 0x3432_5241; // ARGB8888
const XB24: u32 = 0x3432_4258; // XBGR8888
const AB24: u32 = 0x3432_4241; // ABGR8888
const NV12: u32 = 0x3231_564e; // DRM_FORMAT_NV12
match fourcc {
XR24 | AR24 => Some(vk::Format::B8G8R8A8_UNORM),
XB24 | AB24 => Some(vk::Format::R8G8B8A8_UNORM),
NV12 => Some(vk::Format::G8_B8R8_2PLANE_420_UNORM),
_ => None,
}
}
@@ -90,9 +92,10 @@ pub(crate) unsafe fn import_rgb_dmabuf(
)
}
/// [`import_rgb_dmabuf`] with the image usage explicit and an optional video-profile list
/// (chained into the image create) — the RGB-direct encode path imports the captured buffer
/// as a profiled `VIDEO_ENCODE_SRC` image instead of a sampled one.
/// [`import_rgb_dmabuf`] with the image usage explicit and an optional video-profile list.
/// Despite the historical name, this also imports gamescope's one-fd LINEAR NV12: the UV
/// subresource layout comes from the producer's plane-1 chunk when it reported one, falling
/// back to the shared-stride contiguous-plane contract.
#[allow(clippy::too_many_arguments)]
pub(crate) unsafe fn import_rgb_dmabuf_as(
device: &ash::Device,
@@ -108,12 +111,27 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
use std::os::fd::IntoRawFd;
let fmt = fourcc_to_vk(d.fourcc)
.with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?;
let plane = [vk::SubresourceLayout::default()
let planes: Vec<vk::SubresourceLayout> = if fmt == vk::Format::G8_B8R8_2PLANE_420_UNORM {
let (uv_offset, uv_stride) = d.plane1.map(|(o, s)| (o as u64, s as u64)).unwrap_or((
d.offset as u64 + d.stride as u64 * ch as u64,
d.stride as u64,
));
vec![
vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64)];
.row_pitch(d.stride as u64),
vk::SubresourceLayout::default()
.offset(uv_offset)
.row_pitch(uv_stride),
]
} else {
vec![vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64)]
};
let mut drm = vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default()
.drm_format_modifier(d.modifier)
.plane_layouts(&plane);
.plane_layouts(&planes);
let mut ext = vk::ExternalMemoryImageCreateInfo::default()
.handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT);
let mut ci = vk::ImageCreateInfo::default()
+461 -100
View File
@@ -14,7 +14,7 @@ use super::vk_util::{color_range, find_mem, make_plain_image, make_view, pixel_t
use crate::{Codec, EncodedFrame, Encoder, EncoderCaps};
use anyhow::{bail, Context, Result};
use ash::vk;
use pf_frame::{CapturedFrame, FramePayload};
use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
use std::collections::VecDeque;
use std::ffi::c_void;
use std::os::fd::AsRawFd;
@@ -84,17 +84,36 @@ fn rgb_request() -> Option<bool> {
}
}
/// True-extent RGB-direct at unaligned modes (default ON; `PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0`
/// restores the padded-copy staging): direct-import the visible-size capture with the TRUE-SIZE
/// source `codedExtent` — RADV derives nonzero VCN firmware padding from it, so the EFC is told
/// the source lacks the alignment rows (see [`RgbDirect::true_extent`]). Guarded-tested on Van
/// Gogh 2026-07-21 (kernel clean, and the fastest 1080p encode path measured); the EFC only
/// exists on Mesa ≥ 26, where the `codedExtent`-driven `session_init` is guaranteed.
fn rgb_true_extent_request() -> bool {
std::env::var("PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT").as_deref() != Ok("0")
}
/// Live RGB-direct session config: the chroma-siting bits the session was created with
/// (chosen from what the driver advertises — see [`probe_rgb_direct`]).
struct RgbDirect {
x_offset: u32, // vk_valve_rgb::CHROMA_OFFSET_*
y_offset: u32,
/// The mode is not 64x16-aligned, so the captured buffer cannot be the encode source
/// directly (the EFC would read past it — the 2026-07-20 field GPU hang). Instead each
/// frame is copied into a per-slot ALIGNED BGRA staging image with the edge rows/columns
/// duplicated into the padding (transfer-only, no shader) and encoded from there. Aligned
/// modes keep the true zero-copy import.
/// under the session's ALIGNED source extent (the EFC read past it — the 2026-07-20 field
/// GPU hang, when the source `codedExtent` was the aligned size and RADV therefore derived
/// ZERO firmware padding). Each frame is copied into a per-slot ALIGNED BGRA staging image
/// with the edge rows/columns duplicated into the padding (transfer-only, no shader) and
/// encoded from there. Aligned modes keep the true zero-copy import.
padded: bool,
/// The default unaligned-mode source strategy (`PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0` falls
/// back to `padded`): direct-import the visible-size buffer and pass the TRUE-SIZE source
/// `codedExtent` — RADV then programs nonzero firmware padding from it (Mesa ≥ 24.2
/// derives `session_init` padding from `srcPictureResource.codedExtent`; see
/// [`VulkanVideoEncoder::native_nv12`]), telling the VCN the source lacks the alignment
/// rows, which the hardware edge-extends internally. The session/SPS/DPB stay app-aligned.
/// Guarded-tested on Van Gogh (kernel clean; fastest 1080p path measured).
true_extent: bool,
}
/// Stack storage for a complete rgb-chained video profile. Profiled image creation AFTER open
@@ -155,6 +174,51 @@ impl RgbProfileStack {
}
}
/// Non-RGB video profile rebuilt for native NV12 DMA-BUF imports. Image creation after `open`
/// must carry a profile identical by value to the session profile.
struct NativeProfileStack {
usage: vk::VideoEncodeUsageInfoKHR<'static>,
h265: vk::VideoEncodeH265ProfileInfoKHR<'static>,
av1: super::vk_av1_encode::VideoEncodeAV1ProfileInfoKHR,
profile: vk::VideoProfileInfoKHR<'static>,
}
impl NativeProfileStack {
fn new(codec_op: vk::VideoCodecOperationFlagsKHR) -> Self {
use super::vk_av1_encode as av1b;
Self {
usage: vk::VideoEncodeUsageInfoKHR::default()
.video_usage_hints(vk::VideoEncodeUsageFlagsKHR::STREAMING)
.video_content_hints(vk::VideoEncodeContentFlagsKHR::RENDERED)
.tuning_mode(vk::VideoEncodeTuningModeKHR::ULTRA_LOW_LATENCY),
h265: vk::VideoEncodeH265ProfileInfoKHR::default().std_profile_idc(
vk::native::StdVideoH265ProfileIdc_STD_VIDEO_H265_PROFILE_IDC_MAIN,
),
av1: av1b::VideoEncodeAV1ProfileInfoKHR {
s_type: av1b::stype(av1b::ST_PROFILE_INFO),
p_next: std::ptr::null(),
std_profile: vk::native::StdVideoAV1Profile_STD_VIDEO_AV1_PROFILE_MAIN,
},
profile: vk::VideoProfileInfoKHR::default()
.video_codec_operation(codec_op)
.chroma_subsampling(vk::VideoChromaSubsamplingFlagsKHR::TYPE_420)
.luma_bit_depth(vk::VideoComponentBitDepthFlagsKHR::TYPE_8)
.chroma_bit_depth(vk::VideoComponentBitDepthFlagsKHR::TYPE_8),
}
}
fn wire(&mut self, av1: bool) -> &vk::VideoProfileInfoKHR<'static> {
if av1 {
self.av1.p_next = &self.usage as *const _ as *const c_void;
self.profile.p_next = &self.av1 as *const _ as *const c_void;
} else {
self.h265.p_next = &self.usage as *const _ as *const c_void;
self.profile.p_next = &self.h265 as *const _ as *const c_void;
}
&self.profile
}
}
/// The Vulkan codec-operation bit for our codec selection (shared by open and the per-import
/// profile rebuilds — the two must agree, profile identity is by value).
fn codec_op_for(av1: bool) -> vk::VideoCodecOperationFlagsKHR {
@@ -173,12 +237,12 @@ enum SrcAcquire {
/// CSC path: `nv12_src` was written by this frame's compute batch (GENERAL layout; the
/// csc_sem orders the queues).
CscGeneral,
/// RGB-direct, first use of a dmabuf import: acquire from the foreign producer
/// (UNDEFINED preserves the modifier-tiled bytes) with a FOREIGN→encode-family transfer.
RgbFresh,
/// RGB-direct, cached import: the image is already VIDEO_ENCODE_SRC; visibility-only
/// barrier for the producer's out-of-band rewrite of the bytes.
RgbCached,
/// First use of a DMA-BUF imported directly as the video source: acquire from the foreign
/// producer (UNDEFINED preserves modifier-backed bytes) with a FOREIGN→encode-family transfer.
DmabufFresh,
/// Cached direct-source import: already VIDEO_ENCODE_SRC; visibility-only barrier for the
/// producer's out-of-band rewrite of the bytes.
DmabufCached,
/// RGB-direct CPU upload: the compute queue copied the staging buffer in (semaphore
/// ordered); transition TRANSFER_DST → VIDEO_ENCODE_SRC.
CpuUpload,
@@ -376,18 +440,33 @@ pub struct VulkanVideoEncoder {
/// `ENCODE_QUALITY_LEVEL` control and baked into the session-parameters object (the spec
/// requires the two to match).
quality_level: u32,
/// `PUNKTFUNK_PERF` CSC/encode split: >0 ⇒ per-frame GPU timestamps are recorded around the
/// compute batch and a sampled `csc_us` line is logged; the value is the device timestamp
/// period in ns/tick. 0.0 ⇒ off (env unset, or the compute family has no timestamp support).
/// `PUNKTFUNK_PERF` pre-encode split: >0 ⇒ per-frame GPU timestamps bracket either the
/// RGB→NV12 compute batch or the native-NV12 padded copy. The measured duration is logged
/// separately from the host's fence wait; 0.0 means disabled or unsupported.
ts_period_ns: f64,
perf_at: std::time::Instant, // last sampled csc_us log (2 s cadence)
perf_at: std::time::Instant,
/// RGB-direct (EFC) session config — `Some` ⇒ the session's picture format is BGRA, frames
/// are handed to the encoder as RGB (dmabuf import or CPU upload) and the VCN front-end does
/// the CSC; `None` ⇒ the compute-CSC path. Fixed per session (the picture format is baked
/// into the video session).
rgb: Option<RgbDirect>,
/// One-shot warning latch: a cursor bitmap arrived on an RGB-direct session (EFC cannot
/// composite it — the cursor will be missing from the stream until the CSC path is used).
/// Producer supplied native NV12 rather than packed RGB. EVERY mode encodes the imported
/// visible-size buffer directly — safely, because native sessions use TRUE-SIZE headers:
/// the SPS/sequence header is authored at the render size and every picture resource's
/// `codedExtent` matches it, so RADV programs the VCN with `session_init` extent = the true
/// size and a nonzero `padding_width/height`, and the FIRMWARE edge-extends the alignment
/// padding internally (radv_video_enc.c `radv_enc_session_init`; the driver also rounds the
/// bitstream SPS up itself and compensates with a conformance window —
/// `radv_video_patch_encode_session_parameters`, per the VK_KHR_video_encode_h265 proposal's
/// "implementations may override" clause). The source is never read past its extent — unlike
/// the app-aligned-SPS convention the CSC/RGB paths use, where the coded extent is 64x16-
/// aligned and an undersized direct source is the OOB-read class behind the 2026-07-20 field
/// GPU reset (those paths keep their aligned-size sources/staging).
native_nv12: bool,
/// One-shot warning latch: a cursor bitmap arrived on an RGB-direct or native-NV12 session
/// (neither has a compositing stage — the cursor will be missing from the stream until the
/// CSC path is used).
warned_cursor: bool,
/// A [`reconfigure_bitrate`](Encoder::reconfigure_bitrate) rate not yet installed in the video
/// session. The next `record_submit` emits an `ENCODE_RATE_CONTROL` control command carrying it
@@ -422,14 +501,24 @@ impl VulkanVideoEncoder {
/// (B2). `PUNKTFUNK_VULKAN_RGB_DIRECT` overrides both ways (see [`rgb_request`]).
pub fn open(
codec: Codec,
format: PixelFormat,
width: u32,
height: u32,
fps: u32,
bitrate_bps: u64,
cursor_blend: bool,
) -> Result<Self> {
let want_rgb = rgb_request().unwrap_or(!cursor_blend);
Self::open_opts(codec, width, height, fps, bitrate_bps, want_rgb)
let native_nv12 = format == PixelFormat::Nv12;
let want_rgb = !native_nv12 && rgb_request().unwrap_or(!cursor_blend);
Self::open_opts_inner(
codec,
width,
height,
fps,
bitrate_bps,
want_rgb,
native_nv12,
)
}
/// `open` with the RGB-direct request explicit instead of read from the env — the smoke
@@ -443,6 +532,18 @@ impl VulkanVideoEncoder {
fps: u32,
bitrate_bps: u64,
want_rgb: bool,
) -> Result<Self> {
Self::open_opts_inner(codec, width, height, fps, bitrate_bps, want_rgb, false)
}
fn open_opts_inner(
codec: Codec,
width: u32,
height: u32,
fps: u32,
bitrate_bps: u64,
want_rgb: bool,
native_nv12: bool,
) -> Result<Self> {
if !matches!(codec, Codec::H265 | Codec::Av1) {
bail!("vulkan-encode backend supports HEVC + AV1 only (got {codec:?})");
@@ -464,6 +565,7 @@ impl VulkanVideoEncoder {
fps.max(1),
bitrate_bps.max(1_000_000),
want_rgb,
native_nv12,
)
}
}
@@ -478,6 +580,7 @@ impl VulkanVideoEncoder {
fps: u32,
bitrate: u64,
want_rgb: bool,
native_nv12: bool,
) -> Result<Self> {
use super::vk_av1_encode as av1b;
use super::vk_valve_rgb as vrgb;
@@ -551,33 +654,50 @@ impl VulkanVideoEncoder {
0.0
};
// RGB-direct (EFC) resolution — BEFORE the profile is built, because an active rgb
// session changes the profile identity (the rgb-conversion struct rides the chain) and
// the session's picture format. The probe runs unconditionally: its verdict is the
// field telemetry that decides where B2 can default this on.
let rgb_probe = probe_rgb_direct(&instance, &vq_inst, pd, codec_op, av1);
// ALIGNMENT GATE (field GPU-hang, 2026-07-20): the coded extent is 64x16-aligned but the
// captured dmabuf is only the REAL mode size — handing it to the encoder as the direct
// source makes the VCN's EFC read the alignment-padding rows PAST the end of the buffer.
// At 1920x1080 (coded 1088) that is 8 rows = 61 KB of out-of-bounds reads per frame:
// deterministic VM protection faults, vcn_enc ring timeouts, and — through the stall
// watchdog's rebuild-and-refault loop — a full MODE1 GPU reset with VRAM loss. The CSC
// shader absorbs the padding by clamping reads and duplicating the edge; RGB-direct has
// no such stage. Mode select: an aligned mode (720p/1440p/4K) encodes the imported
// buffer directly (true zero-copy); an unaligned one (1080p!) goes through the
// padded-copy staging image (see [`RgbDirect::padded`]) — transfer-only, still no
// compute CSC.
// Resolve the encode source before building the profile: EFC RGB conversion changes
// profile identity; producer-native NV12 uses the ordinary 4:2:0 profile.
let aligned = rw == w && rh == h;
let rgb_probe = if native_nv12 {
Err("not-probed(native NV12 source selected)")
} else {
probe_rgb_direct(&instance, &vq_inst, pd, codec_op, av1)
};
let rgb_cfg: Option<RgbDirect> = match (&rgb_probe, want_rgb) {
(Ok((x, y)), true) => Some(RgbDirect {
(Ok((x, y)), true) => {
let true_extent = !aligned && rgb_true_extent_request();
Some(RgbDirect {
x_offset: *x,
y_offset: *y,
padded: !aligned,
}),
padded: !aligned && !true_extent,
true_extent,
})
}
_ => None,
};
if native_nv12 {
tracing::info!(
native_nv12 = "active(direct-import)",
source_width = rw,
source_height = rh,
fw_padding_width = w - rw,
fw_padding_height = h - rh,
"vulkan-encode: producer-native NV12 encode source (true-size headers: the \
driver aligns the bitstream SPS itself and the firmware edge-extends the \
padding the source is never read past its extent)"
);
} else {
tracing::info!(
rgb_direct = match (&rgb_probe, want_rgb, &rgb_cfg) {
(
_,
_,
Some(RgbDirect {
true_extent: true, ..
}),
) =>
"active(true-extent: unaligned mode, direct import with the true-size \
source codedExtent RADV firmware padding covers the alignment rows; \
PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 restores the padded copy)",
(_, _, Some(RgbDirect { padded: false, .. })) => "active",
(_, _, Some(RgbDirect { padded: true, .. })) =>
"active(padded-copy: mode is not 64x16-aligned — staging blit + edge \
@@ -586,11 +706,11 @@ impl VulkanVideoEncoder {
"available(off: PUNKTFUNK_VULKAN_RGB_DIRECT=0, or a cursor-blend session \
=1 forces)",
(Err(e), _, None) => e,
// (Ok, wanted) always builds Some above.
(Ok(_), true, None) => unreachable!("rgb gate and cfg disagree"),
},
"vulkan-encode: EFC RGB-direct encode source (design/vulkan-rgb-direct-encode.md)"
);
}
// the encode profile — H265 Main, or AV1 Main; chained raw and uniformly (vendored AV1 +
// rgb structs can't `push_next`, and the chain must match [`RgbProfileStack::wire`]'s
@@ -820,13 +940,19 @@ impl VulkanVideoEncoder {
// ---- session parameters + header framing (HEVC: VPS/SPS/PPS on keyframes; AV1: a
// temporal-delimiter OBU per frame + a sequence-header OBU on keyframes) ----
// Native NV12 authors TRUE-SIZE headers (SPS/seq at the render size, no app-side
// conformance window): RADV rounds the bitstream SPS up itself and, keyed off the
// matching true-size codedExtent, programs the VCN with firmware padding so the source
// is never read past its extent. The CSC/RGB paths keep the app-aligned convention
// (their sources genuinely cover the aligned extent).
let (hdr_w, hdr_h) = if native_nv12 { (rw, rh) } else { (w, h) };
let (params, header, frame_prefix) = if av1 {
build_parameters_av1(
&device,
&vq_dev,
session,
w,
h,
hdr_w,
hdr_h,
rw,
rh,
av1_caps.max_level,
@@ -839,8 +965,8 @@ impl VulkanVideoEncoder {
&vq_dev,
&venc_dev,
session,
w,
h,
hdr_w,
hdr_h,
rw,
rh,
quality_level,
@@ -996,9 +1122,14 @@ impl VulkanVideoEncoder {
compute_pool,
bs_size,
sampler,
ts_period_ns > 0.0 && rgb_cfg.is_none(),
rgb_cfg.is_none(),
rgb_cfg.as_ref().is_some_and(|c| c.padded),
ts_period_ns > 0.0
&& ((rgb_cfg.is_none() && !native_nv12)
|| rgb_cfg.as_ref().is_some_and(|c| c.padded)),
rgb_cfg.is_none() && !native_nv12,
rgb_cfg
.as_ref()
.is_some_and(|c| c.padded)
.then_some(vk::Format::B8G8R8A8_UNORM),
guard.frames.last_mut().expect("frame just pushed"),
)?;
}
@@ -1052,6 +1183,7 @@ impl VulkanVideoEncoder {
ts_period_ns,
perf_at: std::time::Instant::now(),
rgb: rgb_cfg,
native_nv12,
warned_cursor: false,
pending_bitrate: None,
width: w,
@@ -1200,15 +1332,31 @@ impl VulkanVideoEncoder {
}
}
/// Import a packed-RGB dmabuf as a VkImage (explicit DRM modifier). CSC sessions import it
/// SAMPLED (the compute shader reads it); RGB-direct sessions import it as a profiled
/// `VIDEO_ENCODE_SRC` — the buffer IS the encode source. Caller destroys.
/// Import a DMA-BUF VkImage with usage/profile matching this session's source mode. Native
/// NV12 and aligned RGB-direct are profiled `VIDEO_ENCODE_SRC` images. Padded RGB-direct
/// imports the producer allocation as transfer-source only.
unsafe fn import_dmabuf(
&self,
d: &pf_frame::DmabufFrame,
cw: u32,
ch: u32,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
if self.native_nv12 {
let mut ps = NativeProfileStack::new(codec_op_for(self.codec == Codec::Av1));
let profile = *ps.wire(self.codec == Codec::Av1);
let arr = [profile];
let mut plist = vk::VideoProfileListInfoKHR::default().profiles(&arr);
return super::vk_util::import_rgb_dmabuf_as(
&self.device,
&self.ext_fd,
&self.mem_props,
d,
cw,
ch,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR,
Some(&mut plist),
);
}
if self.rgb.as_ref().is_some_and(|r| r.padded) {
// Padded-copy mode: the import is only ever a transfer SOURCE (the blit into the
// aligned staging image) — plain TRANSFER_SRC, no video profile involved.
@@ -1503,8 +1651,21 @@ impl VulkanVideoEncoder {
setup_idx = (setup_idx + 1) % DPB_SLOTS as usize;
}
// ---- 2..4 diverge by encode source; the RGB-direct twin returns through the shared
// bookkeeping tail (design/vulkan-rgb-direct-encode.md B1) ----
// ---- 2..4 diverge by encode source; native NV12 and RGB-direct return through the
// shared bookkeeping tail ----
if self.native_nv12 {
self.record_submit_nv12(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?;
self.post_submit_bookkeeping(
slot,
frame.pts_ns,
wire,
is_idr,
recovery,
setup_idx,
poc,
);
return Ok(());
}
if self.rgb.is_some() {
self.record_submit_rgb(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?;
self.post_submit_bookkeeping(
@@ -1831,12 +1992,20 @@ impl VulkanVideoEncoder {
Ok(())
}
/// Padded-copy blit (unaligned-mode RGB-direct): record — into `compute_cmd` — the visible
/// frame copy from the imported capture image into the aligned staging image, plus the edge
/// duplication into the 64x16 padding (the same edge semantics the CSC shader implements
/// with clamped reads). Transfer-only, no shader. The staging image lives in GENERAL — it
/// is both copy dst and, for the right-column pass, copy src — and the encode acquires it
/// via [`SrcAcquire::CscGeneral`] (content produced on the compute queue, csc_sem-ordered).
/// Padded-copy blit (unaligned-mode RGB-direct or native NV12): record — into `compute_cmd`
/// — the visible frame copy from the imported capture image into the aligned staging image,
/// plus the edge duplication into the 64x16 padding (the same edge semantics the CSC shader
/// implements with clamped reads). Transfer-only, no shader. The staging image lives in
/// GENERAL — it is both copy dst and, for the right-column pass, copy src — and the encode
/// acquires it via [`SrcAcquire::CscGeneral`] (content produced on the compute queue,
/// csc_sem-ordered).
///
/// `planes` lists the copy aspects with their subsampling divisor — `[(COLOR, 1)]` for
/// packed RGB, `[(PLANE_0, 1), (PLANE_1, 2)]` for NV12 (multi-planar copy regions are in
/// each plane's own coordinate space; barriers on non-disjoint images stay COLOR-aspect).
/// Every divisor must divide the visible and aligned extents (4:2:0 frames are even, the
/// coded extent is 64x16-aligned).
#[allow(clippy::too_many_arguments)]
unsafe fn record_pad_blit(
&self,
dev: &ash::Device,
@@ -1844,6 +2013,8 @@ impl VulkanVideoEncoder {
src: vk::Image,
src_fresh: bool,
pad: vk::Image,
ts_pool: vk::QueryPool,
planes: &[(vk::ImageAspectFlags, u32)],
) -> Result<()> {
let (rw, rh) = (self.render_w, self.render_h);
let (w, h) = (self.width, self.height);
@@ -1852,6 +2023,10 @@ impl VulkanVideoEncoder {
&vk::CommandBufferBeginInfo::default()
.flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT),
)?;
if self.ts_period_ns > 0.0 {
dev.cmd_reset_query_pool(compute_cmd, ts_pool, 0, 2);
dev.cmd_write_timestamp2(compute_cmd, vk::PipelineStageFlags2::NONE, ts_pool, 0);
}
// Acquire the imported capture buffer for transfer reads (FOREIGN hand-off on first
// import — UNDEFINED preserves the modifier-tiled bytes — visibility-only afterwards),
// and the staging image for transfer writes (prior contents discarded).
@@ -1893,10 +2068,11 @@ impl VulkanVideoEncoder {
compute_cmd,
&vk::DependencyInfo::default().image_memory_barriers(&[src_acq, pad_dst]),
);
let region =
|aspect: vk::ImageAspectFlags, sx: i32, sy: i32, dx: i32, dy: i32, cw: u32, ch: u32| {
let layers = vk::ImageSubresourceLayers::default()
.aspect_mask(vk::ImageAspectFlags::COLOR)
.aspect_mask(aspect)
.layer_count(1);
let region = |sx: i32, sy: i32, dx: i32, dy: i32, cw: u32, ch: u32| {
vk::ImageCopy::default()
.src_subresource(layers)
.dst_subresource(layers)
@@ -1908,11 +2084,16 @@ impl VulkanVideoEncoder {
depth: 1,
})
};
// Pass 1 — from the capture: the visible region, then each bottom padding row as a
// copy of the last visible row (1080p: 8 such rows). One call, disjoint regions.
let mut regions = vec![region(0, 0, 0, 0, rw, rh)];
// Pass 1 — from the capture, per plane: the visible region, then each bottom padding row
// as a copy of the last visible row (1080p: 8 luma + 4 chroma rows). One call, disjoint
// regions.
let mut regions = Vec::new();
for &(aspect, div) in planes {
let (rw, rh, h) = (rw / div, rh / div, h / div);
regions.push(region(aspect, 0, 0, 0, 0, rw, rh));
for y in rh..h {
regions.push(region(0, rh as i32 - 1, 0, y as i32, rw, 1));
regions.push(region(aspect, 0, rh as i32 - 1, 0, y as i32, rw, 1));
}
}
dev.cmd_copy_image(
compute_cmd,
@@ -1942,9 +2123,13 @@ impl VulkanVideoEncoder {
compute_cmd,
&vk::DependencyInfo::default().image_memory_barriers(&[self_dep]),
);
let cols: Vec<vk::ImageCopy> = (rw..w)
.map(|x| region(rw as i32 - 1, 0, x as i32, 0, 1, h))
.collect();
let mut cols = Vec::new();
for &(aspect, div) in planes {
let (rw, w, h) = (rw / div, w / div, h / div);
for x in rw..w {
cols.push(region(aspect, rw as i32 - 1, 0, x as i32, 0, 1, h));
}
}
dev.cmd_copy_image(
compute_cmd,
pad,
@@ -1954,10 +2139,112 @@ impl VulkanVideoEncoder {
&cols,
);
}
if self.ts_period_ns > 0.0 {
dev.cmd_write_timestamp2(
compute_cmd,
vk::PipelineStageFlags2::ALL_COMMANDS,
ts_pool,
1,
);
}
dev.end_command_buffer(compute_cmd)?;
Ok(())
}
/// Producer-native NV12 submit: import the producer's visible-size buffer directly as the
/// encode source. Safe at every mode because native sessions run true-size headers — the
/// picture resources' codedExtent equals the source extent and the VCN firmware edge-extends
/// the alignment padding internally (see the [`Self::native_nv12`] field docs).
#[allow(clippy::too_many_arguments)]
unsafe fn record_submit_nv12(
&mut self,
slot: usize,
frame: &CapturedFrame,
is_idr: bool,
recovery: bool,
ref_slot: usize,
setup_idx: usize,
poc: i32,
) -> Result<()> {
if frame.format != PixelFormat::Nv12 {
bail!(
"vulkan-encode (native NV12): negotiated NV12 but received {:?}",
frame.format
);
}
if frame.width != self.render_w || frame.height != self.render_h {
bail!(
"vulkan-encode (native NV12): frame {}x{} != mode {}x{}",
frame.width,
frame.height,
self.render_w,
self.render_h
);
}
if frame.width % 2 != 0 || frame.height % 2 != 0 {
bail!("vulkan-encode (native NV12): 4:2:0 frame dimensions must be even");
}
let FramePayload::Dmabuf(d) = &frame.payload else {
bail!("vulkan-encode (native NV12): producer frame is not a DMA-BUF");
};
if d.fourcc != pf_frame::drm_fourcc(PixelFormat::Nv12).expect("NV12 FourCC") {
bail!(
"vulkan-encode (native NV12): DMA-BUF FourCC {:#x} is not NV12",
d.fourcc
);
}
if d.modifier != 0 {
bail!(
"vulkan-encode (native NV12): only LINEAR is supported, got modifier {:#x}",
d.modifier
);
}
// No compositing stage exists here (like RGB-direct/EFC): gamescope embeds its pointer
// in the produced pixels, but any other NV12 producer's metadata cursor would be lost —
// say so once instead of silently.
if frame.cursor.is_some() && !self.warned_cursor {
self.warned_cursor = true;
tracing::warn!(
"cursor bitmap on a native-NV12 session — nothing composites it; the cursor \
will be missing from the stream (unset PUNKTFUNK_PIPEWIRE_NV12 for \
metadata-cursor captures)"
);
}
let dev = self.device.clone();
let cmd = self.frames[slot].cmd;
let fence = self.frames[slot].fence;
let query_pool = self.frames[slot].query_pool;
let bs_buf = self.frames[slot].bs_buf;
// The frame-size check above proved the buffer covers the (true-size) coded extent —
// the direct import is the encode source at every mode.
let (src_img, src_view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let acquire = if fresh {
SrcAcquire::DmabufFresh
} else {
SrcAcquire::DmabufCached
};
if self.codec == Codec::Av1 {
self.record_coding_av1(
&dev, cmd, query_pool, bs_buf, src_img, src_view, acquire, is_idr, recovery,
ref_slot, setup_idx, poc,
)?;
} else {
self.record_coding_h265(
&dev, cmd, query_pool, bs_buf, src_img, src_view, acquire, is_idr, ref_slot,
setup_idx, poc,
)?;
}
dev.reset_fences(&[fence])?;
// The whole frame is one submit: the encoder reads the imported NV12 directly.
let ecmds = [cmd];
dev.queue_submit(
self.encode_queue,
&[vk::SubmitInfo::default().command_buffers(&ecmds)],
fence,
)?;
Ok(())
}
/// RGB-direct twin of [`record_submit`]'s steps 24 (step 1 and the bookkeeping tail are
/// shared): resolve the RGB encode source — the imported capture dmabuf itself, or the CPU
/// staging upload — record the encode, and submit. There is no compute CSC: the VCN EFC
@@ -1981,6 +2268,7 @@ impl VulkanVideoEncoder {
let fence = self.frames[slot].fence;
let query_pool = self.frames[slot].query_pool;
let bs_buf = self.frames[slot].bs_buf;
let ts_pool = self.frames[slot].ts_pool;
// EFC cannot composite the cursor bitmap the metadata-cursor captures hand us — say so
// once instead of silently losing the pointer (gamescope, the flagship, embeds it).
if frame.cursor.is_some() && !self.warned_cursor {
@@ -1995,25 +2283,33 @@ impl VulkanVideoEncoder {
let (src_img, src_view, acquire, compute_active) = match &frame.payload {
FramePayload::Dmabuf(d) if !padded => {
// Defense in depth for the OOB class the alignment gate closes at open: the
// imported buffer must cover the FULL coded extent, or the EFC reads past it
// (VM faults → VCN ring hang → GPU reset, the 2026-07-20 field report). A
// mismatched frame (mid-flight mode change, odd capture) errors out here and
// takes the encoder-rebuild path instead of faulting the GPU.
if frame.width != self.width || frame.height != self.height {
// imported buffer must cover the source extent the encode declares — the FULL
// aligned coded extent normally (or the EFC reads past it: VM faults → VCN
// ring hang → GPU reset, the 2026-07-20 field report), the render size in
// true-extent mode (where the declared source codedExtent shrinks with it and
// RADV's firmware padding covers the alignment rows). A mismatched frame
// (mid-flight mode change, odd capture) errors out here and takes the
// encoder-rebuild path instead of faulting the GPU.
let (need_w, need_h) = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
(self.render_w, self.render_h)
} else {
(self.width, self.height)
};
if frame.width != need_w || frame.height != need_h {
bail!(
"vulkan-encode (rgb-direct): frame {}x{} does not cover the coded \
extent {}x{} refusing an out-of-bounds encode source",
"vulkan-encode (rgb-direct): frame {}x{} does not cover the declared \
source extent {}x{} refusing an out-of-bounds encode source",
frame.width,
frame.height,
self.width,
self.height
need_w,
need_h
);
}
let (img, view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let acq = if fresh {
SrcAcquire::RgbFresh
SrcAcquire::DmabufFresh
} else {
SrcAcquire::RgbCached
SrcAcquire::DmabufCached
};
(img, view, acq, false)
}
@@ -2036,7 +2332,15 @@ impl VulkanVideoEncoder {
let (img, _view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let pad_img = self.frames[slot].pad_img;
let pad_view = self.frames[slot].pad_view;
self.record_pad_blit(&dev, compute_cmd, img, fresh, pad_img)?;
self.record_pad_blit(
&dev,
compute_cmd,
img,
fresh,
pad_img,
ts_pool,
&[(vk::ImageAspectFlags::COLOR, 1)],
)?;
// The staging image ends the blit in GENERAL with the csc_sem ordering the
// hand-off — exactly the CscGeneral acquire contract.
(pad_img, pad_view, SrcAcquire::CscGeneral, true)
@@ -2240,13 +2544,13 @@ impl VulkanVideoEncoder {
.src_stage_mask(vk::PipelineStageFlags2::ALL_COMMANDS)
.src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::GENERAL),
SrcAcquire::RgbFresh => src_base
SrcAcquire::DmabufFresh => src_base
.src_stage_mask(vk::PipelineStageFlags2::NONE)
.src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::UNDEFINED)
.src_queue_family_index(vk::QUEUE_FAMILY_FOREIGN_EXT)
.dst_queue_family_index(self.encode_family),
SrcAcquire::RgbCached => src_base
SrcAcquire::DmabufCached => src_base
.src_stage_mask(vk::PipelineStageFlags2::NONE)
.src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::VIDEO_ENCODE_SRC_KHR),
@@ -2284,9 +2588,29 @@ impl VulkanVideoEncoder {
poc: i32,
) -> Result<()> {
use ash::vk::native as h;
let ext2d = vk::Extent2D {
// Setup/reference extent: the aligned size for app-aligned sessions (CSC, RGB — it
// pairs with their aligned SPS), the render size for native NV12's true-size headers.
let ext2d = if self.native_nv12 {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
vk::Extent2D {
width: self.width,
height: self.height,
}
};
// Source extent additionally drops to the render size in RGB true-extent mode: RADV
// derives the VCN firmware padding from srcPictureResource.codedExtent (Mesa ≥ 24.2),
// so the visible-size import is never read past its extent (see RgbDirect::true_extent).
let src_extent = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
ext2d
};
let ref_poc = if is_idr { 0 } else { self.slot_poc[ref_slot] };
@@ -2458,7 +2782,7 @@ impl VulkanVideoEncoder {
}
dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty());
let src_res = vk::VideoPictureResourceInfoKHR::default()
.coded_extent(ext2d)
.coded_extent(src_extent)
.image_view_binding(src_view);
let mut enc = vk::VideoEncodeInfoKHR::default()
.dst_buffer(bs_buf)
@@ -2502,9 +2826,29 @@ impl VulkanVideoEncoder {
) -> Result<()> {
use super::vk_av1_encode as av1;
use ash::vk::native as h;
let ext2d = vk::Extent2D {
// Setup/reference extent: the aligned size for app-aligned sessions (CSC, RGB — it
// pairs with their aligned SPS), the render size for native NV12's true-size headers.
let ext2d = if self.native_nv12 {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
vk::Extent2D {
width: self.width,
height: self.height,
}
};
// Source extent additionally drops to the render size in RGB true-extent mode: RADV
// derives the VCN firmware padding from srcPictureResource.codedExtent (Mesa ≥ 24.2),
// so the visible-size import is never read past its extent (see RgbDirect::true_extent).
let src_extent = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
ext2d
};
// ---- required AV1 frame sub-structs (single tile; no CDEF/LR/segmentation/global-motion) ----
@@ -2726,7 +3070,7 @@ impl VulkanVideoEncoder {
}
dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty());
let src_res = vk::VideoPictureResourceInfoKHR::default()
.coded_extent(ext2d)
.coded_extent(src_extent)
.image_view_binding(src_view);
let mut enc = vk::VideoEncodeInfoKHR::default()
.dst_buffer(bs_buf)
@@ -2784,10 +3128,13 @@ impl VulkanVideoEncoder {
);
}
let (off, len) = (off64 as usize, len64 as usize);
// PUNKTFUNK_PERF CSC split (best-effort): the fence signaled, so the compute batch that
// wrote these timestamps completed long ago — WAIT is a formality. Sampled to one log
// line per ~2 s; `wait_us` in the pump's stage perf minus this ≈ the ASIC encode.
if self.ts_period_ns > 0.0 && f.ts_pool != vk::QueryPool::null() {
// PUNKTFUNK_PERF pre-encode split (best-effort): the fence signaled, so the compute/transfer
// timestamps are available. This is a device duration, not permission to label the
// remaining host fence wait as pure VCN time; queueing and synchronization remain in it.
if self.ts_period_ns > 0.0
&& f.ts_pool != vk::QueryPool::null()
&& self.perf_at.elapsed() >= std::time::Duration::from_secs(2)
{
let mut ts = [0u64; 2];
if dev
.get_query_pool_results(
@@ -2797,16 +3144,26 @@ impl VulkanVideoEncoder {
vk::QueryResultFlags::TYPE_64 | vk::QueryResultFlags::WAIT,
)
.is_ok()
&& self.perf_at.elapsed() >= std::time::Duration::from_secs(2)
{
self.perf_at = std::time::Instant::now();
let csc_us = (ts[1].saturating_sub(ts[0]) as f64 * self.ts_period_ns) / 1000.0;
let pre_encode_us =
(ts[1].saturating_sub(ts[0]) as f64 * self.ts_period_ns) / 1000.0;
if self.rgb.as_ref().is_some_and(|r| r.padded) {
tracing::info!(
csc_us = format!("{csc_us:.0}"),
rgb_copy_us = format!("{pre_encode_us:.0}"),
au_bytes = len,
"vulkan-encode split (sampled): csc=GPU compute batch (import barriers + \
CSC + plane copies); ASIC encode stage-perf wait_us csc"
"vulkan-encode split (sampled): padded RGB copy device time before EFC; \
remaining fence wait still includes queue synchronization + RGBYUV EFC \
+ video encode"
);
} else {
tracing::info!(
csc_us = format!("{pre_encode_us:.0}"),
au_bytes = len,
"vulkan-encode split (sampled): RGB→NV12 compute batch device time; \
remaining fence wait still includes queue synchronization + video encode"
);
}
}
}
let f = &self.frames[slot];
@@ -3412,26 +3769,30 @@ unsafe fn make_frame(
sampler: vk::Sampler,
with_ts: bool,
csc: bool,
rgb_pad: bool,
pad_fmt: Option<vk::Format>,
f: &mut Frame,
) -> Result<()> {
// "no cursor uploaded yet" sentinel — a real serial may be 0 (see `prep_cursor`).
f.cursor_serial = u64::MAX;
// Padded-copy RGB staging (unaligned-mode RGB-direct): aligned BGRA encode-src filled by a
// transfer blit each frame — concurrent compute (copy) + encode (source read).
if rgb_pad {
// Padded-copy staging (unaligned-mode RGB-direct or native NV12): an aligned encode-src in
// the session's picture format, filled by a transfer blit each frame — concurrent compute
// (copy) + encode (source read). TRANSFER_SRC because the width-padding pass self-copies the
// staging image's own last visible column (see `record_pad_blit`).
if let Some(fmt) = pad_fmt {
(f.pad_img, f.pad_mem) = make_video_image(
device,
mem_props,
vk::Format::B8G8R8A8_UNORM,
fmt,
w,
h,
1,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR | vk::ImageUsageFlags::TRANSFER_DST,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR
| vk::ImageUsageFlags::TRANSFER_DST
| vk::ImageUsageFlags::TRANSFER_SRC,
profile_list,
fams,
)?;
f.pad_view = make_view(device, f.pad_img, vk::Format::B8G8R8A8_UNORM, 0)?;
f.pad_view = make_view(device, f.pad_img, fmt, 0)?;
}
// RGB-direct sessions never touch the CSC pipeline: no NV12 encode-src, no Y/UV scratch, no
// cursor overlay, no descriptor set — the encode source is the imported RGB itself (or the
+257 -5
View File
@@ -478,10 +478,14 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
b
});
// HDR signalling (10-bit sessions are the HDR path on Windows — same coupling as NVENC):
// BT.2020/PQ colour description + the source's mastering/CLL grade at every IDR.
// Colour signalling, written UNCONDITIONALLY (mirrors nvenc_core.rs): the input is already
// CSC'd to a specific matrix — BT.709 limited for SDR (the capture-side VideoConverter),
// BT.2020 PQ for HDR (HdrP010Converter) — so the stream must say so. An SDR stream without a
// colour description leaves the choice to the decoder's "unspecified" default, and
// Moonlight/third-party/Android-vendor decoders default to 601 at sub-HD → mis-rendered
// colours. (10-bit sessions are the HDR path on Windows — same coupling as NVENC.)
let hdr = cfg.ten_bit && cfg.codec != Codec::H264;
let vsi = hdr.then(|| {
let vsi = {
// SAFETY: all-zero is valid; header stamped below.
let mut b: Box<vpl::mfxExtVideoSignalInfo> = Box::new(unsafe { std::mem::zeroed() });
b.Header.BufferId = vpl::MFX_EXTBUFF_VIDEO_SIGNAL_INFO as u32;
@@ -489,11 +493,17 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
b.VideoFormat = 5; // unspecified
b.VideoFullRange = 0;
b.ColourDescriptionPresent = 1;
if hdr {
b.ColourPrimaries = 9; // BT.2020
b.TransferCharacteristics = 16; // SMPTE ST 2084 (PQ)
b.MatrixCoefficients = 9; // BT.2020 non-constant
b
});
} else {
b.ColourPrimaries = 1; // BT.709
b.TransferCharacteristics = 1; // BT.709
b.MatrixCoefficients = 1; // BT.709
}
Some(b)
};
let mastering = cfg.hdr_meta.filter(|_| hdr).map(|m| {
// SAFETY: all-zero is valid; header stamped below.
let mut b: Box<vpl::mfxExtMasteringDisplayColourVolume> =
@@ -1994,4 +2004,246 @@ mod tests {
"the bitrate retarget emitted a keyframe (StartNewSequence leak)"
);
}
/// FULL-CHAIN colour check at the field capture size: a known P010 colour-bar source at
/// 1920x1080 — whose height is NOT 16-aligned, so the ingest `CopySubresourceRegion` copies
/// into a 1920x1088 runtime pool surface whose chroma plane sits at a DIFFERENT row offset
/// than the source's (the seam no 640x480 test exercises) — encoded to Main10 HEVC and
/// dumped to `%TEMP%\pf_qsv_1080_bars.h265` for off-box decode verification against the
/// same codes. On-box this asserts stream shape; the pixel verdict needs a decoder.
#[test]
fn qsv_live_p010_1080_colorbars_dump() {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, D3D11_BIND_RENDER_TARGET, D3D11_BIND_SHADER_RESOURCE,
D3D11_SDK_VERSION, D3D11_SUBRESOURCE_DATA, D3D11_USAGE_DEFAULT,
};
use windows::Win32::Graphics::Dxgi::Common::{DXGI_FORMAT_P010, DXGI_SAMPLE_DESC};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
// (Y, Cb, Cr) 10-bit limited codes for the 8 sRGB bars white/yellow/cyan/green/magenta/
// red/blue/black at 80-nit SDR white under PQ/BT.2020 — the same math as pf-capture's
// `p010_reference` (and the bars_pq2020 client fixture). Stored MSB-aligned (`<<6`).
const BARS: [(u16, u16, u16); 8] = [
(490, 512, 512),
(478, 423, 518),
(464, 525, 473),
(450, 432, 476),
(350, 584, 585),
(325, 448, 598),
(226, 650, 535),
(64, 512, 512),
];
const W: u32 = 1920;
const H: u32 = 1080;
init_tracing();
let Ok((_l, impls)) = intel_loader() else {
eprintln!("skipping: no VPL loader");
return;
};
let Some(imp) = impls.iter().find(|i| i.luid_valid) else {
eprintln!("skipping: no Intel VPL implementation on this box");
return;
};
if !probe_can_encode_10bit(Codec::H265) {
eprintln!("skipping: this GPU declines 10-bit HEVC");
return;
}
// P010 initial data: plane 0 = H rows of W u16 luma; plane 1 = H/2 rows of W u16
// (interleaved Cb,Cr pairs), same pitch. Bars are vertical: bar index = x / (W/8).
let bar_w = (W / 8) as usize;
let mut init = vec![0u16; (W as usize) * (H as usize + H as usize / 2)];
for y in 0..H as usize {
for x in 0..W as usize {
init[y * W as usize + x] = BARS[(x / bar_w).min(7)].0 << 6;
}
}
let chroma_base = (W as usize) * (H as usize);
for cy in 0..(H as usize / 2) {
for cx in 0..(W as usize / 2) {
let (_, cb, cr) = BARS[((cx * 2) / bar_w).min(7)];
init[chroma_base + cy * W as usize + cx * 2] = cb << 6;
init[chroma_base + cy * W as usize + cx * 2 + 1] = cr << 6;
}
}
// SAFETY: self-contained harness on one thread/device (same contract as `drive_live`);
// the initial-data pointer outlives the synchronous CreateTexture2D that reads it.
let (device, tex) = unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(imp.luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(imp.luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().expect("dxgi factory");
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).expect("intel adapter");
let mut device = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
windows::Win32::Foundation::HMODULE::default(),
Default::default(),
None,
D3D11_SDK_VERSION,
Some(&mut device),
None,
None,
)
.expect("d3d11 device on intel adapter");
let device: ID3D11Device = device.expect("device");
let desc = D3D11_TEXTURE2D_DESC {
Width: W,
Height: H,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
CPUAccessFlags: 0,
MiscFlags: 0,
};
let data = D3D11_SUBRESOURCE_DATA {
pSysMem: init.as_ptr() as *const std::ffi::c_void,
SysMemPitch: W * 2,
SysMemSlicePitch: 0,
};
let mut t: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, Some(&data), Some(&mut t))
.expect("bar texture");
(device.clone(), t.expect("texture"))
};
let mut enc = QsvEncoder::open(
Codec::H265,
PixelFormat::P010,
W,
H,
30,
10_000_000,
10,
ChromaFormat::Yuv420,
)
.expect("open");
enc.set_hdr_meta(Some(test_hdr_meta()));
let mut stream = Vec::new();
let mut aus = 0usize;
let mut keyframes = 0usize;
for i in 0..12u32 {
let frame = CapturedFrame {
width: W,
height: H,
pts_ns: i as u64 * 33_333_333,
format: PixelFormat::P010,
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
enc.submit_indexed(&frame, i).expect("submit");
if let Some(au) = enc.poll().expect("poll") {
aus += 1;
keyframes += au.keyframe as usize;
stream.extend_from_slice(&au.data);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("drain") {
aus += 1;
keyframes += au.keyframe as usize;
stream.extend_from_slice(&au.data);
}
assert!(aus >= 10, "expected ≥10 AUs, got {aus}");
assert!(keyframes >= 1, "expected an IDR in the dump");
let path = std::env::temp_dir().join("pf_qsv_1080_bars.h265");
std::fs::write(&path, &stream).expect("write dump");
println!(
"wrote {} AUs ({} bytes, {keyframes} keyframes) to {}",
aus,
stream.len(),
path.display()
);
}
/// The PRODUCTION host chain minus the IDD ring: the REAL `HdrP010Converter` renders the 8
/// sRGB bars into a ring-profile P010 texture (`BIND_RENDER_TARGET` only — RTV-written, not
/// CPU-uploaded) on the VPL implementation's own adapter, and THAT texture goes through the
/// unaligned-height ingest copy into a Main10 encode. Dumped to
/// `%TEMP%\pf_qsv_conv_1080_bars.h265`; expected decode codes = the bars_pq2020 fixture set
/// (see `hdr_p010_convert_bars_on_luid`).
#[test]
fn qsv_live_hdr_converter_e2e_1080_dump() {
const W: u32 = 1920;
const H: u32 = 1080;
init_tracing();
let Ok((_l, impls)) = intel_loader() else {
eprintln!("skipping: no VPL loader");
return;
};
let Some(imp) = impls.iter().find(|i| i.luid_valid) else {
eprintln!("skipping: no Intel VPL implementation on this box");
return;
};
if !probe_can_encode_10bit(Codec::H265) {
eprintln!("skipping: this GPU declines 10-bit HEVC");
return;
}
let (device, tex) = pf_capture::dxgi::hdr_p010_convert_bars_on_luid(imp.luid, W, H)
.expect("converter bars");
let mut enc = QsvEncoder::open(
Codec::H265,
PixelFormat::P010,
W,
H,
30,
10_000_000,
10,
ChromaFormat::Yuv420,
)
.expect("open");
enc.set_hdr_meta(Some(test_hdr_meta()));
let mut stream = Vec::new();
let mut aus = 0usize;
for i in 0..12u32 {
let frame = CapturedFrame {
width: W,
height: H,
pts_ns: i as u64 * 33_333_333,
format: PixelFormat::P010,
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
enc.submit_indexed(&frame, i).expect("submit");
if let Some(au) = enc.poll().expect("poll") {
aus += 1;
stream.extend_from_slice(&au.data);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("drain") {
aus += 1;
stream.extend_from_slice(&au.data);
}
assert!(aus >= 10, "expected ≥10 AUs, got {aus}");
let path = std::env::temp_dir().join("pf_qsv_conv_1080_bars.h265");
std::fs::write(&path, &stream).expect("write dump");
println!(
"wrote {aus} AUs ({} bytes) to {}",
stream.len(),
path.display()
);
}
}
+49
View File
@@ -330,6 +330,7 @@ fn open_video_backend(
{
match vulkan_video::VulkanVideoEncoder::open(
codec,
format,
width,
height,
fps,
@@ -344,12 +345,32 @@ fn open_video_backend(
);
return Ok((Box::new(e) as Box<dyn Encoder>, "vulkan"));
}
// Native NV12 (PUNKTFUNK_PIPEWIRE_NV12 capture) has no VAAPI fallback:
// libav's dmabuf lane would import the two-plane buffer as packed RGB
// (silent garbage) and its CPU lane bails per frame — die crisply instead.
Err(e) if format == PixelFormat::Nv12 => {
return Err(e.context(
"Vulkan Video open failed on a native-NV12 capture \
no VAAPI fallback exists; set PUNKTFUNK_PIPEWIRE_NV12=0 to \
restore the packed-RGB negotiation",
));
}
Err(e) => tracing::warn!(
error = %format!("{e:#}"),
"Vulkan Video encode open failed — falling back to libav VAAPI"
),
}
}
// Same rule when the Vulkan backend was never eligible (H264 session,
// PUNKTFUNK_VULKAN_ENCODE=0, or a build without the feature).
if format == PixelFormat::Nv12 {
anyhow::bail!(
"native NV12 capture requires the Vulkan Video encoder (HEVC/AV1 \
session, --features vulkan-encode, PUNKTFUNK_VULKAN_ENCODE not 0) this \
session resolved to libav VAAPI; set PUNKTFUNK_PIPEWIRE_NV12=0 to restore \
the packed-RGB negotiation"
);
}
vaapi::VaapiEncoder::open(
codec,
format,
@@ -390,6 +411,7 @@ fn open_video_backend(
}
vulkan_video::VulkanVideoEncoder::open(
codec,
format,
width,
height,
fps,
@@ -819,6 +841,33 @@ fn vulkan_encode_enabled() -> bool {
.unwrap_or(true)
}
/// Whether THIS session's encoder can ingest a producer-native NV12 capture: only the raw
/// Vulkan Video backend does (libav VAAPI would misread the two-plane buffer as packed RGB —
/// [`open_video`] refuses the combination), so the session's codec must be one it encodes and
/// the backend must be eligible to open. The host facade threads the verdict into the capture
/// negotiation (`OutputFormat::nv12_native` → `ZeroCopyPolicy::native_nv12_session`), which
/// then PREFERS gamescope's producer-side NV12 pod (default-on; `PUNKTFUNK_PIPEWIRE_NV12=0`
/// escapes at the capture gate).
#[cfg(target_os = "linux")]
pub fn linux_native_nv12_ok(codec: Codec) -> bool {
#[cfg(feature = "vulkan-encode")]
{
matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
// NVENC/PyroWave prefs never open the Vulkan Video backend; every other pref
// (auto/vaapi/amd/intel/vulkan) tries it first on AMD/Intel — see [`open_video`].
&& !matches!(
pf_host_config::config().encoder_pref.as_str(),
"nvenc" | "nvidia" | "cuda" | "pyrowave"
)
}
#[cfg(not(feature = "vulkan-encode"))]
{
let _ = codec;
false
}
}
/// Cheap, side-effect-free NVIDIA-presence probe for the `auto` backend selector: the NVIDIA
/// kernel driver exposes these device nodes, AMD/Intel boxes have neither. Deliberately does NOT
/// create a CUDA context (that would allocate GPU state on every host that merely *might* be
+31 -10
View File
@@ -105,13 +105,15 @@ pub fn drm_fourcc(format: PixelFormat) -> Option<u32> {
Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888
Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888
Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888
// Linux native NV12 capture (gamescope PipeWire): one LINEAR dmabuf with contiguous Y then
// interleaved UV, exposed under DRM_FORMAT_NV12.
Nv12 => drm_fourcc_code(b"NV12"),
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020).
X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010
X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010
// 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path.
// Rgb10a2/Nv12/P010 are the Windows HDR / video-processor formats — never produced on
// Linux; Yuv444 is OUR convert's OUTPUT, never a capture source format.
Rgb | Bgr | Rgb10a2 | Nv12 | P010 | Yuv444 => return None,
// Rgb10a2/P010 are Windows formats; Yuv444 is OUR convert output, never a capture source.
Rgb | Bgr | Rgb10a2 | P010 | Yuv444 => return None,
})
}
@@ -144,6 +146,12 @@ pub struct OutputFormat {
/// (never BGRA-passthrough / P010). `false` on every non-PyroWave session and on Linux (the
/// wavelet encoder ingests dmabufs / CPU RGB there, not a D3D11 texture).
pub pyrowave: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (Linux raw Vulkan Video
/// backend on an H265/AV1 session — see `pf_encode::linux_native_nv12_ok`). The Linux capture
/// negotiation only offers gamescope the NV12 pod when this is set: libav VAAPI (the H264
/// codec's backend, and the fallback family) would misread the two-plane buffer as packed
/// RGB. Always `false` on Windows (the IDD-push capturer owns its own formats).
pub nv12_native: bool,
}
impl OutputFormat {
@@ -161,6 +169,11 @@ impl OutputFormat {
chroma_444: false,
// GameStream never negotiates PyroWave (native punktfunk/1 only).
pyrowave: false,
// Conservative: the GameStream + spike paths don't resolve the codec here, and a
// Moonlight client may negotiate H264 (whose VAAPI backend can't ingest NV12) — so
// they never prefer the producer-native NV12 pod. The punktfunk/1 plane opts in via
// `SessionPlan::output_format()`, which knows the codec.
nv12_native: false,
}
}
}
@@ -201,18 +214,26 @@ pub struct CapturedFrame {
pub cursor: Option<CursorOverlay>,
}
/// A captured frame still living in a single-plane packed-RGB dmabuf (the VAAPI zero-copy path).
/// A captured frame still living in a DMA-BUF. Packed RGB uses one plane. Native Linux NV12
/// (gamescope PipeWire) travels in ONE fd: Y starts at `offset`, and the interleaved UV plane
/// lives at `plane1`'s offset/stride when the producer reported them — else at the contiguous
/// fallback `offset + stride * frame_height` with the shared `stride`.
///
/// Owns a *dup* of the PipeWire buffer's fd, so the frame can travel to the encode thread and be
/// imported into a VA surface there without the compositor's buffer being closed underneath it.
/// (Content stability across the brief import window relies on the compositor's buffer pool depth,
/// same as any zero-copy capture — the VAAPI importer copies into its own NV12 surface promptly.)
/// imported there without the compositor's buffer being closed underneath it. Content stability
/// across the brief import window relies on the compositor's buffer pool depth, like any zero-copy
/// capture.
#[cfg(target_os = "linux")]
pub struct DmabufFrame {
pub fd: std::os::fd::OwnedFd,
/// DRM FourCC of the packed-RGB plane (e.g. `XR24` for BGRx).
/// DRM FourCC (`XR24` for BGRx, `NV12` for native 4:2:0).
pub fourcc: u32,
/// DRM format modifier the compositor allocated (0 = LINEAR).
pub modifier: u64,
/// Second-plane `(offset, stride)` within the SAME fd, when the producer reported one (the
/// PipeWire buffer's plane-1 chunk — NV12's interleaved UV). `None` falls back to the
/// contiguous-plane contract above. Always `None` for single-plane packed RGB.
pub plane1: Option<(u32, u32)>,
pub offset: u32,
pub stride: u32,
}
@@ -225,8 +246,8 @@ pub enum FramePayload {
/// The dmabuf has already been imported + copied into this owned device buffer.
#[cfg(target_os = "linux")]
Cuda(pf_zerocopy::DeviceBuffer),
/// A raw packed-RGB dmabuf — the AMD/Intel (VAAPI) zero-copy path. The encoder imports it into
/// a VA surface and does RGB→NV12 on the GPU video engine (no host CSC, no upload).
/// A raw DMA-BUF: packed RGB for the existing GPU CSC paths, or native NV12 from a producer
/// such as gamescope. The encoder imports it without a host copy.
#[cfg(target_os = "linux")]
Dmabuf(DmabufFrame),
/// A GPU-resident D3D11 texture (Windows zero-copy path for NVENC). Owns the copied frame.
+17
View File
@@ -63,6 +63,13 @@ pub struct HostConfig {
/// deliver full chroma, and the GPU/driver passed the encode probe — otherwise 4:2:0.
/// `PUNKTFUNK_444=0`/`false`/`off`/`no` disables. Independent of `ten_bit` (chroma vs depth).
pub four_four_four: bool,
/// `PUNKTFUNK_CHACHA20` — host policy gate for the negotiated ChaCha20-Poly1305 session
/// cipher (design/chacha20-session-cipher.md). **Default ON** (pure rollout safety — perf-only,
/// both AEADs are full-strength): the host merely *allows* it — a session only seals with
/// ChaCha when the client advertised `VIDEO_CAP_CHACHA20` (set by soft-AES armv7 clients,
/// e.g. webOS TVs, whose GCM decrypt caps at ~100 Mbps); everyone else stays AES-128-GCM.
/// `PUNKTFUNK_CHACHA20=0`/`false`/`off`/`no` disables.
pub chacha20: bool,
/// `PUNKTFUNK_PERF` — per-stage timing instrumentation.
pub perf: bool,
/// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic).
@@ -147,6 +154,16 @@ impl HostConfig {
)
})
.unwrap_or(true),
// Default ON, explicit-off grammar (the client's VIDEO_CAP_CHACHA20 bit is the real
// per-session switch; see the field doc).
chacha20: val("PUNKTFUNK_CHACHA20")
.map(|s| {
!matches!(
s.trim().to_ascii_lowercase().as_str(),
"0" | "false" | "off" | "no"
)
})
.unwrap_or(true),
perf: flag("PUNKTFUNK_PERF"),
video_source: val("PUNKTFUNK_VIDEO_SOURCE"),
compositor: val("PUNKTFUNK_COMPOSITOR"),
+6
View File
@@ -36,6 +36,12 @@ pub fn vk_to_evdev(vk: u8) -> Option<u16> {
0x2D => Some(110), // VK_INSERT -> KEY_INSERT
0x2E => Some(111), // VK_DELETE -> KEY_DELETE
// --- Consumer/media keys (Android TV remotes, keyboard media rows) ---
0xB0 => Some(163), // VK_MEDIA_NEXT_TRACK -> KEY_NEXTSONG
0xB1 => Some(165), // VK_MEDIA_PREV_TRACK -> KEY_PREVIOUSSONG
0xB2 => Some(166), // VK_MEDIA_STOP -> KEY_STOPCD
0xB3 => Some(164), // VK_MEDIA_PLAY_PAUSE -> KEY_PLAYPAUSE
// --- Generic modifiers ---
0x10 => Some(42), // VK_SHIFT -> KEY_LEFTSHIFT
0x11 => Some(29), // VK_CONTROL -> KEY_LEFTCTRL
@@ -424,6 +424,9 @@ impl InputInjector for KwinFakeInjector {
self.fake.touch_up(event.code);
self.fake.touch_frame();
}
// fake_input can only press host-layout keycodes — no committed-text path (the
// HOST_CAP_TEXT_INPUT cap is not advertised on this backend).
InputKind::TextInput => {}
// Gamepads are injected through uinput, not the compositor.
InputKind::GamepadState
| InputKind::GamepadButton
+169 -23
View File
@@ -109,7 +109,7 @@ async fn session_main(mut rx: UnboundedReceiver<InputEvent>, source: EiSource) {
// Keep `_rd`/`_session` bound for the whole loop — dropping the portal session closes the
// EIS connection. Bound the setup so a headless approval dialog (un-bypassed grant) can't
// hang the worker forever.
let (_keepalive, context, mut events) = match tokio::time::timeout(
let (_keepalive, context, mut events, output_hint) = match tokio::time::timeout(
Duration::from_secs(30),
connect(source),
)
@@ -130,6 +130,7 @@ async fn session_main(mut rx: UnboundedReceiver<InputEvent>, source: EiSource) {
tracing::info!("libei: EIS connected — awaiting devices");
let mut state = EiState::new();
state.output_hint = output_hint;
// Watchdog: a healthy EIS server adds + resumes an input device within a beat of the handshake.
// If none has resumed by this deadline, the connection is dead-on-arrival (stale/half-ready
// gamescope socket the handshake passed but no real server is behind) — exit so the next
@@ -177,20 +178,28 @@ type Connected = (
Box<dyn Send>,
ei::Context,
reis::tokio::EiConvertEventStream,
// The compositor's output size ("WxH" relay-file hint) — the scale target for absolute
// coordinates when the EIS advertises only a degenerate region (gamescope). `None` for
// the portal/Mutter paths, whose regions are real.
Option<(u32, u32)>,
);
/// Reach an EIS server per `source` and run the EI sender handshake.
async fn connect(source: EiSource) -> Result<Connected> {
let (keepalive, stream): (Box<dyn Send>, UnixStream) = match source {
let (keepalive, stream, output_hint): (Box<dyn Send>, UnixStream, Option<(u32, u32)>) =
match source {
EiSource::Portal => {
let (rd, session, fd) = connect_portal().await?;
(Box::new((rd, session)), UnixStream::from(fd))
(Box::new((rd, session)), UnixStream::from(fd), None)
}
EiSource::MutterEis => {
let (keepalive, fd) = connect_mutter().await?;
(keepalive, UnixStream::from(fd))
(keepalive, UnixStream::from(fd), None)
}
EiSource::SocketPathFile(file) => {
let (stream, hint) = connect_socket_file(&file).await?;
(Box::new(()), stream, hint)
}
EiSource::SocketPathFile(file) => (Box::new(()), connect_socket_file(&file).await?),
};
let context = ei::Context::new(stream).map_err(|e| anyhow!("reis EI context: {e}"))?;
// Bound the handshake. `UnixStream::connect` to a socket *file* succeeds the moment the path
@@ -206,7 +215,7 @@ async fn connect(source: EiSource) -> Result<Connected> {
anyhow!("EI handshake timed out (EIS server not responding — stale/half-ready socket?)")
})?
.map_err(|e| anyhow!("EI handshake: {e}"))?;
Ok((keepalive, context, events))
Ok((keepalive, context, events, output_hint))
}
/// Open a RemoteDesktop portal session (pointer + keyboard) and obtain the EIS socket fd.
@@ -294,8 +303,10 @@ async fn connect_mutter() -> Result<(Box<dyn Send>, std::os::fd::OwnedFd)> {
/// Poll `file` for the EIS socket path (the gamescope backend relays `LIBEI_SOCKET` there once
/// the nested app launches), then connect. A bare name is resolved against `XDG_RUNTIME_DIR`,
/// mirroring libei's own `LIBEI_SOCKET` semantics.
async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
/// mirroring libei's own `LIBEI_SOCKET` semantics. Line 2 of the relay file, when present,
/// carries the compositor's output size as `WxH` — returned as the absolute-coordinate scale
/// hint (gamescope's EIS region is degenerate, so the geometry can't come from the protocol).
async fn connect_socket_file(file: &std::path::Path) -> Result<(UnixStream, Option<(u32, u32)>)> {
// The relay file is rewritten each session with the CURRENT gamescope's `LIBEI_SOCKET`, and the
// socket may not be `listen()`ing the instant its name appears — or the file may briefly still
// hold a prior, now-dead session's name (the host-lifetime injector reconnecting between
@@ -319,7 +330,12 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
));
}
if let Ok(s) = std::fs::read_to_string(file) {
let name = s.trim();
let mut file_lines = s.lines();
let name = file_lines.next().unwrap_or("").trim();
let hint = file_lines.next().and_then(|l| {
let (w, h) = l.trim().split_once('x')?;
Some((w.parse::<u32>().ok()?, h.parse::<u32>().ok()?))
});
if !name.is_empty() {
let full = if name.starts_with('/') {
std::path::PathBuf::from(name)
@@ -334,7 +350,7 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
logged = name.to_string();
}
match UnixStream::connect(&full) {
Ok(stream) => return Ok(stream),
Ok(stream) => return Ok((stream, hint)),
// Refused = socket file exists but no listener yet (or a dead session);
// NotFound = path not created yet. Both heal once the live gamescope's EIS is
// up — retry. Anything else (e.g. permission) is a real failure.
@@ -386,6 +402,22 @@ struct EiState {
held_keys: Vec<u32>,
held_buttons: Vec<u32>,
held_touches: Vec<u32>,
/// The touch id currently degraded to the absolute pointer ([`EiState::degrade_touch`]) —
/// the "primary finger" while the EIS has no touchscreen device. `None` between touches.
degraded_touch: Option<u32>,
/// The compositor's output size (relay-file "WxH" hint) — scale target for absolute
/// coordinates when the device's region is degenerate/absent (gamescope). Without it the
/// fallback is raw client pixels, correct only when the stream runs at the output's size.
output_hint: Option<(u32, u32)>,
}
/// Is this EIS region a plausible OUTPUT geometry — something to map normalized coordinates
/// into? gamescope advertises a degenerate `(0,0,INT32_MAX,INT32_MAX)` "everything" region on
/// its virtual input device, meaning "absolute coordinates are raw"; normalizing into it
/// explodes a center tap to x≈1e9, which the compositor clamps to the far corner. 16384
/// comfortably covers real multi-monitor layouts while rejecting the sentinel.
fn sane_region(r: &reis::event::Region) -> bool {
r.width > 0 && r.height > 0 && r.width <= 16_384 && r.height <= 16_384
}
/// Stable small index per [`InputKind`] for the `seen_kinds` bitmask.
@@ -406,6 +438,7 @@ fn kind_bit(kind: InputKind) -> u32 {
InputKind::GamepadState => 12,
InputKind::GamepadRemove => 13,
InputKind::GamepadArrival => 14,
InputKind::TextInput => 15,
};
1 << i
}
@@ -422,6 +455,8 @@ impl EiState {
held_keys: Vec::new(),
held_buttons: Vec::new(),
held_touches: Vec::new(),
degraded_touch: None,
output_hint: None,
}
}
@@ -430,6 +465,10 @@ impl EiState {
/// normal [`EiState::inject`] path so the compositor sees proper key-up / button-up /
/// touch-up frames before the devices disappear.
fn release_all(&mut self, ctx: &ei::Context) {
// A degraded touch is held as a synthesized left button (in `held_buttons`), so the
// loop below releases it — but the primary-finger latch must clear too, or the NEXT
// session's first TouchDown reads as a second finger and is ignored.
self.degraded_touch = None;
let (keys, buttons, touches) = (
std::mem::take(&mut self.held_keys),
std::mem::take(&mut self.held_buttons),
@@ -506,6 +545,8 @@ impl EiState {
keyboard = dev.has_capability(DeviceCapability::Keyboard),
button = dev.has_capability(DeviceCapability::Button),
scroll = dev.has_capability(DeviceCapability::Scroll),
regions = dev.regions().len(),
region0 = ?dev.regions().first().map(|r| (r.x, r.y, r.width, r.height)),
"libei: device RESUMED (now emittable)"
);
}
@@ -537,8 +578,85 @@ impl EiState {
}
}
/// Degrade touch to a single-finger ABSOLUTE POINTER on compositors whose EIS never
/// creates a touchscreen device (gamescope's "Gamescope Virtual Input" advertises
/// pointer/pointer_abs/button but no touch — observed live; headless KWin the same).
/// Down = abs-move + left press, move = abs-move, up = left release — synthesized through
/// the normal [`EiState::inject`] Mouse* paths so region mapping, held-state tracking and
/// [`EiState::release_all`] all apply. Only the FIRST finger drives the pointer; later
/// fingers are ignored (a pointer has no second contact — a pinch degrades to a drag).
fn degrade_touch(&mut self, ev: &InputEvent, ctx: &ei::Context) {
const GS_BUTTON_LEFT: u32 = 1;
match ev.kind {
InputKind::TouchDown => {
if self.degraded_touch.is_some() {
return; // secondary finger — single-pointer degradation
}
self.degraded_touch = Some(ev.code);
static NOTED: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(false);
if !NOTED.swap(true, std::sync::atomic::Ordering::Relaxed) {
tracing::info!(
"compositor's EIS has no touchscreen device — degrading touch to a \
single-finger absolute pointer (tap = left click; multi-touch \
gestures unavailable)"
);
}
self.inject(
&InputEvent {
kind: InputKind::MouseMoveAbs,
..*ev
},
ctx,
);
self.inject(
&InputEvent {
kind: InputKind::MouseButtonDown,
code: GS_BUTTON_LEFT,
..*ev
},
ctx,
);
}
InputKind::TouchMove if self.degraded_touch == Some(ev.code) => {
self.inject(
&InputEvent {
kind: InputKind::MouseMoveAbs,
..*ev
},
ctx,
);
}
InputKind::TouchUp if self.degraded_touch == Some(ev.code) => {
self.degraded_touch = None;
self.inject(
&InputEvent {
kind: InputKind::MouseButtonUp,
code: GS_BUTTON_LEFT,
..*ev
},
ctx,
);
}
_ => {}
}
}
/// Translate and emit one client input event, committing it as a single `frame`.
fn inject(&mut self, ev: &InputEvent, ctx: &ei::Context) {
// No ei_touchscreen device but an absolute pointer exists → degrade rather than drop
// (the game-mode "touch simply does not work" trap). Checked per event, not latched:
// a touchscreen device appearing later (compositor restart, capability change) takes
// over seamlessly on the next touch.
if matches!(
ev.kind,
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp
) && self.device_for(DeviceCapability::Touch).is_none()
&& self.device_for(DeviceCapability::PointerAbsolute).is_some()
{
self.degrade_touch(ev, ctx);
return;
}
let cap = match ev.kind {
InputKind::MouseMove => DeviceCapability::Pointer,
InputKind::MouseMoveAbs => DeviceCapability::PointerAbsolute,
@@ -553,6 +671,9 @@ impl EiState {
| InputKind::GamepadAxis
| InputKind::GamepadRemove
| InputKind::GamepadArrival => return, // uinput path (later)
// libei presses keycodes against the server's negotiated keymap — no committed-text
// path (the HOST_CAP_TEXT_INPUT cap is not advertised on this backend).
InputKind::TextInput => return,
};
self.injected += 1;
let n = self.injected;
@@ -605,16 +726,31 @@ impl EiState {
InputKind::MouseMoveAbs => {
let w = ((ev.flags >> 16) & 0xffff) as f32;
let h = (ev.flags & 0xffff) as f32;
match (
slot.interface::<ei::PointerAbsolute>(),
slot.regions().first(),
) {
(Some(p), Some(region)) if w > 0.0 && h > 0.0 => {
// Map the normalized client position into the device's first region.
match slot.interface::<ei::PointerAbsolute>() {
Some(p) if w > 0.0 && h > 0.0 => {
// Map the normalized client position into the device's first region —
// but only when the region looks like a real output geometry.
// gamescope's "Gamescope Virtual Input" advertises a degenerate
// (0,0,INT32_MAX,INT32_MAX) region meaning "coordinates are raw":
// normalizing into it explodes a center tap to x≈1e9, which gamescope
// clamps to the far corner (the observed cursor-parked-at-1279,799).
// There the managed session runs at the client's mode, so client
// pixels ARE output pixels: emit them raw.
let nx = (ev.x as f32 / w).clamp(0.0, 1.0);
let ny = (ev.y as f32 / h).clamp(0.0, 1.0);
let x = region.x as f32 + nx * region.width as f32;
let y = region.y as f32 + ny * region.height as f32;
let (x, y) = match slot.regions().first().filter(|r| sane_region(r)) {
Some(region) => (
region.x as f32 + nx * region.width as f32,
region.y as f32 + ny * region.height as f32,
),
// Degenerate/absent region: scale into the relay-file output hint
// (correct even when the client streams at a different resolution
// than the session runs); raw client pixels as the last resort.
None => match self.output_hint {
Some((ow, oh)) => (nx * ow as f32, ny * oh as f32),
None => (ev.x as f32, ev.y as f32),
},
};
p.motion_absolute(x, y);
}
_ => emitted = false,
@@ -680,12 +816,21 @@ impl EiState {
InputKind::TouchDown | InputKind::TouchMove => {
let w = ((ev.flags >> 16) & 0xffff) as f32;
let h = (ev.flags & 0xffff) as f32;
match (slot.interface::<ei::Touchscreen>(), slot.regions().first()) {
(Some(t), Some(region)) if w > 0.0 && h > 0.0 => {
match slot.interface::<ei::Touchscreen>() {
Some(t) if w > 0.0 && h > 0.0 => {
let nx = (ev.x as f32 / w).clamp(0.0, 1.0);
let ny = (ev.y as f32 / h).clamp(0.0, 1.0);
let x = region.x as f32 + nx * region.width as f32;
let y = region.y as f32 + ny * region.height as f32;
// Same degenerate-region fallback ladder as MouseMoveAbs.
let (x, y) = match slot.regions().first().filter(|r| sane_region(r)) {
Some(region) => (
region.x as f32 + nx * region.width as f32,
region.y as f32 + ny * region.height as f32,
),
None => match self.output_hint {
Some((ow, oh)) => (nx * ow as f32, ny * oh as f32),
None => (ev.x as f32, ev.y as f32),
},
};
if ev.kind == InputKind::TouchDown {
t.down(ev.code, x, y);
} else {
@@ -703,7 +848,8 @@ impl EiState {
| InputKind::GamepadButton
| InputKind::GamepadAxis
| InputKind::GamepadRemove
| InputKind::GamepadArrival => emitted = false,
| InputKind::GamepadArrival
| InputKind::TextInput => emitted = false,
}
if emitted {
+96
View File
@@ -98,9 +98,26 @@ pub struct WlrootsInjector {
keyboard: ZwpVirtualKeyboardV1,
xkb_state: xkb::State,
_keymap_file: std::fs::File, // keep the memfd alive for the compositor's mmap
/// Dedicated committed-text device ([`InputKind::TextInput`]), created on first use.
text: Option<TextKeyboard>,
start: Instant,
}
/// Cap on distinct characters the dynamic text keymap holds before it restarts from scratch
/// (keycodes grow upward from 9; xkb tops out at 255, so stay well under).
const TEXT_KEYMAP_MAX: usize = 200;
/// The dedicated **text** virtual keyboard: types committed IME text (`InputKind::TextInput`,
/// one Unicode scalar per event) by growing a keymap of Unicode keysyms on demand and pressing
/// the character's keycode — the `wtype` model. A separate `zwp_virtual_keyboard` so keymap
/// re-uploads never disturb the main device's layout/modifier state that VK key events ride on.
struct TextKeyboard {
keyboard: ZwpVirtualKeyboardV1,
/// Characters in keycode order: `chars[i]` types on wire keycode `i + 1` (xkb `i + 9`).
chars: Vec<char>,
_keymap_file: Option<std::fs::File>, // keep the memfd alive for the compositor's mmap
}
impl WlrootsInjector {
pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env()
@@ -171,6 +188,7 @@ impl WlrootsInjector {
keyboard,
xkb_state,
_keymap_file: file,
text: None,
start: Instant::now(),
})
}
@@ -179,6 +197,54 @@ impl WlrootsInjector {
self.start.elapsed().as_millis() as u32
}
/// Type one committed-text Unicode scalar on the dedicated text device (created lazily),
/// growing its keymap when the character is new. Control characters are dropped — Enter,
/// Backspace and Tab ride the VK key-event path.
fn type_text(&mut self, cp: u32) -> Result<()> {
let Some(ch) = char::from_u32(cp) else {
return Ok(()); // lone surrogate / out of range
};
if ch.is_control() {
return Ok(());
}
if self.text.is_none() {
let (Some(mgr), Some(seat)) =
(self.globals.keyboard_mgr.clone(), self.globals.seat.clone())
else {
return Ok(());
};
let kb = mgr.create_virtual_keyboard(&seat, &self.queue.handle(), ());
self.text = Some(TextKeyboard {
keyboard: kb,
chars: Vec::new(),
_keymap_file: None,
});
}
let t = self.now_ms();
let text = self.text.as_mut().expect("created above");
let code = match text.chars.iter().position(|&c| c == ch) {
Some(i) => (i + 1) as u32,
None => {
if text.chars.len() >= TEXT_KEYMAP_MAX {
text.chars.clear(); // restart the map; old codes are re-assigned lazily
}
text.chars.push(ch);
let keymap_str = text_keymap(&text.chars);
let file = memfd_with(&keymap_str)?;
text.keyboard.keymap(
1, /* XKB_V1 */
file.as_fd(),
keymap_str.len() as u32 + 1,
);
text._keymap_file = Some(file);
text.chars.len() as u32
}
};
text.keyboard.key(t, code, 1);
text.keyboard.key(t, code, 0);
Ok(())
}
/// Update xkb state for a key and tell the compositor the resulting modifier mask.
fn send_modifiers(&mut self, evdev: u16, down: bool) {
let kc = xkb::Keycode::new(evdev as u32 + 8); // evdev -> xkb keycode
@@ -254,6 +320,9 @@ impl InputInjector for WlrootsInjector {
tracing::debug!(vk = event.code, "unmapped VK keycode — dropped");
}
}
InputKind::TextInput => {
self.type_text(event.code)?;
}
InputKind::GamepadState
| InputKind::GamepadButton
| InputKind::GamepadAxis
@@ -271,6 +340,33 @@ impl InputInjector for WlrootsInjector {
}
}
/// Build a minimal xkb keymap whose keycode `i + 9` (wire code `i + 1`) types `chars[i]`, using
/// Unicode keysym names (`U<hex>` — xkbcommon resolves them for any scalar, emoji included).
/// Types/compat `include "complete"` mirrors `wtype`'s generated keymap — proven on wlroots
/// compositors, and the system XKB data is present (the main keymap compiled from it in `open`).
fn text_keymap(chars: &[char]) -> String {
use std::fmt::Write as _;
let mut keycodes = String::new();
let mut symbols = String::new();
for (i, ch) in chars.iter().enumerate() {
let _ = writeln!(keycodes, " <T{i}> = {};", i + 9);
let _ = writeln!(symbols, " key <T{i}> {{ [ U{:04X} ] }};", *ch as u32);
}
format!(
"xkb_keymap {{\n\
xkb_keycodes \"punktfunk-text\" {{\n\
minimum = 8;\n\
maximum = {};\n\
{keycodes}\
}};\n\
xkb_types \"punktfunk-text\" {{ include \"complete\" }};\n\
xkb_compatibility \"punktfunk-text\" {{ include \"complete\" }};\n\
xkb_symbols \"punktfunk-text\" {{\n{symbols} }};\n\
}};\n",
chars.len() + 9,
)
}
/// Create an anonymous in-memory file holding `s` + a trailing NUL (for the keymap fd).
fn memfd_with(s: &str) -> Result<std::fs::File> {
let name = b"punktfunk-keymap\0";
@@ -27,10 +27,10 @@ use windows::Win32::System::StationsAndDesktops::{
use windows::Win32::UI::Input::KeyboardAndMouse::{
GetKeyboardLayout, MapVirtualKeyExW, SendInput, HKL, INPUT, INPUT_0, INPUT_KEYBOARD,
INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY, KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE,
MAPVK_VK_TO_VSC_EX, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL, MOUSEEVENTF_LEFTDOWN,
MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP, MOUSEEVENTF_MOVE,
MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK, MOUSEEVENTF_WHEEL,
MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY,
KEYEVENTF_UNICODE, MAPVK_VK_TO_VSC_EX, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL,
MOUSEEVENTF_LEFTDOWN, MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP,
MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK,
MOUSEEVENTF_WHEEL, MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY,
};
use windows::Win32::UI::WindowsAndMessaging::{
GetForegroundWindow, GetSystemMetrics, GetWindowThreadProcessId, SM_CXVIRTUALSCREEN,
@@ -297,6 +297,33 @@ impl InputInjector for SendInputInjector {
};
self.send(&[key(ki)])
}
InputKind::TextInput => {
// Committed IME text: one Unicode scalar per event, injected as
// `KEYEVENTF_UNICODE` packets (wScan = UTF-16 unit, no scancode/layout involved
// — the receiving app gets the character verbatim via WM_CHAR). An astral-plane
// scalar (emoji) is its surrogate pair, each unit down+up in order — exactly how
// Windows expects supplementary characters from unicode injection.
let Some(ch) = char::from_u32(event.code) else {
return Ok(()); // lone surrogate / out of range — drop
};
if ch.is_control() {
return Ok(()); // control chars ride the VK path (Enter/Backspace/Tab)
}
let mut units = [0u16; 2];
let mut inputs: Vec<INPUT> = Vec::with_capacity(4);
for &unit in ch.encode_utf16(&mut units).iter() {
for flags in [KEYEVENTF_UNICODE, KEYEVENTF_UNICODE | KEYEVENTF_KEYUP] {
inputs.push(key(KEYBDINPUT {
wVk: VIRTUAL_KEY(0),
wScan: unit,
dwFlags: flags,
time: 0,
dwExtraInfo: 0,
}));
}
}
self.send(&inputs)
}
// Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op.
InputKind::GamepadButton
| InputKind::GamepadAxis
+23
View File
@@ -174,6 +174,29 @@ pub fn default_backend() -> Backend {
Backend::Unsupported
}
/// Whether the session's inject backend can type **committed text**
/// ([`InputKind::TextInput`] — see `HOST_CAP_TEXT_INPUT`): Windows always (`KEYEVENTF_UNICODE`);
/// Linux only on the wlroots backend (a dedicated virtual keyboard with a dynamically-grown
/// Unicode keymap) — KWin fake-input/libei/gamescope can only press keycodes of the host layout.
/// Consulted at Welcome time to advertise the cap; a mid-session backend switch away from a
/// capable one just degrades to dropped text events (input is lossy by design).
#[cfg(target_os = "windows")]
pub fn text_input_supported() -> bool {
true
}
/// See the Windows variant: Linux types text only through the wlroots virtual-keyboard backend.
#[cfg(target_os = "linux")]
pub fn text_input_supported() -> bool {
matches!(default_backend(), Backend::WlrVirtual)
}
/// No injector ⇒ no text.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn text_input_supported() -> bool {
false
}
#[path = "inject/service.rs"]
mod service;
pub use service::InjectorService;
+5
View File
@@ -333,6 +333,11 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// bottom-right corner (the reported bug). The menu/library is keyboard+gamepad-driven
// and consumes no mouse, so nothing wanted these synthetic events anyway.
sdl3::hint::set("SDL_TOUCH_MOUSE_EVENTS", "0");
// The Wayland `app_id` (and X11 WM_CLASS) — compositors match it against
// io.unom.Punktfunk.desktop for the window/taskbar icon. Without it SDL uses a generic
// identity and the session window gets the default-Wayland icon (the Linux analog of
// the AppUserModelID adoption above).
sdl3::hint::set("SDL_APP_ID", "io.unom.Punktfunk");
let sdl = sdl3::init().context("SDL init")?;
let video = sdl.video().context("SDL video")?;
let events = sdl.event().context("SDL events")?;
+9 -1
View File
@@ -30,9 +30,17 @@ impl Presenter {
// switch modes before anything touches this frame. Only where the surface
// offers HDR10 — otherwise PQ stays on the SDR swapchain and the CSC shader
// tonemaps (mode 1).
//
// CPU frames NEVER take the HDR10 surface: software decode uploads swscale RGBA with
// no CSC/tonemap pass, so on a mode-0 swapchain that sRGB-encoded content would be
// composed as PQ — the field-reported psychedelic cyan/magenta picture (reproduced
// 2026-07-21: Fedora-class client, no hw HEVC decode, GNOME/Mesa offering HDR10 even
// on an SDR desktop). On the SDR swapchain the same frames are merely untonemapped
// (washed out) — wrong in the known, benign way until the CPU lane grows a real
// PQ→sRGB pass.
let frame_pq = match &input {
FrameInput::Redraw => None,
FrameInput::Cpu(f) => Some(f.color.is_pq()),
FrameInput::Cpu(_) => Some(false),
#[cfg(target_os = "linux")]
FrameInput::Dmabuf(d) => Some(d.color.is_pq()),
FrameInput::VkFrame(v) => Some(v.color.is_pq()),
+9
View File
@@ -617,6 +617,15 @@ pub(super) fn pick_formats(
surface: vk::SurfaceKHR,
colorspace_ext: bool,
) -> Result<(vk::SurfaceFormatKHR, Option<vk::SurfaceFormatKHR>)> {
// `PUNKTFUNK_HDR10=0` (explicit-off grammar) refuses the HDR10/ST.2084 swapchain outright,
// pinning PQ streams to the shader tonemap on an SDR surface. Two reasons this exists:
// desktop compositors newly offer HDR10 even on SDR desktops (GNOME 48 / Plasma 6 with
// Mesa ≥ 25.1 — a lane that otherwise engages silently), and it is the A/B lever that
// splits "HDR10 passthrough composes wrong" from "the decoded planes are wrong" in the
// field without rebuilding anything.
let colorspace_ext = colorspace_ext
&& !std::env::var("PUNKTFUNK_HDR10")
.is_ok_and(|v| matches!(v.as_str(), "0" | "false" | "off" | "no"));
let formats = unsafe { surface_i.get_physical_device_surface_formats(pdev, surface) }?;
let mut sdr = None;
for want in [vk::Format::B8G8R8A8_UNORM, vk::Format::R8G8B8A8_UNORM] {
+30 -2
View File
@@ -72,8 +72,8 @@ pub use session::{session_epoch, try_recover_session};
#[path = "vdisplay/routing.rs"]
pub(crate) mod routing;
pub use routing::{
apply_input_env, restore_managed_session, restore_takeover_on_startup, start_restore_worker,
wants_dedicated_game_session,
apply_input_env, managed_session_available, restore_managed_session,
restore_takeover_on_startup, start_restore_worker, wants_dedicated_game_session,
};
#[cfg(target_os = "linux")]
pub use routing::{
@@ -254,6 +254,34 @@ pub fn detect() -> Result<Compositor> {
}
}
/// Attach-only probes: while any scope is held, backend `create` paths must not stop, relaunch,
/// or take over box sessions — they may only attach to an already-live output, and fail fast
/// otherwise. The capture-loss rebuild holds one for its first seconds: right after a capture
/// loss the active-session detection can be STALE (a Game→Desktop switch observed live: the
/// probe's gamescope re-acquire restarted `gamescope-session.target` and yanked the user out of
/// the KDE session they had just switched to). A counter, so overlapping scopes compose.
static REBUILD_PROBES: std::sync::atomic::AtomicU32 = std::sync::atomic::AtomicU32::new(0);
/// RAII scope marking pipeline builds as attach-only probes (see [`rebuild_probe_active`]).
pub struct RebuildProbeScope(());
pub fn rebuild_probe_scope() -> RebuildProbeScope {
REBUILD_PROBES.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
RebuildProbeScope(())
}
impl Drop for RebuildProbeScope {
fn drop(&mut self) {
REBUILD_PROBES.fetch_sub(1, std::sync::atomic::Ordering::SeqCst);
}
}
/// Is any [`rebuild_probe_scope`] active? Destructive session operations (stop/relaunch/
/// takeover-restart) must be skipped while true.
pub fn rebuild_probe_active() -> bool {
REBUILD_PROBES.load(std::sync::atomic::Ordering::SeqCst) > 0
}
/// Open the virtual-display driver for `compositor`.
pub fn open(compositor: Compositor) -> Result<Box<dyn VirtualDisplay>> {
#[cfg(target_os = "linux")]
@@ -325,6 +325,29 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
if steamos_session_present() {
return create_managed_session_steamos(mode);
}
// Attach-only rebuild probe: reuse a live same-mode session, but NEVER stop/relaunch box
// sessions — right after a capture loss the caller's session detection can be stale, and a
// destructive rebuild here would fight the session the user just switched to.
if crate::rebuild_probe_active() {
let guard = MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner());
let same_mode = guard.as_ref().is_some_and(|s| {
s.width == mode.width && s.height == mode.height && s.refresh_hz == mode.refresh_hz
});
if same_mode {
if let Some(node_id) = find_gamescope_node() {
point_injector_at_eis();
tracing::info!(
node_id,
"gamescope session: attach-only probe reusing live node"
);
return Ok(managed_output(node_id, mode));
}
}
return Err(anyhow!(
"gamescope session has no attachable live node — attach-only rebuild probe refuses \
to stop/relaunch box sessions (re-detection follows the live session)"
));
}
// Steam is single-instance: if the box autologged into gaming mode on a physical display (the
// Bazzite default — `gamescope-session-plus@ogui-steam` on the TV), that session holds Steam and
// renders to the TV's native mode, which we'd capture instead of the client's. Free Steam by
@@ -607,12 +630,17 @@ fn write_steamos_dropin(shim_dir: &std::path::Path, mode: Mode) -> Result<()> {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).with_context(|| format!("mkdir {}", parent.display()))?;
}
// UnsetEnvironment: the same headless-must-not-attach armor `launch_session` gives its
// transient unit — the manager env can carry a stale desktop DISPLAY/WAYLAND_DISPLAY (from a
// portal settle), and gamescope would abort trying to attach to it instead of becoming the
// display server. Unit-scoped belt-and-suspenders on top of the observe_session_instance scrub.
let body = format!(
"[Service]\n\
Environment=PATH={shim}:/usr/bin:/bin:/usr/local/bin\n\
Environment=PF_W={w}\n\
Environment=PF_H={h}\n\
Environment=PF_HZ={hz}\n",
Environment=PF_HZ={hz}\n\
UnsetEnvironment=DISPLAY WAYLAND_DISPLAY\n",
shim = shim_dir.display(),
w = mode.width,
h = mode.height,
@@ -650,6 +678,16 @@ fn create_managed_session_steamos(mode: Mode) -> Result<VirtualOutput> {
}
*guard = None; // tracked session lost its node — fall through to a clean restart
}
// Attach-only rebuild probe: the reuse path above may attach, but a restart of the session
// target is out of bounds — observed live on a Deck: a stale post-capture-loss detection made
// this restart steal the seat back from the KDE session the user had just switched to.
if crate::rebuild_probe_active() {
return Err(anyhow!(
"gamescope has no live node and this is an attach-only rebuild probe — refusing to \
restart {STEAMOS_SESSION_TARGET} (the box may be mid-switch to another session; \
re-detection follows it)"
));
}
let shim_dir = write_headless_shim()?;
write_steamos_dropin(&shim_dir, mode)?;
systemctl_user(&["daemon-reload"]);
@@ -1078,6 +1116,26 @@ pub fn schedule_restore_tv_session() {
}
}
/// Does any DRM connector report a physically `connected` display? Scans
/// `/sys/class/drm/*/status` — only connector nodes (`card0-eDP-1`, `card0-HDMI-A-1`, …) have a
/// `status` file, so the bare `cardN` device dirs and `renderD*` nodes filter themselves out. A
/// headless box (VM, panel-less mini PC) has none — in which case a "restore to the physical
/// panel" can only fail, gamescope having no output to drive. Errors (no DRM at all, sysfs
/// unreadable) read as headless: the safe direction is keeping the working session.
fn physical_display_connected() -> bool {
connected_connector_under(std::path::Path::new("/sys/class/drm"))
}
/// [`physical_display_connected`] against an arbitrary sysfs root (the unit-testable core).
fn connected_connector_under(base: &std::path::Path) -> bool {
let Ok(entries) = std::fs::read_dir(base) else {
return false;
};
entries.flatten().any(|e| {
std::fs::read_to_string(e.path().join("status")).is_ok_and(|s| s.trim() == "connected")
})
}
/// Tear down our host-managed session (freeing Steam) and restart the autologin gaming session(s)
/// we stopped on connect — so the TV returns to gaming mode when no one is streaming. Invoked by
/// [`start_restore_worker`] once the debounce deadline passes; takes the stopped-unit list so a
@@ -1089,6 +1147,19 @@ fn do_restore_tv_session() {
{
let mut took = STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner());
if *took {
// A box with no physically connected display (a VM, a panel-less mini PC) has no
// "physical gaming session" to restore TO: removing the drop-in and restarting the
// target just crash-loops gamescope (no output to drive) and strands every later
// connect on "no usable compositor". Keep the headless session — and the takeover
// state, so a same-mode reconnect reuses it warm — instead. Checked at restore time
// (not connect time) so plugging a panel in later restores normally.
if !physical_display_connected() {
tracing::info!(
"gamescope (SteamOS): no physical display connected — keeping the headless \
session (nothing to restore to)"
);
return;
}
*took = false;
clear_takeover(); // A3: takeover undone — drop the persisted crash-restore marker
*MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None;
@@ -1188,15 +1259,31 @@ pub fn start_restore_worker() -> std::sync::Arc<()> {
/// session). Shared by the attach and host-managed-session paths.
fn point_injector_at_eis() {
match find_gamescope_eis_socket() {
Some(sock) => match std::fs::write(ei_socket_file(), &sock) {
Some(sock) => {
// Relay format: line 1 = socket, optional line 2 = the session's CURRENT output
// size as "WxH". gamescope's EIS advertises only a degenerate INT32_MAX region, so
// the injector can't learn the output geometry from the protocol — the hint lets
// it scale normalized client positions correctly even when the client streams at
// a different resolution than the session runs (foreign attach, supersample).
let size = current_gamescope_output_size();
let body = match size {
Some((w, h)) => format!("{sock}\n{w}x{h}"),
None => sock.clone(),
};
match std::fs::write(ei_socket_file(), body) {
Ok(()) => {
tracing::info!(socket = %sock, "gamescope: pointed injector at the session's EIS socket")
tracing::info!(
socket = %sock,
output = ?size,
"gamescope: pointed injector at the session's EIS socket"
)
}
Err(e) => tracing::warn!(
error = %e,
"gamescope: could not write the EIS relay file — input may not reach the session"
),
},
}
}
None => tracing::warn!(
"gamescope: no connectable gamescope EIS socket found — input won't reach the session"
),
@@ -1486,7 +1573,35 @@ impl Drop for GamescopeProc {
#[cfg(test)]
mod tests {
use super::{cgroup_is_punktfunk_owned, is_steam_launch, shape_dedicated_command};
use super::{
cgroup_is_punktfunk_owned, connected_connector_under, is_steam_launch,
shape_dedicated_command,
};
#[test]
fn connector_status_scan() {
let base = std::env::temp_dir().join(format!("pf-drm-scan-{}", std::process::id()));
let mk = |name: &str, status: Option<&str>| {
let dir = base.join(name);
std::fs::create_dir_all(&dir).unwrap();
if let Some(s) = status {
std::fs::write(dir.join("status"), s).unwrap();
}
};
// Headless layout: device + render nodes only (no status files) → not connected.
mk("card0", None);
mk("renderD128", None);
assert!(!connected_connector_under(&base));
// Connectors present but nothing plugged in → still not connected.
mk("card0-HDMI-A-1", Some("disconnected\n"));
assert!(!connected_connector_under(&base));
// A live panel → connected.
mk("card0-eDP-1", Some("connected\n"));
assert!(connected_connector_under(&base));
// A missing base dir (no DRM at all) reads as headless.
assert!(!connected_connector_under(&base.join("nope")));
std::fs::remove_dir_all(&base).unwrap();
}
#[test]
fn steam_launch_detection() {
@@ -207,6 +207,20 @@ pub fn cancel_pending_tv_restore() {
#[cfg(not(target_os = "linux"))]
pub fn cancel_pending_tv_restore() {}
/// Can the MANAGED gamescope path stand a session up from nothing on this box (SteamOS's
/// `gamescope-session` launcher or Bazzite's `gamescope-session-plus` present)? Lets the connect
/// path route a "no live graphical session" box to the gamescope takeover — which rebuilds the
/// session at the client's mode — instead of failing the connect. Always `false` off Linux.
#[cfg(target_os = "linux")]
pub fn managed_session_available() -> bool {
gamescope::managed_session_available()
}
#[cfg(not(target_os = "linux"))]
pub fn managed_session_available() -> bool {
false
}
/// Call when a client session ends: if the host-managed gamescope path took over a box's autologin
/// gaming session (stopped its single-instance Steam to stream at the client's mode), **schedule** a
/// debounced restore so the TV returns to gaming mode — unless a client reconnects within the window
@@ -57,6 +57,13 @@ pub fn observe_session_instance(active: &ActiveSession) {
if let Some(old) = compositor_for_kind(prev.0) {
registry::invalidate_backend(old.id());
}
// The dead desktop's socket vars may still sit in the systemd --user manager env
// ([`settle_desktop_portal`]'s import-environment) — scrub them NOW, or the next
// `gamescope-session.target` start inherits a stale WAYLAND_DISPLAY and gamescope
// runs NESTED against the dead desktop socket instead of becoming the display
// server ("Failed to connect to wayland socket: wayland-0" — kept a Deck's Game
// Mode from starting at all, observed live 2026-07-21).
scrub_desktop_manager_env();
}
let epoch = bump_session_epoch();
tracing::info!(
@@ -70,6 +77,23 @@ pub fn observe_session_instance(active: &ActiveSession) {
*last = Some(cur);
}
/// Counterpart to [`settle_desktop_portal`]'s `import-environment`: drop the desktop session's
/// socket vars from the systemd `--user` manager env once that desktop instance is GONE. They
/// persist in the manager otherwise, and every later user unit inherits them — including
/// `gamescope-session.target`, whose gamescope then aborts trying to attach to the dead desktop
/// socket. Best-effort; the D-Bus activation env has no unset op, but gamescope-session is
/// systemd-started, so the manager scrub is the one that matters. (A desktop restart re-imports
/// via the next [`settle_desktop_portal`], so scrubbing on a bounce is harmless.)
#[cfg(target_os = "linux")]
fn scrub_desktop_manager_env() {
let _ = std::process::Command::new("systemctl")
.args(["--user", "unset-environment", "WAYLAND_DISPLAY", "DISPLAY"])
.status();
}
#[cfg(not(target_os = "linux"))]
fn scrub_desktop_manager_env() {}
/// Is `kind` a **desktop** compositor (KWin / Mutter / wlroots) — one whose kept PipeWire outputs die
/// with the compositor instance, so the session epoch tracks it? `Gaming` (gamescope) and `None` are
/// not (gamescope spawns are independent nested sessions — see [`observe_session_instance`]).
+12 -1
View File
@@ -861,7 +861,18 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
continue;
}
if p.flags & DISPLAYCONFIG_PATH_ACTIVE != 0 {
p.flags &= !DISPLAYCONFIG_PATH_ACTIVE; // mark this path inactive
// Mark the path inactive AND unpin its modes: per the SetDisplayConfig
// contract a path being turned OFF needs BOTH mode indexes marked invalid,
// and leaving them referencing the queried mode entries gets the whole
// supplied config rejected with 0x57 ERROR_INVALID_PARAMETER on some
// driver/topology combinations (field-reported: exclusive mode left the
// physical panel lit, every retry failing 0x57). Writing the all-ones
// sentinel to the whole union is also correct under the virtual-mode-aware
// interpretation (cloneGroupId/sourceModeInfoIdx both become their 0xffff
// INVALID values).
p.flags &= !DISPLAYCONFIG_PATH_ACTIVE;
p.sourceInfo.Anonymous.modeInfoIdx = DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
p.targetInfo.Anonymous.modeInfoIdx = DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
others += 1;
}
}
+5
View File
@@ -33,6 +33,11 @@ reed-solomon-simd = "3.1" # GF(2^16) Leopard-RS, SIMD, O(n log n) — the w
# NOT interoperable.) See vendor/fec-rs/LICENSE (BSD-2-Clause).
fec-rs = { path = "vendor/fec-rs" }
aes-gcm = "0.10" # AES-128-GCM session crypto, matches GameStream
# ChaCha20-Poly1305 session crypto, negotiated by clients without hardware AES (the soft-AES
# armv7 targets — webOS TVs — where GCM caps decrypt at ~100 Mbps; ARX runs 4-7x faster there).
# Same RustCrypto `aead 0.5` generation as aes-gcm: identical trait/nonce/tag shapes, pure Rust,
# cross-compiles like aes-gcm (no cmake). See design/chacha20-session-cipher.md.
chacha20poly1305 = "0.10"
zerocopy = { version = "0.8", features = ["derive"] }
bytes = "1"
socket2 = { version = "0.6", features = [
+22 -12
View File
@@ -1,7 +1,8 @@
//! Tier-1 microbenchmarks for the punktfunk/1 hot path — GPU-free, so they run in normal CI.
//!
//! Two layers:
//! - `crypto/*` — the isolated AES-128-GCM primitives on one ~MTU shard.
//! - `crypto/*` — the isolated AEAD primitives (AES-128-GCM + the negotiated
//! ChaCha20-Poly1305) on one ~MTU shard.
//! - `pipeline/*`— a whole frame through the real per-frame path end to end over the in-process
//! loopback transport: FEC encode → AES-GCM seal → packetize → (loopback) → reassemble →
//! FEC decode → open. This is what a throughput/latency regression in the core would show up in.
@@ -11,11 +12,11 @@
use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion, Throughput};
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionCrypto;
use punktfunk_core::crypto::{SessionCrypto, SessionKey};
use punktfunk_core::session::Session;
use punktfunk_core::transport::loopback_pair;
const TAG_LEN: usize = 16; // AES-GCM authentication tag
const TAG_LEN: usize = 16; // AEAD authentication tag (GCM and Poly1305 share the size)
const SHARD: usize = punktfunk_core::config::mtu1500_shard_payload(); // one MTU-safe data shard
fn cfg(role: Role, scheme: FecScheme) -> Config {
@@ -38,21 +39,29 @@ fn cfg(role: Role, scheme: FecScheme) -> Config {
shard_payload: SHARD,
max_frame_bytes: 8 * 1024 * 1024,
encrypt: true, // bench the real path — crypto is always on for punktfunk/1
key: [7u8; 16],
key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [1, 2, 3, 4],
loopback_drop_period: 0, // throughput run: no induced loss (loss-harness covers recovery)
}
}
fn bench_crypto(c: &mut Criterion) {
let host = SessionCrypto::new(&[7u8; 16], [1, 2, 3, 4], Role::Host);
let client = SessionCrypto::new(&[7u8; 16], [1, 2, 3, 4], Role::Client);
let mut g = c.benchmark_group("crypto");
g.throughput(Throughput::Bytes(SHARD as u64));
// Both negotiated session AEADs. On the x86 / Apple Silicon this runs on, both must be
// line-rate-trivial — the chacha20 series is the host-side sealing-cost check for the
// negotiated soft-AES-armv7 path (design/chacha20-session-cipher.md §7). The AES series
// keeps its unsuffixed names so the CI regression compare retains its history.
for (suffix, key) in [
("", SessionKey::Aes128Gcm([7u8; 16])),
("_chacha20", SessionKey::ChaCha20Poly1305([7u8; 32])),
] {
let host = SessionCrypto::new(&key, [1, 2, 3, 4], Role::Host);
let client = SessionCrypto::new(&key, [1, 2, 3, 4], Role::Client);
let payload = vec![0xABu8; SHARD];
let sealed = host.seal(0, &payload).unwrap();
let mut g = c.benchmark_group("crypto");
g.throughput(Throughput::Bytes(SHARD as u64));
g.bench_function("seal", |b| {
g.bench_function(format!("seal{suffix}"), |b| {
let mut seq = 0u64;
b.iter(|| {
let ct = host.seal(seq, black_box(&payload)).unwrap();
@@ -60,7 +69,7 @@ fn bench_crypto(c: &mut Criterion) {
black_box(ct)
})
});
g.bench_function("seal_in_place", |b| {
g.bench_function(format!("seal_in_place{suffix}"), |b| {
let mut seq = 0u64;
let mut buf = vec![0xABu8; SHARD + TAG_LEN];
b.iter(|| {
@@ -68,10 +77,10 @@ fn bench_crypto(c: &mut Criterion) {
seq += 1;
})
});
g.bench_function("open", |b| {
g.bench_function(format!("open{suffix}"), |b| {
b.iter(|| black_box(client.open(0, black_box(&sealed)).unwrap()))
});
g.bench_function("open_in_place", |b| {
g.bench_function(format!("open_in_place{suffix}"), |b| {
// In-place open consumes the buffer, so each iteration restores the ciphertext first —
// one memcpy, mirroring what the recv ring does when the next datagram lands in the slot.
let mut buf = sealed.clone();
@@ -80,6 +89,7 @@ fn bench_crypto(c: &mut Criterion) {
black_box(client.open_in_place(0, &mut buf).unwrap());
})
});
}
g.finish();
}
+5 -1
View File
@@ -11,6 +11,7 @@
//! - Panics never cross the boundary: every entry point is wrapped in `catch_unwind`.
use crate::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use crate::crypto::SessionKey;
use crate::error::PunktfunkStatus;
use crate::input::InputEvent;
use crate::reanchor::{GateVerdict, ReanchorGate};
@@ -94,7 +95,10 @@ impl PunktfunkConfig {
shard_payload: self.shard_payload as usize,
max_frame_bytes,
encrypt: self.encrypt != 0,
key: self.key,
// The C ABI keeps its fixed 16-byte key and always selects AES-128-GCM — no
// ABI_VERSION bump. Raw-`Config` C embedders can't negotiate ChaCha; the Swift/
// Kotlin clients are aarch64 with AES CE and never want it.
key: SessionKey::Aes128Gcm(self.key),
salt: self.salt,
loopback_drop_period: self.loopback_drop_period,
};
+1
View File
@@ -393,6 +393,7 @@ impl NativeClient {
launch,
pin,
identity,
connect_timeout: timeout,
frames: frame_chan_w,
audio_tx,
rumble_tx,
+14 -2
View File
@@ -86,10 +86,22 @@ impl NativeClient {
// A typed application close from the host (pairing not armed / armed for a
// different device / rate-limited / version mismatch) beats the generic
// transport error the aborted exchange produced — it is the actual answer.
Err(e) => Err(match reject_from_close(&conn) {
// Same close-vs-stream-error race as the connect handshake: give the
// host's CONNECTION_CLOSE a short grace to be processed before deciding
// the error was plain transport trouble.
Err(e) => {
if conn.close_reason().is_none() {
let _ = tokio::time::timeout(
std::time::Duration::from_millis(300),
conn.closed(),
)
.await;
}
Err(match reject_from_close(&conn) {
Some(r) => PunktfunkError::Rejected(r),
None => e,
}),
})
}
ok => ok,
};
// Always tell the host we're done so it never blocks at its read — code 0 on
@@ -32,21 +32,65 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
identity.as_ref().map(|(c, k)| (c.as_str(), k.as_str())),
);
let ep = ep.map_err(|e| PunktfunkError::Io(std::io::Error::other(e.to_string())))?;
let conn = ep
// Dial with retry across the connect budget, not a single attempt: one quinn dial gives
// up after the transport's idle window (~8 s of silence), which is shorter than a
// suspend-to-RAM resume — the Steam Deck flow fires Wake-on-LAN and connects
// immediately, so the host is still waking while the first Initials go out, and a
// single-shot dial died just before the host came up. Short attempts keep the Initial
// cadence dense (quinn's per-attempt retransmits back off toward multi-second gaps), so
// the connect lands within ~a second of the host's network returning. Only SILENCE is
// retried: a host that answers and rejects us (pin mismatch, ALPN/version, typed close)
// must surface immediately, and the embedder's shutdown flag (budget expiry in
// `connect`, or a user cancel) stops the loop between attempts.
const DIAL_ATTEMPT: std::time::Duration = std::time::Duration::from_secs(3);
// Redial headroom: leave room for the control handshake (Hello/Welcome/clock sync)
// after a late dial success, so it still completes inside the embedder's budget.
const CONTROL_HEADROOM: std::time::Duration = std::time::Duration::from_secs(2);
let start = tokio::time::Instant::now();
let deadline = start + args.connect_timeout;
let redial_until = start + args.connect_timeout.saturating_sub(CONTROL_HEADROOM);
let conn = loop {
let connecting = ep
.connect(remote, "punktfunk")
.map_err(|_| PunktfunkError::InvalidArg("connect"))?
.await
.map_err(|e| {
.map_err(|_| PunktfunkError::InvalidArg("connect"))?;
// Cap the attempt to the remaining budget so a success never lands after the
// embedder's `ready_rx` wait has already given up and flagged a teardown.
let now = tokio::time::Instant::now();
let attempt = DIAL_ATTEMPT.min(deadline.saturating_duration_since(now));
let gave_up = || {
tokio::time::Instant::now() >= redial_until
|| shutdown.load(std::sync::atomic::Ordering::SeqCst)
};
match tokio::time::timeout(attempt, connecting).await {
Ok(Ok(conn)) => break conn,
Ok(Err(e)) => {
// A pin mismatch surfaces as a TLS failure; report it as a crypto error so
// the embedder can distinguish "wrong host identity" from plain IO trouble.
let fp_mismatch =
pin.is_some() && observed.lock().unwrap().map(|fp| Some(fp) != pin) == Some(true);
let fp_mismatch = pin.is_some()
&& observed.lock().unwrap().map(|fp| Some(fp) != pin) == Some(true);
if fp_mismatch {
PunktfunkError::Crypto
} else {
PunktfunkError::Io(std::io::Error::other(e.to_string()))
return Err(PunktfunkError::Crypto);
}
})?;
// The transport's own idle expiry — the host never answered — is the one
// retryable outcome; everything else is a real answer or a local failure.
let host_silent = matches!(e, quinn::ConnectionError::TimedOut);
if !host_silent {
return Err(PunktfunkError::Io(std::io::Error::other(e.to_string())));
}
if gave_up() {
return Err(PunktfunkError::Timeout);
}
}
// Attempt window elapsed with the host still silent; dropping `connecting`
// abandoned that dial — go again unless the budget is spent.
Err(_) => {
if gave_up() {
return Err(PunktfunkError::Timeout);
}
}
}
tracing::debug!(%remote, "host silent — re-dialing (wake/resume tolerant connect)");
};
let fingerprint = observed.lock().unwrap().unwrap_or([0u8; 32]);
// The rest of the handshake runs in an inner future so a failure can consult
// `conn.close_reason()`: a host that turned us away with a typed application close
@@ -196,9 +240,22 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
negotiated,
host_caps,
}),
Err(e) => Err(match reject_from_close(&conn) {
Err(e) => {
// The host's typed close can land a beat AFTER the stream error it caused: the
// stream reset/FIN and the CONNECTION_CLOSE are in flight together, and quinn can
// hand the reader its mid-frame EOF before it processes the close. Give the close a
// short grace to arrive so a host-side setup failure renders as its real reason
// ("the host could not start the stream session") instead of "control stream
// finished mid-frame". No-op when the connection already closed (or never will —
// bounded by the timeout).
if conn.close_reason().is_none() {
let _ = tokio::time::timeout(std::time::Duration::from_millis(300), conn.closed())
.await;
}
Err(match reject_from_close(&conn) {
Some(r) => PunktfunkError::Rejected(r),
None => e,
}),
})
}
}
}
@@ -25,6 +25,10 @@ pub(crate) struct WorkerArgs {
pub(crate) launch: Option<String>,
pub(crate) pin: Option<[u8; 32]>,
pub(crate) identity: Option<(String, String)>,
/// The embedder's connect budget (the same value `connect` bounds `ready_rx` with): the
/// dial loop re-dials a silent host within it, so a host still resuming from Wake-on-LAN
/// is caught the moment its network comes back instead of failing on the first attempt.
pub(crate) connect_timeout: std::time::Duration,
pub(crate) frames: Arc<FrameChannel>,
pub(crate) audio_tx: SyncSender<AudioPacket>,
pub(crate) rumble_tx: SyncSender<RumbleUpdate>,
+15 -6
View File
@@ -1,5 +1,6 @@
//! Session configuration and protocol/FEC parameters.
use crate::crypto::SessionKey;
use crate::error::{PunktfunkError, Result};
use crate::packet::{CRYPTO_OVERHEAD, HEADER_LEN, MAX_DATAGRAM_BYTES};
use zeroize::Zeroize;
@@ -355,9 +356,11 @@ pub struct Config {
/// hostile/corrupt headers; see [`Session`](crate::session::Session)).
pub max_frame_bytes: usize,
pub encrypt: bool,
/// AES-128 session key established during pairing. MUST be unique per session when
/// The negotiated session AEAD + its key, established during pairing/handshake —
/// AES-128-GCM for every peer by default, ChaCha20-Poly1305 when the client negotiated it
/// (soft-AES armv7 targets; see [`SessionKey`]). MUST be unique per session when
/// `encrypt` is set (see the nonce-uniqueness contract in [`crate::crypto`]).
pub key: [u8; 16],
pub key: SessionKey,
/// Per-session nonce salt, established alongside `key` during pairing. MUST be
/// unique per (key, session).
pub salt: [u8; 4],
@@ -382,7 +385,8 @@ impl std::fmt::Debug for Config {
.field("shard_payload", &self.shard_payload)
.field("max_frame_bytes", &self.max_frame_bytes)
.field("encrypt", &self.encrypt)
.field("key", &"<redacted>")
// SessionKey's own Debug redacts the material but keeps the cipher choice visible.
.field("key", &self.key)
.field("salt", &"<redacted>")
.field("loopback_drop_period", &self.loopback_drop_period)
.finish()
@@ -426,7 +430,7 @@ impl Config {
"max_frame_bytes too large for this shard/block configuration (block count overflows u16)",
));
}
if self.encrypt && self.key == [0u8; 16] {
if self.encrypt && self.key.is_zero() {
return Err(PunktfunkError::InvalidArg(
"encrypt requires a non-zero session key (see crypto nonce-uniqueness contract)",
));
@@ -449,7 +453,7 @@ impl Config {
shard_payload: 1024,
max_frame_bytes: 64 * 1024 * 1024,
encrypt: false,
key: [0u8; 16],
key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4],
loopback_drop_period: 0,
}
@@ -465,7 +469,12 @@ mod tests {
let mut c = Config::p1_defaults(Role::Host);
c.encrypt = true; // key is still all-zero
assert!(c.validate().is_err());
c.key = [1u8; 16];
c.key = SessionKey::Aes128Gcm([1u8; 16]);
assert!(c.validate().is_ok());
// The rejection follows whichever cipher variant is active.
c.key = SessionKey::ChaCha20Poly1305([0u8; 32]);
assert!(c.validate().is_err());
c.key = SessionKey::ChaCha20Poly1305([1u8; 32]);
assert!(c.validate().is_ok());
}
+194 -34
View File
@@ -1,9 +1,10 @@
//! AES-128-GCM session sealing, matching GameStream's video crypto in P1.
//! Session sealing with the negotiated AEAD — AES-128-GCM (matching GameStream's video
//! crypto in P1) by default, ChaCha20-Poly1305 (RFC 8439) for clients without hardware AES.
//!
//! ## Nonce uniqueness (the GCM safety requirement)
//! ## Nonce uniqueness (the AEAD safety requirement)
//!
//! The 96-bit nonce is `salt (4 bytes) || sequence (8 bytes, big-endian)`. Reusing a
//! `(key, nonce)` pair under AES-GCM is catastrophic, so two precautions apply:
//! `(key, nonce)` pair is catastrophic under either AEAD, so two precautions apply:
//!
//! 1. **Per-direction salts.** Host and client share one `key` and `salt`, and each
//! counts its sequence from 0. To stop the host's video stream and the client's input
@@ -17,17 +18,96 @@
//! The sequence number is also passed as AEAD associated data, so tampering with the
//! on-wire sequence is detected (the tag check fails) rather than silently shifting the
//! nonce. Note: this layer does not provide anti-replay — see `Session`.
//!
//! ## Why two ciphers
//!
//! Both AEADs are full-strength; the choice (negotiated via `Welcome::cipher`) is purely a
//! performance one. On targets without hardware AES — the soft-AES armv7 clients (webOS TVs) —
//! GCM's fixsliced AES + software GHASH costs ~50100 cycles/byte and caps decrypt at
//! ~100 Mbps, while ChaCha20-Poly1305's ARX construction runs ~1017 cycles/byte in portable
//! software (design/chacha20-session-cipher.md). Same 96-bit nonce, 16-byte tag, and AAD
//! shape, so the entire nonce discipline above carries over verbatim.
use crate::config::Role;
use crate::error::{PunktfunkError, Result};
use aes_gcm::aead::{Aead, AeadInPlace, KeyInit, Payload};
use aes_gcm::{Aes128Gcm, Key, Nonce};
use chacha20poly1305::ChaCha20Poly1305;
use zeroize::Zeroize;
/// 16-byte AEAD authentication tag appended by GCM.
/// 16-byte AEAD authentication tag appended by either session cipher.
pub const TAG_LEN: usize = 16;
// The wire (CRYPTO_OVERHEAD) and every in-place split assume both negotiated AEADs append
// exactly TAG_LEN bytes — a different-tag cipher can never slip in behind this constant.
const _: () = assert!(std::mem::size_of::<aes_gcm::Tag>() == TAG_LEN);
const _: () = assert!(std::mem::size_of::<chacha20poly1305::Tag>() == TAG_LEN);
/// The negotiated session AEAD together with its key material — merged so the invalid state
/// (a ChaCha cipher with an AES-sized key, or vice versa) is unrepresentable. AES-128-GCM is
/// the default every peer speaks; ChaCha20-Poly1305 is granted to clients that advertised
/// [`VIDEO_CAP_CHACHA20`](crate::quic::VIDEO_CAP_CHACHA20) (the soft-AES armv7 targets —
/// see the module docs). 256 bits for ChaCha is what RFC 8439 requires.
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum SessionKey {
Aes128Gcm([u8; 16]),
ChaCha20Poly1305([u8; 32]),
}
impl SessionKey {
/// Canonical lowercase cipher name for session-start logs.
pub fn cipher_name(&self) -> &'static str {
match self {
SessionKey::Aes128Gcm(_) => "aes-128-gcm",
SessionKey::ChaCha20Poly1305(_) => "chacha20-poly1305",
}
}
/// True when the key material is all zeros — the pairing-layer footgun `Config::validate`
/// rejects when encryption is on (see the nonce-uniqueness contract in the module docs).
pub fn is_zero(&self) -> bool {
match self {
SessionKey::Aes128Gcm(k) => k == &[0u8; 16],
SessionKey::ChaCha20Poly1305(k) => k == &[0u8; 32],
}
}
}
/// Key material never appears in logs, whichever variant is active — only the cipher choice
/// (`Config`'s hand-written `Debug` relies on this).
impl std::fmt::Debug for SessionKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
SessionKey::Aes128Gcm(_) => f.write_str("Aes128Gcm(<redacted>)"),
SessionKey::ChaCha20Poly1305(_) => f.write_str("ChaCha20Poly1305(<redacted>)"),
}
}
}
/// Same zeroize-on-drop discipline the raw key array had (`Config`'s `Drop`).
impl Zeroize for SessionKey {
fn zeroize(&mut self) {
match self {
SessionKey::Aes128Gcm(k) => k.zeroize(),
SessionKey::ChaCha20Poly1305(k) => k.zeroize(),
}
}
}
/// The two negotiated AEADs behind one seal/open surface. Both are the same RustCrypto
/// `aead 0.5` generation (identical trait shapes, nonce/tag types), so each call below is a
/// two-arm match right next to the cipher work itself.
// AES's precomputed round keys (~0.7 KB) dwarf ChaCha's 32-byte state, but there is exactly
// one long-lived `SessionCrypto` per session — boxing the variant would trade that one-off
// slack for a pointer chase on every per-datagram seal/open.
#[allow(clippy::large_enum_variant)]
enum Cipher {
Aes128Gcm(Aes128Gcm),
ChaCha20Poly1305(ChaCha20Poly1305),
}
pub struct SessionCrypto {
cipher: Aes128Gcm,
cipher: Cipher,
/// Salt for nonces we seal with (our direction).
send_salt: [u8; 4],
/// Salt for nonces we open with (the peer's direction).
@@ -35,11 +115,18 @@ pub struct SessionCrypto {
}
impl SessionCrypto {
pub fn new(key: &[u8; 16], salt: [u8; 4], role: Role) -> Self {
let key = Key::<Aes128Gcm>::from_slice(key);
pub fn new(key: &SessionKey, salt: [u8; 4], role: Role) -> Self {
let cipher = match key {
SessionKey::Aes128Gcm(k) => {
Cipher::Aes128Gcm(Aes128Gcm::new(Key::<Aes128Gcm>::from_slice(k)))
}
SessionKey::ChaCha20Poly1305(k) => Cipher::ChaCha20Poly1305(ChaCha20Poly1305::new(
Key::<ChaCha20Poly1305>::from_slice(k),
)),
};
let own = direction(role);
SessionCrypto {
cipher: Aes128Gcm::new(key),
cipher,
send_salt: dir_salt(salt, own),
recv_salt: dir_salt(salt, own ^ 1),
}
@@ -49,14 +136,15 @@ impl SessionCrypto {
/// authenticated as associated data.
pub fn seal(&self, seq: u64, plaintext: &[u8]) -> Result<Vec<u8>> {
let nonce = nonce(self.send_salt, seq);
self.cipher
.encrypt(
Nonce::from_slice(&nonce),
Payload {
let aad = seq.to_be_bytes();
let payload = Payload {
msg: plaintext,
aad: &seq.to_be_bytes(),
},
)
aad: &aad,
};
match &self.cipher {
Cipher::Aes128Gcm(c) => c.encrypt(Nonce::from_slice(&nonce), payload),
Cipher::ChaCha20Poly1305(c) => c.encrypt(Nonce::from_slice(&nonce), payload),
}
.map_err(|_| PunktfunkError::Crypto)
}
@@ -69,9 +157,15 @@ impl SessionCrypto {
let nonce = nonce(self.send_salt, seq);
let split = buf.len() - TAG_LEN;
let (plaintext, tag_slot) = buf.split_at_mut(split);
let tag = self
.cipher
.encrypt_in_place_detached(Nonce::from_slice(&nonce), &seq.to_be_bytes(), plaintext)
let aad = seq.to_be_bytes();
let tag = match &self.cipher {
Cipher::Aes128Gcm(c) => {
c.encrypt_in_place_detached(Nonce::from_slice(&nonce), &aad, plaintext)
}
Cipher::ChaCha20Poly1305(c) => {
c.encrypt_in_place_detached(Nonce::from_slice(&nonce), &aad, plaintext)
}
}
.map_err(|_| PunktfunkError::Crypto)?;
tag_slot.copy_from_slice(&tag);
Ok(())
@@ -80,20 +174,21 @@ impl SessionCrypto {
/// Open `ciphertext || tag` for sequence `seq` (also bound as associated data).
pub fn open(&self, seq: u64, ciphertext: &[u8]) -> Result<Vec<u8>> {
let nonce = nonce(self.recv_salt, seq);
self.cipher
.decrypt(
Nonce::from_slice(&nonce),
Payload {
let aad = seq.to_be_bytes();
let payload = Payload {
msg: ciphertext,
aad: &seq.to_be_bytes(),
},
)
aad: &aad,
};
match &self.cipher {
Cipher::Aes128Gcm(c) => c.decrypt(Nonce::from_slice(&nonce), payload),
Cipher::ChaCha20Poly1305(c) => c.decrypt(Nonce::from_slice(&nonce), payload),
}
.map_err(|_| PunktfunkError::Crypto)
}
/// Open in place, no per-packet allocation: `buf` holds `[ciphertext .. ][tag]` on entry and
/// the plaintext in its first `buf.len() - TAG_LEN` bytes on success (returned as the length)
/// — byte-identical to `open`, just written in place. GCM verifies the tag *before*
/// — byte-identical to `open`, just written in place. Both AEADs verify the tag *before*
/// decrypting, so on failure `buf` still holds the ciphertext (the caller drops the packet
/// either way). The hot-path receiver (`Session::poll_frame`) uses this to avoid the `Vec`
/// that `open`'s convenience API allocates for every datagram at line rate — the receive
@@ -105,13 +200,21 @@ impl SessionCrypto {
let nonce = nonce(self.recv_salt, seq);
let split = buf.len() - TAG_LEN;
let (ciphertext, tag) = buf.split_at_mut(split);
self.cipher
.decrypt_in_place_detached(
let aad = seq.to_be_bytes();
match &self.cipher {
Cipher::Aes128Gcm(c) => c.decrypt_in_place_detached(
Nonce::from_slice(&nonce),
&seq.to_be_bytes(),
&aad,
ciphertext,
aes_gcm::Tag::from_slice(tag),
)
),
Cipher::ChaCha20Poly1305(c) => c.decrypt_in_place_detached(
Nonce::from_slice(&nonce),
&aad,
ciphertext,
chacha20poly1305::Tag::from_slice(tag),
),
}
.map_err(|_| PunktfunkError::Crypto)?;
Ok(split)
}
@@ -145,6 +248,13 @@ pub fn random_key() -> [u8; 16] {
k
}
/// Generate a fresh random ChaCha20-Poly1305 session key (RFC 8439's 256-bit size).
pub fn random_key32() -> [u8; 32] {
let mut k = [0u8; 32];
rand::RngCore::fill_bytes(&mut rand::rng(), &mut k);
k
}
/// Generate a fresh random per-session nonce salt.
pub fn random_salt() -> [u8; 4] {
let mut s = [0u8; 4];
@@ -156,9 +266,17 @@ pub fn random_salt() -> [u8; 4] {
mod tests {
use super::*;
/// One fresh key per negotiated cipher — every sealing test below must hold for both.
fn both_keys() -> [SessionKey; 2] {
[
SessionKey::Aes128Gcm(random_key()),
SessionKey::ChaCha20Poly1305(random_key32()),
]
}
#[test]
fn seal_open_roundtrip_cross_direction() {
let key = random_key();
for key in both_keys() {
let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -175,10 +293,11 @@ mod tests {
// open its own outbound packet → distinct nonce spaces per direction.
assert!(host.open(42, &sealed).is_err());
}
}
#[test]
fn directions_use_distinct_nonce_spaces() {
let key = random_key();
for key in both_keys() {
let salt = [0u8; 4]; // even an all-zero base salt must separate the directions
let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -188,10 +307,11 @@ mod tests {
client.seal(0, b"abc").unwrap()
);
}
}
#[test]
fn open_in_place_matches_open_and_rejects_tampering() {
let key = random_key();
for key in both_keys() {
let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -221,10 +341,11 @@ mod tests {
let mut runt = vec![0u8; TAG_LEN - 1];
assert!(client.open_in_place(0, &mut runt).is_err());
}
}
#[test]
fn seal_in_place_matches_seal_and_opens() {
let key = random_key();
for key in both_keys() {
let salt = random_salt();
let host = SessionCrypto::new(&key, salt, Role::Host);
let client = SessionCrypto::new(&key, salt, Role::Client);
@@ -246,3 +367,42 @@ mod tests {
}
}
}
#[test]
fn ciphers_are_not_interchangeable() {
// A packet sealed under one AEAD must not open under the other — negotiation skew has
// to fail loudly (a tag mismatch), never decode garbage. The ChaCha key repeats the AES
// key bytes so even overlapping key material can't accidentally interoperate.
let salt = random_salt();
let aes = SessionKey::Aes128Gcm([7u8; 16]);
let chacha = SessionKey::ChaCha20Poly1305([7u8; 32]);
let sealed = SessionCrypto::new(&aes, salt, Role::Host)
.seal(1, b"cross-cipher")
.unwrap();
assert!(SessionCrypto::new(&chacha, salt, Role::Client)
.open(1, &sealed)
.is_err());
let sealed = SessionCrypto::new(&chacha, salt, Role::Host)
.seal(1, b"cross-cipher")
.unwrap();
assert!(SessionCrypto::new(&aes, salt, Role::Client)
.open(1, &sealed)
.is_err());
}
#[test]
fn session_key_zero_check_and_debug_redaction() {
assert!(SessionKey::Aes128Gcm([0u8; 16]).is_zero());
assert!(SessionKey::ChaCha20Poly1305([0u8; 32]).is_zero());
assert!(!SessionKey::Aes128Gcm([1u8; 16]).is_zero());
assert!(!SessionKey::ChaCha20Poly1305([1u8; 32]).is_zero());
// Key bytes must never reach a log, whichever variant — only the cipher choice.
for key in both_keys() {
let dbg = format!("{key:?}");
assert!(dbg.contains("<redacted>"), "{dbg}");
}
let mut k = SessionKey::ChaCha20Poly1305([9u8; 32]);
k.zeroize();
assert!(k.is_zero());
}
}
+2
View File
@@ -61,6 +61,7 @@ pub enum PunktfunkStatus {
RejectedSuperseded = -26,
RejectedWireVersion = -27,
RejectedBusy = -28,
RejectedSetupFailed = -29,
Panic = -99,
}
@@ -89,6 +90,7 @@ impl PunktfunkError {
R::Superseded => PunktfunkStatus::RejectedSuperseded,
R::WireVersionMismatch => PunktfunkStatus::RejectedWireVersion,
R::Busy => PunktfunkStatus::RejectedBusy,
R::SetupFailed => PunktfunkStatus::RejectedSetupFailed,
}
}
}
+31 -2
View File
@@ -73,6 +73,17 @@ pub enum InputKind {
/// [`HOST_CAP_GAMEPAD_STATE`](crate::quic::HOST_CAP_GAMEPAD_STATE); an older host ignores the
/// unknown tag (every pad then uses the session-default kind — the pre-existing behaviour).
GamepadArrival = 14,
/// One Unicode scalar of **committed text** — `code` = the scalar value, everything else 0.
///
/// The IME path: the layout-independent VK key events cannot express text an input method
/// *commits* (autocorrect, gesture typing, non-Latin scripts, emoji), so a capable client
/// sends the committed characters verbatim and the host injects them directly (Windows
/// `KEYEVENTF_UNICODE`; Linux wlroots via a dynamically-grown Unicode keymap on a dedicated
/// virtual keyboard). A multi-character commit is consecutive events in order. Sent only when
/// the host advertised [`HOST_CAP_TEXT_INPUT`](crate::quic::HOST_CAP_TEXT_INPUT) — toward an
/// older host (or one whose inject backend can't type text) clients keep the best-effort VK
/// synthesis, and an older host ignores the unknown tag entirely.
TextInput = 15,
}
/// Pack a [`InputKind::GamepadRemove`] `flags` word (`seq << 24 | pad`) — the same low-byte-pad /
@@ -158,6 +169,7 @@ impl InputKind {
12 => GamepadState,
13 => GamepadRemove,
14 => GamepadArrival,
15 => TextInput,
_ => return None,
})
}
@@ -392,10 +404,27 @@ mod tests {
};
assert_eq!(InputEvent::decode(&e.encode()), Some(e));
}
// GamepadRemove + GamepadArrival are valid kinds; 15 (one past them) is not.
// GamepadRemove/GamepadArrival/TextInput are valid kinds; 16 (one past them) is not.
assert_eq!(InputKind::from_u8(13), Some(InputKind::GamepadRemove));
assert_eq!(InputKind::from_u8(14), Some(InputKind::GamepadArrival));
assert_eq!(InputKind::from_u8(15), None);
assert_eq!(InputKind::from_u8(15), Some(InputKind::TextInput));
assert_eq!(InputKind::from_u8(16), None);
}
#[test]
fn text_input_roundtrip() {
// One Unicode scalar per event — BMP and astral (emoji) alike.
for cp in ['a' as u32, 'ß' as u32, '語' as u32, 0x1F600 /* 😀 */] {
let e = InputEvent {
kind: InputKind::TextInput,
_pad: [0; 3],
code: cp,
x: 0,
y: 0,
flags: 0,
};
assert_eq!(InputEvent::decode(&e.encode()), Some(e));
}
}
#[test]
+3 -2
View File
@@ -1,6 +1,7 @@
use super::reassemble::LOSS_WINDOW_NS;
use super::*;
use crate::config::{Config, FecScheme};
use crate::crypto::SessionKey;
use crate::fec::coder_for;
use crate::stats::StatsCounters;
use zerocopy::{FromBytes, IntoBytes};
@@ -182,7 +183,7 @@ fn explicit_frame_index_is_stamped_and_internal_counter_untouched() {
shard_payload: 16,
max_frame_bytes: 4096,
encrypt: false,
key: [0u8; 16],
key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4],
loopback_drop_period: 0,
};
@@ -292,7 +293,7 @@ fn e2e_config(scheme: FecScheme, fec_percent: u8) -> Config {
shard_payload: 16,
max_frame_bytes: 4096,
encrypt: false,
key: [0u8; 16],
key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4],
loopback_drop_period: 0,
}
+23
View File
@@ -44,6 +44,17 @@ pub const VIDEO_CAP_PROBE_SEQ: u8 = 0x10;
/// bit; every other client gets today's whole-AU path (chunks concatenated before sealing), so
/// the fallback is zero-risk.
pub const VIDEO_CAP_STREAMED_AU: u8 = 0x20;
/// [`Hello::video_caps`] bit: the client can open **ChaCha20-Poly1305**-sealed session datagrams
/// AND requests them — set by clients without hardware AES (the soft-AES armv7 targets, e.g.
/// webOS TVs), where GCM's software AES + GHASH caps decrypt at ~100 Mbps while ChaCha's ARX
/// construction runs 47× faster in portable code (design/chacha20-session-cipher.md).
/// Support-plus-request in one bit mirrors [`VIDEO_CAP_444`]'s "capable AND turned on"
/// precedent. The host grants it only when its `PUNKTFUNK_CHACHA20` kill-switch (default on)
/// allows, answering with [`Welcome::cipher`] `= 1` + the 32-byte [`Welcome::key_chacha`];
/// toward every other client the Welcome stays byte-identical AES-128-GCM. Purely a
/// performance choice — both AEADs are full-strength, and Hello/Welcome ride the pinned-TLS
/// control channel, so there is no downgrade surface.
pub const VIDEO_CAP_CHACHA20: u8 = 0x40;
/// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
/// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
@@ -60,6 +71,16 @@ pub const HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// trailing `host_caps` byte — no wire-layout change.
pub const HOST_CAP_CLIPBOARD: u8 = 0x02;
/// [`Welcome::host_caps`] bit: the host's active inject backend can type **committed text**
/// ([`InputKind::TextInput`](crate::input::InputKind::TextInput) — one Unicode scalar per event):
/// Windows (`KEYEVENTF_UNICODE`) and Linux wlroots (dynamic Unicode keymap on a dedicated virtual
/// keyboard); the KWin/libei/gamescope backends can only press layout keycodes, so those sessions
/// don't set it. A capable client routes its IME's committed text (autocorrect, gesture typing,
/// non-Latin scripts, emoji) through `TextInput` instead of lossy VK synthesis; absent the bit it
/// keeps the VK fallback. Packs into the existing trailing `host_caps` byte — no wire-layout
/// change; an older host ignores the unknown input tag anyway (input is lossy by design).
pub const HOST_CAP_TEXT_INPUT: u8 = 0x04;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
/// advertise this.
@@ -225,6 +246,8 @@ mod tests {
audio_channels: 2,
codec: CODEC_HEVC,
host_caps: HOST_CAP_GAMEPAD_STATE | HOST_CAP_CLIPBOARD,
cipher: 0,
key_chacha: None,
};
let got = Welcome::decode(&w.encode()).unwrap();
assert_eq!(got.host_caps & HOST_CAP_CLIPBOARD, HOST_CAP_CLIPBOARD);
+152 -3
View File
@@ -4,6 +4,7 @@ use super::*;
use crate::config::{
CompositorPref, Config, FecConfig, FecScheme, GamepadPref, Mode, ProtocolPhase, Role,
};
use crate::crypto::SessionKey;
use crate::error::{PunktfunkError, Result};
/// `client → host`: open the session, requesting a display mode (the host creates its
@@ -107,6 +108,13 @@ pub const HELLO_NAME_MAX: usize = 64;
/// (`steam:<appid>` / `custom:<12 hex>`); the cap just bounds an attacker-controlled field.
pub const HELLO_LAUNCH_MAX: usize = 128;
/// [`Welcome::cipher`] id: AES-128-GCM — the default session AEAD every peer speaks (and the
/// only one pre-cipher builds know).
pub const CIPHER_AES_128_GCM: u8 = 0;
/// [`Welcome::cipher`] id: ChaCha20-Poly1305 (RFC 8439) — negotiated via
/// [`VIDEO_CAP_CHACHA20`] for clients without hardware AES.
pub const CIPHER_CHACHA20_POLY1305: u8 = 1;
/// `host → client`: the complete session offer.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Welcome {
@@ -173,6 +181,22 @@ pub struct Welcome {
/// per-transition events otherwise). Appended after `codec` as a single trailing byte; an
/// older host that omits it decodes to `0` (no capabilities — legacy events only).
pub host_caps: u8,
/// The session AEAD the data plane seals with — [`CIPHER_AES_128_GCM`] (`0`, the default
/// every peer speaks) or [`CIPHER_CHACHA20_POLY1305`] (`1`). The host sets `1` ONLY toward
/// a client that advertised [`VIDEO_CAP_CHACHA20`] (the soft-AES armv7 targets). Appended
/// after `host_caps` at offset 68 and — unlike the earlier trailing fields — emitted only
/// when non-zero, so an AES session's Welcome stays **byte-identical** to the pre-cipher
/// wire form; an older host omits it (→ `0`, AES). Decode is fail-closed: an unknown id is
/// an `Err`, never a silent AES fallback — the host only picks a cipher this client
/// advertised, so an unknown id reaching us is a bug, and falling back would yield an
/// undecryptable session with a confusing failure signature.
pub cipher: u8,
/// The 256-bit ChaCha20-Poly1305 session key (RFC 8439 requires the full 32 bytes; wire
/// cost is once per handshake) — present iff `cipher == 1`, at offsets 69..101. The legacy
/// 16-byte `key` keeps its offset and stays independently random, so nothing downstream
/// ever observes an all-zero key. Decode rejects `cipher == 1` with fewer than 32 key
/// bytes following.
pub key_chacha: Option<[u8; 32]>,
}
/// `client → host`: data plane is bound, begin streaming.
@@ -366,6 +390,21 @@ impl Welcome {
b.push(self.codec);
// Host input caps at offset 67 — older clients stop before this → 0 (legacy input only).
b.push(self.host_caps);
// Session cipher at offset 68 + the 32-byte ChaCha key at 69..101 — emitted ONLY when a
// non-default cipher was negotiated, so an AES session's Welcome stays byte-identical
// to the pre-cipher wire form. The host only sets cipher toward a client that
// advertised VIDEO_CAP_CHACHA20, so an old client never sees these bytes at all.
debug_assert_eq!(
self.cipher == CIPHER_CHACHA20_POLY1305,
self.key_chacha.is_some(),
"key_chacha present iff cipher == 1"
);
if self.cipher != CIPHER_AES_128_GCM {
b.push(self.cipher);
if let Some(k) = &self.key_chacha {
b.extend_from_slice(k);
}
}
b
}
@@ -374,8 +413,10 @@ impl Welcome {
// scheme[22] pct[23] max_data[24..26] shard[26..28] encrypt[28] key[29..45]
// salt[45..49] frames[49..53] compositor[53] gamepad[54] bitrate_kbps[55..59]
// bit_depth[59] color.primaries[60] color.transfer[61] color.matrix[62] color.range[63]
// chroma_format[64] audio_channels[65] codec[66] (everything from compositor on is an
// optional trailing byte; an older host stops earlier).
// chroma_format[64] audio_channels[65] codec[66] host_caps[67] cipher[68]
// key_chacha[69..101] (everything from compositor on is an optional trailing byte; an
// older host stops earlier; cipher/key_chacha are present only when ChaCha was
// negotiated).
if b.len() < 53 || &b[0..4] != MAGIC {
return Err(PunktfunkError::InvalidArg("bad Welcome"));
}
@@ -385,6 +426,24 @@ impl Welcome {
key.copy_from_slice(&b[29..45]);
let mut salt = [0u8; 4];
salt.copy_from_slice(&b[45..49]);
// Session cipher at 68 — absent on an older host → AES-128-GCM. Fail-closed on
// anything else: `cipher == 1` with fewer than 32 key bytes must be an error (a silent
// AES fallback would yield an undecryptable session with a confusing failure
// signature), and an unknown id (≥ 2) reaching us is a bug — a host only picks a
// cipher this client advertised — never a legitimate negotiation.
let cipher = b.get(68).copied().unwrap_or(CIPHER_AES_128_GCM);
let key_chacha = match cipher {
CIPHER_AES_128_GCM => None,
CIPHER_CHACHA20_POLY1305 => {
let bytes = b
.get(69..101)
.ok_or(PunktfunkError::InvalidArg("bad Welcome"))?;
let mut k = [0u8; 32];
k.copy_from_slice(bytes);
Some(k)
}
_ => return Err(PunktfunkError::InvalidArg("bad Welcome")),
};
Ok(Welcome {
abi_version: u32at(4),
udp_port: u16at(8),
@@ -452,6 +511,8 @@ impl Welcome {
// Optional trailing host-caps byte — absent on an older host → 0 (no gamepad-state
// snapshots; the client keeps sending legacy per-transition events).
host_caps: b.get(67).copied().unwrap_or(0),
cipher,
key_chacha,
})
}
@@ -462,7 +523,12 @@ impl Welcome {
c.fec = self.fec;
c.shard_payload = self.shard_payload as usize;
c.encrypt = self.encrypt;
c.key = self.key;
// The negotiated AEAD: the ChaCha key when cipher == 1 (guaranteed present by decode —
// the `(1, None)` shape is unreachable off the wire), the legacy AES key otherwise.
c.key = match (self.cipher, self.key_chacha) {
(CIPHER_CHACHA20_POLY1305, Some(k)) => SessionKey::ChaCha20Poly1305(k),
_ => SessionKey::Aes128Gcm(self.key),
};
c.salt = self.salt;
// Client-side reassembler ceiling: p1_defaults' 64 MiB hostile-header memory bound is
// ~10x larger than any real access unit. Derive it from the negotiated rate instead:
@@ -531,6 +597,8 @@ mod tests {
audio_channels: 2,
codec: CODEC_H264, // exercise a non-default codec through the roundtrip
host_caps: HOST_CAP_GAMEPAD_STATE,
cipher: 0,
key_chacha: None,
};
assert_eq!(Welcome::decode(&w.encode()).unwrap(), w);
@@ -564,6 +632,81 @@ mod tests {
assert!(derived > (8 << 20) && derived < (64 << 20));
}
#[test]
fn welcome_cipher_negotiation_wire_and_back_compat() {
use crate::crypto::SessionKey;
let base = Welcome {
abi_version: 2,
udp_port: 7000,
mode: Mode {
width: 1920,
height: 1080,
refresh_hz: 60,
},
fec: FecConfig {
scheme: FecScheme::Gf16,
fec_percent: 20,
max_data_per_block: 4096,
},
shard_payload: 1200,
encrypt: true,
key: [7u8; 16],
salt: [9, 8, 7, 6],
frames: 0,
compositor: CompositorPref::Auto,
gamepad: GamepadPref::Auto,
bitrate_kbps: 50_000,
bit_depth: 8,
color: ColorInfo::SDR_BT709,
chroma_format: CHROMA_IDC_420,
audio_channels: 2,
codec: CODEC_HEVC,
host_caps: 0,
cipher: CIPHER_AES_128_GCM,
key_chacha: None,
};
// An AES session's Welcome is byte-identical to the pre-cipher wire form (68 bytes) —
// the old-client × new-host interop guarantee.
let enc = base.encode();
assert_eq!(enc.len(), 68);
assert_eq!(Welcome::decode(&enc).unwrap(), base);
// ChaCha roundtrip: cipher byte at 68, the 32-byte key at 69..101.
let k32: [u8; 32] = core::array::from_fn(|i| i as u8 + 1);
let cha = Welcome {
cipher: CIPHER_CHACHA20_POLY1305,
key_chacha: Some(k32),
..base
};
let cenc = cha.encode();
assert_eq!(cenc.len(), 68 + 1 + 32);
assert_eq!(Welcome::decode(&cenc).unwrap(), cha);
// A truncated old-host Welcome (no cipher byte) decodes to the AES default.
let old_host = Welcome::decode(&cenc[..68]).unwrap();
assert_eq!(old_host.cipher, CIPHER_AES_128_GCM);
assert_eq!(old_host.key_chacha, None);
// cipher == 1 with a missing / short key → Err, fail-closed (a silent AES fallback
// would yield an undecryptable session with a confusing failure signature).
assert!(Welcome::decode(&cenc[..69]).is_err());
assert!(Welcome::decode(&cenc[..100]).is_err());
// An unknown cipher id (≥ 2) → Err: the host only picks a cipher we advertised, so an
// unknown id reaching us is a bug, never a legitimate negotiation.
let mut bad = cenc.clone();
bad[68] = 2;
assert!(Welcome::decode(&bad).is_err());
// session_config maps both variants onto the data-plane key, and both validate.
let aes_cfg = base.session_config(Role::Client);
assert_eq!(aes_cfg.key, SessionKey::Aes128Gcm([7u8; 16]));
aes_cfg.validate().expect("AES config validates");
let cha_cfg = cha.session_config(Role::Client);
assert_eq!(cha_cfg.key, SessionKey::ChaCha20Poly1305(k32));
cha_cfg.validate().expect("ChaCha config validates");
}
#[test]
fn codec_negotiation_and_back_compat() {
// resolve_codec precedence (HEVC > AV1 > H.264), no preference (0).
@@ -656,6 +799,8 @@ mod tests {
audio_channels: 2,
codec: CODEC_PYROWAVE,
host_caps: 0,
cipher: 0,
key_chacha: None,
}
.encode(),
)
@@ -726,6 +871,8 @@ mod tests {
audio_channels: 2,
codec: CODEC_H264,
host_caps: 0,
cipher: 0,
key_chacha: None,
}
.encode(),
)
@@ -831,6 +978,8 @@ mod tests {
audio_channels: 6, // 5.1 — exercises the non-default trailing byte
codec: CODEC_HEVC,
host_caps: HOST_CAP_GAMEPAD_STATE,
cipher: 0,
key_chacha: None,
};
let wenc = w.encode();
assert_eq!(wenc.len(), 68); // 60 base + 4 colour + chroma + audio-channels + codec + host-caps
+16 -2
View File
@@ -33,6 +33,12 @@ pub const PAIR_APPROVAL_TIMEOUT_CLOSE_CODE: u32 = 0x65;
pub const PAIR_SUPERSEDED_CLOSE_CODE: u32 = 0x66;
/// The client's wire (protocol) version does not match the host's — one side needs updating.
pub const WIRE_VERSION_CLOSE_CODE: u32 = 0x67;
/// The host admitted the connection but could not stand the stream session up (compositor /
/// capture / encoder setup failed host-side). The close reason bytes carry the specific error
/// text for logs/diagnostics; clients render a stable "host-side failure" sentence. Before this
/// code, a setup failure reached the client as a bare dropped connection ("control stream
/// finished mid-frame") — indistinguishable from transport trouble.
pub const SETUP_FAILED_CLOSE_CODE: u32 = 0x68;
/// Why a host turned a connection away, decoded from the QUIC application close code — the
/// client-side view of [`PAIR_NOT_ARMED_CLOSE_CODE`]..[`WIRE_VERSION_CLOSE_CODE`] plus
@@ -59,6 +65,9 @@ pub enum RejectReason {
WireVersionMismatch,
/// The host refused admission because a conflicting session is live.
Busy,
/// The host admitted the connection but failed to start the stream session (host-side
/// setup error — the host log has the specific cause).
SetupFailed,
}
impl RejectReason {
@@ -75,6 +84,7 @@ impl RejectReason {
PAIR_SUPERSEDED_CLOSE_CODE => Self::Superseded,
WIRE_VERSION_CLOSE_CODE => Self::WireVersionMismatch,
REJECT_BUSY_CLOSE_CODE => Self::Busy,
SETUP_FAILED_CLOSE_CODE => Self::SetupFailed,
_ => return None,
})
}
@@ -91,6 +101,7 @@ impl RejectReason {
Self::Superseded => PAIR_SUPERSEDED_CLOSE_CODE,
Self::WireVersionMismatch => WIRE_VERSION_CLOSE_CODE,
Self::Busy => REJECT_BUSY_CLOSE_CODE,
Self::SetupFailed => SETUP_FAILED_CLOSE_CODE,
}
}
@@ -107,6 +118,7 @@ impl RejectReason {
Self::Superseded => "superseded",
Self::WireVersionMismatch => "wire-version",
Self::Busy => "busy",
Self::SetupFailed => "setup-failed",
}
}
}
@@ -125,6 +137,7 @@ impl std::fmt::Display for RejectReason {
Self::Superseded => "a newer request from this device replaced this one",
Self::WireVersionMismatch => "client and host versions do not match",
Self::Busy => "the host is busy with another session",
Self::SetupFailed => "the host could not start the stream session",
})
}
}
@@ -133,7 +146,7 @@ impl std::fmt::Display for RejectReason {
mod tests {
use super::*;
const ALL: [RejectReason; 9] = [
const ALL: [RejectReason; 10] = [
RejectReason::PairingNotArmed,
RejectReason::PairingBoundToOtherDevice,
RejectReason::PairingRateLimited,
@@ -143,6 +156,7 @@ mod tests {
RejectReason::Superseded,
RejectReason::WireVersionMismatch,
RejectReason::Busy,
RejectReason::SetupFailed,
];
#[test]
@@ -164,7 +178,7 @@ mod tests {
fn foreign_codes_stay_untyped() {
// Bare closes, the client's own pair-done codes, and the deliberate-end codes must
// never read as a host rejection.
for code in [0u32, 1, 0x41, 0x51, 0x52, 0x5f, 0x68, u32::MAX] {
for code in [0u32, 1, 0x41, 0x51, 0x52, 0x5f, 0x69, u32::MAX] {
assert_eq!(RejectReason::from_close_code(code), None);
}
}
+3 -2
View File
@@ -781,6 +781,7 @@ impl Session {
mod wire_equivalence_tests {
use super::*;
use crate::config::{FecConfig, FecScheme, ProtocolPhase};
use crate::crypto::SessionKey;
use crate::transport::loopback_pair;
fn host_cfg(scheme: FecScheme, fec_percent: u8, encrypt: bool) -> Config {
@@ -798,7 +799,7 @@ mod wire_equivalence_tests {
shard_payload: 64,
max_frame_bytes: 8 * 1024 * 1024,
encrypt,
key: [7u8; 16],
key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [3, 1, 4, 1],
loopback_drop_period: 0,
}
@@ -930,7 +931,7 @@ mod wire_equivalence_tests {
shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024,
encrypt: false,
key: [0u8; 16],
key: SessionKey::Aes128Gcm([0u8; 16]),
salt: [0u8; 4],
loopback_drop_period: 0,
};
+26 -1
View File
@@ -5,6 +5,7 @@
use proptest::prelude::*;
use punktfunk_core::config::{Config, FecConfig, FecScheme, ProtocolPhase, Role};
use punktfunk_core::crypto::SessionKey;
use punktfunk_core::fec::coder_for;
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::session::Session;
@@ -25,7 +26,7 @@ fn config(role: Role, scheme: FecScheme, encrypt: bool, drop_period: u32) -> Con
shard_payload: 1024,
max_frame_bytes: 8 * 1024 * 1024,
encrypt,
key: [7u8; 16],
key: SessionKey::Aes128Gcm([7u8; 16]),
salt: [1, 2, 3, 4],
loopback_drop_period: drop_period,
}
@@ -101,6 +102,30 @@ fn encrypted_stream_recovers_under_loss() {
assert_eq!(stats.frames_completed, frames.len() as u64);
}
/// The negotiated ChaCha20-Poly1305 session cipher through the same lossy full-stream path:
/// loss/replay behavior is cipher-independent (the replay window keys off the authenticated
/// seq), so recovery must be byte-identical to the AES run above.
#[test]
fn chacha20_encrypted_stream_recovers_under_loss() {
let frames = sample_frames();
let mk = |role| {
let mut c = config(role, FecScheme::Gf16, true, 8);
c.key = SessionKey::ChaCha20Poly1305([7u8; 32]);
c
};
let (host_tp, client_tp) = loopback_pair(8, 0);
let mut host = Session::new(mk(Role::Host), Box::new(host_tp)).unwrap();
let mut client = Session::new(mk(Role::Client), Box::new(client_tp)).unwrap();
for (i, frame) in frames.iter().enumerate() {
host.submit_frame(frame, i as u64 * 1_000_000, 0).unwrap();
let got = client
.poll_frame()
.expect("frame should recover despite loss");
assert_eq!(&got.data, frame, "frame {i} mismatched after recovery");
}
assert!(client.stats().fec_recovered_shards > 0);
}
#[test]
fn lossless_stream_is_exact() {
let frames = sample_frames();
+9 -3
View File
@@ -26,7 +26,10 @@ pub use pf_capture::{dxgi, synthetic_nv12};
/// capture→encode cycle). Resolved here (the host facade) and threaded in, so the edge stays one-way
/// (plan §2.4 / §W6).
#[cfg(target_os = "linux")]
fn zero_copy_policy(pyrowave_session: bool) -> pf_capture::ZeroCopyPolicy {
fn zero_copy_policy(
pyrowave_session: bool,
native_nv12_session: bool,
) -> pf_capture::ZeroCopyPolicy {
let backend_is_vaapi = crate::encode::linux_zero_copy_is_vaapi();
// The raw-dmabuf passthrough serves a PyroWave session on ANY vendor (the wavelet encoder's
// own Vulkan device imports the dmabuf) — per-session from the negotiated codec, plus the
@@ -56,6 +59,7 @@ fn zero_copy_policy(pyrowave_session: bool) -> pf_capture::ZeroCopyPolicy {
backend_is_gpu: crate::encode::resolved_backend_is_gpu(),
pyrowave_session,
pyrowave_modifiers,
native_nv12_session,
}
}
@@ -70,7 +74,9 @@ pub fn open_portal_monitor(want_hdr: bool) -> Result<Box<dyn Capturer>> {
let anchored = crate::inject::default_backend() == crate::inject::Backend::Libei;
// Monitor mirrors never carry the native PyroWave plane (GameStream protocol) — per-session
// passthrough is virtual-output-only; the global encoder-pref lever still applies inside.
pf_capture::open_portal_monitor(anchored, want_hdr, zero_copy_policy(false))
// Native NV12 stays off too: the mirror path doesn't resolve the codec here, and the desktop
// compositors it mirrors (GNOME/KWin) don't produce NV12 anyway.
pf_capture::open_portal_monitor(anchored, want_hdr, zero_copy_policy(false, false))
}
#[cfg(not(target_os = "linux"))]
@@ -101,7 +107,7 @@ pub fn capture_virtual_output(
vout.keepalive,
want.gpu,
want.chroma_444,
zero_copy_policy(want.pyrowave),
zero_copy_policy(want.pyrowave, want.nv12_native),
)
}
+30
View File
@@ -30,6 +30,36 @@ pub fn input_test() -> Result<()> {
y,
flags: 0,
};
// `PUNKTFUNK_INPUT_TEST_ABS=WxH` (e.g. 1280x800): exercise ABSOLUTE pointer moves instead —
// steps through the corners + center of the given surface, 1s apart, so an observer
// (`DISPLAY=:0 xdotool getmouselocation`) can verify each jump. This is the degraded-touch
// path (touch → MouseMoveAbs), so it validates game-mode touch without a client.
if let Ok(dims) = std::env::var("PUNKTFUNK_INPUT_TEST_ABS") {
let (w, h) = dims
.split_once('x')
.and_then(|(w, h)| Some((w.parse::<u32>().ok()?, h.parse::<u32>().ok()?)))
.unwrap_or((1280, 800));
let flags = (w << 16) | (h & 0xffff);
let pts = [
(100, 100),
(w as i32 - 100, 100),
(w as i32 - 100, h as i32 - 100),
(100, h as i32 - 100),
(w as i32 / 2, h as i32 / 2),
];
tracing::info!(w, h, "input-test: ABS mode — corners + center, 1s apart");
for (x, y) in pts {
let mut e = ev(InputKind::MouseMoveAbs, 0, x, y);
e.flags = flags;
if let Err(err) = inj.inject(&e) {
tracing::warn!(error = %format!("{err:#}"), "input-test: abs inject failed");
}
tracing::info!(x, y, "input-test: abs move emitted");
std::thread::sleep(Duration::from_secs(1));
}
tracing::info!("input-test: done (abs)");
return Ok(());
}
tracing::info!(
"input-test: injecting a mouse square + 'A'/click taps for ~8s (watch wev / focused app)"
);
+34 -2
View File
@@ -35,7 +35,24 @@ pub fn decode(plaintext: &[u8]) -> Vec<InputEvent> {
if plaintext.len() < 4 || u16::from_le_bytes([plaintext[0], plaintext[1]]) != INPUT_DATA_TYPE {
return Vec::new();
}
decode_input_packet(&plaintext[4..]).into_iter().collect()
let p = &plaintext[4..];
// UTF-8 text (Moonlight's client-side keyboard commit) expands to one `TextInput` event per
// Unicode scalar — the only magic yielding more than one event, so it's handled before the
// single-event dispatch. Injected the same way as the native plane's IME text.
if p.len() >= 8 && u32::from_le_bytes([p[4], p[5], p[6], p[7]]) == MAGIC_UTF8 {
// NV_INPUT_HEADER.size (BE, excludes itself) counts magic + body.
let size = u32::from_be_bytes([p[0], p[1], p[2], p[3]]) as usize;
let body_len = size.saturating_sub(4).min(p.len() - 8);
return match std::str::from_utf8(&p[8..8 + body_len]) {
Ok(s) => s
.chars()
.filter(|c| !c.is_control())
.map(|c| ev(InputKind::TextInput, c as u32, 0, 0, 0))
.collect(),
Err(_) => Vec::new(),
};
}
decode_input_packet(p).into_iter().collect()
}
fn decode_input_packet(p: &[u8]) -> Option<InputEvent> {
@@ -89,7 +106,7 @@ fn decode_input_packet(p: &[u8]) -> Option<InputEvent> {
modifiers | crate::inject::KEY_FLAG_SEMANTIC_VK,
)
}
// UTF-8 text, gamepad, pen, touch, haptics — not yet injected.
// Gamepad, pen, touch, haptics — not yet injected. (UTF-8 text is handled in `decode`.)
_ => return None,
})
}
@@ -144,6 +161,21 @@ mod tests {
assert_eq!(ev[0].flags, 0x04 | crate::inject::KEY_FLAG_SEMANTIC_VK);
}
#[test]
fn decodes_utf8_text_per_scalar() {
// "aß😀" — ASCII, Latin-1, and an astral scalar; one TextInput event per scalar.
let pt = wrap(MAGIC_UTF8, "aß😀".as_bytes());
let ev = decode(&pt);
assert_eq!(ev.len(), 3);
assert!(ev.iter().all(|e| e.kind == InputKind::TextInput));
assert_eq!(ev[0].code, 'a' as u32);
assert_eq!(ev[1].code, 'ß' as u32);
assert_eq!(ev[2].code, 0x1F600);
// Truncated / invalid UTF-8 decodes to nothing rather than mojibake.
let bad = wrap(MAGIC_UTF8, &[0xff, 0xfe]);
assert!(decode(&bad).is_empty());
}
#[test]
fn ignores_non_input_type() {
let mut pt = vec![0x00, 0x02]; // type 0x0200 (keepalive)
@@ -60,6 +60,9 @@ pub fn start(
// Same scheduling posture as the native path's capture/encode thread (Linux nice -10 /
// Windows HIGHEST + session tuning) — GameStream previously ran unboosted on Linux.
crate::native::boost_thread_priority(true);
// A GameStream viewer may be video-only too — hold the suspend/idle inhibitor for
// this stream's lifetime (plane parity with the native LiveSessionGuard).
let _sleep = crate::sleep_inhibit::hold();
tracing::info!(?cfg, "video stream starting");
// Lifecycle events + the script-facing marker file, plane parity with the native loop
// (RFC §4): `announce` emits `stream.started`/`stream.stopped` and holds the marker for
@@ -86,6 +89,12 @@ pub fn start(
crate::events::emit(crate::events::EventKind::ClientConnected {
client: event_client.clone(),
});
// GPU clock pin (Linux, opt-in `PUNKTFUNK_PIN_CLOCKS`): hold the box-wide vendor clock
// floor while this compat-plane stream runs, refcounted with every other live session
// across both planes. Released when the closure exits (stream stopped) — so idle clocks
// aren't pinned between Moonlight sessions. No-op off Linux / when the flag is unset.
#[cfg(target_os = "linux")]
let _clock_pin = crate::gpuclocks::session_pin();
let result = run(
cfg,
app.as_ref(),
+97 -21
View File
@@ -2,11 +2,15 @@
//! plan Tier 1B): the driver's adaptive P-state ramps clocks down between bursty encode frames,
//! so every frame re-pays a spin-up. This is NOT theoretical — measured on the 780M (VCN 4),
//! a 1440p HEVC encode takes ~4.4 ms/frame with the clocks hot (120 fps pacing) but ~8 ms/frame
//! at a 60 fps duty cycle: the sag doubles per-frame encode latency.
//! at a 60 fps duty cycle: the sag doubles per-frame encode latency. So the pin is worth holding
//! only while a client is actively streaming: it is armed on the first live client and released
//! when the last one disconnects (refcounted across both streaming planes — see [`session_pin`]),
//! leaving the driver's idle power management alone the rest of the time.
//!
//! **AMD** (`PUNKTFUNK_PIN_CLOCKS=1`, root-gated by sysfs ownership): write `high` into each
//! amdgpu card's `power_dpm_force_performance_level` for the host lifetime, restoring the prior
//! value on exit. Non-root gets EACCES → logged once with the privilege recipe. Deliberately
//! amdgpu card's `power_dpm_force_performance_level` while a client is streaming, restoring the
//! prior value when the last client disconnects. Non-root gets EACCES → logged once with the
//! privilege recipe. Deliberately
//! opt-in: it defeats power management box-wide and is wrong on battery (Steam Deck!).
//!
//! **NVIDIA** — two independent halves, both no-ops off NVIDIA:
@@ -28,14 +32,16 @@
//! while leaving boost headroom — NVIDIA's own latency guidance is "raise the floor, don't pin
//! the max" (locking above base just gets throttled; a max pin only burns idle watts). Non-root
//! callers get `NVML_ERROR_NO_PERMISSION` — logged once with the privilege recipe, then the
//! host runs unpinned. The pin is undone on drop (host exit); after a crash it persists until
//! driver reload/reboot, which the reset-before-pin on the next start self-heals. Deliberately
//! host runs unpinned. The pin is undone on drop (when the last client disconnects); after a
//! crash it persists until driver reload/reboot, which the reset-before-pin on the next arm
//! self-heals. Deliberately
//! NOT default-on: it defeats idle downclocking for the whole box and is wrong on
//! battery-powered hosts.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use std::os::raw::{c_char, c_int, c_uint, c_void};
use std::sync::{Mutex, OnceLock};
/// `nvmlDevice_t` — an opaque driver handle.
type NvmlDevice = *mut c_void;
@@ -133,8 +139,9 @@ struct AmdPin {
restore: String,
}
/// Host-lifetime guard: holds the armed clock pins (NVML floor and/or amdgpu perf level) and
/// undoes them on drop.
/// Holds the armed clock pins (NVML floor and/or amdgpu perf level) and undoes them on drop. Owned
/// by the [`session_pin`] refcount: constructed when the first live client arms the pin, dropped
/// (clocks restored) when the last one disconnects.
pub struct ClockGuard {
nvml: Option<NvmlPin>,
amd: Vec<AmdPin>,
@@ -142,9 +149,10 @@ pub struct ClockGuard {
// SAFETY: `ClockGuard` holds opaque NVML device handles + resolved fn pointers from the loaded
// driver library (plus plain sysfs paths/strings). NVML is documented thread-safe, the handles are
// plain driver tokens with no thread affinity, and the guard is only ever *moved* (held in `main`,
// dropped once at exit) and used through `&mut`/ownership — never shared. Transfer across threads
// is therefore sound.
// plain driver tokens with no thread affinity, and the guard is only ever *moved* (into the
// `pin_refcount` mutex when armed, taken back out and dropped when the last client disconnects) and
// used through exclusive ownership behind that mutex — never shared. Transfer across threads is
// therefore sound.
unsafe impl Send for ClockGuard {}
impl Drop for ClockGuard {
@@ -182,22 +190,89 @@ impl Drop for ClockGuard {
}
/// Startup hook for the host subcommands (`serve` / `punktfunk1-host`): install the NVIDIA P2-cap
/// application profile and, when `PUNKTFUNK_PIN_CLOCKS` is set, arm the vendor clock pin (NVML
/// core-clock floor / amdgpu `high` performance level). Returns the guard keeping the pins for
/// the host lifetime. `None` when nothing was armed.
pub fn on_host_start() -> Option<ClockGuard> {
/// application profile — the process-scoped, no-root half of the NVIDIA lever, which the driver
/// only acts on once the host holds a live CUDA/NVENC context (i.e. during a session).
///
/// The vendor clock *pin* is deliberately NOT armed here anymore: held for the whole host lifetime
/// it kept the box's clocks hot even with no client connected. It is now refcounted per live client
/// via [`session_pin`] (armed on both streaming planes), so idle clocks are left to the driver's
/// power management until someone actually streams.
pub fn on_host_start() {
if nvidia_present() {
ensure_cuda_perf_profile();
}
if !flag_truthy("PUNKTFUNK_PIN_CLOCKS") {
return None;
}
/// The box-wide clock-pin refcount, shared across BOTH streaming planes (native + GameStream): the
/// vendor pin is a single global GPU setting, so N concurrent sessions share ONE pin — armed when
/// the first client goes live, released when the last one leaves.
struct PinRefcount {
/// Number of live [`SessionClockPin`] handles.
live: usize,
/// The armed pins, present iff `live > 0` and something was actually pinnable.
guard: Option<ClockGuard>,
}
fn pin_refcount() -> &'static Mutex<PinRefcount> {
static STATE: OnceLock<Mutex<PinRefcount>> = OnceLock::new();
STATE.get_or_init(|| {
Mutex::new(PinRefcount {
live: 0,
guard: None,
})
})
}
/// RAII handle that keeps the box-wide clock pin armed while it is alive. Obtain one per live client
/// session on either plane via [`session_pin`]; when the last outstanding handle drops, the pin is
/// released and the driver's idle downclocking resumes. A no-op handle when `PUNKTFUNK_PIN_CLOCKS`
/// is unset (the opt-in gate) — the refcount only ticks for sessions that asked for pinning.
pub struct SessionClockPin {
/// Whether this handle actually incremented the refcount (false = opt-in gate off → no-op).
counted: bool,
}
/// Arm the box-wide clock pin for one live client session, refcounted across every plane. Returns
/// an RAII handle; the pin is released when the last handle drops. A no-op (returns immediately,
/// touches no GPU state) unless `PUNKTFUNK_PIN_CLOCKS` is set.
pub fn session_pin() -> SessionClockPin {
if !flag_truthy("PUNKTFUNK_PIN_CLOCKS") {
return SessionClockPin { counted: false };
}
let mut state = pin_refcount().lock().unwrap();
state.live += 1;
if state.live == 1 {
// 0 → 1: the first live client — arm the vendor pin (reset-before-pin inside `pin_nvidia`
// heals a stale pin from a crashed previous run).
let nvml = if nvidia_present() { pin_nvidia() } else { None };
let amd = pin_amdgpu();
if nvml.is_none() && amd.is_empty() {
return None;
}
state.guard = if nvml.is_none() && amd.is_empty() {
None // nothing pinnable (no perms / no supported GPU) — the session streams unpinned
} else {
Some(ClockGuard { nvml, amd })
};
}
SessionClockPin { counted: true }
}
impl Drop for SessionClockPin {
fn drop(&mut self) {
if !self.counted {
return;
}
// Take the guard out under the lock but drop it *outside* — releasing the pin does NVML +
// sysfs I/O we don't want to hold the refcount lock across.
let release = {
let mut state = pin_refcount().lock().unwrap();
state.live = state.live.saturating_sub(1);
if state.live == 0 {
state.guard.take()
} else {
None
}
};
drop(release);
}
}
/// Force every amdgpu card's DPM performance level to `high` for the session — the encode-latency
@@ -238,7 +313,7 @@ fn pin_amdgpu() -> Vec<AmdPin> {
card = %name,
was = %prev,
"amdgpu performance level pinned to high (encode clock sag removed) — \
restored on host exit"
restored when the last client disconnects"
);
pins.push(AmdPin {
path,
@@ -327,7 +402,8 @@ fn pin_nvidia() -> Option<NvmlPin> {
}
tracing::info!(
devices = pinned.len(),
"NVIDIA core-clock floor armed (min=TDP/base, max=boost) — released on host exit"
"NVIDIA core-clock floor armed (min=TDP/base, max=boost) — released when the last \
client disconnects"
);
Some(NvmlPin { nvml, pinned })
}
+38 -8
View File
@@ -72,6 +72,7 @@ mod send_pacing;
mod service;
mod session_plan;
mod session_status;
mod sleep_inhibit;
mod spike;
mod stats_recorder;
// The plugin store: signed catalogs, tiered trust, and install/uninstall jobs that run through the
@@ -246,14 +247,17 @@ fn real_main() -> Result<()> {
crate::capture::dxgi::install_gpu_pref_hook();
}
// NVIDIA clock hygiene (Linux, host subcommands only): install the P2-cap driver profile and,
// under PUNKTFUNK_PIN_CLOCKS, hold the NVML core-clock floor for the host lifetime (reset on
// exit via the guard's Drop). No-op off NVIDIA / on the tool subcommands.
// NVIDIA clock hygiene (Linux, host subcommands only): install the P2-cap driver profile. The
// vendor clock *pin* (PUNKTFUNK_PIN_CLOCKS) is no longer held for the host lifetime — it is
// armed per live client via `gpuclocks::session_pin()` on both streaming planes, so idle clocks
// are left alone while nobody is connected. No-op off NVIDIA / on the tool subcommands.
#[cfg(target_os = "linux")]
let _nv_clocks = match args.first().map(String::as_str) {
Some("serve") | Some("punktfunk1-host") => gpuclocks::on_host_start(),
_ => None,
};
if matches!(
args.first().map(String::as_str),
Some("serve") | Some("punktfunk1-host")
) {
gpuclocks::on_host_start();
}
match args.first().map(String::as_str) {
// The host: the native punktfunk/1 plane + management API by default (secure), and — with
@@ -322,7 +326,33 @@ fn real_main() -> Result<()> {
// `PUNKTFUNK_HDR_SHADER_P010` colour math without green-screening a live HDR stream. Prints
// PASS/FAIL + max Y/Cb/Cr error.
#[cfg(target_os = "windows")]
Some("hdr-p010-selftest") => crate::capture::dxgi::hdr_p010_selftest(),
Some("hdr-p010-selftest") => {
// Optional args: a `WxH` size (default 64x64 — pass the real capture size: heights
// like 1080 are NOT 16-aligned and exercise a different driver path) and a GPU
// vendor (`intel`|`nvidia`|`amd` — dual-GPU boxes otherwise test the default
// adapter, which may not be the one that encodes).
let mut size = (64u32, 64u32);
let mut vendor = None;
for a in args.iter().skip(2) {
match a.as_str() {
"intel" => vendor = Some(0x8086),
"nvidia" => vendor = Some(0x10de),
"amd" => vendor = Some(0x1002),
s => {
let parsed = s
.split_once('x')
.and_then(|(w, h)| Some((w.parse().ok()?, h.parse().ok()?)));
match parsed {
Some(wh) => size = wh,
None => anyhow::bail!(
"hdr-p010-selftest: unrecognized arg {s:?} (want WxH or intel|nvidia|amd)"
),
}
}
}
}
crate::capture::dxgi::hdr_p010_selftest_at(size.0, size.1, vendor)
}
// Linux HDR readiness probe (GNOME 50+ portal path): prints whether a monitor is currently
// in BT.2100 (HDR) colour mode, whether the NVENC/VAAPI backend probes Main10 for
// HEVC/AV1, and the GameStream HDR capability the two combine into — the "why isn't my
+58 -7
View File
@@ -423,6 +423,10 @@ pub(crate) async fn serve(
// permit's lifetime: it's released while a knock is parked for delegated approval and
// re-acquired on approval, so the hold is no longer a simple closure-scoped binding.
let sem_session = sem.clone();
// Kept for the error path below: `serve_session` consumes `conn`, but a setup failure
// must still close the connection with a typed reason (quinn connections are cheap
// Arc-handle clones).
let conn_err = conn.clone();
sessions.spawn(async move {
match serve_session(
conn,
@@ -439,9 +443,29 @@ pub(crate) async fn serve(
)
.await
{
Ok(()) => tracing::info!(%peer, "session complete"),
Ok(Served::Session) => tracing::info!(%peer, "session complete"),
Ok(Served::ProbeClose) => tracing::debug!(
%peer,
"closed before the control handshake (reachability probe)"
),
Err(e) => {
tracing::warn!(%peer, error = %format!("{e:#}"), "session ended with error")
// Make the failure legible to the client (the [`close_rejected`] discipline,
// extended to EVERY session error): a setup failure that just drops the
// connection reaches the client as a bare close mid-control-frame ("control
// stream finished mid-frame") — indistinguishable from transport trouble.
// Close with the typed setup-failed code, carrying the error text in the
// reason bytes for client-side logs. When a gate already closed with its own
// typed code, or the peer closed first, this close is a no-op (first wins).
let detail = format!("{e:#}");
let mut cut = detail.len().min(256);
while !detail.is_char_boundary(cut) {
cut -= 1;
}
conn_err.close(
punktfunk_core::reject::SETUP_FAILED_CLOSE_CODE.into(),
&detail.as_bytes()[..cut],
);
tracing::warn!(%peer, error = %detail, "session ended with error")
}
}
});
@@ -629,6 +653,15 @@ type AudioCapSlot = Arc<std::sync::Mutex<Option<Box<dyn crate::audio::AudioCaptu
/// connection (the host stops waiting at once).
const PENDING_APPROVAL_WAIT: std::time::Duration = std::time::Duration::from_secs(180);
/// How a served connection ended. A peer that completes the QUIC handshake and closes cleanly
/// (code 0) without ever opening the control stream is a reachability probe (the clients'
/// hosts-page "online" pips / `--reachable`) or an abandoned connect — routine, and logged
/// quietly: as a WARN it buried the real failures in a wake-on-LAN triage log.
enum Served {
Session,
ProbeClose,
}
/// One client session: handshake → input/audio planes → data plane until done/disconnect.
/// Everything torn down on return (RAII: virtual output, encoder, threads via channel close).
/// A connection whose first message is a PairRequest runs the pairing ceremony instead.
@@ -651,14 +684,23 @@ async fn serve_session(
// parked knock can't hold a streaming slot. `sem` is the pool it re-acquires from.
mut permit: tokio::sync::OwnedSemaphorePermit,
sem: Arc<tokio::sync::Semaphore>,
) -> Result<()> {
) -> Result<Served> {
let peer = conn.remote_address();
// First message decides what this connection is: a pairing ceremony or a session.
let (mut send, mut recv) = tokio::time::timeout(HANDSHAKE_TIMEOUT, conn.accept_bi())
let (mut send, mut recv) = match tokio::time::timeout(HANDSHAKE_TIMEOUT, conn.accept_bi())
.await
.map_err(|_| anyhow!("control stream timeout"))?
.context("accept control stream")?;
{
// A clean close before any control stream: a reachability probe / abandoned connect,
// not a failed session (see [`Served::ProbeClose`]).
Err(quinn::ConnectionError::ApplicationClosed(ref ac))
if ac.error_code == quinn::VarInt::from_u32(0) =>
{
return Ok(Served::ProbeClose);
}
r => r.context("accept control stream")?,
};
let first = tokio::time::timeout(HANDSHAKE_TIMEOUT, io::read_msg(&mut recv))
.await
.map_err(|_| anyhow!("first message timeout"))??;
@@ -709,7 +751,9 @@ async fn serve_session(
}
*last = Some(std::time::Instant::now());
}
return pair_ceremony(&conn, send, recv, req, host_fp, np, &pin).await;
return pair_ceremony(&conn, send, recv, req, host_fp, np, &pin)
.await
.map(|()| Served::Session);
}
// Pairing gate for a session Hello (a PairRequest was handled above). Lifted OUT of the
@@ -1175,6 +1219,13 @@ async fn serve_session(
launch: hello.launch.clone(),
plane: crate::events::Plane::Native,
});
// GPU clock pin (Linux, opt-in `PUNKTFUNK_PIN_CLOCKS`): hold the box-wide vendor clock floor for
// as long as THIS session streams, refcounted with every other live session across both planes.
// RAII like the marker above — armed on the first live client, released when the last one
// disconnects, so idle clocks aren't pinned while nobody is connected. No-op off Linux / when
// the flag is unset.
#[cfg(target_os = "linux")]
let _clock_pin = crate::gpuclocks::session_pin();
// The session's launch, threaded into the data plane. Windows carries the store-qualified id
// (spawned into the interactive user session once capture is live); other hosts resolve the id
// to its shell command HERE against the host's own library — a client can only ever pick an
@@ -1392,7 +1443,7 @@ async fn serve_session(
// host-managed gamescope path on a box that autologs into gaming mode (Bazzite default), put the
// TV's gaming session back so it's the default when no one is streaming.
crate::vdisplay::restore_managed_session();
result
result.map(|()| Served::Session)
}
/// Backoff between reopen attempts after a host-lifetime service's backend (a capturer) fails
+17 -1
View File
@@ -92,8 +92,24 @@ pub(super) fn resolve_compositor(
let available = crate::vdisplay::available();
let chosen = match pick_compositor(pref, &available, detected) {
Some(c) => c,
// No live session, but the MANAGED gamescope infra exists (SteamOS's
// `gamescope-session`, Bazzite's `gamescope-session-plus`): route to the gamescope
// backend anyway — its managed path stands the session up from nothing at the
// client's mode (drop-in takeover / session relaunch), so a dead gaming session
// self-heals on the next connect instead of bouncing every client until someone
// restarts it by hand. (The trap that motivated this: a headless SteamOS box whose
// gamescope died — every connect failed "no usable compositor" even though the
// takeover could rebuild it.) Not under an operator pin: an explicit
// `PUNKTFUNK_COMPOSITOR` keeps its exact, hand-configured meaning.
None if !overridden && crate::vdisplay::managed_session_available() => {
tracing::info!(
"no live graphical session — managed gamescope infra present; routing to \
the managed takeover to revive the session"
);
Compositor::Gamescope
}
None => {
// No live session — the state a compositor crash leaves behind (gnome-shell
// The state a compositor crash leaves behind (gnome-shell
// SIGSEGV → GDM greeter, whose auto-login is once-per-boot). If the operator
// configured a recovery hook, fire it (debounced) and tell the client to retry:
// its next knock lands in the recovered desktop.
@@ -354,6 +354,28 @@ pub(super) async fn negotiate(
// just follow.
let mut salt = [0u8; 4];
rand::thread_rng().fill_bytes(&mut salt);
// Session AEAD: ChaCha20-Poly1305 when the client asked for it (VIDEO_CAP_CHACHA20 — the
// soft-AES armv7 targets, whose GCM decrypt caps at ~100 Mbps) and the operator
// kill-switch allows (PUNKTFUNK_CHACHA20, default on — pure rollout safety; perf-only,
// both AEADs are full-strength). The fresh-per-session discipline above applies to this
// key identically; the legacy 16-byte `key` stays independently random so nothing
// downstream ever observes an all-zero key.
let client_wants_chacha = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_CHACHA20 != 0;
let chacha = client_wants_chacha && pf_host_config::config().chacha20;
let key_chacha = chacha.then(|| {
let mut k = [0u8; 32];
rand::thread_rng().fill_bytes(&mut k);
k
});
tracing::info!(
cipher = if chacha {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
client_wants_chacha,
"session cipher"
);
let welcome = Welcome {
abi_version: punktfunk_core::WIRE_VERSION,
udp_port,
@@ -422,7 +444,26 @@ pub(super) async fn negotiate(
punktfunk_core::quic::HOST_CAP_CLIPBOARD
} else {
0
}
// Committed-text injection (InputKind::TextInput): only where the session's inject
// backend can actually type text — Windows SendInput (KEYEVENTF_UNICODE) and the
// Linux wlroots virtual keyboard (dynamic Unicode keymap). Clients without the bit
// keep their VK-synthesis fallback for IME text.
| if crate::inject::text_input_supported() {
punktfunk_core::quic::HOST_CAP_TEXT_INPUT
} else {
0
},
// The negotiated session AEAD (resolved above) + its 32-byte key toward a ChaCha
// client; toward everyone else cipher 0 keeps the Welcome byte-identical to the
// pre-cipher wire form. The host's own data plane picks the cipher up via
// `welcome.session_config` — no other host change.
cipher: if chacha {
punktfunk_core::quic::CIPHER_CHACHA20_POLY1305
} else {
punktfunk_core::quic::CIPHER_AES_128_GCM
},
key_chacha,
};
io::write_msg(send, &welcome.encode()).await?;
bringup.mark("welcome");
+74 -12
View File
@@ -1111,6 +1111,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
plan,
&quit,
&stop,
8,
Some(bringup.as_ref()),
)?;
// Setup done — the IDD-push setup lock releases as the guard leaves this arm's scope,
@@ -1426,6 +1427,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
plan,
&quit,
&stop,
8,
None,
)?;
Ok((new_vd, pipe))
@@ -1522,6 +1524,10 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
bit_depth,
plan,
&quit,
// No first-frame shortening here: this direct call has no retry wrapper to
// absorb an early bail, and the resize source is a live compositor (the
// takeover race doesn't apply) — keep the patient default.
None,
Some(resize_trace.as_ref()),
) {
Ok(next_pipe) => {
@@ -1878,7 +1884,15 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// connected, frozen on the last frame, and the stream resumes when the new output
// appears — no reconnect.
const REBUILD_BUDGET: std::time::Duration = std::time::Duration::from_secs(40);
let rebuild_deadline = std::time::Instant::now() + REBUILD_BUDGET;
// Attach-only holdoff: for the first seconds after a capture loss the session
// detection can be STALE (the new session isn't up yet), and a rebuild acting on
// a stale "Gaming" answer restarts gamescope-session.target — which on SteamOS
// steals the seat back from the session the user just switched to (observed
// live). While the holdoff lasts, builds run under a vdisplay rebuild-probe
// scope: attach to live outputs only, never stop/relaunch/take over sessions.
const PROBE_HOLDOFF: std::time::Duration = std::time::Duration::from_secs(4);
let loss_at = std::time::Instant::now();
let rebuild_deadline = loss_at + REBUILD_BUDGET;
let (new_cap, new_enc, new_frame, new_interval, new_node_id, new_display_gen) = loop {
// Follow the active session unless an explicit PUNKTFUNK_COMPOSITOR pin forbids
// retargeting (then we stick to the pinned backend and just rebuild it).
@@ -1912,6 +1926,8 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
}
}
}
let _probe = (loss_at.elapsed() < PROBE_HOLDOFF)
.then(crate::vdisplay::rebuild_probe_scope);
match build_pipeline_with_retry(
&mut vd,
cur_mode,
@@ -1920,6 +1936,16 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
plan,
&quit,
&stop,
// 1, not 8: this loop re-detects the active session per iteration — short
// inner cycles are what let it FOLLOW a session switch instead of burning
// retries against a compositor that no longer exists. One attempt per
// cycle also keeps every probe on the SHORT first-frame window (attempt 1
// = 2.5 s): a patient 10 s attempt here just waits on a stale backend
// (observed live: it made a Game→Desktop switch 20 s instead of ~9 —
// the winning KWin rebuild took 0.7 s once detection caught up). The
// slow-new-session case is the OUTER loop's job (40 s budget, fresh
// 2.5 s probes until the new compositor delivers).
1,
None,
) {
Ok(p) => break p,
@@ -1932,6 +1958,9 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
}
tracing::warn!(error = %format!("{e2:#}"),
"capture lost — new session not up yet, retrying");
// Probe failures are instant (attach-only bail) — pace the loop so
// re-detection runs at ~2 Hz instead of spinning.
std::thread::sleep(std::time::Duration::from_millis(500));
}
}
};
@@ -2655,6 +2684,7 @@ pub(super) fn prepare_display(
plan,
quit,
stop,
8,
Some(trace),
)?;
Ok(PreparedDisplay { vd, pipeline })
@@ -2679,15 +2709,22 @@ fn build_pipeline_with_retry(
plan: crate::session_plan::SessionPlan,
quit: &Arc<AtomicBool>,
stop: &Arc<AtomicBool>,
// Retry budget: 8 everywhere EXCEPT the capture-loss rebuild (2). That path wraps this call
// in its own outer loop that RE-DETECTS the active session between calls — during a
// Gaming↔Desktop switch the old compositor is simply gone, so burning 8 attempts (~13 s)
// against its dead socket only delays following the box to the session that replaced it
// (observed live: a Desktop→Gaming switch spent 13 of its 27 s retrying gone-KWin).
max_attempts: u32,
// Transition trace (P0.1): `Some` for the traced builds (bring-up, resize); each stage stamps
// once (first crossing) so the retry loop can pass it through unconditionally.
trace: Option<&crate::bringup::Trace>,
) -> Result<Pipeline> {
// ~10s first-frame wait per attempt. 8 gives a ~90s budget for the SLOW case: a host-managed
// gamescope session cold-starting Steam Big Picture (the SteamOS/Bazzite takeover) can take
// 30-60s to produce its first frame, and a first-connect timeout would tear down the warm
// session (forcing another cold start on reconnect). A genuinely permanent failure still fails
// fast via `is_permanent_build_error`; only transient "no frame yet" retries consume the budget.
// ~10s first-frame wait per attempt (attempt 1: see FIRST_ATTEMPT_FRAME_BUDGET below). 8
// gives a ~80s budget for the SLOW case: a host-managed gamescope session cold-starting Steam
// Big Picture (the SteamOS/Bazzite takeover) can take 30-60s to produce its first frame, and
// a first-connect timeout would tear down the warm session (forcing another cold start on
// reconnect). A genuinely permanent failure still fails fast via `is_permanent_build_error`;
// only transient "no frame yet" retries consume the budget.
// IDD-push only: HOLD one monitor lease across all build attempts. A failed attempt's capturer
// drop releases ITS lease, but this held lease keeps the shared monitor Active (refs >= 1), so the
// next attempt's `vd.create` JOINS it (refcount++) instead of finding it Lingering and tripping the
@@ -2705,9 +2742,16 @@ fn build_pipeline_with_retry(
} else {
None
};
const MAX_ATTEMPTS: u32 = 8;
// Attempt 1 waits only briefly for the first frame: a PipeWire stream connected while
// gamescope re-initializes its headless takeover negotiates a format and reaches `Streaming`
// but never receives a buffer — a FRESH connect then delivers within ~0.5 s (observed on
// SteamOS: every gamescope bring-up burned the full 10 s on attempt 1, then attempt 2 got
// frames instantly → 17 s bring-ups). Healthy compositors deliver the first frame well inside
// this window (KWin ~0.3 s), and the genuinely-slow cold start above still gets the patient
// 10 s window on every later attempt.
const FIRST_ATTEMPT_FRAME_BUDGET: std::time::Duration = std::time::Duration::from_millis(2500);
let mut backoff = std::time::Duration::from_millis(500);
for attempt in 1..=MAX_ATTEMPTS {
for attempt in 1..=max_attempts {
// The client is gone (connection closed → `stop`): every further attempt only churns the
// box for a session no one is watching — on a Bazzite takeover that means SIGKILLing and
// relaunching the box's Steam session once per attempt for minutes (the .181 storm
@@ -2719,7 +2763,17 @@ fn build_pipeline_with_retry(
attempt - 1
);
}
match build_pipeline(vd, mode, bitrate_kbps, bit_depth, plan, quit, trace) {
let first_frame_budget = (attempt == 1).then_some(FIRST_ATTEMPT_FRAME_BUDGET);
match build_pipeline(
vd,
mode,
bitrate_kbps,
bit_depth,
plan,
quit,
first_frame_budget,
trace,
) {
Ok(pipe) => {
if attempt > 1 {
tracing::info!(attempt, "pipeline up after retry");
@@ -2729,7 +2783,7 @@ fn build_pipeline_with_retry(
Err(e) => {
let chain = format!("{e:#}");
let permanent = is_permanent_build_error(&chain);
if permanent || attempt == MAX_ATTEMPTS {
if permanent || attempt == max_attempts {
let why = if permanent {
"permanent"
} else {
@@ -2741,7 +2795,7 @@ fn build_pipeline_with_retry(
}
tracing::warn!(
attempt,
max = MAX_ATTEMPTS,
max = max_attempts,
backoff_ms = backoff.as_millis() as u64,
error = %chain,
"pipeline build failed — retrying"
@@ -2802,6 +2856,10 @@ fn build_pipeline(
bit_depth: u8,
plan: crate::session_plan::SessionPlan,
quit: &Arc<AtomicBool>,
// First-frame wait override (`None` = the backend's default 10 s): the retry loop shortens
// its FIRST attempt so a stream stuck in the gamescope takeover race fails over to the
// reconnect that fixes it (see FIRST_ATTEMPT_FRAME_BUDGET in `build_pipeline_with_retry`).
first_frame_budget: Option<std::time::Duration>,
// Transition trace (P0.1): stamps the build's stages (display acquire, capture attach, first
// frame, encoder open) into the bring-up/resize timeline. `None` on untraced rebuilds.
trace: Option<&crate::bringup::Trace>,
@@ -2857,7 +2915,11 @@ fn build_pipeline(
t.mark("capture_attached");
}
capturer.set_active(true);
let frame = match capturer.next_frame().context("first frame") {
let first = match first_frame_budget {
Some(budget) => capturer.next_frame_within(budget),
None => capturer.next_frame(),
};
let frame = match first.context("first frame") {
Ok(f) => f,
Err(e) => {
// A reused kept display was dead — invalidate it so the next attempt creates fresh (A2).
+17 -1
View File
@@ -158,9 +158,25 @@ pub(crate) fn runner_command() -> Result<(std::path::PathBuf, Vec<String>)> {
if bun.exists() && runner.exists() {
return Ok((bun, vec![runner.to_string_lossy().into_owned()]));
}
// Immutable-/usr distros (SteamOS): scripts/steamdeck/install.sh lays the SAME payload
// out user-scoped under ~/.local — wrapper, private bun, and bundle mirroring the deb's
// /usr layout — because a system package can't exist there.
if let Ok(home) = std::env::var("HOME") {
let home = std::path::Path::new(&home);
let wrapper = home.join(".local/bin/punktfunk-scripting");
if wrapper.exists() {
return Ok((wrapper, Vec::new()));
}
let bun = home.join(".local/lib/punktfunk-scripting/bun");
let runner = home.join(".local/share/punktfunk-scripting/runner-cli.js");
if bun.exists() && runner.exists() {
return Ok((bun, vec![runner.to_string_lossy().into_owned()]));
}
}
bail!(
"the plugin runner isn't installed — install it first (Debian/Ubuntu: \
`sudo apt install punktfunk-scripting`)"
`sudo apt install punktfunk-scripting`; SteamOS: re-run \
scripts/steamdeck/install.sh)"
)
}
}
@@ -179,6 +179,13 @@ impl SessionPlan {
// Vulkan device; on Linux the capture facade flips the zero-copy policy to the
// raw-dmabuf passthrough (see above).
pyrowave: self.codec == crate::encode::Codec::PyroWave,
// Producer-native NV12 (gamescope) is consumable only by the Linux Vulkan Video
// backend — resolved HERE from the plan's codec so the capturer never reaches back
// into encode (the same one-way edge as `gpu` above).
#[cfg(target_os = "linux")]
nv12_native: crate::encode::linux_native_nv12_ok(self.codec),
#[cfg(not(target_os = "linux"))]
nv12_native: false,
}
}
}
+7 -1
View File
@@ -114,12 +114,18 @@ pub fn register(
session: session_ref(&session),
});
registry().lock().unwrap().push(session);
LiveSessionGuard { id }
LiveSessionGuard {
id,
_sleep: crate::sleep_inhibit::hold(),
}
}
/// Removes its session from the registry when dropped (any scope exit of the native video loop).
pub struct LiveSessionGuard {
id: u64,
/// While any native session lives, the box must not auto-suspend under a passive viewer
/// ([`crate::sleep_inhibit`]) — refcounted, released with the guard.
_sleep: crate::sleep_inhibit::StreamHold,
}
impl Drop for LiveSessionGuard {
+100
View File
@@ -0,0 +1,100 @@
//! Session-scoped suspend/idle inhibition: while at least one client is streaming, the host
//! holds a logind `sleep:idle` BLOCK inhibitor so the box doesn't auto-suspend out from under a
//! passive viewer. Remote INPUT resets the compositor's idle timers, but a video-only viewer
//! sends none — observed live on a SteamOS Game-Mode host, which s2idled mid-stream-day and
//! dropped off the network (and, in a VM with GPU passthrough, never woke again). Refcounted
//! across planes (native sessions + GameStream media): the first hold acquires, the last drop
//! releases. Best-effort — no logind (containers, non-systemd boxes) logs once and streams on.
//! Off Linux this is a no-op: macOS/Windows hosts manage their own power assertions.
use std::sync::{Mutex, OnceLock};
/// RAII share of the host-wide inhibitor — hold one per live session/stream.
pub struct StreamHold(());
struct State {
count: u32,
/// The logind inhibitor pipe fd — inhibition lasts exactly as long as it stays open.
#[cfg(target_os = "linux")]
fd: Option<ashpd::zbus::zvariant::OwnedFd>,
}
fn state() -> &'static Mutex<State> {
static S: OnceLock<Mutex<State>> = OnceLock::new();
S.get_or_init(|| {
Mutex::new(State {
count: 0,
#[cfg(target_os = "linux")]
fd: None,
})
})
}
/// Take a share; the underlying inhibitor is acquired on the 0→1 edge.
pub fn hold() -> StreamHold {
let mut st = state().lock().unwrap_or_else(|e| e.into_inner());
st.count += 1;
#[cfg(target_os = "linux")]
if st.count == 1 && st.fd.is_none() {
st.fd = acquire();
}
StreamHold(())
}
impl Drop for StreamHold {
fn drop(&mut self) {
let mut st = state().lock().unwrap_or_else(|e| e.into_inner());
st.count = st.count.saturating_sub(1);
#[cfg(target_os = "linux")]
if st.count == 0 && st.fd.take().is_some() {
tracing::info!("released the sleep/idle inhibitor (no live sessions)");
}
}
}
/// One logind `Inhibit` call on a dedicated plain thread — zbus's blocking API must not run on
/// a tokio worker (its internal `block_on` panics there), and callers of [`hold`] may be either.
/// The join blocks the caller for the D-Bus round-trip (~ms), which every call site tolerates.
#[cfg(target_os = "linux")]
fn acquire() -> Option<ashpd::zbus::zvariant::OwnedFd> {
let fd = std::thread::spawn(|| -> Option<ashpd::zbus::zvariant::OwnedFd> {
use ashpd::zbus;
// zbus's blocking API is configured out by ashpd's feature set — drive the async API on
// a private current-thread runtime instead (still on this plain thread; see fn doc).
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.ok()?;
let attempt: zbus::Result<zbus::zvariant::OwnedFd> = rt.block_on(async {
let conn = zbus::Connection::system().await?;
let reply = conn
.call_method(
Some("org.freedesktop.login1"),
"/org/freedesktop/login1",
Some("org.freedesktop.login1.Manager"),
"Inhibit",
&("sleep:idle", "Punktfunk", "a client is streaming", "block"),
)
.await?;
reply.body().deserialize()
});
match attempt {
Ok(fd) => Some(fd),
Err(e) => {
tracing::warn!(
error = %e,
"could not take a logind sleep/idle inhibitor — the box may auto-suspend \
under a passive (video-only) viewer"
);
None
}
}
})
.join()
.ok()
.flatten();
if fd.is_some() {
tracing::info!("holding a logind sleep/idle inhibitor while clients stream");
}
fd
}
+60 -9
View File
@@ -168,12 +168,29 @@ The canonical "decide, don't just observe" pattern — approve pairing from your
## Recipe: full controller passthrough (VirtualHere)
To get a controller's *native* features on the host — DualSense gyro, touchpad, adaptive
triggers, USB rumble — instead of the emulated pad, share the physical device from the couch with
[VirtualHere](https://www.virtualhere.com/) (USB-over-IP) and bind it to the host only while a
client is connected. The couch runs the VirtualHere **server** (sharing the pad); the host runs
the VirtualHere **client** and this automation drives its `-t` IPC.
triggers, USB rumble — instead of the emulated pad, hand the physical device from the couch to the
host over [VirtualHere](https://www.virtualhere.com/) (USB-over-IP), bound only while a client is
connected so the couch keeps its own controller the rest of the time.
Zero-code, bracketed on the stream with two hooks:
**The two sides.** VirtualHere is a server/client pair, and you run both:
- **Server — on the couch** (where the pad is physically plugged in). Run the VirtualHere USB
Server there; it shares the pad on the LAN. Leave it running.
- **Client — on the host** (where the game and this automation run). Install the VirtualHere
Client (as a service, or the tray app). It auto-discovers the couch's shared pad on the same
LAN; across subnets, add it once with `<VH_CLIENT> -t "MANUAL HUB ADD,<couch-ip>:7575"`. The
client binary is `vhclientx86_64` on Linux, `vhui64.exe` on Windows, `vhclientosx` on macOS,
`vhclientarm64` on ARM Linux.
The client's `-t` flag is a one-shot IPC to the already-running client: `-t LIST` prints every
visible device with its address (`server.port`, e.g. `couch-deck.11`); `-t "USE,<addr>"` mounts it
onto the host; `-t "STOP USING,<addr>"` hands it back. The automation just brackets
`USE` / `STOP USING` around a session.
### Zero-code: two hooks
Bracket it on the stream with two [hooks](#hooks-hooksjson) — mount when video starts, release
when it stops. This is all most setups need:
```json
{
@@ -184,8 +201,42 @@ Zero-code, bracketed on the stream with two hooks:
}
```
`couch-deck.11` is the device's VirtualHere address (`vhclientx86_64 -t LIST`); same-LAN setups
auto-discover it, otherwise `MANUAL HUB ADD,<couch-ip>:7575` once. For a version that resolves the
device by name, filters to one couch, and releases the pad on a clean shutdown, see the
`couch-deck.11` is the device's VirtualHere address from `vhclientx86_64 -t LIST`.
### Scripted: resolve by name, release on shutdown
The hooks above hard-code the address, and can strand the pad on the host if it's stopped
mid-stream (the `stream.stopped` hook never fires). The
[`virtualhere-dualsense.ts`](https://git.unom.io/unom/punktfunk/src/branch/main/sdk/examples/virtualhere-dualsense.ts)
SDK example.
SDK example is the robust version: it resolves the device by **name substring** (survives the
address changing), can filter to **one couch** in a multi-client setup, and **releases the pad on
a clean shutdown** (`systemctl stop`, ^C) so the couch always gets its controller back.
It's a standalone [`@punktfunk/host` script](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#running-a-single-script-as-a-service)
— run it in its own directory. The one edit is its import: the in-repo copy imports from
`../src/index.js`; outside the repo it's the published package:
```sh
mkdir ~/punktfunk-scripts && cd ~/punktfunk-scripts
bun init -y
bun add @punktfunk/host # point the @punktfunk scope at the registry first — see the SDK README
# save the example as virtualhere-dualsense.ts, and change its first import to the package:
# - import { connect } from "../src/index.js";
# + import { connect } from "@punktfunk/host";
VH_DEVICE=DualSense bun virtualhere-dualsense.ts
```
`VH_DEVICE` is required — a VirtualHere address (`couch-deck.11`) or a device-name substring
(`DualSense`). Optional: `VH_CLIENT` overrides the client binary (default `vhclientx86_64`);
`VH_ONLY_CLIENT` binds only for one punktfunk client label. Running on the host box the SDK needs
no token or URL — it reads the host's loopback credentials itself (see
[Connection resolution](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#connection-resolution)).
Keep it running as a
[systemd user unit](https://git.unom.io/unom/punktfunk/src/branch/main/sdk#running-a-single-script-as-a-service)
(its default `SIGTERM` triggers the script's own release step — so `systemctl stop` hands the pad
back), or drop it under the [scripting runner](/docs/plugins) with your other units.
> The example brackets on `client.connected` / `client.disconnected` — the pad returns to the
> couch the moment they disconnect. Switch to `stream.started` / `stream.stopped` if you'd rather
> pass it through only while video is actually flowing; both are noted in the file's header.
+1
View File
@@ -90,6 +90,7 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t
| `PUNKTFUNK_FEC_PCT` | `N` (percent) | Forward-error-correction redundancy for lossy links (the default is sensible for a normal LAN). Higher = more loss-resilient, more bandwidth. |
| `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. Windows host, plus the Linux **GNOME 50+ GameStream desktop mirror** (`PUNKTFUNK_VIDEO_SOURCE=portal`, mirrored monitor in HDR mode — check with `punktfunk-host hdr-probe`). Linux **virtual displays** (native protocol, GameStream default) stay 8-bit: Mutter's virtual-monitor screencast is SDR-only upstream. |
| `PUNKTFUNK_444` | `1` · `0` *(default on)* | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **On by default** on the host; the client's own 4:4:4 setting (default off) is the real switch. Set `0` to force 4:2:0. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. |
| `PUNKTFUNK_CHACHA20` | `1` · `0` *(default on)* | ChaCha20-Poly1305 session encryption for clients without hardware AES (old ARM TVs, e.g. webOS), lifting their ~100 Mbps software-AES decrypt ceiling. **On by default** on the host; a session uses it only when the client requests it — everyone else stays on AES-GCM. Purely a performance choice (both ciphers are full-strength); set `0` to force AES-GCM for all sessions. |
| `PUNKTFUNK_PYROWAVE_MAX_MBPS` | `N` (Mbps) | Cap the [PyroWave](/docs/pyrowave) Automatic bitrate pin, for a host on a link that the open-loop pin can outrun (e.g. 4:4:4 + HDR at 5120×1440@240 pins ~5.3 Gbps, over a 5GbE link). Unset = no cap. Only affects Automatic (bitrate `0`) PyroWave sessions; an explicit client bitrate bypasses it. |
| `PUNKTFUNK_DSCP` | `1` | Opt-in DSCP / `SO_PRIORITY` QoS tagging on the media sockets. No-op on the wire on Windows without a qWAVE policy. |
| `PUNKTFUNK_OH264_THREADS` / `PUNKTFUNK_OH264_GOP` | `N` | Software (openh264) encoder tuning: encode threads (default 2 — latency over throughput) and GOP length (default 0 = encoder-auto). Only relevant with `PUNKTFUNK_ENCODER=software`. |
+4
View File
@@ -70,6 +70,10 @@ These apply to the **Gaming Mode (gamescope)** path only; the desktop path is un
- **gamescope 3.16.22 or newer is required.** Older versions can deadlock during capture. Bazzite's
and SteamOS's current gamescope is fine; this only bites if you've pinned an old one.
- **The mouse cursor isn't included in the captured image** — a gamescope limitation for now.
- **Touch arrives as a single-finger pointer.** gamescope's virtual input device has no
touchscreen, so the host maps a client's touchscreen to an absolute pointer: taps click exactly
where you touch and drags work, but multi-touch gestures (pinch) aren't available in Gaming
Mode. The desktop path has full multi-touch.
- **HDR isn't supported on the gamescope path** — gamescope's capture output is 8-bit. SDR streams
normally.
+4
View File
@@ -69,6 +69,10 @@ punktfunk-host plugins add playnite # or: rom-manager
punktfunk-host plugins enable # turn the runner on (once)
```
On **SteamOS** the [host installer](/docs/steamos-host) ships the runner automatically (user-scoped
under `~/.local` — the read-only `/usr` can't take the package). If the console reports the runner
isn't installed on an older setup, re-run `scripts/steamdeck/update.sh` once.
</Tab>
<Tab value="Windows">
+30 -6
View File
@@ -56,12 +56,17 @@ bash ~/punktfunk/scripts/steamdeck/install.sh
It is idempotent — safe to re-run. In one pass it:
1. creates the `pf2` Debian-trixie distrobox and installs the build toolchain,
2. builds `punktfunk-host` (and the web console),
2. builds `punktfunk-host`, the web console, and the **plugin/script runner** (so the console's
[plugin store](/docs/plugins) works out of the box — the runner service itself stays opt-in),
3. writes config to `~/.config/punktfunk/` (a generated web-console login password),
4. raises the UDP socket buffers to 32 MB and adds you to the `input` group (needs `sudo`; skipped
with a warning if unavailable),
4. raises the UDP socket buffers to 32 MB, installs the gamepad udev rule + the `vhci-hcd` autoload
and adds you to the `input` group (virtual gamepads / **native Steam Deck controller passthrough**),
seeds the KDE RemoteDesktop grant for Desktop-mode input, and **registers all of it on SteamOS's
atomic-update keep list** so OS updates carry it over — this step **prompts for your `sudo`
password** (a stock Steam Deck requires one; without it gamepad passthrough and the UDP tuning are skipped),
5. installs + starts the `punktfunk-host` and `punktfunk-web` **systemd user services** (with linger,
so they run without a login session).
so they run without a login session) plus a boot-time **rebuild check** that repairs the host
automatically if a SteamOS update ever breaks its library links.
Useful flags:
@@ -82,6 +87,14 @@ When it finishes it prints the web-console URL and how to pair.
> If you only ever use native clients, install with `--no-gamestream` for a host with no GameStream
> surface at all.
> **First install — reboot once before streaming.** KWin only authorizes Desktop-mode screen capture
> on a fresh session, and the new `input` group (native Steam Deck controller passthrough) only takes
> effect on a new login — so after the **first** install, **reboot the Deck** (a re-run that changes
> nothing doesn't need it). Streaming **Game Mode** with a generic Xbox pad works right away; **Desktop
> capture and the native Steam Deck controller need the reboot.** If a client connects and every
> session ends with `KWin does not expose zkde_screencast_unstable_v1` or the pad shows up as an Xbox
> 360 controller, you haven't rebooted yet.
## 3. Pair a device
By default the host **requires PIN pairing** (secure). Two ways to pair:
@@ -128,8 +141,19 @@ bash ~/punktfunk/scripts/steamdeck/update.sh
thrash the managed session. Pick one mode per session.
- **Keep the device awake.** On handhelds, Game Mode auto-suspends on idle, which drops the host off
the network mid stream — disable auto-suspend (Settings → Power) for a headless host.
- **It survives OS updates**, but a major SteamOS bump can move library versions; if the host fails to
start after an update, just re-run `update.sh` to rebuild against the new base.
- **Native Steam Deck controller passthrough** presents the client's pad as a real Steam Deck
controller (paddles, trackpads, gyro) via a virtual USB device — that needs the `input` group and the
`vhci-hcd` module live, so it only works **after the first-install reboot** above; until then the pad
degrades to a generic Xbox 360 controller (still fully playable). If you're streaming *to* another
Steam Deck, also set Steam Input to **Off** for Punktfunk on that Deck — see
[Stream to a Steam Deck](/docs/steam-deck).
- **It survives OS updates — automatically.** SteamOS A/B updates rebuild `/etc` and can move
library versions; the installer defends both sides. The system tuning (gamepad udev rule,
`vhci-hcd`, UDP buffers) is registered on SteamOS's own atomic-update keep list
(`/etc/atomic-update.conf.d/`), so updates carry it over; and a boot-time check
(`punktfunk-rebuild-check`) probes the host binary and re-runs the build only if the new OS
actually broke its library links — you should never need to intervene. (Re-running `update.sh`
by hand still works and is harmless.)
- Deeper reference (services, container, manual steps): [`scripts/steamdeck/README.md`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/steamdeck/README.md).
Trouble? See [Troubleshooting](/docs/troubleshooting) and [Pairing](/docs/pairing).
+58 -1
View File
@@ -272,7 +272,7 @@
#define INBOUND_REQ_FLAG 2147483648
#endif
// 16-byte AEAD authentication tag appended by GCM.
// 16-byte AEAD authentication tag appended by either session cipher.
#define TAG_LEN 16
// Wire tag distinguishing an input datagram from a video packet.
@@ -465,6 +465,20 @@
#define VIDEO_CAP_STREAMED_AU 32
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_caps`] bit: the client can open **ChaCha20-Poly1305**-sealed session datagrams
// AND requests them — set by clients without hardware AES (the soft-AES armv7 targets, e.g.
// webOS TVs), where GCM's software AES + GHASH caps decrypt at ~100 Mbps while ChaCha's ARX
// construction runs 47× faster in portable code (design/chacha20-session-cipher.md).
// Support-plus-request in one bit mirrors [`VIDEO_CAP_444`]'s "capable AND turned on"
// precedent. The host grants it only when its `PUNKTFUNK_CHACHA20` kill-switch (default on)
// allows, answering with [`Welcome::cipher`] `= 1` + the 32-byte [`Welcome::key_chacha`];
// toward every other client the Welcome stays byte-identical AES-128-GCM. Purely a
// performance choice — both AEADs are full-strength, and Hello/Welcome ride the pinned-TLS
// control channel, so there is no downgrade surface.
#define VIDEO_CAP_CHACHA20 64
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
@@ -484,6 +498,18 @@
#define HOST_CAP_CLIPBOARD 2
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host's active inject backend can type **committed text**
// ([`InputKind::TextInput`](crate::input::InputKind::TextInput) — one Unicode scalar per event):
// Windows (`KEYEVENTF_UNICODE`) and Linux wlroots (dynamic Unicode keymap on a dedicated virtual
// keyboard); the KWin/libei/gamescope backends can only press layout keycodes, so those sessions
// don't set it. A capable client routes its IME's committed text (autocorrect, gesture typing,
// non-Latin scripts, emoji) through `TextInput` instead of lossy VK synthesis; absent the bit it
// keeps the VK fallback. Packs into the existing trailing `host_caps` byte — no wire-layout
// change; an older host ignores the unknown input tag anyway (input is lossy by design).
#define HOST_CAP_TEXT_INPUT 4
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
@@ -855,6 +881,18 @@
#define HELLO_LAUNCH_MAX 128
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::cipher`] id: AES-128-GCM — the default session AEAD every peer speaks (and the
// only one pre-cipher builds know).
#define CIPHER_AES_128_GCM 0
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::cipher`] id: ChaCha20-Poly1305 (RFC 8439) — negotiated via
// [`VIDEO_CAP_CHACHA20`] for clients without hardware AES.
#define CIPHER_CHACHA20_POLY1305 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Type byte of [`PairRequest`].
#define MSG_PAIR_REQUEST 16
@@ -951,6 +989,13 @@
// The client's wire (protocol) version does not match the host's — one side needs updating.
#define WIRE_VERSION_CLOSE_CODE 103
// The host admitted the connection but could not stand the stream session up (compositor /
// capture / encoder setup failed host-side). The close reason bytes carry the specific error
// text for logs/diagnostics; clients render a stable "host-side failure" sentence. Before this
// code, a setup failure reached the client as a bare dropped connection ("control stream
// finished mid-frame") — indistinguishable from transport trouble.
#define SETUP_FAILED_CLOSE_CODE 104
// Minimum supported multiplier (renders under native, upscaled on present).
#define MIN_SCALE 0.5
@@ -984,6 +1029,7 @@ enum PunktfunkStatus
PUNKTFUNK_STATUS_REJECTED_SUPERSEDED = -26,
PUNKTFUNK_STATUS_REJECTED_WIRE_VERSION = -27,
PUNKTFUNK_STATUS_REJECTED_BUSY = -28,
PUNKTFUNK_STATUS_REJECTED_SETUP_FAILED = -29,
PUNKTFUNK_STATUS_PANIC = -99,
};
#ifndef __cplusplus
@@ -1060,6 +1106,17 @@ enum PunktfunkInputKind
// [`HOST_CAP_GAMEPAD_STATE`](crate::quic::HOST_CAP_GAMEPAD_STATE); an older host ignores the
// unknown tag (every pad then uses the session-default kind — the pre-existing behaviour).
PUNKTFUNK_INPUT_KIND_GAMEPAD_ARRIVAL = 14,
// One Unicode scalar of **committed text** — `code` = the scalar value, everything else 0.
//
// The IME path: the layout-independent VK key events cannot express text an input method
// *commits* (autocorrect, gesture typing, non-Latin scripts, emoji), so a capable client
// sends the committed characters verbatim and the host injects them directly (Windows
// `KEYEVENTF_UNICODE`; Linux wlroots via a dynamically-grown Unicode keymap on a dedicated
// virtual keyboard). A multi-character commit is consecutive events in order. Sent only when
// the host advertised [`HOST_CAP_TEXT_INPUT`](crate::quic::HOST_CAP_TEXT_INPUT) — toward an
// older host (or one whose inject backend can't type text) clients keep the best-effort VK
// synthesis, and an older host ignores the unknown tag entirely.
PUNKTFUNK_INPUT_KIND_TEXT_INPUT = 15,
};
#ifndef __cplusplus
#if __STDC_VERSION__ >= 202311L
+17
View File
@@ -0,0 +1,17 @@
# punktfunk — SteamOS atomic-update preserve list (drop-in for /etc/atomic-update.conf.d/).
#
# SteamOS's A/B updates rebuild /etc from scratch and carry over ONLY the files matched by
# /usr/lib/rauc/atomic-update-keep.conf plus these operator drop-ins (the drop-in dir itself is
# on Valve's keep list, so this file is self-preserving). Without it, every OS update silently
# strips the host's system integration: /dev/uhid access (virtual pads degrade to Xbox 360),
# the vhci-hcd autoload (no native Steam Deck pad), and the UDP buffer sizing (stream stutter).
# Same wildcard rules as the stock list: '*' matches within a path segment, '**' across.
## Virtual-gamepad device access (uinput/uhid/vhci for the input group)
/etc/udev/rules.d/60-punktfunk.rules
## vhci-hcd autoload (usbip transport for the native Steam Deck pad)
/etc/modules-load.d/punktfunk.conf
## UDP socket buffer sizing (32 MB, matches the deb's sysctl drop-in)
/etc/sysctl.d/99-punktfunk-net.conf
+28 -7
View File
@@ -22,6 +22,15 @@ is only the build environment; `punktfunk-host` is launched directly, not via `d
rebuild always matches the running OS. Encode is **VAAPI** on the Deck's AMD GPU (NVENC on NVIDIA),
auto-selected by `PUNKTFUNK_ENCODER=auto`.
The honest trade-off: on-device building costs a slow first install (~1015 min, ~1 GB of image +
toolchain) and adds moving parts (apt mirrors, rustup, bun) — the price of an install that can
always chase the OS. Both failure modes of that chase are automated away now:
`punktfunk-rebuild-check` rebuilds when an OS update breaks the binary's links, and the
atomic-update keep list preserves the `/etc` tuning. The eventual lighter-weight alternative is a
CI-prebuilt bundle with the volatile libraries (FFmpeg et al.) vendored under an `$ORIGIN` rpath —
OS-update-proof without a toolchain on the device — worth it once SteamOS host volume justifies
per-release artifact signing/hosting; the from-source path would stay as the dev/fallback route.
The web console is the one part that stays in the container at runtime: it's a Nitro **`bun`**
build (`bun` both builds **and runs** it — the bun-preset output uses `Bun.serve` with TLS,
serving HTTPS (HTTP/1.1 over TLS) with the host's identity cert), so its service does
@@ -31,8 +40,9 @@ serving HTTPS (HTTP/1.1 over TLS) with the host's identity cert), so its service
| Script | What it does |
|--------|--------------|
| `install.sh` | Idempotent installer: ensure the `pf2` distrobox + toolchain → build host (+web) → write config → tune sysctl + `input` group (sudo) → install + start `punktfunk-host` / `punktfunk-web` systemd **user** services with linger. |
| `update.sh` | Rebuild from the current source and restart the services (config + pairings persist). `--pull` does `git pull` first. |
| `install.sh` | Idempotent installer: ensure the `pf2` distrobox + toolchain → build host + web + **plugin runner** → write config → tune sysctl + udev + `vhci-hcd` + `input` group and **register it on SteamOS's atomic-update keep list** (sudo) → install + start `punktfunk-host` / `punktfunk-web` systemd **user** services with linger, plus the **rebuild check** below. |
| `update.sh` | Rebuild everything from the current source and restart the services (config + pairings persist). `--pull` does `git pull` first. Also retrofits anything a newer install.sh writes (runner, keep-list registration, rebuild check) onto older installs. |
| `rebuild-check.sh` | The post-OS-update self-heal (run by `punktfunk-rebuild-check.service` before the host at session start): `ldd`-probes the binary — milliseconds when healthy, a full `update.sh` rebuild only when a SteamOS update actually broke its library links. |
```sh
git clone https://git.unom.io/unom/punktfunk ~/punktfunk
@@ -57,10 +67,19 @@ default `pf2`), `PUNKTFUNK_MGMT_PORT` (47990), `PUNKTFUNK_WEB_PORT` (47992).
here too and persists across updates.
- **Services:** `~/.config/systemd/user/punktfunk-host.service` (runs `serve --gamestream --mgmt-bind
0.0.0.0:47990`, `+ --open` if chosen — `--gamestream` adds the Moonlight-compat planes so the Deck's
Game Mode also streams to stock Moonlight; the native `punktfunk/1` plane is always on) and
`punktfunk-web.service`. Linger is enabled so they run without a login session.
Game Mode also streams to stock Moonlight; the native `punktfunk/1` plane is always on),
`punktfunk-web.service`, `punktfunk-rebuild-check.service` (post-OS-update self-heal, enabled), and
`punktfunk-scripting.service` (plugin runner, **opt-in** — enable it once you use plugins/scripts).
Linger is enabled so they run without a login session.
- **Plugin runner:** the deb's payload laid out user-scoped (read-only `/usr` can't take the
package): wrapper `~/.local/bin/punktfunk-scripting`, pinned `bun` in
`~/.local/lib/punktfunk-scripting/`, bundle in `~/.local/share/punktfunk-scripting/`.
- **System tuning (sudo):** `/etc/sysctl.d/99-punktfunk-net.conf` (32 MB UDP buffers — the #1
high-bitrate lever), `/etc/udev/rules.d/60-punktfunk.rules`, and `$USER` in the `input` group.
high-bitrate lever), `/etc/udev/rules.d/60-punktfunk.rules` (`uinput`/`uhid` access),
`/etc/modules-load.d/punktfunk.conf` (`vhci-hcd` for the native Deck pad), `$USER` in the `input`
group — and `/etc/atomic-update.conf.d/punktfunk.conf`, which registers the three files on
SteamOS's atomic-update keep list so A/B OS updates carry them over (verified: without it an
update silently strips them — pads degrade to Xbox 360, buffers drop to 208 KB).
## Operating
@@ -83,5 +102,7 @@ host advertises over mDNS as `_punktfunk._udp`, so clients discover it automatic
for a headless host.
- **WiFi tx ceiling** ≈ 250 Mbps goodput (a Deck hardware/driver packet-rate limit, band-independent);
fine for 1080p/1440p60. A wired dock lifts it.
- **After a major SteamOS update**, if the host won't start, run `update.sh` to rebuild against the new
base libraries.
- **After a SteamOS update** nothing should be needed: the `/etc` tuning survives via the
atomic-update keep list, and `punktfunk-rebuild-check` rebuilds the binary automatically if the
new base actually broke its library links (first session start after the update takes the build's
few minutes in that case). A manual `update.sh` remains harmless.
+173 -12
View File
@@ -48,6 +48,9 @@ BIN="$TARGET_DIR/release/punktfunk-host"
CONFIG="$HOME/.config/punktfunk"
UNITS="$HOME/.config/systemd/user"
XRD="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}"
# Set when this run does something that only a fresh login picks up (input-group add, first-time
# KWin .desktop grant). Drives the loud "reboot before streaming" note in the summary.
NEED_RELOGIN=0
# --- 0. preflight ----------------------------------------------------------
log "Preflight"
@@ -66,6 +69,33 @@ fi
DISTROBOX="$(command -v distrobox)" # baked into the web unit (may be /usr/bin or ~/.local/bin)
ok "distrobox: $DISTROBOX"
# --- acquire sudo up front (before the ~15-min build) ----------------------
# Steps 4-5 (UDP buffers, gamepad udev rule, vhci-hcd, input group, linger) need root. Prompt NOW,
# not after the build — so you authorize once and walk away, and a non-interactive run fails LOUDLY
# here instead of silently skipping the tuning at the very end. A stock SteamOS 'deck' has no
# password, so sudo can't work until you set one.
SUDO_OK=0
if sudo -n true 2>/dev/null; then
SUDO_OK=1
elif [ -t 0 ]; then
warn "sudo is needed once (UDP buffers, gamepad udev rule, vhci-hcd, input group, linger):"
if sudo -v; then
SUDO_OK=1
# keep the sudo timestamp warm across the long build so steps 4-5 don't re-prompt / expire
( while sudo -n -v 2>/dev/null; do sleep 50; done ) &
_pf_sudo_keepalive=$!
trap '[ -n "${_pf_sudo_keepalive:-}" ] && kill "$_pf_sudo_keepalive" 2>/dev/null || true' EXIT
fi
fi
if [ "$SUDO_OK" != 1 ]; then
if [ -t 0 ]; then
warn "No sudo — a stock SteamOS 'deck' account has no password. Set one and re-run: passwd"
else
warn "No TTY for the sudo prompt (non-interactive run) — system tuning + linger will be SKIPPED."
warn "Run in Konsole or an interactive 'ssh -t' session (or pre-authorize sudo) to enable them."
fi
fi
# --- 1. build container + toolchain ---------------------------------------
log "Build container '$BOX' ($BOX_IMAGE)"
if distrobox list 2>/dev/null | awk -F'|' '{gsub(/ /,"",$2); print $2}' | grep -qx "$BOX"; then
@@ -84,7 +114,7 @@ set -e
export DEBIAN_FRONTEND=noninteractive
sudo apt-get update -qq
sudo apt-get install -y -qq --no-install-recommends \
build-essential pkg-config clang curl git ca-certificates \
build-essential pkg-config clang cmake curl git ca-certificates \
libavcodec-dev libavformat-dev libavutil-dev libavfilter-dev libswscale-dev libavdevice-dev \
libpipewire-0.3-dev libspa-0.2-dev \
libgbm-dev libegl-dev libgl-dev libdrm-dev libva-dev \
@@ -101,10 +131,13 @@ ok "build deps ready"
# --- 2. build host (+ web) -------------------------------------------------
log "Building punktfunk-host (release) — first build is slow (~10-15 min)"
# vulkan-encode matches the packaged builds (deb/arch): the raw Vulkan Video HEVC/AV1 backend
# (real RFI loss recovery). Pure-Rust ash — no extra system dep. A featureless hand build would
# silently fall back to libav VAAPI.
distrobox enter "$BOX" -- bash -lc "
set -e
export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR'
cd '$SRC' && cargo build -r -p punktfunk-host
cd '$SRC' && cargo build -r -p punktfunk-host --features punktfunk-host/vulkan-encode
"
[ -x "$BIN" ] || die "build did not produce $BIN"
ok "host binary: $BIN"
@@ -120,14 +153,47 @@ cd '$SRC/web' && bun install --frozen-lockfile && bun run build
ok "web console built"
fi
# --- 2b. plugin runner (scripting) -----------------------------------------
# The console's plugin store and `punktfunk-host plugins …` shell out to the scripting runner —
# the SDK's runner CLI bundled to one self-contained JS, run on a pinned bun (it import()s the
# operator's .ts plugins; see packaging/debian/build-scripting-deb.sh). The .deb lays it out under
# /usr, which is read-only here, so ship the SAME payload user-scoped — wrapper + private bun +
# bundle under ~/.local, where the host's runner discovery looks after the /usr layouts.
log "Building the plugin runner (scripting)"
mkdir -p "$HOME/.local/bin" "$HOME/.local/lib/punktfunk-scripting" "$HOME/.local/share/punktfunk-scripting"
distrobox enter "$BOX" -- bash -lc "
set -e
export PATH=\$HOME/.bun/bin:\$PATH
cd '$SRC/sdk'
bun install --frozen-lockfile --ignore-scripts
bun build src/runner-cli.ts --target=bun --outfile \"\$HOME/.local/share/punktfunk-scripting/runner-cli.js\"
# Pin the runtime: copy the box's bun next to the bundle so a bun self-update (or a box rebuild)
# never changes what the runner executes under.
install -m0755 \"\$(command -v bun)\" \"\$HOME/.local/lib/punktfunk-scripting/bun\"
"
grep -q 'attempt=' "$HOME/.local/share/punktfunk-scripting/runner-cli.js" \
|| die "runner bundle missing the dynamic plugin import — wrong build"
cat > "$HOME/.local/bin/punktfunk-scripting" <<'WRAP'
#!/bin/sh
# Generated by scripts/steamdeck/install.sh — user-scoped punktfunk-scripting (the .deb's /usr/bin
# wrapper, relocated): the runner bundle on its private pinned bun.
exec "$HOME/.local/lib/punktfunk-scripting/bun" "$HOME/.local/share/punktfunk-scripting/runner-cli.js" "$@"
WRAP
chmod 0755 "$HOME/.local/bin/punktfunk-scripting"
ok "plugin runner: ~/.local/bin/punktfunk-scripting"
# --- 3. config -------------------------------------------------------------
log "Configuration ($CONFIG)"
mkdir -p "$CONFIG"
if [ ! -f "$CONFIG/host.env" ]; then
cat > "$CONFIG/host.env" <<'EOF'
# punktfunk Steam Deck host config (sourced by the punktfunk-host user service).
# Auto encoder: VAAPI on the Deck's AMD GPU, NVENC on NVIDIA.
# Auto encoder: Vulkan Video (or VAAPI fallback) on the Deck's AMD GPU, NVENC on NVIDIA.
PUNKTFUNK_ENCODER=auto
# Van Gogh (LCD/OLED Deck) RADV still gates VK_KHR_video_encode_* behind this perftest flag;
# without it the Vulkan backend can't open and sessions fall back to libav VAAPI. Harmless on
# GPUs where encode is exposed by default.
RADV_PERFTEST=video_encode
# The host auto-detects the live session (Game Mode gamescope / Desktop KDE) per connect.
# Override the compositor only if detection misbehaves: PUNKTFUNK_COMPOSITOR=gamescope
EOF
@@ -136,6 +202,34 @@ else
ok "host.env exists (left as-is)"
fi
# KWin authorization for Desktop-Mode streaming (and mid-stream Game↔Desktop switches): KWin
# resolves a connecting client's /proc/<pid>/exe against a .desktop `Exec=` and only then grants
# the restricted Wayland globals it lists (see packaging/linux/io.unom.Punktfunk.Host.desktop).
# Exec must therefore be THIS install's binary path, not the packaged /usr/bin one. KWin reads
# grants at session start — after first install, restart the Desktop session (Game Mode and back).
DESKTOP_DST="$HOME/.local/share/applications/io.unom.Punktfunk.Host.desktop"
# First-time install of the grant: KWin only reads it at session start, so a fresh login is required
# before Desktop-mode capture works. A re-run that just rewrites it needs no relogin.
[ -f "$DESKTOP_DST" ] || NEED_RELOGIN=1
mkdir -p "$HOME/.local/share/applications"
sed "s|^Exec=.*|Exec=$BIN|" "$SRC/packaging/linux/io.unom.Punktfunk.Host.desktop" > "$DESKTOP_DST"
ok "KWin desktop-capture authorization (io.unom.Punktfunk.Host.desktop → $BIN)"
# KDE Desktop-mode INPUT: a normal Plasma login lacks the RemoteDesktop portal grant the host's libei
# input path needs, so it would pop an "Allow remote control?" dialog a headless host can't answer.
# Seed it once (per-user, no root) — mirrors packaging/bazzite/kde-desktop-setup.sh. Game Mode
# (gamescope) needs none of this; the .desktop above already grants org_kde_kwin_fake_input.
GRANT_SRC="$SRC/scripts/headless/kde-authorized"
GRANT_DST="$HOME/.local/share/flatpak/db/kde-authorized"
if [ -s "$GRANT_DST" ]; then
ok "KDE RemoteDesktop grant already present"
elif [ -s "$GRANT_SRC" ]; then
mkdir -p "$(dirname "$GRANT_DST")"
install -m644 "$GRANT_SRC" "$GRANT_DST"
systemctl --user restart xdg-permission-store 2>/dev/null || true
ok "seeded KDE RemoteDesktop grant (Desktop-mode input)"
fi
if [ "$WITH_WEB" = 1 ] && [ ! -f "$CONFIG/web.env" ]; then
# Random login password + session secret for the web console, generated once.
# `|| true` swallows the SIGPIPE `tr` takes when `head` closes the pipe (pipefail would abort).
@@ -151,9 +245,11 @@ else
[ "$WITH_WEB" = 1 ] && ok "web.env exists (login password unchanged)"
fi
# --- 4. system tuning (needs sudo; skipped gracefully if unavailable) ------
log "System tuning (UDP buffers + input group) — needs sudo"
if sudo -n true 2>/dev/null; then
# --- 4. system tuning (needs sudo: UDP buffers + gamepad udev rule + vhci-hcd + input group) --------
log "System tuning (UDP buffers + gamepad rules + vhci-hcd + input group)"
# sudo was acquired up front in preflight (SUDO_OK) so this never stalls behind the long build; a
# skip here (no password / no TTY) was already reported loudly there.
if [ "$SUDO_OK" = 1 ]; then
printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' \
| sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
@@ -161,13 +257,42 @@ if sudo -n true 2>/dev/null; then
if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then
sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules
sudo udevadm control --reload-rules && sudo udevadm trigger || true
ok "installed udev rule (virtual gamepads)"
ok "installed udev rule (virtual gamepads + native Steam Deck controller)"
fi
id -nG "$USER" | grep -qw input || { sudo usermod -aG input "$USER"; warn "added $USER to 'input' group — log out/in (or reboot) for gamepad support"; }
# vhci-hcd: the usbip transport that makes the virtual Steam Deck pad a *real* USB device so Steam
# Input adopts it (else it degrades to plain UHID, which Steam ignores — "no controller appears").
# Persist the autoload AND load it now so passthrough works without waiting for a reboot.
if [ -f "$SRC/scripts/punktfunk-modules.conf" ]; then
sudo install -m644 "$SRC/scripts/punktfunk-modules.conf" /etc/modules-load.d/punktfunk.conf
sudo modprobe vhci-hcd 2>/dev/null || warn "could not load vhci-hcd now (loads on next boot) — needed for the native Steam Deck pad"
ok "vhci-hcd autoload installed (native Steam Deck controller transport)"
fi
if id -nG "$USER" | grep -qw input; then
ok "already in the 'input' group"
else
warn "passwordless sudo unavailable — skipping UDP-buffer + udev tuning."
warn "Without it, high-bitrate streaming drops packets. Apply manually later:"
warn " echo -e 'net.core.wmem_max=33554432\\nnet.core.rmem_max=33554432' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf && sudo sysctl --system"
sudo usermod -aG input "$USER"
NEED_RELOGIN=1
warn "added $USER to the 'input' group (applies on next login)"
fi
# SteamOS A/B updates rebuild /etc and DROP everything not on Valve's keep list — verified
# live: an OS update stripped the udev rule + vhci autoload + UDP sysctl (gamepads silently
# degrade to Xbox 360, buffers back to 208 KB). The sanctioned fix is a preserve drop-in in
# /etc/atomic-update.conf.d/ (itself on the stock keep list, so it self-preserves).
if [ -f "$SRC/scripts/punktfunk-atomic-keep.conf" ]; then
sudo install -Dm644 "$SRC/scripts/punktfunk-atomic-keep.conf" /etc/atomic-update.conf.d/punktfunk.conf
ok "system tuning registered to survive SteamOS updates (atomic-update.conf.d)"
fi
else
warn "no usable sudo — SKIPPED system tuning. Gamepad passthrough + clean streaming need root (udev"
warn "rule, 'input' group, vhci-hcd, UDP buffers) — there is no user-space way to do these."
warn "A stock SteamOS 'deck' account has NO password, so sudo can't work until you set one:"
warn " passwd # set a sudo password once, then re-run this script"
warn "Or apply it by hand (then reboot):"
warn " sudo install -m644 $SRC/scripts/60-punktfunk.rules /etc/udev/rules.d/ &&"
warn " sudo install -m644 $SRC/scripts/punktfunk-modules.conf /etc/modules-load.d/punktfunk.conf &&"
warn " sudo usermod -aG input $USER &&"
warn " printf 'net.core.wmem_max=33554432\\nnet.core.rmem_max=33554432\\n' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf &&"
warn " sudo sysctl --system && sudo udevadm control --reload-rules && sudo udevadm trigger"
fi
# --- 5. systemd user services ---------------------------------------------
@@ -218,6 +343,35 @@ EOF
ok "punktfunk-web.service (port $WEB_PORT)"
fi
# The runner's user unit (OPT-IN, matching the .deb: installed but NOT enabled — the runner is
# inert until you add scripts/plugins). ExecStart is rewritten from the packaged /usr wrapper to
# the user-scoped one section 2b installed.
sed 's|^ExecStart=.*|ExecStart=%h/.local/bin/punktfunk-scripting|' \
"$SRC/scripts/punktfunk-scripting.service" > "$UNITS/punktfunk-scripting.service"
ok "punktfunk-scripting.service (opt-in: systemctl --user enable --now punktfunk-scripting)"
# Post-OS-update self-heal: SteamOS A/B updates can bump library sonames the host binary links
# (FFmpeg/PipeWire/libva) — this oneshot probes the binary with ldd before punktfunk-host starts
# and re-runs update.sh only when it actually stopped loading. Milliseconds on a normal boot.
cat > "$UNITS/punktfunk-rebuild-check.service" <<EOF
# Generated by scripts/steamdeck/install.sh — rebuild the host if a SteamOS update broke its libs.
[Unit]
Description=punktfunk SteamOS post-update rebuild check
Before=punktfunk-host.service
[Service]
Type=oneshot
ExecStart=$SRC/scripts/steamdeck/rebuild-check.sh
# A cold-ish rebuild is minutes, not seconds.
TimeoutStartSec=1800
[Install]
WantedBy=default.target
EOF
chmod +x "$SRC/scripts/steamdeck/rebuild-check.sh" 2>/dev/null || true
systemctl --user enable punktfunk-rebuild-check.service 2>/dev/null || true
ok "punktfunk-rebuild-check.service (auto-rebuild after SteamOS updates)"
systemctl --user daemon-reload
loginctl show-user "$USER" 2>/dev/null | grep -q 'Linger=yes' || { sudo loginctl enable-linger "$USER" 2>/dev/null && ok "enabled linger (services run without login)" || warn "could not enable linger — services stop when you log out (sudo loginctl enable-linger $USER)"; }
# enable + restart (not `enable --now`): restart picks up unit-file changes on a re-run, where
@@ -239,7 +393,7 @@ echo
log "Done — punktfunk host is running on this Steam Deck"
echo " • Host status: systemctl --user status punktfunk-host"
if [ "$WITH_WEB" = 1 ]; then
echo " • Web console: http://${IP:-steamdeck.local}:$WEB_PORT (login: see $CONFIG/web.env)"
echo " • Web console: https://${IP:-steamdeck.local}:$WEB_PORT (login: see $CONFIG/web.env)"
echo " • Pair a device: open the web console → Devices → arm pairing → enter the PIN on the client"
fi
if [ "$OPEN" = 1 ]; then
@@ -248,3 +402,10 @@ else
echo " • Pairing required (secure default). From a client, pick this host and enter the PIN the host shows."
fi
echo " • Update later: bash $SRC/scripts/steamdeck/update.sh"
if [ "$NEED_RELOGIN" = 1 ]; then
echo
warn "ONE MORE STEP before streaming — reboot the Deck (or fully log out and back in)."
echo " KWin only authorizes Desktop-mode screen capture on a fresh session, and the new 'input'"
echo " group (native Steam Deck controller passthrough) only applies to a new login. Streaming"
echo " Game Mode with a generic Xbox pad works now; Desktop capture + the native Deck pad need the reboot."
fi
+31
View File
@@ -0,0 +1,31 @@
#!/usr/bin/env bash
# punktfunk — SteamOS post-OS-update self-heal (runs before punktfunk-host at session start).
#
# The host binary links SteamOS system libraries (FFmpeg, PipeWire, libva, …). A SteamOS A/B
# update that bumps a soname leaves the binary unable to load — a silently dead host until
# someone remembers to re-run update.sh. This probe is the reliability backstop:
# * healthy binary → exit in milliseconds (every normal boot);
# * loader breakage → run scripts/steamdeck/update.sh (rebuild host + web + runner against
# the new library tree, restart services). The build container, source, and cargo caches
# all live under /home, which SteamOS updates never touch — so the rebuild is warm.
#
# Root is NOT needed: the /etc system tuning survives updates via the atomic-update keep list
# (see punktfunk-atomic-keep.conf); only the binary has to chase the OS libraries.
set -euo pipefail
# systemd user units get a minimal PATH — distrobox commonly lives in ~/.local/bin.
export PATH="$HOME/.local/bin:$PATH"
SRC="${PUNKTFUNK_SRC:-$HOME/punktfunk}"
BIN="$SRC/target-steamos/release/punktfunk-host"
if [ ! -x "$BIN" ]; then
echo "punktfunk-host binary missing at $BIN — running a full rebuild" >&2
elif ! ldd "$BIN" 2>/dev/null | grep -q "not found"; then
exit 0 # every library resolves — nothing to do
else
echo "punktfunk-host no longer loads after a SteamOS update — its missing libraries:" >&2
ldd "$BIN" 2>/dev/null | grep "not found" >&2 || true
echo "rebuilding against the new OS tree (this takes a few minutes; streaming resumes after)" >&2
fi
exec bash "$SRC/scripts/steamdeck/update.sh"
+124 -4
View File
@@ -8,6 +8,9 @@
set -euo pipefail
log() { printf '\033[1;36m==>\033[0m %s\n' "$*"; }
ok() { printf '\033[1;32m ok\033[0m %s\n' "$*"; }
# warn was USED below but never defined — under `set -e` the first warn call ("command not
# found") aborted the whole update before the service restarts.
warn() { printf '\033[1;33m !!\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31merror:\033[0m %s\n' "$*" >&2; exit 1; }
SRC="${PUNKTFUNK_SRC:-$HOME/punktfunk}"
@@ -21,7 +24,8 @@ if [ "${1:-}" = "--pull" ]; then
fi
log "Rebuilding host (release)"
distrobox enter "$BOX" -- bash -lc "set -e; export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR'; cd '$SRC' && cargo build -r -p punktfunk-host"
# vulkan-encode matches the packaged builds (deb/arch) — see install.sh.
distrobox enter "$BOX" -- bash -lc "set -e; export PATH=\$HOME/.cargo/bin:\$PATH CARGO_TARGET_DIR='$TARGET_DIR'; cd '$SRC' && cargo build -r -p punktfunk-host --features punktfunk-host/vulkan-encode"
ok "host rebuilt"
if [ "$WEB" = 1 ]; then
log "Rebuilding web console"
@@ -29,9 +33,125 @@ if [ "$WEB" = 1 ]; then
ok "web rebuilt"
fi
# Plugin runner (scripting): rebuild the user-scoped runner payload (install.sh §2b) — also
# RETROFITS it onto older installs that predate it (the "plugin runner isn't installed" console
# state on SteamOS).
log "Rebuilding plugin runner (scripting)"
mkdir -p "$HOME/.local/bin" "$HOME/.local/lib/punktfunk-scripting" "$HOME/.local/share/punktfunk-scripting"
distrobox enter "$BOX" -- bash -lc "set -e; export PATH=\$HOME/.bun/bin:\$PATH; cd '$SRC/sdk' && bun install --frozen-lockfile --ignore-scripts && bun build src/runner-cli.ts --target=bun --outfile \"\$HOME/.local/share/punktfunk-scripting/runner-cli.js\" && install -m0755 \"\$(command -v bun)\" \"\$HOME/.local/lib/punktfunk-scripting/bun\""
grep -q 'attempt=' "$HOME/.local/share/punktfunk-scripting/runner-cli.js" \
|| die "runner bundle missing the dynamic plugin import — wrong build"
cat > "$HOME/.local/bin/punktfunk-scripting" <<'WRAP'
#!/bin/sh
# Generated by scripts/steamdeck/update.sh — user-scoped punktfunk-scripting (see install.sh §2b).
exec "$HOME/.local/lib/punktfunk-scripting/bun" "$HOME/.local/share/punktfunk-scripting/runner-cli.js" "$@"
WRAP
chmod 0755 "$HOME/.local/bin/punktfunk-scripting"
sed 's|^ExecStart=.*|ExecStart=%h/.local/bin/punktfunk-scripting|' \
"$SRC/scripts/punktfunk-scripting.service" > "$HOME/.config/systemd/user/punktfunk-scripting.service"
systemctl --user daemon-reload
ok "plugin runner rebuilt (opt-in service: systemctl --user enable --now punktfunk-scripting)"
# Retrofit the post-OS-update rebuild check (install.sh §5) onto older installs: probes the host
# binary with ldd at session start and re-runs this script when a SteamOS update broke its links.
if [ ! -f "$HOME/.config/systemd/user/punktfunk-rebuild-check.service" ]; then
cat > "$HOME/.config/systemd/user/punktfunk-rebuild-check.service" <<EOF
# Generated by scripts/steamdeck/update.sh — rebuild the host if a SteamOS update broke its libs.
[Unit]
Description=punktfunk SteamOS post-update rebuild check
Before=punktfunk-host.service
[Service]
Type=oneshot
ExecStart=$SRC/scripts/steamdeck/rebuild-check.sh
TimeoutStartSec=1800
[Install]
WantedBy=default.target
EOF
chmod +x "$SRC/scripts/steamdeck/rebuild-check.sh" 2>/dev/null || true
systemctl --user daemon-reload
systemctl --user enable punktfunk-rebuild-check.service 2>/dev/null || true
ok "punktfunk-rebuild-check.service installed (auto-rebuild after SteamOS updates)"
fi
# Retrofit config that install.sh now writes but older installs predate (both idempotent):
# RADV_PERFTEST — Van Gogh RADV still gates VK_KHR_video_encode_* behind it; without it the
# Vulkan backend can't open and sessions silently fall back to libav VAAPI. The KWin .desktop —
# KWin only grants the restricted capture/input globals to the exe a .desktop authorizes.
HOST_ENV="$HOME/.config/punktfunk/host.env"
if [ -f "$HOST_ENV" ] && ! grep -q '^RADV_PERFTEST=' "$HOST_ENV"; then
printf '\n# Van Gogh RADV gates VK_KHR_video_encode_* behind this (Vulkan Video encode).\nRADV_PERFTEST=video_encode\n' >> "$HOST_ENV"
ok "host.env: added RADV_PERFTEST=video_encode"
fi
mkdir -p "$HOME/.local/share/applications"
sed "s|^Exec=.*|Exec=$TARGET_DIR/release/punktfunk-host|" "$SRC/packaging/linux/io.unom.Punktfunk.Host.desktop" \
> "$HOME/.local/share/applications/io.unom.Punktfunk.Host.desktop"
ok "KWin desktop-capture authorization refreshed"
# Retrofit the system bits install.sh now sets up but older installs predate (idempotent). vhci-hcd =
# usbip transport for the native Steam Deck pad; 60-punktfunk.rules = /dev/uhid + vhci access; input
# group = uhid write; the kde-authorized grant (per-user, no root) = Desktop-mode input. A stock Deck
# needs a sudo PASSWORD, so PROMPT for it rather than silently skipping (skipping = gamepads stay dead).
SUDO_OK=0
if sudo -n true 2>/dev/null; then
SUDO_OK=1
elif [ -t 0 ]; then
warn "sudo needs your password to (re)apply the gamepad udev rule, vhci-hcd, input group, and UDP buffers:"
sudo -v && SUDO_OK=1 || true
fi
if [ "$SUDO_OK" = 1 ]; then
if [ -f "$SRC/scripts/60-punktfunk.rules" ]; then
sudo install -m644 "$SRC/scripts/60-punktfunk.rules" /etc/udev/rules.d/60-punktfunk.rules
sudo udevadm control --reload-rules >/dev/null 2>&1 || true
sudo udevadm trigger >/dev/null 2>&1 || true
ok "gamepad udev rule ensured"
fi
if [ -f "$SRC/scripts/punktfunk-modules.conf" ]; then
sudo install -m644 "$SRC/scripts/punktfunk-modules.conf" /etc/modules-load.d/punktfunk.conf
sudo modprobe vhci-hcd 2>/dev/null || true
ok "vhci-hcd autoload ensured (native Steam Deck controller)"
fi
# UDP buffers: older installs (or sudo-skipped ones) still run the stock 416 KB cap.
if [ ! -f /etc/sysctl.d/99-punktfunk-net.conf ]; then
printf 'net.core.wmem_max=33554432\nnet.core.rmem_max=33554432\n' | sudo tee /etc/sysctl.d/99-punktfunk-net.conf >/dev/null
sudo sysctl -q -p /etc/sysctl.d/99-punktfunk-net.conf >/dev/null 2>&1 || true
ok "UDP socket buffers raised to 32 MB (persisted)"
fi
if id -nG "$USER" | grep -qw input; then :; else
sudo usermod -aG input "$USER"
warn "added $USER to the 'input' group — REBOOT (or log out/in) for it to apply"
fi
# Register the tuning on Valve's atomic-update preserve list (see install.sh §4): without
# this, every SteamOS A/B update strips the three files above again (verified live —
# gamepads silently degrade to Xbox 360, UDP buffers back to 208 KB).
if [ -f "$SRC/scripts/punktfunk-atomic-keep.conf" ]; then
sudo install -Dm644 "$SRC/scripts/punktfunk-atomic-keep.conf" /etc/atomic-update.conf.d/punktfunk.conf
ok "system tuning registered to survive SteamOS updates (atomic-update.conf.d)"
fi
else
warn "no usable sudo — SKIPPED gamepad/udev/vhci/UDP tuning (all root-only; no user-space alternative)."
warn "A stock SteamOS 'deck' account has NO password — set one with 'passwd', then re-run. Gamepads stay"
warn "Xbox-360 until this runs and you reboot."
fi
echo
warn "If the controller still shows as an Xbox 360 pad, REBOOT the Deck once — the 'input' group and the"
warn "vhci-hcd module only become live for the host service on a fresh login."
GRANT_SRC="$SRC/scripts/headless/kde-authorized"
GRANT_DST="$HOME/.local/share/flatpak/db/kde-authorized"
if [ ! -s "$GRANT_DST" ] && [ -s "$GRANT_SRC" ]; then
mkdir -p "$(dirname "$GRANT_DST")"
install -m644 "$GRANT_SRC" "$GRANT_DST"
ok "seeded KDE RemoteDesktop grant (Desktop-mode input)"
fi
log "Restarting services"
systemctl --user restart punktfunk-host.service
ok "punktfunk-host restarted"
if [ "$WEB" = 1 ]; then systemctl --user restart punktfunk-web.service; ok "punktfunk-web restarted"; fi
# --no-block: when this script runs INSIDE punktfunk-rebuild-check.service (ordered
# Before=punktfunk-host), a blocking restart would deadlock — the restart job waits for the
# check unit, which waits for this script, which waits for the restart. Enqueue and move on;
# systemd starts the service the moment the ordering allows.
systemctl --user restart --no-block punktfunk-host.service
ok "punktfunk-host restart queued"
if [ "$WEB" = 1 ]; then systemctl --user restart --no-block punktfunk-web.service; ok "punktfunk-web restart queued"; fi
echo
log "Updated. Status: systemctl --user status punktfunk-host"
+4 -1
View File
@@ -89,7 +89,10 @@ A complexity ladder in [`examples/`](./examples) — start at the top:
4. [`couch-preset.effect.ts`](./examples/couch-preset.effect.ts) — **advanced, Effect-native**: only if you're composing Effect programs.
Examples 13 are the plain Promise facade and cover most automation; you only need example 4's
Effect surface for composed, interruptible programs. Run any with `bun examples/<file>.ts`.
Effect surface for composed, interruptible programs. Run any **in the repo** with
`bun examples/<file>.ts`. To **deploy** one on a host, install the package into its own directory
(`bun add @punktfunk/host`) and change its `../src/…` import to `@punktfunk/host` — see
[Running a single script as a service](#running-a-single-script-as-a-service).
Plus a real-world recipe:
+5
View File
@@ -15,6 +15,11 @@
// VH_DEVICE=couch-deck.11 bun examples/virtualhere-dualsense.ts # address from `-t LIST`
// VH_DEVICE=DualSense bun examples/virtualhere-dualsense.ts # …or match by name substring
//
// To run this *outside* the repo (the normal case), drop it in its own dir, `bun add
// @punktfunk/host`, and change the import below from `../src/index.js` to `@punktfunk/host`; then
// keep it alive as a systemd user unit (its SIGTERM release hands the pad back on `systemctl
// stop`). Full walkthrough: docs → Events & hooks → "full controller passthrough (VirtualHere)".
//
// Env: VH_DEVICE required — a VirtualHere address (`server.port`) or a device-name substring.
// VH_CLIENT client binary. Default `vhclientx86_64` (Linux); Windows `vhui64.exe`,
// macOS `vhclientosx`, ARM Linux `vhclientarm64`.

Some files were not shown because too many files have changed in this diff Show More