35 Commits

Author SHA1 Message Date
enricobuehler 71e1865519 fix(android): gate the latency overhaul behind an experimental toggle, default off
apple / swift (push) Successful in 2m36s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 50s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 59s
apple / screenshots (push) Successful in 5m55s
windows-host / package (push) Successful in 7m32s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m19s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m13s
release / apple (push) Successful in 12m3s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m18s
ci / rust (push) Successful in 4m52s
ci / bench (push) Successful in 6m44s
android / android (push) Successful in 5m9s
arch / build-publish (push) Successful in 5m40s
decky / build-publish (push) Successful in 15s
deb / build-publish (push) Successful in 3m20s
linux-client-screenshots / screenshots (push) Successful in 2m0s
flatpak / build-publish (push) Successful in 4m52s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m21s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m13s
docker / deploy-docs (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
web-screenshots / screenshots (push) Successful in 2m59s
android-screenshots / screenshots (push) Failing after 3m25s
The 27c53a4 low-latency overhaul regressed badly on some phones. Every piece
of it — decoder ranking, per-SoC vendor keys, the async decode loop, pipeline
thread boosts, the ADPF max-performance bias, game-tagged AAudio, DSCP marking,
the Wi-Fi low-latency lock, HDMI ALLM and the forced TV mode switch — now rides
the "Low-latency mode (experimental)" toggle, default OFF. Off restores the
pre-overhaul pipeline byte-for-byte: the sync poll loop, the platform-default
decoder, and the original format keys (standard low-latency + blind Qualcomm
twin + priority=0 + operating-rate=MAX together).

- New pref key (low_latency_mode_experimental): the old key shipped default-ON,
  so any install that ever saved settings persisted true — flipping the default
  under the old key would leave exactly the regressed devices stuck on.
- DSCP is applied at socket creation, so the toggle reaches the transport via
  NativeBridge.nativeSetLowLatencyMode → transport::set_dscp_default, called in
  the connect choke point before nativeConnect; the core DSCP default reverts
  to off everywhere.
- nativeStartAudio(handle, lowLatencyMode) gates AAudio usage=Game.
- VideoDecoders.pickDecoder now skips `.secure` decoder twins and decoders that
  require FEATURE_SecurePlayback: they need a secure surface, and a secure twin
  could out-score its plain sibling (only it advertising FEATURE_LowLatency),
  which black-screens a clear stream.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 20:18:59 +02:00
enricobuehler 0ea9c46d9f Merge remote-tracking branch 'origin/main'
apple / swift (push) Successful in 1m14s
apple / screenshots (push) Successful in 5m59s
android / android (push) Successful in 4m56s
arch / build-publish (push) Successful in 6m10s
ci / web (push) Successful in 1m6s
ci / docs-site (push) Successful in 1m17s
ci / rust (push) Successful in 4m56s
deb / build-publish (push) Successful in 3m20s
decky / build-publish (push) Successful in 18s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
ci / bench (push) Successful in 5m9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m49s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m55s
docker / deploy-docs (push) Successful in 20s
2026-07-06 20:02:48 +02:00
enricobuehler 0429e93167 fix(android): auto-wake opt-out + console glyphs default by form factor
An mDNS discovery miss was forcing connects through the Wake-on-LAN wait
even for a host that's already up; add a Settings toggle ("Auto-wake on
connect") that skips the mDNS-liveness gate and dials straight through
when off.

Also default the console UI's button glyphs by form factor instead of
always starting in TV-remote style: a phone/tablet only ever enters the
console UI via a real controller, so it should show gamepad glyphs from
the first frame, not a remote's select/back glyphs. TV keeps the remote
default.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 19:56:31 +02:00
enricobuehler 6e25b91b8e chore(release): regenerate openapi.json for 0.8.1
apple / swift (push) Successful in 1m19s
apple / screenshots (push) Failing after 2m56s
android / android (push) Successful in 4m17s
arch / build-publish (push) Successful in 6m15s
ci / rust (push) Successful in 5m6s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 3m20s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 30s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 5m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m58s
docker / deploy-docs (push) Successful in 18s
The 0.8.1 version bump (215a8aa) bumped Cargo.toml but not the generated
OpenAPI doc, leaving info.version stale at 0.8.0. Regenerated — version
string only, no API surface change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:43:06 +00:00
enricobuehler 05a6f39550 style: rustfmt drift from the native AMF SDK encoder
apple / swift (push) Successful in 1m16s
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
decky / build-publish (push) Successful in 22s
apple / screenshots (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
windows-host / package (push) Successful in 8m12s
deb / build-publish (push) Successful in 3m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m21s
docker / deploy-docs (push) Successful in 24s
The native AMF encoder (7649ccb) landed unformatted, failing CI's Format
step (and short-circuiting Clippy/Build/Test). Reformatted amf.rs with the
pinned rustfmt 1.96.0 — no functional change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 17:39:40 +00:00
enricobuehler 27d25b5f6d fix(client-linux): share one GamepadService across app activations
apple / swift (push) Successful in 1m30s
arch / build-publish (push) Successful in 9m24s
apple / screenshots (push) Successful in 6m48s
ci / rust (push) Failing after 36s
ci / web (push) Successful in 58s
android / android (push) Successful in 11m35s
ci / docs-site (push) Successful in 1m2s
ci / bench (push) Successful in 4m59s
decky / build-publish (push) Successful in 26s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m48s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 33s
deb / build-publish (push) Successful in 9m19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m24s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 55s
flatpak / build-publish (push) Successful in 4m49s
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
`build_ui` (the GTK `activate` handler) started a fresh SDL3 gamepad
worker thread on every reactivation of the already-running singleton
(another --connect, the launcher clicked twice, ...). sdl3 only lets
the first thread ever to call sdl3::init() hold "main thread" status,
so every later activation's worker thread failed permanently with
"Cannot initialize `Sdl` from a thread other than the main thread",
silently disabling controller support for the rest of the process.

Start the GamepadService once in run() and clone it into build_ui
instead of starting a new one per activation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 19:11:45 +02:00
enricobuehler f94978f820 feat(android): add higher bitrate options up to 500 Mbps
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
arch / build-publish (push) Has been cancelled
deb / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
decky / build-publish (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 16:31:27 +00:00
enricobuehler 58d7b64978 Merge remote-tracking branch 'origin/main'
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
windows-host / package (push) Has been cancelled
2026-07-06 16:29:48 +00:00
enricobuehler b6f59f5000 fix(apple): resolve macOS modifiers by keyCode — Control was silently dropped
apple / swift (push) Successful in 1m40s
apple / screenshots (push) Has been cancelled
android / android (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
Modifier keys arrive only as flagsChanged, and the direction was recovered
by diffing the device-dependent L/R bits (NX_DEVICE*KEYMASK) alone. Those
bits are undocumented and some keyboards omit them (only the class bit,
e.g. NX_CONTROLMASK, is set), so the diff saw no transition and the key
never reached the host — no Ctrl shortcuts. SDL/Moonlight key off the
event's keyCode for exactly this reason; do the same: keyCode names the
changed key, the class bit says up, the device bits (when present) pick
the side, and a tracked-held-state flip covers keyboards without them.

PUNKTFUNK_INPUT_DEBUG=1 now also logs every flagsChanged (keyCode + raw
flags) so a field report is diagnosable from client logs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 18:29:40 +02:00
enricobuehler a041b0aa96 fix(host): stop force-disabling zero-copy on Mutter+NVIDIA by default
apply_session_env unconditionally forced PUNKTFUNK_FORCE_SHM=1 for every
GNOME/Mutter session, added 2026-06-14 after a same-day stale-frame bug hunt
found Mutter has no implicit dmabuf fence on NVIDIA. That override silently
contradicted the documented "zero-copy is on by default for all Linux GPU
backends" behavior and left Mutter+NVIDIA hosts on the slower CPU/SHM path
unconditionally, with no way to opt back in.

Live retesting (192.168.1.21, RTX 5070 Ti, real client with cursor
movement/window drag/typing — the historical trigger) shows no visible
staleness with the override removed. Drop the automatic force; PUNKTFUNK_FORCE_SHM
stays as a manual escape hatch for anyone who does hit flashing/stale frames
on a Mutter+NVIDIA host.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 16:04:25 +00:00
enricobuehler 7649ccb66b feat(host): native AMF SDK encoder for Windows AMD — drop libavcodec
apple / swift (push) Successful in 1m26s
ci / rust (push) Failing after 38s
ci / web (push) Successful in 52s
windows-host / package (push) Successful in 7m26s
apple / screenshots (push) Successful in 6m8s
decky / build-publish (push) Successful in 13s
android / android (push) Successful in 4m58s
arch / build-publish (push) Successful in 5m31s
ci / docs-site (push) Successful in 1m1s
deb / build-publish (push) Successful in 3m20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m52s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m11s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m11s
Direct-SDK AMF encoder (encode/windows/amf.rs), the AMD analogue of the
direct-NVENC path, replacing the libavcodec *_amf dispatch. C-vtable FFI
pinned to AMF headers v1.4.36, runtime-loaded from the driver's amfrt64.dll
(no build feature, no new dependency) exactly as NVENC loads its DLL.

- AVC/HEVC (SDR NV12 + 10-bit HDR P010) and AV1 (RDNA3+, probed); a bounded
  poll retires the libavcodec ~2-frame output hold; native in-place reset().
- Intra-refresh wave (PUNKTFUNK_INTRA_REFRESH), in-band HDR mastering/CLL
  metadata (*InHDRMetadata -> HEVC SEI / AV1 OBU), and a native codec probe
  feeding the GameStream advertisement (windows_backend_is_ffmpeg ->
  windows_backend_is_probed).
- AMD dispatch / advertisement / 4:4:4 are native-only; the libavcodec AMF
  fallback and the PUNKTFUNK_AMF_FFMPEG hatch are removed. FFmpeg serves QSV
  only (its AMF path retained solely as the latency A/B comparator).
- Overload back-pressure: submit bounds in-flight surfaces below the input
  ring, draining finished AUs (buffered for poll, FIFO-preserved) to free a
  slot and retry on AMF_INPUT_FULL instead of tearing the encoder down and
  forcing an IDR; this also closes a latent ring-overwrite corruption seen
  under load on-glass.

Validated on the lab Ryzen iGPU (AMF runtime 1.4.37): HEVC/AVC across a
native reset, HEVC Main10 mastering+CLL SEIs byte-verified, intra-refresh
accepted, a backpressure burst FIFO-clean, and end-to-end via the macOS
client. Measured §5.2 latency A/B: native encode_us p50 ~5 ms (0.31 frame
periods) vs libavcodec ~17 ms (1.01). 4:4:4 stays unsupported (VCN hardware
limit). Live-gated tests skip cleanly on non-AMD boxes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 17:33:23 +02:00
enricobuehler efb49c5afd feat(windows-client): WOL wait-until-up + IP re-key (Apple/Android parity)
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m9s
apple / swift (push) Successful in 1m26s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m7s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 41s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 46s
apple / screenshots (push) Successful in 5m42s
android / android (push) Successful in 5m22s
arch / build-publish (push) Successful in 6m57s
ci / web (push) Successful in 57s
ci / rust (push) Successful in 5m20s
ci / docs-site (push) Successful in 1m1s
deb / build-publish (push) Successful in 3m20s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
ci / bench (push) Successful in 4m50s
flatpak / build-publish (push) Successful in 4m1s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m22s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m39s
Like the Linux client, the Windows client had WOL send + MAC storage + a Wake
action + fire-and-forget auto-wake, but no poll-until-up + IP re-key. Add the
polished flow (mirrors Apple HostWaker + the request_access screen pattern):

- connect::wake_and_connect — send the magic packet, show a cancelable
  Screen::Waking busy page, poll discovery::browse() until the host reappears
  (re-sending every 6 s, 90 s budget), then dial; re-key the saved host
  (KnownHosts::upsert) if it woke on a new IP.
- Screen::Waking + waking_page, routed in app/mod.rs (mirrors RequestAccess).
- the saved-host tile routes an offline-with-MAC tap to wake_and_connect;
  MENU_WAKE stays a pure send-only button.

Reviewed against the request_access reference — DiscoveredHost/KnownHost/Target
types, the widgets, .call()/.lock(), and the initiate signature all match;
compile-verified by Windows CI (no local Windows toolchain).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:50:51 +00:00
enricobuehler a5254c8798 feat(linux-client): WOL wait-until-up + IP re-key (Apple/Android parity)
The Linux client already had WOL send + MAC storage + a Wake action + auto-wake-
on-connect, but the auto-wake just fired a packet and did one dial to the stored
address — so a host that woke on a new DHCP lease failed, and there was no
"waiting" feedback. Add the polished flow (mirrors Apple/Android HostWaker):

- ui_trust::wake_and_connect — send the magic packet, poll mDNS until the host
  reappears (re-sending every 6 s, 90 s budget) behind a cancelable "Waking…"
  dialog, then connect; if it woke on a new IP, re-key the saved host first.
- trust::rekey_addr — no-churn addr/port update keyed by fingerprint.
- the hosts page routes an offline saved-host-with-MAC tap to on_wake_connect
  (the new flow) instead of fire-and-forget wake + immediate dial.

Builds + clippy + fmt clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:47:37 +00:00
enricobuehler 6081502949 feat(clients): signal explicit exit (QUIT_CLOSE_CODE) on deliberate disconnect
apple / swift (push) Successful in 1m10s
android / android (push) Successful in 5m5s
arch / build-publish (push) Successful in 5m28s
ci / web (push) Successful in 1m10s
windows-host / package (push) Successful in 7m9s
ci / docs-site (push) Successful in 1m21s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m19s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m11s
release / apple (push) Successful in 8m40s
ci / rust (push) Successful in 4m58s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 49s
apple / screenshots (push) Failing after 1m12s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 53s
ci / bench (push) Successful in 5m2s
decky / build-publish (push) Successful in 13s
deb / build-publish (push) Successful in 3m22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
flatpak / build-publish (push) Successful in 4m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 12m3s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m1s
docker / deploy-docs (push) Successful in 18s
The core's deliberate-quit close (NativeClient::disconnect_quit → QUIT_CLOSE_CODE,
host skips the keep-alive linger) was implemented but never called by any client.
Wire it to each client's explicit user-disconnect action — NOT to a network drop /
host-ended / app-background (those keep the linger for a reconnect):

- core: new C-ABI punktfunk_connection_disconnect_quit(c) for the ABI clients
- Linux (direct-core): Ctrl+Alt+Shift+D + the controller escape chord
- Windows (direct-core): Ctrl+Alt+Shift+D
- Apple (C-ABI): PunktfunkConnection.disconnectQuit() + a `deliberate` flag on
  SessionModel.disconnect() (sessionEnded passes false → keeps the linger)
- Android (JNI): new nativeDisconnectQuit export, called from the back gesture +
  the Select+Start+L1+R1 chord (not the host-gone watchdog)
- probe already did this via --quit (b71dc94)

Verified: core + Linux client + Android (cargo-ndk + gradle) build clean;
Windows/Apple compile-checked by CI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:37:45 +00:00
enricobuehler a2723e34a1 style: rustfmt drift from the AMF/watchdog commits
apple / swift (push) Successful in 1m32s
android / android (push) Successful in 4m57s
arch / build-publish (push) Successful in 5m42s
ci / web (push) Successful in 1m4s
windows-host / package (push) Successful in 7m57s
ci / docs-site (push) Successful in 1m22s
apple / screenshots (push) Successful in 6m50s
ci / rust (push) Successful in 4m55s
ci / bench (push) Successful in 4m54s
deb / build-publish (push) Successful in 3m24s
decky / build-publish (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m44s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m51s
docker / deploy-docs (push) Successful in 19s
`cargo fmt --all --check` (a CI gate) failed on main: config.rs (the new
`zerocopy: val(...).map { !matches!(...) }` from 8fb1264) and punktfunk1.rs
(the `reset_stalled_encoder` conditions from 06d594b) were left unwrapped by
the pinned rustfmt. Pure reformat, no semantic change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 13:00:23 +00:00
enricobuehler 8df6d07cc6 fix(android): launch games from the library
The library browser was browse-only — the A button (and a tap) did nothing.
Wire it to connect + boot straight into the selected title: thread a `launch`
id (the store-qualified library id `steam:<appid>` / `custom:<id>`) through
nativeConnect → NativeClient's Hello.launch (was hardcoded None), add a shared
connectToHost() the ConnectScreen and the library launcher both use, and have
LibraryScreen dial the host with launch=game.id on A / tap — with a launching
overlay + an "A Launch" hint. Verified: native compiles (cargo-ndk arm64),
app+kit Kotlin compiles (gradle, 3 ABIs).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 12:59:39 +00:00
enricobuehler 2c1bb4de93 docs(design): native AMF encoder handoff — drop ffmpeg for AMF, keep for QSV
apple / swift (push) Successful in 1m13s
android / android (push) Failing after 2m16s
ci / rust (push) Failing after 40s
ci / web (push) Successful in 53s
apple / screenshots (push) Failing after 3m21s
ci / docs-site (push) Successful in 1m0s
arch / build-publish (push) Successful in 5m17s
windows-host / package (push) Successful in 7m30s
deb / build-publish (push) Successful in 3m19s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
ci / bench (push) Successful in 4m51s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m39s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 47s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m47s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m33s
docker / deploy-docs (push) Successful in 20s
Design + decision record for replacing the libavcodec *_amf path with a
direct AMF SDK encoder (encode/windows/amf.rs, the AMD analogue of the direct
NVENC path). Motivation is measured, not speculative: the libavcodec wrapper
structurally holds ~2 frames (36 ms p50 at 720p60 on VCN, un-tunable), driver
wedges surface as forever-EAGAIN instead of typed AMF_RESULTs, and
intra-refresh / in-band HDR SEI are inexpressible through it. Covers the C
vtable FFI strategy (amfrt64.dll runtime-load, FFmpeg amfenc.c and OBS
texture-amf.cpp as references), bounded-poll retrieval, the property table,
watchdog/reset interplay, exact dispatch seams, a lab-iGPU validation plan
with today's zero-copy baselines, and three phases ending with the ffmpeg-AMF
arm deleted (FFmpeg stays QSV-only).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 14:36:55 +02:00
enricobuehler 8fb126482e feat(host): default the Windows AMF encode input to zero-copy D3D11
On-glass A/B on the Ryzen 7000 iGPU (1080p120 HDR P010, hevc_amf,
PUNKTFUNK_PERF stage split): the system-memory readback costs the encode
thread 2.7-2.9 ms p50 (6.6 ms p99) per frame in submit; the zero-copy D3D11
pool path does the same work in 0.26 ms p50 (0.5 ms p99) — and on an iGPU the
readback also burns the shared memory bandwidth the game needs. The docs-site
already promised "on by default ... D3D11 on Windows" since the Linux flip
(2f9e61d was Linux-only); the Windows code now delivers it.

PUNKTFUNK_ZEROCOPY becomes a tri-state override: unset defers to a per-vendor
default in zerocopy_enabled(vendor) — ON for AMF (validated above; open
failures still fall back to system-memory readback), OFF for QSV until it is
validated on Intel glass (the fallback only catches *setup* errors; a QSV
derive that opens but maps wrong would corrupt silently, so probe-never-assume
applies). Explicit values force either way: 0|false|off|no = readback,
anything else = zero-copy, so the old presence-style =1 keeps working.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 14:36:43 +02:00
enricobuehler 06d594bc2a fix(host): encode-stall watchdog — heal the silent AMF/QSV freeze in place
Field reports: Windows AMD/Intel streams freeze after ~3-5 min regardless of
desktop activity. Root cause: the libavcodec AMF/QSV poll is non-blocking
(EAGAIN -> Ok(None)), and the encode loop's drain treated None as benign
without popping `inflight` — a wedged driver (QueryOutput stops producing)
meant frames kept being submitted, inflight grew unboundedly, no AU ever
reached the send thread, and nothing logged: a silent permanent freeze. The
input-side twin: once libavcodec's one-frame buffer fills, avcodec_send_frame
EAGAINs and the submit `?` killed the whole session.

Add `Encoder::reset()` (in-place encoder rebuild; implemented for AMF/QSV by
dropping the wedged libavcodec encoder so the next submit re-opens it on the
current device, forced IDR) and an encode-stall watchdog in the stream loop:
trip on a poll error, on no AU within max(2 s, 8 frame intervals) while frames
are owed, or on an owed backlog worth more than the window's frames (the
slow-leak latency-runaway form). Recovery is a bounded (5 consecutive, cleared
by any delivered AU) in-place rebuild + forced IDR — a logged ~one-second
hiccup instead of a dead stream; exhaustion or a reset-less backend still
fails the session with a clear error. Submit failures route through the same
bounded recovery. The three existing pipeline-rebuild paths (session switch,
mode switch, capture loss) now also clear the stale in-flight records that
pointed at the dropped encoder.

Backends whose poll blocks (direct NVENC sync, software) can't false-trip:
they never return Ok(None) mid-stream and drain inflight below depth each
tick. Validated: clippy -D warnings (nvenc,amf-qsv), 191 host tests, synthetic
E2E 300/300 frames, and an on-glass AMD iGPU session (1080p120 HDR hevc_amf).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 14:36:29 +02:00
enricobuehler 39889c3102 fix(host): silence write-only cur_node_id on non-Linux encoder builds
apple / swift (push) Successful in 1m10s
android / android (push) Successful in 6m0s
arch / build-publish (push) Successful in 6m5s
ci / web (push) Successful in 1m1s
windows-host / package (push) Successful in 7m17s
apple / screenshots (push) Successful in 6m43s
ci / docs-site (push) Successful in 1m18s
ci / bench (push) Successful in 4m58s
ci / rust (push) Successful in 9m27s
decky / build-publish (push) Successful in 26s
deb / build-publish (push) Successful in 6m37s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 4m9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 28s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m41s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m17s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m30s
docker / deploy-docs (push) Successful in 24s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m17s
cur_node_id (the capture 5-tuple's node id, added for the Linux dedicated-
game-exit check) is read only under #[cfg(target_os = "linux")], so on the
Windows nvenc/amf-qsv build it was assigned but never read — failing
`clippy -D warnings`. Read it on non-Linux platforms (the `let _ = &launch`
idiom already used in this file).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:49:45 +00:00
enricobuehler 215a8aa9dc chore(release): bump workspace version to 0.8.1
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-host / package (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
release / apple (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
decky / build-publish (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
audit / cargo-audit (push) Has been cancelled
audit / bun-audit (push) Has been cancelled
apple / swift (push) Has been cancelled
deb / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
arch / build-publish (push) Has been cancelled
apple / screenshots (push) Has been cancelled
android / android (push) Has been cancelled
Release 0.8.1: game-mode + dedicated game sessions, zero-copy GPU import
process-isolation (and zero-copy on by default on all backends), user-defined
custom display presets, and a physical-monitor refresh-preservation fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:38:14 +00:00
enricobuehler 2f9e61d191 feat(host): enable GPU zero-copy by default on all backends
apple / swift (push) Failing after 45s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m15s
android / android (push) Successful in 4m56s
windows-host / package (push) Failing after 3m31s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
arch / build-publish (push) Successful in 5m39s
release / apple (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
apple / screenshots (push) Has been skipped
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
decky / build-publish (push) Has been cancelled
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m23s
ci / rust (push) Successful in 4m46s
ci / bench (push) Successful in 4m57s
deb / build-publish (push) Successful in 4m22s
Now that the per-capture worker subprocess makes an NVENC EGL/CUDA driver
fault survivable (design/zerocopy-worker-isolation.md), the reason the NVENC
zero-copy path stayed opt-in is gone. zerocopy::enabled() now defaults ON for
both GPU backends (was ON VAAPI / OFF NVENC). Fallbacks are intact: VAAPI's
one-shot CPU auto-downgrade (VAAPI-gated, never trips for NVENC) and NVENC's
per-capture fallback + worker-death latch.

Reframe the shipped host.env examples and setup guides to rely on the default
rather than force PUNKTFUNK_ZEROCOPY=1 (an explicit =1 skips the VAAPI
auto-downgrade).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler a947f48d29 fix(host): preserve a physical monitor's refresh when adding a virtual display
Connecting reset an existing physical monitor's refresh (e.g. 120->60 Hz)
because the topology code read the physical's mode AFTER the virtual output
perturbed the compositor layout — by which point it had already been
downgraded. Read/preserve each physical's mode from a pre-connect snapshot.

- Mutter: build_primary_keeping_physicals takes the pre-virtual snapshot and
  preserves each physical's real mode (pick_keep_mode, unit-tested)
- KWin: capture each output's mode when disabling for exclusive, re-assert it
  on re-enable (a bare enable defaulted to ~60 Hz)
- Windows: skip the refresh-resetting SDC_TOPOLOGY_EXTEND when a physical is
  already active

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler a4f81dec48 feat(host): user-defined custom display presets
Save named bundles of the display-management policy (the six behavior axes
plus the game-session axis) as custom presets, alongside the built-ins. A
custom preset is data — stored in <config>/display-presets.json — not a Preset
enum variant, so DisplayPolicy::effective() stays pure and the built-in set is
untouched; applying one writes a Custom policy via the existing PUT
/display/settings.

- policy.rs: CustomPreset/CustomPresetInput + load/add/update/delete store
- mgmt.rs: GET/POST /display/presets + PUT/DELETE /display/presets/{id},
  surfaced on GET /display/settings
- web console: custom-preset cards with save-as / edit / delete + i18n
- regenerated api/openapi.json; docs

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler 077d8f85ca feat(host): isolate the zero-copy GPU import in a worker process
The tiled EGL/GL→CUDA import crashed the whole host (SIGSEGV inside
libnvidia-eglcore via cuGraphicsMapResources) when the compositor
invalidated an imported dmabuf mid-map — reproduced on the Bazzite F44
Game→Desktop switch (design/zerocopy-hardening-handoff.md). A driver
SIGSEGV is uncatchable in-process, so the whole EglImporter (tiled
EGL/GL→CUDA and LINEAR Vulkan→CUDA) now runs in a per-capture
`zerocopy-worker` subprocess: dmabuf fds go over a SEQPACKET socketpair
(SCM_RIGHTS, sent once per buffer keyed by dmabuf st_ino; NeedFd resend
self-heals cache desync), frames come back as CUDA-IPC pooled device
buffers (still zero-copy, +one socket RTT/frame). Worker death poisons
the capturer so the existing capture-loss rebuild runs — the host
survives; 3 consecutive deaths latch the GPU import off (CPU/SHM path).
PUNKTFUNK_ZEROCOPY_INPROC=1 keeps the old in-process import for
debugging/A-B.

Also fixed along the way: a failed *tiled* import no longer falls
through to the CPU mmap de-pad (which scrambled tiled bytes; LINEAR
keeps the fallback); Nv12Blit dropped its GL textures while still
CUDA-registered (unregister now runs first); GlBlit had no Drop at all
(GL objects leaked per size change); VkBridge's per-fd src cache is now
invalidated on renegotiation/eviction instead of never.

Design: design/zerocopy-worker-isolation.md. Unit tests: 14 new
(protocol fd-passing, worker dispatch, client handshake/death/NeedFd,
death latch). On-glass validated on the RTX 5070 Ti/GNOME box (.21):
the worker path streams at p50 1.30 ms (NV12, 1800 frames 0-mismatched,
parity with the in-process path), and a kill -9 of the worker
mid-stream is survived by the host and recovered — poison -> capture
lost, rebuilding pipeline in place -> a fresh worker in ~185 ms ->
streaming resumes (2385 frames, 0 mismatched). A real KWin
compositor-crash repro is still pending (a worker kill -9 is strictly
harsher, so it corroborates).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler 07d46f865c docs: zero-copy EGL/CUDA capture-crash hardening handoff
Describes a reproduced host SIGSEGV in cuGraphicsMapResources (inside
libnvidia-eglcore), reached via zerocopy::cuda / zerocopy::egl on the tiled
EGL/GL->CUDA capture path, when the KWin dmabuf is invalidated mid-map (observed
on .181 during a Game->Desktop switch under zero-copy, with the compositor itself
crashing). Pre-existing capture-layer issue, not the gamemode work. Issue
description + root cause + solution-space considerations only -- the next agent
plans the implementation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler 1271fd5ea2 feat(host): game-mode integration + dedicated game sessions
Implements design/gamemode-and-dedicated-sessions.md (Parts A1-A5 + B0-B2):
reconciles the merged display-management registry with session-mobile
Bazzite/SteamOS hosts and adds a per-launch dedicated gamescope mode.

- A1 DisplayOwnership {Owned,External,SessionManaged} + poolable_now(): the
  registry pools only what it owns, so gamescope managed/attach outputs are no
  longer double-owned by the registry AND the gamescope restore worker (fixes
  the game-mode-reconnect stale-node wedge).
- A2 validated reuse: (backend,mode,launch,epoch) reuse key + kept_display_alive
  liveness probe + reused_gen/mark_failed on a reused-display first-frame failure.
- A3 policy-driven managed restore (keep_alive replaces the hardcoded 5s debounce;
  forever = held = gaming-rig truthful) + crash-restore persist + SIGKILL teardown
  (kill_unit, applied to our transient unit AND the autologin stop -- validated
  live on .181 to avoid the F44 GPU-context leak).
- A4 session epoch: observe_session_instance bumps the epoch + invalidate_backend
  on a desktop-compositor instance change; gamescope spawns are exempt.
- A5 per-spawn log + PID-scoped gamescope node discovery.
- B0 game_session {auto,dedicated} policy (top-level, preset-orthogonal) +
  pick_gamescope_mode dedicated_launch + steam -silent command shaping.
- B1 free the autologin Steam before a dedicated Steam spawn (single-instance).
- B2 game-exit -> APP_EXITED_CLOSE_CODE (0x52) clean session end.

Adversarially reviewed (11 findings fixed). Validated on glass (.181 Bazzite F44,
RTX 4090): dedicated spawn streams a real game smoothly; keep-alive reuse; the
SIGKILL fix avoids the F44 vkCreateDevice leak. Workspace green
(build / test --workspace / clippy -D warnings / fmt), OpenAPI + C header
regenerated, web console tsc + vite build green. clients/probe: bump the
no-video timeout 8s->45s for gamescope cold starts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 10:33:43 +00:00
enricobuehler 42b45113a9 fix(android): pass dropped streams through to the ui
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-host / package (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
2026-07-06 12:14:40 +02:00
enricobuehler d76a42e0e9 docs(security): add 2026-07-05 whole-project posture audit
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
arch / build-publish (push) Has been cancelled
audit / bun-audit (push) Has been cancelled
audit / cargo-audit (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
Assessment of the overall security architecture, the state of the prior
reviews' remediations, and the process/supply-chain controls. Flags the
two web-console residuals (no brute-force throttle; cookie seal key from
the login password) that the accompanying web-hardening change fixes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:28:51 +02:00
enricobuehler 5a51b0d8e7 ci: SHA-pin secret-bearing third-party actions + audit the web dep tree
- Pin android-actions/setup-android, appleboy/scp-action, and
  appleboy/ssh-action to commit SHAs (version kept in a trailing comment).
  These run in jobs holding the Android signing keystore, Play
  service-account, and deploy SSH key, so a moved tag on a third-party
  action could exfiltrate them.
- Add a bun-audit job to audit.yml over web/bun.lock — the console holds
  the login gate, session sealing, and mgmt token, so its deps matter too
  — and trigger the workflow on web/bun.lock changes alongside Cargo.lock.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:28:45 +02:00
enricobuehler bda5556d37 feat(web): harden login gate — throttle, scoped TLS, token-derived seal key
Remediates the two web-console residuals from the 2026-07-05 posture audit:

- Brute-force throttle (loginThrottle.ts): per-IP exponential backoff
  after 5 free attempts, plus a global floor for spread-out floods, keyed
  on the socket peer IP (not spoofable X-Forwarded-For) with a size-capped
  map. The constant-time compare already stopped the timing leak; this
  bounds guess *volume* against a by-design LAN-exposed console.
- Session seal key now derives from the high-entropy mgmt token instead of
  the low-entropy login password, so a captured cookie is no longer an
  offline password oracle. Falls back to the password only when no token
  is configured (dev/local). Rotating the token now invalidates sessions.
- Replace the process-wide NODE_TLS_REJECT_UNAUTHORIZED=0 with per-request
  Bun TLS scoped to the loopback proxy hop; a non-loopback mgmt URL now
  verifies normally. Dropped the env var from the systemd unit, Steam Deck
  installer, Windows run scripts, env examples, and web README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:28:39 +02:00
enricobuehler ab56536842 docs(readme): Windows host capture is IDD-push, not DXGI/WGC
The DXGI Desktop Duplication + WGC relay paths were removed; sealed
IDD-push (finished frames pushed straight into the host's own IddCx
driver, no screen-scraping) is now the sole Windows capture path. Fix the
stale "DXGI/WGC capture" claims in the root and punktfunk-host READMEs,
which also contradicted the push-based IDD description already present in
the root README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:28:29 +02:00
enricobuehler 27c53a4b53 perf(android): low-latency decode overhaul — vendor keys, async loop, system tuning
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 9m21s
Close the latency gap on the Android client with per-SoC decoder tuning, an
event-driven decode loop, and full system integration.

- Decoder selection: rank MediaCodecList decoders in Kotlin (hardware/vendor
  preferred, software avoided, FEATURE_LowLatency probed) and create the chosen
  one by name. Per-SoC low-latency keys gated on the codec-name prefix: Qualcomm
  picture-order + low-latency, Exynos (also Google Tensor), Amlogic, HiSilicon;
  MediaTek vdec-lowlatency set unconditionally. operating-rate = MAX (Qualcomm)
  vs priority = 0 (else) are mutually exclusive. NVIDIA/Rockchip/Realtek have no
  vendor key — covered by ranking + the standard low-latency key.

- Async decode loop: AMediaCodec async-notify replaces the poll loop, presenting a
  decoded frame the instant it is ready instead of waiting out a poll interval.
  Behind USE_ASYNC_DECODE with the synchronous loop kept for A/B during bring-up.

- System integration: Wi-Fi FULL_LOW_LATENCY lock and HDMI ALLM
  (setPreferMinimalPostProcessing) for the stream's lifetime; game_mode_config.xml
  opting out of OEM downscaling / FPS overrides.

- Pipeline: boost the data-plane pump + audio thread priorities, AAudio usage=Game,
  DSCP marking on by default on Android, ADPF setPreferPowerEfficiency(false),
  and setFrameRateWithChangeStrategy(ALWAYS) to force the HDMI mode switch on TV.

- lowLatencyMode master toggle (default on) as the escape hatch; the stats HUD now
  shows the resolved decoder name.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:25:16 +02:00
enricobuehler 69fcb6e0b1 docs: restructure host setup by distro, configuration by compositor
apple / swift (push) Successful in 1m8s
apple / screenshots (push) Successful in 5m33s
android / android (push) Successful in 4m43s
arch / build-publish (push) Successful in 5m38s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m17s
ci / rust (push) Successful in 4m48s
ci / bench (push) Successful in 5m7s
decky / build-publish (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
deb / build-publish (push) Successful in 4m29s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m24s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m3s
Split the docs' single distro×desktop axis (ubuntu-gnome / ubuntu-kde / fedora-kde) into two,
which deduplicates the shared mechanics and scales to distros that run several desktops (Arch):

- Install the host — per distro/OS (ubuntu, fedora, arch, bazzite, steamos-host, windows-host):
  GPU driver + package + input group, then a canonical "Configure your desktop" funnel.
- Configure your desktop — per compositor (kde, gnome, gamescope, sway): host.env, compositor
  quirks, the headless session, and starting the host.

New shared web-console page (enable · login password · arm pairing) removes the console/password
block that was copy-pasted across all seven host pages. Merged ubuntu-gnome + ubuntu-kde into
ubuntu; renamed fedora-kde to fedora; kept bazzite and steamos-host as dedicated appliance guides
(trimmed of duplication). Moved the KWin headless session, the GNOME EGL/lock traps, and the
gamescope attach/managed model out of the distro pages onto their compositor pages.

Fixed while restructuring: distro-specific paths on kde (kde-desktop-setup.sh is Fedora/Bazzite-only;
the .deb ships host.env.kde under /usr/share/punktfunk-host), the interactive "start the host" step
that was lost in the merge, sway over-claiming Hyprland, and a pre-existing broken anchor in
how-it-works.

Removal of the three old pages was captured by the preceding commit 8ebb614 (a concurrent commit
swept up the staged git-rm); the net docs tree is correct. Fumadocs build + internal link/anchor
check green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 21:04:31 +00:00
enricobuehler 8ebb61400c fix(web): clearer topology/identity copy, capped description width, mobile More-nav, preset spacing
apple / swift (push) Successful in 1m12s
apple / screenshots (push) Successful in 5m18s
windows-host / package (push) Successful in 7m38s
android / android (push) Successful in 12m11s
arch / build-publish (push) Successful in 9m7s
ci / web (push) Successful in 1m7s
ci / docs-site (push) Successful in 1m18s
ci / rust (push) Successful in 4m47s
ci / bench (push) Successful in 5m3s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 32s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6s
deb / build-publish (push) Successful in 4m31s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 57s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 10m48s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 10m28s
docker / deploy-docs (push) Successful in 20s
Console polish on the Virtual displays card + shell:
- Topology help now leads with the streamed display's role (Extend/Primary/Exclusive) instead of
  the confusing physical-monitor-only framing; notes the headless case. Identity help spells out the
  actual behavior (stable per-client identity → the desktop reapplies that client's scaling/resolution
  on reconnect) + what Shared / Per-client / Per-client+resolution each do.
- Cap description/help width at max-w-prose so long help text isn't a full-viewport line on large screens.
- Mobile bottom nav: 8 flat tabs were too cramped → 4 pinned tabs + a "More" tab whose sheet holds the
  rest (Performance/Logs/Pairing/Settings), "More" highlighted when the active route is in the overflow.
- More breathing room under the "Preset" heading.

web tsc + biome + vite build green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 20:40:35 +00:00
128 changed files with 11708 additions and 1210 deletions
+2 -1
View File
@@ -25,7 +25,8 @@ jobs:
java-version: "21"
- name: Android SDK
uses: android-actions/setup-android@v3
# SHA-pinned for parity with android.yml (third-party action). v3 = 9fc6c4e.
uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3
# No NDK/CMake — the screenshot unit tests are pure JVM. compileSdk 37 auto-downloads via AGP
# if the platform channel lacks it (same note as android.yml).
+3 -1
View File
@@ -43,7 +43,9 @@ jobs:
"$RUSTUP" target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android
- name: Android SDK
uses: android-actions/setup-android@v3
# SHA-pinned: this workflow's release job carries the signing keystore + Play service-account
# secrets, so a moved tag on a third-party action could exfiltrate them. v3 = 9fc6c4e.
uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3
- name: NDK r30 + platform 36 + build-tools + CMake (libopus cross-build)
# cmake;3.22.1 installs cmake + ninja under $ANDROID_SDK/cmake/3.22.1/bin — the exact path
+30 -5
View File
@@ -1,7 +1,10 @@
# Supply-chain advisory scan for the (network-facing, crypto-heavy) Rust dependency tree.
# Runs `cargo audit` against the RustSec advisory DB: weekly (catch newly-disclosed CVEs in
# pinned deps), on every Cargo.lock change (catch a bad dep the moment it lands), and on demand.
# To silence a known-unfixable advisory, add it to `.cargo/audit.toml` ([advisories] ignore = [...]).
# Supply-chain advisory scan for BOTH dependency trees the project ships to users:
# * cargo-audit → the (network-facing, crypto-heavy) Rust tree, against the RustSec advisory DB.
# * bun audit → the web management console (Nitro/Bun BFF) — the component that holds the login
# gate, session sealing, and the mgmt bearer token, so its deps matter too.
# Triggers: weekly (catch newly-disclosed CVEs in pinned deps), on every lockfile change (catch a bad
# dep the moment it lands), and on demand.
# To silence a known-unfixable Rust advisory, add it to `.cargo/audit.toml` ([advisories] ignore=[…]).
name: audit
on:
@@ -9,7 +12,7 @@ on:
- cron: '0 6 * * 1' # Mondays 06:00 UTC
push:
branches: [main]
paths: ['Cargo.lock', '.gitea/workflows/audit.yml']
paths: ['Cargo.lock', 'web/bun.lock', '.gitea/workflows/audit.yml']
workflow_dispatch:
jobs:
@@ -31,3 +34,25 @@ jobs:
git config --global --add safe.directory "$PWD"
command -v cargo-audit >/dev/null 2>&1 || cargo install --locked cargo-audit
cargo audit
bun-audit:
runs-on: ubuntu-24.04
container:
image: oven/bun:1
timeout-minutes: 15
defaults:
run:
working-directory: web
steps:
# oven/bun's slim base lacks a CA bundle + git — actions/checkout's HTTPS fetch needs them
# (same preamble as web-screenshots.yml / ci.yml's web job).
- name: Install git + CA certs
working-directory: /
run: apt-get update && apt-get install -y --no-install-recommends ca-certificates git
- uses: actions/checkout@v4
# `bun audit` queries the registry advisory DB for the versions pinned in web/bun.lock. No
# install/build needed — it reads the manifest + lockfile. Fails the job on any advisory, the
# same fail-on-vulnerability stance as cargo-audit above; triage a finding by bumping the dep
# (or, if genuinely unfixable + inapplicable, pinning a resolution and noting why here).
- name: bun audit
run: bun audit
+6 -2
View File
@@ -86,7 +86,10 @@ jobs:
- uses: actions/checkout@v4
- name: Sync compose file
uses: appleboy/scp-action@v0.1.7
# SHA-pinned (not tag-pinned): this action receives DEPLOY_SSH_KEY + host/user/port, so a
# moved tag would mean credential exfiltration. v0.1.7 = 917f8b8. Bump both the SHA and the
# trailing version together when upgrading.
uses: appleboy/scp-action@917f8b81dfc1ccd331fef9e2d61bdc6c8be94634 # v0.1.7
with:
host: ${{ secrets.DEPLOY_HOST }}
username: ${{ secrets.DEPLOY_USER }}
@@ -97,7 +100,8 @@ jobs:
overwrite: true
- name: Pull and start docs
uses: appleboy/ssh-action@v1.2.5
# SHA-pinned: receives DEPLOY_SSH_KEY + REGISTRY_TOKEN (see the scp step above). v1.2.5 = 0ff4204.
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2 # v1.2.5
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
with:
Generated
+9 -9
View File
@@ -2129,7 +2129,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.8.0"
version = "0.8.1"
[[package]]
name = "lazy_static"
@@ -2261,7 +2261,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"punktfunk-core",
]
@@ -2908,7 +2908,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"android_logger",
"jni",
@@ -2922,7 +2922,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"anyhow",
"async-channel",
@@ -2945,7 +2945,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"anyhow",
"async-channel",
@@ -2968,7 +2968,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"aes-gcm",
"bytes",
@@ -2999,7 +2999,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"aes",
"aes-gcm",
@@ -3071,7 +3071,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3085,7 +3085,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.8.0"
version = "0.8.1"
dependencies = [
"anyhow",
"ksni",
+1 -1
View File
@@ -17,7 +17,7 @@ members = [
exclude = ["packaging/linux/steam-deck-gadget/usbip-poc"]
[workspace.package]
version = "0.8.0"
version = "0.8.1"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
+1 -1
View File
@@ -57,7 +57,7 @@ protocol, FEC, and crypto, linked into the host and every client over a stable C
| **Core**`punktfunk-core` + C ABI (protocol · FEC · crypto · QUIC) | ✅ Complete & hardened |
| **GameStream host** → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads |
| **Native protocol**`punktfunk/1` | ✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation |
| **Windows host** (Windows 11 22H2+, x64) | 🟡 Implemented & shipping as a signed installer: DXGI/WGC capture · its own all-Rust IddCx **virtual display** (secure-desktop capable) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel, software H.264 without a GPU) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
| **Windows host** (Windows 11 22H2+, x64) | 🟡 Implemented & shipping as a signed installer: its own all-Rust IddCx **virtual display** (secure-desktop capable) with a **sealed IDD-push** capture path — finished frames pushed straight into its own driver, not screen-scraped (no DDA/WGC) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel, software H.264 without a GPU) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
| **macOS / iOS / tvOS client** (`clients/apple`) | ✅ Streaming live: VideoToolbox decode, controllers incl. DualSense, discovery, pairing, speed test |
| **Linux client** (`clients/linux`, GTK4) | ✅ Streaming live: FFmpeg + VAAPI zero-copy decode, PipeWire audio, SDL3 controllers; ships as Flatpak/apt/rpm/Arch |
| **Android client** (`clients/android`, phone + TV) | ✅ Streaming live: AMediaCodec decode + HDR10, AAudio audio, controllers, discovery, pairing |
+298 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0"
},
"version": "0.7.4"
"version": "0.8.1"
},
"paths": {
"/api/v1/clients": {
@@ -190,6 +190,237 @@
}
}
},
"/api/v1/display/presets": {
"get": {
"tags": [
"display"
],
"summary": "List the saved custom presets",
"description": "The operator's named field-bundles (`display-presets.json`). These also ride the\n`GET /display/settings` response (`custom_presets`), so the console rarely needs this directly.",
"operationId": "listCustomPresets",
"responses": {
"200": {
"description": "The saved custom presets",
"content": {
"application/json": {
"schema": {
"type": "array",
"items": {
"$ref": "#/components/schemas/CustomPreset"
}
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
},
"post": {
"tags": [
"display"
],
"summary": "Save a custom preset",
"description": "Stores a named bundle of the display-behavior axes (+ the game-session axis) the operator can\napply later. The host assigns a stable id, returned in the body. Applying a preset is a\n`PUT /display/settings` with a `Custom` policy carrying its `fields` — no separate apply route.",
"operationId": "createCustomPreset",
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CustomPresetInput"
}
}
},
"required": true
},
"responses": {
"201": {
"description": "Preset created",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CustomPreset"
}
}
}
},
"400": {
"description": "Empty name",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Could not persist the catalog",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/display/presets/{id}": {
"put": {
"tags": [
"display"
],
"summary": "Update a custom preset",
"operationId": "updateCustomPreset",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The custom preset id",
"required": true,
"schema": {
"type": "string"
}
}
],
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CustomPresetInput"
}
}
},
"required": true
},
"responses": {
"200": {
"description": "Preset updated",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/CustomPreset"
}
}
}
},
"400": {
"description": "Empty name",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No custom preset with that id",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Could not persist the catalog",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
},
"delete": {
"tags": [
"display"
],
"summary": "Delete a custom preset",
"description": "Removes it from the catalog. The active policy is untouched — if this preset was the one applied,\nthe running behavior stays exactly as it was (the catalog and `display-settings.json` are decoupled).",
"operationId": "deleteCustomPreset",
"parameters": [
{
"name": "id",
"in": "path",
"description": "The custom preset id",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"204": {
"description": "Preset deleted"
},
"401": {
"description": "Missing or invalid bearer token",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"404": {
"description": "No custom preset with that id",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
},
"500": {
"description": "Could not persist the catalog",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ApiError"
}
}
}
}
}
}
},
"/api/v1/display/release": {
"post": {
"tags": [
@@ -2220,6 +2451,52 @@
}
}
},
"CustomPreset": {
"type": "object",
"description": "A user-defined named preset: a saved bundle of the six display-behavior axes (exactly what a\nbuilt-in [`Preset`] expands to) plus the orthogonal game-session axis, that the operator names\nand applies from the console.\n\nUnlike the built-in [`Preset`]s (a closed enum), custom presets are **data** — a catalog stored in\n`<config>/display-presets.json`. Applying one writes a `Custom` [`DisplayPolicy`] carrying these\nfields (the console reuses `PUT /display/settings`), so [`DisplayPolicy::effective`] stays pure and\nthe built-in set is never touched. The catalog is decoupled from the active `display-settings.json`:\nediting or deleting a preset never mutates the running policy (re-apply to adopt a change).",
"required": [
"id",
"name",
"fields"
],
"properties": {
"fields": {
"$ref": "#/components/schemas/EffectivePolicy",
"description": "The six display-behavior axes this preset applies (the same shape a built-in preset expands to)."
},
"game_session": {
"$ref": "#/components/schemas/GameSession",
"description": "The game-session routing this preset applies (orthogonal to the six axes; see [`GameSession`]).\nA custom preset captures the operator's *full* setup, so — unlike a built-in preset — applying\none does set this axis."
},
"id": {
"type": "string",
"description": "Host-assigned, stable for the life of the entry (the `{id}` in the CRUD path)."
},
"name": {
"type": "string",
"description": "User-facing name shown on the preset card; editable."
}
}
},
"CustomPresetInput": {
"type": "object",
"description": "Request body to create or replace a custom preset (no `id` — the host owns it).",
"required": [
"name",
"fields"
],
"properties": {
"fields": {
"$ref": "#/components/schemas/EffectivePolicy"
},
"game_session": {
"$ref": "#/components/schemas/GameSession"
},
"name": {
"type": "string"
}
}
},
"DisplayLayoutRequest": {
"type": "object",
"description": "Request body for `setDisplayLayout`: per-identity-slot desktop offsets, keyed by the identity-slot\nid as a string (the same id `/display/state` reports as `identity_slot`).",
@@ -2240,6 +2517,10 @@
"type": "object",
"description": "The user-facing display-management policy — what `display-settings.json` holds and what the mgmt\nAPI GETs/PUTs. When [`preset`](Self::preset) is not [`Preset::Custom`] the explicit fields are\nignored (the console writes one or the other); [`effective`](Self::effective) resolves both to a\nsingle [`EffectivePolicy`].",
"properties": {
"game_session": {
"$ref": "#/components/schemas/GameSession",
"description": "How a game-launching session is served (`design/gamemode-and-dedicated-sessions.md` §5.2).\nOrthogonal to `preset`/lifecycle — preserved across preset changes; `#[serde(default)]` = `Auto`\nso existing `display-settings.json` files are untouched."
},
"identity": {
"$ref": "#/components/schemas/Identity"
},
@@ -2280,6 +2561,7 @@
"configured",
"effective",
"presets",
"custom_presets",
"enforced"
],
"properties": {
@@ -2287,6 +2569,13 @@
"type": "boolean",
"description": "True once a `display-settings.json` exists (the console has configured this host)."
},
"custom_presets": {
"type": "array",
"items": {
"$ref": "#/components/schemas/CustomPreset"
},
"description": "The operator's saved custom presets (`display-presets.json`) — named field-bundles rendered\nalongside the built-ins. Managed via `POST/PUT/DELETE /display/presets`; applied by writing a\n`Custom` policy carrying the preset's fields."
},
"effective": {
"$ref": "#/components/schemas/EffectivePolicy",
"description": "The effective (preset-expanded) policy currently in force."
@@ -2399,6 +2688,14 @@
}
}
},
"GameSession": {
"type": "string",
"description": "How a session that **launches a game** (a library id on the Hello / apps.json / Decky pin) is\nserved (`design/gamemode-and-dedicated-sessions.md` §5.2). Orthogonal to the preset/lifecycle axes\n— a top-level [`DisplayPolicy`] field, NOT part of [`EffectivePolicy`], so a preset never clobbers\nit. Linux-only in effect (a launching Windows session opens into the one desktop).",
"enum": [
"auto",
"dedicated"
]
},
"GpuState": {
"type": "object",
"description": "Full GPU-selection state for the console: inventory, the persisted preference, what the next\nsession will use, and what is in use right now.",
@@ -43,6 +43,14 @@
android:supportsRtl="true"
android:theme="@style/Theme.PunktfunkAndroid">
<!-- Game Mode config (Android 13+): declare we support Performance mode and opt OUT of the
OEM interventions that would fight the negotiated stream — resolution downscaling and
FPS overrides. A game-streaming client renders exactly the host's mode; a platform
downscale/FPS-cap corrupts that. Ignored below API 33. -->
<meta-data
android:name="android.game_mode_config"
android:resource="@xml/game_mode_config" />
<activity
android:name=".MainActivity"
android:exported="true"
@@ -208,6 +208,8 @@ fun GamepadShell(
GamepadScreen.Library -> libraryHost?.let { host ->
LibraryScreen(
host = host,
settings = settings,
onLaunched = onConnected,
onBack = { screen = GamepadScreen.Home; libraryHost = null },
navActive = s == screen,
)
@@ -63,9 +63,6 @@ import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
/** Handshake budget for a normal connect (the prior hardcoded value, now passed explicitly). */
private const val CONNECT_TIMEOUT_MS = 10_000
/**
* Handshake budget for the no-PIN "request access" connect. Must exceed the host's approval-park
* window (~180 s) so a slow operator approval still lands on this same parked connection rather than
@@ -181,25 +178,10 @@ fun ConnectScreen(
// it survives a DHCP address change; else by address:port). Mirrors the Apple client.
val discoveredUnsaved = discovered.filter { dh -> savedHosts.none { it.matches(dh) } }
// The one place the full nativeConnect is issued (shared by the normal connect and the
// request-access path), including the HDR/gamepad derivation both need.
suspend fun connectNative(id: ClientIdentity, targetHost: String, targetPort: Int, pinHex: String, timeoutMs: Int): Long {
// Advertise HDR only when the user enabled it AND this device's display can present it
// (else the host sends a proper SDR stream rather than PQ the panel would mis-tone-map).
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID
// (Android exposes no controller-type enum) — parity with the Linux/Apple clients. An
// explicit choice is passed through unchanged.
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
return withContext(Dispatchers.IO) {
NativeBridge.nativeConnect(
targetHost, targetPort, w, h, hz,
id.certPem, id.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, settings.preferredCodec(), timeoutMs,
)
}
}
// Issue the native connect (shared by the normal connect and the request-access path). A plain
// desktop connect (no library launch) — the library launcher calls [connectToHost] with an id.
suspend fun connectNative(id: ClientIdentity, targetHost: String, targetPort: Int, pinHex: String, timeoutMs: Int): Long =
connectToHost(context, settings, id, targetHost, targetPort, pinHex, launch = null, timeoutMs = timeoutMs)
// The actual dial (identity already ready). On a TOFU connect (pinHex null), pin the fingerprint
// the host presented (as an unpaired known host) so the next connect goes straight through and it
@@ -230,11 +212,12 @@ fun ConnectScreen(
}
}
// Wake-aware connect. If the target is a saved host with a learned MAC that ISN'T currently
// advertising (asleep/off), wake it and WAIT for it to reappear on mDNS (WakeController shows the
// "Waking…" overlay) before dialing — discovery stays running meanwhile so we can see it come
// back. A fire-and-forget packet + the connect timeout wasn't enough for a cold boot. Otherwise
// dial straight through.
// Wake-aware connect. If auto-wake is on (Settings.autoWakeEnabled) and the target is a saved
// host with a learned MAC that ISN'T currently advertising (asleep/off, or just missing from
// mDNS), wake it and WAIT for it to reappear on mDNS (WakeController shows the "Waking…" overlay)
// before dialing — discovery stays running meanwhile so we can see it come back. A fire-and-forget
// packet + the connect timeout wasn't enough for a cold boot. Otherwise (auto-wake off, no MAC, or
// already seen live) dial straight through.
fun doConnect(targetHost: String, targetPort: Int, name: String, pinHex: String?) {
if (identity == null) {
status = "Identity not ready yet — try again in a moment"
@@ -248,7 +231,7 @@ fun ConnectScreen(
fun liveAdvert(): DiscoveredHost? =
if (kh != null) discovered.firstOrNull { kh.matches(it) }
else discovered.firstOrNull { it.host == targetHost && it.port == targetPort }
if (macs.isNotEmpty() && liveAdvert() == null) {
if (settings.autoWakeEnabled && macs.isNotEmpty() && liveAdvert() == null) {
waker.start(
hostName = name,
connectsAfter = true,
@@ -0,0 +1,48 @@
package io.unom.punktfunk
import android.content.Context
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.security.ClientIdentity
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
/** Handshake budget for a normal / library-launch connect (not the long request-access park). */
const val CONNECT_TIMEOUT_MS = 10_000
/**
* The one place [NativeBridge.nativeConnect] is assembled — shared by [ConnectScreen] and the library
* launcher ([LibraryScreen]). Derives the mode / HDR / gamepad settings the host needs from
* [settings]. [pinHex] is the pinned fingerprint (empty ⇒ TOFU). [launch] is a store-qualified library
* id (`steam:<appid>` / `custom:<id>`) to boot straight into a game, or `null` for the desktop.
* Returns the session handle, or `0` on failure. Call off the main thread.
*/
suspend fun connectToHost(
context: Context,
settings: Settings,
identity: ClientIdentity,
host: String,
port: Int,
pinHex: String,
launch: String?,
timeoutMs: Int = CONNECT_TIMEOUT_MS,
): Long {
// Advertise HDR only when the user enabled it AND this device's display can present it (else the
// host sends a proper SDR stream rather than PQ the panel would mis-tone-map).
val (w, h, hz) = settings.effectiveMode(context)
val hdrEnabled = settings.hdrEnabled && displaySupportsHdr(context)
// "Automatic" resolves to a concrete pad type from the connected controller's VID/PID.
val gamepadPref = Gamepad.resolvePref(settings.gamepad)
return withContext(Dispatchers.IO) {
// Transport-level half of "Low-latency mode (experimental)" (DSCP marking on the media
// sockets) — must be applied before connect, since sockets are tagged at creation.
NativeBridge.nativeSetLowLatencyMode(settings.lowLatencyMode)
NativeBridge.nativeConnect(
host, port, w, h, hz,
identity.certPem, identity.privateKeyPem, pinHex,
settings.bitrateKbps, settings.compositor, gamepadPref,
hdrEnabled, settings.audioChannels, settings.preferredCodec(), timeoutMs,
launch,
)
}
}
@@ -1,8 +1,10 @@
package io.unom.punktfunk
import android.widget.Toast
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.BoxWithConstraints
@@ -57,6 +59,7 @@ import io.unom.punktfunk.kit.library.GameEntry
import io.unom.punktfunk.kit.library.LibraryClient
import io.unom.punktfunk.kit.library.LibraryResult
import io.unom.punktfunk.kit.library.mtlsHttpClient
import io.unom.punktfunk.kit.security.ClientIdentity
import io.unom.punktfunk.kit.security.IdentityStore
import io.unom.punktfunk.kit.security.KnownHost
import io.unom.punktfunk.kit.security.obtainIdentity
@@ -73,17 +76,27 @@ import kotlinx.coroutines.withContext
private sealed class LibState {
object Loading : LibState()
data class Ready(val games: List<GameEntry>, val loader: ImageLoader) : LibState()
// Carries the client identity so a launch can dial the host over the same pinned mTLS trust.
data class Ready(val games: List<GameEntry>, val loader: ImageLoader, val identity: ClientIdentity) : LibState()
data class Message(val text: String) : LibState() // unauthorized / empty / error
}
@Composable
fun LibraryScreen(host: KnownHost, onBack: () -> Unit, navActive: Boolean = true) {
fun LibraryScreen(
host: KnownHost,
settings: Settings,
onLaunched: (Long) -> Unit,
onBack: () -> Unit,
navActive: Boolean = true,
) {
BackHandler(onBack = onBack)
val context = LocalContext.current
val scope = rememberCoroutineScope()
val hazeState = remember { HazeState() }
val landscape = LocalConfiguration.current.orientation == Configuration.ORIENTATION_LANDSCAPE
var state by remember { mutableStateOf<LibState>(LibState.Loading) }
// A launch (connect) in flight: shows an overlay + gates the pad so a second press can't dial twice.
var launching by remember { mutableStateOf(false) }
LaunchedEffect(host.address, host.port, host.fpHex) {
state = LibState.Loading
@@ -101,7 +114,7 @@ fun LibraryScreen(host: KnownHost, onBack: () -> Unit, navActive: Boolean = true
LibState.Message("No games found on this host.")
} else {
val client = mtlsHttpClient(id.certPem, id.privateKeyPem, host.address, host.fpHex)
LibState.Ready(res.games, ImageLoader.Builder(context).okHttpClient(client).build())
LibState.Ready(res.games, ImageLoader.Builder(context).okHttpClient(client).build(), id)
}
is LibraryResult.Unauthorized -> LibState.Message(res.message)
is LibraryResult.Error -> LibState.Message(res.message)
@@ -118,11 +131,45 @@ fun LibraryScreen(host: KnownHost, onBack: () -> Unit, navActive: Boolean = true
when (val s = state) {
is LibState.Loading -> LoadingState()
is LibState.Message -> MessageState(s.text)
is LibState.Ready -> Coverflow(s.games, s.loader, navActive)
is LibState.Ready -> Coverflow(s.games, s.loader, navActive && !launching) { game ->
if (!launching) {
launching = true
scope.launch {
// Dial the host over the same pinned mTLS trust, booting straight
// into this title (the host resolves `launch` = its library id).
val handle = connectToHost(
context, settings, s.identity,
host.address, host.port, host.fpHex, launch = game.id,
)
launching = false
if (handle != 0L) onLaunched(handle)
else Toast.makeText(
context,
"Launch failed — check the host and try again.",
Toast.LENGTH_LONG,
).show()
}
}
}
}
}
}
}
// Launching overlay — the connect + host-side game boot takes a moment; block the pad while it runs.
if (launching) {
Box(
Modifier.fillMaxSize().background(Color.Black.copy(alpha = 0.6f)),
contentAlignment = Alignment.Center,
) {
Column(
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(14.dp),
) {
CircularProgressIndicator(color = Color.White)
Text("Launching…", color = Color.White, style = MaterialTheme.typography.bodyLarge)
}
}
}
// Floating legend at the shared spot — same landscape-aware inset as every other console
// screen (ignore the safe area in landscape, where the bottom edge isn't a tap target).
Box(
@@ -130,7 +177,13 @@ fun LibraryScreen(host: KnownHost, onBack: () -> Unit, navActive: Boolean = true
.then(if (landscape) Modifier else Modifier.systemBarsPadding())
.padding(ConsoleLegendInset),
) {
GamepadHintBar(listOf(PadGlyph.hint('B', "Close", onClick = onBack)), hazeState = hazeState)
GamepadHintBar(
buildList {
if (state is LibState.Ready) add(PadGlyph.hint('A', "Launch"))
add(PadGlyph.hint('B', "Close", onClick = onBack))
},
hazeState = hazeState,
)
}
}
}
@@ -155,7 +208,12 @@ private fun MessageState(text: String) {
}
@Composable
private fun Coverflow(games: List<GameEntry>, loader: ImageLoader, navActive: Boolean) {
private fun Coverflow(
games: List<GameEntry>,
loader: ImageLoader,
navActive: Boolean,
onLaunch: (GameEntry) -> Unit,
) {
BoxWithConstraints(Modifier.fillMaxSize()) {
// Fit a 2:3 poster into the height the detail line leaves; clamp so it never dwarfs the screen.
val coverHeight = (maxHeight * 0.72f).coerceAtMost(360.dp)
@@ -167,16 +225,15 @@ private fun Coverflow(games: List<GameEntry>, loader: ImageLoader, navActive: Bo
LaunchedEffect(pagerState.settledPage) { navTarget = pagerState.settledPage }
val current = games.getOrNull(navTarget)
// Controller nav: the pad drives the coverflow (it wasn't captured before). Left/right steps a
// coalesced target the pager chases; A is reserved for launch (browse-only for now); B closes
// via the screen's BackHandler.
// Controller nav: the pad drives the coverflow. Left/right steps a coalesced target the pager
// chases; A launches the centred title; B closes via the screen's BackHandler.
GamepadNavEffect(
active = navActive && games.isNotEmpty(),
onMove = { dir ->
val t = (navTarget + dir).coerceIn(0, games.lastIndex)
if (t != navTarget) { navTarget = t; scope.launch { pagerState.animateScrollToPage(t) } }
},
onActivate = { /* launch a title — browse-only for now */ },
onActivate = { games.getOrNull(navTarget)?.let(onLaunch) },
)
Column(Modifier.fillMaxSize(), verticalArrangement = Arrangement.Center) {
@@ -198,6 +255,11 @@ private fun Coverflow(games: List<GameEntry>, loader: ImageLoader, navActive: Bo
.zIndex(-d) // centred cover on top, neighbours stacked behind
.width(coverWidth)
.height(coverHeight)
// Touch: tap the centred cover to launch it; tap a neighbour to bring it centre.
.clickable {
if (page == pagerState.currentPage) onLaunch(games[page])
else scope.launch { pagerState.animateScrollToPage(page) }
}
.graphicsLayer {
// Centre at full size; EVERY neighbour settles to one size, so an even pitch
// yields even VISUAL gaps. (A progressive shrink made the outer gaps grow —
@@ -51,8 +51,12 @@ class MainActivity : ComponentActivity() {
* Whether the last console input came from a real gamepad (face buttons / stick) vs. a TV D-pad
* remote (which has no A/B/X/Y). The console UI reads this to show glyphs the user recognises — pad
* face buttons, or a select glyph + arrows for a remote. Compose observes it (a snapshot state).
* Defaults to the remote glyphs on a TV (its D-pad remote is the typical first input, and often the
* only one) and to gamepad glyphs everywhere else (the console UI on a phone/tablet only activates
* via a real controller, so a TV-remote glyph would be a wrong first impression there) — set from
* [onCreate] once a [Context] is available, then kept live by real input.
*/
var lastPadIsGamepad by mutableStateOf(false)
var lastPadIsGamepad by mutableStateOf(true)
private set
/** The panel's highest-refresh display mode (0 = unknown/unsupported), resolved once at startup. */
@@ -60,6 +64,7 @@ class MainActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
lastPadIsGamepad = !isTvDevice(this)
resolveHighRefreshMode()
setConsoleHighRefreshRate(true) // the console UI wants max refresh; streaming manages its own
// Dark, transparent system bars regardless of the system theme — our UI is always dark, so
@@ -54,6 +54,24 @@ data class Settings(
* client's `libraryEnabled`.
*/
val libraryEnabled: Boolean = true,
/**
* "Low-latency mode (experimental)" — the master switch over the whole latency overhaul: decoder
* ranking + per-SoC vendor keys + the async decode loop (native), pipeline thread boosts + ADPF
* max-performance, game-tagged AAudio, DSCP marking on the media sockets, the Wi-Fi low-latency
* lock, HDMI ALLM, and the forced TV mode switch. Off (default): the original pre-overhaul
* pipeline, kept byte-for-byte as the known-good baseline — the overhaul regressed badly on some
* phones, so it's opt-in until it's proven per-device.
*/
val lowLatencyMode: Boolean = false,
/**
* Wake-on-LAN a saved host before connecting when it isn't currently seen on mDNS. On (default):
* a connect to a host with a learned MAC that isn't advertising sends a magic packet and waits
* for it to reappear (see [WakeController]) before dialing. Off: always dial straight through,
* skipping the mDNS-presence check entirely — for a host that's actually up but not visible on
* mDNS (a flaky discovery path, a VLAN/subnet that blocks multicast, etc.), where auto-wake would
* otherwise misfire and wait out its timeout despite the host already being reachable.
*/
val autoWakeEnabled: Boolean = true,
)
/** [Settings.touchMode] values; persisted by name. */
@@ -82,6 +100,8 @@ class SettingsStore(context: Context) {
?: if (prefs.getBoolean(K_TRACKPAD, true)) TouchMode.TRACKPAD else TouchMode.POINTER,
gamepadUiEnabled = prefs.getBoolean(K_GAMEPAD_UI, true),
libraryEnabled = prefs.getBoolean(K_LIBRARY, true),
lowLatencyMode = prefs.getBoolean(K_LOW_LATENCY, false),
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
)
fun save(s: Settings) {
@@ -100,6 +120,8 @@ class SettingsStore(context: Context) {
.putString(K_TOUCH_MODE, s.touchMode.name)
.putBoolean(K_GAMEPAD_UI, s.gamepadUiEnabled)
.putBoolean(K_LIBRARY, s.libraryEnabled)
.putBoolean(K_LOW_LATENCY, s.lowLatencyMode)
.putBoolean(K_AUTO_WAKE, s.autoWakeEnabled)
.apply()
}
@@ -119,6 +141,15 @@ class SettingsStore(context: Context) {
const val K_GAMEPAD_UI = "gamepad_ui_enabled"
const val K_LIBRARY = "library_enabled"
/**
* Deliberately NOT the original `"low_latency_mode"` key: that one shipped default-ON, so
* any install that ever saved settings persisted `true` — under the old key, flipping the
* default to off would leave exactly the regressed devices stuck on the overhaul. The fresh
* key restarts everyone at the safe default; the stale one is abandoned unread.
*/
const val K_LOW_LATENCY = "low_latency_mode_experimental"
const val K_AUTO_WAKE = "auto_wake_enabled"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */
const val K_TRACKPAD = "trackpad_mode"
}
@@ -215,6 +246,10 @@ val BITRATE_OPTIONS = listOf(
20_000 to "20 Mbps",
50_000 to "50 Mbps",
100_000 to "100 Mbps",
150_000 to "150 Mbps",
200_000 to "200 Mbps",
300_000 to "300 Mbps",
500_000 to "500 Mbps",
)
/** index = CompositorPref wire byte. */
@@ -324,6 +324,15 @@ private fun DisplaySettings(s: Settings, update: (Settings) -> Unit, context: an
options = COMPOSITOR_OPTIONS.mapIndexed { i, lbl -> i to lbl },
selected = s.compositor,
) { c -> update(s.copy(compositor = c)) }
ToggleRow(
title = "Low-latency mode (experimental)",
subtitle = "Aggressive decoder and system tuning (per-device decoder selection, async " +
"decode, Wi-Fi and HDMI hints). Can lower latency, but may stutter or glitch on " +
"some devices — turn off if the stream misbehaves.",
checked = s.lowLatencyMode,
onCheckedChange = { on -> update(s.copy(lowLatencyMode = on)) },
)
}
}
@@ -386,6 +395,14 @@ private fun InterfaceSettings(s: Settings, update: (Settings) -> Unit) {
checked = s.libraryEnabled,
onCheckedChange = { on -> update(s.copy(libraryEnabled = on)) },
)
ToggleRow(
title = "Auto-wake on connect",
subtitle = "Send Wake-on-LAN and wait for a saved host to reappear on mDNS before " +
"connecting. Turn off if a host that's already on isn't seen on mDNS, so connects " +
"go straight through instead of waiting out the wake timeout.",
checked = s.autoWakeEnabled,
onCheckedChange = { on -> update(s.copy(autoWakeEnabled = on)) },
)
ToggleRow(
title = "Stats overlay",
subtitle = "Show FPS, throughput and latency while streaming (3-finger tap toggles it live)",
@@ -27,7 +27,7 @@ import kotlin.math.roundToInt
* older layouts just omit those lines.
*/
@Composable
internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
internal fun StatsOverlay(s: DoubleArray, decoderLabel: String = "", modifier: Modifier = Modifier) {
if (s.size < 10) return
val w = s[6].toInt()
val h = s[7].toInt()
@@ -46,6 +46,14 @@ internal fun StatsOverlay(s: DoubleArray, modifier: Modifier = Modifier) {
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
if (decoderLabel.isNotEmpty()) {
Text(
decoderLabel,
color = Color(0xFFB0D0FF),
fontFamily = FontFamily.Monospace,
fontSize = 12.sp,
)
}
videoFeedLine(s)?.let { feed ->
Text(
feed,
@@ -1,11 +1,15 @@
package io.unom.punktfunk
import android.Manifest
import android.content.Context
import android.content.pm.ActivityInfo
import android.content.pm.PackageManager
import android.net.wifi.WifiManager
import android.os.Build
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.WindowManager
import android.widget.Toast
import androidx.activity.compose.BackHandler
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
@@ -30,6 +34,7 @@ import androidx.core.view.WindowInsetsControllerCompat
import io.unom.punktfunk.kit.Gamepad
import io.unom.punktfunk.kit.GamepadFeedback
import io.unom.punktfunk.kit.NativeBridge
import io.unom.punktfunk.kit.VideoDecoders
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.delay
@@ -55,29 +60,89 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// comes from Settings.
val initialSettings = remember { SettingsStore(context).load() }
var stats by remember { mutableStateOf<DoubleArray?>(null) }
var decoderLabel by remember { mutableStateOf("") }
var showStats by remember { mutableStateOf(initialSettings.statsHudEnabled) }
// Touch model is fixed per session (re-keys the gesture handler below if it ever changes).
val touchMode = initialSettings.touchMode
// "Low-latency mode (experimental)" master toggle, resolved once for the session. Off (the
// default) runs the original pre-overhaul pipeline; on enables the whole aggressive stack —
// decoder ranking + vendor keys + async loop (native side), the Wi-Fi low-latency lock and
// HDMI ALLM below, game-tagged audio, and DSCP marking (applied earlier, at connect).
val lowLatencyMode = initialSettings.lowLatencyMode
// TV form factor (leanback): the decoder actively switches the HDMI output mode to the stream
// refresh; a phone/tablet gets the softer seamless frame-rate hint instead.
val isTv = remember { context.packageManager.hasSystemFeature(PackageManager.FEATURE_LEANBACK) }
LaunchedEffect(handle, showStats) {
NativeBridge.nativeSetVideoStatsEnabled(handle, showStats)
if (showStats) {
while (true) {
delay(1000)
stats = NativeBridge.nativeVideoStats(handle)
// The decoder is fixed for the session; fetch its label once it's resolved.
if (decoderLabel.isEmpty()) decoderLabel = NativeBridge.nativeVideoDecoderLabel(handle)
}
} else {
stats = null // drop the last snapshot so a re-show never flashes stale numbers
}
}
// Host-gone watchdog. When the host suspends/sleeps (or crashes, or drops off the network) it
// stops answering the QUIC keep-alive and the connection idle-times out (~8 s) — no more frames
// arrive and the decoder would otherwise sit frozen on its last decoded frame until the user
// manually backed out. Poll the native session-liveness flag (one atomic load, independent of the
// stats HUD) and, the moment the session is dead, drop back to the menu so the user can
// Wake-on-LAN the host instead of being stranded on a frozen picture. Mirrors the Apple client's
// onSessionEnd → sessionEnded() → disconnect(). The 1 s cadence + the ~8 s idle timeout is a
// deliberately generous window: the keep-alive holds a merely-quiet connection (a static desktop)
// open, so this fires only on a genuinely dead peer, never a false positive. Keyed on `handle`, so
// it stops the moment we navigate away (the handle is only freed later, in onDispose).
LaunchedEffect(handle) {
while (true) {
delay(1000)
if (NativeBridge.nativeSessionEnded(handle)) {
Toast.makeText(
context,
"Connection lost — the host may be asleep. Wake it to reconnect.",
Toast.LENGTH_LONG,
).show()
onDisconnect()
return@LaunchedEffect
}
}
}
// One-shot teardown guard. Both the SurfaceView callback and DisposableEffect tear down on the
// way out, but `nativeClose` frees the handle — so once it's closed, NO path may touch the handle
// again (use-after-free → SIGSEGV: the consistent back-while-streaming crash). Both run on the
// main thread, so a plain flag is race-free; AtomicBoolean just makes the intent explicit.
val closed = remember { AtomicBoolean(false) }
// A Wi-Fi low-latency lock held for the stream's duration: asks the Wi-Fi firmware to drop its
// power-save polling (a common source of tens-of-ms jitter). WIFI_MODE_FULL_LOW_LATENCY (API
// 29+) is the strongest; older releases fall back to FULL_HIGH_PERF. Needs no extra permission
// beyond ACCESS_WIFI_STATE (already declared). Non-reference-counted: one explicit acquire/release.
// Part of the experimental low-latency stack — not created at all when the toggle is off.
val wifiLock = remember(handle) {
if (!lowLatencyMode) return@remember null
val wm = context.applicationContext.getSystemService(Context.WIFI_SERVICE) as? WifiManager
val mode = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
WifiManager.WIFI_MODE_FULL_LOW_LATENCY
} else {
@Suppress("DEPRECATION")
WifiManager.WIFI_MODE_FULL_HIGH_PERF
}
wm?.createWifiLock(mode, "punktfunk:stream")?.apply { setReferenceCounted(false) }
}
DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
runCatching { wifiLock?.acquire() }
// HDMI Auto Low-Latency Mode: ask the display to drop its post-processing (game mode) —
// the biggest panel-side latency win on the TV boxes. No-op where ALLM isn't supported. API
// 30+. Part of the experimental low-latency stack.
if (lowLatencyMode && Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
window?.setPreferMinimalPostProcessing(true)
}
controller?.let {
it.systemBarsBehavior = WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
it.hide(WindowInsetsCompat.Type.systemBars())
@@ -91,7 +156,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
activity?.requestedOrientation = ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE
activity?.streamHandle = handle // route hardware keys to this session
activity?.axisMapper = Gamepad.AxisMapper(handle) // route joystick axes
activity?.requestStreamExit = onDisconnect // Select+Start+L1+R1 chord leaves the stream
// Select+Start+L1+R1 chord leaves the stream — a deliberate quit (signal it so the host skips
// the keep-alive linger), unlike a host-ended / backgrounded drop.
activity?.requestStreamExit = { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate
// Host→client feedback (rumble + DualSense lightbar/LEDs); poll threads stopped before close.
val feedback = GamepadFeedback(handle).also { it.start() }
@@ -105,6 +172,10 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
activity?.setConsoleHighRefreshRate(true) // back to the console UI's max refresh
controller?.show(WindowInsetsCompat.Type.systemBars())
window?.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
if (lowLatencyMode && Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
window?.setPreferMinimalPostProcessing(false)
}
runCatching { if (wifiLock?.isHeld == true) wifiLock.release() }
// Release the landscape lock so the rest of the app follows the device/system again.
activity?.requestedOrientation =
priorOrientation ?: ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED
@@ -116,7 +187,8 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
}
}
BackHandler { onDisconnect() }
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
Box(modifier = Modifier.fillMaxSize()) {
AndroidView(
@@ -125,8 +197,22 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
SurfaceView(ctx).apply {
holder.addCallback(object : SurfaceHolder.Callback {
override fun surfaceCreated(holder: SurfaceHolder) {
NativeBridge.nativeStartVideo(handle, holder.surface)
NativeBridge.nativeStartAudio(handle)
// Low-latency mode: rank MediaCodecList decoders for the negotiated
// MIME (framework-only API) and hand the chosen one to Rust, which
// creates it by name and applies the per-SoC vendor low-latency keys.
// Off ⇒ no ranking: the platform resolves its default decoder for the
// MIME, exactly as before the overhaul.
val mime = NativeBridge.nativeVideoMime(handle)
val choice = if (lowLatencyMode) VideoDecoders.pickDecoder(mime) else null
NativeBridge.nativeStartVideo(
handle,
holder.surface,
choice?.name ?: "",
lowLatencyMode,
choice?.lowLatencyFeature ?: false,
isTv,
)
NativeBridge.nativeStartAudio(handle, lowLatencyMode)
if (micWanted) NativeBridge.nativeStartMic(handle)
}
@@ -150,7 +236,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Live stats HUD (FPS / throughput / capture→client latency), drawn over the video but
// BEFORE the transparent gesture layer below, so it shows through and never eats touches.
if (showStats) {
stats?.let { StatsOverlay(it, Modifier.align(Alignment.TopStart).padding(12.dp)) }
stats?.let { StatsOverlay(it, decoderLabel, Modifier.align(Alignment.TopStart).padding(12.dp)) }
}
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt.
@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Game Mode config (Android 13 / API 33+). We support the Performance game mode; and we opt OUT of
the two OEM interventions that would corrupt a game-streaming session:
- allowGameDownscaling=false: the client renders exactly the host's negotiated resolution; a
platform downscale would blur the stream.
- allowGameFpsOverride=false: the stream is paced to the host's frame rate; a platform FPS cap
would drop frames / add judder.
Ignored on releases below API 33.
-->
<game-mode-config xmlns:android="http://schemas.android.com/apk/res/android"
android:supportsPerformanceGameMode="true"
android:allowGameDownscaling="false"
android:allowGameFpsOverride="false" />
@@ -51,11 +51,23 @@ object NativeBridge {
/** Preferred video codec as a `quic::CODEC_*` bit (`0` = auto). Soft — the host falls back. */
preferredCodec: Int,
timeoutMs: Int,
/** Store-qualified library id (`steam:<appid>` / `custom:<id>`) to boot straight into a game,
* or `null`/empty for a plain desktop connect. Rides the Hello as `launch`. */
launch: String?,
): Long
/** 64-hex SHA-256 of the cert the host presented on [handle]; valid after a successful connect. */
external fun nativeHostFingerprint(handle: Long): String
/**
* Has the underlying QUIC session ended? `true` once the connection closed — a host suspend /
* crash / network drop idle-timed it out (~8 s), or the host closed it — from then on no frame
* ever arrives and the video sits frozen on its last one. The stream watchdog polls this (~1 Hz)
* to leave a dead stream and return to the menu, where the user can Wake-on-LAN the host, instead
* of stranding them on a frozen frame. `false` on a `0` handle. Cheap (one atomic load); UI-safe.
*/
external fun nativeSessionEnded(handle: Long): Boolean
/**
* Run the SPAKE2 PIN ceremony, presenting [certPem]/[keyPem]. Returns the host's verified
* fingerprint (64-hex) to persist + pin, or `""` on failure (wrong PIN / MITM / unreachable).
@@ -70,6 +82,14 @@ object NativeBridge {
name: String,
): String
/**
* Signal a **deliberate** user disconnect on [handle] before [nativeClose]: the session closes
* with `QUIT_CLOSE_CODE` so the host tears it down immediately instead of holding the keep-alive
* linger for a reconnect. Call from an explicit disconnect gesture only — NOT from a
* host-ended/network-drop end or an app-background (those keep the linger). No-op on `0`.
*/
external fun nativeDisconnectQuit(handle: Long)
/** Tear down a session handle returned by [nativeConnect]. No-op on `0`. */
external fun nativeClose(handle: Long)
@@ -104,14 +124,51 @@ object NativeBridge {
external fun nativeWakeOnLan(macsCsv: String, lastIp: String): Boolean
/**
* Start the HEVC decode thread rendering onto [surface] (a SurfaceView's surface). Decode runs
* entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI. No-op if already started.
* Apply the user's "Low-latency mode (experimental)" toggle to the process-wide transport
* defaults — today just DSCP/QoS marking on the media sockets. Must be called BEFORE
* [nativeConnect] (the tag is applied at socket creation); `HostConnect.connectToHost` does.
* The rest of the toggle rides explicit per-session parameters ([nativeStartVideo] /
* [nativeStartAudio]). Cheap (one atomic store); UI-safe.
*/
external fun nativeStartVideo(handle: Long, surface: android.view.Surface)
external fun nativeSetLowLatencyMode(enabled: Boolean)
/**
* The MediaCodec MIME the host resolved for this session (`"video/hevc"` / `"video/avc"` /
* `"video/av01"`), or `""` on a `0` handle. Kotlin ranks `MediaCodecList` decoders for this
* MIME (see [io.unom.punktfunk.kit.VideoDecoders]) before [nativeStartVideo]. Cheap; UI-safe.
*/
external fun nativeVideoMime(handle: Long): String
/**
* Start the decode thread rendering onto [surface] (a SurfaceView's surface). Decode runs
* entirely in Rust (NDK AMediaCodec → ANativeWindow) — no per-frame JNI. [decoderName] is the
* decoder Kotlin ranked from `MediaCodecList` (`""` = let the platform resolve the default for
* the MIME — what the pre-overhaul client always did); [lowLatencyMode] is the user's
* "Low-latency mode (experimental)" toggle (off, the default, runs the original decode
* pipeline; on, the aggressive per-SoC tuning + async loop); [lowLatencyFeature] is whether
* [decoderName] advertised `FEATURE_LowLatency` (HUD label only). [isTv] drives an active HDMI
* mode switch to the stream refresh on TV boxes when the toggle is on (vs. the softer seamless
* hint otherwise). No-op if already started.
*/
external fun nativeStartVideo(
handle: Long,
surface: android.view.Surface,
decoderName: String,
lowLatencyMode: Boolean,
lowLatencyFeature: Boolean,
isTv: Boolean,
)
/** Stop + join the decode thread without closing the session. No-op on `0`. */
external fun nativeStopVideo(handle: Long)
/**
* The resolved decoder identity for the HUD, e.g. `c2.qti.avc.decoder · low-latency`, or `""`
* before the decode thread has resolved one. One-shot (fixed for the session); poll once after
* the HUD appears.
*/
external fun nativeVideoDecoderLabel(handle: Long): String
/**
* Drain ~1 s of live decode stats for the on-stream HUD, or `null` when no decode thread runs.
* Returns 18 doubles (unified stats spec, `design/stats-unification.md`):
@@ -137,10 +194,12 @@ object NativeBridge {
external fun nativeSetVideoStatsEnabled(handle: Long, enabled: Boolean)
/**
* Start host→client audio: Opus decode → jitter ring → AAudio (LowLatency), all in Rust. No-op
* if already started. Best-effort — a failure leaves video streaming.
* Start host→client audio: Opus decode → jitter ring → AAudio (LowLatency), all in Rust.
* [lowLatencyMode] (the experimental toggle) additionally tags the stream usage=Game for the
* HAL's game-audio routing. No-op if already started. Best-effort — a failure leaves video
* streaming.
*/
external fun nativeStartAudio(handle: Long)
external fun nativeStartAudio(handle: Long, lowLatencyMode: Boolean)
/** Stop + join the audio thread and close AAudio, without closing the session. No-op on `0`. */
external fun nativeStopAudio(handle: Long)
@@ -0,0 +1,95 @@
package io.unom.punktfunk.kit
import android.media.MediaCodecInfo.CodecCapabilities
import android.media.MediaCodecList
import android.os.Build
/** The decoder Kotlin ranked for a MIME, handed to [NativeBridge.nativeStartVideo]. */
data class DecoderChoice(val name: String, val lowLatencyFeature: Boolean)
/**
* Rank the platform's `MediaCodecList` decoders for a video MIME and pick the best one for
* low-latency streaming, the way Moonlight-Android does. There is no NDK `MediaCodecList`, so this
* enumeration must live on the Kotlin (framework) side; Rust then creates the chosen decoder by
* name (`AMediaCodec_createCodecByName`) and derives the per-SoC vendor low-latency keys from it.
*
* Ranking (best first): hardware over software; a real SoC-vendor decoder (Qualcomm/Amlogic/…) over
* the generic AOSP software fallback; a decoder advertising `FEATURE_LowLatency` over one that
* doesn't. Known-bad software decoders (`omx.google.*`, `c2.android.*`, Qualcomm/Samsung SW HEVC)
* are dropped outright — matching Moonlight's blacklist.
*/
object VideoDecoders {
/** Decoder-name prefixes/names we never want, mirroring Moonlight's blacklist. */
private val BLOCKED_PREFIXES = listOf("omx.google.", "c2.android.", "avcdecoder", "omx.ffmpeg.")
private val BLOCKED_EXACT = listOf("omx.qcom.video.decoder.hevcswvdec", "omx.sec.hevc.sw.dec")
/**
* Real SoC-vendor decoder prefixes we prefer over the generic AOSP fallback, covering the common
* targets: Qualcomm Snapdragon and MediaTek (most phones + many TV boxes), Samsung Exynos (+
* Google Tensor, whose decoder is `c2.exynos.*`), NVIDIA Tegra (Shield TV), Amlogic / Rockchip /
* Realtek (TV boxes & smart TVs), and HiSilicon Kirin (older Huawei).
*/
private val VENDOR_PREFIXES = listOf(
"omx.qcom", "c2.qti",
"omx.mtk", "c2.mtk",
"omx.exynos", "c2.exynos",
"omx.nvidia", "c2.nvidia",
"omx.amlogic", "c2.amlogic",
"omx.rk", "c2.rk",
"omx.realtek", "c2.realtek",
"omx.hisi", "c2.hisi",
)
/**
* Pick the best decoder for [mime] (`"video/hevc"` / `"video/avc"` / `"video/av01"`), or `null`
* to let the platform resolve its default. Enumerates once — call at stream start.
*/
fun pickDecoder(mime: String): DecoderChoice? {
if (mime.isEmpty()) return null
val infos = runCatching { MediaCodecList(MediaCodecList.REGULAR_CODECS).codecInfos }
.getOrNull() ?: return null
var bestName: String? = null
var bestLowLatency = false
var bestScore = Int.MIN_VALUE
for (info in infos) {
if (info.isEncoder) continue
val name = info.name
val lower = name.lowercase()
if (BLOCKED_PREFIXES.any { lower.startsWith(it) } || lower in BLOCKED_EXACT) continue
// Never a secure decoder: `.secure` names are the DRM-pipeline twins of the real
// decoder and require a secure surface — configuring one for a clear stream fails (or
// renders black). The plain twin is also in the list, so drop rather than rank
// (a `.secure` twin can otherwise OUT-score its plain sibling when only it advertises
// FEATURE_LowLatency). Moonlight filters the same way.
if (lower.endsWith(".secure")) continue
val caps = runCatching { info.getCapabilitiesForType(mime) }.getOrNull() ?: continue
val secureRequired = runCatching {
caps.isFeatureRequired(CodecCapabilities.FEATURE_SecurePlayback)
}.getOrDefault(false)
if (secureRequired) continue
val hardware = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
info.isHardwareAccelerated
} else {
// Pre-Q heuristic: the software decoders are the ones we can name (already blocked
// above), so anything surviving the blacklist is treated as hardware.
true
}
val lowLatency = Build.VERSION.SDK_INT >= Build.VERSION_CODES.R &&
runCatching { caps.isFeatureSupported(CodecCapabilities.FEATURE_LowLatency) }
.getOrDefault(false)
val vendor = VENDOR_PREFIXES.any { lower.startsWith(it) }
val score = (if (hardware) 100 else 0) +
(if (vendor) 40 else 0) +
(if (lowLatency) 20 else 0)
if (score > bestScore) {
bestScore = score
bestName = name
bestLowLatency = lowLatency
}
}
return bestName?.let { DecoderChoice(it, bestLowLatency) }
}
}
+26 -2
View File
@@ -28,6 +28,7 @@ type CreateSessionFn = unsafe extern "C" fn(*mut c_void, *const i32, usize, i64)
type ReportFn = unsafe extern "C" fn(*mut c_void, i64) -> c_int;
type UpdateTargetFn = unsafe extern "C" fn(*mut c_void, i64) -> c_int;
type CloseFn = unsafe extern "C" fn(*mut c_void);
type SetPreferPowerEfficiencyFn = unsafe extern "C" fn(*mut c_void, bool) -> c_int;
/// The entry points we use, resolved once from `libandroid.so`, plus the process-wide manager.
struct Api {
@@ -35,6 +36,9 @@ struct Api {
report: ReportFn,
update_target: UpdateTargetFn,
close: CloseFn,
/// `APerformanceHint_setPreferPowerEfficiency` — NDK **API 35**, so `Option`al even when the
/// rest of ADPF resolved (a 33/34 device has the session API but not this one).
set_prefer_power_efficiency: Option<SetPreferPowerEfficiencyFn>,
manager: *mut c_void,
}
@@ -70,11 +74,20 @@ fn resolve_api() -> Option<Api> {
if manager.is_null() {
return None;
}
// Optional (API 35): resolve if present, else `None` — the session still works without it.
let set_prefer_power_efficiency =
libc::dlsym(lib, c"APerformanceHint_setPreferPowerEfficiency".as_ptr());
let set_prefer_power_efficiency = (!set_prefer_power_efficiency.is_null()).then(|| {
std::mem::transmute::<*mut c_void, SetPreferPowerEfficiencyFn>(
set_prefer_power_efficiency,
)
});
Some(Api {
create_session: std::mem::transmute::<*mut c_void, CreateSessionFn>(create_session),
report: std::mem::transmute::<*mut c_void, ReportFn>(report),
update_target: std::mem::transmute::<*mut c_void, UpdateTargetFn>(update_target),
close: std::mem::transmute::<*mut c_void, CloseFn>(close),
set_prefer_power_efficiency,
manager,
})
}
@@ -90,8 +103,10 @@ pub struct HintSession {
impl HintSession {
/// Open a session hinting `tids` with an initial per-frame target of `target_ns` nanoseconds.
/// `None` when ADPF is unavailable (device API < 33) or the platform declines — the caller then
/// runs unhinted (a no-op, not an error).
pub fn create(target_ns: i64, tids: &[i32]) -> Option<Self> {
/// runs unhinted (a no-op, not an error). `prefer_performance` (the experimental low-latency
/// mode) additionally biases the governor away from power efficiency (API 35+); off, the
/// session runs with the platform default, as it did before the overhaul.
pub fn create(target_ns: i64, tids: &[i32], prefer_performance: bool) -> Option<Self> {
if target_ns <= 0 || tids.is_empty() {
return None;
}
@@ -103,6 +118,15 @@ impl HintSession {
if session.is_null() {
return None;
}
// Tell the governor NOT to bias this session toward power efficiency (API 35+): our loop is
// latency-critical, so we want it kept on fast cores at high clocks over battery savings.
// Best-effort; absent below API 35.
if prefer_performance {
if let Some(f) = api.set_prefer_power_efficiency {
// SAFETY: `session` is the live session just created; the fn takes it + a bool.
unsafe { f(session, false) };
}
}
Some(Self { api, session })
}
+20 -6
View File
@@ -18,8 +18,8 @@
//! grown on XRuns (Google's anti-glitch technique).
use ndk::audio::{
AudioCallbackResult, AudioDirection, AudioFormat, AudioPerformanceMode, AudioSharingMode,
AudioStream, AudioStreamBuilder,
AudioCallbackResult, AudioContentType, AudioDirection, AudioFormat, AudioPerformanceMode,
AudioSharingMode, AudioStream, AudioStreamBuilder, AudioUsage,
};
use punktfunk_core::client::NativeClient;
use punktfunk_core::error::PunktfunkError;
@@ -116,8 +116,10 @@ pub struct AudioPlayback {
impl AudioPlayback {
/// Open AAudio (LowLatency, 48 kHz/f32, the host-resolved channel layout) with a realtime
/// callback draining a jitter ring, then spawn the Opus decode thread. `None` on failure (the
/// caller leaves video streaming).
pub fn start(client: Arc<NativeClient>) -> Option<AudioPlayback> {
/// caller leaves video streaming). `game_audio` (the experimental low-latency mode) tags the
/// stream usage=Game for the HAL's game-audio routing; off, the stream is untagged as it was
/// before the overhaul.
pub fn start(client: Arc<NativeClient>, game_audio: bool) -> Option<AudioPlayback> {
// Build playback from the host-RESOLVED channel count (never the request): 2 = stereo /
// 6 = 5.1 / 8 = 7.1, canonical wire order FL FR FC LFE RL RR SL SR.
let channels = punktfunk_core::audio::normalize_channels(client.audio_channels) as usize;
@@ -226,7 +228,7 @@ impl AudioPlayback {
AudioCallbackResult::Continue
};
let stream = AudioStreamBuilder::new()?
let builder = AudioStreamBuilder::new()?
.direction(AudioDirection::Output)
.sample_rate(SAMPLE_RATE)
// The wire order (FL FR FC LFE RL RR SL SR) is the standard AAudio/Android channel
@@ -234,7 +236,19 @@ impl AudioPlayback {
// from `channel_count` (the ndk crate's builder exposes no setChannelMask); the host
// captures + Opus-encodes in exactly this order.
.channel_count(channels as i32)
.format(AudioFormat::PCM_Float)
.format(AudioFormat::PCM_Float);
// Tag the stream as game audio (usage=Game / content=Movie): the audio HAL applies
// its low-latency game-audio routing/policy and it's grouped correctly with the
// game-mode profile. Advisory — ignored where the device has no such policy. Part of
// the experimental low-latency stack; off, the stream stays untagged.
let builder = if game_audio {
builder
.usage(AudioUsage::Game)
.content_type(AudioContentType::Movie)
} else {
builder
};
let stream = builder
.performance_mode(AudioPerformanceMode::LowLatency)
.sharing_mode(sharing)
.data_callback(Box::new(callback))
+755 -41
View File
@@ -8,8 +8,8 @@
use ndk::data_space::DataSpace;
use ndk::media::media_codec::{
DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec, MediaCodecDirection,
OutputBuffer,
AsyncNotifyCallback, DequeuedInputBufferResult, DequeuedOutputBufferInfoResult, MediaCodec,
MediaCodecDirection, OutputBuffer,
};
use ndk::media::media_format::MediaFormat;
use ndk::native_window::NativeWindow;
@@ -19,9 +19,14 @@ use punktfunk_core::session::Frame;
use std::collections::VecDeque;
use std::ffi::c_void;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::sync::{mpsc, Arc, Mutex};
use std::time::{Duration, Instant};
/// Cap on AUs parked in the async loop awaiting a free codec input slot. Matches the connector's
/// own frame-channel depth; on sustained overflow the oldest is dropped and a keyframe requested
/// (same recovery as a reassembler drop). In steady state this stays near-empty.
const FRAME_PARK_CAP: usize = 16;
/// Cap on the pts→received-timestamp map below: MediaCodec holds only a handful of frames in
/// flight, so anything beyond this is stale (codec flushed / HUD toggled) and gets evicted.
const IN_FLIGHT_CAP: usize = 64;
@@ -31,29 +36,83 @@ const IN_FLIGHT_CAP: usize = 64;
/// this deep is a lost datagram (or an old host that never sends any) and gets evicted.
const PENDING_SPLIT_CAP: usize = 256;
/// The decode loop. Runs on the `pf-decode` thread until `shutdown` is set or the session closes.
/// Whether low-latency mode uses the event-driven async decode loop (default) or the synchronous
/// poll loop. Flip to `false` to A/B the two on the HUD (`design/…`); the async loop presents a
/// decoded frame the instant it's ready instead of waiting out a poll interval. Only consulted when
/// the user's "Low-latency mode (experimental)" toggle is ON — off, the sync loop always runs (the
/// original pipeline).
const USE_ASYNC_DECODE: bool = true;
/// Per-session decode configuration, resolved by the JNI layer (`nativeStartVideo`) and passed to
/// the decode loop. Bundled so the loop entry points don't sprout a wide argument list.
pub(crate) struct DecodeOptions {
/// The decoder Kotlin ranked from `MediaCodecList` (`VideoDecoders.pickDecoder`). `None`/empty ⇒
/// let the platform resolve the default decoder for the MIME.
pub decoder_name: Option<String>,
/// Whether Kotlin found the chosen decoder advertises `FEATURE_LowLatency` (queryable only via
/// the Java `CodecCapabilities` API) — surfaced on the HUD next to the decoder name.
pub ll_feature: bool,
/// The user's "Low-latency mode (experimental)" master toggle. On ⇒ the full overhaul: async
/// decode loop, per-SoC vendor keys, pipeline thread boosts, ADPF max-performance, forced TV
/// mode switch. Off (default) ⇒ the original pre-overhaul pipeline, kept as the known-good
/// baseline while the overhaul is experimental.
pub low_latency_mode: bool,
/// TV form factor (Kotlin's `UiModeManager`): actively drive the HDMI output into the stream's
/// refresh mode, vs. the softer seamless hint on a phone/tablet.
pub is_tv: bool,
}
/// The decode entry point on the `pf-decode` thread: dispatches to the async or synchronous loop.
/// Both run until `shutdown` is set or the session closes.
pub fn run(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
if opts.low_latency_mode && USE_ASYNC_DECODE {
run_async(client, window, shutdown, stats, opts);
} else {
run_sync(client, window, shutdown, stats, opts);
}
}
/// The synchronous poll loop — the original decode path: the only one when low-latency mode is off,
/// and the [`USE_ASYNC_DECODE`] A/B fallback when it's on. Feeds and drains on this one thread; the
/// only blocking wait is a short output dequeue while input is backed up.
fn run_sync(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
let DecodeOptions {
decoder_name,
ll_feature,
low_latency_mode,
is_tv,
} = opts;
boost_thread_priority();
let mode = client.mode();
// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`): HEVC or H.264. AMediaCodec
// needs no out-of-band extradata — the in-band VPS/SPS/PPS on every IDR configure it either way.
let mime = match client.codec {
punktfunk_core::quic::CODEC_H264 => "video/avc",
_ => "video/hevc",
};
let codec = match MediaCodec::from_decoder_type(mime) {
// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`). AMediaCodec needs no
// out-of-band extradata — the in-band VPS/SPS/PPS on every IDR configure it either way.
let mime = codec_mime(client.codec);
let codec = match create_codec(mime, decoder_name.as_deref()) {
Some(c) => c,
None => {
log::error!("decode: no {mime} decoder on this device");
return;
}
};
log::info!("decode: codec mime = {mime}");
// The decoder's *actual* resolved name (Kotlin's pick, or the platform default when it fell
// back) drives both the HUD label and which vendor low-latency keys apply below.
let codec_name = codec.name().unwrap_or_default();
stats.set_decoder(&codec_name, ll_feature);
log::info!(
"decode: codec mime = {mime}, decoder = {codec_name} (low-latency feature: {ll_feature})"
);
let mut format = MediaFormat::new();
format.set_str("mime", mime);
@@ -64,23 +123,9 @@ pub fn run(
"max-input-size",
(mode.width * mode.height).max(2_000_000) as i32,
);
// Ask for the low-latency decode path where the decoder supports it (no reordering buffer).
format.set_i32("low-latency", 1);
// Best-effort vendor twin of the standard key: older Qualcomm decoders only honor their own
// extension. Unknown keys are ignored by other vendors' codecs, so this is safe to set blind.
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
// Advisory low-latency hints (KEY_PRIORITY / KEY_OPERATING_RATE), ignored where unsupported:
// realtime priority + the target frame rate, so vendor decoders (e.g. Qualcomm) run at full
// clocks instead of a power-saving cadence that adds dequeue latency.
format.set_i32("priority", 0); // 0 = realtime
// Operating rate = the codec's clock hint. Setting it to the display rate merely asks the
// decoder to *sustain* that cadence — a Qualcomm decoder can meet 60/120 fps at a power-saving
// clock that adds a millisecond-plus of decode latency per frame. Setting it to the AOSP
// "unbounded" sentinel (Short.MAX) instead asks the decoder to run each frame at max clocks and
// finish ASAP, minimising per-frame decode latency — the right trade for a real-time stream
// (costs power/heat; the dial to lower if a device thermally throttles over a long session).
// Ignored where unsupported.
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
// Standard + per-SoC vendor low-latency keys and the clock hints, gated on the resolved decoder
// name and the master toggle (see `configure_low_latency`).
configure_low_latency(&mut format, &codec_name, low_latency_mode);
// HDR static metadata (ST.2086 mastering + content light level): when an HDR session was
// negotiated, set KEY_HDR_STATIC_INFO so the display tone-maps from the source's real grade.
@@ -118,7 +163,11 @@ pub fn run(
// above our API-28 floor, so we resolve it at runtime (see `try_set_frame_rate`) rather than link
// it — a hard import would stop `libpunktfunk_android.so` loading at all on API 28/29. Absent
// there ⇒ we simply skip the hint (non-fatal; the stream renders fine without it).
if mode.refresh_hz > 0 && !try_set_frame_rate(&window, mode.refresh_hz as f32) {
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
"decode: set_frame_rate({} Hz) unavailable/declined (non-fatal)",
mode.refresh_hz
@@ -277,7 +326,12 @@ pub fn run(
// or where the platform declines → `None`, and the loop runs unhinted).
hint_tried = true;
let tids = client.hot_thread_ids();
hint = crate::adpf::HintSession::create(frame_period_ns, &tids);
// The pump/audio priority boost is part of the experimental low-latency stack; the
// ADPF session itself predates it and always runs (max-performance bias gated inside).
if low_latency_mode {
boost_hot_threads(&tids);
}
hint = crate::adpf::HintSession::create(frame_period_ns, &tids, low_latency_mode);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
@@ -326,6 +380,626 @@ fn now_realtime_ns() -> i128 {
.unwrap_or(0)
}
/// The MediaCodec MIME for the codec the host resolved (`Welcome.codec`). Shared by the decode
/// thread and `nativeVideoMime` (which tells Kotlin what to rank decoders for). AV1 uses the
/// AOSP `video/av01` type; anything not H.264/AV1 is treated as HEVC (every pre-negotiation host
/// emitted HEVC).
pub(crate) fn codec_mime(codec: u8) -> &'static str {
match codec {
punktfunk_core::quic::CODEC_H264 => "video/avc",
punktfunk_core::quic::CODEC_AV1 => "video/av01",
_ => "video/hevc",
}
}
/// Create the decoder: prefer the specific codec Kotlin ranked from `MediaCodecList`
/// (`from_codec_name`), falling back to the platform's default decoder for the MIME
/// (`from_decoder_type`) if that name can't be created (codec busy / renamed across an OS update).
fn create_codec(mime: &str, preferred: Option<&str>) -> Option<MediaCodec> {
if let Some(name) = preferred.filter(|n| !n.is_empty()) {
if let Some(c) = MediaCodec::from_codec_name(name) {
return Some(c);
}
log::warn!(
"decode: from_codec_name({name}) failed — falling back to default {mime} decoder"
);
}
MediaCodec::from_decoder_type(mime)
}
/// Apply the low-latency MediaFormat keys for `codec_name`.
///
/// `aggressive` = the "Low-latency mode (experimental)" master toggle. **Off** (default) ⇒ the
/// pre-overhaul key set, byte-for-byte — the standard `low-latency` key, the blind Qualcomm vendor
/// twin, `priority = 0` AND `operating-rate = MAX` set together — kept as the known-good baseline
/// (the profile every device streamed with before the overhaul). **On** ⇒ the Moonlight-parity
/// profile: MediaTek's `vdec-lowlatency` (unconditionally — ignored off MediaTek), the per-SoC
/// vendor extension keys (gated on the decoder-name prefix the way Moonlight-Android does, since a
/// key one vendor honours is meaningless on another), and one *mutually exclusive* clock hint.
///
/// Vendor keys mirror Moonlight's `MediaCodecHelper` (verified against current source): Qualcomm
/// picture-order + low-latency, Exynos (also Google Tensor), Amlogic, HiSilicon, MediaTek. NVIDIA
/// Tegra / Rockchip / Realtek expose no such key (nor does Moonlight) — they're covered by the
/// standard key + clock hint + being ranked first in `VideoDecoders`.
fn configure_low_latency(format: &mut MediaFormat, codec_name: &str, aggressive: bool) {
// Standard key: request the no-reorder low-latency path where the platform decoder supports it.
format.set_i32("low-latency", 1);
if !aggressive {
// The original profile: the Qualcomm vendor twin set blind (unknown keys are ignored by
// other vendors' codecs), realtime priority, and the AOSP "unbounded" operating-rate
// sentinel — decode each frame at max clocks rather than pacing to the frame rate.
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
format.set_i32("priority", 0); // 0 = realtime
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
return;
}
// MediaTek's low-latency key — very common (mid/budget phones + many Google TV / Fire TV boxes).
// Set unconditionally like the standard key: MediaTek decoders honour it, others ignore it, so it
// covers MediaTek whatever the exact decoder name (omx.mtk / c2.mtk / an OEM rename). Moonlight
// does the same, and also relies on it for Amazon's Amlogic fork.
format.set_i32("vdec-lowlatency", 1);
let name = codec_name.to_ascii_lowercase();
let is = |prefix: &str| name.starts_with(prefix);
// Qualcomm Snapdragon (the most common phone SoC): picture-order forces decode-order output
// (kills the reorder buffer on decoders that predate the standard key); low-latency is the older
// vendor twin.
if is("omx.qcom") || is("c2.qti") {
format.set_i32("vendor.qti-ext-dec-picture-order.enable", 1);
format.set_i32("vendor.qti-ext-dec-low-latency.enable", 1);
}
// Samsung Exynos — also covers Google Tensor (Pixel 6+), whose hardware decoder is `c2.exynos.*`.
if is("omx.exynos") || is("c2.exynos") {
format.set_i32("vendor.rtc-ext-dec-low-latency.enable", 1);
}
// Amlogic — the Android TV boxes (onn 4K, Chromecast w/ Google TV, Homatics).
if is("omx.amlogic") || is("c2.amlogic") {
format.set_i32("vendor.low-latency.enable", 1);
}
// HiSilicon / Kirin (older Huawei; paired req/rdy keys).
if is("omx.hisi") || is("c2.hisi") {
format.set_i32(
"vendor.hisi-ext-low-latency-video-dec.video-scene-for-low-latency-req",
1,
);
format.set_i32(
"vendor.hisi-ext-low-latency-video-dec.video-scene-for-low-latency-rdy",
-1,
);
}
// NVIDIA Tegra (Shield TV) and Rockchip/Realtek (budget TV boxes / smart TVs) expose no
// low-latency vendor key (Moonlight has none either) — their decoders are already low-latency
// oriented, so the standard `low-latency` key + the clock hint below + being ranked first
// (see `VideoDecoders`) is their treatment.
//
// Clock hint, mutually exclusive (matching Moonlight): the AOSP "unbounded" operating-rate
// sentinel (Short.MAX) tells the decoder to run each frame at max clocks and finish ASAP rather
// than pace to the frame rate — shaving per-frame decode latency at a power/heat cost. Only
// Qualcomm is known to handle the sentinel; every other vendor mis-paces on it, so they get the
// plain realtime `priority` hint instead.
if decoder_supports_max_operating_rate(&name) {
format.set_i32("operating-rate", i16::MAX as i32); // 32767 = "as fast as possible"
} else {
format.set_i32("priority", 0); // 0 = realtime
}
}
/// Whether a decoder tolerates `operating-rate = Short.MAX` rather than regressing on it. Follows
/// Moonlight's allowlist: Qualcomm decoders honour the sentinel (the Adreno 620 generation is the
/// known exception Moonlight excludes by GPU model — undetectable from native code here, so it
/// rides the master toggle as its escape hatch). Other vendors fall back to the plain `priority`
/// hint above.
fn decoder_supports_max_operating_rate(name_lower: &str) -> bool {
name_lower.starts_with("omx.qcom") || name_lower.starts_with("c2.qti")
}
/// One decoded output buffer ready to release: its codec buffer index + the pts the codec echoed
/// (from the output callback's `BufferInfo`), used to pair the `decode` HUD stat.
struct OutputReady {
index: usize,
pts_us: u64,
}
/// Events the async decode loop reacts to. The codec's async-notify callbacks (which run on its
/// internal looper thread) push the codec ones; the feeder thread pushes `Au`. Each carries only
/// owned/`Copy` data so the callback closures satisfy the `Send` bound and never touch the codec.
enum DecodeEvent {
/// A received access unit from the feeder, ready to queue into the decoder.
Au(Frame),
/// An input buffer slot freed (index) — we can queue an AU into it.
InputAvailable(usize),
/// A decoded frame is ready (buffer index + echoed pts).
OutputAvailable { index: usize, pts_us: u64 },
/// The output format changed — re-check the stream's colour signalling (HDR DataSpace).
FormatChanged,
/// The codec reported an error; `fatal` when neither recoverable nor transient.
Error { fatal: bool },
}
/// The event-driven async decode loop (default; see [`run`]/[`USE_ASYNC_DECODE`]). The codec drives
/// us: an async-notify callback fires the instant an input buffer frees or a frame finishes
/// decoding, so a decoded frame is presented immediately instead of waiting out a poll interval (the
/// latency the sync loop left on the table). The callbacks run on the codec's internal looper thread
/// and only *push events* — every `AMediaCodec` buffer op stays on this thread, which owns the codec,
/// sidestepping the self-reference that would arise from a callback calling back into the codec it's
/// stored in. A small `pf-decode-feed` thread blocks on the network so this loop never does.
fn run_async(
client: Arc<NativeClient>,
window: NativeWindow,
shutdown: Arc<AtomicBool>,
stats: Arc<crate::stats::VideoStats>,
opts: DecodeOptions,
) {
let DecodeOptions {
decoder_name,
ll_feature,
low_latency_mode,
is_tv,
} = opts;
boost_thread_priority();
let mode = client.mode();
let mime = codec_mime(client.codec);
let mut codec = match create_codec(mime, decoder_name.as_deref()) {
Some(c) => c,
None => {
log::error!("decode: no {mime} decoder on this device");
return;
}
};
let codec_name = codec.name().unwrap_or_default();
stats.set_decoder(&codec_name, ll_feature);
log::info!(
"decode: codec mime = {mime}, decoder = {codec_name} (async, low-latency feature: {ll_feature})"
);
// The event channel: the callbacks + feeder push, this loop pulls. `Sender` is `Send`, so the
// callback closures (each capturing a clone) satisfy the async-notify `Send` bound.
let (ev_tx, ev_rx) = mpsc::channel::<DecodeEvent>();
// Install the callbacks BEFORE configure()/start() so we're in async mode from the first buffer.
// Each just forwards an index/flag — no codec access here (the codec owns these closures).
{
let out_tx = ev_tx.clone();
let in_tx = ev_tx.clone();
let fmt_tx = ev_tx.clone();
let err_tx = ev_tx.clone();
let cb = AsyncNotifyCallback {
on_input_available: Some(Box::new(move |idx| {
let _ = in_tx.send(DecodeEvent::InputAvailable(idx));
})),
on_output_available: Some(Box::new(move |idx, info| {
let _ = out_tx.send(DecodeEvent::OutputAvailable {
index: idx,
pts_us: info.presentation_time_us().max(0) as u64,
});
})),
on_format_changed: Some(Box::new(move |_fmt| {
let _ = fmt_tx.send(DecodeEvent::FormatChanged);
})),
on_error: Some(Box::new(move |e, code, _detail| {
let fatal = !code.is_recoverable() && !code.is_transient();
log::warn!("decode: codec error {e:?} (fatal={fatal})");
let _ = err_tx.send(DecodeEvent::Error { fatal });
})),
};
if let Err(e) = codec.set_async_notify_callback(Some(cb)) {
log::error!("decode: set_async_notify_callback failed: {e}");
return;
}
}
// Build the low-latency format (identical keys to the sync path).
let mut format = MediaFormat::new();
format.set_str("mime", mime);
format.set_i32("width", mode.width as i32);
format.set_i32("height", mode.height as i32);
format.set_i32(
"max-input-size",
(mode.width * mode.height).max(2_000_000) as i32,
);
configure_low_latency(&mut format, &codec_name, low_latency_mode);
if client.color.is_hdr() {
match client.next_hdr_meta(Duration::from_millis(250)) {
Ok(meta) => {
format.set_buffer("hdr-static-info", &android_hdr_static_info(&meta));
log::info!("decode: HDR static metadata applied (KEY_HDR_STATIC_INFO)");
}
Err(_) => {
log::info!("decode: HDR session but no mastering metadata yet — DataSpace only")
}
}
}
if let Err(e) = codec.configure(&format, Some(&window), MediaCodecDirection::Decoder) {
log::error!("decode: configure failed: {e}");
return;
}
if let Err(e) = codec.start() {
log::error!("decode: start failed: {e}");
return;
}
log::info!(
"decode: decoder started (async) at {}x{}",
mode.width,
mode.height
);
// The forced TV mode switch (`is_tv` ⇒ ALWAYS strategy) is part of the experimental stack;
// off, every form factor gets the original soft seamless hint.
if mode.refresh_hz > 0
&& !try_set_frame_rate(&window, mode.refresh_hz as f32, is_tv && low_latency_mode)
{
log::debug!(
"decode: set_frame_rate({} Hz) unavailable/declined (non-fatal)",
mode.refresh_hz
);
}
// Skew-corrected latency stats (spec: design/stats-unification.md). Receipt stamps (keyed by the
// pts we queue) live in a shared map: the feeder writes them at receipt, this loop pairs decoded
// output back to them. Behind a `Mutex` since two threads touch it — only ever locked while the
// HUD is visible.
let clock_offset = client.clock_offset_ns;
let in_flight = Arc::new(Mutex::new(VecDeque::<(u64, i128)>::new()));
// Feeder thread: block on the network so this loop doesn't (an AU's arrival becomes an event that
// wakes us immediately, with no input-side poll latency). It also records the `received` HUD stat.
let feeder = {
let client = client.clone();
let stats = stats.clone();
let in_flight = in_flight.clone();
let shutdown = shutdown.clone();
let ev_tx = ev_tx.clone();
std::thread::Builder::new()
.name("pf-decode-feed".into())
.spawn(move || {
feeder_loop(
client,
stats,
in_flight,
clock_offset as i128,
shutdown,
ev_tx,
);
})
.ok()
};
drop(ev_tx); // only the feeder + callbacks keep the channel alive now
// ADPF: same as the sync path — register this thread now, create the session lazily on the first
// presented frame (by when the pump + audio + feeder threads have registered their tids too).
let frame_period_ns = if mode.refresh_hz > 0 {
1_000_000_000i64 / mode.refresh_hz as i64
} else {
0
};
client.register_hot_thread();
let mut hint: Option<crate::adpf::HintSession> = None;
let mut hint_tried = false;
let mut free_inputs: VecDeque<usize> = VecDeque::new();
let mut pending_aus: VecDeque<Frame> = VecDeque::new();
let mut ready: Vec<OutputReady> = Vec::new();
let mut applied_ds: Option<DataSpace> = None;
let mut fed: u64 = 0;
let mut rendered: u64 = 0;
let mut discarded: u64 = 0;
let mut last_dropped = client.frames_dropped();
let mut last_kf_req: Option<Instant> = None;
// Productive (dispatch+feed+present) time between displayed frames; reported to ADPF once one is
// presented. The blocking event wait is excluded (idle, not work) — same accounting as the sync loop.
let mut work_accum_ns: i64 = 0;
let mut fatal = false;
while !shutdown.load(Ordering::Relaxed) && !fatal {
// Block for the next event (idle wait — excluded from the work tally). The short timeout
// drives loss-recovery housekeeping when the pipeline is momentarily quiet.
let ev0 = match ev_rx.recv_timeout(Duration::from_millis(5)) {
Ok(ev) => Some(ev),
Err(mpsc::RecvTimeoutError::Timeout) => None,
Err(mpsc::RecvTimeoutError::Disconnected) => break,
};
let work_t0 = Instant::now();
let mut fmt_dirty = false;
let mut au_dropped = false;
if let Some(ev) = ev0 {
au_dropped |= dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
);
}
// Coalesce every other event already queued into this one work pass — correct newest-only
// presentation across a decode burst, and batched feeding.
while let Ok(ev) = ev_rx.try_recv() {
au_dropped |= dispatch_event(
ev,
&mut pending_aus,
&mut free_inputs,
&mut ready,
&mut fmt_dirty,
&mut fatal,
);
}
if fmt_dirty {
apply_hdr_dataspace(&codec, &window, &mut applied_ds);
}
feed_ready(&codec, &mut pending_aus, &mut free_inputs, &mut fed);
let had_output = !ready.is_empty();
present_ready(
&codec,
&mut ready,
&stats,
&in_flight,
clock_offset,
&mut rendered,
&mut discarded,
);
work_accum_ns += work_t0.elapsed().as_nanos() as i64;
if had_output {
if !hint_tried {
hint_tried = true;
let tids = client.hot_thread_ids();
// The pump/audio priority boost is part of the experimental low-latency stack; the
// ADPF session itself predates it and always runs (max-performance bias gated inside).
if low_latency_mode {
boost_hot_threads(&tids);
}
hint = crate::adpf::HintSession::create(frame_period_ns, &tids, low_latency_mode);
log::info!(
"decode: ADPF hint session {} — {} hot thread(s), target {frame_period_ns} ns",
if hint.is_some() {
"active"
} else {
"unavailable"
},
tids.len(),
);
}
if let Some(h) = &hint {
h.report_actual(work_accum_ns);
}
work_accum_ns = 0;
if rendered > 0 && rendered % 300 == 0 {
log::info!("decode: fed={fed} rendered={rendered} discarded={discarded}");
}
}
// Loss recovery: request an IDR when the reassembler's unrecoverable-drop count climbs (or we
// dropped a parked AU on overflow), throttled so a multi-frame recovery gap doesn't flood the
// control stream.
let dropped = client.frames_dropped();
if dropped > last_dropped || au_dropped {
last_dropped = dropped;
let now = Instant::now();
if last_kf_req.is_none_or(|t| now.duration_since(t) >= Duration::from_millis(100)) {
last_kf_req = Some(now);
let _ = client.request_keyframe();
}
}
}
let _ = codec.stop();
shutdown.store(true, Ordering::SeqCst); // ensure the feeder wakes and exits, then join it
if let Some(j) = feeder {
let _ = j.join();
}
log::info!("decode: stopped (async, fed={fed} rendered={rendered} discarded={discarded})");
}
/// The `pf-decode-feed` thread: block on the connector for the next access unit so the async loop
/// never has to. Records the `received` HUD stat (receipt point) — including the Phase-2 host/network
/// split from any matching 0xCF host timings — then hands the AU to the loop via the event channel.
/// Exits when `shutdown` is set, the session closes, or the loop's receiver is gone.
fn feeder_loop(
client: Arc<NativeClient>,
stats: Arc<crate::stats::VideoStats>,
in_flight: Arc<Mutex<VecDeque<(u64, i128)>>>,
clock_offset: i128,
shutdown: Arc<AtomicBool>,
ev_tx: mpsc::Sender<DecodeEvent>,
) {
// Received AUs awaiting their 0xCF host timing (Phase-2 split), as (pts_ns, capture→received µs).
let mut pending_split: VecDeque<(u64, u64)> = VecDeque::new();
while !shutdown.load(Ordering::Relaxed) {
match client.next_frame(Duration::from_millis(5)) {
Ok(frame) => {
if stats.enabled() {
let received_ns = now_realtime_ns();
let lat_ns = received_ns + clock_offset - frame.pts_ns as i128;
let lat_us =
(lat_ns > 0 && lat_ns < 10_000_000_000).then_some((lat_ns / 1000) as u64);
stats.note_received(frame.data.len(), lat_us, clock_offset != 0);
{
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
g.push_back((frame.pts_ns / 1000, received_ns));
if g.len() > IN_FLIGHT_CAP {
g.pop_front(); // stale — codec never echoed it back
}
}
if let Some(hostnet_us) = lat_us {
pending_split.push_back((frame.pts_ns, hostnet_us));
if pending_split.len() > PENDING_SPLIT_CAP {
pending_split.pop_front();
}
}
while let Ok(t) = client.next_host_timing(Duration::ZERO) {
if let Some(i) = pending_split.iter().position(|&(p, _)| p == t.pts_ns) {
let (_, hostnet_us) = pending_split.remove(i).unwrap();
stats.note_host_split(
t.host_us as u64,
hostnet_us.saturating_sub(t.host_us as u64),
);
}
}
}
if ev_tx.send(DecodeEvent::Au(frame)).is_err() {
break; // the decode loop is gone
}
}
Err(PunktfunkError::NoFrame) => {} // timeout — re-check shutdown and poll again
Err(_) => break, // session closed
}
}
}
/// Route one [`DecodeEvent`] into the loop's working sets. Returns `true` only when a parked AU was
/// dropped on overflow (the caller then requests a keyframe).
fn dispatch_event(
ev: DecodeEvent,
pending_aus: &mut VecDeque<Frame>,
free_inputs: &mut VecDeque<usize>,
ready: &mut Vec<OutputReady>,
fmt_dirty: &mut bool,
fatal: &mut bool,
) -> bool {
match ev {
DecodeEvent::Au(f) => {
pending_aus.push_back(f);
if pending_aus.len() > FRAME_PARK_CAP {
pending_aus.pop_front(); // sustained overflow — drop oldest, signal a keyframe request
return true;
}
}
DecodeEvent::InputAvailable(i) => free_inputs.push_back(i),
DecodeEvent::OutputAvailable { index, pts_us } => ready.push(OutputReady { index, pts_us }),
DecodeEvent::FormatChanged => *fmt_dirty = true,
DecodeEvent::Error { fatal: f } => {
if f {
*fatal = true;
}
}
}
false
}
/// Queue as many parked AUs as there are free input buffer slots (async mode: the indices come from
/// `InputAvailable` callbacks, not a dequeue). Each AU is copied into its codec input buffer and
/// submitted; a too-large AU is truncated (logged) rather than dropped.
fn feed_ready(
codec: &MediaCodec,
pending_aus: &mut VecDeque<Frame>,
free_inputs: &mut VecDeque<usize>,
fed: &mut u64,
) {
while !pending_aus.is_empty() && !free_inputs.is_empty() {
let idx = free_inputs.pop_front().unwrap();
let frame = pending_aus.pop_front().unwrap();
let pts_us = frame.pts_ns / 1000;
let Some(dst) = codec.input_buffer(idx) else {
log::warn!("decode: input_buffer({idx}) returned None — dropping AU");
continue;
};
let au = &frame.data;
let n = au.len().min(dst.len());
if n < au.len() {
log::warn!(
"decode: AU {} > input buffer {}, truncated",
au.len(),
dst.len()
);
}
// SAFETY: `au` (wire AU) and `dst` (codec input buffer) are distinct allocations, both valid
// for `n` bytes; `MaybeUninit<u8>` is layout-identical to `u8`, so this initializes dst[..n].
unsafe {
std::ptr::copy_nonoverlapping(au.as_ptr(), dst.as_mut_ptr().cast::<u8>(), n);
}
if let Err(e) = codec.queue_input_buffer_by_index(idx, 0, n, pts_us, 0) {
log::warn!("decode: queue_input_buffer_by_index: {e}");
} else {
*fed += 1;
}
}
}
/// Present only the NEWEST ready output (render = true) and release the rest without rendering — a
/// burst of stale frames on glass is worse than skipping to the freshest (the sync loop's newest-ready
/// policy, callback-driven). Every dequeued buffer, rendered or not, is the HUD's `decoded`
/// measurement point (it finished decoding either way); samples are recorded in pts order so the
/// receipt-map eviction stays monotonic. `ready` is drained.
fn present_ready(
codec: &MediaCodec,
ready: &mut Vec<OutputReady>,
stats: &crate::stats::VideoStats,
in_flight: &Mutex<VecDeque<(u64, i128)>>,
clock_offset: i64,
rendered: &mut u64,
discarded: &mut u64,
) {
if ready.is_empty() {
return;
}
if stats.enabled() {
let mut g = in_flight
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
for o in ready.iter() {
note_decoded_pts(stats, &mut g, clock_offset, o.pts_us);
}
}
let last = ready.len() - 1;
for (i, o) in ready.drain(..).enumerate() {
let render = i == last;
match codec.release_output_buffer_by_index(o.index, render) {
Ok(()) if render => *rendered += 1,
Ok(()) => *discarded += 1,
Err(e) => {
log::warn!(
"decode: release_output_buffer_by_index({}, {render}): {e}",
o.index
)
}
}
}
}
/// React to an output-format change by signalling the stream's HDR dataspace on the Surface (SDR
/// streams leave the default alone). The AMediaCodec analogue of the sync loop's `OutputFormatChanged`
/// handling; safe to call repeatedly (`applied_ds` dedups).
fn apply_hdr_dataspace(
codec: &MediaCodec,
window: &NativeWindow,
applied_ds: &mut Option<DataSpace>,
) {
if let Some(ds) = hdr_dataspace(codec) {
if *applied_ds != Some(ds) {
match window.set_buffers_data_space(ds) {
Ok(()) => {
*applied_ds = Some(ds);
log::info!("decode: HDR stream → Surface dataspace {ds}");
}
Err(e) => {
log::warn!("decode: set_buffers_data_space({ds}) failed (non-fatal): {e}")
}
}
}
}
}
/// Raise the pipeline's OTHER hot threads — the core's data-plane pump (UDP receive + FEC
/// reassembly) and the audio decode thread — toward the display band, matching this decode thread's
/// own boost. `setpriority(PRIO_PROCESS, tid)` targets any task in the process, so we do it from
/// here once their tids are known (the same set ADPF hints), without a per-platform priority hook
/// in the shared core. Slightly below the decode thread's -10 so the display path still wins.
/// Best-effort; skips this thread (already boosted) and is non-fatal if the platform refuses.
fn boost_hot_threads(tids: &[i32]) {
// SAFETY: `gettid` is an always-safe syscall on the calling thread.
let self_tid = unsafe { libc::gettid() };
for &tid in tids {
if tid == self_tid {
continue;
}
// SAFETY: `setpriority` with PRIO_PROCESS + a live tid in our own process is an always-safe
// syscall; a refusal is reported via the return value, not UB.
unsafe {
if libc::setpriority(libc::PRIO_PROCESS, tid as libc::id_t, -8) != 0 {
log::debug!("decode: setpriority(-8) on hot tid {tid} failed (non-fatal)");
}
}
}
}
/// Best-effort: raise the decode thread toward Android's URGENT_DISPLAY band so background work
/// can't preempt it under load (which shows up as late/dropped frames). Non-fatal if the platform
/// refuses (foreground apps may set their own threads; the exact floor is policy-dependent).
@@ -343,23 +1017,48 @@ fn boost_thread_priority() {
}
}
/// `ANativeWindow_setFrameRate` (NDK **API 30**) resolved from `libandroid.so` at runtime, so the lib
/// still loads on our API-28 floor — a hard import of a >floor symbol makes `dlopen`/`System.load`
/// fail on every API-28/29 device, even where this path is never hit. Mirrors the dlsym approach in
/// [`crate::adpf`]. Returns `true` when the platform accepted the hint; `false` on API < 30 (symbol
/// absent) or when the platform declined. `compatibility` is fixed to the DEFAULT (0) policy.
fn try_set_frame_rate(window: &NativeWindow, frame_rate: f32) -> bool {
/// Set the surface's frame-rate hint to the stream's refresh so SurfaceFlinger picks a matching
/// display mode and aligns vsync (no 60-in-120 judder). Both NDK entry points sit above our API-28
/// floor, so both are dlsym-resolved at runtime (a hard import of a >floor symbol makes
/// `dlopen`/`System.load` fail on every API-28/29 device, even where this path is never hit —
/// mirrors [`crate::adpf`]):
/// - On a **TV** (`is_tv`): `ANativeWindow_setFrameRateWithChangeStrategy` (**API 31**) with
/// `changeFrameRateStrategy = ALWAYS`, which actively drives the HDMI output into the matching
/// mode (e.g. 60↔120) instead of leaving the panel at its default and judder-matching. The
/// forced switch may blank the panel briefly — acceptable once at stream start, not wanted on a
/// phone. Falls through to the 2-arg hint on API 30.
/// - Otherwise: `ANativeWindow_setFrameRate` (**API 30**) with `compatibility = DEFAULT` — the
/// softer, seamless-preferred hint for phones/tablets and the universal fallback.
///
/// Returns `true` when the platform accepted a hint; `false` on API < 30 (symbols absent) or a
/// decline.
fn try_set_frame_rate(window: &NativeWindow, frame_rate: f32, is_tv: bool) -> bool {
// int32_t ANativeWindow_setFrameRate(ANativeWindow*, float frameRate, int8_t compatibility)
type SetFrameRateFn = unsafe extern "C" fn(*mut c_void, f32, i8) -> i32;
// int32_t ANativeWindow_setFrameRateWithChangeStrategy(
// ANativeWindow*, float frameRate, int8_t compatibility, int8_t changeFrameRateStrategy)
type SetFrameRateStrategyFn = unsafe extern "C" fn(*mut c_void, f32, i8, i8) -> i32;
// SAFETY: `dlopen` of the always-mapped `libandroid.so` (only bumps its refcount; never closed —
// process-lifetime handle). `dlsym` returns null when the symbol is absent (device API < 30),
// checked before transmuting the non-null pointer to its fn-pointer type. `window.ptr()` is the
// live `ANativeWindow` this `NativeWindow` owns for the call's duration.
// process-lifetime handle). Each `dlsym` returns null when the symbol is absent (device below the
// symbol's API level), checked before transmuting the non-null pointer to its fn-pointer type.
// `window.ptr()` is the live `ANativeWindow` this `NativeWindow` owns for the call's duration.
unsafe {
let lib = libc::dlopen(c"libandroid.so".as_ptr(), libc::RTLD_NOW);
if lib.is_null() {
return false;
}
// TV: prefer the API-31 change-strategy form to force the mode switch (strategy 1 = ALWAYS,
// compatibility 0 = DEFAULT). Absent on API 30 ⇒ fall through to the 2-arg hint below.
if is_tv {
let sym = libc::dlsym(
lib,
c"ANativeWindow_setFrameRateWithChangeStrategy".as_ptr(),
);
if !sym.is_null() {
let set = std::mem::transmute::<*mut c_void, SetFrameRateStrategyFn>(sym);
return set(window.ptr().as_ptr().cast(), frame_rate, 0, 1) == 0;
}
}
let sym = libc::dlsym(lib, c"ANativeWindow_setFrameRate".as_ptr());
if sym.is_null() {
return false; // device API < 30 — no per-surface frame-rate hint
@@ -499,7 +1198,22 @@ fn note_decoded(
clock_offset: i64,
buf: &OutputBuffer<'_>,
) {
let pts_us = buf.info().presentation_time_us().max(0) as u64;
note_decoded_pts(
stats,
in_flight,
clock_offset,
buf.info().presentation_time_us().max(0) as u64,
);
}
/// The [`note_decoded`] body keyed by the echoed `presentationTimeUs` directly — the async loop has
/// the pts (from the output callback's `BufferInfo`) but no borrowed `OutputBuffer`, so it calls this.
fn note_decoded_pts(
stats: &crate::stats::VideoStats,
in_flight: &mut VecDeque<(u64, i128)>,
clock_offset: i64,
pts_us: u64,
) {
let decoded_ns = now_realtime_ns();
// Pair the echoed pts back to its receipt stamp, evicting stale (older) entries as we go.
let mut received_ns = None;
+71 -2
View File
@@ -32,8 +32,23 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeGenerateIde
}
}
/// `NativeBridge.nativeSetLowLatencyMode(enabled)` — apply the user's "Low-latency mode
/// (experimental)" toggle to the process-wide transport defaults, today just DSCP/QoS marking on
/// the media sockets. Must be called BEFORE `nativeConnect` (the tag is applied at socket
/// creation); Kotlin's one connect choke point (`HostConnect.connectToHost`) does. The rest of the
/// toggle rides explicit per-session parameters (`nativeStartVideo` / `nativeStartAudio`).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSetLowLatencyMode(
_env: JNIEnv,
_this: JObject,
enabled: jboolean,
) {
punktfunk_core::transport::set_dscp_default(enabled != 0);
}
/// `NativeBridge.nativeConnect(host, port, w, h, hz, certPem, keyPem, pinHex, bitrateKbps,
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, preferredCodec, timeoutMs): Long`.
/// compositorPref, gamepadPref, hdrEnabled, audioChannels, preferredCodec, timeoutMs, launch): Long`.
/// `launch` (empty ⇒ none) is a store-qualified library id to boot straight into a game.
/// `certPem`/`keyPem` empty = anonymous, else presented as the persistent identity. `pinHex` empty
/// = TOFU (read `nativeHostFingerprint` after), else 64-hex SHA-256 to pin the host (mismatch → 0).
/// `bitrateKbps` 0 = host default. `compositorPref`/`gamepadPref` are `CompositorPref`/`GamepadPref`
@@ -63,6 +78,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
audio_channels: jint,
preferred_codec: jint,
timeout_ms: jint,
launch: JString<'local>,
) -> jlong {
let host: String = match env.get_string(&host) {
Ok(s) => s.into(),
@@ -74,6 +90,13 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
.unwrap_or_default();
let key: String = env.get_string(&key_pem).map(Into::into).unwrap_or_default();
let pin_hex: String = env.get_string(&pin_hex).map(Into::into).unwrap_or_default();
// A store-qualified library id (`steam:<appid>` / `custom:<id>`) to boot straight into a game;
// null / empty ⇒ None (a plain desktop connect). Rides the Hello as `launch`.
let launch: Option<String> = env
.get_string(&launch)
.map(Into::into)
.ok()
.filter(|s: &String| !s.is_empty());
let identity: Option<(String, String)> = if cert.is_empty() || key.is_empty() {
None
@@ -124,7 +147,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// + the soft `preferred_codec` and echoes it in `connector.codec`, which drives the mime below.
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC,
preferred_codec.clamp(0, u8::MAX as jint) as u8,
None, // launch: default app
launch, // a store-qualified library id to boot into a game, or None for the desktop
pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous)
// Handshake budget from Kotlin: ~10 s for a normal connect, ~185 s for "request access"
@@ -170,6 +193,30 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClose(
})
}
/// `NativeBridge.nativeDisconnectQuit(handle)` — signal a DELIBERATE user quit before `nativeClose`,
/// so the session closes with `QUIT_CLOSE_CODE` and the host tears it down immediately instead of
/// holding the keep-alive linger for a reconnect. Call from an explicit disconnect action only (a
/// plain drop / app-background keeps the linger). The handle is only BORROWED (not freed). No-op on `0`.
///
/// # Safety contract
/// `handle` must be `0` or a live handle from [`Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect`],
/// not freed / closed concurrently with this call (Kotlin still owns it and closes it via `nativeClose`).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeDisconnectQuit(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) {
jni_guard((), || {
if handle != 0 {
// SAFETY: per the contract, `handle` is a live `Box<SessionHandle>` — we only borrow it
// (no drop), so it stays owned by Kotlin for the later `nativeClose`.
let sh = unsafe { &*(handle as *const SessionHandle) };
sh.client.disconnect_quit();
}
})
}
/// `NativeBridge.nativeHostFingerprint(handle): String` — the SHA-256 (64-hex) of the cert the host
/// presented on this connection. Valid after a successful `nativeConnect`; Kotlin pins it on a TOFU
/// connect. `""` on a `0` handle.
@@ -192,6 +239,28 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeHostFingerp
}
}
/// `NativeBridge.nativeSessionEnded(handle): Boolean` — has the underlying QUIC session ended?
/// `true` once the connection closed (a host suspend / crash / network drop idle-timed it out, or the
/// host closed it) — from then on no more frames arrive and the video sits frozen on its last one.
/// Kotlin's stream watchdog polls this (~1 Hz) to leave a dead stream and return to the menu (where
/// the user can Wake-on-LAN the host) instead of stranding them on a frozen frame. `false` on a `0`
/// handle. Cheap (one atomic load); safe on the UI thread.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSessionEnded(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
jni_guard(0, || {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
jboolean::from(h.client.is_session_ended())
})
}
/// `NativeBridge.nativePair(host, port, certPem, keyPem, pin, name): String` — run the SPAKE2 PIN
/// ceremony, presenting our persistent identity. On success returns the host's verified fingerprint
/// (64-hex) to persist + pin; on any failure (wrong PIN / MITM / host reject / unreachable) returns
+82 -8
View File
@@ -2,20 +2,31 @@
//! ~1 Hz decode-stats drain for the HUD.
use jni::objects::JObject;
use jni::sys::{jboolean, jdoubleArray, jlong, jsize};
// Used only by the android-gated `nativeStartVideo`; on the host build that fn is cfg'd out.
#[cfg(target_os = "android")]
use jni::objects::JString;
use jni::sys::{jboolean, jdoubleArray, jlong, jsize, jstring};
use jni::JNIEnv;
use super::{jni_guard, SessionHandle};
/// `NativeBridge.nativeStartVideo(handle, surface)` — wrap the SurfaceView's `Surface` as an
/// `ANativeWindow` and start the HEVC decode thread rendering onto it. No-op if already started.
/// `NativeBridge.nativeStartVideo(handle, surface, decoderName, lowLatencyMode, lowLatencyFeature)`
/// — wrap the SurfaceView's `Surface` as an `ANativeWindow` and start the decode thread rendering
/// onto it. `decoderName` is the codec Kotlin ranked from `MediaCodecList` (`""` = let the platform
/// resolve the default for the MIME); `lowLatencyMode` is the user's master toggle;
/// `lowLatencyFeature` is whether that decoder advertised `FEATURE_LowLatency` (HUD label only).
/// No-op if already started.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo(
env: JNIEnv,
mut env: JNIEnv,
_this: JObject,
handle: jlong,
surface: JObject,
decoder_name: JString,
low_latency_mode: jboolean,
ll_feature: jboolean,
is_tv: jboolean,
) {
use super::VideoThread;
use std::sync::atomic::AtomicBool;
@@ -24,6 +35,12 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo(
if handle == 0 {
return;
}
// The decoder name Kotlin picked (empty string / read failure ⇒ None ⇒ default resolver).
let decoder = env
.get_string(&decoder_name)
.ok()
.map(String::from)
.filter(|s| !s.is_empty());
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
let mut guard = h.video.lock().unwrap();
@@ -48,13 +65,67 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo(
let client = h.client.clone();
let sd = shutdown.clone();
let st = h.stats.clone(); // session-lifetime stats (gate survives surface recreate)
let opts = crate::decode::DecodeOptions {
decoder_name: decoder,
ll_feature: ll_feature != 0,
low_latency_mode: low_latency_mode != 0,
is_tv: is_tv != 0,
};
let join = std::thread::Builder::new()
.name("pf-decode".into())
.spawn(move || crate::decode::run(client, window, sd, st))
.spawn(move || crate::decode::run(client, window, sd, st, opts))
.ok();
*guard = Some(VideoThread { shutdown, join });
}
/// `NativeBridge.nativeVideoMime(handle): String` — the MediaCodec MIME for the codec the host
/// resolved (`"video/hevc"` / `"video/avc"` / `"video/av01"`), so Kotlin can rank `MediaCodecList`
/// decoders for it before calling [`Java_io_unom_punktfunk_kit_NativeBridge_nativeStartVideo`].
/// Empty string on a `0` handle. Cheap; safe on the UI thread.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoMime<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
handle: jlong,
) -> jstring {
jni_guard(std::ptr::null_mut(), || {
if handle == 0 {
return std::ptr::null_mut();
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
match env.new_string(crate::decode::codec_mime(h.client.codec)) {
Ok(s) => s.into_raw(),
Err(_) => std::ptr::null_mut(),
}
})
}
/// `NativeBridge.nativeVideoDecoderLabel(handle): String` — the resolved decoder identity for the
/// HUD, e.g. `c2.qti.avc.decoder · low-latency`, or `""` before the decode thread has resolved one.
/// One-shot (the decoder is fixed for the session); poll once after the HUD appears. Not
/// android-gated — pure `jni` + a lock, so it links on the host build too (Kotlin only calls it on
/// device).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeVideoDecoderLabel<'local>(
env: JNIEnv<'local>,
_this: JObject<'local>,
handle: jlong,
) -> jstring {
jni_guard(std::ptr::null_mut(), || {
if handle == 0 {
return std::ptr::null_mut();
}
// SAFETY: live handle per the nativeConnect/nativeClose contract.
let h = unsafe { &*(handle as *const SessionHandle) };
match env.new_string(h.stats.decoder_label()) {
Ok(s) => s.into_raw(),
Err(_) => std::ptr::null_mut(),
}
})
}
/// `NativeBridge.nativeStopVideo(handle)` — stop + join the decode thread (without closing the
/// session). No-op on `0`.
#[no_mangle]
@@ -162,14 +233,17 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSetVideoSta
})
}
/// `NativeBridge.nativeStartAudio(handle)` — start the Opus→AAudio playback thread. No-op if already
/// started or on a `0` handle. Best-effort: a failure leaves video streaming.
/// `NativeBridge.nativeStartAudio(handle, lowLatencyMode)` — start the Opus→AAudio playback thread.
/// `lowLatencyMode` (the experimental toggle) tags the stream usage=Game for the HAL's game-audio
/// routing. No-op if already started or on a `0` handle. Best-effort: a failure leaves video
/// streaming.
#[cfg(target_os = "android")]
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartAudio(
_env: JNIEnv,
_this: JObject,
handle: jlong,
low_latency_mode: jboolean,
) {
if handle == 0 {
return;
@@ -180,7 +254,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeStartAudio(
if guard.is_some() {
return; // already playing
}
match crate::audio::AudioPlayback::start(h.client.clone()) {
match crate::audio::AudioPlayback::start(h.client.clone(), low_latency_mode != 0) {
Some(p) => *guard = Some(p),
None => log::error!("nativeStartAudio: playback init failed (video unaffected)"),
}
+43
View File
@@ -22,9 +22,21 @@ pub struct VideoStats {
/// they (and the caller's latency computation — see `enabled`) early-out on this flag alone.
/// Off until Kotlin shows the HUD.
enabled: AtomicBool,
/// The resolved decoder identity for the HUD: the codec's actual `AMediaCodec` name (e.g.
/// `c2.qti.avc.decoder`) and whether it advertised `FEATURE_LowLatency`. Set once when the
/// decode thread creates the codec (`set_decoder`), read one-shot by `nativeVideoDecoderLabel`.
/// Separate from `inner` (never touched per-frame) so naming it costs nothing on the hot path.
decoder: Mutex<Option<DecoderInfo>>,
inner: Mutex<Inner>,
}
/// The chosen decoder's identity, surfaced on the stats HUD so before/after latency comparisons
/// name the codec that produced them.
struct DecoderInfo {
name: String,
low_latency: bool,
}
struct Inner {
window_start: Instant,
frames: u64,
@@ -79,6 +91,7 @@ impl VideoStats {
pub fn new() -> VideoStats {
VideoStats {
enabled: AtomicBool::new(false),
decoder: Mutex::new(None),
inner: Mutex::new(Inner {
window_start: Instant::now(),
frames: 0,
@@ -121,6 +134,36 @@ impl VideoStats {
}
}
/// Record the resolved decoder identity for the HUD — the codec's real `AMediaCodec` name and
/// whether it reported `FEATURE_LowLatency`. Called once from the decode thread right after the
/// codec is created (before `configure`), overwriting any prior value on a surface recreate.
// Set only by the android-only decode thread; unreferenced on the host build — expected.
#[cfg_attr(not(target_os = "android"), allow(dead_code))]
pub fn set_decoder(&self, name: &str, low_latency: bool) {
let mut g = self
.decoder
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
*g = Some(DecoderInfo {
name: name.to_owned(),
low_latency,
});
}
/// The decoder label for the HUD, e.g. `c2.qti.avc.decoder · low-latency`, or `""` before the
/// decode thread has resolved one. Cheap (a lock + a string build); safe on the UI thread.
pub fn decoder_label(&self) -> String {
let g = self
.decoder
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
match &*g {
Some(d) if d.low_latency => format!("{} · low-latency", d.name),
Some(d) => d.name.clone(),
None => String::new(),
}
}
/// Record one received access unit: its wire size and (if in range) its capture→received
/// `host+network` stage sample. Receipt is the fps/goodput counting point per the spec.
// Driven only by the android-only decode thread; unreferenced on the host build — expected.
@@ -432,6 +432,7 @@
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -471,6 +472,7 @@
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = Config/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = Punktfunk;
INFOPLIST_KEY_GCSupportsControllerUserInteraction = YES;
INFOPLIST_KEY_LSApplicationCategoryType = "public.app-category.utilities";
INFOPLIST_KEY_NSLocalNetworkUsageDescription = "Punktfunk connects directly to your punktfunk host on the local network to stream video, audio, and input.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "Your microphone is streamed to the connected punktfunk host, where it appears as a virtual microphone.";
@@ -276,7 +276,10 @@ final class SessionModel: ObservableObject {
disconnect()
}
func disconnect() {
/// Tear the session down. `deliberate` (the default) means a user-initiated quit signal
/// `disconnectQuit()` so the host skips the keep-alive linger; `sessionEnded()` (a host-ended /
/// dropped session) passes `false` to leave the linger intact.
func disconnect(deliberate: Bool = true) {
statsTimer?.invalidate()
statsTimer = nil
let audio = self.audio
@@ -294,6 +297,8 @@ final class SessionModel: ObservableObject {
Task.detached {
audio?.stop()
feedback?.stop()
// Deliberate user quit tell the host to skip the keep-alive linger (must precede close).
if deliberate { conn.disconnectQuit() }
conn.close()
}
} else {
@@ -321,7 +326,7 @@ final class SessionModel: ObservableObject {
func sessionEnded() {
guard connection != nil else { return }
let name = activeHost?.displayName ?? "host"
disconnect()
disconnect(deliberate: false) // host/network ended it keep the linger for a reconnect
errorMessage = "Session ended by \(name)."
}
@@ -759,6 +759,17 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_send_input(h, &ev)
}
/// Signal a **deliberate** user-initiated quit before ``close()``: the connection closes with
/// `QUIT_CLOSE_CODE` (81) so the host tears the session down immediately instead of holding the
/// keep-alive linger for a reconnect. Call only from an explicit "Disconnect" action NOT from a
/// network drop / host-ended / app-background (those keep the linger). Idempotent, safe pre-close.
public func disconnectQuit() {
abiLock.lock()
defer { abiLock.unlock() }
guard let h = handle, !closeRequested else { return }
punktfunk_connection_disconnect_quit(h)
}
/// Close the connection and free the handle. Safe from any thread, idempotent; waits
/// for in-flight pulls ( their timeouts) before tearing down.
public func close() {
@@ -84,15 +84,6 @@ public final class InputCapture {
/// its Esc suppression need it in both states).
private var cmdKeysDown: Set<UInt32> = []
#if os(macOS)
/// Previous raw `NSEvent.modifierFlags.rawValue` (LOW 16 bits intact those carry the
/// device-dependent L/R bits). Modifier keys never fire keyDown/keyUp on macOS; they
/// arrive as flagsChanged, which doesn't carry down-vs-up we recover that by diffing
/// this snapshot. Resynced (not diffed) while forwarding is off so a modifier held
/// across a capture toggle can't produce a phantom transition on re-engage.
private var prevModFlags: UInt = 0
#endif
/// While true, mouse/keyboard flow to the host and key NSEvents are swallowed
/// locally; while false the user is interacting with the local UI (dragging the
/// window, clicking the HUD) and nothing is forwarded. Main-queue only.
@@ -279,12 +270,6 @@ public final class InputCapture {
residualY = 0
residualScrollX = 0
residualScrollY = 0
#if os(macOS)
// Drop the modifier snapshot too: a flagsChanged transition can be missed if focus
// leaves mid-chord, and the next handleFlagsChanged resyncs from a clean slate (it
// resyncs while released anyway, but this keeps stuck state from outliving a blur).
prevModFlags = 0
#endif
}
/// Release any held MOUSE buttons host-side, leaving keyboard state untouched. Used when
@@ -359,39 +344,52 @@ public final class InputCapture {
}
/// NSEvent modifier path (macOS): modifier keys never fire keyDown/keyUp they arrive
/// as flagsChanged, which carries no down-vs-up. We diff the raw flags against the prior
/// snapshot to recover each transition, and the changed key's L/R identity from the
/// device-dependent bits in the LOW 16 bits (the .deviceIndependentFlagsMask the
/// monitor uses deliberately strips exactly these do NOT pre-mask here). Each side maps
/// to the same L/R modifier VK `hidToVK` already emits, so the host needs no change.
/// Fed `UInt(event.modifierFlags.rawValue)`.
public func handleFlagsChanged(_ rawFlags: UInt) {
// While released we only resync the snapshot, so a modifier held across a capture
// toggle doesn't show up as a spurious transition the moment forwarding re-engages.
guard forwarding else {
prevModFlags = rawFlags
return
/// as flagsChanged, which carries no down-vs-up. `keyCode` names the key that changed
/// (kVK_Control & co., already L/R-specific); `resolveModifier` recovers the direction
/// from the flags. Fed `event.keyCode` + `UInt(event.modifierFlags.rawValue)` LOW 16
/// bits intact, they carry the device-dependent L/R bits (the .deviceIndependentFlagsMask
/// the monitor uses deliberately strips exactly these do NOT pre-mask here).
public func handleFlagsChanged(keyCode: UInt16, rawFlags: UInt) {
if inputDebug {
inputLog.debug(
"flagsChanged keyCode \(keyCode, privacy: .public) flags 0x\(String(rawFlags, radix: 16), privacy: .public) forwarding \(self.forwarding, privacy: .public)")
}
// (device-dependent mask, VK). LOW-16-bit masks from IOLLEvent.h (NX_DEVICE*MASK):
// Lshift 0x2 Rshift 0x4 | Lctrl 0x1 Rctrl 0x2000 | Lalt 0x20 Ralt 0x40 | Lcmd 0x8 Rcmd 0x10.
let table: [(UInt, UInt32)] = [
(0x2, 0xA0), (0x4, 0xA1), // VK_LSHIFT / VK_RSHIFT
(0x1, 0xA2), (0x2000, 0xA3), // VK_LCONTROL / VK_RCONTROL
(0x20, 0xA4), (0x40, 0xA5), // VK_LMENU / VK_RMENU (left/right alt-option)
(0x8, 0x5B), (0x10, 0x5C), // VK_LWIN / VK_RWIN (left/right command)
]
for (mask, vk) in table {
let now = (rawFlags & mask) != 0
let was = (prevModFlags & mask) != 0
guard now != was else { continue }
// Keep cmdKeysDown in step (the toggle + Esc suppression read it); sendKey
// adds the VK to pressedVKs so releaseAll/blur flushes a held modifier cleanly.
if vk == 0x5B || vk == 0x5C {
if now { cmdKeysDown.insert(vk) } else { cmdKeysDown.remove(vk) }
}
sendKey(vk, down: now)
guard forwarding else { return }
guard let (vk, down) = Self.resolveModifier(
keyCode: keyCode, rawFlags: rawFlags, isDown: { pressedVKs.contains($0) })
else { return } // Fn / Caps Lock / unknown nothing the host consumes on this path
// Keep cmdKeysDown in step (the toggle + Esc suppression read it); sendKey
// adds the VK to pressedVKs so releaseAll/blur flushes a held modifier cleanly.
if vk == 0x5B || vk == 0x5C {
if down { cmdKeysDown.insert(vk) } else { cmdKeysDown.remove(vk) }
}
prevModFlags = rawFlags
sendKey(vk, down: down)
}
/// Resolve one flagsChanged transition to (Windows VK, down). The changed key is
/// `keyCode`; the direction comes from the flags. The device-dependent L/R bits (LOW
/// 16 bits, NX_DEVICE*KEYMASK) disambiguate the two same-class keys, but some
/// keyboards ship flagsChanged WITHOUT them only the device-independent class
/// bit (NX_CONTROLMASK & co.) is set. A pure diff of the device bits silently drops
/// those keys (seen live: Control never forwarded), so this is keyCode-driven with the
/// flags as evidence: class bit clear the key went up; device bits present they
/// say which side is held now; class bit set with NO device bits flip the held state
/// we track (`isDown`, from pressedVKs SDL ships the same fallback). Each keyCode
/// maps to the L/R modifier VK `hidToVK` already emits, so the host needs no change.
/// Returns nil for modifiers the host doesn't consume on this path (Fn, Caps Lock).
static func resolveModifier(
keyCode: UInt16, rawFlags: UInt, isDown: (UInt32) -> Bool
) -> (vk: UInt32, down: Bool)? {
guard let mod = modifierBits[keyCode] else { return nil }
let down: Bool
if rawFlags & mod.classMask == 0 {
down = false
} else if rawFlags & (mod.deviceBit | mod.siblingBit) != 0 {
down = rawFlags & mod.deviceBit != 0
} else {
down = !isDown(mod.vk)
}
return (mod.vk, down)
}
#endif
@@ -98,5 +98,23 @@ extension InputCapture {
m[0x47] = 0x90 // KP clear sits where NumLock is VK_NUMLOCK. (KP equals 0x51 dropped.)
return m
}()
/// NSEvent.keyCode of each modifier key (kVK_Shift & co. modifiers arrive only as
/// flagsChanged) its Windows VK plus the `NSEvent.modifierFlags` bits that describe
/// it: `classMask` is the device-INDEPENDENT NX_*MASK for the modifier class,
/// `deviceBit`/`siblingBit` the device-dependent bits (LOW 16 bits, NX_DEVICE*KEYMASK
/// in IOLLEvent.h) for this key and its opposite-side twin. Consumed by
/// `resolveModifier`, which explains why both kinds of bit are needed.
static let modifierBits:
[UInt16: (vk: UInt32, classMask: UInt, deviceBit: UInt, siblingBit: UInt)] = [
56: (0xA0, 0x2_0000, 0x2, 0x4), // left shift VK_LSHIFT
60: (0xA1, 0x2_0000, 0x4, 0x2), // right shift VK_RSHIFT
59: (0xA2, 0x4_0000, 0x1, 0x2000), // left control VK_LCONTROL
62: (0xA3, 0x4_0000, 0x2000, 0x1), // right control VK_RCONTROL
58: (0xA4, 0x8_0000, 0x20, 0x40), // left option VK_LMENU
61: (0xA5, 0x8_0000, 0x40, 0x20), // right option VK_RMENU
55: (0x5B, 0x10_0000, 0x8, 0x10), // left command VK_LWIN
54: (0x5C, 0x10_0000, 0x10, 0x8), // right command VK_RWIN
]
#endif
}
@@ -346,10 +346,13 @@ public final class StreamLayerView: NSView {
super.keyUp(with: event)
}
/// Modifier keys (shift/control/option/command) arrive ONLY as flagsChanged on macOS,
/// never keyDown/keyUp InputCapture diffs the raw flags to recover each L/R down/up.
/// never keyDown/keyUp the changed key is `event.keyCode`; InputCapture resolves the
/// down-vs-up direction from the flags (diffing the device-dependent flag bits alone
/// proved unreliable some keyboards omit them, which silently dropped Control).
public override func flagsChanged(with event: NSEvent) {
if captured, let inputCapture {
inputCapture.handleFlagsChanged(UInt(event.modifierFlags.rawValue))
inputCapture.handleFlagsChanged(
keyCode: event.keyCode, rawFlags: UInt(event.modifierFlags.rawValue))
return
}
super.flagsChanged(with: event)
@@ -0,0 +1,87 @@
#if os(macOS)
import XCTest
@testable import PunktfunkKit
/// Pins the macOS flagsChanged modifier-VK resolution (InputCapture.resolveModifier).
/// Modifier keys arrive only as flagsChanged, which carries no down-vs-up: the changed key
/// is the event's keyCode, and the direction is recovered from the flag bits with a
/// held-state fallback for keyboards that omit the device-dependent L/R bits (the gap that
/// used to silently drop Control when the transition was diffed from those bits alone).
final class ModifierResolveTests: XCTestCase {
/// Resolve with a fixed already-held answer for the fallback path.
private func resolve(
keyCode: UInt16, rawFlags: UInt, held: Bool = false
) -> (vk: UInt32, down: Bool)? {
InputCapture.resolveModifier(keyCode: keyCode, rawFlags: rawFlags) { _ in held }
}
// MARK: Keyboards that report the device-dependent L/R bits (the common case)
func testControlPressAndReleaseWithDeviceBits() {
// Real left-Control down: NX_CONTROLMASK | NX_DEVICELCTLKEYMASK (+ misc low bits).
let down = resolve(keyCode: 59, rawFlags: 0x4_0101)
XCTAssertEqual(down?.vk, 0xA2) // VK_LCONTROL
XCTAssertEqual(down?.down, true)
// Release: the class mask is gone entirely.
let up = resolve(keyCode: 59, rawFlags: 0x100)
XCTAssertEqual(up?.vk, 0xA2)
XCTAssertEqual(up?.down, false)
}
func testRightControlUsesItsOwnDeviceBit() {
let down = resolve(keyCode: 62, rawFlags: 0x4_2000)
XCTAssertEqual(down?.vk, 0xA3) // VK_RCONTROL
XCTAssertEqual(down?.down, true)
}
func testReleasingOneOfTwoHeldControls() {
// Left goes up while right stays held: class mask still set, right device bit
// still set, LEFT device bit cleared the left key must resolve as UP.
let leftUp = resolve(keyCode: 59, rawFlags: 0x4_2000, held: true)
XCTAssertEqual(leftUp?.vk, 0xA2)
XCTAssertEqual(leftUp?.down, false)
}
func testEverySideMapsToItsOwnVK() {
XCTAssertEqual(resolve(keyCode: 56, rawFlags: 0x2_0002)?.vk, 0xA0) // VK_LSHIFT
XCTAssertEqual(resolve(keyCode: 60, rawFlags: 0x2_0004)?.vk, 0xA1) // VK_RSHIFT
XCTAssertEqual(resolve(keyCode: 58, rawFlags: 0x8_0020)?.vk, 0xA4) // VK_LMENU
XCTAssertEqual(resolve(keyCode: 61, rawFlags: 0x8_0040)?.vk, 0xA5) // VK_RMENU
XCTAssertEqual(resolve(keyCode: 55, rawFlags: 0x10_0008)?.vk, 0x5B) // VK_LWIN
XCTAssertEqual(resolve(keyCode: 54, rawFlags: 0x10_0010)?.vk, 0x5C) // VK_RWIN
for (_, down) in [56, 60, 58, 61, 55, 54].compactMap({
self.resolve(keyCode: UInt16($0), rawFlags: 0xFF_FFFF)
}) {
XCTAssertTrue(down)
}
}
// MARK: Keyboards that DON'T report the device bits (the bug this resolver fixes)
func testControlPressWithoutDeviceBitsFallsBackToHeldState() {
// Only NX_CONTROLMASK, no low bits at all: a flag diff of the device bits sees no
// transition and drops the key the fallback must infer DOWN from "not held yet".
let down = resolve(keyCode: 59, rawFlags: 0x4_0000, held: false)
XCTAssertEqual(down?.vk, 0xA2)
XCTAssertEqual(down?.down, true)
// And the mirror release (class cleared) still resolves as UP.
let up = resolve(keyCode: 59, rawFlags: 0, held: true)
XCTAssertEqual(up?.down, false)
}
func testClassBitStillSetButKeyAlreadyHeldResolvesUp() {
// Device-bit-less keyboard, second same-class key still holding the class bit:
// the best available answer for the key that changed is to flip its held state.
let up = resolve(keyCode: 59, rawFlags: 0x4_0000, held: true)
XCTAssertEqual(up?.down, false)
}
// MARK: Modifiers the host doesn't consume on this path
func testFnAndCapsLockResolveToNothing() {
XCTAssertNil(resolve(keyCode: 63, rawFlags: 0x80_0000)) // Fn / Globe
XCTAssertNil(resolve(keyCode: 57, rawFlags: 0x1_0000)) // Caps Lock
}
}
#endif
+15 -3
View File
@@ -154,13 +154,21 @@ pub fn run() -> glib::ExitCode {
builder = builder.flags(gtk::gio::ApplicationFlags::NON_UNIQUE);
}
let app = builder.build();
app.connect_activate(build_ui);
// One SDL context for the whole process: `activate` fires again on every subsequent
// launch forwarded to this already-running singleton (another `--connect`, the desktop
// icon clicked twice, …). SDL only ever lets the FIRST thread that calls `sdl3::init()`
// hold the "main thread" — a second `GamepadService::start()` from a later `activate`
// would spawn a new thread that fails that check forever. Starting it once here and
// cloning it into each `build_ui` keeps the worker thread (and its pad state) shared
// across every window instead.
let gamepad = crate::gamepad::GamepadService::start();
app.connect_activate(move |gtk_app| build_ui(gtk_app, gamepad.clone()));
// GTK doesn't see our argv (`--connect` is handled in `build_ui`); an empty argv also
// keeps GApplication from rejecting unknown options.
app.run_with_args(&[] as &[&str])
}
fn build_ui(gtk_app: &adw::Application) {
fn build_ui(gtk_app: &adw::Application, gamepad: crate::gamepad::GamepadService) {
let identity = match crate::trust::load_or_create_identity() {
Ok(i) => i,
Err(e) => {
@@ -203,7 +211,7 @@ fn build_ui(gtk_app: &adw::Application) {
toasts,
settings: Rc::new(RefCell::new(Settings::load())),
identity,
gamepad: crate::gamepad::GamepadService::start(),
gamepad,
busy: std::cell::Cell::new(false),
fullscreen,
// (`--browse` makes cli_connect_request None — browse mode returns to the
@@ -242,6 +250,10 @@ fn build_ui(gtk_app: &adw::Application) {
let app = app.clone();
Rc::new(move |req| crate::ui_trust::initiate_connect(app.clone(), req))
},
on_wake_connect: {
let app = app.clone();
Rc::new(move |req| crate::ui_trust::wake_and_connect(app.clone(), req))
},
on_speed_test: {
let app = app.clone();
Rc::new(move |req| speed_test(app.clone(), req))
+20
View File
@@ -168,6 +168,26 @@ pub fn learn_mac(fp_hex: &str, addr: &str, port: u16, mac: &[String]) {
let _ = known.save();
}
/// Re-key a saved host's address/port after it rediscovered on a new DHCP lease (matched by
/// fingerprint). No-op — and no disk write — when unchanged. Called from the wake-and-wait flow when
/// a woken host reappears on a different IP than the stored one, so this and future connects dial the
/// live address instead of the stale one.
pub fn rekey_addr(fp_hex: &str, addr: &str, port: u16) {
if fp_hex.is_empty() {
return;
}
let mut known = KnownHosts::load();
let Some(h) = known.hosts.iter_mut().find(|h| h.fp_hex == fp_hex) else {
return;
};
if h.addr == addr && h.port == port {
return;
}
h.addr = addr.to_string();
h.port = port;
let _ = known.save();
}
/// Stamp "now" as this host's last successful connect (drives the hosts page's
/// most-recent accent). No-op when the fingerprint isn't stored.
pub fn touch_last_used(fp_hex: &str) {
+10 -5
View File
@@ -48,6 +48,9 @@ impl ConnectRequest {
/// the library browser).
pub struct HostsCallbacks {
pub on_connect: Rc<dyn Fn(ConnectRequest)>,
/// Connect to an OFFLINE saved host with a known MAC: wake it, poll until it's up (re-keying a
/// new DHCP IP), then connect. Falls back to `on_connect` when there's nothing to wake.
pub on_wake_connect: Rc<dyn Fn(ConnectRequest)>,
pub on_speed_test: Rc<dyn Fn(ConnectRequest)>,
pub on_pair: Rc<dyn Fn(ConnectRequest)>,
pub on_library: Rc<dyn Fn(ConnectRequest)>,
@@ -546,15 +549,17 @@ fn saved_card(
overlay.add_controller(right_click);
let on_connect = state.cbs.on_connect.clone();
// Auto-wake: if the host wasn't advertising when this card was built and we have a MAC, fire a
// magic packet before connecting — the connect's own retry/timeout gives a woken host time to
// come up. A host that's genuinely off/unreachable then fails the connect as before.
let on_wake_connect = state.cbs.on_wake_connect.clone();
// Auto-wake: if the host wasn't advertising when this card was built and we have a MAC, route to
// the wake-and-wait flow (send a magic packet, poll mDNS until it's up — re-keying a new DHCP IP —
// then connect). Otherwise a plain connect. A host that's genuinely off then times out as before.
let wake_first = !online && !req.mac.is_empty();
child.connect_activate(move |_| {
if wake_first {
crate::wol::wake(&req.mac, req.addr.parse().ok());
on_wake_connect(req.clone());
} else {
on_connect(req.clone());
}
on_connect(req.clone());
});
child
}
+5
View File
@@ -817,6 +817,9 @@ fn attach_keyboard(
// the capture toggle alone can't end a stream, so this is the keyboard's explicit exit.
if state.contains(chord) && keyval.to_lower() == gdk::Key::d {
cap.release();
// Deliberate user exit → close with QUIT_CLOSE_CODE so the host tears the session down
// immediately instead of holding the keep-alive linger for a reconnect.
cap.connector.disconnect_quit();
stop_kb.store(true, Ordering::SeqCst);
return glib::Propagation::Stop;
}
@@ -1024,6 +1027,8 @@ fn spawn_disconnect_watch(
glib::spawn_future_local(async move {
if disconnect_rx.recv().await.is_ok() {
cap.release();
// Deliberate user exit (the controller escape chord) → QUIT_CLOSE_CODE, host skips linger.
cap.connector.disconnect_quit();
if window.is_fullscreen() {
window.unfullscreen();
}
+81
View File
@@ -60,6 +60,87 @@ pub fn initiate_connect(app: Rc<App>, req: ConnectRequest) {
}
}
/// Wake-and-wait: an **offline** saved host with a known MAC is sent a magic packet, then we poll
/// mDNS until it comes back online — re-sending every few seconds up to a timeout — and dial it via
/// [`initiate_connect`], **re-keying the saved record if the host woke on a new DHCP IP** (matched by
/// fingerprint). A "Waking…" dialog lets the user cancel. Mirrors the Apple/Android `HostWaker` (a
/// 90 s budget, resend every 6 s). The online path stays on the fast [`initiate_connect`]; this runs
/// only from the hosts page's auto-wake when a saved host isn't advertising.
pub fn wake_and_connect(app: Rc<App>, req: ConnectRequest) {
if app.busy.get() {
return;
}
let cancel = Rc::new(std::cell::Cell::new(false));
let waiting = adw::AlertDialog::new(
Some("Waking Host"),
Some(&format!(
"Sent a wake signal to “{}”. Waiting for it to come online…",
req.name
)),
);
waiting.add_responses(&[("cancel", "Cancel")]);
waiting.set_close_response("cancel");
{
let cancel = cancel.clone();
waiting.connect_response(Some("cancel"), move |_, _| cancel.set(true));
}
waiting.present(Some(&app.window));
glib::spawn_future_local(async move {
use std::time::{Duration, Instant};
let events = crate::discovery::browse();
let started = Instant::now();
let budget = Duration::from_secs(90);
let resend = Duration::from_secs(6);
// Fire the first packet now, then re-send on the resend cadence.
crate::wol::wake(&req.mac, req.addr.parse().ok());
let mut last_wake = Instant::now();
loop {
if cancel.get() {
waiting.close();
return;
}
if last_wake.elapsed() >= resend {
crate::wol::wake(&req.mac, req.addr.parse().ok());
last_wake = Instant::now();
}
// Drain resolved adverts; a match (by fingerprint, else addr:port) means the host is up.
while let Ok(ev) = events.try_recv() {
let crate::discovery::DiscoveryEvent::Resolved(h) = ev else {
continue;
};
let matched = match &req.fp_hex {
Some(fp) => !h.fp_hex.is_empty() && &h.fp_hex == fp,
None => h.addr == req.addr && h.port == req.port,
};
if matched {
waiting.close();
let mut req = req.clone();
// Re-key on a new DHCP lease so this + future connects dial the live address.
if h.addr != req.addr || h.port != req.port {
if let Some(fp) = &req.fp_hex {
trust::rekey_addr(fp, &h.addr, h.port);
}
req.addr = h.addr;
req.port = h.port;
}
initiate_connect(app.clone(), req);
return;
}
}
if started.elapsed() >= budget {
waiting.close();
app.toast(&format!(
"Couldn't reach “{}” — is it powered and on the network?",
req.name
));
return;
}
glib::timeout_future(Duration::from_millis(500)).await;
}
});
}
/// The certificate fingerprint as grouped monospaced hex — 4-char groups over 2 lines
/// (the Apple TrustCardView format), far easier to compare against the host's log than
/// one 64-char run.
+1 -1
View File
@@ -1090,7 +1090,7 @@ async fn session(args: Args) -> Result<()> {
break;
}
if started.elapsed() > std::time::Duration::from_secs(cap_secs)
|| last_rx.elapsed() > std::time::Duration::from_secs(8)
|| last_rx.elapsed() > std::time::Duration::from_secs(45)
{
break;
}
+124
View File
@@ -5,6 +5,7 @@
use super::style::*;
use super::{AppCtx, Screen, Svc, Target};
use crate::discovery::DiscoveredHost;
use crate::session::{self, SessionEvent, SessionParams, Stats};
use crate::trust::{self, KnownHost, KnownHosts, Settings};
use crate::video::DecoderPref;
@@ -313,6 +314,97 @@ pub(crate) fn request_access(props: &Svc, target: &Target) {
);
}
/// The Wake-on-LAN "wait until up" flow (mirrors the Apple `HostWaker`): the tapped saved host is
/// offline but has a MAC, so send a magic packet, show a cancelable "Waking…" screen, and POLL mDNS
/// for the host to reappear — re-sending the packet periodically — on a bounded deadline. A cold box
/// takes far longer to POST/boot/re-advertise than a connect attempt will sit, so we can't just
/// fire-and-dial. On reappearance we dial it (re-keying the saved host when it came back on a new
/// IP); on timeout or Cancel we return to the host list.
pub(crate) fn wake_and_connect(
ctx: &Arc<AppCtx>,
target: Target,
set_screen: &AsyncSetState<Screen>,
set_status: &AsyncSetState<String>,
) {
// First packet now; the poll loop re-sends every RESEND_SECS (a single one can be missed, and
// some NICs only wake on a fresh packet after dropping into a deeper sleep state).
crate::wol::wake(&target.mac, target.addr.parse().ok());
// A fresh cancel flag per wake, installed where the "Waking…" screen's Cancel button reads it
// back (the same shared channel as the request-access flow); the poll loop checks the same `Arc`.
let cancel = Arc::new(AtomicBool::new(false));
*ctx.shared.cancel.lock().unwrap() = Some(cancel.clone());
// The busy page reads the host name from the shared target.
*ctx.shared.target.lock().unwrap() = target.clone();
set_status.call(String::new());
set_screen.call(Screen::Waking);
let (ctx, ss, st) = (ctx.clone(), set_screen.clone(), set_status.clone());
std::thread::spawn(move || {
// Generous — a cold boot + service start can be a minute-plus; re-send periodically.
const TIMEOUT_SECS: u64 = 90;
const RESEND_SECS: u64 = 6;
let rx = crate::discovery::browse();
let mut seen: Vec<DiscoveredHost> = Vec::new();
let mut elapsed: u64 = 0;
loop {
// Cancel already returned the UI to the host list — stop re-sending and tear down.
if cancel.load(Ordering::SeqCst) {
return;
}
// Drain freshly-resolved adverts into the accumulator (newest wins per key).
while let Ok(h) = rx.try_recv() {
if let Some(e) = seen.iter_mut().find(|e| e.key == h.key) {
*e = h;
} else {
seen.push(h);
}
}
// Match on the pinned fingerprint first (it survives an IP change), else last address.
let resolved = seen
.iter()
.find(|h| match &target.fp_hex {
Some(fp) if !h.fp_hex.is_empty() => h.fp_hex == *fp,
_ => h.addr == target.addr && h.port == target.port,
})
.map(|h| (h.addr.clone(), h.port));
if let Some((addr, port)) = resolved {
let mut target = target.clone();
// Came back on a new IP (DHCP): dial the fresh address and re-key the saved host so
// the pin stays reachable next time (keyed by fingerprint; addr/port overwritten,
// `paired`/`mac` preserved by `upsert`).
if addr != target.addr || port != target.port {
target.addr = addr;
target.port = port;
if let Some(fp) = target.fp_hex.clone() {
let mut k = KnownHosts::load();
k.upsert(KnownHost {
name: target.name.clone(),
addr: target.addr.clone(),
port: target.port,
fp_hex: fp,
paired: false,
mac: target.mac.clone(),
});
let _ = k.save();
}
}
initiate(&ctx, target, &ss, &st);
return;
}
if elapsed >= TIMEOUT_SECS {
st.call("The host didn't come online.".to_string());
ss.call(Screen::Hosts);
return;
}
std::thread::sleep(Duration::from_secs(1));
elapsed += 1;
if elapsed % RESEND_SECS == 0 {
crate::wol::wake(&target.mac, target.addr.parse().ok());
}
}
});
}
/// The plain "Connecting…" screen shown while the session worker handshakes. No hooks.
pub(crate) fn connecting_page(ctx: &Arc<AppCtx>, status: &str) -> Element {
let target_name = ctx.shared.target.lock().unwrap().name.clone();
@@ -365,3 +457,35 @@ pub(crate) fn request_access_page(
vec![cancel_btn.into()],
)
}
/// The cancelable "Waking…" screen (Wake-on-LAN wait-until-up flow): a spinner + guidance while the
/// poll loop waits for the woken host to reappear on mDNS, plus a Cancel that returns to the host
/// list and trips the shared cancel flag so the poll loop stops re-sending and tears down. No hooks.
pub(crate) fn waking_page(ctx: &Arc<AppCtx>, set_screen: &AsyncSetState<Screen>) -> Element {
let target_name = ctx.shared.target.lock().unwrap().name.clone();
let headline = if target_name.is_empty() {
"Waking the host\u{2026}".to_string()
} else {
format!("Waking {target_name}\u{2026}")
};
let cancel_btn = {
let (ctx, ss) = (ctx.clone(), set_screen.clone());
button("Cancel")
.icon(Symbol::Cancel)
.on_click(move || {
// Return the UI immediately and trip the flag the poll loop is watching so it stops
// re-sending and exits without touching a screen a later action may already own.
if let Some(c) = ctx.shared.cancel.lock().unwrap().as_ref() {
c.store(true, Ordering::SeqCst);
}
ss.call(Screen::Hosts);
})
.horizontal_alignment(HorizontalAlignment::Center)
};
busy_page(
&headline,
"Sent a wake signal and waiting for the host to come online \u{2014} this can take up to a \
minute for a sleeping or powered-off machine.",
vec![cancel_btn.into()],
)
}
+7 -5
View File
@@ -2,7 +2,7 @@
//! tiles in a responsive grid, with a per-host "…" menu (connect / speed test / rename /
//! forget) and a manual connect entry — the same card layout as the Linux and Apple clients.
use super::connect::initiate;
use super::connect::{initiate, wake_and_connect};
use super::speed::SpeedState;
use super::style::*;
use super::{Screen, Svc, Target};
@@ -386,12 +386,14 @@ pub(crate) fn hosts_page(props: &HostsProps, cx: &mut RenderCx) -> Element {
),
Some(menu),
Some(Box::new(move || {
// Auto-wake an offline saved host before connecting; the connect's own
// retry/timeout gives a woken host time to come up.
// Offline saved host with a known MAC: wake it and WAIT for it to reappear on
// the network (re-sending periodically) before dialing — a cold box boots far
// slower than a connect will sit. An online host dials straight away.
if can_wake {
crate::wol::wake(&target.mac, target.addr.parse().ok());
wake_and_connect(&ctx2, target.clone(), &ss, &st);
} else {
initiate(&ctx2, target.clone(), &ss, &st);
}
initiate(&ctx2, target.clone(), &ss, &st)
})),
));
}
+6 -2
View File
@@ -50,6 +50,9 @@ pub(crate) enum Screen {
/// The no-PIN "request access" wait: an identified connect is in flight, parked by the host
/// until the operator approves this device in its console. Cancelable.
RequestAccess,
/// Wake-on-LAN "wait until up": a magic packet was sent to an offline saved host and we're
/// polling mDNS for it to reappear (re-sending periodically) before dialing. Cancelable.
Waking,
Stream,
Settings,
/// Open-source / third-party license notices (reached from Settings).
@@ -378,10 +381,11 @@ fn root(cx: &mut RenderCx, ctx: &Arc<AppCtx>) -> Element {
set_hover,
},
),
// connecting_page / request_access_page / settings_page / licenses_page use no hooks
// (they never touch `cx`), so calling them inline is sound.
// connecting_page / request_access_page / waking_page / settings_page / licenses_page use
// no hooks (they never touch `cx`), so calling them inline is sound.
Screen::Connecting => connect::connecting_page(ctx, &status),
Screen::RequestAccess => connect::request_access_page(ctx, &set_screen),
Screen::Waking => connect::waking_page(ctx, &set_screen),
Screen::Settings => settings::settings_page(
ctx,
&set_screen,
+3
View File
@@ -281,6 +281,9 @@ unsafe extern "system" fn kbd_proc(code: i32, wparam: WPARAM, lparam: LPARAM) ->
// the cursor is free while the session winds down and the UI navigates home.
if !up && vk == VK_D.0 && st.ctrl && st.alt && st.shift {
set_captured(st, false);
// Deliberate user exit → close with QUIT_CLOSE_CODE so the host tears the session
// down immediately instead of holding the keep-alive linger for a reconnect.
st.connector.disconnect_quit();
st.stop.store(true, Ordering::SeqCst);
tracing::info!("disconnect requested (Ctrl+Alt+Shift+D)");
return LRESULT(1);
+16
View File
@@ -2432,6 +2432,22 @@ pub unsafe extern "C" fn punktfunk_connection_probe_result(
})
}
/// Signal a **deliberate quit** (a user "stop", not a network drop) before closing: the connection
/// closes with [`QUIT_CLOSE_CODE`] instead of code 0, so the host tears the session down immediately
/// (skips the keep-alive linger) rather than holding it for a reconnect. Call this right before
/// [`punktfunk_connection_close`] on a user-initiated disconnect; a plain close (network drop,
/// backgrounding) leaves the linger intact. NULL is a no-op.
///
/// # Safety
/// `c` was returned by [`punktfunk_connect`] and remains valid (closed via `punktfunk_connection_close`).
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_disconnect_quit(c: *mut PunktfunkConnection) {
if let Some(c) = unsafe { c.as_ref() } {
c.inner.disconnect_quit();
}
}
/// Close the connection and free the handle (joins the internal threads). NULL is a no-op.
///
/// # Safety
+11
View File
@@ -582,6 +582,17 @@ impl NativeClient {
self.frames_dropped.load(Ordering::Relaxed)
}
/// Whether the underlying QUIC session has ended — the worker's connection-close watcher set the
/// shutdown flag (`conn.closed()` fired: a host suspend / crash / network drop idle-timed the
/// connection out, or the host closed it), or a deliberate [`disconnect_quit`](Self::disconnect_quit)
/// / drop did. Once `true`, every `next_*` plane returns [`PunktfunkError::Closed`] and no more
/// frames will ever arrive. A client watchdog polls this so it can leave a frozen stream and
/// return to the menu (where the user can wake the host) instead of sitting on the last decoded
/// frame forever — the poll-friendly counterpart to reacting to a `Closed` in a plane loop.
pub fn is_session_ended(&self) -> bool {
self.shutdown.load(Ordering::SeqCst)
}
/// Register the calling thread as latency-critical so a later
/// [`hot_thread_ids`](Self::hot_thread_ids) includes it. An embedder calls this from its own
/// plane threads (e.g. the Android client's decode + audio threads) to fold them into the same
+8
View File
@@ -129,6 +129,14 @@ pub const VIDEO_CAP_HOST_TIMING: u8 = 0x08;
/// reconnect can resume. Shared so host + every client agree on the code.
pub const QUIT_CLOSE_CODE: u32 = 0x51;
/// QUIC application error code the **host** closes the control connection with when a **dedicated game
/// session's game process exits** (the nested gamescope died — the user quit the game), so a launcher
/// client can distinguish "the game ended" from an error and return to its library cleanly rather than
/// surfacing a failure (`design/gamemode-and-dedicated-sessions.md` §5.3). Sibling of
/// [`QUIT_CLOSE_CODE`]; a client that doesn't special-case it still ends the session (every client
/// returns to its launcher on session end), so it is purely refinement. Shared so host + clients agree.
pub const APP_EXITED_CLOSE_CODE: u32 = 0x52;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
/// advertise this.
+1 -1
View File
@@ -6,7 +6,7 @@ mod qos;
mod udp;
pub use loopback::{loopback_pair, LoopbackTransport};
pub use qos::{grow_socket_buffers, set_media_qos, MediaClass};
pub use qos::{grow_socket_buffers, set_dscp_default, set_media_qos, MediaClass};
/// Windows-only: reusable USO (UDP Send Offload) batch send for callers that own their own connected
/// socket (the GameStream video sender) rather than going through [`UdpTransport`].
#[cfg(target_os = "windows")]
+24 -6
View File
@@ -7,11 +7,13 @@
//! [`set_media_qos`] DSCP-tags the latency-sensitive video/audio traffic (+ Linux `SO_PRIORITY`) so a
//! QoS-aware path (Wi-Fi WMM access categories, a managed switch, a shaped uplink) can prioritize it
//! over bulk flows. Mirrors what Apollo/Sunshine tag — DSCP **CS5** for video, **CS6** for audio. It
//! is **opt-in** (`PUNKTFUNK_DSCP=1`): DSCP can interact badly with some consumer ISPs/routers, and on
//! is **opt-in** (`PUNKTFUNK_DSCP=1`, or [`set_dscp_default`] from an embedder — the Android client
//! ties it to its experimental low-latency mode): DSCP can interact badly with some consumer ISPs/routers, and on
//! Windows a plain `IP_TOS` is silently stripped unless a qWAVE policy is active (Apollo uses the
//! qWAVE API there — that port is a follow-up; today this is a no-op on the wire on Windows).
use std::net::UdpSocket;
use std::sync::atomic::{AtomicBool, Ordering};
/// Target kernel socket-buffer size (`SO_SNDBUF`/`SO_RCVBUF`). A high-resolution frame is a burst (a
/// 5120×1440 keyframe is ~130 packets the send thread hands to `sendmmsg` at once); the default UDP
@@ -66,12 +68,28 @@ impl MediaClass {
}
}
/// Whether DSCP/QoS marking is enabled (`PUNKTFUNK_DSCP=1`). Off by default.
/// Runtime default for DSCP marking when `PUNKTFUNK_DSCP` is unset (see [`set_dscp_default`]).
/// Off unless an embedder opts in — on Wi-Fi, access points commonly map DSCP to WMM access
/// categories (a real airtime-priority win), but wired paths rarely honour it and some bleach or
/// reject marked packets, so it never turns on by itself.
static DSCP_DEFAULT: AtomicBool = AtomicBool::new(false);
/// Opt in to (or back out of) DSCP marking for sockets created from now on. Must be called BEFORE
/// connecting — the tag is applied at socket creation. The Android client ties this to its
/// experimental low-latency mode; `PUNKTFUNK_DSCP` still overrides in either direction.
pub fn set_dscp_default(enabled: bool) {
DSCP_DEFAULT.store(enabled, Ordering::Relaxed);
}
/// Whether DSCP/QoS marking is enabled: `PUNKTFUNK_DSCP` when set (`1`/`true`/`on` forces it on,
/// `0`/`false`/`off` forces it off — e.g. to rule QoS out while debugging a flaky AP), else the
/// [`set_dscp_default`] runtime default.
pub(crate) fn dscp_enabled() -> bool {
matches!(
std::env::var("PUNKTFUNK_DSCP").as_deref(),
Ok("1") | Ok("true") | Ok("on")
)
match std::env::var("PUNKTFUNK_DSCP").as_deref() {
Ok("1") | Ok("true") | Ok("on") => true,
Ok("0") | Ok("false") | Ok("off") => false,
_ => DSCP_DEFAULT.load(Ordering::Relaxed),
}
}
/// Best-effort: tag `socket`'s outgoing packets for prioritized delivery of its media class. A no-op
+4 -3
View File
@@ -20,8 +20,9 @@ platform-facing around it.
- **Per-client virtual displays at the exact WxH@Hz.** Linux uses per-compositor backends — **KWin**,
**gamescope**, **Mutter**, and **Sway/wlroots**; Windows uses its own all-Rust IddCx virtual display,
even on the secure desktop (UAC / lock screen).
- **GPU zero-copy capture → encode.** dmabuf → CUDA/Vulkan → NVENC on Linux; DXGI/WGC → GPU encode on
Windows. Encoders auto-select by GPU vendor: **NVENC** (NVIDIA), **VAAPI** (Linux AMD/Intel),
- **GPU zero-copy capture → encode.** dmabuf → CUDA/Vulkan → NVENC on Linux; on Windows the host
pushes frames straight into its own IDD (sealed IDD-push, no screen-scraping) → GPU encode.
Encoders auto-select by GPU vendor: **NVENC** (NVIDIA), **VAAPI** (Linux AMD/Intel),
**AMF/QSV** (Windows AMD/Intel), or software H.264 as a floor. HDR/10-bit and HEVC 4:4:4 supported.
- **Input injection.** Mouse/keyboard (libei / gamescope EIS / wlr / Windows SendInput) and virtual
**gamepads** — Xbox 360/One, DualSense, DualShock 4 — with rumble and HID feedback back-channels.
@@ -70,7 +71,7 @@ src/
main.rs CLI + subcommand dispatch
config.rs · session_plan.rs · session_tuning.rs · pipeline.rs session setup + the frame pipeline
vdisplay/ per-compositor virtual outputs (kwin · gamescope · mutter · wlroots)
capture/ · capture.rs screen/dmabuf capture (+ Windows DXGI/WGC)
capture/ · capture.rs screen/dmabuf capture (+ Windows IDD-push)
encode/ · encode.rs per-GPU encoders (nvenc · vaapi · ffmpeg_win (AMF/QSV) · sw)
zerocopy/ dmabuf → CUDA → NVENC bridges (EGL/GL tiled, Vulkan LINEAR)
inject/ · inject.rs input backends (libei · wlr · uinput gamepads · UHID DualSense/DS4)
+165 -56
View File
@@ -43,6 +43,12 @@ pub struct PortalCapturer {
/// True only while the PipeWire stream is `Streaming`. [`try_latest`](Self::try_latest) reads it
/// to distinguish a static desktop (alive, no new buffers) from a dead source (left `Streaming`).
streaming: Arc<AtomicBool>,
/// Poison flag: the zero-copy GPU import is irrecoverably gone for this stream (the import
/// worker died — e.g. it absorbed the driver fault of a crashing compositor — or tiled imports
/// failed repeatedly, where the CPU fallback would de-pad scrambled tiled bytes). Both
/// [`next_frame`](Capturer::next_frame) and [`try_latest`](Self::try_latest) surface it as an
/// error so the session's capture-loss rebuild runs instead of freezing/corrupting.
broken: Arc<AtomicBool>,
/// When the stream first dropped out of `Streaming` with no new frame; used to grace a transient
/// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream
/// is `Streaming`.
@@ -130,6 +136,8 @@ struct PwHandles {
active: Arc<AtomicBool>,
negotiated: Arc<AtomicBool>,
streaming: Arc<AtomicBool>,
/// See [`PortalCapturer::broken`].
broken: Arc<AtomicBool>,
/// This capture will offer LINEAR-dmabuf-only for the VAAPI passthrough (see
/// [`PortalCapturer::vaapi_dmabuf`]).
vaapi_dmabuf: bool,
@@ -146,6 +154,7 @@ impl PwHandles {
active: self.active,
negotiated: self.negotiated,
streaming: self.streaming,
broken: self.broken,
stall_since: None,
vaapi_dmabuf: self.vaapi_dmabuf,
node_id,
@@ -178,6 +187,8 @@ fn spawn_pipewire(
let negotiated_cb = negotiated.clone();
let streaming = Arc::new(AtomicBool::new(false));
let streaming_cb = streaming.clone();
let broken = Arc::new(AtomicBool::new(false));
let broken_cb = broken.clone();
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope.
@@ -199,6 +210,7 @@ fn spawn_pipewire(
active_cb,
negotiated_cb,
streaming_cb,
broken_cb,
zerocopy,
preferred,
quit_rx,
@@ -212,6 +224,7 @@ fn spawn_pipewire(
active,
negotiated,
streaming,
broken,
vaapi_dmabuf,
quit: quit_tx,
join,
@@ -220,48 +233,36 @@ fn spawn_pipewire(
impl Capturer for PortalCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> {
// First frame can lag behind format negotiation; later frames arrive at ~fps.
match self.frames.recv_timeout(Duration::from_secs(10)) {
Ok(frame) => Ok(frame),
Err(RecvTimeoutError::Timeout) => {
// Split the two black-screen root causes apart so the operator gets a cause, not
// just a symptom: did the format negotiate (compositor produced no buffers) or
// not (no acceptable format / node never emitted a param)?
if self.negotiated.load(Ordering::Relaxed) {
Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiated but no buffers \
arrived — the compositor produced no frames (virtual output idle/unmapped, \
or capture never started)",
self.node_id
))
} else if self.vaapi_dmabuf && !crate::zerocopy::vaapi_dmabuf_forced() {
// The LINEAR-dmabuf-only offer (VAAPI passthrough default) was never accepted.
// Latch the process-wide downgrade so the encode loop's pipeline rebuild
// retries on the CPU offer instead of failing this same negotiation forever.
crate::zerocopy::note_vaapi_dmabuf_failed();
Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \
the LINEAR-dmabuf offer (VAAPI zero-copy) — downgrading this host to the \
CPU capture path; the pipeline rebuild will renegotiate without dmabuf",
self.node_id
))
} else {
Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiation never \
completed — the compositor offered no format this consumer accepts \
(pixel-format/modifier mismatch) or the node never emitted a Format param",
self.node_id
))
}
// First frame can lag behind format negotiation; later frames arrive at ~fps. Wait in
// short slices so a GPU-import poison (worker death) fails the capture within ~0.5 s
// instead of sitting out the full first-frame budget.
let deadline = std::time::Instant::now() + Duration::from_secs(10);
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly — rebuilding capture",
self.node_id
));
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e),
}
Err(RecvTimeoutError::Disconnected) => Err(anyhow!(
"PipeWire capture thread ended before a frame (node {})",
self.node_id
)),
}
}
fn try_latest(&mut self) -> Result<Option<CapturedFrame>> {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly — rebuilding capture",
self.node_id
));
}
// Drain to the newest queued frame without blocking; `None` means the compositor
// hasn't produced a new frame since last call (static/idle desktop).
let mut latest = None;
@@ -304,6 +305,50 @@ impl Capturer for PortalCapturer {
}
}
impl PortalCapturer {
/// The [`Capturer::next_frame`] budget expired (or the thread ended) — turn it into the
/// diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged.
fn next_frame_timed_out(&self, err: RecvTimeoutError) -> Result<CapturedFrame> {
match err {
RecvTimeoutError::Timeout => {
// Split the two black-screen root causes apart so the operator gets a cause, not
// just a symptom: did the format negotiate (compositor produced no buffers) or
// not (no acceptable format / node never emitted a param)?
if self.negotiated.load(Ordering::Relaxed) {
Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiated but no buffers \
arrived — the compositor produced no frames (virtual output idle/unmapped, \
or capture never started)",
self.node_id
))
} else if self.vaapi_dmabuf && !crate::zerocopy::vaapi_dmabuf_forced() {
// The LINEAR-dmabuf-only offer (VAAPI passthrough default) was never accepted.
// Latch the process-wide downgrade so the encode loop's pipeline rebuild
// retries on the CPU offer instead of failing this same negotiation forever.
crate::zerocopy::note_vaapi_dmabuf_failed();
Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \
the LINEAR-dmabuf offer (VAAPI zero-copy) — downgrading this host to the \
CPU capture path; the pipeline rebuild will renegotiate without dmabuf",
self.node_id
))
} else {
Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiation never \
completed — the compositor offered no format this consumer accepts \
(pixel-format/modifier mismatch) or the node never emitted a Format param",
self.node_id
))
}
}
RecvTimeoutError::Disconnected => Err(anyhow!(
"PipeWire capture thread ended before a frame (node {})",
self.node_id
)),
}
}
}
impl Drop for PortalCapturer {
fn drop(&mut self) {
// Stop the PipeWire loop and wait for the thread to unwind BEFORE the keepalive (virtual
@@ -548,8 +593,15 @@ mod pipewire {
/// `Paused`/`Unconnected`/`Error` — the source vanished (compositor torn down on a session
/// switch). Read by [`PortalCapturer::try_latest`] to surface a sustained drop as a loss.
streaming: Arc<AtomicBool>,
/// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer.
importer: Option<crate::zerocopy::EglImporter>,
/// Poison flag (see [`PortalCapturer::broken`]): set here when the GPU import is
/// irrecoverably gone for this stream — the import worker died, or tiled imports failed
/// [`IMPORT_FAIL_POISON`] times in a row.
broken: Arc<AtomicBool>,
/// Consecutive tiled-import failures (reset on success); see [`IMPORT_FAIL_POISON`].
import_fail_streak: u32,
/// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer,
/// normally via the isolated worker process (`crate::zerocopy::Importer::Remote`).
importer: Option<crate::zerocopy::Importer>,
/// VAAPI zero-copy: hand the raw dmabuf to the encoder (which imports + GPU-CSCs it) instead
/// of a CUDA import. Set when zero-copy is on, the EGL→CUDA importer is unavailable, and the
/// encoder backend is VAAPI (AMD/Intel).
@@ -561,6 +613,12 @@ mod pipewire {
dbg_log_n: u64,
}
/// Consecutive tiled-import failures (worker alive, e.g. a per-buffer `EGL_BAD_MATCH`) before
/// the stream is poisoned for rebuild. A tiled import failure must NEVER fall through to the
/// CPU mmap path — de-padding tiled bytes as linear produces a scrambled image — so after a
/// short streak of dropped frames the capturer fails loudly and the session renegotiates.
const IMPORT_FAIL_POISON: u32 = 3;
/// Log a frame-drop reason once per process (the process callback runs per frame; a stuck
/// pipeline must say why without flooding).
fn warn_once(msg: &'static str) {
@@ -814,6 +872,11 @@ mod pipewire {
if !ud.active.load(Ordering::Relaxed) {
return;
}
// Poisoned (GPU import lost): the capturer is already surfacing an error to the encode
// loop; skip per-frame work until the rebuild tears this stream down.
if ud.broken.load(Ordering::Relaxed) {
return;
}
// SAFETY: `spa_buf` is the `*mut spa_buffer` of the PipeWire buffer we dequeued and still hold for
// this `.process` callback (not requeued until after `consume_frame` returns), so it is live. The
// block null-checks `spa_buf`, requires `n_datas != 0`, and null-checks the `datas` array pointer
@@ -965,6 +1028,8 @@ mod pipewire {
};
match imported {
Ok(devbuf) => {
ud.import_fail_streak = 0;
crate::zerocopy::note_gpu_import_ok();
static ONCE: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(true);
if ONCE.swap(false, Ordering::Relaxed) {
@@ -990,12 +1055,32 @@ mod pipewire {
return;
}
Err(e) => {
// GPU import unavailable for this buffer kind (e.g. the
// driver rejects LINEAR external-memory import). Disable
// the importer and fall through to the CPU mmap path —
// degraded, not dead.
let dead = importer.dead();
if dead {
crate::zerocopy::note_gpu_import_death();
}
if modifier.is_some() {
// Tiled buffer: the CPU fallback below would mmap TILED bytes
// and de-pad them as linear — a scrambled image, worse than no
// frame. Drop the frame instead; on a dead worker (it absorbed a
// driver fault) or a short failure streak, poison the stream so
// the session's capture-loss rebuild renegotiates cleanly.
ud.import_fail_streak += 1;
if dead || ud.import_fail_streak >= IMPORT_FAIL_POISON {
tracing::error!(error = %format!("{e:#}"), dead,
"tiled GPU import lost — failing this capture for rebuild");
ud.broken.store(true, Ordering::Relaxed);
} else {
tracing::warn!(error = %format!("{e:#}"),
streak = ud.import_fail_streak,
"tiled dmabuf GPU import failed — frame dropped");
}
return;
}
// LINEAR dmabuf: CPU-mappable, so disable the importer and fall
// through to the CPU mmap path — degraded, not dead.
tracing::warn!(error = %format!("{e:#}"),
"dmabuf GPU import failed — falling back to the CPU copy path");
"LINEAR dmabuf GPU import failed — falling back to the CPU copy path");
gpu_import_broken = true;
}
}
@@ -1138,6 +1223,7 @@ mod pipewire {
active: Arc<AtomicBool>,
negotiated: Arc<AtomicBool>,
streaming: Arc<AtomicBool>,
broken: Arc<AtomicBool>,
zerocopy: bool,
preferred: Option<(u32, u32, u32)>,
quit_rx: pw::channel::Receiver<()>,
@@ -1165,26 +1251,40 @@ mod pipewire {
.context("pw connect (default daemon)")?,
};
// Build the EGL→CUDA importer up front; if it fails, log and fall back to the CPU path
// Build the GPU importer up front — normally the ISOLATED worker process
// (design/zerocopy-worker-isolation.md), so a driver fault on a dying compositor's
// dmabuf kills the worker, not this host. If it fails, log and fall back to the CPU path
// (we simply won't request dmabuf below). Skipped entirely when the encode backend is
// VAAPI: those frames go to the raw-dmabuf passthrough, and building the importer there
// would waste a CUDA probe — or worse, on an NVIDIA box forced to PUNKTFUNK_ENCODER=vaapi,
// succeed and produce CUDA payloads the VAAPI encoder must reject.
// succeed and produce CUDA payloads the VAAPI encoder must reject. Also skipped once
// repeated worker deaths latched the import off (a wedged GPU stack must not crash-loop).
let backend_is_vaapi = crate::encode::linux_zero_copy_is_vaapi();
let importer = if zerocopy && !backend_is_vaapi {
match crate::zerocopy::EglImporter::new() {
Ok(i) => Some(i),
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "zero-copy import unavailable — using CPU path");
None
let mut importer = if zerocopy && !backend_is_vaapi {
if crate::zerocopy::gpu_import_disabled() {
tracing::warn!(
"zero-copy GPU import disabled after repeated import-worker deaths — using CPU path"
);
None
} else {
match crate::zerocopy::Importer::new_for_capture() {
Ok(i) => Some(i),
Err(e) => {
tracing::warn!(error = %format!("{e:#}"), "zero-copy import unavailable — using CPU path");
None
}
}
}
} else {
None
};
// PUNKTFUNK_FORCE_SHM=1 forces the race-free download path (SHM, no dmabuf) — required on
// Mutter+NVIDIA where dmabuf capture has no working sync and shows stale frames. KWin/
// gamescope don't need it (they blit into the buffer, so no read-before-render race).
// PUNKTFUNK_FORCE_SHM=1 forces the race-free download path (SHM, no dmabuf) — a manual
// escape hatch, mainly for Mutter+NVIDIA: that combo has no implicit dmabuf fence, so
// zero-copy capture can in principle race the compositor's render and show stale frames.
// Zero-copy is the Mutter+NVIDIA default (no unconditional override) since live retesting
// found no visible staleness; set this if you do see flashing/stale content on such a
// host. KWin/gamescope don't need it (they blit into the buffer, so no read-before-render
// race).
let force_shm = std::env::var("PUNKTFUNK_FORCE_SHM").as_deref() == Ok("1");
// VAAPI zero-copy passthrough: zero-copy on, no EGL→CUDA importer (any non-NVIDIA host), and
// the encoder backend is VAAPI → hand the raw dmabuf to the encoder (it imports + GPU-CSCs).
@@ -1194,7 +1294,7 @@ mod pipewire {
// CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only:
// radeonsi/iHD import it and any compositor can allocate it.
let mut modifiers = importer
.as_ref()
.as_mut()
.map(|i| i.supported_modifiers(crate::zerocopy::drm_fourcc(PixelFormat::Bgrx).unwrap()))
.unwrap_or_default();
if (importer.is_some() || vaapi_passthrough) && !modifiers.contains(&0) {
@@ -1247,6 +1347,8 @@ mod pipewire {
active,
negotiated,
streaming,
broken,
import_fail_streak: 0,
importer,
vaapi_passthrough,
nv12: crate::zerocopy::nv12_enabled(),
@@ -1300,6 +1402,13 @@ mod pipewire {
}
if ud.info.parse(param).is_ok() {
ud.negotiated.store(true, Ordering::Relaxed);
// A (re)negotiation replaces the buffer pool: every cached per-buffer import
// (stored fds in the worker, the Vulkan bridge's per-fd sources) keys on
// buffers that no longer exist — and a recycled fd number/inode must never
// resolve to a stale import. No-op on the first negotiation (empty caches).
if let Some(imp) = ud.importer.as_mut() {
imp.clear_cache();
}
let sz = ud.info.size();
ud.format = map_format(ud.info.format());
ud.modifier = ud.info.modifier();
+15 -6
View File
@@ -25,9 +25,12 @@
//! - **Path / genuinely-dynamic reads**: the config-dir resolution, `PATH` executable search, the
//! env-forward-to-child loop, `PUNKTFUNK_MGMT_TOKEN`, `PUNKTFUNK_HOST_CMD`, `PUNKTFUNK_RENDER_NODE`.
//!
//! `PUNKTFUNK_ZEROCOPY` note: this field uses **presence** semantics (`var_os(..).is_some()`) to match the
//! Windows `encode/ffmpeg_win.rs` reader. The Linux `zerocopy` module keeps its own *truthy* parser
//! (`1|true|yes|on`) — the two are independent features that share a name; do NOT conflate them.
//! `PUNKTFUNK_ZEROCOPY` note: this field is a **tri-state override** (`None` = unset). Unset defers to
//! the per-vendor default in `encode/ffmpeg_win.rs::zerocopy_enabled` (AMF on — on-glass validated
//! 2026-07-06; QSV off until validated on Intel glass); an explicit value forces it (`0|false|off|no`
//! = off, anything else = on, so the old presence-style `=1` keeps working). The Linux `zerocopy`
//! module keeps its own *truthy* parser (`1|true|yes|on`) — the two are independent features that
//! share a name; do NOT conflate them.
use std::sync::OnceLock;
@@ -43,8 +46,9 @@ pub struct HostConfig {
pub render_adapter: Option<String>,
/// `PUNKTFUNK_IDD_DEPTH` — IDD-push pipeline depth override (default 2; the call site clamps to its `OUT_RING`).
pub idd_depth: usize,
/// `PUNKTFUNK_ZEROCOPY` — opt into the Windows D3D11 zero-copy encode path (presence semantics; see module docs).
pub zerocopy: bool,
/// `PUNKTFUNK_ZEROCOPY` — Windows D3D11 zero-copy encode input override. `None` (unset) defers to
/// the per-vendor default (AMF on, QSV off — see module docs and `encode/ffmpeg_win.rs`).
pub zerocopy: Option<bool>,
/// `PUNKTFUNK_10BIT` — host policy gate for HEVC Main10 (only honored when the client also advertised 10-bit).
pub ten_bit: bool,
/// `PUNKTFUNK_444` — host policy gate for full-chroma HEVC 4:4:4 (Range Extensions). Honored only
@@ -84,7 +88,12 @@ impl HostConfig {
idd_depth: val("PUNKTFUNK_IDD_DEPTH")
.and_then(|s| s.parse::<usize>().ok())
.unwrap_or(2),
zerocopy: flag("PUNKTFUNK_ZEROCOPY"),
zerocopy: val("PUNKTFUNK_ZEROCOPY").map(|s| {
!matches!(
s.trim().to_ascii_lowercase().as_str(),
"0" | "false" | "off" | "no"
)
}),
ten_bit: flag("PUNKTFUNK_10BIT"),
four_four_four: flag("PUNKTFUNK_444"),
perf: flag("PUNKTFUNK_PERF"),
+108 -44
View File
@@ -194,6 +194,15 @@ pub trait Encoder: Send {
}
/// Pull the next encoded AU if one is ready.
fn poll(&mut self) -> Result<Option<EncodedFrame>>;
/// Tear the underlying hardware encoder down and rebuild it in place, keeping the session's
/// negotiated parameters — the encode-stall watchdog's recovery lever (a wedged AMF/QSV
/// driver stops emitting AUs or accepting frames without ever returning an error). Returns
/// `true` when the encoder was rebuilt: every submitted-but-unpolled frame is forfeited and
/// the next submitted frame starts a fresh stream (IDR). Default `false`: the backend has no
/// in-place rebuild and the caller must treat the stall as fatal instead.
fn reset(&mut self) -> bool {
false
}
/// Signal end-of-stream. After this, drain the remaining AUs with [`poll`](Self::poll)
/// until it returns `None` — NVENC buffers frames internally even at `delay=0`.
fn flush(&mut self) -> Result<()>;
@@ -370,6 +379,9 @@ impl Encoder for TrackedEncoder {
fn poll(&mut self) -> Result<Option<EncodedFrame>> {
self.inner.poll()
}
fn reset(&mut self) -> bool {
self.inner.reset()
}
fn flush(&mut self) -> Result<()> {
self.inner.flush()
}
@@ -534,17 +546,40 @@ fn open_video_backend(
)
}
}
backend @ (WindowsBackend::Amf | WindowsBackend::Qsv) => {
// AMD AMF / Intel QSV via libavcodec (the Windows analogue of the Linux VAAPI path).
WindowsBackend::Amf => {
// AMD: the native AMF SDK encoder, unconditionally (design/native-amf-encoder.md
// Phase 3). The libavcodec AMF fallback and the `PUNKTFUNK_AMF_FFMPEG` hatch were
// removed once the native path was validated — two permanently-maintained AMF
// paths double the driver-matrix burden, and the one kept "for safety" is exactly
// the one with the wedge/latency pathology. No build feature: amfrt64.dll resolves
// at runtime like NVENC's DLL. A missing/ancient runtime fails HERE with the
// "install/update the AMD driver" message `AmfEncoder::open` raises (§6), rather
// than silently degrading — FFmpeg now serves QSV only.
amf::AmfEncoder::open(
codec,
format,
width,
height,
fps,
bitrate_bps,
bit_depth,
chroma,
)
.map(|e| Box::new(e) as Box<dyn Encoder>)
.map_err(|e| {
e.context(
"native AMF encode failed to open (update the AMD driver / amfrt64.dll \
runtime)",
)
})
}
WindowsBackend::Qsv => {
// Intel QSV via libavcodec (stays on FFmpeg — design/native-amf-encoder.md §2:
// async_depth=1 + low_power VDEnc is already near the hardware latency floor).
#[cfg(feature = "amf-qsv")]
{
let vendor = if matches!(backend, WindowsBackend::Amf) {
ffmpeg_win::WinVendor::Amf
} else {
ffmpeg_win::WinVendor::Qsv
};
ffmpeg_win::FfmpegWinEncoder::open(
vendor,
ffmpeg_win::WinVendor::Qsv,
codec,
format,
width,
@@ -558,11 +593,10 @@ fn open_video_backend(
}
#[cfg(not(feature = "amf-qsv"))]
{
let _ = backend;
anyhow::bail!(
"AMD/Intel (AMF/QSV) encode requested/detected but this host was built \
without it — rebuild with `--features amf-qsv` (needs ffmpeg-next + a \
FFMPEG_DIR with the AMF/QSV encoders at build time)"
"Intel (QSV) encode requested/detected but this host was built without \
it — rebuild with `--features amf-qsv` (needs ffmpeg-next + a FFMPEG_DIR \
with the QSV encoders at build time)"
)
}
}
@@ -773,14 +807,13 @@ pub fn can_encode_444(codec: Codec) -> bool {
false
}
}
WindowsBackend::Amf | WindowsBackend::Qsv => {
// AMD: native AMF never encodes 4:4:4 — VCN hardware limit, permanent, no probe
// needed (design/native-amf-encoder.md §3.5, Phase 3).
WindowsBackend::Amf => false,
WindowsBackend::Qsv => {
#[cfg(feature = "amf-qsv")]
{
let vendor = match windows_resolved_backend() {
WindowsBackend::Qsv => ffmpeg_win::WinVendor::Qsv,
_ => ffmpeg_win::WinVendor::Amf,
};
ffmpeg_win::probe_can_encode_444(vendor, codec)
ffmpeg_win::probe_can_encode_444(ffmpeg_win::WinVendor::Qsv, codec)
}
#[cfg(not(feature = "amf-qsv"))]
{
@@ -847,16 +880,18 @@ pub(crate) fn windows_resolved_backend() -> WindowsBackend {
}
}
/// True if the active Windows backend is the libavcodec AMF/QSV path (so the codec advertisement
/// consults a real GPU probe rather than the NVENC static superset). Always false when the
/// `amf-qsv` feature is off — there's then no ffmpeg backend to probe.
/// True if the active Windows backend's codec advertisement comes from a **real GPU probe**
/// ([`windows_codec_support`]) rather than the NVENC static superset. AMF always qualifies — the
/// native factory probe (`amf::probe_can_encode`) needs no build feature — while QSV still needs
/// the `amf-qsv` (libavcodec) build. Formerly `windows_backend_is_ffmpeg`, renamed when the
/// native AMF probe replaced the ffmpeg open-probe (design/native-amf-encoder.md §4, Phase 2).
#[cfg(target_os = "windows")]
pub fn windows_backend_is_ffmpeg() -> bool {
cfg!(feature = "amf-qsv")
&& matches!(
windows_resolved_backend(),
WindowsBackend::Amf | WindowsBackend::Qsv
)
pub fn windows_backend_is_probed() -> bool {
match windows_resolved_backend() {
WindowsBackend::Amf => true,
WindowsBackend::Qsv => cfg!(feature = "amf-qsv"),
WindowsBackend::Nvenc | WindowsBackend::Software => false,
}
}
/// Detect the encode-GPU vendor from the **selected render adapter** ([`crate::gpu::selected_gpu`]:
@@ -885,32 +920,55 @@ fn windows_gpu_vendor() -> Option<GpuVendor> {
})
}
/// Probe the active Windows AMF/QSV backend for its encodable codecs (opens a tiny encoder per
/// codec; cached **per (backend, selected GPU)** — a web-console preference change re-probes on the
/// newly selected adapter instead of serving the old GPU's answer for the process lifetime).
/// Mirrors [`vaapi_codec_support`]; called only when [`windows_backend_is_ffmpeg`] is true. AV1 is
/// narrow (AMD RDNA3+, Intel Arc/Xe2+), so it must be probed, not assumed.
#[cfg(all(target_os = "windows", feature = "amf-qsv"))]
/// Probe the active Windows AMF/QSV backend for its encodable codecs (cached **per (backend,
/// selected GPU)** — a web-console preference change re-probes on the newly selected adapter
/// instead of serving the old GPU's answer for the process lifetime). Mirrors
/// [`vaapi_codec_support`]; called only when [`windows_backend_is_probed`] is true. AV1 is narrow
/// (AMD RDNA3+, Intel Arc/Xe2+), so it must be probed, not assumed.
///
/// Mirrors the session dispatch (design/native-amf-encoder.md Phase 3): **AMD advertises from the
/// native AMF factory probe alone** (`amf::probe_can_encode`, on the selected adapter — the same
/// path the session opens, so the advertisement can never claim a codec the session can't emit);
/// **Intel/QSV uses the libavcodec probe** (all-`false` without the `amf-qsv` feature, matching a
/// build that cannot open QSV at all).
#[cfg(target_os = "windows")]
pub fn windows_codec_support() -> CodecSupport {
use std::collections::HashMap;
use std::sync::{Mutex, OnceLock};
static CACHE: OnceLock<Mutex<HashMap<String, CodecSupport>>> = OnceLock::new();
let vendor = match windows_resolved_backend() {
WindowsBackend::Qsv => ffmpeg_win::WinVendor::Qsv,
_ => ffmpeg_win::WinVendor::Amf,
};
let key = format!("{vendor:?}:{}", crate::gpu::selection_key());
let backend = windows_resolved_backend();
let key = format!("{backend:?}:{}", crate::gpu::selection_key());
let cache = CACHE.get_or_init(|| Mutex::new(HashMap::new()));
if let Some(c) = cache.lock().unwrap().get(&key) {
return *c;
}
let probe_one = |codec: Codec| -> bool {
match backend {
// AMD: the native factory probe is authoritative — it opens exactly the component the
// session will, so the advertisement matches what the encoder can emit by construction.
WindowsBackend::Amf => amf::probe_can_encode(codec),
WindowsBackend::Qsv => {
#[cfg(feature = "amf-qsv")]
{
ffmpeg_win::probe_can_encode(ffmpeg_win::WinVendor::Qsv, codec)
}
#[cfg(not(feature = "amf-qsv"))]
{
false
}
}
// Callers gate on `windows_backend_is_probed` — defensively answer "nothing probed"
// (the advertisement then falls back to the static superset).
WindowsBackend::Nvenc | WindowsBackend::Software => false,
}
};
let caps = CodecSupport {
h264: ffmpeg_win::probe_can_encode(vendor, Codec::H264),
h265: ffmpeg_win::probe_can_encode(vendor, Codec::H265),
av1: ffmpeg_win::probe_can_encode(vendor, Codec::Av1),
h264: probe_one(Codec::H264),
h265: probe_one(Codec::H265),
av1: probe_one(Codec::Av1),
};
tracing::info!(
backend = ?vendor,
?backend,
h264 = caps.h264,
h265 = caps.h265,
av1 = caps.av1,
@@ -921,8 +979,14 @@ pub fn windows_codec_support() -> CodecSupport {
caps
}
// Goal-1 stage 6: GPU/CPU encoders confined to `encode/windows/` (NVENC, AMF/QSV ffmpeg, software) and
// `encode/linux/` (NVENC/CUDA + VAAPI); `#[path]` keeps the `crate::encode::*` module names flat.
// Goal-1 stage 6: GPU/CPU encoders confined to `encode/windows/` (NVENC, native AMF, AMF/QSV
// ffmpeg, software) and `encode/linux/` (NVENC/CUDA + VAAPI); `#[path]` keeps the
// `crate::encode::*` module names flat.
// Native AMF (direct SDK, design/native-amf-encoder.md): compiled unconditionally on Windows —
// no build feature, the driver-installed amfrt64.dll resolves at runtime like NVENC's DLL.
#[cfg(target_os = "windows")]
#[path = "encode/windows/amf.rs"]
mod amf;
#[cfg(all(target_os = "windows", feature = "amf-qsv"))]
#[path = "encode/windows/ffmpeg_win.rs"]
mod ffmpeg_win;
File diff suppressed because it is too large Load Diff
@@ -1,28 +1,37 @@
//! AMD **AMF** and Intel **QSV** hardware encode on Windows via `ffmpeg-next` — the Windows
//! analogue of the Linux [`super::vaapi`] backend (one libavcodec backend per vendor, selected by
//! encoder name: `*_amf` / `*_qsv`). This is the sibling of the direct-SDK [`super::nvenc`] path
//! behind the shared [`Encoder`] trait, selected in [`super::open_video`] (NVIDIA → NVENC,
//! AMD → AMF, Intel → QSV).
//! Intel **QSV** (and, retained-but-no-longer-dispatched, AMD **AMF**) hardware encode on Windows
//! via `ffmpeg-next` — the Windows analogue of the Linux [`super::vaapi`] backend (one libavcodec
//! backend per vendor, selected by encoder name: `*_qsv` / `*_amf`). Sibling of the direct-SDK
//! [`super::nvenc`] path behind the shared [`Encoder`] trait.
//!
//! **Dispatch (design/native-amf-encoder.md Phase 3):** [`super::open_video`] routes AMD to the
//! direct-SDK [`super::amf`] encoder, not this module — the libavcodec AMF wrapper's ~2-frame
//! output hold and its silent-wedge failure mode are exactly why the native path exists. So in
//! production this file serves **QSV only**. The `WinVendor::Amf` machinery is kept (not deleted)
//! because it is the comparator in the native-vs-libavcodec latency A/B (`amf::tests::
//! amf_latency_ab_bench`), and excising it would churn the shared, Intel-unvalidated QSV code for
//! no production benefit. Treat every `WinVendor::Amf` arm below as benchmark-only.
//!
//! The capturer hands a `FramePayload::D3d11` texture (NV12/P010 from the D3D11 video processor, or
//! BGRA/Rgb10a2 as a fallback) on the capturer's own `ID3D11Device`. Two input paths, chosen lazily
//! from the first frame and the `PUNKTFUNK_ZEROCOPY` knob:
//!
//! * **System-memory** ([`SystemInner`], the default): read the captured D3D11 surface back to a CPU
//! * **System-memory** ([`SystemInner`]): read the captured D3D11 surface back to a CPU
//! NV12/P010 [`AVFrame`] (a same-format `CopyResource` → staging → `Map`, plus a `swscale` step for
//! the BGRA fallback) and `avcodec_send_frame` it. AMF/QSV upload it internally. One
//! GPU→CPU→GPU round-trip per frame — the robust path, and the only one that can be brought up
//! without on-glass validation (it is the analogue of the VAAPI "CPU input" fallback).
//! * **Zero-copy D3D11** ([`ZeroCopyInner`], `PUNKTFUNK_ZEROCOPY=1`): wrap the capturer's
//! `ID3D11Device` as an `AV_HWDEVICE_TYPE_D3D11VA` hwdevice (shared, *not* a second device — the
//! capture textures are not shared-handle, so a different device couldn't read them), keep an
//! FFmpeg D3D11 frames pool, `CopySubresourceRegion` the captured texture into a pooled array
//! slice (a GPU-local copy, like NVENC's CUDA path), then feed AMF `AV_PIX_FMT_D3D11` directly,
//! or map the D3D11 frame to a derived QSV surface for QSV. If the hw setup fails to open, this
//! falls back to the system-memory path for the session.
//! GPU→CPU→GPU round-trip per frame — the robust path, the QSV default, and the automatic
//! fallback when the zero-copy setup fails (it is the analogue of the VAAPI "CPU input" fallback).
//! * **Zero-copy D3D11** ([`ZeroCopyInner`], the AMF default; see [`zerocopy_enabled`]): wrap the
//! capturer's `ID3D11Device` as an `AV_HWDEVICE_TYPE_D3D11VA` hwdevice (shared, *not* a second
//! device — the capture textures are not shared-handle, so a different device couldn't read them),
//! keep an FFmpeg D3D11 frames pool, `CopySubresourceRegion` the captured texture into a pooled
//! array slice (a GPU-local copy, like NVENC's CUDA path), then feed AMF `AV_PIX_FMT_D3D11`
//! directly, or map the D3D11 frame to a derived QSV surface for QSV. If the hw setup fails to
//! open, this falls back to the system-memory path for the session.
//!
//! **Status: compiles in CI; not yet on-glass validated** (no AMD/Intel Windows box in the lab as of
//! 2026-06-22). The system path is the conservative default; zero-copy is opt-in until validated.
//! **Status:** AMF on-glass validated 2026-07-06 (Ryzen 7000 iGPU, 1080p120 HDR P010, both input
//! paths; zero-copy cut `submit_us` p50 2.8 ms → 0.26 ms) — zero-copy is the AMF default. QSV is
//! still not on-glass validated (no Intel Windows box in the lab), so its zero-copy path stays
//! opt-in via `PUNKTFUNK_ZEROCOPY=1`.
//!
//! Raw FFI: `ffmpeg-next` has no hwcontext wrappers for D3D11VA, so the hwdevice/hwframes calls go
//! through `ffmpeg::ffi` (= `ffmpeg_sys_next`), exactly as the Linux CUDA/VAAPI paths do. The
@@ -108,10 +117,16 @@ impl WinVendor {
}
}
/// Is the zero-copy D3D11 path enabled? Opt-in (`PUNKTFUNK_ZEROCOPY=1`) until on-glass validated;
/// the default is the robust system-memory readback path.
fn zerocopy_enabled() -> bool {
crate::config::config().zerocopy
/// Is the zero-copy D3D11 path enabled for this vendor? An explicit `PUNKTFUNK_ZEROCOPY`
/// (`0|false|off|no` = off, anything else = on) overrides; unset defers to the per-vendor default:
/// **on for AMF** — on-glass validated 2026-07-06 (Ryzen iGPU, 1080p120 HDR P010: `submit_us` p50
/// 2.8 ms → 0.26 ms vs readback) — and **off for QSV** until validated on Intel glass (the
/// open-failure fallback only catches *setup* errors; a derive that opens but maps wrong would
/// corrupt silently, so it stays opt-in per the probe-never-assume rule).
fn zerocopy_enabled(vendor: WinVendor) -> bool {
crate::config::config()
.zerocopy
.unwrap_or(matches!(vendor, WinVendor::Amf))
}
/// The swscale *source* pixel format for a captured packed-RGB/BGR layout (8-bit BGRA fallback only).
@@ -771,9 +786,9 @@ impl Drop for SystemInner {
}
// ---------------------------------------------------------------------------------------------
// Zero-copy D3D11 path (PUNKTFUNK_ZEROCOPY=1): share the capture device, pool D3D11 frames, copy
// the captured texture into a pooled slice, feed AMF directly / map to QSV. Falls back to the
// system path if the hw setup fails to open. Untested on glass — opt-in only for now.
// Zero-copy D3D11 path (the AMF default; QSV opt-in — see `zerocopy_enabled`): share the capture
// device, pool D3D11 frames, copy the captured texture into a pooled slice, feed AMF directly /
// map to QSV. Falls back to the system path if the hw setup fails to open.
// ---------------------------------------------------------------------------------------------
struct D3d11Hw {
@@ -1199,7 +1214,7 @@ impl FfmpegWinEncoder {
}
self.inner = None;
self.bound_device = dev_raw;
let inner = if zerocopy_enabled() {
let inner = if zerocopy_enabled(self.vendor) {
match ZeroCopyInner::open(
self.vendor,
self.codec,
@@ -1307,6 +1322,18 @@ impl Encoder for FfmpegWinEncoder {
self.force_kf = true;
}
/// Encode-stall recovery: drop the wedged libavcodec encoder (its `Drop` releases the AMF/QSV
/// runtime state) and let the next `submit` rebuild it lazily on the current device, exactly
/// like first-frame bring-up. The owed AUs are forfeited (`in_flight` zeroed) and the rebuilt
/// encoder's first frame is forced IDR so the client resyncs immediately.
fn reset(&mut self) -> bool {
self.inner = None;
self.bound_device = 0;
self.in_flight = 0;
self.force_kf = true;
true
}
/// Poll for the next finished AU (single non-blocking `receive_packet`).
///
/// libavcodec's `hevc_amf`/`av1_amf` wrapper holds ~2 frames before releasing the oldest
@@ -77,9 +77,10 @@ fn base_codec_mode_support() -> u32 {
}
}
// Windows AMD/Intel (AMF/QSV): advertise only what the GPU actually encodes (AV1 is narrow, an
// old iGPU might lack HEVC). NVENC and the GPU-less software path keep the static superset.
#[cfg(all(target_os = "windows", feature = "amf-qsv"))]
if crate::encode::windows_backend_is_ffmpeg() {
// old iGPU might lack HEVC). AMF probes natively (no build feature needed); QSV needs the
// libavcodec build. NVENC and the GPU-less software path keep the static superset.
#[cfg(target_os = "windows")]
if crate::encode::windows_backend_is_probed() {
if let Some(m) = probed_mask(crate::encode::windows_codec_support()) {
return m;
}
@@ -91,7 +92,7 @@ fn base_codec_mode_support() -> u32 {
/// or `None` if the probe found nothing — meaning the GPU wasn't usable at probe time (GPU-less CI,
/// a misconfigured/wrong-vendor host), NOT that it encodes zero codecs; the caller then advertises
/// the static superset (pre-probe behaviour) rather than claiming nothing.
#[cfg(any(target_os = "linux", all(target_os = "windows", feature = "amf-qsv")))]
#[cfg(any(target_os = "linux", target_os = "windows"))]
fn probed_mask(caps: crate::encode::CodecSupport) -> Option<u32> {
use super::{SCM_AV1_MAIN8, SCM_H264, SCM_HEVC};
let mut m = 0;
+25 -6
View File
@@ -246,14 +246,33 @@ fn open_gs_virtual_source(
}
#[cfg(not(target_os = "windows"))]
{
// A client is (re)connecting → cancel any pending TV-session restore (review #3).
crate::vdisplay::cancel_pending_tv_restore();
let active = crate::vdisplay::detect_active_session();
// A4: fold any compositor-instance change (idle-time Game↔Desktop switch) into the epoch
// before acquiring, so a GameStream reconnect never reuses a dead-instance node.
crate::vdisplay::observe_session_instance(&active);
crate::vdisplay::apply_session_env(&active);
let c = crate::vdisplay::compositor_for_kind(active.kind)
.map(Ok)
.unwrap_or_else(crate::vdisplay::detect)
.context("detect compositor")?;
crate::vdisplay::apply_input_env(c);
c
// Dedicated game session (B0): a GameStream app whose launch RESOLVES to a command (library
// id / apps.json command), under `game_session=dedicated` with gamescope available, gets its
// own headless gamescope spawn at the client mode — same routing as the native plane. Gate on
// the resolved command so an unresolvable entry falls back to auto routing (review #9).
let has_launch = crate::library::resolve_session_launch(
app.and_then(|a| a.library_id.as_deref()),
app.and_then(|a| a.cmd.as_deref()),
)
.is_some();
if crate::vdisplay::wants_dedicated_game_session(has_launch) {
crate::vdisplay::apply_input_env(crate::vdisplay::Compositor::Gamescope, true);
crate::vdisplay::Compositor::Gamescope
} else {
let c = crate::vdisplay::compositor_for_kind(active.kind)
.map(Ok)
.unwrap_or_else(crate::vdisplay::detect)
.context("detect compositor")?;
crate::vdisplay::apply_input_env(c, false);
c
}
}
};
let mut vd = crate::vdisplay::open(compositor).context("open virtual display")?;
@@ -0,0 +1,609 @@
//! Host side of the isolated zero-copy GPU import (design:
//! [`design/zerocopy-worker-isolation.md`]): spawns the `zerocopy-worker` subprocess, mirrors the
//! [`super::egl::EglImporter`] entry points over the [`super::proto`] socket, and materializes
//! the worker's pooled CUDA buffers in this process via CUDA IPC (each buffer's handles are
//! opened exactly once and reused as the pool recycles). A worker death — the whole point of the
//! isolation — surfaces as an `Err` with [`RemoteImporter::dead`] set, never as a host fault.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::cuda::{self, CUdeviceptr, DeviceBuffer, CU_IPC_HANDLE_SIZE};
use super::egl::DmabufPlane;
use super::proto::{self, BufferDesc, ImportKind, Reply, Request};
use anyhow::{bail, Context, Result};
use std::collections::{HashMap, HashSet};
use std::io;
use std::os::fd::{AsFd, AsRawFd, BorrowedFd, OwnedFd};
use std::path::Path;
use std::process::{Child, Command};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;
/// Handshake budget: EGL + CUDA bring-up is ~200 ms; a cold driver load can take seconds.
const HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(20);
/// Per-request budget. An import is a few ms of GPU work; if the worker can't answer in this
/// window it is wedged (GPU fault in progress) and gets treated as dead.
const REPLY_TIMEOUT: Duration = Duration::from_secs(10);
/// State shared with in-flight frames: the socket (their release messages) and the CUDA IPC
/// mappings (their device pointers). Lives until the LAST in-flight [`DeviceBuffer`] drops, so a
/// mapping is never closed under a frame the encoder still reads — and only then does the socket
/// close, which is what tells an idle worker to exit.
struct Shared {
sock: OwnedFd,
mappings: Mutex<HashMap<u32, Mapping>>,
dead: AtomicBool,
}
/// One pooled worker buffer, opened in this process.
#[derive(Clone, Copy)]
struct Mapping {
y: CUdeviceptr,
y_pitch: usize,
uv: Option<(CUdeviceptr, usize)>,
width: u32,
height: u32,
}
impl Drop for Shared {
fn drop(&mut self) {
// Last reference gone — no DeviceBuffer can still point into these mappings.
for (_, m) in self.mappings.lock().unwrap().drain() {
cuda::ipc_close(m.y);
if let Some((uv, _)) = m.uv {
cuda::ipc_close(uv);
}
}
}
}
/// Children whose worker hasn't exited yet at `RemoteImporter` drop time (it exits on socket
/// EOF, i.e. after the last in-flight frame drops). Swept on every spawn and every drop so
/// workers don't linger as zombies for more than one capture generation.
static REAPER: Mutex<Vec<Child>> = Mutex::new(Vec::new());
fn sweep_reaper() {
let mut list = REAPER.lock().unwrap();
list.retain_mut(|c| !matches!(c.try_wait(), Ok(Some(_))));
}
/// The remote (isolated) importer — one per capture. Method-for-method mirror of the in-process
/// [`super::egl::EglImporter`] surface the capture thread uses.
pub struct RemoteImporter {
shared: Arc<Shared>,
child: Option<Child>,
/// Reused receive scratch buffer (all replies are read by the single capture thread).
rbuf: Vec<u8>,
/// Dmabuf keys (`st_ino`) whose fd the worker already holds — the fd is passed only once.
sent_keys: HashSet<u64>,
}
impl RemoteImporter {
/// Spawn the worker from this host binary and complete the readiness handshake. An `Err`
/// here means "no isolated zero-copy available" — callers fall back to the CPU path, exactly
/// like an in-process `EglImporter::new()` failure.
pub fn spawn() -> Result<RemoteImporter> {
let exe = std::env::current_exe().context("resolve /proc/self/exe for the worker")?;
Self::spawn_exe(&exe)
}
/// [`Self::spawn`] with an explicit executable (separated for tests).
fn spawn_exe(exe: &Path) -> Result<RemoteImporter> {
sweep_reaper();
let (host_end, worker_end) = proto::socketpair_seqpacket().context("worker socketpair")?;
let mut cmd = Command::new(exe);
cmd.arg("zerocopy-worker").arg("--fd").arg("3");
let raw = worker_end.as_raw_fd();
// SAFETY: `pre_exec` runs between fork and exec, so only async-signal-safe calls are
// allowed — `dup2` and `fcntl` both are, and the closure captures only the `Copy` int
// `raw` (no allocation, no locks). `dup2(raw, 3)` installs the socket at the fd number
// the subcommand expects and clears CLOEXEC on the copy; if the parent's fd already IS 3,
// `dup2(3,3)` would preserve CLOEXEC, so that case clears the flag explicitly instead.
unsafe {
use std::os::unix::process::CommandExt;
cmd.pre_exec(move || {
if raw == 3 {
let flags = libc::fcntl(3, libc::F_GETFD);
if flags < 0 || libc::fcntl(3, libc::F_SETFD, flags & !libc::FD_CLOEXEC) < 0 {
return Err(io::Error::last_os_error());
}
} else if libc::dup2(raw, 3) < 0 {
return Err(io::Error::last_os_error());
}
Ok(())
});
}
let child = cmd.spawn().context("spawn zerocopy-worker")?;
drop(worker_end); // the child holds its own copy now
Self::from_socket(host_end, Some(child))
}
/// Complete the handshake on an already-connected socket (the unit tests drive this against
/// a mock server thread instead of a real subprocess).
fn from_socket(sock: OwnedFd, child: Option<Child>) -> Result<RemoteImporter> {
let mut importer = RemoteImporter {
shared: Arc::new(Shared {
sock,
mappings: Mutex::new(HashMap::new()),
dead: AtomicBool::new(false),
}),
child,
rbuf: Vec::new(),
sent_keys: HashSet::new(),
};
proto::set_recv_timeout(importer.shared.sock.as_fd(), Some(HANDSHAKE_TIMEOUT))?;
let ready = proto::recv::<Reply>(importer.shared.sock.as_fd(), &mut importer.rbuf);
proto::set_recv_timeout(importer.shared.sock.as_fd(), Some(REPLY_TIMEOUT))?;
match ready {
Ok((Reply::Ready { version }, _)) if version == proto::PROTO_VERSION => {
tracing::info!(
pid = importer.child.as_ref().map(|c| c.id()),
"zero-copy GPU import isolated in a worker process"
);
Ok(importer)
}
Ok((Reply::Ready { version }, _)) => {
importer.mark_dead();
bail!(
"zerocopy worker protocol mismatch (worker v{version}, host v{})",
proto::PROTO_VERSION
)
}
Ok((Reply::InitErr { message }, _)) => {
// The worker exits by itself after reporting; not a death, just "no GPU here".
bail!("zerocopy worker init failed: {message}")
}
Ok((other, _)) => {
importer.mark_dead();
bail!("unexpected zerocopy worker handshake: {other:?}")
}
Err(e) => {
importer.mark_dead();
Err(e).context("zerocopy worker handshake (died on startup?)")
}
}
}
/// True once any exchange failed at the transport level — the worker is gone (or wedged) and
/// every further call fails fast. The capture layer poisons its stream on this.
pub fn dead(&self) -> bool {
self.shared.dead.load(Ordering::Relaxed)
}
fn mark_dead(&self) {
self.shared.dead.store(true, Ordering::Relaxed);
}
/// Mirror of [`super::egl::EglImporter::supported_modifiers`] (worker round-trip; empty on
/// any failure, which makes the capture fall back like an importless negotiation).
pub fn supported_modifiers(&mut self, fourcc: u32) -> Vec<u64> {
if self.dead() {
return Vec::new();
}
if let Err(e) = proto::send(
self.shared.sock.as_fd(),
&Request::Modifiers { fourcc },
None,
) {
tracing::warn!(error = %e, "zerocopy worker modifier query failed");
self.mark_dead();
return Vec::new();
}
match proto::recv::<Reply>(self.shared.sock.as_fd(), &mut self.rbuf) {
Ok((Reply::Modifiers { modifiers }, _)) => modifiers,
Ok((other, _)) => {
tracing::warn!(?other, "unexpected zerocopy worker reply to Modifiers");
self.mark_dead();
Vec::new()
}
Err(e) => {
tracing::warn!(error = %e, "zerocopy worker modifier reply failed");
self.mark_dead();
Vec::new()
}
}
}
/// Mirror of [`super::egl::EglImporter::import`] (tiled dmabuf → BGRx CUDA buffer).
pub fn import(
&mut self,
plane: &DmabufPlane,
width: u32,
height: u32,
fourcc: u32,
modifier: Option<u64>,
) -> Result<DeviceBuffer> {
self.import_impl(plane, ImportKind::Tiled, width, height, fourcc, modifier)
}
/// Mirror of [`super::egl::EglImporter::import_nv12`].
pub fn import_nv12(
&mut self,
plane: &DmabufPlane,
width: u32,
height: u32,
fourcc: u32,
modifier: Option<u64>,
) -> Result<DeviceBuffer> {
self.import_impl(
plane,
ImportKind::TiledNv12,
width,
height,
fourcc,
modifier,
)
}
/// Mirror of [`super::egl::EglImporter::import_linear`] (LINEAR dmabuf → Vulkan bridge).
pub fn import_linear(
&mut self,
plane: &DmabufPlane,
width: u32,
height: u32,
) -> Result<DeviceBuffer> {
self.import_impl(plane, ImportKind::Linear, width, height, 0, None)
}
fn import_impl(
&mut self,
plane: &DmabufPlane,
kind: ImportKind,
width: u32,
height: u32,
fourcc: u32,
modifier: Option<u64>,
) -> Result<DeviceBuffer> {
if self.dead() {
bail!("zerocopy worker is dead");
}
let key = dmabuf_key(plane.fd)?;
// One retry: a `NeedFd` reply (the worker's fd cache evicted this key) clears our
// "already sent" note so the second attempt carries the fd again.
let mut attempts = 0;
let reply = loop {
attempts += 1;
let has_fd = self.sent_keys.insert(key);
// SAFETY: `plane.fd` is the dmabuf fd of the PipeWire buffer the capture thread still
// holds for this callback (`consume_frame`'s contract), so it is open and stays open
// for this synchronous call; the `BorrowedFd` never outlives it (used only for the
// `send`).
let pass = has_fd.then(|| unsafe { BorrowedFd::borrow_raw(plane.fd) });
let req = Request::Import {
key,
kind,
width,
height,
fourcc,
modifier,
offset: plane.offset,
stride: plane.stride,
has_fd,
};
if let Err(e) = proto::send(self.shared.sock.as_fd(), &req, pass) {
self.mark_dead();
return Err(e).context("zerocopy worker died (send)");
}
let reply = match proto::recv::<Reply>(self.shared.sock.as_fd(), &mut self.rbuf) {
Ok((reply, _)) => reply,
Err(e) => {
self.mark_dead();
return Err(e).context("zerocopy worker died (no reply)");
}
};
match reply {
Reply::NeedFd if attempts == 1 => {
self.sent_keys.remove(&key);
continue;
}
Reply::NeedFd => {
self.mark_dead();
bail!("zerocopy worker still lacks the fd after a resend (desync)");
}
other => break other,
}
};
match reply {
Reply::Frame { id, desc } => {
if let Some(desc) = desc {
let mapping = open_mapping(&desc).with_context(|| {
// An unopenable mapping poisons every future frame in this buffer —
// treat it as a dead worker so the capture rebuilds cleanly.
self.mark_dead();
format!("open CUDA IPC mapping for worker buffer {id}")
})?;
self.shared.mappings.lock().unwrap().insert(id, mapping);
}
let m = self
.shared
.mappings
.lock()
.unwrap()
.get(&id)
.copied()
.ok_or_else(|| {
self.mark_dead();
anyhow::anyhow!("worker delivered unknown buffer id {id} (desync)")
})?;
let shared = self.shared.clone();
Ok(DeviceBuffer::remote(
m.y,
m.y_pitch,
m.width,
m.height,
m.uv,
Box::new(move || {
// Fire-and-forget recycle; a dead worker just means EPIPE, ignored. The
// captured `shared` Arc is what keeps the mapping + socket alive until
// the last frame drops.
let _ = proto::send(shared.sock.as_fd(), &Request::Release { id }, None);
}),
))
}
Reply::Err { message } => bail!("zerocopy worker import failed: {message}"),
other => {
self.mark_dead();
bail!("unexpected zerocopy worker reply: {other:?}")
}
}
}
/// The PipeWire stream renegotiated — reset both sides' per-buffer caches.
pub fn clear_cache(&mut self) {
self.sent_keys.clear();
if !self.dead() {
if let Err(e) = proto::send(self.shared.sock.as_fd(), &Request::ClearCache, None) {
tracing::warn!(error = %e, "zerocopy worker ClearCache failed");
self.mark_dead();
}
}
}
}
impl Drop for RemoteImporter {
fn drop(&mut self) {
// The worker exits on socket EOF, which happens when the last `Shared` reference (this
// importer, or the final in-flight frame on the encode side) drops. Reap what's already
// gone; park the rest for the next sweep.
if let Some(mut child) = self.child.take() {
if !matches!(child.try_wait(), Ok(Some(_))) {
REAPER.lock().unwrap().push(child);
}
}
sweep_reaper();
}
}
/// Identity of the dma-buf behind `fd`, stable across frames and across `SCM_RIGHTS` re-numbering:
/// every dma-buf gets a unique inode on the kernel's dmabuf pseudo-fs for its lifetime. Used as
/// the worker's fd-cache key so the fd itself is only passed once.
fn dmabuf_key(fd: i32) -> Result<u64> {
// SAFETY: `libc::stat` is plain-old-data for which all-zero is a valid value, so
// `mem::zeroed()` is a sound initializer. `fd` is the caller's live dmabuf fd; `fstat` writes
// into `&mut st`, a live, correctly-sized stack struct that outlives the synchronous call,
// and `st_ino` is read only after the return value is checked.
unsafe {
let mut st: libc::stat = std::mem::zeroed();
if libc::fstat(fd, &mut st) != 0 {
bail!("fstat(dmabuf fd): {}", io::Error::last_os_error());
}
Ok(st.st_ino)
}
}
/// Open a worker buffer's CUDA IPC handles in this process.
fn open_mapping(desc: &BufferDesc) -> Result<Mapping> {
cuda::make_current()?;
let y_handle: [u8; CU_IPC_HANDLE_SIZE] = desc
.y_handle
.as_slice()
.try_into()
.context("worker sent a malformed Y IPC handle")?;
let y = cuda::ipc_open(&y_handle).context("open Y plane IPC handle")?;
let uv = match &desc.uv {
Some((handle, pitch)) => {
let handle: [u8; CU_IPC_HANDLE_SIZE] = handle
.as_slice()
.try_into()
.context("worker sent a malformed UV IPC handle")?;
match cuda::ipc_open(&handle) {
Ok(ptr) => Some((ptr, *pitch)),
Err(e) => {
// Don't leak the Y mapping on a half-open failure.
cuda::ipc_close(y);
return Err(e).context("open UV plane IPC handle");
}
}
}
None => None,
};
Ok(Mapping {
y,
y_pitch: desc.y_pitch,
uv,
width: desc.width,
height: desc.height,
})
}
#[cfg(test)]
mod tests {
use super::*;
use std::thread;
fn handshake_server(reply: Reply) -> OwnedFd {
let (host, worker) = proto::socketpair_seqpacket().unwrap();
proto::send(worker.as_fd(), &reply, None).unwrap();
// Keep the worker end alive alongside the host end for the test's duration by leaking it
// into the reply thread below? Not needed: the handshake reply is already queued in the
// socket buffer, so the worker end may drop — recv still delivers queued data first.
drop(worker);
host
}
#[test]
fn handshake_ready_and_version_gate() {
let host = handshake_server(Reply::Ready {
version: proto::PROTO_VERSION,
});
let imp = RemoteImporter::from_socket(host, None).unwrap();
assert!(!imp.dead());
let host = handshake_server(Reply::Ready { version: 999 });
assert!(RemoteImporter::from_socket(host, None).is_err());
}
#[test]
fn handshake_init_err() {
let host = handshake_server(Reply::InitErr {
message: "no GPU".into(),
});
let Err(err) = RemoteImporter::from_socket(host, None) else {
panic!("InitErr handshake must fail")
};
assert!(format!("{err:#}").contains("no GPU"), "{err:#}");
}
#[test]
fn handshake_eof_is_an_error() {
let (host, worker) = proto::socketpair_seqpacket().unwrap();
drop(worker);
assert!(RemoteImporter::from_socket(host, None).is_err());
}
#[test]
fn spawning_a_non_worker_fails_cleanly() {
// `true` exits immediately without a handshake → EOF → clean spawn error, the same
// fallback path a GPU-less box takes.
let Err(err) = RemoteImporter::spawn_exe(Path::new("true")) else {
panic!("spawning a non-worker must fail")
};
assert!(format!("{err:#}").contains("handshake"), "{err:#}");
}
/// A scripted peer: answers the handshake, then serves canned replies per request.
fn scripted_server(replies: Vec<Reply>) -> (RemoteImporter, thread::JoinHandle<Vec<Request>>) {
let (host, worker) = proto::socketpair_seqpacket().unwrap();
proto::send(
worker.as_fd(),
&Reply::Ready {
version: proto::PROTO_VERSION,
},
None,
)
.unwrap();
let join = thread::spawn(move || {
let mut buf = Vec::new();
let mut seen = Vec::new();
let mut replies = replies.into_iter();
while let Ok((req, _fd)) = proto::recv::<Request>(worker.as_fd(), &mut buf) {
let needs_reply = matches!(req, Request::Modifiers { .. } | Request::Import { .. });
seen.push(req);
if needs_reply {
match replies.next() {
Some(r) => proto::send(worker.as_fd(), &r, None).unwrap(),
None => break, // close → client sees a dead worker
}
}
}
seen
});
let imp = RemoteImporter::from_socket(host, None).unwrap();
(imp, join)
}
#[test]
fn modifiers_round_trip() {
let (mut imp, join) = scripted_server(vec![Reply::Modifiers {
modifiers: vec![1, 2, 3],
}]);
assert_eq!(imp.supported_modifiers(0x3432_5258), vec![1, 2, 3]);
assert!(!imp.dead());
drop(imp);
let seen = join.join().unwrap();
assert_eq!(
seen,
vec![Request::Modifiers {
fourcc: 0x3432_5258
}]
);
}
#[test]
fn need_fd_triggers_one_resend_with_the_fd() {
let (mut imp, join) = scripted_server(vec![
Reply::Err {
message: "one".into(),
},
Reply::NeedFd,
Reply::Err {
message: "two".into(),
},
]);
let (pr, _pw) = std::io::pipe().unwrap();
let plane = DmabufPlane {
fd: pr.as_fd().as_raw_fd(),
offset: 0,
stride: 256,
};
// First import: first sight of the key → fd rides along; the Err reply keeps the key
// marked as sent (the worker cached the fd before failing).
assert!(imp.import(&plane, 64, 64, 1, Some(2)).is_err());
// Second import: no fd (already sent) → worker answers NeedFd → one retry WITH the fd.
assert!(imp.import(&plane, 64, 64, 1, Some(2)).is_err());
assert!(!imp.dead(), "NeedFd handling must not mark the worker dead");
drop(imp);
let fd_flags: Vec<bool> = join
.join()
.unwrap()
.iter()
.map(|r| match r {
Request::Import { has_fd, .. } => *has_fd,
other => panic!("unexpected request {other:?}"),
})
.collect();
assert_eq!(fd_flags, vec![true, false, true]);
}
#[test]
fn import_error_reply_keeps_worker_alive_and_death_is_detected() {
let (mut imp, join) = scripted_server(vec![Reply::Err {
message: "EGL_BAD_MATCH".into(),
}]);
// Any pipe works as a stand-in fd for key derivation.
let (pr, _pw) = std::io::pipe().unwrap();
let plane = DmabufPlane {
fd: pr.as_fd().as_raw_fd(),
offset: 0,
stride: 256,
};
let Err(err) = imp.import(&plane, 64, 64, 1, Some(2)) else {
panic!("scripted Err reply must fail the import")
};
assert!(format!("{err:#}").contains("EGL_BAD_MATCH"));
assert!(!imp.dead(), "an Err reply must not mark the worker dead");
// The scripted replies are exhausted → the server closes → the next import dies.
let Err(err) = imp.import(&plane, 64, 64, 1, Some(2)) else {
panic!("a closed worker must fail the import")
};
assert!(format!("{err:#}").contains("died"), "{err:#}");
assert!(imp.dead());
drop(imp);
let seen = join.join().unwrap();
// First import carried the fd (first sight of the key); the retry didn't re-send it.
match (&seen[0], &seen[1]) {
(
Request::Import {
has_fd: true,
kind: ImportKind::Tiled,
..
},
Request::Import { has_fd: false, .. },
) => {}
other => panic!("unexpected requests {other:?}"),
}
}
}
+159 -11
View File
@@ -90,6 +90,21 @@ pub struct CUDA_EXTERNAL_MEMORY_BUFFER_DESC {
pub const CU_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD: c_uint = 1;
/// `CUipcMemHandle` (cuda.h): an opaque 64-byte struct identifying a device allocation across
/// processes. Produced by `cuIpcGetMemHandle` in the exporting process, consumed by
/// `cuIpcOpenMemHandle` in the importer — passed **by value**, matching the C
/// `struct { char reserved[64]; }`. Plain bytes — safe to ship over a socket.
pub const CU_IPC_HANDLE_SIZE: usize = 64;
#[repr(C)]
#[derive(Clone, Copy)]
pub struct CUipcMemHandle {
pub reserved: [u8; CU_IPC_HANDLE_SIZE],
}
/// `CUipcMem_flags`: lazily enable peer access on open (the documented flag for
/// `cuIpcOpenMemHandle`; a no-op for a same-device open, which is our only case).
const CU_IPC_MEM_LAZY_ENABLE_PEER_ACCESS: c_uint = 0x1;
/// CUDA Driver API entry points, resolved at runtime from `libcuda.so.1` via `dlopen` rather than
/// a link-time `#[link(name = "cuda")]`. This is what lets ONE host binary run on NVIDIA
/// (zero-copy via CUDA → NVENC) *and* on AMD/Intel (VAAPI, where the NVIDIA driver — and thus
@@ -129,6 +144,9 @@ struct CudaApi {
*const CUDA_EXTERNAL_MEMORY_BUFFER_DESC,
) -> CUresult,
cuDestroyExternalMemory: unsafe extern "C" fn(CUexternalMemory) -> CUresult,
cuIpcGetMemHandle: unsafe extern "C" fn(*mut CUipcMemHandle, CUdeviceptr) -> CUresult,
cuIpcOpenMemHandle: unsafe extern "C" fn(*mut CUdeviceptr, CUipcMemHandle, c_uint) -> CUresult,
cuIpcCloseMemHandle: unsafe extern "C" fn(CUdeviceptr) -> CUresult,
}
// SAFETY: every field is a bare `extern "C" fn` address into the leaked, process-lifetime
// `libcuda` mapping (`cuda_api` `forget`s the `Library`, so it is never unloaded) — an immutable
@@ -192,6 +210,14 @@ fn cuda_api() -> Option<&'static CudaApi> {
.get(b"cuExternalMemoryGetMappedBuffer\0")
.ok()?,
cuDestroyExternalMemory: *lib.get(b"cuDestroyExternalMemory\0").ok()?,
cuIpcGetMemHandle: *lib.get(b"cuIpcGetMemHandle\0").ok()?,
// CUDA 11 renamed the entry point (per-thread-stream ABI split); every modern
// driver exports `_v2`, but accept the unsuffixed one too (same signature).
cuIpcOpenMemHandle: *lib
.get(b"cuIpcOpenMemHandle_v2\0")
.or_else(|_| lib.get(b"cuIpcOpenMemHandle\0"))
.ok()?,
cuIpcCloseMemHandle: *lib.get(b"cuIpcCloseMemHandle\0").ok()?,
};
std::mem::forget(lib); // keep libcuda mapped for the fn pointers' lifetime (process)
Some(api)
@@ -346,6 +372,28 @@ unsafe fn cuDestroyExternalMemory(ext_mem: CUexternalMemory) -> CUresult {
None => CU_ERROR_NOT_LOADED,
}
}
unsafe fn cuIpcGetMemHandle(handle: *mut CUipcMemHandle, dptr: CUdeviceptr) -> CUresult {
match cuda_api() {
Some(a) => (a.cuIpcGetMemHandle)(handle, dptr),
None => CU_ERROR_NOT_LOADED,
}
}
unsafe fn cuIpcOpenMemHandle(
dptr: *mut CUdeviceptr,
handle: CUipcMemHandle,
flags: c_uint,
) -> CUresult {
match cuda_api() {
Some(a) => (a.cuIpcOpenMemHandle)(dptr, handle, flags),
None => CU_ERROR_NOT_LOADED,
}
}
unsafe fn cuIpcCloseMemHandle(dptr: CUdeviceptr) -> CUresult {
match cuda_api() {
Some(a) => (a.cuIpcCloseMemHandle)(dptr),
None => CU_ERROR_NOT_LOADED,
}
}
#[inline]
fn ck(r: CUresult, what: &str) -> Result<()> {
@@ -387,6 +435,55 @@ pub fn read_plane_to_host(
Ok(host)
}
/// Export a device allocation (from `cuMemAllocPitch`/`cuMemAlloc`) as a cross-process CUDA IPC
/// handle — an opaque 64-byte blob another process opens with [`ipc_open`]. The allocation must
/// stay alive for as long as any importer has it open. The shared context must be current.
pub fn ipc_export(ptr: CUdeviceptr) -> Result<[u8; CU_IPC_HANDLE_SIZE]> {
let mut handle = CUipcMemHandle {
reserved: [0; CU_IPC_HANDLE_SIZE],
};
// SAFETY: `&mut handle` is a live, correctly-sized stack out-param the driver fills with the
// opaque IPC blob; `ptr` is the caller's live device allocation (by-value integer). The call is
// synchronous and retains no pointer into Rust memory. Wrapper → live table (context current).
unsafe { ck(cuIpcGetMemHandle(&mut handle, ptr), "cuIpcGetMemHandle")? };
Ok(handle.reserved)
}
/// Open an IPC handle exported by *another* process ([`ipc_export`]); returns a device pointer
/// valid in this process until [`ipc_close`]. The shared context must be current.
pub fn ipc_open(handle: &[u8; CU_IPC_HANDLE_SIZE]) -> Result<CUdeviceptr> {
let h = CUipcMemHandle { reserved: *handle };
let mut ptr: CUdeviceptr = 0;
// SAFETY: `h` is passed by value (matching the C `CUipcMemHandle` struct ABI); `&mut ptr` is a
// live zero-init stack out-param the driver writes the mapped device address into. Synchronous
// call, distinct locals, no aliasing. Wrapper → live table (context current).
unsafe {
ck(
cuIpcOpenMemHandle(&mut ptr, h, CU_IPC_MEM_LAZY_ENABLE_PEER_ACCESS),
"cuIpcOpenMemHandle",
)?
};
Ok(ptr)
}
/// Close a mapping opened with [`ipc_open`] (best-effort teardown; makes the shared context
/// current itself since drops may run off-thread).
pub fn ipc_close(ptr: CUdeviceptr) {
if ptr == 0 {
return;
}
// SAFETY: `ptr` is a device pointer previously returned by `cuIpcOpenMemHandle` (the only
// caller path), closed exactly once by the owning cache. We make the shared context current
// first because this runs from `Drop` on whatever thread holds the last reference. Result
// ignored (best-effort teardown). Wrapper → live table (the mapping exists ⇒ driver present).
unsafe {
if let Some(c) = CONTEXT.get() {
let _ = cuCtxSetCurrent(c.0);
}
let _ = cuIpcCloseMemHandle(ptr);
}
}
/// The shared process-wide CUDA context (created once). Wrapped so it's `Send`/`Sync` to live
/// in a `OnceLock`; the raw `CUcontext` is thread-safe to make current from any thread.
#[derive(Clone, Copy)]
@@ -676,6 +773,7 @@ impl BufferPool {
height: self.height,
uv: Some((uv_ptr, uv_pitch)),
pool: Some(self.inner.clone()),
remote_release: None,
});
}
let reuse = self.inner.lock().unwrap().free.pop();
@@ -690,6 +788,7 @@ impl BufferPool {
height: self.height,
uv: None,
pool: Some(self.inner.clone()),
remote_release: None,
})
}
}
@@ -706,6 +805,10 @@ pub struct DeviceBuffer {
/// `None` for the default 4-byte RGB/BGRx path. When `Some`, [`ptr`] is the Y plane (1 byte/px).
pub uv: Option<(CUdeviceptr, usize)>,
pool: Option<Arc<Mutex<PoolInner>>>,
/// Set for buffers whose device memory is owned by ANOTHER process (the zero-copy import
/// worker, reached via CUDA IPC): drop runs this exactly once (telling the owner to recycle)
/// and must neither free nor pool-recycle the pointers locally.
remote_release: Option<Box<dyn FnOnce() + Send>>,
}
impl DeviceBuffer {
@@ -719,6 +822,7 @@ impl DeviceBuffer {
height,
uv: None,
pool: None,
remote_release: None,
})
}
@@ -733,6 +837,7 @@ impl DeviceBuffer {
height,
uv: Some((uv_ptr, uv_pitch)),
pool: None,
remote_release: None,
})
}
@@ -740,10 +845,38 @@ impl DeviceBuffer {
pub fn is_nv12(&self) -> bool {
self.uv.is_some()
}
/// Wrap device planes owned by ANOTHER process (opened here via [`ipc_open`]) as a frame
/// buffer. `release` runs exactly once on drop — it tells the owning process to recycle the
/// buffer; nothing is freed or pooled locally (the IPC mapping itself is closed by the cache
/// that opened it, after the last remote buffer referencing it has dropped).
pub fn remote(
ptr: CUdeviceptr,
pitch: usize,
width: u32,
height: u32,
uv: Option<(CUdeviceptr, usize)>,
release: Box<dyn FnOnce() + Send>,
) -> DeviceBuffer {
DeviceBuffer {
ptr,
pitch,
width,
height,
uv,
pool: None,
remote_release: Some(release),
}
}
}
impl Drop for DeviceBuffer {
fn drop(&mut self) {
if let Some(release) = self.remote_release.take() {
// Remote (IPC) buffer: the worker owns the memory — just hand it back.
release();
return;
}
if self.ptr == 0 {
return;
}
@@ -988,19 +1121,34 @@ pub fn copy_nv12_to_device(
}
}
impl RegisteredTexture {
/// Unregister now (idempotent; the later `Drop` then no-ops). Teardown-order helper: the blit
/// destructors call this to release the CUDA registration BEFORE deleting the GL texture it
/// wraps — deleting a still-registered texture leaves the driver holding a registration onto
/// freed GL state, exactly the stale-driver-state class this path once crashed on.
pub fn release(&mut self) {
if self.resource.is_null() {
return;
}
// SAFETY: `self.resource` is non-null (just checked) and is the valid `CUgraphicsResource`
// from `register_gl`, owned exclusively by this `RegisteredTexture`; nulling the field
// right after makes this (and the `Drop` below) unregister it exactly once — no
// use-after-free or double-unregister. We make the shared context current first because a
// release may run during teardown on a thread where it isn't. Wrapper → live table (the
// resource exists ⇒ the driver was present). Result ignored (best-effort teardown).
unsafe {
if let Some(c) = CONTEXT.get() {
let _ = cuCtxSetCurrent(c.0);
}
let _ = cuGraphicsUnregisterResource(self.resource);
}
self.resource = std::ptr::null_mut();
}
}
impl Drop for RegisteredTexture {
fn drop(&mut self) {
if !self.resource.is_null() {
// SAFETY: `self.resource` is non-null (just checked) and is the valid
// `CUgraphicsResource` from `register_gl`, owned exclusively by this `RegisteredTexture`
// and unregistered exactly once here (drop runs once) — no use-after-free or
// double-unregister. `cuGraphicsUnregisterResource` releases the GL↔CUDA registration;
// wrapper → live table (the resource exists ⇒ the driver was present). Result ignored
// (best-effort teardown).
unsafe {
let _ = cuGraphicsUnregisterResource(self.resource);
}
}
self.release();
}
}
@@ -270,6 +270,27 @@ impl GlBlit {
}
}
impl Drop for GlBlit {
fn drop(&mut self) {
// Unregister the CUDA graphics resource BEFORE deleting the GL texture it wraps (see
// `Nv12Blit::drop` — same ordering hazard). Previously `GlBlit` had no `Drop` at all, so
// its GL objects leaked on every size change and on importer teardown.
self.registered.release();
// SAFETY: these GL names were all created by THIS `GlBlit` in `GlBlit::new` on the current
// GL context, still current here (the owning `EglImporter` drops on its single capture
// thread and never releases the context). Each `glDelete*` gets a count of 1 and a `&u32`
// to one live field; the symbols dispatch through libGL to the driver for the current
// context. Each name is deleted exactly once, after its CUDA registration was released.
unsafe {
glDeleteTextures(1, &self.dst_tex);
glDeleteTextures(1, &self.src_tex);
glDeleteFramebuffers(1, &self.fbo);
glDeleteVertexArrays(1, &self.vao);
glDeleteProgram(self.program);
}
}
}
/// Per-size GL machinery to convert a dmabuf EGLImage into an NV12 (BT.709 limited-range) pair —
/// the [`GlBlit`] analogue for the `PUNKTFUNK_NV12` path. Two passes share `src_tex`: a full-res Y
/// pass into a CUDA-registrable `GL_R8` texture and a half-res UV pass into a `GL_RG8` texture.
@@ -417,6 +438,12 @@ impl Nv12Blit {
impl Drop for Nv12Blit {
fn drop(&mut self) {
// Unregister the CUDA graphics resources BEFORE deleting the GL textures they wrap.
// `Drop::drop` runs before the fields' own drops, so without this the `glDeleteTextures`
// below would destroy `y_tex`/`uv_tex` while still CUDA-registered — leaving the driver a
// registration onto freed GL state (the stale-driver-state class that crashed this path).
self.y_registered.release();
self.uv_registered.release();
// SAFETY: these GL names (textures/FBOs/VAO/programs) were all created by THIS `Nv12Blit`
// in `Nv12Blit::new` on the current GL context, which is still current because the owning
// `EglImporter` is dropped on its single capture thread (fields drop before
@@ -424,7 +451,8 @@ impl Drop for Nv12Blit {
// pointer to that many names: `&self.y_tex`/`&self.vao` are `&u32` to one live field (n=1);
// `[self.y_fbo, self.uv_fbo].as_ptr()` points at a 2-element temporary that lives for the
// whole `glDeleteFramebuffers` call (n=2 matches). The symbols dispatch through libGL
// (libglvnd) to the driver for the current context. Each name is deleted exactly once.
// (libglvnd) to the driver for the current context. Each name is deleted exactly once,
// after its CUDA registration was released above.
unsafe {
glDeleteTextures(1, &self.y_tex);
glDeleteTextures(1, &self.uv_tex);
@@ -637,6 +665,22 @@ impl EglImporter {
)
}
/// Drop the Vulkan bridge's cached per-fd import (see [`super::vulkan::VkBridge::forget_fd`]).
/// No-op when the bridge hasn't been built (tiled-only captures).
pub fn forget_linear_fd(&mut self, fd: i32) {
if let Some(vk) = self.vk.as_mut() {
vk.forget_fd(fd);
}
}
/// Tear down the whole LINEAR-path import cache (the Vulkan bridge and every per-fd source
/// buffer in it). Called when the PipeWire stream renegotiates — the buffer pool the cache
/// keyed on is gone, and a recycled fd number must never resolve to a stale import. The
/// bridge lazily rebuilds on the next LINEAR frame (renegotiations are rare).
pub fn clear_linear_cache(&mut self) {
self.vk = None;
}
/// The DRM format modifiers the NVIDIA EGL stack can import for `fourcc`, via
/// `eglQueryDmaBufModifiersEXT`. We advertise these to PipeWire so the compositor allocates
/// a dmabuf in a layout we can import. Empty on failure (caller falls back).
+157 -10
View File
@@ -10,11 +10,14 @@
//! headless EGLDisplay + dmabuf→`EGLImage`→CUDA import). The encoder's CUDA-frame path lives in
//! `encode/linux.rs`; the dmabuf negotiation lives in `capture/linux.rs`.
pub mod client;
pub mod cuda;
pub mod egl;
pub mod proto;
pub mod vulkan;
pub mod worker;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
pub use cuda::DeviceBuffer;
pub use egl::{DmabufPlane, EglImporter};
@@ -48,18 +51,21 @@ pub fn vaapi_dmabuf_forced() -> bool {
flag_opt("PUNKTFUNK_ZEROCOPY") == Some(true)
}
/// Whether the zero-copy path is on. `PUNKTFUNK_ZEROCOPY` decides when set (truthy = on, else
/// off). Unset defaults **on for the VAAPI (AMD/Intel) backend** — the stock AMD/Intel install
/// gets the GPU dmabuf path, not three full-frame CPU touches — unless a failed negotiation
/// downgraded it ([`note_vaapi_dmabuf_failed`]); and **off for NVENC**, whose EGL→CUDA import
/// stays opt-in (Mutter+NVIDIA has known dmabuf-capture races; see `PUNKTFUNK_FORCE_SHM`).
/// Whether the zero-copy path is on. `PUNKTFUNK_ZEROCOPY` decides when set (truthy = on, else off).
/// **Unset defaults ON for both GPU backends** — the stock install gets the GPU dmabuf path, not
/// three full-frame CPU touches. This includes NVENC (previously opt-in): the EGL→CUDA (tiled) and
/// Vulkan (LINEAR) imports now run in a per-capture worker subprocess
/// (`design/zerocopy-worker-isolation.md`), so a driver fault on a producer-invalidated dmabuf kills
/// the worker and the host degrades to its capture-loss rebuild instead of dying — the reason the
/// NVENC path stayed opt-in is gone. Fallbacks stay in place: VAAPI has a one-shot CPU downgrade if
/// the LINEAR-dmabuf offer never negotiates ([`note_vaapi_dmabuf_failed`]); NVENC falls back per
/// capture when no importer/importable modifier is available and latches the import off after
/// repeated worker deaths. `PUNKTFUNK_ZEROCOPY=0` opts out; `PUNKTFUNK_FORCE_SHM` forces the
/// race-free SHM path.
pub fn enabled() -> bool {
match flag_opt("PUNKTFUNK_ZEROCOPY") {
Some(v) => v,
None => {
crate::encode::linux_zero_copy_is_vaapi()
&& !VAAPI_DMABUF_FAILED.load(Ordering::Relaxed)
}
None => !VAAPI_DMABUF_FAILED.load(Ordering::Relaxed),
}
}
@@ -73,6 +79,127 @@ pub fn nv12_enabled() -> bool {
flag_opt("PUNKTFUNK_NV12").unwrap_or(true)
}
/// The GPU importer a capture uses — normally the [`client::RemoteImporter`] worker subprocess
/// (design: `design/zerocopy-worker-isolation.md`), so a driver fault on a producer-invalidated
/// dmabuf kills the worker instead of the host. `PUNKTFUNK_ZEROCOPY_INPROC=1` keeps the import
/// in-process (the pre-isolation behavior) for debugging and A/B latency comparison.
pub enum Importer {
Remote(client::RemoteImporter),
InProc(Box<EglImporter>),
}
impl Importer {
/// Build the importer for a capture session, honoring the `PUNKTFUNK_ZEROCOPY_INPROC`
/// escape hatch. An `Err` means "no GPU import available" — callers fall back to the CPU path.
pub fn new_for_capture() -> anyhow::Result<Importer> {
if flag("PUNKTFUNK_ZEROCOPY_INPROC") {
tracing::warn!(
"PUNKTFUNK_ZEROCOPY_INPROC=1 — GPU import runs IN-PROCESS; a driver fault on a \
dying compositor's dmabuf can take the whole host down (debug/A-B use only)"
);
return Ok(Importer::InProc(Box::new(EglImporter::new()?)));
}
Ok(Importer::Remote(client::RemoteImporter::spawn()?))
}
pub fn supported_modifiers(&mut self, fourcc: u32) -> Vec<u64> {
match self {
Importer::Remote(r) => r.supported_modifiers(fourcc),
Importer::InProc(i) => i.supported_modifiers(fourcc),
}
}
pub fn import(
&mut self,
plane: &DmabufPlane,
width: u32,
height: u32,
fourcc: u32,
modifier: Option<u64>,
) -> anyhow::Result<DeviceBuffer> {
match self {
Importer::Remote(r) => r.import(plane, width, height, fourcc, modifier),
Importer::InProc(i) => i.import(plane, width, height, fourcc, modifier),
}
}
pub fn import_nv12(
&mut self,
plane: &DmabufPlane,
width: u32,
height: u32,
fourcc: u32,
modifier: Option<u64>,
) -> anyhow::Result<DeviceBuffer> {
match self {
Importer::Remote(r) => r.import_nv12(plane, width, height, fourcc, modifier),
Importer::InProc(i) => i.import_nv12(plane, width, height, fourcc, modifier),
}
}
pub fn import_linear(
&mut self,
plane: &DmabufPlane,
width: u32,
height: u32,
) -> anyhow::Result<DeviceBuffer> {
match self {
Importer::Remote(r) => r.import_linear(plane, width, height),
Importer::InProc(i) => i.import_linear(plane, width, height),
}
}
/// True once the worker process is gone/wedged (every further call fails fast). Always
/// `false` in-process — an in-process driver fault doesn't return.
pub fn dead(&self) -> bool {
match self {
Importer::Remote(r) => r.dead(),
Importer::InProc(_) => false,
}
}
/// The PipeWire stream renegotiated its format (the buffer pool is replaced) — drop all
/// per-buffer caches so a recycled fd number can never resolve to a stale import.
pub fn clear_cache(&mut self) {
match self {
Importer::Remote(r) => r.clear_cache(),
Importer::InProc(i) => i.clear_linear_cache(),
}
}
}
/// Consecutive zero-copy worker deaths without a successful import in between. A short streak is
/// normal (the observed trigger — a compositor crash — kills the worker once, and the rebuilt
/// session's fresh worker succeeds); a sustained streak means the GPU stack itself is wedged and
/// respawning would crash-loop, so [`note_gpu_import_death`] latches [`GPU_IMPORT_DISABLED`] and
/// every later capture negotiates the safe CPU/SHM path instead.
static GPU_IMPORT_DEATH_STREAK: AtomicU32 = AtomicU32::new(0);
static GPU_IMPORT_DISABLED: AtomicBool = AtomicBool::new(false);
const GPU_IMPORT_DEATH_LATCH: u32 = 3;
/// Record a worker death (transport-level failure). Latches the process-wide disable after
/// [`GPU_IMPORT_DEATH_LATCH`] consecutive deaths.
pub fn note_gpu_import_death() {
let streak = GPU_IMPORT_DEATH_STREAK.fetch_add(1, Ordering::Relaxed) + 1;
if streak >= GPU_IMPORT_DEATH_LATCH && !GPU_IMPORT_DISABLED.swap(true, Ordering::Relaxed) {
tracing::error!(
streak,
"zero-copy GPU import disabled for this host process: the import worker died {streak} \
times in a row (GPU/driver stack unstable) captures fall back to the CPU path"
);
}
}
/// Record a successful GPU import — resets the death streak (the stack works again).
pub fn note_gpu_import_ok() {
GPU_IMPORT_DEATH_STREAK.store(0, Ordering::Relaxed);
}
/// True once repeated worker deaths latched the GPU import off (see [`note_gpu_import_death`]).
pub fn gpu_import_disabled() -> bool {
GPU_IMPORT_DISABLED.load(Ordering::Relaxed)
}
/// DRM FourCC for a packed 32-bit format name (little-endian, e.g. `b"XR24"`).
const fn fourcc(c: &[u8; 4]) -> u32 {
(c[0] as u32) | ((c[1] as u32) << 8) | ((c[2] as u32) << 16) | ((c[3] as u32) << 24)
@@ -250,3 +377,23 @@ pub fn nv12_selftest() -> anyhow::Result<()> {
bail!("NV12 self-test FAILED (Y={max_y_err:.2} U={max_u_err:.2} V={max_v_err:.2})");
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Single test owning the process-global latch statics (they are never reset by design).
#[test]
fn gpu_import_death_latch() {
note_gpu_import_death();
note_gpu_import_ok(); // a successful import resets the streak
note_gpu_import_death();
note_gpu_import_death();
assert!(
!gpu_import_disabled(),
"two consecutive deaths must not latch"
);
note_gpu_import_death(); // third consecutive death
assert!(gpu_import_disabled());
}
}
@@ -0,0 +1,390 @@
//! Wire protocol between the PipeWire capture thread and the isolated zero-copy GPU-import
//! worker process (`punktfunk-host zerocopy-worker`; design:
//! [`design/zerocopy-worker-isolation.md`]). Transport is a `SOCK_SEQPACKET` unix socketpair —
//! reliable, ordered, message-framed (one `sendmsg` = one message) — with dmabuf fds riding as
//! `SCM_RIGHTS` control data. Bodies are small serde_json blobs (~200 B/frame); pixels never
//! cross the socket (they move GPU-side via CUDA IPC, see [`super::cuda::ipc_export`]).
//!
//! Zero-length messages are reserved: `recvmsg` returning 0 on a SEQPACKET socket is EOF (the
//! peer died/closed), and every serialized message here is non-empty JSON, so the two can't be
//! confused.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize};
use std::io;
use std::os::fd::{AsRawFd, BorrowedFd, FromRawFd, OwnedFd};
use std::time::Duration;
/// Bumped on any wire change; the worker echoes it in [`Reply::Ready`] and the host refuses a
/// mismatch. Host and worker are the same binary (`/proc/self/exe`), so this only ever trips on
/// exotic deployment mistakes (a stale binary re-exec'd across an upgrade).
pub const PROTO_VERSION: u32 = 1;
/// Upper bound for one serialized message (the largest real message — a modifier list — is far
/// below this). A message reported truncated at this size is a protocol error.
pub const MAX_MSG: usize = 64 * 1024;
/// How a dmabuf should be imported — mirrors the three `EglImporter` entry points.
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq)]
pub enum ImportKind {
/// Tiled dmabuf → EGL/GL de-tile blit → BGRx CUDA buffer.
Tiled,
/// Tiled dmabuf → EGL/GL NV12 convert → two-plane CUDA buffer (`PUNKTFUNK_NV12`).
TiledNv12,
/// LINEAR dmabuf → Vulkan bridge → BGRx CUDA buffer (gamescope's only offer).
Linear,
}
/// host → worker.
#[derive(Serialize, Deserialize, Debug, PartialEq)]
pub enum Request {
/// The EGL-importable DRM modifiers for `fourcc` (startup, before the stream connects —
/// the host advertises these to PipeWire).
Modifiers { fourcc: u32 },
/// Import one frame. `key` identifies the underlying dmabuf across frames (the host uses
/// the fd's `st_ino` — unique per dma-buf object); the fd itself rides along as
/// `SCM_RIGHTS` only on first sight of `key` (`has_fd`), and the worker keeps its dup.
Import {
key: u64,
kind: ImportKind,
width: u32,
height: u32,
fourcc: u32,
modifier: Option<u64>,
offset: u32,
stride: u32,
has_fd: bool,
},
/// The frame buffer previously delivered as `id` is no longer in use — recycle it into the
/// worker's pool. Fire-and-forget (no reply); may be sent from any host thread.
Release { id: u32 },
/// The PipeWire stream renegotiated its format: the buffer pool is gone, so drop all cached
/// per-`key` state (stored fds, Vulkan per-fd imports). Fire-and-forget.
ClearCache,
}
/// worker → host.
#[derive(Serialize, Deserialize, Debug, PartialEq)]
pub enum Reply {
/// Sent once at startup after EGL + CUDA came up.
Ready {
version: u32,
},
/// Startup failed (no NVIDIA driver, EGL error, …) — the host falls back to the CPU path,
/// exactly like an in-process `EglImporter::new()` failure.
InitErr {
message: String,
},
Modifiers {
modifiers: Vec<u64>,
},
/// The imported frame is complete (the GPU copy already synced worker-side) in buffer `id`.
/// `desc` rides along the first time `id` is ever delivered — the host opens its CUDA IPC
/// handles once and caches the mapping for every later frame in the same buffer.
Frame {
id: u32,
desc: Option<BufferDesc>,
},
/// The worker has no cached fd for the import's `key` (evicted, or the two sides' caches
/// diverged) — the host forgets its "already sent" note and retries once WITH the fd.
NeedFd,
/// This import failed but the worker is alive (e.g. `EGL_BAD_MATCH` on one buffer).
Err {
message: String,
},
}
/// CUDA IPC identity of one pooled device buffer (sent once per buffer, then referenced by id).
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
pub struct BufferDesc {
pub width: u32,
pub height: u32,
/// `cuIpcGetMemHandle` blob for the (Y or BGRx) plane — exactly 64 bytes.
pub y_handle: Vec<u8>,
pub y_pitch: usize,
/// NV12 only: the interleaved chroma plane's `(handle, pitch)`.
pub uv: Option<(Vec<u8>, usize)>,
}
/// A CLOEXEC `SOCK_SEQPACKET` socketpair — `(host_end, worker_end)`.
pub fn socketpair_seqpacket() -> io::Result<(OwnedFd, OwnedFd)> {
let mut fds = [0i32; 2];
// SAFETY: `socketpair` writes two fds into `fds`, a live 2-element stack array matching the
// API contract; it reads no other Rust memory. The result is checked before the fds are used,
// and each returned fd is fresh (owned by no other wrapper), so the two `OwnedFd::from_raw_fd`
// each take sole ownership of a distinct, valid descriptor — no alias, no double-close.
unsafe {
if libc::socketpair(
libc::AF_UNIX,
libc::SOCK_SEQPACKET | libc::SOCK_CLOEXEC,
0,
fds.as_mut_ptr(),
) != 0
{
return Err(io::Error::last_os_error());
}
Ok((OwnedFd::from_raw_fd(fds[0]), OwnedFd::from_raw_fd(fds[1])))
}
}
/// Set (or clear) the receive timeout: a blocked [`recv`] then fails with
/// `ErrorKind::WouldBlock`. Used by the host so a hung worker can't wedge the capture thread.
pub fn set_recv_timeout(sock: BorrowedFd, timeout: Option<Duration>) -> io::Result<()> {
let tv = match timeout {
Some(d) => libc::timeval {
tv_sec: d.as_secs() as libc::time_t,
tv_usec: d.subsec_micros() as libc::suseconds_t,
},
None => libc::timeval {
tv_sec: 0,
tv_usec: 0,
},
};
// SAFETY: `setsockopt(SO_RCVTIMEO)` reads `size_of::<timeval>()` bytes from `&tv`, a live
// stack `timeval` that outlives this synchronous call; `sock` is the caller's live socket fd.
// Nothing is retained or written through Rust pointers.
let r = unsafe {
libc::setsockopt(
sock.as_raw_fd(),
libc::SOL_SOCKET,
libc::SO_RCVTIMEO,
&tv as *const libc::timeval as *const libc::c_void,
std::mem::size_of::<libc::timeval>() as libc::socklen_t,
)
};
if r != 0 {
return Err(io::Error::last_os_error());
}
Ok(())
}
/// Send one message (+ optionally one fd as `SCM_RIGHTS`) as a single SEQPACKET datagram.
/// Atomic per message, so concurrent senders on the same socket (the capture thread's imports,
/// the encode thread's releases) need no lock. `MSG_NOSIGNAL` turns a dead peer into `EPIPE`
/// instead of `SIGPIPE`.
pub fn send<T: Serialize>(
sock: BorrowedFd,
msg: &T,
pass_fd: Option<BorrowedFd>,
) -> io::Result<()> {
let body =
serde_json::to_vec(msg).map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e))?;
debug_assert!(
!body.is_empty(),
"zero-length messages are reserved for EOF"
);
if body.len() > MAX_MSG {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
"zerocopy proto message too large",
));
}
let mut iov = libc::iovec {
iov_base: body.as_ptr() as *mut libc::c_void,
iov_len: body.len(),
};
// Control buffer for one fd: CMSG_SPACE(4) = 24 bytes on 64-bit; [u64; 4] gives 32 bytes at
// the 8-byte alignment `cmsghdr` requires.
let mut cmsg_store = [0u64; 4];
// SAFETY: `mhdr` is a plain-old-data C struct for which all-zero is a valid value.
let mut mhdr: libc::msghdr = unsafe { std::mem::zeroed() };
mhdr.msg_iov = &mut iov;
mhdr.msg_iovlen = 1;
if let Some(fd) = pass_fd {
mhdr.msg_control = cmsg_store.as_mut_ptr() as *mut libc::c_void;
// SAFETY: `CMSG_SPACE`/`CMSG_LEN` are pure size computations (no memory access).
// `CMSG_FIRSTHDR(&mhdr)` returns a pointer into `cmsg_store` (non-null: msg_controllen
// ≥ one cmsghdr), which is live, 8-aligned, and large enough (32 ≥ CMSG_SPACE(4) = 24)
// for the header fields and the 4-byte fd written via `CMSG_DATA`; `write_unaligned`
// handles the data area's byte alignment. All writes stay within `cmsg_store`, which
// outlives the synchronous `sendmsg` below.
unsafe {
mhdr.msg_controllen = libc::CMSG_SPACE(4) as _;
let c = libc::CMSG_FIRSTHDR(&mhdr);
(*c).cmsg_level = libc::SOL_SOCKET;
(*c).cmsg_type = libc::SCM_RIGHTS;
(*c).cmsg_len = libc::CMSG_LEN(4) as _;
std::ptr::write_unaligned(libc::CMSG_DATA(c) as *mut i32, fd.as_raw_fd());
}
}
// SAFETY: `sock` is the caller's live socket; `mhdr` points at the live `iov` (over `body`,
// which outlives the call) and — when an fd is passed — at `cmsg_store` (ditto). `sendmsg`
// only reads these buffers. The kernel dups the fd into the message; our `BorrowedFd` stays
// owned by the caller.
let n = unsafe { libc::sendmsg(sock.as_raw_fd(), &mhdr, libc::MSG_NOSIGNAL) };
if n < 0 {
return Err(io::Error::last_os_error());
}
if n as usize != body.len() {
return Err(io::Error::new(
io::ErrorKind::WriteZero,
"short sendmsg on SEQPACKET socket",
));
}
Ok(())
}
/// Receive one message (+ up to one `SCM_RIGHTS` fd). `buf` is a caller-owned scratch buffer
/// (grown to [`MAX_MSG`] once, then reused frame to frame). Errors:
/// `UnexpectedEof` = the peer is gone; `WouldBlock` = the [`set_recv_timeout`] expired.
pub fn recv<T: DeserializeOwned>(
sock: BorrowedFd,
buf: &mut Vec<u8>,
) -> io::Result<(T, Option<OwnedFd>)> {
buf.resize(MAX_MSG, 0);
let mut iov = libc::iovec {
iov_base: buf.as_mut_ptr() as *mut libc::c_void,
iov_len: buf.len(),
};
let mut cmsg_store = [0u64; 4];
// SAFETY: `mhdr` is a plain-old-data C struct for which all-zero is a valid value.
let mut mhdr: libc::msghdr = unsafe { std::mem::zeroed() };
mhdr.msg_iov = &mut iov;
mhdr.msg_iovlen = 1;
mhdr.msg_control = cmsg_store.as_mut_ptr() as *mut libc::c_void;
mhdr.msg_controllen = std::mem::size_of_val(&cmsg_store) as _;
// SAFETY: `sock` is the caller's live socket. `recvmsg` writes at most `iov_len` bytes into
// `buf` (live for the call) and at most `msg_controllen` control bytes into `cmsg_store`
// (live, 8-aligned). `MSG_CMSG_CLOEXEC` makes any received fd CLOEXEC atomically.
let n = unsafe { libc::recvmsg(sock.as_raw_fd(), &mut mhdr, libc::MSG_CMSG_CLOEXEC) };
if n < 0 {
return Err(io::Error::last_os_error());
}
if n == 0 {
return Err(io::Error::new(
io::ErrorKind::UnexpectedEof,
"zerocopy proto peer closed",
));
}
// Collect a passed fd (if any) BEFORE any early return below, so it can't leak.
let mut got_fd: Option<OwnedFd> = None;
// SAFETY: `CMSG_FIRSTHDR`/`CMSG_NXTHDR` walk the control area the kernel just wrote inside
// `cmsg_store` (bounded by the updated `mhdr.msg_controllen`), returning either null or a
// pointer to a complete `cmsghdr` within it — each dereference reads kernel-initialized
// fields in bounds. For an `SCM_RIGHTS` cmsg the data area holds whole `i32` fds; we read the
// first via `read_unaligned`. The kernel gave us ownership of that fd (it is a fresh
// descriptor in our table), so `OwnedFd::from_raw_fd` takes sole ownership — any previously
// collected `got_fd` is dropped (closed) first, so nothing leaks even with multiple cmsgs.
unsafe {
let mut c = libc::CMSG_FIRSTHDR(&mhdr);
while !c.is_null() {
if (*c).cmsg_level == libc::SOL_SOCKET && (*c).cmsg_type == libc::SCM_RIGHTS {
let fd = std::ptr::read_unaligned(libc::CMSG_DATA(c) as *const i32);
if fd >= 0 {
got_fd = Some(OwnedFd::from_raw_fd(fd));
}
}
c = libc::CMSG_NXTHDR(&mhdr, c);
}
}
if mhdr.msg_flags & libc::MSG_TRUNC != 0 {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
"zerocopy proto message truncated",
));
}
let msg = serde_json::from_slice(&buf[..n as usize])
.map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e))?;
Ok((msg, got_fd))
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::{Read, Write};
use std::os::fd::AsFd;
#[test]
fn round_trip_no_fd() {
let (a, b) = socketpair_seqpacket().unwrap();
let mut buf = Vec::new();
let req = Request::Import {
key: 0xdead_beef_u64,
kind: ImportKind::TiledNv12,
width: 5120,
height: 1440,
fourcc: 0x3432_5258,
modifier: Some(0x0300_0000_0000_1234),
offset: 0,
stride: 5120 * 4,
has_fd: false,
};
send(a.as_fd(), &req, None).unwrap();
let (got, fd) = recv::<Request>(b.as_fd(), &mut buf).unwrap();
assert_eq!(got, req);
assert!(fd.is_none());
let reply = Reply::Frame {
id: 7,
desc: Some(BufferDesc {
width: 5120,
height: 1440,
y_handle: vec![1u8; 64],
y_pitch: 5632,
uv: Some((vec![2u8; 64], 5632)),
}),
};
send(b.as_fd(), &reply, None).unwrap();
let (got, fd) = recv::<Reply>(a.as_fd(), &mut buf).unwrap();
assert_eq!(got, reply);
assert!(fd.is_none());
}
#[test]
fn passes_an_fd() {
let (a, b) = socketpair_seqpacket().unwrap();
let mut buf = Vec::new();
// A pipe stands in for a dmabuf: pass the read end, write through the original write end,
// and read the bytes back through the RECEIVED fd.
let (mut pr, mut pw) = std::io::pipe().unwrap();
send(a.as_fd(), &Request::ClearCache, Some(pr.as_fd())).unwrap();
let (got, fd) = recv::<Request>(b.as_fd(), &mut buf).unwrap();
assert_eq!(got, Request::ClearCache);
let fd = fd.expect("fd should have been passed");
pw.write_all(b"hello").unwrap();
drop(pw);
let mut file = std::fs::File::from(fd);
let mut s = String::new();
file.read_to_string(&mut s).unwrap();
assert_eq!(s, "hello");
// The original read end still works independently of the passed dup.
let mut nothing = [0u8; 1];
assert_eq!(pr.read(&mut nothing).unwrap(), 0);
}
#[test]
fn eof_when_peer_closes() {
let (a, b) = socketpair_seqpacket().unwrap();
drop(a);
let mut buf = Vec::new();
let err = recv::<Reply>(b.as_fd(), &mut buf).unwrap_err();
assert_eq!(err.kind(), io::ErrorKind::UnexpectedEof);
}
#[test]
fn send_to_dead_peer_is_epipe_not_sigpipe() {
let (a, b) = socketpair_seqpacket().unwrap();
drop(b);
let err = send(a.as_fd(), &Request::ClearCache, None).unwrap_err();
// MSG_NOSIGNAL: a dead peer surfaces as EPIPE (BrokenPipe), never a process-killing signal.
assert_eq!(err.kind(), io::ErrorKind::BrokenPipe);
}
#[test]
fn recv_timeout_fires() {
let (a, _b) = socketpair_seqpacket().unwrap();
set_recv_timeout(a.as_fd(), Some(Duration::from_millis(50))).unwrap();
let mut buf = Vec::new();
let err = recv::<Reply>(a.as_fd(), &mut buf).unwrap_err();
assert!(
matches!(
err.kind(),
io::ErrorKind::WouldBlock | io::ErrorKind::TimedOut
),
"unexpected error kind: {err:?}"
);
}
}
@@ -302,6 +302,23 @@ impl VkBridge {
Ok(())
}
/// Drop the cached import for `fd` (the PipeWire buffer it wrapped is gone — pool recycle /
/// renegotiation — or the caller is about to store a different dmabuf under the same slot).
/// Without this the cache could serve a stale imported buffer for a reused fd number, or
/// leak an entry per recycled pool buffer.
pub fn forget_fd(&mut self, fd: i32) {
if let Some(s) = self.src_cache.remove(&fd) {
// SAFETY: `s.buffer`/`s.memory` were created by this bridge's `import_src` and are
// exclusively owned by the removed cache entry, so each is destroyed exactly once.
// No GPU work can still reference them: every `import_linear` fence-waits its copy to
// completion before returning, and this runs on the same single owning thread.
unsafe {
self.device.destroy_buffer(s.buffer, None);
self.device.free_memory(s.memory, None);
}
}
}
/// Bridge one LINEAR dmabuf frame into a pooled CUDA buffer: GPU copy dmabuf→exportable,
/// then pitched CUDA copy exportable→`pool` buffer.
pub fn import_linear(
@@ -0,0 +1,465 @@
//! The isolated zero-copy GPU-import worker (`punktfunk-host zerocopy-worker`; design:
//! [`design/zerocopy-worker-isolation.md`]). It owns the fragile driver stack — the headless
//! EGLDisplay + GL context, the CUDA context, and the Vulkan bridge — so that a driver fault on a
//! producer-invalidated dmabuf (the `cuGraphicsMapResources` SIGSEGV the F44 Game→Desktop switch
//! reproduced) kills THIS process, not the streaming host. The host observes the dead socket,
//! fails the frame cleanly, and its existing capture-loss rebuild takes over.
//!
//! One worker serves one capture (spawned per `pipewire_thread`). It exits on socket EOF — which
//! only happens after the capturer AND every in-flight frame on the host side are gone, so pooled
//! device memory is never freed under a frame the host still reads.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::cuda::{self, CUdeviceptr, DeviceBuffer};
use super::egl::{DmabufPlane, EglImporter};
use super::proto::{self, BufferDesc, ImportKind, Reply, Request};
use anyhow::{bail, Context, Result};
use std::collections::{HashMap, VecDeque};
use std::io;
use std::os::fd::{AsFd, AsRawFd, FromRawFd, OwnedFd};
/// Cap on cached per-key dmabuf fds. PipeWire buffer pools are ≤ ~16 buffers; the cap only
/// matters if a misbehaving producer churns buffers without a renegotiation.
const FD_CACHE_CAP: usize = 64;
/// Entry point for the hidden `zerocopy-worker` subcommand. `args` are the subcommand's own
/// arguments (`--fd N`, default 3 — the socket end the spawning host `dup2`'d in).
pub fn run_from_args(args: &[String]) -> Result<()> {
let fd: i32 = args
.iter()
.skip_while(|a| *a != "--fd")
.nth(1)
.map(|s| s.parse())
.transpose()
.context("parse --fd")?
.unwrap_or(3);
// SAFETY: the spawning host `dup2`'d its socketpair end onto exactly this fd number before
// exec (the subcommand's contract) and nothing else in this fresh process owns it, so
// `OwnedFd` takes sole ownership and closes it exactly once at exit.
let sock = unsafe { OwnedFd::from_raw_fd(fd) };
run(sock)
}
/// Bring up the GPU stack, report readiness, and serve until the host goes away.
fn run(sock: OwnedFd) -> Result<()> {
let importer = match EglImporter::new() {
Ok(i) => i,
Err(e) => {
// Init failure is an ANSWER, not a crash: the host falls back to the CPU path,
// exactly like an in-process `EglImporter::new()` failure.
let _ = proto::send(
sock.as_fd(),
&Reply::InitErr {
message: format!("{e:#}"),
},
None,
);
return Ok(());
}
};
proto::send(
sock.as_fd(),
&Reply::Ready {
version: proto::PROTO_VERSION,
},
None,
)
.context("send Ready")?;
tracing::info!(pid = std::process::id(), "zerocopy import worker ready");
let mut backend = EglBackend::new(importer);
serve(&sock, &mut backend)
}
/// What [`serve`] needs from an import implementation — split out so the dispatch loop is
/// unit-testable without a GPU.
pub(crate) trait ImportBackend {
fn modifiers(&mut self, fourcc: u32) -> Vec<u64>;
/// Answers with [`Reply::Frame`] (buffer id + [`BufferDesc`] iff first delivery of that id),
/// [`Reply::NeedFd`] (this side lacks the key's fd — host resends it once), or [`Reply::Err`].
fn import(&mut self, req: &ImportReq, fd: Option<OwnedFd>) -> Reply;
fn release(&mut self, id: u32);
fn clear_cache(&mut self);
}
/// The [`Request::Import`] fields, destructured for [`ImportBackend::import`].
pub(crate) struct ImportReq {
pub key: u64,
pub kind: ImportKind,
pub width: u32,
pub height: u32,
pub fourcc: u32,
pub modifier: Option<u64>,
pub offset: u32,
pub stride: u32,
pub has_fd: bool,
}
/// The request loop. Returns `Ok(())` on host EOF (normal end-of-life); any other socket error
/// propagates (the process exits — the host treats it like a death, which it is).
pub(crate) fn serve(sock: &OwnedFd, backend: &mut dyn ImportBackend) -> Result<()> {
let mut buf = Vec::new();
loop {
let (req, fd) = match proto::recv::<Request>(sock.as_fd(), &mut buf) {
Ok(v) => v,
Err(e) if e.kind() == io::ErrorKind::UnexpectedEof => return Ok(()),
Err(e) => return Err(e).context("worker recv"),
};
match req {
Request::Modifiers { fourcc } => {
let reply = Reply::Modifiers {
modifiers: backend.modifiers(fourcc),
};
if send_or_eof(sock, &reply)? {
return Ok(());
}
}
Request::Import {
key,
kind,
width,
height,
fourcc,
modifier,
offset,
stride,
has_fd,
} => {
let req = ImportReq {
key,
kind,
width,
height,
fourcc,
modifier,
offset,
stride,
has_fd,
};
let reply = backend.import(&req, fd);
if send_or_eof(sock, &reply)? {
return Ok(());
}
}
Request::Release { id } => backend.release(id),
Request::ClearCache => backend.clear_cache(),
}
}
}
/// Send a reply; `Ok(true)` means the host is gone (EPIPE) and the loop should end quietly.
fn send_or_eof(sock: &OwnedFd, reply: &Reply) -> Result<bool> {
match proto::send(sock.as_fd(), reply, None) {
Ok(()) => Ok(false),
Err(e) if e.kind() == io::ErrorKind::BrokenPipe => Ok(true),
Err(e) => Err(e).context("worker send"),
}
}
/// The real backend: the in-process [`EglImporter`] plus the cross-process bookkeeping —
/// per-key dmabuf fds, in-flight frames (held until `Release`), and stable buffer ids.
struct EglBackend {
importer: EglImporter,
/// The dmabuf fd for each host key (`st_ino`), kept because the tiled path re-imports the fd
/// every frame (`eglCreateImage`) and the LINEAR path caches per fd inside the Vulkan bridge.
fds: HashMap<u64, OwnedFd>,
/// Insertion order of `fds` keys for the LRU cap.
fd_lru: VecDeque<u64>,
/// Frames delivered to the host and not yet released — holding the `DeviceBuffer` is what
/// keeps its device memory alive (pool `Arc`s) while the host encodes from it.
inflight: HashMap<u32, DeviceBuffer>,
/// Buffer id per device allocation. Valid only within one pool generation: pools never free
/// allocations while alive, so a device VA can't repeat until a size change replaces the pool
/// — at which point [`Self::note_dims`] clears this map (ids themselves are never reused;
/// `next_id` only counts up).
ids: HashMap<CUdeviceptr, u32>,
next_id: u32,
/// The (kind, width, height) of the last import — a change means the importer replaced its
/// pool, invalidating the VA→id map (see [`Self::ids`]).
last_shape: Option<(ImportKind, u32, u32)>,
}
impl EglBackend {
fn new(importer: EglImporter) -> EglBackend {
EglBackend {
importer,
fds: HashMap::new(),
fd_lru: VecDeque::new(),
inflight: HashMap::new(),
ids: HashMap::new(),
next_id: 0,
last_shape: None,
}
}
/// Store (or replace) the cached fd for `key`, evicting beyond the cap. A replaced or
/// evicted fd is first forgotten by the Vulkan bridge so its per-fd import can't go stale.
fn store_fd(&mut self, key: u64, fd: OwnedFd) {
if let Some(old) = self.fds.insert(key, fd) {
self.importer.forget_linear_fd(old.as_raw_fd());
self.fd_lru.retain(|k| *k != key);
}
self.fd_lru.push_back(key);
while self.fds.len() > FD_CACHE_CAP {
let Some(oldest) = self.fd_lru.pop_front() else {
break;
};
if let Some(old) = self.fds.remove(&oldest) {
self.importer.forget_linear_fd(old.as_raw_fd());
}
}
}
/// Clear the VA→id map when the importer is about to replace its per-size pool (see
/// [`Self::ids`]).
fn note_dims(&mut self, kind: ImportKind, width: u32, height: u32) {
if self.last_shape != Some((kind, width, height)) {
self.last_shape = Some((kind, width, height));
self.ids.clear();
}
}
}
impl ImportBackend for EglBackend {
fn modifiers(&mut self, fourcc: u32) -> Vec<u64> {
self.importer.supported_modifiers(fourcc)
}
fn import(&mut self, req: &ImportReq, fd: Option<OwnedFd>) -> Reply {
if let Some(fd) = fd {
self.store_fd(req.key, fd);
} else if req.has_fd {
return Reply::Err {
message: "Import said has_fd but no fd arrived".into(),
};
}
let Some(raw) = self.fds.get(&req.key).map(|f| f.as_raw_fd()) else {
// We no longer hold this buffer's fd (LRU eviction / cache desync) — ask the host to
// resend it rather than failing the frame.
return Reply::NeedFd;
};
match self.import_inner(req, raw) {
Ok((id, desc)) => Reply::Frame { id, desc },
Err(e) => Reply::Err {
message: format!("{e:#}"),
},
}
}
fn release(&mut self, id: u32) {
if self.inflight.remove(&id).is_none() {
tracing::warn!(id, "release for a frame not in flight (host/worker desync)");
}
}
fn clear_cache(&mut self) {
for (_, fd) in self.fds.drain() {
self.importer.forget_linear_fd(fd.as_raw_fd());
}
self.fd_lru.clear();
self.importer.clear_linear_cache();
}
}
impl EglBackend {
/// The fallible core of [`ImportBackend::import`], once the fd for `req.key` is resolved.
fn import_inner(&mut self, req: &ImportReq, raw: i32) -> Result<(u32, Option<BufferDesc>)> {
let plane = DmabufPlane {
fd: raw,
offset: req.offset,
stride: req.stride,
};
self.note_dims(req.kind, req.width, req.height);
let buf = match req.kind {
ImportKind::Tiled => {
self.importer
.import(&plane, req.width, req.height, req.fourcc, req.modifier)?
}
ImportKind::TiledNv12 => self.importer.import_nv12(
&plane,
req.width,
req.height,
req.fourcc,
req.modifier,
)?,
ImportKind::Linear => self.importer.import_linear(&plane, req.width, req.height)?,
};
// Assign / look up the buffer's id and export its CUDA IPC identity on first delivery.
cuda::make_current()?;
let (id, desc) = match self.ids.get(&buf.ptr) {
Some(&id) => (id, None),
None => {
let id = self.next_id;
self.next_id = self.next_id.wrapping_add(1);
let y_handle = cuda::ipc_export(buf.ptr)?.to_vec();
let uv = match buf.uv {
Some((uv_ptr, uv_pitch)) => {
Some((cuda::ipc_export(uv_ptr)?.to_vec(), uv_pitch))
}
None => None,
};
self.ids.insert(buf.ptr, id);
(
id,
Some(BufferDesc {
width: buf.width,
height: buf.height,
y_handle,
y_pitch: buf.pitch,
uv,
}),
)
}
};
if self.inflight.insert(id, buf).is_some() {
// A pool never hands out a buffer that hasn't been recycled, so a duplicate id means
// corrupted bookkeeping — fail the import rather than alias two frames.
bail!("buffer id {id} already in flight");
}
Ok((id, desc))
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::mpsc;
/// Records calls; import behavior is scripted per key.
struct MockBackend {
calls: mpsc::Sender<String>,
next: u32,
}
impl ImportBackend for MockBackend {
fn modifiers(&mut self, fourcc: u32) -> Vec<u64> {
let _ = self.calls.send(format!("modifiers:{fourcc}"));
vec![7, 8, 9]
}
fn import(&mut self, req: &ImportReq, fd: Option<OwnedFd>) -> Reply {
let _ = self.calls.send(format!(
"import:key={} kind={:?} fd={}",
req.key,
req.kind,
fd.is_some()
));
if req.key == 0xbad {
return Reply::Err {
message: "scripted failure".into(),
};
}
if req.key == 0xfeed && !req.has_fd {
return Reply::NeedFd;
}
let id = self.next;
self.next += 1;
let desc = (id == 0).then(|| BufferDesc {
width: req.width,
height: req.height,
y_handle: vec![0u8; 64],
y_pitch: 256,
uv: None,
});
Reply::Frame { id, desc }
}
fn release(&mut self, id: u32) {
let _ = self.calls.send(format!("release:{id}"));
}
fn clear_cache(&mut self) {
let _ = self.calls.send("clear".into());
}
}
fn start_server() -> (
OwnedFd,
mpsc::Receiver<String>,
std::thread::JoinHandle<Result<()>>,
) {
let (host, worker) = proto::socketpair_seqpacket().unwrap();
let (tx, rx) = mpsc::channel();
let join = std::thread::spawn(move || {
let mut backend = MockBackend { calls: tx, next: 0 };
serve(&worker, &mut backend)
});
(host, rx, join)
}
fn import_req(key: u64, has_fd: bool) -> Request {
Request::Import {
key,
kind: ImportKind::Tiled,
width: 64,
height: 64,
fourcc: 1,
modifier: None,
offset: 0,
stride: 256,
has_fd,
}
}
#[test]
fn dispatch_and_eof() {
let (host, rx, join) = start_server();
let mut buf = Vec::new();
proto::send(host.as_fd(), &Request::Modifiers { fourcc: 42 }, None).unwrap();
let (reply, _) = proto::recv::<Reply>(host.as_fd(), &mut buf).unwrap();
assert_eq!(
reply,
Reply::Modifiers {
modifiers: vec![7, 8, 9]
}
);
// First import delivers the desc; the second (same mock id sequence continues) doesn't.
proto::send(host.as_fd(), &import_req(1, false), None).unwrap();
let (reply, _) = proto::recv::<Reply>(host.as_fd(), &mut buf).unwrap();
match reply {
Reply::Frame {
id: 0,
desc: Some(_),
} => {}
other => panic!("unexpected reply {other:?}"),
}
proto::send(host.as_fd(), &import_req(1, false), None).unwrap();
let (reply, _) = proto::recv::<Reply>(host.as_fd(), &mut buf).unwrap();
assert_eq!(reply, Reply::Frame { id: 1, desc: None });
// A missing worker-side fd is a NeedFd reply (host resends), not a failure.
proto::send(host.as_fd(), &import_req(0xfeed, false), None).unwrap();
let (reply, _) = proto::recv::<Reply>(host.as_fd(), &mut buf).unwrap();
assert_eq!(reply, Reply::NeedFd);
// A failed import is an Err reply, not a dead worker.
proto::send(host.as_fd(), &import_req(0xbad, false), None).unwrap();
let (reply, _) = proto::recv::<Reply>(host.as_fd(), &mut buf).unwrap();
match reply {
Reply::Err { message } => assert!(message.contains("scripted failure")),
other => panic!("unexpected reply {other:?}"),
}
// Fire-and-forget ops reach the backend without replies.
proto::send(host.as_fd(), &Request::Release { id: 0 }, None).unwrap();
proto::send(host.as_fd(), &Request::ClearCache, None).unwrap();
// Closing the host end terminates serve() cleanly.
drop(host);
join.join().unwrap().unwrap();
let calls: Vec<String> = rx.iter().collect();
assert_eq!(
calls,
vec![
"modifiers:42",
"import:key=1 kind=Tiled fd=false",
"import:key=1 kind=Tiled fd=false",
"import:key=65261 kind=Tiled fd=false", // 0xfeed
"import:key=2989 kind=Tiled fd=false", // 0xbad
"release:0",
"clear",
]
);
}
}
+5
View File
@@ -181,6 +181,11 @@ fn real_main() -> Result<()> {
// Zero-copy FFI/GPU probe: init the EGL importer + CUDA context (no capture needed).
#[cfg(target_os = "linux")]
Some("zerocopy-probe") => zerocopy::probe(),
// Hidden: the isolated GPU-import worker the capture path spawns from /proc/self/exe
// (design/zerocopy-worker-isolation.md) — never run by hand; --fd names the inherited
// socketpair end.
#[cfg(target_os = "linux")]
Some("zerocopy-worker") => zerocopy::worker::run_from_args(&args[1..]),
// NV12 colour self-test (no display/capture needed): convert a known RGBA pattern to NV12
// on the GPU and compare against a BT.709 limited-range reference. Validates the Tier 2A
// `PUNKTFUNK_NV12` convert is colour-correct. Prints PASS/FAIL + max Y/U/V error.
+115 -1
View File
@@ -161,6 +161,8 @@ fn api_router_parts() -> (Router<Arc<MgmtState>>, utoipa::openapi::OpenApi) {
.routes(routes!(get_display_state))
.routes(routes!(release_display))
.routes(routes!(set_display_layout))
.routes(routes!(list_custom_presets, create_custom_preset))
.routes(routes!(update_custom_preset, delete_custom_preset))
.routes(routes!(get_status))
.routes(routes!(get_local_summary))
.routes(routes!(list_paired_clients))
@@ -993,6 +995,10 @@ struct DisplaySettingsState {
effective: crate::vdisplay::policy::EffectivePolicy,
/// Every named preset and what it expands to (for the picker's preview).
presets: Vec<PresetInfo>,
/// The operator's saved custom presets (`display-presets.json`) — named field-bundles rendered
/// alongside the built-ins. Managed via `POST/PUT/DELETE /display/presets`; applied by writing a
/// `Custom` policy carrying the preset's fields.
custom_presets: Vec<crate::vdisplay::policy::CustomPreset>,
/// Option names this build enforces right now. All five axes are now acted on (keep_alive +
/// topology since Stage 0-2, identity Stage 3, mode_conflict Stage 4, layout Stage 5) — the console
/// reads this to know which controls are live vs. "coming soon" (per-backend nuance, e.g. layout
@@ -1037,12 +1043,14 @@ fn display_settings_state() -> DisplaySettingsState {
settings,
configured,
presets,
custom_presets: policy::load_custom_presets(),
enforced: vec![
"keep_alive".into(),
"topology".into(),
"mode_conflict".into(),
"identity".into(),
"layout".into(),
"game_session".into(),
],
}
}
@@ -1248,7 +1256,10 @@ async fn set_display_layout(ApiJson(req): ApiJson<DisplayLayoutRequest>) -> Resp
// Lock the current effective behavior into explicit fields + set the manual arrangement (pure
// transform, unit-tested in `policy.rs`) — so arranging displays is orthogonal to the other policy
// axes. (`effective` keep_alive is never `Forever` via the API — the settings PUT rejects it.)
let policy = store.get().effective().with_manual_layout(req.positions);
let policy = store
.get()
.effective()
.with_manual_layout(req.positions, store.game_session());
if let Err(e) = store.set(policy) {
return api_error(
StatusCode::INTERNAL_SERVER_ERROR,
@@ -1262,6 +1273,109 @@ async fn set_display_layout(ApiJson(req): ApiJson<DisplayLayoutRequest>) -> Resp
Json(display_settings_state()).into_response()
}
/// List the saved custom presets
///
/// The operator's named field-bundles (`display-presets.json`). These also ride the
/// `GET /display/settings` response (`custom_presets`), so the console rarely needs this directly.
#[utoipa::path(
get,
path = "/display/presets",
tag = "display",
operation_id = "listCustomPresets",
responses(
(status = OK, description = "The saved custom presets", body = Vec<crate::vdisplay::policy::CustomPreset>),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
)
)]
async fn list_custom_presets() -> Json<Vec<crate::vdisplay::policy::CustomPreset>> {
Json(crate::vdisplay::policy::load_custom_presets())
}
/// Save a custom preset
///
/// Stores a named bundle of the display-behavior axes (+ the game-session axis) the operator can
/// apply later. The host assigns a stable id, returned in the body. Applying a preset is a
/// `PUT /display/settings` with a `Custom` policy carrying its `fields` — no separate apply route.
#[utoipa::path(
post,
path = "/display/presets",
tag = "display",
operation_id = "createCustomPreset",
request_body = crate::vdisplay::policy::CustomPresetInput,
responses(
(status = CREATED, description = "Preset created", body = crate::vdisplay::policy::CustomPreset),
(status = BAD_REQUEST, description = "Empty name", body = ApiError),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
(status = INTERNAL_SERVER_ERROR, description = "Could not persist the catalog", body = ApiError),
)
)]
async fn create_custom_preset(
ApiJson(input): ApiJson<crate::vdisplay::policy::CustomPresetInput>,
) -> Response {
if input.name.trim().is_empty() {
return api_error(StatusCode::BAD_REQUEST, "preset name must not be empty");
}
match crate::vdisplay::policy::add_custom_preset(input) {
Ok(preset) => (StatusCode::CREATED, Json(preset)).into_response(),
Err(e) => api_error(StatusCode::INTERNAL_SERVER_ERROR, &e.to_string()),
}
}
/// Update a custom preset
#[utoipa::path(
put,
path = "/display/presets/{id}",
tag = "display",
operation_id = "updateCustomPreset",
params(("id" = String, Path, description = "The custom preset id")),
request_body = crate::vdisplay::policy::CustomPresetInput,
responses(
(status = OK, description = "Preset updated", body = crate::vdisplay::policy::CustomPreset),
(status = BAD_REQUEST, description = "Empty name", body = ApiError),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
(status = NOT_FOUND, description = "No custom preset with that id", body = ApiError),
(status = INTERNAL_SERVER_ERROR, description = "Could not persist the catalog", body = ApiError),
)
)]
async fn update_custom_preset(
Path(id): Path<String>,
ApiJson(input): ApiJson<crate::vdisplay::policy::CustomPresetInput>,
) -> Response {
if input.name.trim().is_empty() {
return api_error(StatusCode::BAD_REQUEST, "preset name must not be empty");
}
match crate::vdisplay::policy::update_custom_preset(&id, input) {
Ok(Some(preset)) => Json(preset).into_response(),
Ok(None) => api_error(StatusCode::NOT_FOUND, "no custom preset with that id"),
Err(e) => api_error(StatusCode::INTERNAL_SERVER_ERROR, &e.to_string()),
}
}
/// Delete a custom preset
///
/// Removes it from the catalog. The active policy is untouched — if this preset was the one applied,
/// the running behavior stays exactly as it was (the catalog and `display-settings.json` are decoupled).
#[utoipa::path(
delete,
path = "/display/presets/{id}",
tag = "display",
operation_id = "deleteCustomPreset",
params(("id" = String, Path, description = "The custom preset id")),
responses(
(status = NO_CONTENT, description = "Preset deleted"),
(status = UNAUTHORIZED, description = "Missing or invalid bearer token", body = ApiError),
(status = NOT_FOUND, description = "No custom preset with that id", body = ApiError),
(status = INTERNAL_SERVER_ERROR, description = "Could not persist the catalog", body = ApiError),
)
)]
async fn delete_custom_preset(Path(id): Path<String>) -> Response {
match crate::vdisplay::policy::delete_custom_preset(&id) {
Ok(true) => StatusCode::NO_CONTENT.into_response(),
Ok(false) => api_error(StatusCode::NOT_FOUND, "no custom preset with that id"),
Err(e) => api_error(StatusCode::INTERNAL_SERVER_ERROR, &e.to_string()),
}
}
/// Live host status
#[utoipa::path(
get,
+234 -16
View File
@@ -285,6 +285,9 @@ pub(crate) async fn serve(
// restores the box's autologin gaming session on idle, not per-disconnect — see
// `vdisplay::restore_managed_session`). Held for serve()'s lifetime; dropping it stops it.
let _restore_worker = crate::vdisplay::start_restore_worker();
// A3: recover a TV takeover stranded by a crashed previous host instance (persisted to
// $XDG_RUNTIME_DIR) — schedule a restore after a reconnect grace. No-op on a clean start.
crate::vdisplay::restore_takeover_on_startup();
// Host-lifetime cover-art warmer: fetches + caches GOG/Xbox cover art (no-auth api.gog.com /
// displaycatalog) off the hot path so `all_games()` (the library list + launch resolve) never
// blocks on the network. A no-op on a host whose stores all carry their own art.
@@ -826,8 +829,23 @@ async fn serve_session(
let compositor = match source {
Punktfunk1Source::Virtual => {
let pref = hello.compositor;
// Dedicated game session (B0): a launching client under `game_session=dedicated`
// (gamescope available) gets its own headless gamescope spawn at the client mode. Gate on
// whether the launch id actually RESOLVES to a command in the host's library — an unknown
// id must fall back to normal auto routing, not a blank "sleep infinity" gamescope
// (review #9). (dedicated is Linux-only; the resolver is the non-Windows launch_command.)
#[cfg(not(target_os = "windows"))]
let has_resolvable_launch = hello
.launch
.as_deref()
.and_then(crate::library::launch_command)
.is_some();
#[cfg(target_os = "windows")]
let has_resolvable_launch = false;
let dedicated =
crate::vdisplay::wants_dedicated_game_session(has_resolvable_launch);
Some(
tokio::task::spawn_blocking(move || resolve_compositor(pref))
tokio::task::spawn_blocking(move || resolve_compositor(pref, dedicated))
.await
.context("resolve compositor task")??,
)
@@ -2223,17 +2241,24 @@ fn pick_compositor(
/// [`pick_compositor`]): enumerate what's available, auto-detect the default, pick, and log
/// whether the explicit request was honored or fell back. Runs blocking probes — call off the
/// async reactor (`spawn_blocking`).
fn resolve_compositor(pref: CompositorPref) -> Result<crate::vdisplay::Compositor> {
fn resolve_compositor(
pref: CompositorPref,
dedicated_launch: bool,
) -> Result<crate::vdisplay::Compositor> {
use crate::vdisplay::Compositor;
// Windows has a single virtual-display backend (SudoVDA); vdisplay::open ignores the compositor
// arg there, so short-circuit the Linux session-detection state machine with a placeholder.
#[cfg(target_os = "windows")]
{
let _ = pref;
let _ = (pref, dedicated_launch);
Ok(Compositor::Kwin)
}
#[cfg(not(target_os = "windows"))]
{
// A client is (re)connecting → cancel any pending TV-session restore so the box stays in the
// streamed session (covers the keep-alive REUSE reconnect, which skips create_managed_session's
// own cancel — review #3). No-op when nothing is pending.
crate::vdisplay::cancel_pending_tv_restore();
// Explicit operator override (legacy / CI / forcing a backend for a test) wins and is assumed
// to come with a hand-set env — don't retarget the process env in that case.
let overridden = crate::config::config().compositor.is_some();
@@ -2244,6 +2269,10 @@ fn resolve_compositor(pref: CompositorPref) -> Result<crate::vdisplay::Composito
// every backend (video capture + input) this connect opens against the active session —
// this is the state machine that lets one host follow a Bazzite box across Gaming↔Desktop.
let active = crate::vdisplay::detect_active_session();
// A4: if the compositor instance changed since the last connect (an idle-time Game↔Desktop
// switch), bump the epoch + invalidate the old backend's kept displays so this connect never
// reuses a node id from the dead instance.
crate::vdisplay::observe_session_instance(&active);
crate::vdisplay::apply_session_env(&active);
tracing::info!(
active = ?active.kind,
@@ -2252,6 +2281,18 @@ fn resolve_compositor(pref: CompositorPref) -> Result<crate::vdisplay::Composito
);
crate::vdisplay::compositor_for_kind(active.kind)
};
// Dedicated game session (design/gamemode-and-dedicated-sessions.md B0): a launching session
// under `game_session=dedicated` (gamescope confirmed available) forces its OWN headless
// gamescope spawn at the client's mode, overriding the detected desktop/game-mode backend. The
// env was already retargeted above (for XDG_RUNTIME_DIR / the PipeWire daemon); we just pin the
// backend + input to the spawn sub-mode. Skipped under an explicit operator compositor pin.
if dedicated_launch && !overridden {
crate::vdisplay::apply_input_env(Compositor::Gamescope, true);
tracing::info!(
"dedicated game session — routing to a headless gamescope spawn at the client mode"
);
return Ok(Compositor::Gamescope);
}
let available = crate::vdisplay::available();
let chosen = pick_compositor(pref, &available, detected).ok_or_else(|| {
anyhow!("no usable compositor (no live graphical session for this uid; set PUNKTFUNK_COMPOSITOR or start a desktop/gaming session)")
@@ -2259,7 +2300,7 @@ fn resolve_compositor(pref: CompositorPref) -> Result<crate::vdisplay::Composito
if !overridden {
// Point input at the same backend and resolve the gamescope sub-mode (managed where the
// session infra exists, attach to a foreign gamescope, else per-session bare spawn).
crate::vdisplay::apply_input_env(chosen);
crate::vdisplay::apply_input_env(chosen, false);
}
let avail_ids: Vec<&str> = available.iter().map(|c| c.id()).collect();
match Compositor::from_pref(pref) {
@@ -2886,6 +2927,11 @@ fn session_watcher_loop(tx: std::sync::mpsc::Sender<SessionSwitch>, stop: Arc<At
break;
}
let active = vdisplay::detect_active_session();
// A4: bump the session epoch + invalidate the old backend the moment the compositor instance
// changes (kind change OR same-kind restart) — even for a same-kind restart the watcher won't
// signal a full SessionSwitch for. Self-dedupes; the debounced SessionSwitch below still drives
// the in-place rebuild.
vdisplay::observe_session_instance(&active);
let cur = active.kind;
if cur == current {
pending = None; // back to the current backend before debounce elapsed — no switch
@@ -3049,7 +3095,7 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
#[cfg(target_os = "windows")]
let _idd_setup_guard = (plan.capture == crate::session_plan::CaptureBackend::IddPush)
.then(|| crate::vdisplay::manager::vdm().begin_idd_setup(stop.clone()));
let (mut capturer, mut enc, mut frame, mut interval) =
let (mut capturer, mut enc, mut frame, mut interval, mut cur_node_id) =
build_pipeline_with_retry(&mut vd, mode, bitrate_kbps, bit_depth, plan, &quit)?;
// Setup done — release the IDD-push setup lock so the next reconnect can begin (and preempt us).
#[cfg(target_os = "windows")]
@@ -3152,6 +3198,18 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
let mut cur_mode = mode;
const MAX_CAPTURE_REBUILDS: u32 = 5;
let mut capture_rebuilds: u32 = 0;
// Encode-stall watchdog: AMF/QSV (and async NVENC) poll non-blocking, so a wedged driver
// shows up as poll() returning None forever while submits keep succeeding — `inflight` grows,
// no AU ever reaches the send thread, and the client freezes on the last frame with nothing
// logged (field reports: AMD/Intel Windows streams freezing after minutes). Track when the
// encoder last produced an AU and rebuild it in place (bounded, like the capture rebuilds)
// when it stops. `ENCODE_STALL_WINDOW` also sizes the in-flight backlog bound: a backlog worth
// more than the window's frames means AUs still trickle (so the gap never trips) but latency
// is growing without bound — the slow-leak form of the same stall.
const ENCODE_STALL_WINDOW: std::time::Duration = std::time::Duration::from_secs(2);
const MAX_ENCODER_RESETS: u32 = 5;
let mut encoder_resets: u32 = 0;
let mut last_au_at = std::time::Instant::now();
// Last HDR mastering metadata we forwarded — re-sent as 0xCE on change/keyframe (see below).
let mut last_hdr_meta: Option<punktfunk_core::quic::HdrMeta> = None;
// Frames submitted to NVENC but not yet polled (wire pts, submit stamp, pacing deadline). With a
@@ -3196,8 +3254,11 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
crate::vdisplay::apply_session_env(&crate::vdisplay::ActiveSession {
kind: sw.kind,
env: sw.env,
compositor_pid: None,
});
crate::vdisplay::apply_input_env(sw.compositor);
// A mid-stream Game↔Desktop switch is not a fresh dedicated launch — route input at the
// switched-to backend's normal sub-mode.
crate::vdisplay::apply_input_env(sw.compositor, false);
// Switching INTO a desktop mid-stream: the xdg portal / systemd-user env may still
// point at the old session, so input would silently not land until a reconnect.
// Settle it (env push + KWin portal restart) before the injector reopens against it.
@@ -3223,16 +3284,22 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
Ok((new_vd, pipe))
})();
match rebuilt {
Ok((new_vd, (new_cap, new_enc, new_frame, new_interval))) => {
Ok((new_vd, (new_cap, new_enc, new_frame, new_interval, new_node_id))) => {
// Replace the pipeline first (drops the old capturer → old PipeWire stream +
// virtual output), then the factory (drops e.g. the old KWin connection).
capturer = new_cap;
enc = new_enc;
frame = new_frame;
interval = new_interval;
cur_node_id = new_node_id;
vd = new_vd;
compositor = sw.compositor;
next = std::time::Instant::now();
// The owed AUs died with the old encoder — drop their in-flight records
// and restart the encode-stall clock for the fresh one.
inflight.clear();
last_au_at = std::time::Instant::now();
encoder_resets = 0;
tracing::info!(
compositor = compositor.id(),
"session switch — backend rebuilt, stream continues"
@@ -3264,9 +3331,14 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
// healthy session — keep streaming the current mode and log instead.
match build_pipeline(&mut vd, new_mode, bitrate_kbps, bit_depth, plan, &quit) {
Ok(next_pipe) => {
(capturer, enc, frame, interval) = next_pipe;
(capturer, enc, frame, interval, cur_node_id) = next_pipe;
cur_mode = new_mode;
next = std::time::Instant::now();
// The owed AUs died with the old encoder — drop their in-flight records
// and restart the encode-stall clock for the fresh one.
inflight.clear();
last_au_at = std::time::Instant::now();
encoder_resets = 0;
}
Err(e) => {
tracing::error!(error = %format!("{e:#}"), ?new_mode,
@@ -3331,6 +3403,32 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
// bounded retry is exhausted; the consecutive cap stops a flapping source from looping the
// client through endless cold IDRs.
Err(e) => {
// B2: a DEDICATED gamescope game session whose gamescope node is gone = the game
// exited (gamescope is a single-app compositor — it dies with its app). End the session
// CLEANLY — close with `APP_EXITED_CLOSE_CODE` so a launcher client returns to its
// library instead of surfacing a failure — rather than the capture-loss rebuild + 40 s
// timeout. Gated to the dedicated bare-spawn launch (`launch_is_nested`), so a normal
// Bazzite/desktop capture loss still rebuilds in place.
// `cur_node_id` (the capture 5-tuple's node id) is read only by the Linux
// dedicated-game-exit check below; keep it read on other platforms so it isn't a
// write-only variable under `-D warnings` (the `let _ = &launch` idiom above).
#[cfg(not(target_os = "linux"))]
let _ = &cur_node_id;
#[cfg(target_os = "linux")]
if launch.is_some()
&& crate::vdisplay::launch_is_nested(compositor)
&& crate::vdisplay::dedicated_game_exited(cur_node_id)
{
tracing::info!(
"dedicated game session: the game exited — ending the session cleanly"
);
quit.store(true, Ordering::SeqCst); // skip keep-alive linger — the game is gone
conn.close(
punktfunk_core::quic::APP_EXITED_CLOSE_CODE.into(),
b"game exited",
);
break;
}
capture_rebuilds += 1;
if capture_rebuilds > MAX_CAPTURE_REBUILDS {
return Err(e).context("capture lost — rebuild attempts exhausted");
@@ -3348,14 +3446,18 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
// appears — no reconnect.
const REBUILD_BUDGET: std::time::Duration = std::time::Duration::from_secs(40);
let rebuild_deadline = std::time::Instant::now() + REBUILD_BUDGET;
let (new_cap, new_enc, new_frame, new_interval) = loop {
let (new_cap, new_enc, new_frame, new_interval, new_node_id) = loop {
// Follow the active session unless an explicit PUNKTFUNK_COMPOSITOR pin forbids
// retargeting (then we stick to the pinned backend and just rebuild it).
if crate::config::config().compositor.is_none() {
let active = crate::vdisplay::detect_active_session();
// A4: fold any compositor-instance change into the epoch/invalidation before we
// rebuild, so the rebuild's acquire won't reuse a dead-instance node.
crate::vdisplay::observe_session_instance(&active);
if let Some(c) = crate::vdisplay::compositor_for_kind(active.kind) {
crate::vdisplay::apply_session_env(&active);
crate::vdisplay::apply_input_env(c);
// Capture-loss rebuild follows the live box session, not a fresh dedicated launch.
crate::vdisplay::apply_input_env(c, false);
if c != compositor {
if matches!(
c,
@@ -3402,8 +3504,15 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
enc = new_enc;
frame = new_frame;
interval = new_interval;
cur_node_id = new_node_id;
enc.request_keyframe(); // belt-and-suspenders; a fresh encoder opens on an IDR anyway
next = std::time::Instant::now();
// The owed AUs died with the old encoder — drop their in-flight records and
// restart the encode-stall clock (the rebuild loop above may have eaten seconds,
// which must not count against the fresh encoder).
inflight.clear();
last_au_at = std::time::Instant::now();
encoder_resets = 0;
tracing::info!(
compositor = compositor.id(),
"capture loss: pipeline rebuilt — stream resumes"
@@ -3470,7 +3579,28 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
st_queue.push(queue_us);
}
let t_submit = std::time::Instant::now();
enc.submit(&frame).context("encoder submit")?;
if let Err(e) = enc.submit(&frame) {
// The input half of an encode stall: once the driver stops draining AUs, libavcodec's
// one-frame buffer fills and avcodec_send_frame starts failing (EAGAIN) — the same
// wedge the watchdog below catches, seen from submit. Rebuild the encoder in place
// (bounded) instead of killing an otherwise healthy session; a backend without an
// in-place rebuild keeps today's fail-fast behavior.
encoder_resets += 1;
if encoder_resets > MAX_ENCODER_RESETS
|| !reset_stalled_encoder(&mut enc, &mut inflight)
{
return Err(e).context("encoder submit");
}
tracing::error!(error = %format!("{e:#}"), reset = encoder_resets,
max = MAX_ENCODER_RESETS,
"encoder submit failed — encoder rebuilt in place, forcing an IDR");
last_au_at = std::time::Instant::now();
// Re-pace from the rebuild and retry this frame next tick (gives the fresh encoder
// one frame period to come up instead of hammering it in a hot loop).
next = std::time::Instant::now() + interval;
std::thread::sleep(interval);
continue;
}
let submit_us = if measure {
t_submit.elapsed().as_micros() as u32
} else {
@@ -3488,9 +3618,12 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
// so the encode of N overlaps the convert/copy of N+1. NVENC's `pending` is FIFO, so poll() returns
// the oldest submitted frame's AU — matching `inflight.pop_front()`.
let mut send_gone = false;
// A poll error is the explicit form of an encode stall (e.g. a QSV device failure);
// carry it to the shared stall recovery below instead of killing the session outright.
let mut poll_err: Option<anyhow::Error> = None;
while inflight.len() >= depth {
let t_wait = std::time::Instant::now();
let polled = enc.poll().context("encoder poll")?;
let polled = enc.poll();
let wait_us = if measure {
t_wait.elapsed().as_micros() as u32
} else {
@@ -3500,9 +3633,20 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
st_wait.push(wait_us);
}
let au = match polled {
Some(au) => au,
None => break, // no AU ready for a submitted frame (shouldn't happen — poll blocks)
Ok(Some(au)) => au,
// No AU ready for a submitted frame. Routine on the non-blocking backends (the
// libavcodec AMF/QSV wrapper holds ~2 frames; async NVENC drains a ready queue) —
// the frame stays in flight and the next tick re-polls. The stall watchdog below
// decides when "not ready yet" has become "the driver is wedged".
Ok(None) => break,
Err(e) => {
poll_err = Some(e);
break;
}
};
// The encoder is alive: feed the stall watchdog, clear the consecutive-reset counter.
last_au_at = std::time::Instant::now();
encoder_resets = 0;
let (cap_ns, sub_ns, deadline) = inflight.pop_front().expect("inflight non-empty");
let flags = if au.keyframe {
(FLAG_PIC | FLAG_SOF) as u32
@@ -3543,6 +3687,40 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
if send_gone {
break;
}
// Encode-stall watchdog. Trip on: an explicit poll error; no AU within the window while
// frames are owed (the full wedge — AMF/QSV's non-blocking poll returns None forever and
// nothing else ever errors); or an owed backlog worth more than the window's frames (the
// slow leak — AUs still trickle, so the gap never trips, but latency grows without bound).
// Recovery rebuilds the encoder in place and forces an IDR — a logged ~one-second hiccup
// instead of a silent permanent freeze — bounded so a genuinely dead encoder still ends
// the session with a clear error. The window scales with the frame interval so low-fps
// modes (where the AMF wrapper's ~2-frame hold spans seconds) can't false-trip.
let stall_window = ENCODE_STALL_WINDOW.max(interval * 8);
let stall_backlog =
depth + (stall_window.as_secs_f64() / interval.as_secs_f64().max(1e-6)).ceil() as usize;
if poll_err.is_some()
|| (!inflight.is_empty()
&& (last_au_at.elapsed() >= stall_window || inflight.len() > stall_backlog))
{
let why = match &poll_err {
Some(e) => format!("poll failed: {e:#}"),
None => format!(
"no AU for {} ms with {} frame(s) in flight",
last_au_at.elapsed().as_millis(),
inflight.len()
),
};
encoder_resets += 1;
if encoder_resets > MAX_ENCODER_RESETS
|| !reset_stalled_encoder(&mut enc, &mut inflight)
{
return Err(poll_err.unwrap_or_else(|| anyhow!("{why}")))
.context("encoder stalled — in-place rebuild unavailable or exhausted");
}
tracing::error!(reset = encoder_resets, max = MAX_ENCODER_RESETS, %why,
"encode stall detected — encoder rebuilt in place, forcing an IDR");
last_au_at = std::time::Instant::now();
}
match next.checked_duration_since(std::time::Instant::now()) {
Some(d) => std::thread::sleep(d),
None => next = std::time::Instant::now(),
@@ -3592,6 +3770,10 @@ type Pipeline = (
Box<dyn crate::encode::Encoder>,
crate::capture::CapturedFrame,
std::time::Duration,
// The virtual output's PipeWire node id — used by the B2 dedicated game-exit probe to check THIS
// session's own node (scoped), not any gamescope node. `0` for backends without a PipeWire node
// (Windows IDD-push), which never take the dedicated-gamescope B2 path anyway.
u32,
);
/// Build the pipeline, retrying *transient* failures with bounded exponential backoff.
@@ -3694,6 +3876,24 @@ fn is_permanent_build_error(chain: &str) -> bool {
PERMANENT.iter().any(|p| lower.contains(p))
}
/// Encode-stall recovery: rebuild the encoder in place (keeping capture + the session up) and
/// discard the owed in-flight frame records — their AUs died with the old encoder instance.
/// Returns `false` when the backend has no in-place rebuild ([`crate::encode::Encoder::reset`]'s
/// default); the caller then surfaces the stall as a session error instead. The forced keyframe
/// makes the rebuilt encoder's first frame an immediate decoder resync point (belt-and-suspenders:
/// a fresh encoder opens on an IDR anyway).
fn reset_stalled_encoder(
enc: &mut Box<dyn crate::encode::Encoder>,
inflight: &mut std::collections::VecDeque<(u64, u64, std::time::Instant)>,
) -> bool {
if !enc.reset() {
return false;
}
inflight.clear();
enc.request_keyframe();
true
}
fn build_pipeline(
vd: &mut Box<dyn crate::vdisplay::VirtualDisplay>,
mode: punktfunk_core::Mode,
@@ -3709,6 +3909,14 @@ fn build_pipeline(
// `quit` flag rides into the lease so a deliberate-quit teardown skips the keep-alive linger.
let vout = crate::vdisplay::registry::acquire(vd, mode, quit.clone())
.context("create virtual output")?;
// A2: if this was a REUSED kept display and its first frame fails, tear the (dead) pool entry down
// so the retry loop's next acquire creates fresh instead of re-wedging on the same corpse. Read the
// gen BEFORE `capture_virtual_output` consumes `vout`. (Linux-only — the pool is Linux.)
#[cfg(target_os = "linux")]
let reused_gen = vout.reused_gen;
// The virtual output's PipeWire node id — kept for the B2 dedicated game-exit probe (scoped to
// this session's own node). Read before `capture_virtual_output` consumes `vout`.
let node_id = vout.node_id;
// The backend reports the refresh it actually achieved in `preferred_mode.2` (KWin may cap a
// virtual output at 60 Hz if the custom-mode install was rejected). Pace the encoder + frame
// clock to that, not the requested rate, so we don't emit phantom duplicate frames over a
@@ -3733,7 +3941,17 @@ fn build_pipeline(
crate::capture::capture_virtual_output(vout, plan.output_format(), plan.capture)
.context("capture virtual output")?;
capturer.set_active(true);
let frame = capturer.next_frame().context("first frame")?;
let frame = match capturer.next_frame().context("first frame") {
Ok(f) => f,
Err(e) => {
// A reused kept display was dead — invalidate it so the next attempt creates fresh (A2).
#[cfg(target_os = "linux")]
if let Some(g) = reused_gen {
crate::vdisplay::registry::mark_failed(g);
}
return Err(e);
}
};
// `bit_depth` is the handshake-negotiated value (8, or 10 = HEVC Main10 when the client
// advertised VIDEO_CAP_10BIT and the host opted in). Threaded down from the Welcome.
let enc = crate::encode::open_video(
@@ -3760,7 +3978,7 @@ fn build_pipeline(
);
}
let interval = std::time::Duration::from_secs_f64(1.0 / effective_hz.max(1) as f64);
Ok((capturer, enc, frame, interval))
Ok((capturer, enc, frame, interval, node_id))
}
#[cfg(test)]
+272 -16
View File
@@ -21,6 +21,29 @@ pub use punktfunk_core::Mode;
#[cfg(target_os = "linux")]
use std::os::fd::OwnedFd;
/// Who owns a [`VirtualOutput`]'s lifecycle — the honest declaration that lets the registry
/// (`design/gamemode-and-dedicated-sessions.md` Part A1) pool **only what it owns** instead of
/// keeping outputs whose real lifecycle lives elsewhere (the gamescope managed/attach paths, which
/// are governed by the gamescope module's own session machinery). Extends the CLAUDE.md invariant
/// "the registry owns display lifecycle" with its converse: what the registry does not own, it must
/// not pretend to keep.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub enum DisplayOwnership {
/// The registry owns the lifecycle: it may pool, linger, pin, and tear this display down (KWin,
/// Mutter, wlroots, gamescope **bare spawn**, and the Windows manager-delegated monitor). The
/// default — a backend that says nothing is registry-owned.
#[default]
Owned,
/// Someone else's display, merely mirrored: no keep-alive, no topology, no reuse (gamescope
/// **attach** to a foreign session). Codifies the design-doc §7 "attach = unmanaged pass-through"
/// row.
External,
/// A box-level session the gamescope module manages (the managed `gamescope-session-plus` /
/// SteamOS takeover). Passed through by the registry (its restore lifecycle is the gamescope
/// module's until Part A3 hands the registry a real keepalive + restore duty).
SessionManaged,
}
/// A created virtual output: a PipeWire source to capture, plus an owned keepalive whose drop
/// tears the output down (releases the compositor-side resource).
///
@@ -44,6 +67,41 @@ pub struct VirtualOutput {
pub win_capture: Option<crate::capture::dxgi::WinCaptureTarget>,
/// Keeps the output — and whatever connection/thread backs it — alive; dropped on teardown.
pub keepalive: Box<dyn Send>,
/// Who owns this display's lifecycle (`design/gamemode-and-dedicated-sessions.md` A1). The
/// registry pools/keep-alives only [`DisplayOwnership::Owned`] outputs; `External`/`SessionManaged`
/// pass through (the capturer holds the keepalive, teardown on drop). Defaults to `Owned`.
pub ownership: DisplayOwnership,
/// `Some(gen)` when [`registry::acquire`](crate::vdisplay::registry::acquire) handed this back as a
/// **reused** kept display (`design/gamemode-and-dedicated-sessions.md` A2), so the pipeline builder
/// can [`registry::mark_failed(gen)`](crate::vdisplay::registry::mark_failed) if the first frame
/// fails on it — tearing the corpse down so the retry loop's next acquire creates fresh instead of
/// re-wedging on the same dead node. `None` on a fresh create / non-poolable output. Linux-only (the
/// keep-alive pool is Linux).
#[cfg(target_os = "linux")]
pub reused_gen: Option<u64>,
}
impl VirtualOutput {
/// A registry-[owned](DisplayOwnership::Owned) output — the common case (KWin/Mutter/wlroots,
/// gamescope bare-spawn, Windows). Fills `ownership: Owned`; the caller sets the platform fields.
pub fn owned(
node_id: u32,
preferred_mode: Option<(u32, u32, u32)>,
keepalive: Box<dyn Send>,
) -> VirtualOutput {
VirtualOutput {
node_id,
#[cfg(target_os = "linux")]
remote_fd: None,
preferred_mode,
#[cfg(target_os = "windows")]
win_capture: None,
keepalive,
ownership: DisplayOwnership::Owned,
#[cfg(target_os = "linux")]
reused_gen: None,
}
}
}
/// Pluggable virtual-output creation, per compositor.
@@ -101,6 +159,110 @@ pub trait VirtualDisplay: Send {
/// runtime by output name (first-slot-wins + a group-aware disable filter), and single-display
/// backends never have a sibling.
fn set_first_in_group(&mut self, _first: bool) {}
/// Will a [`create`](Self::create) for the CURRENT request produce a registry-poolable
/// ([`DisplayOwnership::Owned`], keep-alive-able) display? The registry consults this **before**
/// its keep-alive reuse lookup, so it never hands a kept display of one flavor to a request of
/// another — specifically a gamescope managed/attach acquire must not reuse a kept **bare-spawn**
/// (they share the backend name `"gamescope"`). Default `true`; only gamescope overrides it,
/// returning `false` when the env selects attach/managed (consistent with the `ownership` its
/// `create` will report). See `design/gamemode-and-dedicated-sessions.md` A1.
fn poolable_now(&self) -> bool {
true
}
/// The resolved launch command carried on this backend instance (set via
/// [`set_launch_command`](Self::set_launch_command)). The registry reads it to key keep-alive reuse
/// on `(backend, mode, launch)` (`design/gamemode-and-dedicated-sessions.md` A2) — a kept display
/// running game A must never be handed to a session that asked to launch game B. Default `None`
/// (backends that never nest a command); only gamescope reports its `cmd`.
fn launch_command(&self) -> Option<String> {
None
}
/// Is the kept display's `node_id` still live, checked **before** the registry REUSES it on a
/// reconnect (`design/gamemode-and-dedicated-sessions.md` A2)? A `false` tells the registry to tear
/// the dead entry down and create fresh instead of handing back a corpse (which would then fail
/// capture and burn a retry). Default `true` (honest optimism — the [`mark_failed`] path is the
/// backstop for a display that dies between this check and first frame). Only gamescope overrides
/// it (its nested session dies when the game exits, independently of any compositor); KWin/Mutter
/// nodes die only with their compositor, which the session-epoch invalidation (A4) already reaps.
///
/// [`mark_failed`]: crate::vdisplay::registry::mark_failed
fn kept_display_alive(&mut self, _node_id: u32) -> bool {
true
}
}
/// The **session epoch** — bumped whenever session detection observes a different compositor
/// *instance*: an [`ActiveKind`] change, **or** a new compositor PID for the same kind (the
/// Desktop→Game→Desktop bounce that brings up a fresh KWin/gamescope with an unrelated node-id space).
/// Pooled displays stamp the epoch at creation; the registry only reuses an entry whose epoch still
/// matches, and its linger timer reaps entries from dead epochs — so a switch can never hand back a
/// node id that now means nothing (`design/gamemode-and-dedicated-sessions.md` A4).
static SESSION_EPOCH: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1);
/// The current [session epoch](SESSION_EPOCH). Read by the registry at acquire (to stamp new entries
/// and gate reuse) and by its linger timer (to reap dead-epoch zombies).
pub fn session_epoch() -> u64 {
SESSION_EPOCH.load(std::sync::atomic::Ordering::Relaxed)
}
/// Bump the [session epoch](SESSION_EPOCH) — call when session detection sees a new compositor
/// instance (kind change, or same-kind new PID). Returns the new value.
pub fn bump_session_epoch() -> u64 {
SESSION_EPOCH.fetch_add(1, std::sync::atomic::Ordering::Relaxed) + 1
}
/// The last-observed compositor instance `(kind, pid)`, so [`observe_session_instance`] can tell a
/// genuine instance change from a stable re-detect.
static LAST_INSTANCE: std::sync::Mutex<Option<(ActiveKind, Option<u32>)>> =
std::sync::Mutex::new(None);
/// Observe the freshly-[detected](detect_active_session) live session and, if the compositor
/// *instance* changed since the last observation — a different [`ActiveKind`], **or** the same kind
/// with a new PID (a compositor restart / Desktop→Game→Desktop bounce) — bump the [session
/// epoch](SESSION_EPOCH) and [invalidate](registry::invalidate_backend) the previous backend's kept
/// displays, so a reconnect can never reuse a node id from the dead instance (A4). Idempotent per
/// instance; the first observation just records the baseline. Cheap on the steady state (one mutex
/// read); the registry lock is taken only on an actual change. Call from every site that detects the
/// session (the per-connect resolve, the mid-stream watcher, the capture-loss re-detect).
pub fn observe_session_instance(active: &ActiveSession) {
let cur = (active.kind, active.compositor_pid);
let mut last = LAST_INSTANCE.lock().unwrap_or_else(|e| e.into_inner());
if let Some(prev) = *last {
// Only a **desktop** compositor (KWin / Mutter / wlroots) instance change bumps the epoch +
// invalidates its kept displays — its PipeWire node dies with the compositor. A **gamescope**
// session (`ActiveKind::Gaming`) is NOT the epoch's subject: the box's game-mode / managed
// gamescope isn't pooled, and dedicated **spawns** are independent nested sessions whose nodes
// outlive any active-session change. So a game-mode gamescope restart, a Gaming↔Gaming winning-PID
// flap (e.g. B1 stopping the autologin before a dedicated spawn), or a coexisting-gamescope set
// change must NOT bump/invalidate — that would tear down a live/kept dedicated session (review
// findings #6/#7/#10). Gate the whole action on a desktop kind being involved.
if prev != cur && (is_desktop_kind(prev.0) || is_desktop_kind(cur.0)) {
// Invalidate only the OLD backend, and only if it was a desktop compositor (never gamescope).
if is_desktop_kind(prev.0) {
if let Some(old) = compositor_for_kind(prev.0) {
registry::invalidate_backend(old.id());
}
}
let epoch = bump_session_epoch();
tracing::info!(
from = ?prev.0,
to = ?cur.0,
epoch,
"desktop compositor instance changed — session epoch bumped"
);
}
}
*last = Some(cur);
}
/// Is `kind` a **desktop** compositor (KWin / Mutter / wlroots) — one whose kept PipeWire outputs die
/// with the compositor instance, so the session epoch tracks it? `Gaming` (gamescope) and `None` are
/// not (gamescope spawns are independent nested sessions — see [`observe_session_instance`]).
fn is_desktop_kind(kind: ActiveKind) -> bool {
matches!(
kind,
ActiveKind::DesktopKde | ActiveKind::DesktopGnome | ActiveKind::DesktopWlroots
)
}
/// Compositors punktfunk knows how to drive (plan §6).
@@ -241,6 +403,10 @@ pub struct SessionEnv {
pub struct ActiveSession {
pub kind: ActiveKind,
pub env: SessionEnv,
/// PID of the winning compositor process (`None` when nothing live). The session watcher compares
/// it across polls so a **same-kind** compositor restart (Desktop→Game→Desktop) bumps the session
/// epoch — a fresh instance's node-id space is unrelated to the old one's (A4).
pub compositor_pid: Option<u32>,
}
impl ActiveSession {
@@ -253,6 +419,7 @@ impl ActiveSession {
dbus_session_bus_address: default_bus(&default_runtime_dir()),
..Default::default()
},
compositor_pid: None,
}
}
}
@@ -304,6 +471,9 @@ pub fn detect_active_session() -> ActiveSession {
// `pkill -x` discipline (exact, ≤15 chars so untruncated).
let mut kind = ActiveKind::None;
let mut best = 0u8;
// The winning compositor's PID — kept so a same-kind compositor RESTART (a new PID) bumps the
// session epoch (A4), not just a kind change.
let mut winning_pid: Option<u32> = None;
if let Ok(entries) = std::fs::read_dir("/proc") {
for e in entries.flatten() {
let name = e.file_name();
@@ -328,9 +498,22 @@ pub fn detect_active_session() -> ActiveSession {
"sway" | "Hyprland" | "hyprland" | "river" => (ActiveKind::DesktopWlroots, 4),
_ => continue,
};
let pid = name.parse::<u32>().ok();
if prio > best {
best = prio;
kind = k;
winning_pid = pid;
} else if prio == best {
// Deterministic tie-break among same-top-priority processes: keep the LOWEST pid, so a
// duplicate same-kind compositor (two `kwin_wayland`) can't make `winning_pid` flap with
// `/proc` enumeration order — which `observe_session_instance` would misread as a
// compositor restart and tear a live display down (re-review low-severity note).
if let (Some(p), Some(w)) = (pid, winning_pid) {
if p < w {
kind = k;
winning_pid = Some(p);
}
}
}
}
}
@@ -358,6 +541,7 @@ pub fn detect_active_session() -> ActiveSession {
dbus_session_bus_address: dbus,
xdg_current_desktop,
},
compositor_pid: winning_pid,
}
}
@@ -435,11 +619,6 @@ pub fn apply_session_env(active: &ActiveSession) {
if let Some(d) = &e.xdg_current_desktop {
std::env::set_var("XDG_CURRENT_DESKTOP", d);
}
// Mutter on NVIDIA has no working dmabuf capture sync — force SHM there; the KWin/gamescope
// tiled/LINEAR paths keep zero-copy.
if active.kind == ActiveKind::DesktopGnome {
std::env::set_var("PUNKTFUNK_FORCE_SHM", "1");
}
// Topology (Stage 2): the per-compositor backends (KWin/Mutter) now read
// [`effective_topology`] directly at create time — the console policy, else the legacy
// `PUNKTFUNK_{KWIN,MUTTER}_VIRTUAL_PRIMARY` env, else the Auto default (exclusive on the
@@ -518,6 +697,7 @@ pub enum GamescopeMode {
/// default is a per-session bare spawn — the path that nests the client's launch command.
#[cfg(target_os = "linux")]
fn pick_gamescope_mode(
dedicated_launch: bool,
force_managed: bool,
attach_env: bool,
node_env: bool,
@@ -529,6 +709,11 @@ fn pick_gamescope_mode(
GamescopeMode::Managed
} else if attach_env || node_env {
GamescopeMode::Attach
} else if dedicated_launch {
// A dedicated game session always spawns its own headless gamescope at the client's mode,
// nesting just the game — outranking managed-infra / foreign-attach, but not the explicit
// operator MANAGED/ATTACH/NODE overrides above (debug/CI). (design/gamemode-and-dedicated-sessions.md §5.3)
GamescopeMode::Spawn
} else if session_env || managed_infra {
GamescopeMode::Managed
} else if foreign_gamescope {
@@ -548,7 +733,7 @@ fn pick_gamescope_mode(
/// nesting the session's launch command — the plain-distro default). `PUNKTFUNK_GAMESCOPE_MANAGED`
/// forces managed over all of it.
#[cfg(target_os = "linux")]
pub fn apply_input_env(chosen: Compositor) {
pub fn apply_input_env(chosen: Compositor, dedicated_launch: bool) {
let _env_guard = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let backend = match chosen {
Compositor::Gamescope => "gamescope",
@@ -562,6 +747,7 @@ pub fn apply_input_env(chosen: Compositor) {
std::env::set_var("PUNKTFUNK_INPUT_BACKEND", backend);
if chosen == Compositor::Gamescope {
let mode = pick_gamescope_mode(
dedicated_launch,
std::env::var_os("PUNKTFUNK_GAMESCOPE_MANAGED").is_some(),
std::env::var_os("PUNKTFUNK_GAMESCOPE_ATTACH").is_some(),
std::env::var_os("PUNKTFUNK_GAMESCOPE_NODE").is_some(),
@@ -593,7 +779,34 @@ pub fn apply_input_env(chosen: Compositor) {
}
}
#[cfg(not(target_os = "linux"))]
pub fn apply_input_env(_chosen: Compositor) {}
pub fn apply_input_env(_chosen: Compositor, _dedicated_launch: bool) {}
/// Should a game-launching session get a **dedicated** headless gamescope (`game_session=dedicated`
/// policy, `design/gamemode-and-dedicated-sessions.md` B0)? True only when the session carries a
/// launch, the policy selects `dedicated`, AND gamescope is actually available (else it degrades to
/// `auto` honestly). Computed at the handshake and threaded into [`apply_input_env`] /
/// [`resolve_compositor`] as a value (no new env knob — the `ENV_LOCK` discipline).
pub fn wants_dedicated_game_session(has_launch: bool) -> bool {
use policy::GameSession;
if !has_launch || policy::prefs().game_session() != GameSession::Dedicated {
return false;
}
#[cfg(target_os = "linux")]
{
if gamescope::is_available() {
true
} else {
tracing::info!(
"game_session=dedicated but gamescope is unavailable — falling back to auto routing"
);
false
}
}
#[cfg(not(target_os = "linux"))]
{
false // Windows: a launching session opens into the one desktop (no gamescope)
}
}
/// Will `vd.create` on this backend NEST the session's launch command itself (gamescope's bare
/// spawn runs it inside the new gamescope)? When true the session must NOT also spawn the command
@@ -616,6 +829,27 @@ pub fn launch_into_gamescope_session(cmd: &str) -> Result<std::process::Child> {
gamescope::launch_into_session(cmd)
}
/// B2: has a **dedicated** gamescope game session's game exited (its `node_id` doesn't reappear within a
/// short window after capture loss)? The dedicated-spawn session ends cleanly on `true` instead of the
/// capture-loss rebuild. Scoped to the session's OWN node so a coexisting gamescope doesn't mask the
/// exit (review #4/#8). Always `false` off Linux.
#[cfg(target_os = "linux")]
pub fn dedicated_game_exited(node_id: u32) -> bool {
gamescope::game_session_exited(node_id)
}
#[cfg(not(target_os = "linux"))]
pub fn dedicated_game_exited(_node_id: u32) -> bool {
false
}
/// Cancel any pending TV-session restore because a client (re)connected (review #3). No-op off Linux.
#[cfg(target_os = "linux")]
pub fn cancel_pending_tv_restore() {
gamescope::cancel_pending_restore();
}
#[cfg(not(target_os = "linux"))]
pub fn cancel_pending_tv_restore() {}
/// Detect the compositor to drive: explicit `PUNKTFUNK_COMPOSITOR` override (legacy / CI / forcing
/// a backend for a test), else the **live session** ([`detect_active_session`] — so a Bazzite box
/// follows Gaming↔Desktop switches), else a last-resort `XDG_CURRENT_DESKTOP` read.
@@ -750,6 +984,16 @@ pub fn start_restore_worker() -> std::sync::Arc<()> {
std::sync::Arc::new(())
}
/// Recover a stranded TV takeover from a crashed previous host instance
/// (`design/gamemode-and-dedicated-sessions.md` A3). Call once at `serve` startup, alongside
/// [`start_restore_worker`]. No-op when no takeover was persisted (a clean start).
#[cfg(target_os = "linux")]
pub fn restore_takeover_on_startup() {
gamescope::restore_takeover_on_startup();
}
#[cfg(not(target_os = "linux"))]
pub fn restore_takeover_on_startup() {}
// The user-configurable management policy (keep-alive / topology / conflict / identity / layout),
// layered above the per-compositor backends — platform-neutral (the mgmt API + both host paths read
// it), so no cfg gate. See `design/display-management.md`.
@@ -878,21 +1122,33 @@ mod tests {
fn gamescope_mode_ladder() {
use GamescopeMode::*;
let pick = pick_gamescope_mode;
// (force_managed, attach_env, node_env, session_env, managed_infra, foreign_gamescope)
// (dedicated_launch, force_managed, attach_env, node_env, session_env, managed_infra, foreign_gamescope)
// Plain distro, nothing running: bare spawn — the path that nests the launch command.
assert_eq!(pick(false, false, false, false, false, false), Spawn);
assert_eq!(pick(false, false, false, false, false, false, false), Spawn);
// Bazzite/SteamOS (session infra present): managed, as validated live.
assert_eq!(pick(false, false, false, false, true, false), Managed);
assert_eq!(pick(false, false, false, false, true, true), Managed);
assert_eq!(
pick(false, false, false, false, false, true, false),
Managed
);
assert_eq!(pick(false, false, false, false, false, true, true), Managed);
// Foreign gamescope on an infra-less box: attach and mirror it.
assert_eq!(pick(false, false, false, false, false, true), Attach);
assert_eq!(pick(false, false, false, false, false, false, true), Attach);
// Operator-set PUNKTFUNK_GAMESCOPE_SESSION keeps managed even without detected infra.
assert_eq!(pick(false, false, false, true, false, false), Managed);
assert_eq!(
pick(false, false, false, false, true, false, false),
Managed
);
// Explicit attach/node wins over infra…
assert_eq!(pick(false, true, false, false, true, false), Attach);
assert_eq!(pick(false, false, true, true, true, false), Attach);
assert_eq!(pick(false, false, true, false, false, true, false), Attach);
assert_eq!(pick(false, false, false, true, true, true, false), Attach);
// …and force-managed wins over everything.
assert_eq!(pick(true, true, true, false, false, false), Managed);
assert_eq!(pick(false, true, true, true, false, false, false), Managed);
// A dedicated launch forces Spawn, outranking managed-infra + foreign-attach…
assert_eq!(pick(true, false, false, false, false, true, true), Spawn);
// …but the explicit operator overrides still win over dedicated.
assert_eq!(pick(true, true, false, false, false, true, false), Managed);
assert_eq!(pick(true, false, true, false, false, false, false), Attach);
assert_eq!(pick(true, false, false, true, false, false, false), Attach);
}
#[test]
@@ -14,7 +14,7 @@
//! Input uses gamescope's own libei/EIS socket (`LIBEI_SOCKET`), relayed to the libei backend (see
//! `inject/libei.rs`) — wired and live-validated.
use super::{Mode, VirtualDisplay, VirtualOutput};
use super::{DisplayOwnership, Mode, VirtualDisplay, VirtualOutput};
use anyhow::{anyhow, bail, Context, Result};
use std::process::{Child, Command, Stdio};
use std::time::{Duration, Instant};
@@ -62,6 +62,18 @@ static PENDING_RESTORE: std::sync::Mutex<Option<Instant>> = std::sync::Mutex::ne
/// instead of triggering a stop/relaunch.
const RESTORE_DEBOUNCE: Duration = Duration::from_secs(5);
/// Per-spawn instance counter (A5): each bare-spawn gets a unique id addressing its own log so two
/// coexisting gamescopes (a kept lingering spawn + a fresh one) never parse each other's node id.
static SPAWN_SEQ: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1);
/// This spawn instance's log path, under `$XDG_RUNTIME_DIR` (per-user, tmpfs; falls back to `/tmp`
/// only if unset). Replaces the shared `/tmp/punktfunk-gamescope.log` so concurrent spawns don't
/// clobber each other's `stream available on node ID:` line.
fn spawn_log_path(inst: u64) -> std::path::PathBuf {
let base = std::env::var("XDG_RUNTIME_DIR").unwrap_or_else(|_| "/tmp".to_string());
std::path::Path::new(&base).join(format!("punktfunk-gamescope-{inst}.log"))
}
/// systemd --user transient unit name for the host-managed gamescope-session-plus session.
const SESSION_UNIT: &str = "punktfunk-gamescope";
/// The gamescope-session-plus launcher script (Bazzite / SteamOS-like hosts).
@@ -82,6 +94,80 @@ const STEAMOS_SESSION_TARGET: &str = "gamescope-session.target";
/// restart the physical session.
static STEAMOS_TOOK_OVER: std::sync::Mutex<bool> = std::sync::Mutex::new(false);
/// Persisted takeover state (`design/gamemode-and-dedicated-sessions.md` A3): the takeover mechanics
/// ([`STOPPED_AUTOLOGIN`] / [`STEAMOS_TOOK_OVER`]) are process memory, so a host **crash** mid-stream
/// would strand the box out of gaming mode with no restore. Mirroring the statics to a file lets
/// [`restore_takeover_on_startup`] put the TV back after a restart.
#[derive(serde::Serialize, serde::Deserialize, Default)]
struct TakeoverState {
/// Autologin `gamescope-session-plus@*.service` units we stopped (to restart on restore).
stopped_autologin: Vec<String>,
/// Whether we took over SteamOS's `gamescope-session.target` (restore = remove drop-in + restart).
steamos: bool,
}
/// Path of the persisted [`TakeoverState`], under `$XDG_RUNTIME_DIR` (per-user, 0700, tmpfs — cleared
/// on reboot, which is correct: a reboot restarts the autologin itself).
fn takeover_state_path() -> std::path::PathBuf {
let base = std::env::var("XDG_RUNTIME_DIR").unwrap_or_else(|_| "/tmp".to_string());
std::path::Path::new(&base).join("punktfunk-session-takeover.json")
}
/// Persist the current takeover mechanics so a host crash doesn't strand the box out of gaming mode.
/// Best-effort (a write failure just loses crash-restore, not correctness).
fn persist_takeover() {
let state = TakeoverState {
stopped_autologin: STOPPED_AUTOLOGIN
.lock()
.unwrap_or_else(|e| e.into_inner())
.clone(),
steamos: *STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()),
};
if state.stopped_autologin.is_empty() && !state.steamos {
clear_takeover();
return;
}
if let Ok(bytes) = serde_json::to_vec(&state) {
let _ = std::fs::write(takeover_state_path(), bytes);
}
}
/// Remove the persisted takeover file (after a completed restore, or when there's nothing to restore).
fn clear_takeover() {
let _ = std::fs::remove_file(takeover_state_path());
}
/// On host startup, restore the TV's gaming session if a previous host instance took it over and
/// crashed before restoring (`design/gamemode-and-dedicated-sessions.md` A3). Loads the persisted
/// [`TakeoverState`] into the statics and schedules a restore after a short reconnect grace (so a
/// client reconnecting right after the restart keeps the streamed session instead of bouncing the
/// box back to gaming mode). No-op when no takeover file exists (a clean start). Call once from
/// `serve` alongside [`start_restore_worker`].
pub fn restore_takeover_on_startup() {
let Ok(bytes) = std::fs::read(takeover_state_path()) else {
return; // no takeover file — clean start
};
let Ok(state) = serde_json::from_slice::<TakeoverState>(&bytes) else {
clear_takeover();
return;
};
if state.stopped_autologin.is_empty() && !state.steamos {
clear_takeover();
return;
}
tracing::warn!(
units = ?state.stopped_autologin,
steamos = state.steamos,
"gamescope: found a stranded takeover from a previous host instance — scheduling TV restore"
);
*STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()) = state.stopped_autologin;
*STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()) = state.steamos;
// A generous grace so a client reconnecting right after the restart cancels it (create_managed_session
// clears PENDING_RESTORE) and keeps the streamed session rather than bouncing to gaming mode.
*PENDING_RESTORE.lock().unwrap_or_else(|e| e.into_inner()) =
Some(Instant::now() + Duration::from_secs(15));
}
impl GamescopeDisplay {
pub fn new() -> Result<Self> {
Ok(GamescopeDisplay::default())
@@ -97,6 +183,32 @@ impl VirtualDisplay for GamescopeDisplay {
self.cmd = cmd;
}
fn poolable_now(&self) -> bool {
// Only a bare SPAWN is registry-poolable (its `create` reports `Owned`); managed
// (`PUNKTFUNK_GAMESCOPE_SESSION`) and attach (`PUNKTFUNK_GAMESCOPE_NODE`) report
// `SessionManaged`/`External`, so the registry must not reuse a kept spawn for them (same
// backend name). Mirrors [`crate::vdisplay::launch_is_nested`]; read under the env lock the
// sub-mode ladder writes these keys under.
crate::vdisplay::with_env_lock(|| {
std::env::var_os("PUNKTFUNK_GAMESCOPE_SESSION").is_none()
&& std::env::var_os("PUNKTFUNK_GAMESCOPE_NODE").is_none()
})
}
fn launch_command(&self) -> Option<String> {
// The registry keys keep-alive reuse on (backend, mode, launch): a kept bare-spawn running
// game A must never be reused for a session launching game B (A2).
self.cmd.clone()
}
fn kept_display_alive(&mut self, node_id: u32) -> bool {
// The nested gamescope dies when its game exits (independently of any compositor), leaving a
// dead pooled node. Before the registry reuses that node on a reconnect, confirm it still
// exists on the daemon; a `false` makes the registry recreate instead of handing back a corpse
// (which would then burn a ~10 s first-frame retry before `mark_failed` recovered it).
gamescope_node_present(node_id)
}
fn create(&mut self, mode: Mode) -> Result<VirtualOutput> {
// Host-managed gamescope-session-plus at the CLIENT's mode (the Bazzite path): launch the
// full Steam-Deck-UI session headless at the client's resolution + refresh — so games SEE
@@ -121,26 +233,51 @@ impl VirtualDisplay for GamescopeDisplay {
};
point_injector_at_eis();
tracing::info!(node_id, "gamescope: attaching to existing PipeWire node");
// ATTACH = mirror a foreign gamescope we don't own → External (no keep-alive/reuse).
return Ok(VirtualOutput {
node_id,
remote_fd: None,
preferred_mode: Some((mode.width, mode.height, mode.refresh_hz)),
keepalive: Box::new(()),
ownership: DisplayOwnership::External,
reused_gen: None,
});
}
check_gamescope_version(); // diagnostic only — warns on known-deadlock-prone versions
let proc = GamescopeProc(spawn(
// B1: a dedicated STEAM launch needs Steam's single instance free. If the box autologged into
// game mode (Bazzite) its Steam holds the instance, and a nested second Steam would see the
// first and exit (crashing the spawn) — so free the autologin session first. Its restore is the
// A3 takeover machinery (recorded in STOPPED_AUTOLOGIN + persisted; restarted on session end via
// schedule_restore_tv_session). Non-Steam launches don't conflict, so they skip this.
if self.cmd.as_deref().is_some_and(is_steam_launch) {
stop_autologin_sessions();
}
// A5: a per-spawn instance id addresses this spawn's log + node discovery, so two coexisting
// bare-spawns (a kept lingering one + a fresh one) never parse each other's node id from a
// shared log. The nested-command's LIBEI relay stays on the global path (per-instance input
// isolation is `design/gamescope-multiuser.md` scope, not addressed here).
let inst = SPAWN_SEQ.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let log = spawn_log_path(inst);
let child = spawn(
mode.width,
mode.height,
mode.refresh_hz.max(1),
self.cmd.as_deref(),
)?);
&log,
)?;
let child_pid = child.id();
let proc = GamescopeProc {
child,
log: log.clone(),
};
// gamescope creates its PipeWire node a moment after start; poll for it (the proc is held
// alive meanwhile, and killed if we give up).
let node_id = wait_for_node(Duration::from_secs(15)).ok_or_else(|| {
// alive meanwhile, and killed if we give up). Discovery reads THIS spawn's log, and the
// fallback is scoped to this spawn's process tree.
let node_id = wait_for_node(Duration::from_secs(15), &log, child_pid).ok_or_else(|| {
anyhow!(
"gamescope PipeWire node did not appear within 15s — gamescope may have failed to \
start or headless capture is unsupported on this GPU/driver (see /tmp/punktfunk-gamescope.log)"
start or headless capture is unsupported on this GPU/driver (see {})",
log.display()
)
})?;
tracing::info!(
@@ -150,12 +287,12 @@ impl VirtualDisplay for GamescopeDisplay {
hz = mode.refresh_hz,
"gamescope virtual output ready"
);
Ok(VirtualOutput {
// Bare SPAWN: we own the nested gamescope process → registry-poolable (keep-alive-able).
Ok(VirtualOutput::owned(
node_id,
remote_fd: None,
preferred_mode: Some((mode.width, mode.height, mode.refresh_hz)),
keepalive: Box::new(proc),
})
Some((mode.width, mode.height, mode.refresh_hz)),
Box::new(proc),
))
}
}
@@ -192,12 +329,7 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
hz = mode.refresh_hz,
"gamescope session: reusing the running session (same mode — no Steam restart)"
);
return Ok(VirtualOutput {
node_id,
remote_fd: None,
preferred_mode: Some((mode.width, mode.height, mode.refresh_hz)),
keepalive: Box::new(()),
});
return Ok(managed_output(node_id, mode));
}
tracing::warn!("gamescope session: tracked session has no live node — relaunching");
*guard = None;
@@ -218,12 +350,23 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
hz = mode.refresh_hz,
"gamescope session: launched gamescope-session-plus at the client's mode"
);
Ok(VirtualOutput {
Ok(managed_output(node_id, mode))
}
/// The [`VirtualOutput`] for a managed / SteamOS-takeover session: a box-level session whose restore
/// lifecycle is (at Part A1) the gamescope module's own machinery (`schedule_restore_tv_session`), so
/// it is [`DisplayOwnership::SessionManaged`] — the registry passes it through (no pooling), and the
/// capturer's unit keepalive tears nothing down on drop. (Part A3 replaces the unit keepalive with a
/// real `ManagedSessionHandle` and flips this to `Owned`.)
fn managed_output(node_id: u32, mode: Mode) -> VirtualOutput {
VirtualOutput {
node_id,
remote_fd: None,
preferred_mode: Some((mode.width, mode.height, mode.refresh_hz)),
keepalive: Box::new(()),
})
ownership: DisplayOwnership::SessionManaged,
reused_gen: None,
}
}
/// SteamOS detection: its session launcher is present and Bazzite's session-plus is NOT (so the
@@ -483,12 +626,7 @@ fn create_managed_session_steamos(mode: Mode) -> Result<VirtualOutput> {
hz = mode.refresh_hz,
"gamescope (SteamOS): reusing the headless session (same mode — no Steam restart)"
);
return Ok(VirtualOutput {
node_id,
remote_fd: None,
preferred_mode: Some((mode.width, mode.height, mode.refresh_hz)),
keepalive: Box::new(()),
});
return Ok(managed_output(node_id, mode));
}
*guard = None; // tracked session lost its node — fall through to a clean restart
}
@@ -497,9 +635,11 @@ fn create_managed_session_steamos(mode: Mode) -> Result<VirtualOutput> {
systemctl_user(&["daemon-reload"]);
systemctl_user(&["restart", STEAMOS_SESSION_TARGET]);
*STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()) = true;
// gamescope's node appears within a few seconds of the restart; Steam's first FRAME is slower
// (Big Picture cold start) and is awaited by the caller's first-frame retry loop.
let node_id = wait_for_node(Duration::from_secs(30)).ok_or_else(|| {
persist_takeover(); // A3: survive a host crash mid-stream
// gamescope's node appears within a few seconds of the restart; Steam's first FRAME is slower
// (Big Picture cold start) and is awaited by the caller's first-frame retry loop. The managed
// session logs to journald (not a per-spawn file), so poll `find_gamescope_node` directly.
let node_id = poll_managed_node(Duration::from_secs(30)).ok_or_else(|| {
anyhow!(
"SteamOS headless gamescope node did not appear within 30s after restarting \
{STEAMOS_SESSION_TARGET} check `journalctl --user -u gamescope-session.service`"
@@ -518,12 +658,7 @@ fn create_managed_session_steamos(mode: Mode) -> Result<VirtualOutput> {
hz = mode.refresh_hz,
"gamescope (SteamOS): took over gamescope-session.target headless at the client's mode"
);
Ok(VirtualOutput {
node_id,
remote_fd: None,
preferred_mode: Some((mode.width, mode.height, mode.refresh_hz)),
keepalive: Box::new(()),
})
Ok(managed_output(node_id, mode))
}
/// ATTACH at the CLIENT's resolution: ensure the box's own game-mode session is running at `mode`'s
@@ -670,11 +805,31 @@ fn running_autologin_gamescope_unit() -> Option<String> {
.map(|u| u.to_string())
}
/// Tear a gamescope `systemd --user` unit down with **SIGKILL** rather than the default SIGTERM stop
/// (`design/gamemode-and-dedicated-sessions.md` A3 / `session-aware-host-followups.md` #1): the
/// hypothesis — validated as the fix on the F44 repro box `.181` — is that gamescope's SIGTERM
/// teardown handler (the one that SIGSEGVs, exit 139) LEAKS the NVIDIA GPU context, after which every
/// subsequent gamescope fails `vkCreateDevice` with `VK_ERROR_INITIALIZATION_FAILED` (-3) until a
/// reboot. SIGKILL skips that handler so the driver reclaims the context cleanly via normal process
/// exit. Follow with `stop` + `reset-failed` to clear the unit's state so a relaunch is clean.
fn kill_unit(unit: &str) {
let _ = Command::new("systemctl")
.args(["--user", "kill", "--signal=SIGKILL", unit])
.status();
let _ = Command::new("systemctl")
.args(["--user", "stop", unit])
.status();
let _ = Command::new("systemctl")
.args(["--user", "reset-failed", unit])
.status();
}
/// Stop every running autologin gaming-mode session (`gamescope-session-plus@*.service`) so its
/// single-instance Steam is free for our own host-managed session. Records the units so
/// [`schedule_restore_tv_session`] can restart them on disconnect. Our own session is the transient
/// `punktfunk-gamescope` unit (not a `@`-instance), so it's never matched here. No-op when nothing
/// is autologged in (e.g. a box that boots headless).
/// is autologged in (e.g. a box that boots headless). Uses the **SIGKILL** teardown ([`kill_unit`])
/// to avoid the F44 GPU-context leak that the autologin's SIGTERM stop triggers.
fn stop_autologin_sessions() {
let Ok(out) = Command::new("systemctl")
.args([
@@ -694,12 +849,10 @@ fn stop_autologin_sessions() {
for line in String::from_utf8_lossy(&out.stdout).lines() {
if let Some(unit) = line.split_whitespace().next() {
if unit.starts_with("gamescope-session-plus@") && unit.ends_with(".service") {
let _ = Command::new("systemctl")
.args(["--user", "stop", unit])
.status();
kill_unit(unit); // SIGKILL teardown — avoid the F44 GPU-context leak
tracing::info!(
unit,
"freed Steam: stopped the autologin gaming session for this stream"
"freed Steam: SIGKILL-stopped the autologin gaming session for this stream"
);
stopped.push(unit.to_string());
}
@@ -707,15 +860,57 @@ fn stop_autologin_sessions() {
}
if !stopped.is_empty() {
*STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()) = stopped;
persist_takeover(); // A3: survive a host crash mid-stream
}
}
/// Client disconnected: **schedule** a debounced restore of the TV's autologin gaming session(s) we
/// stopped on connect — the actual restore fires [`RESTORE_DEBOUNCE`] later (via [`start_restore_worker`])
/// unless a client reconnects first, which cancels it and reuses the warm managed session. Debouncing
/// means at most one gamescope stop/relaunch per quiet period instead of one per disconnect — the
/// per-connect churn is what leaked GPU context on F44. No-op when nothing was stolen (non-Bazzite /
/// headless box). Idempotent / safe to call on every session end.
/// Cancel any pending TV-session restore — a client has (re)connected, so the box must stay in the
/// streamed session, not bounce back to gaming mode. This covers the **keep-alive reuse** reconnect
/// path (a kept dedicated / managed gamescope), which never calls `create_managed_session` (where the
/// managed path already clears `PENDING_RESTORE`) — so without this, a dedicated Steam reconnect within
/// the linger window would restart the autologin *underneath* the live session (review finding #3).
/// Called from the connect path (native `resolve_compositor`, GameStream `open_gs_virtual_source`).
/// No-op when nothing is pending; the stopped-unit list stays armed for a later real disconnect.
pub fn cancel_pending_restore() {
let mut g = PENDING_RESTORE.lock().unwrap_or_else(|e| e.into_inner());
if g.is_some() {
*g = None;
tracing::info!(
"gamescope: client (re)connected — cancelled the pending TV-session restore"
);
}
}
/// The delay before restoring the TV's autologin session after the last client disconnects — the
/// display-management **keep-alive policy**, replacing the hardcoded [`RESTORE_DEBOUNCE`]
/// (`design/gamemode-and-dedicated-sessions.md` A3). The managed gamescope session is a single
/// box-level singleton (not a registry pool entry — A1), so its keep-alive lives here rather than in
/// the registry, but reads the same policy the pooled backends do:
/// * `off` → restore immediately (0 s);
/// * `duration(s)` → restore after `s`;
/// * `forever` → **`None`**: never auto-restore — the managed session is HELD until host stop or a
/// manual return to gaming mode (the `gaming-rig` "the TV model" story, now truthful on gamescope);
/// * unconfigured → the historical 5 s [`RESTORE_DEBOUNCE`] (bit-for-bit today's behavior).
fn restore_delay() -> Option<Duration> {
use crate::vdisplay::policy::{self, Linger};
match policy::prefs()
.configured_effective()
.map(|e| e.keep_alive.linger())
{
Some(Linger::Immediate) => Some(Duration::from_secs(0)),
Some(Linger::For(d)) => Some(d),
Some(Linger::Forever) => None,
None => Some(RESTORE_DEBOUNCE),
}
}
/// Client disconnected: **schedule** a policy-timed restore of the TV's autologin gaming session(s) we
/// stopped on connect ([`restore_delay`], via [`start_restore_worker`]) — unless a client reconnects
/// first, which cancels it and reuses the warm managed session. Debouncing means at most one gamescope
/// stop/relaunch per quiet period instead of one per disconnect — the per-connect churn is what leaked
/// GPU context on F44. Under `keep_alive=forever` ([`restore_delay`] `None`) NO restore is scheduled:
/// the managed session is pinned (gaming-rig). No-op when nothing was stolen (non-Bazzite / headless
/// box). Idempotent / safe to call on every session end.
pub fn schedule_restore_tv_session() {
let nothing_to_restore = STOPPED_AUTOLOGIN
.lock()
@@ -725,12 +920,24 @@ pub fn schedule_restore_tv_session() {
if nothing_to_restore {
return; // nothing was taken over → nothing to restore (also the non-managed path)
}
*PENDING_RESTORE.lock().unwrap_or_else(|e| e.into_inner()) =
Some(Instant::now() + RESTORE_DEBOUNCE);
tracing::info!(
secs = RESTORE_DEBOUNCE.as_secs(),
"gamescope: scheduled debounced TV-session restore (cancelled if a client reconnects)"
);
match restore_delay() {
None => {
// keep_alive=forever → pin the managed session; leave PENDING_RESTORE unset.
*PENDING_RESTORE.lock().unwrap_or_else(|e| e.into_inner()) = None;
tracing::info!(
"gamescope: keep-alive=forever — managed session held (no TV-restore scheduled; \
return to gaming mode or restart the host to free it)"
);
}
Some(delay) => {
*PENDING_RESTORE.lock().unwrap_or_else(|e| e.into_inner()) =
Some(Instant::now() + delay);
tracing::info!(
secs = delay.as_secs(),
"gamescope: scheduled TV-session restore (keep-alive policy; cancelled on reconnect)"
);
}
}
}
/// Tear down our host-managed session (freeing Steam) and restart the autologin gaming session(s)
@@ -745,6 +952,7 @@ fn do_restore_tv_session() {
let mut took = STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner());
if *took {
*took = false;
clear_takeover(); // A3: takeover undone — drop the persisted crash-restore marker
*MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None;
remove_steamos_dropin();
systemctl_user(&["daemon-reload"]);
@@ -770,6 +978,7 @@ fn do_restore_tv_session() {
if units.is_empty() {
return; // nothing was stolen → nothing to restore (also the non-Bazzite path)
}
clear_takeover(); // A3: takeover consumed — drop the persisted crash-restore marker
stop_session(SESSION_UNIT); // our gamescope/Steam session, so Steam is free for the autologin
*MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None;
// Only bring the gaming autologin BACK if the box is still meant to be in gaming mode. If the
@@ -923,12 +1132,10 @@ fn launch_session(client: &str, unit_name: &str, mode: Mode) -> Result<u32> {
}
}
/// Stop the host-managed session's transient unit (best-effort) and clear the EIS relay so a dead
/// session's socket name can't be reconnected.
/// Stop the host-managed session's transient unit ([`kill_unit`] — SIGKILL teardown to avoid the F44
/// GPU-context leak) and clear the EIS relay so a dead session's socket name can't be reconnected.
fn stop_session(unit_name: &str) {
let _ = Command::new("systemctl")
.args(["--user", "stop", unit_name])
.status();
kill_unit(unit_name);
let _ = std::fs::remove_file(ei_socket_file());
}
@@ -949,13 +1156,36 @@ pub fn ei_socket_file() -> std::path::PathBuf {
}
}
/// Shape a resolved launch command for a bare-spawn gamescope session. A Steam URI launch
/// (`steam steam://rungameid/<id>`, produced by `library::command_for`) gets `-silent` inserted so
/// the game is the gamescope focus with no Steam client window to navigate
/// (`design/gamemode-and-dedicated-sessions.md` §5.3). Operator-typed custom commands and non-Steam
/// launches are returned unchanged. Idempotent (never double-inserts `-silent`). Pure + unit-tested.
/// Does this resolved launch command start Steam (`steam … steam://…`)? Such a launch needs Steam's
/// single instance free before a dedicated spawn (B1). Pure + unit-tested.
fn is_steam_launch(cmd: &str) -> bool {
let mut it = cmd.split_whitespace();
it.next() == Some("steam") && cmd.contains("steam://")
}
fn shape_dedicated_command(app: &str) -> String {
let mut it = app.split_whitespace();
if it.next() == Some("steam") {
let rest: Vec<&str> = it.collect();
if !rest.contains(&"-silent") && rest.iter().any(|t| t.starts_with("steam://")) {
return format!("steam -silent {}", rest.join(" "));
}
}
app.to_string()
}
/// Spawn `gamescope --backend headless -W w -H h -r hz -- <app>`. The app comes from
/// `PUNKTFUNK_GAMESCOPE_APP` (default a no-op that just keeps gamescope alive — set it to a real
/// game/GL app for actual content, e.g. `steam -gamepadui` for the SteamOS-like session).
/// stdout/stderr go to `/tmp/punktfunk-gamescope.log`. The app is launched through a tiny shell
/// wrapper that relays gamescope's `LIBEI_SOCKET` (set for its children) to [`ei_socket_file`]
/// stdout/stderr go to `log` (this spawn's per-instance log, A5). The app is launched through a tiny
/// shell wrapper that relays gamescope's `LIBEI_SOCKET` (set for its children) to [`ei_socket_file`]
/// so the input injector can connect to gamescope's EIS server from outside.
fn spawn(w: u32, h: u32, hz: u32, cmd: Option<&str>) -> Result<Child> {
fn spawn(w: u32, h: u32, hz: u32, cmd: Option<&str>, log: &std::path::Path) -> Result<Child> {
// A non-empty per-session command (set via `set_launch_command`) wins; else the
// `PUNKTFUNK_GAMESCOPE_APP` env var (the documented manual fallback); else a no-op that keeps
// gamescope alive. Each level is taken only if non-empty, so a blank per-session cmd transparently
@@ -970,6 +1200,9 @@ fn spawn(w: u32, h: u32, hz: u32, cmd: Option<&str>) -> Result<Child> {
})
.filter(|s| !s.trim().is_empty())
.unwrap_or_else(|| "sleep infinity".to_string());
// Dedicated-launch command shaping (Part B): a Steam URI runs with `-silent` so the game is the
// gamescope focus with no Steam client window to navigate.
let app = shape_dedicated_command(&app);
let relay = ei_socket_file();
let _ = std::fs::remove_file(&relay); // stale socket path from a previous session
let mut cmd = Command::new("gamescope");
@@ -990,14 +1223,14 @@ fn spawn(w: u32, h: u32, hz: u32, cmd: Option<&str>) -> Result<Child> {
.args(app.split_whitespace())
// Prefer the NVIDIA GL vendor for the nested session (harmless on a pure-NVIDIA box).
.env("__GLX_VENDOR_LIBRARY_NAME", "nvidia");
if let Ok(log) = std::fs::File::create("/tmp/punktfunk-gamescope.log") {
if let Ok(log2) = log.try_clone() {
cmd.stdout(Stdio::from(log)).stderr(Stdio::from(log2));
if let Ok(logf) = std::fs::File::create(log) {
if let Ok(log2) = logf.try_clone() {
cmd.stdout(Stdio::from(logf)).stderr(Stdio::from(log2));
}
} else {
cmd.stdout(Stdio::null()).stderr(Stdio::null());
}
tracing::info!(w, h, hz, %app, "spawning gamescope (headless)");
tracing::info!(w, h, hz, %app, log = %log.display(), "spawning gamescope (headless)");
cmd.spawn()
.context("spawn gamescope (is it installed? `apt install gamescope`)")
}
@@ -1006,22 +1239,59 @@ fn spawn(w: u32, h: u32, hz: u32, cmd: Option<&str>) -> Result<Child> {
/// line `stream available on node ID: N` (its node carries `node.name=gamescope` on TWO objects
/// — the adapter and the inner stream — and only the advertised id is the correct capture
/// target). Falls back to `pw-dump` discovery if the log line doesn't show.
fn wait_for_node(timeout: Duration) -> Option<u32> {
/// B2 (game-exit detection): confirm a **dedicated** gamescope session's game has exited. gamescope is
/// a single-app compositor — it exits when its nested app exits — so once capture is lost, THIS
/// session's `node_id` not reappearing within a short confirmation window means the game quit (vs. a
/// transient PipeWire hiccup). Scoped to the session's own `node_id` (via [`gamescope_node_present`]),
/// so a **coexisting** gamescope (a second dedicated session, or the box's game-mode gamescope beside a
/// non-Steam dedicated launch) doesn't mask the exit (review findings #4/#8). Returns `true` when the
/// node stays absent across the window.
pub fn game_session_exited(node_id: u32) -> bool {
let deadline = Instant::now() + Duration::from_millis(1500);
loop {
if gamescope_node_present(node_id) {
return false; // OUR node is (still) present → not an exit (transient loss)
}
if Instant::now() >= deadline {
return true; // our node stayed gone across the window → the game exited
}
std::thread::sleep(Duration::from_millis(250));
}
}
/// Poll [`find_gamescope_node`] (unscoped) up to `timeout` — for the managed / SteamOS session, which
/// logs to journald (no per-spawn file) and is single-session (no scoping needed).
fn poll_managed_node(timeout: Duration) -> Option<u32> {
let deadline = Instant::now() + timeout;
loop {
if let Some(id) = node_from_log() {
if let Some(id) = find_gamescope_node() {
return Some(id);
}
if Instant::now() >= deadline {
return find_gamescope_node(); // last-resort fallback
return None;
}
std::thread::sleep(Duration::from_millis(300));
}
}
/// Parse `stream available on node ID: N` from the spawned gamescope's log (ANSI-colored).
fn node_from_log() -> Option<u32> {
let log = std::fs::read_to_string("/tmp/punktfunk-gamescope.log").ok()?;
fn wait_for_node(timeout: Duration, log: &std::path::Path, child_pid: u32) -> Option<u32> {
let deadline = Instant::now() + timeout;
loop {
if let Some(id) = node_from_log(log) {
return Some(id);
}
if Instant::now() >= deadline {
// Last-resort fallback scoped to THIS spawn's process tree (A5), so a coexisting gamescope's
// node isn't picked by mistake.
return find_gamescope_node_scoped(Some(child_pid));
}
std::thread::sleep(Duration::from_millis(300));
}
}
/// Parse `stream available on node ID: N` from a spawned gamescope's per-instance log (ANSI-colored).
fn node_from_log(log: &std::path::Path) -> Option<u32> {
let log = std::fs::read_to_string(log).ok()?;
for line in log.lines().rev() {
if let Some(pos) = line.find("stream available on node ID:") {
let tail = &line[pos + "stream available on node ID:".len()..];
@@ -1034,6 +1304,27 @@ fn node_from_log() -> Option<u32> {
None
}
/// Is a PipeWire node with exactly `node_id` present on the default daemon right now? Used by the
/// keep-alive reuse liveness probe ([`GamescopeDisplay::kept_display_alive`]): a kept gamescope node
/// vanishes when its nested game exits, so a missing id means "recreate, don't reuse the corpse".
fn gamescope_node_present(node_id: u32) -> bool {
let Ok(out) = Command::new("pw-dump").arg(node_id.to_string()).output() else {
// pw-dump unavailable → don't block reuse (mark_failed is the backstop on a genuinely dead node).
return true;
};
let Ok(dump) = serde_json::from_slice::<serde_json::Value>(&out.stdout) else {
return true;
};
dump.as_array()
.map(|objs| {
objs.iter().any(|o| {
o.get("id").and_then(|i| i.as_u64()) == Some(node_id as u64)
&& o.get("type").and_then(|t| t.as_str()) == Some("PipeWire:Interface:Node")
})
})
.unwrap_or(true)
}
/// Find the `gamescope` `Video/Source` node id in a `pw-dump` snapshot of the default daemon.
///
/// `node.name=gamescope` appears on TWO objects (the adapter *and* the inner stream node); only
@@ -1041,10 +1332,18 @@ fn node_from_log() -> Option<u32> {
/// other wedges the link. So we require `Video/Source` first and fall back to a bare name match
/// only if no class-tagged node is present (older gamescope that doesn't set media.class).
fn find_gamescope_node() -> Option<u32> {
find_gamescope_node_scoped(None)
}
/// Like [`find_gamescope_node`], but when `scope` is `Some(pid)` only a node whose owning process
/// (`application.process.id`) is `pid` or a descendant of it qualifies (A5 — a spawn's node must
/// belong to OUR gamescope's process tree, so a coexisting foreign / other-session gamescope node is
/// never mistaken for ours). `None` = any gamescope node (the managed/attach paths, single-session).
fn find_gamescope_node_scoped(scope: Option<u32>) -> Option<u32> {
let out = Command::new("pw-dump").output().ok()?;
let dump: serde_json::Value = serde_json::from_slice(&out.stdout).ok()?;
let nodes = dump.as_array()?;
let node_props = |obj: &serde_json::Value| -> Option<(u32, String, String)> {
let node_props = |obj: &serde_json::Value| -> Option<(u32, String, String, Option<u32>)> {
if obj.get("type").and_then(|t| t.as_str()) != Some("PipeWire:Interface:Node") {
return None;
}
@@ -1060,20 +1359,40 @@ fn find_gamescope_node() -> Option<u32> {
.and_then(|n| n.as_str())
.unwrap_or("")
.to_string();
Some((id, name, class))
// PipeWire records the owning process id as a string or an int depending on version.
let pid = props
.and_then(|p| p.get("application.process.id"))
.and_then(|v| {
v.as_u64()
.or_else(|| v.as_str().and_then(|s| s.parse().ok()))
.map(|n| n as u32)
});
Some((id, name, class, pid))
};
// Preferred: a Video/Source node named (or containing) "gamescope".
// A node is in-scope when no scope is asked, or its owning pid descends from the scope pid. When
// the pid prop is absent (older gamescope / PipeWire) we DON'T exclude it — falling back to the
// per-instance log is the primary addressing (design §7 risk note).
let in_scope = |pid: Option<u32>| -> bool {
match scope {
None => true,
Some(root) => pid.map(|p| descends_from(p, root)).unwrap_or(true),
}
};
// Preferred: a Video/Source node named (or containing) "gamescope", in scope.
for obj in nodes {
if let Some((id, name, class)) = node_props(obj) {
if class == "Video/Source" && (name == "gamescope" || name.contains("gamescope")) {
if let Some((id, name, class, pid)) = node_props(obj) {
if class == "Video/Source"
&& (name == "gamescope" || name.contains("gamescope"))
&& in_scope(pid)
{
return Some(id);
}
}
}
// Fallback: a node literally named "gamescope" with no usable class tag.
// Fallback: a node literally named "gamescope" with no usable class tag, in scope.
for obj in nodes {
if let Some((id, name, _)) = node_props(obj) {
if name == "gamescope" {
if let Some((id, name, _, pid)) = node_props(obj) {
if name == "gamescope" && in_scope(pid) {
tracing::warn!(
node_id = id,
"gamescope node has no media.class=Video/Source tag — capturing it anyway"
@@ -1168,22 +1487,62 @@ fn parse_version(text: &str) -> Option<(u32, u32, u32)> {
None
}
/// Owns the spawned gamescope process; killing it tears the virtual output down.
struct GamescopeProc(Child);
/// Owns the spawned gamescope process (and its per-instance log, A5); killing it tears the virtual
/// output down.
struct GamescopeProc {
child: Child,
log: std::path::PathBuf,
}
impl Drop for GamescopeProc {
fn drop(&mut self) {
let _ = self.0.kill();
let _ = self.0.wait();
let _ = self.child.kill();
let _ = self.child.wait();
// Clear the relayed EIS socket name so the host-lifetime injector can't reconnect to this
// now-dead session's socket between sessions (the stale path is the "Connection refused").
let _ = std::fs::remove_file(ei_socket_file());
// Drop this spawn's per-instance log (A5) so `$XDG_RUNTIME_DIR` doesn't accumulate them.
let _ = std::fs::remove_file(&self.log);
}
}
#[cfg(test)]
mod tests {
use super::{parse_version, MIN_GAMESCOPE};
use super::{is_steam_launch, parse_version, shape_dedicated_command, MIN_GAMESCOPE};
#[test]
fn steam_launch_detection() {
assert!(is_steam_launch("steam steam://rungameid/570"));
assert!(is_steam_launch("steam -silent steam://rungameid/570"));
assert!(!is_steam_launch("vkcube"));
assert!(!is_steam_launch("lutris lutris:rungameid/42"));
assert!(!is_steam_launch("steam -bigpicture")); // no URI = not a game launch
}
#[test]
fn dedicated_command_shaping() {
// Steam URI → -silent inserted so the game is the gamescope focus.
assert_eq!(
shape_dedicated_command("steam steam://rungameid/570"),
"steam -silent steam://rungameid/570"
);
// Idempotent: an already-silent command is left alone.
assert_eq!(
shape_dedicated_command("steam -silent steam://rungameid/570"),
"steam -silent steam://rungameid/570"
);
// Non-Steam launches and operator custom commands are untouched.
assert_eq!(shape_dedicated_command("vkcube"), "vkcube");
assert_eq!(
shape_dedicated_command("lutris lutris:rungameid/42"),
"lutris lutris:rungameid/42"
);
// A bare `steam` with no URI is left alone (not a game launch).
assert_eq!(
shape_dedicated_command("steam -bigpicture"),
"steam -bigpicture"
);
}
#[test]
fn parses_version_banner() {
@@ -212,31 +212,46 @@ impl VirtualDisplay for KwinDisplay {
});
// Layout position (§6.2) is applied by the registry via `apply_position` right after create
// (it owns the display group, so it computes auto-row / manual placement over the whole group).
Ok(VirtualOutput {
Ok(VirtualOutput::owned(
node_id,
remote_fd: None,
preferred_mode: Some((mode.width, mode.height, achieved_hz)),
keepalive: Box::new(StopGuard { stop }),
})
Some((mode.width, mode.height, achieved_hz)),
Box::new(StopGuard { stop }),
))
}
}
/// Re-enable the outputs an `exclusive` topology disabled (bootstrap / physical), so KWin re-homes onto
/// them. Called by the registry when the display group's last member is torn down (design §6.1), BEFORE
/// that member's output is reclaimed — so KWin is never momentarily left with zero enabled outputs.
fn reenable_outputs(outputs: &[String]) {
fn reenable_outputs(outputs: &[(String, String)]) {
if outputs.is_empty() {
return;
}
let args: Vec<String> = outputs
// Enable FIRST, as a standalone apply — a bare `output.X.enable` always succeeds, so a physical
// can never be left DARK. (Batching a possibly-stale `mode` arg into the same invocation risks
// kscreen-doctor rejecting the whole config and leaving the output disabled.)
let enable_args: Vec<String> = outputs
.iter()
.map(|o| format!("output.{o}.enable"))
.map(|(name, _)| format!("output.{name}.enable"))
.collect();
let _ = std::process::Command::new("kscreen-doctor")
.args(&args)
.args(&enable_args)
.status();
// THEN re-assert each captured mode, best-effort — a bare re-enable lets KWin fall back to the
// EDID-preferred mode (a 120 Hz panel returns at ~60 Hz); this restores the exact refresh. The
// output is enabled now, so the mode set is valid; a rejected mode just leaves KWin's default.
let mode_args: Vec<String> = outputs
.iter()
.filter(|(_, mode)| !mode.is_empty())
.map(|(name, mode)| format!("output.{name}.mode.{mode}"))
.collect();
if !mode_args.is_empty() {
let _ = std::process::Command::new("kscreen-doctor")
.args(&mode_args)
.status();
}
std::thread::sleep(Duration::from_millis(200));
tracing::info!(reenabled = ?outputs, "KWin: restored the physical/bootstrap outputs (group empty)");
tracing::info!(reenabled = ?outputs, "KWin: restored the physical/bootstrap outputs at their captured modes (group empty)");
}
/// Best-effort: raise the just-created virtual output's refresh above KWin's default 60 Hz by
@@ -328,12 +343,39 @@ fn read_active_refresh(output: &str) -> Option<u32> {
/// recognised by this prefix, so we never have to thread the live set through the backend.
const MANAGED_PREFIX: &str = "Virtual-punktfunk";
/// Names of currently-ENABLED outputs that are **not managed by us** — the headless session's
/// bootstrap output(s) + any physical monitor, i.e. exactly what `exclusive` must disable.
/// The current mode of an output as a kscreen-doctor mode setter, from its `-j` entry — preferring
/// the human `WxH@Hz` form (survives a mode-id re-enumeration across disable→enable) and falling back
/// to the raw `currentModeId`. `None` if the current mode can't be resolved.
fn output_current_mode_spec(o: &serde_json::Value) -> Option<String> {
let as_id = |v: &serde_json::Value| -> Option<String> {
v.as_str()
.map(|s| s.to_string())
.or_else(|| v.as_u64().map(|n| n.to_string()))
};
let current = o.get("currentModeId").and_then(&as_id)?;
let mode = o
.get("modes")?
.as_array()?
.iter()
.find(|m| m.get("id").and_then(&as_id).as_deref() == Some(current.as_str()))?;
let human = (|| {
let size = mode.get("size")?;
let w = size.get("width").and_then(|v| v.as_u64())?;
let h = size.get("height").and_then(|v| v.as_u64())?;
let hz = mode.get("refreshRate").and_then(|r| r.as_f64())?.round() as u64;
Some(format!("{w}x{h}@{hz}"))
})();
Some(human.unwrap_or(current))
}
/// Currently-ENABLED outputs that are **not managed by us** — the headless session's bootstrap
/// output(s) + any physical monitor, i.e. exactly what `exclusive` must disable — EACH PAIRED WITH ITS
/// CURRENT MODE (`WxH@Hz`, empty if unresolved) so teardown can put it back at that exact refresh (a
/// bare re-enable drops a 120 Hz panel to KWin's default ~60 Hz).
/// **Group-aware (§6.1):** excludes the WHOLE managed family (the [`MANAGED_PREFIX`]), not just this
/// session's own output — so a 2nd `exclusive` session (with a distinct per-slot name) never disables
/// the 1st session's live output. Parsed from `kscreen-doctor -j` (same source as [`read_active_refresh`]).
fn other_enabled_outputs() -> Vec<String> {
fn other_enabled_outputs() -> Vec<(String, String)> {
let out = match std::process::Command::new("kscreen-doctor")
.arg("-j")
.output()
@@ -350,9 +392,15 @@ fn other_enabled_outputs() -> Vec<String> {
.map(|outs| {
outs.iter()
.filter(|o| o.get("enabled").and_then(|e| e.as_bool()).unwrap_or(false))
.filter_map(|o| o.get("name").and_then(|n| n.as_str()))
.filter(|n| !n.starts_with(MANAGED_PREFIX))
.map(String::from)
.filter_map(|o| {
let name = o.get("name").and_then(|n| n.as_str())?;
(!name.starts_with(MANAGED_PREFIX)).then(|| {
(
name.to_string(),
output_current_mode_spec(o).unwrap_or_default(),
)
})
})
.collect()
})
.unwrap_or_default()
@@ -393,7 +441,7 @@ fn a_managed_output_is_primary() -> bool {
/// the sole desktop (KWin re-homes plasmashell + windows onto it). Returns the disabled outputs for
/// the keepalive to re-enable on teardown. Best-effort: on failure, streaming continues (just possibly
/// showing only the wallpaper) rather than failing the session.
fn apply_virtual_primary(name: &str) -> Vec<String> {
fn apply_virtual_primary(name: &str) -> Vec<(String, String)> {
let ours = format!("Virtual-{name}");
let kscreen = |args: &[String]| {
std::process::Command::new("kscreen-doctor")
@@ -416,11 +464,12 @@ fn apply_virtual_primary(name: &str) -> Vec<String> {
}
// Disable everything still enabled that ISN'T a managed group member (bootstrap / physical), so
// the group is unambiguously the desktop — never a sibling session's output (group-aware filter).
// Each is captured WITH its current mode so teardown restores its real refresh, not KWin's default.
let others = other_enabled_outputs();
if !others.is_empty() {
let args: Vec<String> = others
.iter()
.map(|o| format!("output.{o}.disable"))
.map(|(o, _mode)| format!("output.{o}.disable"))
.collect();
let _ = kscreen(&args);
}
@@ -97,12 +97,11 @@ impl VirtualDisplay for MutterDisplay {
h = mode.height,
"Mutter virtual monitor ready"
);
Ok(VirtualOutput {
Ok(VirtualOutput::owned(
node_id,
remote_fd: None,
preferred_mode: Some((mode.width, mode.height, mode.refresh_hz)),
keepalive: Box::new(StopGuard(stop)),
})
Some((mode.width, mode.height, mode.refresh_hz)),
Box::new(StopGuard(stop)),
))
}
}
@@ -413,8 +412,8 @@ fn mode_flag(md: &DbusMode, key: &str) -> bool {
matches!(md.6.get(key).map(|v| &**v), Some(&Value::Bool(true)))
}
/// The current (else preferred, else first) mode of `connector` → (mode_id, width, height).
fn current_mode(state: &CurrentState, connector: &str) -> Option<(String, i32, i32)> {
/// The current (else preferred, else first) mode of `connector` → `(mode_id, width, height, refresh)`.
fn current_mode_full(state: &CurrentState, connector: &str) -> Option<(String, i32, i32, f64)> {
let mon = state.1.iter().find(|m| m.0 .0 == connector)?;
let pick = mon
.1
@@ -422,7 +421,83 @@ fn current_mode(state: &CurrentState, connector: &str) -> Option<(String, i32, i
.find(|md| mode_flag(md, "is-current"))
.or_else(|| mon.1.iter().find(|md| mode_flag(md, "is-preferred")))
.or_else(|| mon.1.first())?;
Some((pick.0.clone(), pick.1, pick.2))
Some((pick.0.clone(), pick.1, pick.2, pick.3))
}
/// As [`current_mode_full`] but dropping the refresh (callers that only place by width).
fn current_mode(state: &CurrentState, connector: &str) -> Option<(String, i32, i32)> {
current_mode_full(state, connector).map(|(id, w, h, _)| (id, w, h))
}
/// Pure mode-pick for a KEPT physical (unit-tested). Given the physical's PRE-connect mode
/// (`pre_mode = (id, w, h, refresh)`; `None` when the connector is new since the snapshot) and the
/// mode list Mutter reports for it in the POST-virtual state
/// (`(id, w, h, refresh, is_current, is_preferred)`), return the `(mode_id, width)` to re-apply.
///
/// Mutter re-derives its layout when the `RecordVirtual` output appears and can silently drop a
/// 120 Hz panel to its EDID-preferred 60 Hz — so the post-virtual `is-current` is *already* 60 Hz.
/// We therefore prefer the PRE mode (its real refresh), resolved to a mode id valid at apply time;
/// only when the physical genuinely no longer offers that mode do we fall back to the post-virtual
/// current (never inventing a mode id `ApplyMonitorsConfig` would reject).
fn pick_keep_mode(
pre_mode: Option<(String, i32, i32, f64)>,
state_modes: &[(String, i32, i32, f64, bool, bool)],
) -> Option<(String, i32)> {
let state_current = || {
state_modes
.iter()
.find(|m| m.4)
.or_else(|| state_modes.iter().find(|m| m.5))
.or_else(|| state_modes.first())
.map(|m| (m.0.clone(), m.1))
};
let Some((pre_id, w, h, hz)) = pre_mode else {
return state_current();
};
// The exact pre mode id, if the connector still offers it (same session ⇒ usually true).
if state_modes.iter().any(|m| m.0 == pre_id) {
return Some((pre_id, w));
}
// Else a re-keyed id with the same geometry + refresh (still the real 120 Hz).
if let Some(m) = state_modes
.iter()
.find(|m| m.1 == w && m.2 == h && (m.3 - hz).abs() < 0.5)
{
return Some((m.0.clone(), m.1));
}
// The physical genuinely no longer offers that mode — use whatever is valid now.
state_current()
}
/// The `(mode_id, width)` a kept physical should be RE-APPLIED at — its PRE-connect mode preserved
/// across Mutter's virtual-output layout re-derive. See [`pick_keep_mode`].
fn physical_keep_mode(
pre: &CurrentState,
state: &CurrentState,
conn: &str,
) -> Option<(String, i32)> {
let pre_mode = current_mode_full(pre, conn);
let state_modes: Vec<(String, i32, i32, f64, bool, bool)> = state
.1
.iter()
.find(|m| m.0 .0 == conn)
.map(|mon| {
mon.1
.iter()
.map(|md| {
(
md.0.clone(),
md.1,
md.2,
md.3,
mode_flag(md, "is-current"),
mode_flag(md, "is-preferred"),
)
})
.collect()
})
.unwrap_or_default();
pick_keep_mode(pre_mode, &state_modes)
}
/// Wait for the virtual output to appear in DisplayConfig (its size follows PipeWire negotiation,
@@ -466,7 +541,7 @@ async fn make_virtual_primary(
let config = if exclusive {
build_exclusive_config(&vconn, &vmode)
} else {
build_primary_keeping_physicals(&state, &vconn, &vmode, mode.width as i32)
build_primary_keeping_physicals(pre, &state, &vconn, &vmode, mode.width as i32)
};
let _: () = dc
.call(
@@ -506,13 +581,20 @@ fn build_exclusive_config(vconn: &str, vmode: &str) -> Vec<ApplyLogical> {
}
/// **Primary** — the virtual output primary at `(0, 0)`, with every currently-active physical
/// monitor KEPT as a secondary (laid left-to-right past the virtual, each at its current mode). So
/// the shell + new windows land on the streamed surface, but the operator's physical screen stays
/// on. On a headless host (no physicals) this is identical to [`build_exclusive_config`].
/// monitor KEPT as a secondary (laid left-to-right past the virtual, each at its **pre-connect**
/// mode). So the shell + new windows land on the streamed surface, but the operator's physical
/// screen stays on **at its real refresh**. On a headless host (no physicals) this is identical to
/// [`build_exclusive_config`].
///
/// `pre` is the snapshot taken *before* the virtual output existed (physical still at its true
/// refresh); `state` is the post-virtual state. We read each physical's mode from `pre` because
/// Mutter can knock a 120 Hz panel down to 60 Hz when it re-derives the layout for the virtual
/// monitor — reading `state` would cement that 60 Hz (`physical_keep_mode`).
///
/// *Physical-keep is unvalidated on-glass* — the lab boxes are headless (no attached display to keep
/// on); the layout math is conservative (append to the right) but wants a display-attached box.
fn build_primary_keeping_physicals(
pre: &CurrentState,
state: &CurrentState,
vconn: &str,
vmode: &str,
@@ -526,15 +608,15 @@ fn build_primary_keeping_physicals(
true,
vec![(vconn.to_string(), vmode.to_string(), HashMap::new())],
)];
// Append each physical (non-virtual) connector that has a usable current mode, to the right of
// the virtual output, as a non-primary secondary.
// Append each physical (non-virtual) connector that has a usable mode, to the right of the
// virtual output, as a non-primary secondary — at its PRE-connect mode (real refresh preserved).
let mut x = virt_width.max(0);
for mon in &state.1 {
let conn = &mon.0 .0;
if conn == vconn {
continue;
}
if let Some((mode_id, w, _h)) = current_mode(state, conn) {
if let Some((mode_id, w)) = physical_keep_mode(pre, state, conn) {
logicals.push((
x,
0,
@@ -548,3 +630,84 @@ fn build_primary_keeping_physicals(
}
logicals
}
#[cfg(test)]
mod tests {
use super::pick_keep_mode;
// (id, w, h, refresh, is_current, is_preferred)
fn m(
id: &str,
w: i32,
h: i32,
hz: f64,
cur: bool,
pref: bool,
) -> (String, i32, i32, f64, bool, bool) {
(id.to_string(), w, h, hz, cur, pref)
}
#[test]
fn keep_mode_prefers_pre_refresh_over_downgraded_state() {
// Physical was 2560x1440@120 pre-connect; after the virtual appeared Mutter marked 60 Hz
// current (the reported bug). We must re-apply the 120 Hz mode, not the state's 60 Hz.
let pre = Some(("M120".to_string(), 2560, 1440, 120.0));
let state = vec![
m("M120", 2560, 1440, 120.0, false, false),
m("M60", 2560, 1440, 60.0, true, true),
];
assert_eq!(
pick_keep_mode(pre, &state),
Some(("M120".to_string(), 2560))
);
}
#[test]
fn keep_mode_rekeyed_id_matches_by_geometry_and_refresh() {
// The pre id is no longer offered (Mutter re-keyed the mode list), but a 120 Hz mode of the
// same geometry exists — match it so the real refresh survives.
let pre = Some(("old-120".to_string(), 2560, 1440, 120.0));
let state = vec![
m("new-120", 2560, 1440, 119.998, false, false),
m("new-60", 2560, 1440, 60.0, true, true),
];
assert_eq!(
pick_keep_mode(pre, &state),
Some(("new-120".to_string(), 2560))
);
}
#[test]
fn keep_mode_falls_back_to_state_current_when_pre_mode_gone() {
// The physical genuinely no longer offers its pre mode (e.g. cable renegotiated to a lower
// max) — never invent an id; use the post-virtual current.
let pre = Some(("gone-165".to_string(), 3440, 1440, 165.0));
let state = vec![
m("s-100", 3440, 1440, 100.0, true, false),
m("s-60", 3440, 1440, 60.0, false, true),
];
assert_eq!(
pick_keep_mode(pre, &state),
Some(("s-100".to_string(), 3440))
);
}
#[test]
fn keep_mode_no_pre_uses_state_current_then_preferred() {
// A connector new since the pre-snapshot (no pre mode): is-current wins, else is-preferred.
let state = vec![
m("A", 1920, 1080, 60.0, true, false),
m("B", 1920, 1080, 144.0, false, true),
];
assert_eq!(pick_keep_mode(None, &state), Some(("A".to_string(), 1920)));
let no_current = vec![
m("A", 1920, 1080, 60.0, false, false),
m("B", 1920, 1080, 144.0, false, true),
];
assert_eq!(
pick_keep_mode(None, &no_current),
Some(("B".to_string(), 1920))
);
}
}
@@ -19,7 +19,7 @@
//! `systemctl --user`, see `scripts/headless/prepare-session.sh`), with the ScreenCast
//! interface routed to xdpw (`scripts/headless/portals.conf`).
use super::{Mode, VirtualDisplay, VirtualOutput};
use super::{DisplayOwnership, Mode, VirtualDisplay, VirtualOutput};
use anyhow::{anyhow, bail, Context, Result};
use std::os::fd::OwnedFd;
use std::process::Command;
@@ -130,6 +130,11 @@ impl VirtualDisplay for WlrootsDisplay {
_stop: StopGuard(stop),
_output: output,
}),
// Owned (the compositor output is ours to tear down), but not registry-poolable: the
// portal fd can't be re-opened per attach, so the registry passes it through on
// `remote_fd.is_some()` (keep-alive stays off for wlroots until fresh-portal re-attach).
ownership: DisplayOwnership::Owned,
reused_gen: None,
})
}
}
+194 -3
View File
@@ -23,10 +23,11 @@
use std::collections::BTreeMap;
use std::path::PathBuf;
use std::sync::{Mutex, OnceLock};
use std::time::Duration;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use anyhow::Result;
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
use utoipa::ToSchema;
/// How long a virtual display (and, on gamescope's bare spawn, the nested session + its game)
@@ -158,6 +159,22 @@ pub struct Layout {
pub positions: BTreeMap<String, Position>,
}
/// How a session that **launches a game** (a library id on the Hello / apps.json / Decky pin) is
/// served (`design/gamemode-and-dedicated-sessions.md` §5.2). Orthogonal to the preset/lifecycle axes
/// — a top-level [`DisplayPolicy`] field, NOT part of [`EffectivePolicy`], so a preset never clobbers
/// it. Linux-only in effect (a launching Windows session opens into the one desktop).
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
#[serde(rename_all = "snake_case")]
pub enum GameSession {
/// Today's routing: the launch rides whatever session the box is in (managed Steam session on
/// Bazzite/SteamOS, bare spawn on plain distros, spawned into the live desktop on KWin/Mutter/wlroots).
#[default]
Auto,
/// A launching session always gets its OWN headless gamescope at the client's mode, nesting just
/// the game — no Steam Big Picture, no game mode. Degrades to `auto` when gamescope is unavailable.
Dedicated,
}
/// A named bundle of the fields below. `Custom` (the default) means the explicit fields rule; any
/// other preset ignores the stored fields and expands to its own ([`DisplayPolicy::effective`]).
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
@@ -202,6 +219,11 @@ pub struct DisplayPolicy {
/// Upper bound on simultaneously-live virtual displays (clamped to `1..=16` on write).
#[serde(default = "default_max_displays")]
pub max_displays: u32,
/// How a game-launching session is served (`design/gamemode-and-dedicated-sessions.md` §5.2).
/// Orthogonal to `preset`/lifecycle — preserved across preset changes; `#[serde(default)]` = `Auto`
/// so existing `display-settings.json` files are untouched.
#[serde(default)]
pub game_session: GameSession,
}
fn one() -> u32 {
@@ -224,6 +246,7 @@ impl Default for DisplayPolicy {
identity: Identity::default(),
layout: Layout::default(),
max_displays: 4,
game_session: GameSession::default(),
}
}
}
@@ -279,7 +302,11 @@ impl EffectivePolicy {
/// transform, factored out pure so arranging displays stays orthogonal to the other axes and is
/// unit-tested without touching the global store. (`Custom` so the explicit fields — incl. the new
/// layout — rule; a named preset would ignore them.)
pub fn with_manual_layout(&self, positions: BTreeMap<String, Position>) -> DisplayPolicy {
pub fn with_manual_layout(
&self,
positions: BTreeMap<String, Position>,
game_session: GameSession,
) -> DisplayPolicy {
DisplayPolicy {
version: 1,
preset: Preset::Custom,
@@ -292,6 +319,8 @@ impl EffectivePolicy {
positions,
},
max_displays: self.max_displays,
// Preserve the orthogonal game-session axis (EffectivePolicy doesn't carry it).
game_session,
}
}
}
@@ -398,6 +427,13 @@ impl DisplayPolicyStore {
self.configured().map(|p| p.effective())
}
/// The game-session routing axis (`design/gamemode-and-dedicated-sessions.md` §5.2). Orthogonal to
/// the preset — read directly off the stored policy (or the default `Auto` when unconfigured), so a
/// preset selection never resets it.
pub fn game_session(&self) -> GameSession {
self.get().game_session
}
/// Persist + adopt a new policy (sanitized first). The in-memory value changes only if the disk
/// write succeeds, so a full disk can't leave memory and file disagreeing.
pub fn set(&self, policy: DisplayPolicy) -> Result<()> {
@@ -423,10 +459,163 @@ pub fn prefs() -> &'static DisplayPolicyStore {
})
}
// ---------------------------------------------------------------------------------------
// User-defined custom presets (`<config>/display-presets.json`)
// ---------------------------------------------------------------------------------------
/// A user-defined named preset: a saved bundle of the six display-behavior axes (exactly what a
/// built-in [`Preset`] expands to) plus the orthogonal game-session axis, that the operator names
/// and applies from the console.
///
/// Unlike the built-in [`Preset`]s (a closed enum), custom presets are **data** — a catalog stored in
/// `<config>/display-presets.json`. Applying one writes a `Custom` [`DisplayPolicy`] carrying these
/// fields (the console reuses `PUT /display/settings`), so [`DisplayPolicy::effective`] stays pure and
/// the built-in set is never touched. The catalog is decoupled from the active `display-settings.json`:
/// editing or deleting a preset never mutates the running policy (re-apply to adopt a change).
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
pub struct CustomPreset {
/// Host-assigned, stable for the life of the entry (the `{id}` in the CRUD path).
pub id: String,
/// User-facing name shown on the preset card; editable.
pub name: String,
/// The six display-behavior axes this preset applies (the same shape a built-in preset expands to).
pub fields: EffectivePolicy,
/// The game-session routing this preset applies (orthogonal to the six axes; see [`GameSession`]).
/// A custom preset captures the operator's *full* setup, so — unlike a built-in preset — applying
/// one does set this axis.
#[serde(default)]
pub game_session: GameSession,
}
/// Request body to create or replace a custom preset (no `id` — the host owns it).
#[derive(Clone, Debug, Deserialize, ToSchema)]
pub struct CustomPresetInput {
pub name: String,
pub fields: EffectivePolicy,
#[serde(default)]
pub game_session: GameSession,
}
fn custom_presets_path() -> PathBuf {
crate::gamestream::config_dir().join("display-presets.json")
}
/// Clamp a saved preset's fields to their valid ranges — the same bounds [`DisplayPolicy::sanitized`]
/// enforces, so a preset can never carry an out-of-range `max_displays` that a later apply would reject.
fn sanitize_preset_fields(mut fields: EffectivePolicy) -> EffectivePolicy {
fields.max_displays = fields.max_displays.clamp(1, 16);
fields
}
/// Load the saved custom presets (empty + non-fatal if the file is absent or malformed — a bad
/// catalog never breaks the console's settings GET).
pub fn load_custom_presets() -> Vec<CustomPreset> {
match std::fs::read(custom_presets_path()) {
Ok(bytes) => serde_json::from_slice(&bytes).unwrap_or_else(|e| {
tracing::warn!(error = %e, "display-presets.json malformed — ignoring custom presets");
Vec::new()
}),
Err(_) => Vec::new(),
}
}
/// Persist the catalog (private dir, temp-write + atomic rename — the [`DisplayPolicyStore::set`]
/// discipline, so a crash mid-write never truncates it).
fn save_custom_presets(presets: &[CustomPreset]) -> Result<()> {
let path = custom_presets_path();
if let Some(dir) = path.parent() {
crate::gamestream::create_private_dir(dir)?;
}
let tmp = path.with_extension("json.tmp");
crate::gamestream::write_secret_file(&tmp, &serde_json::to_vec_pretty(presets)?)?;
std::fs::rename(&tmp, &path)?;
Ok(())
}
/// 12 hex chars from the name + wall-clock nanos — collision-free in practice, no uuid dep (the
/// [`crate::library`] custom-entry id scheme).
fn new_preset_id(name: &str) -> String {
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0);
hex::encode(&Sha256::digest(format!("{name}:{nanos}").as_bytes())[..6])
}
/// Create a custom preset, returning it with its assigned id.
pub fn add_custom_preset(input: CustomPresetInput) -> Result<CustomPreset> {
let mut presets = load_custom_presets();
let preset = CustomPreset {
id: new_preset_id(&input.name),
name: input.name,
fields: sanitize_preset_fields(input.fields),
game_session: input.game_session,
};
presets.push(preset.clone());
save_custom_presets(&presets)?;
Ok(preset)
}
/// Replace a custom preset's fields (id preserved). `None` ⇒ no preset with that id.
pub fn update_custom_preset(id: &str, input: CustomPresetInput) -> Result<Option<CustomPreset>> {
let mut presets = load_custom_presets();
let Some(slot) = presets.iter_mut().find(|p| p.id == id) else {
return Ok(None);
};
slot.name = input.name;
slot.fields = sanitize_preset_fields(input.fields);
slot.game_session = input.game_session;
let updated = slot.clone();
save_custom_presets(&presets)?;
Ok(Some(updated))
}
/// Delete a custom preset. `false` ⇒ no preset with that id.
pub fn delete_custom_preset(id: &str) -> Result<bool> {
let mut presets = load_custom_presets();
let before = presets.len();
presets.retain(|p| p.id != id);
if presets.len() == before {
return Ok(false);
}
save_custom_presets(&presets)?;
Ok(true)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn custom_preset_serde_roundtrips_and_defaults_game_session() {
let preset = CustomPreset {
id: "abc123".into(),
name: "My Rig".into(),
fields: preset_fields(Preset::GamingRig).unwrap(),
game_session: GameSession::Dedicated,
};
let json = serde_json::to_string(&preset).unwrap();
assert_eq!(serde_json::from_str::<CustomPreset>(&json).unwrap(), preset);
// A catalog written before `game_session` existed still loads (defaults to `Auto`).
let legacy: CustomPreset = serde_json::from_value(serde_json::json!({
"id": "x",
"name": "Legacy",
"fields": serde_json::to_value(preset_fields(Preset::Default).unwrap()).unwrap(),
}))
.unwrap();
assert_eq!(legacy.game_session, GameSession::Auto);
}
#[test]
fn sanitize_preset_fields_clamps_max_displays() {
let mut f = preset_fields(Preset::Default).unwrap();
f.max_displays = 999;
assert_eq!(sanitize_preset_fields(f.clone()).max_displays, 16);
f.max_displays = 0;
assert_eq!(sanitize_preset_fields(f).max_displays, 1);
}
#[test]
fn keep_alive_serializes_tagged_on_mode() {
assert_eq!(
@@ -560,7 +749,9 @@ mod tests {
let mut positions = BTreeMap::new();
positions.insert("1".to_string(), Position { x: 0, y: 0 });
positions.insert("7".to_string(), Position { x: 2560, y: 0 });
let p = eff.with_manual_layout(positions);
let p = eff.with_manual_layout(positions, GameSession::Dedicated);
// The orthogonal game-session axis is preserved through the layout transform.
assert_eq!(p.game_session, GameSession::Dedicated);
// Preset drops to Custom so the explicit fields (incl. the layout) rule…
assert_eq!(p.preset, Preset::Custom);
// …every other behavior axis is preserved verbatim…
+244 -33
View File
@@ -164,6 +164,28 @@ pub fn release(slot: Option<u64>) -> usize {
}
}
/// Tear down a **reused-but-dead** pool entry by its generation stamp (A2). Called by the pipeline
/// builder when the first frame fails on a display [`acquire`] handed back as REUSED — so the retry
/// loop's next `acquire` creates fresh instead of re-wedging on the same corpse. No-op off Linux / if
/// the entry is already gone (idempotent — the subsequent stale-gen lease drop no-ops too).
pub fn mark_failed(gen: u64) {
#[cfg(target_os = "linux")]
linux::mark_failed(gen);
#[cfg(not(target_os = "linux"))]
let _ = gen;
}
/// Invalidate every kept display of `backend` — its compositor instance is gone (a Game↔Desktop switch
/// tore it down), so `/display/state` must stop listing it and its keepalive must be reaped
/// (`design/gamemode-and-dedicated-sessions.md` A4). Called from the session-switch watcher / a
/// per-connect re-detect that finds the previous backend's compositor gone. No-op off Linux.
pub fn invalidate_backend(backend: &str) {
#[cfg(target_os = "linux")]
linux::invalidate_backend(backend);
#[cfg(not(target_os = "linux"))]
let _ = backend;
}
// ---------------------------------------------------------------------------------------------
// Linux keep-alive pool
// ---------------------------------------------------------------------------------------------
@@ -202,6 +224,13 @@ mod linux {
/// exclusive session); on teardown it hands off to a surviving sibling, and only runs when the
/// group's last member drops. `None` for extend/primary and non-first / non-exclusive members.
topology_restore: Option<Restore>,
/// The launch command this display was created with (`design/gamemode-and-dedicated-sessions.md`
/// A2): keep-alive reuse requires an exact match, so a kept spawn running game A never serves a
/// session launching game B. `None` = a plain desktop / no nested command.
launch: Option<String>,
/// The session epoch at creation (A4). Reuse requires an epoch match; the linger timer reaps
/// entries whose epoch is stale (their compositor instance was replaced under them).
epoch: u64,
/// Generation stamp: a [`DisplayLease`] only releases if its gen still matches (a stale lease
/// — its entry was reused + re-stamped — is a no-op).
gen: u64,
@@ -210,6 +239,18 @@ mod linux {
/// A per-group topology-restore action (see [`Entry::topology_restore`]).
type Restore = Box<dyn FnOnce() + Send>;
/// The result of the keep-alive reuse lookup (A2 validated reuse): a live kept display was reused,
/// a dead one was pulled out (recreate), or nothing matched.
enum ReuseOutcome {
/// A live kept display — the session-facing output to return.
Reused(VirtualOutput),
/// A dead kept display, removed from the pool, plus its group restore (run before the corpse's
/// keepalive drops); the caller falls through to a fresh create.
Dead(Entry, Option<Restore>),
/// No matching kept display.
Miss,
}
/// Hand off a torn-down display's topology restore (design §6.1 — per-group restore): if a
/// same-group (backend) sibling survives in `remaining`, MOVE the restore onto it (a later teardown
/// runs it); if the group is now empty, RETURN the action so the caller runs it (before dropping the
@@ -245,6 +286,19 @@ mod linux {
})
}
/// Does a pooled entry's session `epoch` still match the current one for reuse / expiry purposes?
/// The session epoch tracks the box's **active-session (desktop) compositor** instance (KWin /
/// Mutter / wlroots) — whose PipeWire node dies with the compositor, so a stale-epoch kept output
/// is a corpse. A **gamescope** spawn is the exact opposite: an independent nested session (its own
/// group), whose node lives with its own child process, wholly unrelated to whatever desktop /
/// game-mode compositor the epoch tracks. So gamescope entries are EXEMPT from the epoch — a desktop
/// switch, or a game-mode gamescope restart, must never invalidate a kept dedicated game session
/// (review findings #2/#5/#6/#7/#10). Their liveness is the `kept_display_alive` node probe + the B2
/// game-exit path + `mark_failed`, not the epoch.
fn epoch_matches(backend: &str, entry_epoch: u64, cur_epoch: u64) -> bool {
backend == "gamescope" || entry_epoch == cur_epoch
}
/// The linger resolution for Linux: the console policy's `keep_alive` when configured, else
/// **Immediate** (today's behavior — a Linux disconnect tears the output down at once).
fn linger() -> Linger {
@@ -262,9 +316,17 @@ mod linux {
fn take_expired(entries: &mut Vec<Entry>, now: Instant) -> (Vec<Entry>, Vec<Restore>) {
let mut expired = Vec::new();
let mut restores = Vec::new();
// A4 backstop: also reap a KEPT (non-Active) DESKTOP display whose session epoch is stale — its
// compositor instance was replaced (a Game↔Desktop switch / same-kind restart), so its node id
// now means nothing. gamescope spawns are exempt (`epoch_matches` — independent nested sessions).
// An Active entry is left to its own session's capture-loss rebuild (which, under the bumped
// epoch, won't reuse it); `invalidate_backend` clears a whole desktop backend on a known switch.
let cur_epoch = crate::vdisplay::session_epoch();
let mut i = 0;
while i < entries.len() {
if entries[i].life.poll_expiry(now) {
let dead_epoch = !epoch_matches(entries[i].backend, entries[i].epoch, cur_epoch)
&& !matches!(entries[i].life, lifecycle::State::Active { .. });
if entries[i].life.poll_expiry(now) || dead_epoch {
let mut e = entries.remove(i);
let backend = e.backend;
if let Some(r) = hand_off_restore(entries, backend, e.topology_restore.take()) {
@@ -312,13 +374,18 @@ mod linux {
preferred_mode: Option<(u32, u32, u32)>,
gen: u64,
quit: Arc<AtomicBool>,
reused: bool,
) -> VirtualOutput {
VirtualOutput {
// The pooled display is registry-owned; the session holds a gen-stamped lease as its keepalive.
let mut out = VirtualOutput::owned(
node_id,
remote_fd: None,
preferred_mode,
keepalive: Box::new(DisplayLease { gen, quit }),
}
Box::new(DisplayLease { gen, quit }),
);
// A2: tell the pipeline builder this was a REUSED kept display, so a first-frame failure can
// `mark_failed(gen)` (tear the corpse down) rather than re-wedge the retry loop on the same node.
out.reused_gen = reused.then_some(gen);
out
}
pub(super) fn acquire(
@@ -328,6 +395,10 @@ mod linux {
) -> Result<VirtualOutput> {
ensure_timer();
let backend = vd.name();
// A2 reuse key: the launch command this acquire carries (a kept spawn running game A must never
// be reused for a session launching game B). A4 reuse key: the current session epoch.
let launch = vd.launch_command();
let cur_epoch = crate::vdisplay::session_epoch();
let r = reg();
// Reap expired first (run any group restores + drop outside the lock).
@@ -340,28 +411,94 @@ mod linux {
}
drop(expired);
// Reuse: a kept (lingering/pinned) display of the same backend + mode. A reconnecting session
// re-attaches a fresh PipeWire consumer to the still-live `node_id`.
{
let mut es = r.entries.lock().unwrap();
if let Some(e) = es.iter_mut().find(|e| {
matches!(
e.life,
lifecycle::State::Lingering { .. } | lifecycle::State::Pinned
) && e.backend == backend
&& e.mode == mode
}) {
// Lingering/Pinned → Active (Acquire::Reuse); side effect matters, value is known.
e.life.acquire();
let gen = r.gen.fetch_add(1, Ordering::Relaxed);
e.gen = gen;
let out = output_for(e.node_id, e.preferred_mode, gen, quit);
tracing::info!(
backend,
node_id = e.node_id,
"virtual display reused (keep-alive reconnect)"
);
return Ok(out);
// Reuse: a kept (lingering/pinned) display of the same backend + mode + launch + epoch. A
// reconnecting session re-attaches a fresh PipeWire consumer to the still-live `node_id`. Gated
// on `vd.poolable_now()` (A1): a gamescope managed/attach acquire must NOT reuse a kept bare-spawn
// (they share the backend name `"gamescope"`); its `create` builds a `SessionManaged`/`External`
// output that passes through below.
if vd.poolable_now() {
// Reuse a kept display, matching backend + mode + launch (+ epoch for the desktop backends;
// gamescope spawns are independent nested sessions, exempt from the active-session epoch —
// see `epoch_matches`). The liveness probe (`kept_display_alive`, which may shell `pw-dump`
// for gamescope) must NOT run under the pool lock (it can block / hang the daemon), so:
// 1. find the candidate + snapshot (gen, node_id) UNDER the lock, then release it;
// 2. probe liveness OUTSIDE the lock;
// 3. re-lock and re-find the SAME entry by its gen (another thread may have reused/removed
// it meanwhile — then we just miss and create fresh).
let candidate = {
let es = r.entries.lock().unwrap();
es.iter()
.find(|e| {
matches!(
e.life,
lifecycle::State::Lingering { .. } | lifecycle::State::Pinned
) && e.backend == backend
&& e.mode == mode
&& e.launch == launch
&& epoch_matches(e.backend, e.epoch, cur_epoch)
})
.map(|e| (e.gen, e.node_id))
};
if let Some((cand_gen, node_id)) = candidate {
let alive = vd.kept_display_alive(node_id); // OUTSIDE the lock (may block)
let reuse = {
let mut es = r.entries.lock().unwrap();
// Re-find the SAME entry by its snapshot gen; skip if it's gone or no longer kept
// (a concurrent reconnect adopted it) — we then miss and create fresh.
match es.iter().position(|e| {
e.gen == cand_gen
&& matches!(
e.life,
lifecycle::State::Lingering { .. } | lifecycle::State::Pinned
)
}) {
Some(idx) if alive => {
es[idx].life.acquire();
let gen = r.gen.fetch_add(1, Ordering::Relaxed);
es[idx].gen = gen;
let preferred_mode = es[idx].preferred_mode;
tracing::info!(
backend,
node_id,
"virtual display reused (keep-alive reconnect)"
);
ReuseOutcome::Reused(output_for(
node_id,
preferred_mode,
gen,
quit.clone(),
true,
))
}
Some(idx) => {
// Dead kept display: remove it, hand off its group restore, create fresh.
let mut dead = es.remove(idx);
let restore = hand_off_restore(
&mut es,
dead.backend,
dead.topology_restore.take(),
);
ReuseOutcome::Dead(dead, restore)
}
None => ReuseOutcome::Miss, // adopted/removed by another thread
}
};
match reuse {
ReuseOutcome::Reused(out) => return Ok(out),
ReuseOutcome::Dead(dead, restore) => {
// Outside the lock: re-enable physicals (if the group emptied) then drop the
// corpse's keepalive (may block) — then fall through to a fresh create below.
if let Some(rst) = restore {
rst();
}
tracing::info!(
backend,
"virtual display: kept display was dead — recreating (validated reuse)"
);
drop(dead);
}
ReuseOutcome::Miss => {}
}
}
}
@@ -381,13 +518,18 @@ mod linux {
// the group arrangement (manual per-slot positions) + the state slot.
let identity_slot = vd.last_identity_slot();
// wlroots (remote_fd = Some, sandboxed xdpw portal) can't be kept without re-opening the
// portal fd per attach — pass it through unchanged (capturer owns it, teardown on drop). The
// poolable backends put their node on the default daemon (remote_fd = None).
if real.remote_fd.is_some() {
// Pool ONLY a registry-owned display on the default PipeWire daemon
// (design/gamemode-and-dedicated-sessions.md A1). Pass through, unchanged (capturer owns the
// keepalive, teardown on drop), everything else:
// * `External`/`SessionManaged` — gamescope attach / managed session: the gamescope module
// owns their lifecycle (its own restore machinery), so the registry must not keep them
// (the stale-node reuse wedge). Their unit keepalive tears nothing down on drop.
// * `remote_fd = Some` — wlroots' sandboxed xdpw portal fd can't be re-opened per attach.
if real.ownership != crate::vdisplay::DisplayOwnership::Owned || real.remote_fd.is_some() {
tracing::debug!(
backend,
"virtual display not poolable (portal fd) — keep-alive off for this backend"
ownership = ?real.ownership,
"virtual display not registry-poolable — keep-alive off (owner keeps it / portal fd)"
);
return Ok(real);
}
@@ -410,6 +552,8 @@ mod linux {
backend,
identity_slot,
topology_restore,
launch: launch.clone(),
epoch: cur_epoch,
gen,
};
@@ -455,7 +599,7 @@ mod linux {
if (position.x, position.y) != (0, 0) {
vd.apply_position(position.x, position.y);
}
Ok(output_for(node_id, preferred_mode, gen, quit))
Ok(output_for(node_id, preferred_mode, gen, quit, false))
}
/// The [`DisplayLease`] `Drop` path: release the session's hold on the pooled display. The
@@ -704,6 +848,71 @@ mod linux {
n
}
/// A2 — tear down a reused-but-dead pool entry by its generation stamp. Removes it (hand off /
/// run its group restore), drops the keepalive outside the lock. Idempotent (already gone → no-op).
pub(super) fn mark_failed(gen: u64) {
let Some(r) = REG.get() else { return };
let (torn, restore) = {
let mut es = r.entries.lock().unwrap();
let Some(idx) = es.iter().position(|e| e.gen == gen) else {
return; // already gone — the subsequent stale-gen lease drop no-ops too
};
let mut e = es.remove(idx);
let backend = e.backend;
let restore = hand_off_restore(&mut es, backend, e.topology_restore.take());
(e, restore)
};
if let Some(rst) = restore {
rst(); // outside the lock, before the keepalive drops
}
tracing::warn!(
backend = torn.backend,
"virtual display: reused kept display was dead on first frame — torn down (A2 mark_failed)"
);
drop(torn); // keepalive Drop outside the lock (may block)
}
/// A4 — invalidate every kept display of `backend` (its compositor instance is gone). Removes them
/// all (any lifecycle state — a dead compositor's Active entries are doomed too; their sessions
/// rebuild), runs/hands off group restores, drops keepalives outside the lock (they hit dead
/// sockets and fail fast). Mirrors `force_release`'s shape but selects by backend, not slot/state.
pub(super) fn invalidate_backend(backend: &str) {
let Some(r) = REG.get() else { return };
let (removed, restores) = {
let mut es = r.entries.lock().unwrap();
let mut out = Vec::new();
let mut restores = Vec::new();
let mut i = 0;
while i < es.len() {
if es[i].backend == backend {
let mut e = es.remove(i);
let b = e.backend;
if let Some(rst) = hand_off_restore(&mut es, b, e.topology_restore.take()) {
restores.push(rst);
}
out.push(e);
} else {
i += 1;
}
}
(out, restores)
};
if removed.is_empty() {
return;
}
for restore in restores {
restore();
}
tracing::info!(
backend,
count = removed.len(),
"virtual displays invalidated — compositor instance gone (A4 session switch)"
);
for e in removed {
drop(e); // outside the lock
}
}
/// The session's refcount handle — the `keepalive` the capturer holds. `Drop` releases the
/// registry hold; a stale lease (its entry was reused + re-stamped, or torn down) is a no-op.
struct DisplayLease {
@@ -744,6 +953,8 @@ mod linux {
backend,
identity_slot: None,
topology_restore: restore,
launch: None,
epoch: 0,
gen,
}
}
@@ -31,10 +31,10 @@ use windows::Win32::System::Threading::{
CreateMutexW, OpenProcess, WaitForSingleObject, PROCESS_SYNCHRONIZE,
};
use super::{Mode, VirtualOutput};
use super::{DisplayOwnership, Mode, VirtualOutput};
use crate::win_display::{
force_extend_topology, isolate_displays_ccd, resolve_gdi_name, restore_displays_ccd,
set_active_mode, set_virtual_primary_ccd, SavedConfig,
count_other_active, force_extend_topology, isolate_displays_ccd, resolve_gdi_name,
restore_displays_ccd, set_active_mode, set_virtual_primary_ccd, SavedConfig,
};
/// The per-backend REMOVE key the driver stamps on ADD and consumes on REMOVE. SudoVDA keys monitors by
@@ -531,6 +531,9 @@ impl VirtualDisplayManager {
mgr: self,
gen: mon.gen,
}),
// The Windows manager owns the monitor lifecycle (refcount/linger/pin), so the registry
// (which delegates to it via `vd.create`) treats it as Owned.
ownership: DisplayOwnership::Owned,
}
}
@@ -670,16 +673,32 @@ impl VirtualDisplayManager {
ccd_saved = unsafe { isolate_displays_ccd(added.target_id) };
}
Topology::Primary => {
// The IDD auto-activates as the SOLE display on a headless box, so the
// physical (if present) is deactivated and QueryDisplayConfig sees only the
// virtual. Force EXTEND first to (re)activate every CONNECTED display
// alongside the virtual, THEN reposition to make the virtual primary — so the
// physical stays active. (The bring-up above only force-EXTENDs when the
// virtual FAILS to auto-resolve; here it resolved, so we do it explicitly.)
// SAFETY: `force_extend_topology` drives the CCD topology FFI (no args, no borrowed
// memory), under the `state` lock — the sole topology mutator.
unsafe { force_extend_topology() };
thread::sleep(Duration::from_millis(300));
// On a headless box the IDD auto-activates as the SOLE display, so a physical
// (if present) is deactivated and QueryDisplayConfig sees only the virtual —
// force EXTEND to (re)activate every connected display alongside the virtual,
// THEN reposition to make the virtual primary. BUT on a box whose physical is
// ALREADY active (the IDD came up extended beside it — the common desktop case),
// that physical is already lit at its real mode; re-applying the bare
// `SDC_TOPOLOGY_EXTEND` preset would only re-pull each display's mode from the
// persistence DB, RESETTING a 120 Hz panel to 60 Hz. So force-EXTEND only when the
// virtual is currently sole; otherwise skip straight to the reposition, which
// re-supplies each physical's QUERIED mode verbatim (preserving its refresh).
// SAFETY: `count_other_active` runs the CCD QueryDisplayConfig FFI (Copy target id
// by value, owned result), under the `state` lock.
let already_extended =
unsafe { count_other_active(added.target_id) }.unwrap_or(0) > 0;
if already_extended {
tracing::info!(
"display topology=primary — a physical display is already active; \
skipping force-EXTEND (preserves its refresh) before making the \
virtual primary"
);
} else {
// SAFETY: `force_extend_topology` drives the CCD topology FFI (no args, no
// borrowed memory), under the `state` lock — the sole topology mutator.
unsafe { force_extend_topology() };
thread::sleep(Duration::from_millis(300));
}
// SAFETY: `set_virtual_primary_ccd` takes the `Copy` target id by value and returns
// an owned `SavedConfig` (no borrowed memory crosses), under the `state` lock.
ccd_saved = unsafe { set_virtual_primary_ccd(added.target_id) };
@@ -384,8 +384,10 @@ unsafe fn query_active_config() -> Option<SavedConfig> {
}
/// Count currently-ACTIVE display paths whose target id != `keep_target_id` — i.e. displays that would
/// still be lit besides the virtual one. `None` on query failure. Used to VERIFY isolation actually took.
unsafe fn count_other_active(keep_target_id: u32) -> Option<u32> {
/// still be lit besides the virtual one. `None` on query failure. Used to VERIFY isolation actually
/// took, and (in the `primary` topology) to detect a physical that is ALREADY active so we can skip a
/// force-EXTEND that would reset its refresh.
pub(crate) unsafe fn count_other_active(keep_target_id: u32) -> Option<u32> {
let (paths, _) = query_active_config()?;
Some(
paths
+11 -2
View File
@@ -33,7 +33,11 @@
Companion docs: `design/implementation-plan.md` §6 (virtual displays), `design/vrr-plan.md`
(pacing — out of scope here), `design/gamescope-multiuser.md` (per-session isolation — adjacent,
not required).
not required), **`design/gamemode-and-dedicated-sessions.md`** (PLANNED — reconciles this layer
with session-mobile Bazzite/SteamOS hosts: display **ownership classes** so the registry stops
pooling gamescope managed/attach outputs it doesn't own, validated reuse + invalidation, the
§5.1 "policy replaces the managed 5 s debounce" promise actually implemented, and the dedicated
per-launch gamescope game sessions built on it).
## 1. Goal
@@ -590,7 +594,12 @@ out per-host instead of lying:
The **attach** gamescope sub-mode never owns the display (it mirrors a foreign gamescope) — the
registry records it as an unmanaged pass-through slot: no keep-alive, no topology, no identity,
conflict = join-only. That's just codifying reality.
conflict = join-only. That's just codifying reality. **Gap (2026-07-05):** the shipped registry
does NOT implement this row — it pools every `remote_fd == None` output, including the
managed/attach/SteamOS paths' unit-keepalive outputs, which double-owns the managed session
against the gamescope module's own restore worker (stale-node reuse wedge on game-mode
reconnect). The fix — explicit display **ownership classes** + registry-owned managed-session
restore — is designed in `design/gamemode-and-dedicated-sessions.md` Part A.
## 8. Management API, web console, tray
+439
View File
@@ -0,0 +1,439 @@
# Game-mode integration & dedicated game sessions — design
> **Status: IMPLEMENTED (2026-07-05), Linux on-glass validation pending.** Parts A (A1A5) and B
> (B0B2) landed; `cargo build`/`test --workspace`/`clippy -D warnings`/`fmt` green, OpenAPI
> regenerated. It reconciles the shipped display-management layer (`design/display-management.md`,
> Stages 05 merged `95b3496`) with **session-mobile hosts** — Bazzite/SteamOS boxes that flip
> between Steam Game Mode (gamescope) and a KDE/GNOME desktop — and adds **dedicated game sessions**:
> a `game_session=dedicated` policy that serves every library launch from its own host-spawned
> headless gamescope at the client's exact mode, booting straight into the game with no Steam Big
> Picture and no game mode.
>
> ## Implementation status (as-built — read the deviations)
>
> - **A1 (ownership classes) — DONE.** `DisplayOwnership {Owned, External, SessionManaged}` on
> `VirtualOutput`; the registry pools only `Owned` outputs on the default daemon. gamescope
> spawn = `Owned`, attach = `External`, managed/SteamOS = `SessionManaged`, KWin/Mutter/Windows =
> `Owned`, wlroots = `Owned` but gated out by `remote_fd`. Plus `VirtualDisplay::poolable_now()`
> so a managed/attach acquire never reuses a kept bare-spawn. **This alone removes the reported
> Bazzite game-mode-reconnect stale-node wedge** (managed sessions are no longer pooled).
> - **A2 (validated reuse) — DONE.** Reuse keys on `(backend, mode, launch, epoch)`;
> `VirtualDisplay::kept_display_alive()` (gamescope checks its node is still present) tears a dead
> kept display down and recreates; `VirtualOutput::reused_gen` + `registry::mark_failed(gen)` on a
> reused-display first-frame failure so the retry loop creates fresh instead of re-wedging.
> - **A3 (managed restore) — DONE, but DEVIATES from the plan below.** The managed/SteamOS session
> is a **single-instance box singleton** (one Steam per uid), so making it a registry-`Owned` pool
> entry (as §4-A3 sketches) collides with its single-instance nature on a mode-change relaunch
> (two `ManagedSessionHandle` drops fighting over the shared `SESSION_UNIT`/autologin). Since A1
> already fixed the wedge, A3 instead **keeps the managed session's own single-instance lifecycle**
> (`MANAGED_SESSION`/`STOPPED_AUTOLOGIN`/`STEAMOS_TOOK_OVER` + the restore worker) and makes only
> its **restore policy-driven**: `schedule_restore_tv_session` reads `restore_delay()` = the
> `keep_alive` policy (`off`→0s, `duration`→N s, **`forever`→never** = gaming-rig held, unconfigured
> →5s bit-for-bit). Plus **crash-restore persistence** (`$XDG_RUNTIME_DIR/punktfunk-session-takeover.json`
> → `restore_takeover_on_startup`) and the **SIGKILL-teardown** experiment in `stop_session`
> (followups.md #1/#7). *Not done:* `/display/release` freeing a `forever`-held managed session
> (managed isn't a registry entry) — a forever managed box returns to gaming mode by manual switch
> or host restart (documented; acceptable for a dedicated couch appliance).
> - **A4 (session epoch) — DONE.** `session_epoch()`/`bump_session_epoch()` + `ActiveSession.compositor_pid`;
> `observe_session_instance()` (called from the per-connect resolve, the mid-stream watcher, the
> capture-loss re-detect, and the GameStream acquire) bumps the epoch + `registry::invalidate_backend`
> on a compositor-instance change (kind change OR same-kind restart). `take_expired` also reaps
> kept dead-epoch entries.
> - **A5 (addressed discovery) — PARTIAL.** Per-spawn **log** (`$XDG_RUNTIME_DIR/punktfunk-gamescope-<inst>.log`)
> + **scoped node discovery** (`find_gamescope_node_scoped` by `application.process.id` /
> `descends_from`) — fixes concurrent bare-spawn VIDEO-node ambiguity (the load-bearing correctness
> part). **DEFERRED:** the per-instance **EIS input relay** — the injector-coupling rework the map
> warned about ("EIS setup timed out"); input for concurrent gamescopes stays on the global relay,
> which is exactly `design/gamescope-multiuser.md` scope. Noted there.
> - **B0 (dedicated routing) — DONE.** `GameSession {Auto, Dedicated}` — a **top-level** `DisplayPolicy`
> field (NOT in `EffectivePolicy`, so presets don't clobber it). `pick_gamescope_mode` gains a leading
> `dedicated_launch` that forces `Spawn` (below explicit operator env, above managed-infra/foreign);
> `wants_dedicated_game_session()` (launch present ∧ policy dedicated ∧ gamescope available, else
> honest `auto` fallback) → `resolve_compositor(pref, dedicated)` forces gamescope; threaded through
> `apply_input_env(chosen, dedicated_launch)`. Steam URIs are shaped `steam -silent steam://…`
> (`shape_dedicated_command`). GameStream shares the dispatch.
> - **B1 (Steam single-instance takeover) — FOCUSED.** A dedicated **Steam** spawn frees the box's
> autologin/game-mode Steam first (`is_steam_launch``stop_autologin_sessions`, restored on session
> end via the A3 machinery). *Not done:* force-releasing a *kept dedicated Steam* session for a
> *different* Steam game (two concurrent dedicated Steam sessions) — rare; documented degradation.
> - **B2 (game-exit clean end) — DONE.** A dedicated gamescope session whose node vanishes
> (`dedicated_game_exited()` — gamescope is single-app, dies with its game) ends the session with the
> new `APP_EXITED_CLOSE_CODE` (0x52) so launcher clients return to their library, instead of the 40 s
> capture-loss rebuild timeout.
> - **B3 (docs/console) — this block + docs-site + CLAUDE.md; the web console toggle is the remaining
> surface.** The `game_session` axis is in the API + OpenAPI now.
>
> **Adversarial review pass (2026-07-05):** a multi-agent review confirmed 11 findings, all fixed:
> (1) `registry::acquire` held the pool mutex across the `pw-dump` liveness probe → the probe now runs
> OUTSIDE the lock (snapshot candidate under lock → probe → re-lock + re-find by gen); (25,7,10) the
> session epoch / `invalidate_backend` wrongly tore down **independent gamescope spawns** on any Gaming
> pid flap (the CRITICAL "dedicated Steam on a game-mode box self-destructs" — B1 stopping the autologin
> flipped the winning PID) → `observe_session_instance` now acts ONLY on **desktop** compositor changes,
> and `epoch_matches` exempts gamescope from the epoch (reuse + reap); (3) a dedicated-Steam reconnect
> didn't cancel the pending TV restore (reuse skips `create`) → `cancel_pending_tv_restore()` on every
> connect; (4,8) B2 game-exit used the UNSCOPED node scan → now scoped to the session's own `node_id`
> (threaded through the pipeline); (9) an unknown launch id → blank gamescope → dedicated now gated on
> the launch RESOLVING to a command. All green after fixes (`test --workspace` 360).
>
> **Remaining before this is "shipped":** Linux on-glass validation (`.116` Bazzite-KWin game-mode
> reconnect under presets, `.21`/`.116` dedicated launch, gaming-rig-forever held-managed, crash-restore,
> SIGKILL-teardown F44 check), and the two documented deferrals (A5 per-instance EIS relay →
> gamescope-multiuser; B1 concurrent-dedicated-Steam release).
>
> The **PLAN as originally written follows** — it still reads as "PLANNED"; the status block above is
> the authority on what actually shipped and where it deviates.
Companion docs: `design/display-management.md` (the policy/registry layer this reconciles — its
§7 capability matrix and §5.1 gamescope semantics are amended here),
`design/session-aware-host-followups.md` (open session-switch limitations #1/#7 are resolved by
Part A3 below), `design/gamescope-multiuser.md` (per-session input/audio isolation — adjacent;
Part B deliberately stops short of it and cross-references instead).
## 1. The two problems
**(A) The display-management rewrite does not compose with Bazzite game mode / KDE switching.**
The registry (`vdisplay/registry.rs`) assumes it owns every display it pools: it holds the
backend keepalive, decides linger/pin/teardown from the policy, and reuses kept displays by
`(backend, mode)`. On a session-mobile box that assumption is false twice over — the gamescope
managed/attach paths hand it outputs whose lifecycle is owned by *other* machinery
(`MANAGED_SESSION` + the debounced TV-restore worker), and the compositor under **every** Linux
backend can be killed and replaced at any moment by a Game↔Desktop switch. The result is pool
entries that lie: they linger or pin while the session behind them is torn down, then get reused
as dead nodes and wedge the whole retry budget.
**(B) Library launches deserve a first-class "just the game" mode.** Today a library launch on a
Bazzite box rides the managed `gamescope-session-plus` Steam session (Big Picture at the client's
mode, launch forwarded into the running Steam) — the game appears, but inside game mode's whole
UX, with Steam BPM boot time and game-mode ownership of the box. The bare-spawn path that nests
*just* the resolved command in a headless gamescope already exists and works — but the sub-mode
ladder makes it unreachable exactly where users want it most (any box with session-plus/SteamOS
infra picks managed). The ask: a host option so a library launch **always** gets a dedicated
gamescope session at the client's requested mode — game boots directly; non-Steam titles
instantly, Steam titles without any UI to navigate.
These are one design because both reduce to the same question: **who owns a gamescope session's
lifecycle, and how does the registry know?** Part A answers it for what exists; Part B builds the
new launch mode on the answer.
## 2. Failure inventory (code-anchored)
What actually goes wrong today, in dependency order:
1. **The registry pools externally-owned outputs.** `registry::linux::acquire` pools every output
with `remote_fd == None`. The gamescope **managed**, **SteamOS-takeover**, and **attach** paths
all return `remote_fd: None` with `keepalive: Box::new(())` (`gamescope.rs::create_managed_session`
/ `create_managed_session_steamos` / the attach arm) — a unit keepalive that keeps *nothing*
alive. The pool entry claims ownership of a display it cannot hold or tear down.
2. **Two lifecycle owners for the managed session → stale-node reuse wedge.** The real managed
lifecycle is `MANAGED_SESSION` + `PENDING_RESTORE` + `RESTORE_DEBOUNCE` (hardcoded 5 s) + the
restore worker, which stops the session and restarts the TV autologin. With any `keep_alive`
policy configured, a disconnect leaves a registry entry Lingering (e.g. 300 s) while the restore
worker kills the session at 5 s. A reconnect inside the linger window **reuses the dead
`node_id`**; capture fails "no PipeWire frame within 10s"; the capturer drop returns the entry to
Lingering; `build_pipeline_with_retry` re-acquires the **same corpse** on every attempt — all 8
retries wedge (~90 s). This is the `.41`-class "game-mode reconnect broken" symptom, now
*manufacturable on any Bazzite box by clicking a console preset*.
3. **`gaming-rig` (keep_alive=forever) is a lie on gamescope.** Design §5.1 promised "the policy
duration replaces the hardcoded 5 s debounce; forever = the TV session is never auto-restored".
Never implemented: the restore worker doesn't read the policy, so `forever` pins an entry whose
session the worker restores away regardless.
4. **A session switch leaves zombie entries.** The mid-stream watcher rebuild and the per-connect
re-detect drop the old backend's lease → the old entry lingers/pins **for a compositor that no
longer exists** (KWin dies on the switch to game mode; the game-mode gamescope dies on the switch
to desktop). `/display/state` lies; expiry drops keepalives into dead compositors; a
Desktop→Game→Desktop bounce brings up a *new* KWin whose node-id space has no relation to the
kept entry's — reuse hands out a number that now means nothing.
5. **Discovery is ambient and ambiguous.** One shared spawn log (`/tmp/punktfunk-gamescope.log`),
name-based node discovery (`find_gamescope_node` returns *any* `gamescope` `Video/Source`), and
one global EIS relay file (`punktfunk-gamescope-ei`). Correct only while at most one gamescope
exists per uid. A kept spawn + live game mode, two concurrent spawns, or (Part B) a dedicated
session next to game mode can capture the wrong node and inject into the wrong session.
6. **Stock Bazzite restarts the systemd user manager on every Game↔Desktop switch** (observed on
`.41`: user-manager PID churn), killing the host, the compositor, and the pool together. Inherent
— keep-alive **cannot** span a switch on a stock setup (the headless-appliance setup —
`enable-linger` + `multi-user.target` — keeps the host alive and can). Separately, the takeover
bookkeeping (`STOPPED_AUTOLOGIN`, `STEAMOS_TOOK_OVER`, the SteamOS drop-in) is process memory
only: a host crash mid-stream strands the box out of game mode with no restore on restart.
## 3. Design principles
- **One owner per display.** Every `VirtualOutput` declares who owns its lifecycle; the registry
pools only what it owns. This extends the CLAUDE.md invariant ("display lifecycle is owned by
the registry; sessions hold leases") with its honest converse: *what the registry does not own,
it must not pretend to keep.*
- **Keep-alive is an optimization, never a failure mode** (display-management §3, now enforced):
reuse validates liveness; a failed reuse invalidates and creates fresh; the retry budget is
never spent twice on the same corpse.
- **The compositor instance, not the host process, is the unit of session truth.** A session
epoch invalidates every display created under a previous compositor instance — kind changes
*and* same-kind restarts.
- **Addressed, not ambient, discovery.** Every spawned gamescope gets its own log, its own node
resolution, its own EIS relay. Global singletons are what break the moment two sessions coexist.
- **Launch identity is part of display identity.** A kept display running game A must never be
handed to a session asking for game B.
## 4. Part A — ownership & session mobility
### A1. Ownership classes (the structural fix — ship first)
`VirtualOutput` gains an ownership declaration:
```rust
pub enum DisplayOwnership {
/// The registry owns lifecycle: pool, linger, pin, tear down. (KWin, Mutter, gamescope
/// SPAWN, Windows manager-delegated.)
Owned,
/// Someone else's display, mirrored: no keep-alive, no topology, no reuse — codifies the
/// display-management §7 "attach = unmanaged pass-through" row the code never implemented.
/// (gamescope ATTACH; wlroots stays gated on remote_fd as today.)
External,
/// A box-level session the gamescope module manages (managed session-plus / SteamOS
/// takeover). Pass-through at A1 (capturer owns the lease as pre-Stage-1); A3 converts it
/// to Owned by giving the registry the real keepalive + restore duty.
SessionManaged,
}
```
The registry pools `Owned` only; `External`/`SessionManaged` pass through unchanged (today's
pre-registry behavior — teardown-on-capturer-drop is a no-op for their unit keepalives, and the
existing gamescope-module machinery keeps doing what it does). `remote_fd == Some` keeps its gate.
Effect: failures 13 stop being reachable from the console — a preset can no longer create lying
entries for game-mode sessions. Smallest possible diff; everything else layers on it. Until A3
lands, keep-alive on gamescope managed/attach honestly reports **unsupported** in
`/display/state` capabilities instead of pretending.
### A2. Validated reuse, invalidation, launch key
- **Launch key.** `Entry` gains `launch: Option<String>` (the resolved launch command the display
was created with; `None` = plain desktop). Reuse requires `(backend, mode, launch)` equality.
Without this, a kept spawn running game A is reused for a session that asked to launch game B —
latent today (unconfigured Linux lingers Immediate), live the moment keep-alive + launches
combine, load-bearing for Part B.
- **Liveness probe at reuse.** New trait method, default honest:
`fn kept_display_alive(&mut self, node_id: u32) -> bool { true }` — gamescope-spawn checks the
child (`try_wait`) *and* the node; KWin/Mutter check the node still exists on the default
daemon (one cheap PipeWire registry roundtrip — no capture attach). A dead entry is torn down
(its group restore handed off / run) and the acquire falls through to a fresh create.
- **Invalidation on reuse failure.** `acquire` marks the returned `VirtualOutput` as reused (a
flag on the lease/gen); when `build_pipeline`'s first-frame fails on a **reused** display, it
calls `registry::mark_failed(gen)` before returning the error — the entry is torn down, so the
retry loop's next `acquire` **creates fresh** instead of re-wedging on the same corpse. This is
the direct fix for failure 2's 8×10 s wedge shape, and it holds for every future way a kept
display can silently die.
### A3. One owner for the managed session (registry-owned restore)
The managed/SteamOS paths become `Owned` by giving the registry the two things it lacks:
- **A real keepalive.** `create_managed_session*` returns a `ManagedSessionHandle` whose `Drop`
performs today's `do_restore_tv_session` duty: stop the transient unit / remove the SteamOS
drop-in, then restart the autologin **iff no desktop session is active** (the existing guard
moves in verbatim — a user who switched to KDE mid-linger is never yanked back to game mode).
- **The policy as the debounce.** The registry linger *is* the restore delay: unconfigured
default = 5 s (bit-for-bit today's `RESTORE_DEBOUNCE`), a configured duration replaces it,
`forever` = never auto-restore — `gaming-rig` finally means on a Bazzite couch box what its
story says, released via `/display/release` or the tray. A reconnect inside the linger is a
registry **reuse** (the same warm-session fast path `PENDING_RESTORE`-cancel gives today, now
with A2 validation); a different requested mode tears down + relaunches through the registry
(gamescope's honest "reconfigure = recreate").
- **Retire the parallel machinery.** `PENDING_RESTORE`, `RESTORE_DEBOUNCE`,
`start_restore_worker`, and the `restore_managed_session()` call sites go away — the registry
linger timer is the one timer. `MANAGED_SESSION` survives only as the module's mode/unit cache.
`STOPPED_AUTOLOGIN`/`STEAMOS_TOOK_OVER` stay as the *mechanics* the handle's Drop consumes.
- **Persist the takeover.** The stopped-unit list + SteamOS-drop-in marker are written to
`$XDG_RUNTIME_DIR/punktfunk-session-takeover.json` at takeover, cleared at restore. On host
startup, a leftover file (crash / service restart mid-stream) schedules a restore after a short
reconnect grace — with the same active-desktop guard. A crashed host no longer strands the TV.
- **Teardown signal.** Adopt the parked follow-ups doc #1 experiment here: the handle's stop uses
`systemctl --user kill --signal=SIGKILL <unit>` (+ `stop`/`reset-failed` to clear unit state)
instead of plain SIGTERM stop, testing the hypothesis that skipping gamescope's crashy SIGTERM
teardown avoids the F44 GPU-context leak. A3's validation pass is the natural place to measure
it; fall back to SIGTERM if SIGKILL misbehaves. Follow-ups #7 (restore-guard/keep-warm
interaction) dissolves: "keep warm" is now just `keep_alive: forever`.
### A4. Session epoch & backend invalidation
- **`vdisplay::session_epoch()`** — a host-global counter bumped whenever session detection
observes a different compositor **instance**: an `ActiveKind` change *or* a new compositor PID
for the same kind (the Desktop→Game→Desktop bounce). Entries stamp their creation epoch; reuse
requires an epoch match; the linger timer reaps entries from dead epochs (their keepalive Drops
hit dead sockets and fail fast; the registry already drops outside the lock).
- **Watcher hook.** A `SessionSwitch` (and a per-connect re-detect that finds the previous
backend's compositor gone) additionally calls `registry::invalidate_backend(old)` so
`/display/state` is honest immediately rather than at the next expiry poll.
- **Stock-Bazzite user-manager restarts** stay inherent: the host dies with the switch, the pool
dies with the host, and that is documented ("keep-alive spans a Game↔Desktop switch only on the
headless-appliance setup"). The persisted takeover file (A3) is what survives.
### A5. Addressed discovery (Part-B prerequisite)
- **Per-spawn log**: `$XDG_RUNTIME_DIR/punktfunk-gamescope-<gen>.log`; the spawned instance's
node id is parsed from *its* log only.
- **Scoped node discovery**: PipeWire node props carry `application.process.id` — a spawn's node
must belong to our child's PID tree (`descends_from`, already written); managed/attach
discovery conversely **excludes** nodes owned by our spawned children. `find_gamescope_node`
grows a scope parameter instead of "first node named gamescope".
- **Per-instance EIS relay**: `punktfunk-gamescope-ei-<gen>`, path carried on `VirtualOutput`
(the gamescope-multiuser doc's item 1), with the injector service resolving the session's own
relay for gamescope sessions (a narrow slice of its item 2 — the shared injector stays for the
portal backends, where shared input is correct and where per-session churn caused the historic
"EIS setup timed out" wedge).
### Part A validation (on-glass, `.116` Bazzite KWin/AMD + Deck `.253`; `.21` for spawn-on-GNOME)
1. Game-mode reconnect: connect → disconnect → reconnect inside game mode, under *every* preset
incl. `gaming-rig` (the failure-2/3 repro — must reuse the warm session or cleanly recreate,
never wedge the retry budget).
2. Game↔Desktop switch mid-linger both directions; Desktop→Game→Desktop bounce (epoch test);
`/display/state` never lists a display whose compositor is dead.
3. `gaming-rig` on Bazzite: TV stays off until `/display/release`; a KDE switch mid-linger is not
yanked back to game mode.
4. Kill -9 the host mid-takeover → restart → TV session restored (persisted-takeover test).
5. Two gamescopes coexisting (kept spawn + game mode): capture and input land in the right one.
## 5. Part B — dedicated game sessions for library launches
### 5.1 What it is
A session that carries a launch id (native `Hello.launch`, the GTK `--browse`/`--launch` flows,
Decky pins, GameStream apps with a library id) can be served by a **dedicated gamescope
session**: a host-spawned headless gamescope at exactly the client's WxH@Hz whose nested command
is the resolved game. No Steam Big Picture, no game mode, no desktop involvement. Session end
returns the client to its launcher (already shipped behavior); the game survives disconnects per
`keep_alive` — the Apollo-style detach/reattach the design always wanted, now per-game.
### 5.2 Policy surface
One new axis in `display-settings.json` (same store/PUT/console pattern; serde-defaulted so
existing files are untouched):
```json5
// How a session that LAUNCHES a game is served:
// "auto" today's routing: the launch rides whatever session the box is in (managed
// Steam session on Bazzite/SteamOS, bare spawn on plain distros, spawned into
// the live desktop on KWin/Mutter/wlroots)
// "dedicated" a launching session always gets its own headless gamescope at the client's
// mode, nesting just the game
"game_session": "auto"
```
Sessions **without** a launch id are untouched by this axis — desktop streaming routes exactly as
today. `dedicated` degrades honestly: no gamescope binary → log + fall back to `auto`. Console: a
toggle on the Virtual displays card (it is a display-lifecycle decision) with one story line.
Deliberate non-options for v1: per-entry overrides (schema keys allow a later
`"per_entry": {"<id>": …}` overlay) and a client-requested Hello byte ("launch dedicated") — host
policy first, protocol growth when a client actually wants to differ per connect.
### 5.3 Routing & command shaping
- **Sub-mode ladder.** `pick_gamescope_mode` (pure, unit-tested) gains a leading
`dedicated_launch: bool` input that forces `Spawn`, outranking managed-infra/foreign-attach
(explicit operator `MANAGED`/`ATTACH`/`NODE` envs still win — they are debug/CI overrides).
`resolve_compositor` computes it: launch id present ∧ policy `dedicated` ∧ gamescope available
→ chosen = `Gamescope`, spawn sub-mode; `launch_is_nested` then routes the command into the
spawn as today.
- **Non-Steam entries** (custom / lutris / heroic): the resolved command nests directly — truly
instant (gamescope up in ~1 s, then the game's own boot).
- **Steam entries** (`steam steam://rungameid/<id>`): Steam is single-instance per uid, so:
1. Command shape becomes `steam -silent steam://rungameid/<id>` in dedicated mode — `-silent`
suppresses the Steam main window so the game is the gamescope focus. **Empirical validation
item** (behavior of `-silent` under a fresh nested Steam); fallback is the plain URI form
(a briefly-visible Steam client, still no BPM navigation).
2. If another same-uid Steam is live (game mode autologin, a kept managed session, a kept
dedicated Steam session): **take Steam over first** — force-release kept entries whose
`launch` is a Steam title, stop the autologin via the A3-owned takeover (persisted state,
policy-driven restore). This reuses the exact machinery game mode streaming already needs;
`dedicated` adds no new churn class.
- **Game exit ends the session.** When the nested command exits (user quits the game), the
gamescope child dies; today's capture-loss path would rebuild an empty session. Dedicated
sessions instead end cleanly: the rebuild path consults the keepalive child (`try_wait`) and a
confirmed child exit becomes a typed session end (host closes QUIC with a new
`APP_EXITED_CLOSE_CODE`, sibling of `QUIT_CLOSE_CODE` 0x51, so launchers can distinguish "game
ended" from an error) with `force_immediate` release. Clients need no change to *work* (they
already return to the launcher on session end); the typed code is polish they can adopt.
- **Mid-stream `Reconfigure`** on a dedicated session = teardown + respawn (gamescope cannot
live-resize its output; the game restarts) — the same honest §7 caveat as managed, documented.
### 5.4 Lifecycle composition (where Part A pays off)
- Dedicated outputs are `Owned` (A1) with the child as a real keepalive → they pool naturally.
- Reuse keys on `(backend, mode, launch)` (A2): reconnect to the same game re-attaches to the
still-running session instantly; a different game never falsely reuses. `keep_alive` then reads
exactly as the presets promise: `off` = game dies with the disconnect; a duration = the detach
window; `forever` = the game runs until released (`gaming-rig`, per-game).
- Each dedicated session is already its own registry **group** (`group_key` — no topology or
restore interaction with the desktop); `max_displays` bounds how many can be kept; Steam
titles are additionally bounded to one by the single-instance takeover above.
- Input rides the per-instance EIS relay (A5); uinput gamepads are per-session already. Audio and
mic stay the host-wide shared services — one *active* dedicated session is the designed case;
concurrent independent-audio sessions are exactly `design/gamescope-multiuser.md` scope, not
re-solved here.
- Admission (`mode_conflict`) applies unchanged across clients; a second client asking for a
different dedicated game under `separate` gets its own spawn (non-Steam) or the single-instance
takeover rules (Steam) — the honest per-backend gating pattern.
### 5.5 GameStream
Same dispatch (the launch path was unified in the 2026-07-01 rebuild): an apps.json/library-id
launch under `dedicated` spawns the same way; serverinfo/RTSP negotiate the client's mode as
today. Moonlight's quit-app (`h_cancel`) maps to a `force_immediate` release — killing the game
on explicit quit — which folds into the already-deferred "GameStream quit-code" follow-up from
display-management §5.1.
### 5.6 Client experience & latency honesty
No client changes are required: GTK `--browse`/`--launch`, Decky pins, and the Apple/Android
library grids just launch, and session end already returns to the launcher. Boot-time
expectations, stated plainly in docs: non-Steam titles are gamescope-spawn (~1 s) + game boot;
Steam titles pay the Steam client's own cold boot inside the fresh session (~1025 s class)
before the game process starts — "no UI to navigate", not "zero seconds". A pre-warmed parked
Steam (`steam -silent` held inside a background headless gamescope at host start) would close
that gap but fights game mode over the single instance — explicitly out of scope for v1, noted
as the one candidate v2 if Steam cold boot proves to be the complaint. **Stretch** (needs a small
mgmt surface, not v1): launchers showing "Resume" for a game the host reports as kept
(`/display/state` already exposes the entries; adding `launch` to `DisplayInfo` is the only
schema growth).
## 6. Staging & dependencies
| Stage | Contents | Depends on | Validation |
|---|---|---|---|
| **A1** | `DisplayOwnership`, pool `Owned` only, honest capabilities | — | unit + `.116` game-mode reconnect under presets no longer wedges |
| **A2** | launch key, `kept_display_alive`, `mark_failed` on reused-display capture failure | A1 | probe-driven: kill a kept display's session → reconnect creates fresh on attempt 2 |
| **A3** | `ManagedSessionHandle`, policy-as-debounce, retire restore worker, persisted takeover, SIGKILL-teardown experiment | A1 | `.116`/Deck: gaming-rig semantics, crash-restore, desktop-guard |
| **A4** | session epoch, `invalidate_backend` on switch | A1 | `.116`: switch matrix incl. same-kind bounce; `/display/state` honesty |
| **A5** | per-spawn log, scoped node discovery, per-instance EIS relay | — (parallel) | two-gamescope coexistence test |
| **B0** | `game_session` policy + ladder input + steam `-silent` shaping | A1 (correctness), A5 (if game mode may be live) | `.21`/plain box: non-Steam + Steam dedicated launch on glass |
| **B1** | Steam takeover integration (autologin stop / kept-Steam release) | A3 | `.116`/Deck: dedicated launch from game mode, TV restore per policy |
| **B2** | reuse-by-launch reattach + game-exit-ends-session (`APP_EXITED` close) | A2 | disconnect → game keeps running → reattach; quit game → launcher |
| **B3** | docs (virtual-displays + steamos-host pages), console toggle polish, "Resume" stretch | B0B2 | — |
Every stage lands green (`cargo test/clippy/fmt`, OpenAPI drift) and independently shippable,
per the display-management discipline. A1+A2 alone fix the user-visible breakage; A3 makes the
presets truthful; B0 is the first user-visible new capability.
## 7. Risks & open questions
- **`steam -silent` inside a fresh nested gamescope** — the load-bearing empirical unknown for
B0's Steam polish (the launch itself works either way; only the cosmetic Steam-window flash is
at stake). Validate first on `.21`/`.116` desktop mode.
- **Bare spawn vs session-plus environment** — session-plus wraps Steam in MangoApp/runtime/
controller-config scaffolding a bare `gamescope -- steam` lacks. The historic "nested Steam
crashes" finding was Steam-vs-Steam single-instance (both dying), *not* a missing-scaffolding
failure — with Steam taken over first, a bare spawn should hold, but this is exactly what B1's
on-glass pass must prove per box (Bazzite, Deck).
- **PipeWire `application.process.id` availability** across gamescope versions (A5's scoped
discovery) — fall back to log-derived ids (per-spawn logs make those unambiguous already).
- **Keepalive drops into dead compositors** (A4 reaping): Wayland conns fail fast; Mutter's D-Bus
`Stop` can block — the registry already drops outside the lock, keep it that way and bound the
damage to one reaper tick.
- **Epoch granularity**: detecting "new compositor instance, same kind" needs the compositor PID
in `ActiveSession` — cheap (the `/proc` scan already visits it), but the watcher must debounce
PID flaps during a switch (its existing 3 s debounce covers it).
- **Injector rework caution** (A5): per-session EIS injectors only for gamescope; the shared
service stays for portal backends — re-learning the "EIS setup timed out" lesson is the failure
mode to guard in review.
- **Windows**: entirely untouched — `game_session` is Linux-only for now (`launch_title` on
Windows opens via the shell into the one desktop); the policy field documents that honestly
rather than pretending.
+287
View File
@@ -0,0 +1,287 @@
# Native AMF encoder — handoff design
> **Status: PHASES 1 + 2 + 3 IMPLEMENTED (2026-07-06).** `encode/windows/amf.rs` ships the
> direct-SDK encoder per §3 — FFI pinned to AMF headers v1.4.36, bounded poll, native `reset()`.
> Phase 2: **AV1** (open-time probe gate; per-codec enum divergences honored — AV1 swaps the
> ULL/LL usage values and uses GOP=0 + FORCE_FRAME_TYPE_KEY=1), **intra-refresh**
> (`PUNKTFUNK_INTRA_REFRESH` opt-in mirroring Linux NVENC; `caps().intra_refresh` reflects the
> driver's actual acceptance), **in-band HDR mastering/CLL metadata** (`*InHDRMetadata` host
> buffer; HEVC + AV1), and the **native codec probe**. Phase 3: **the ffmpeg-AMF dispatch
> fallback + `PUNKTFUNK_AMF_FFMPEG` hatch are deleted** — AMD dispatch / codec advertisement /
> 4:4:4 answer are native-only; FFmpeg serves QSV only (`ffmpeg_win.rs`'s AMF machinery is kept
> solely as the A/B comparator). `windows_backend_is_ffmpeg``windows_backend_is_probed`. **The
> §7 field-silence gate on Phase 3 was pre-empted on explicit direction** — see the §7 gate note
> for what that costs (VP-format-fallback now fails the session; AMFVideoConverter is the owed
> native fix).
> Live-validated on the lab Ryzen iGPU (VCN 3): AVC + HEVC batches across a native in-place reset
> (Annex-B IDR contract, FIFO pairing); HEVC Main10 P010 with the mastering + CLL prefix SEIs
> **confirmed present in the encoded IDR**; intra-refresh property accepted on both codecs; probe
> honestly answers h264/h265=true, av1=false on this RDNA2 part. The **§5.2 latency A/B is
> measured** (`amf_latency_ab_bench`, 1080p60 HEVC): native `encode_us` p50 **5.18 ms (0.31 frame
> periods)** vs libavcodec-AMF **16.9 ms (1.01)** — 3.3× lower, the frame-hold gone. Owed: §5.3
> on-glass session behaviors + soak (macOS-client on-glass test in progress), and field
> validation on ≥2 VCN generations (AV1/RDNA3 has no lab hardware).
> Companion context: the encode-stall watchdog + `Encoder::reset()` (punktfunk1.rs / encode.rs,
> shipped 2026-07-06) and the QSV backend module docs in `encode/windows/ffmpeg_win.rs`.
## 1. Why (measured, not speculative)
Three independent reasons, in order of weight:
1. **The libavcodec AMF wrapper's structural ~2-frame output hold.** `hevc_amf`/`av1_amf`
need frame N+2 submitted before they release frame N's AU. Measured on the Ryzen 7000
iGPU (VCN, 720p60): encode→retrieve **36 ms p50, dead-stable**, invariant across pipeline
depth 1/2, every `usage` preset, and any spin budget (a 150 ms poll spin provably never
produced the owed AU — it pegged at exactly 150 ms). See the `poll` doc comment in
`ffmpeg_win.rs`. The direct-SDK NVENC path retrieves in ~12 ms. At 60 Hz this is ~33 ms
of pure pipeline latency no FFmpeg-side knob can remove; at 120 Hz it is two whole frame
budgets.
2. **Silent driver wedges surface as forever-EAGAIN, not errors.** The field failure
(AMD/Intel streams freezing after ~35 min) was invisible because the wrapper's only
"not ready" signal is EAGAIN, indistinguishable from a healthy pipeline warming up. The
2026-07-06 watchdog converts that into a bounded rebuild + IDR, but it is a safety net
with a ~2 s detection floor. The AMF runtime itself returns typed `AMF_RESULT` codes
(`AMF_INPUT_FULL`, device-lost, etc.) — a native path sees the wedge on the frame it
happens.
3. **Feature gaps libavcodec cannot express.** No intra-refresh wave (every
FEC-unrecoverable loss is answered with a full IDR — the 2040× frame-size spike the
Linux NVENC intra-refresh mode exists to avoid), no in-band HDR mastering SEI
(`EncoderCaps::supports_hdr_metadata` is NVENC-only today), coarse per-frame control.
## 2. The decision: drop FFmpeg for AMF, keep it for QSV
**Native AMF replaces the libavcodec AMF path (phased, §7). FFmpeg stays for QSV.**
- QSV via libavcodec with `async_depth=1` + `low_power` VDEnc is already near the hardware
latency floor; a direct libvpl port would buy little for its cost. Revisit only if Intel
field data shows a QSV-specific gap (separate doc if so).
- Because QSV stays on FFmpeg, the FFmpeg DLLs keep shipping and the `amf-qsv` build
feature keeps existing. Dropping FFmpeg *entirely* is therefore not on the table here —
"drop" means: the AMF dispatch stops going through it once the native path is validated.
- During bring-up the ffmpeg-AMF path remains as an automatic open-failure fallback and an
explicit escape hatch (§7), then its AMF dispatch is deleted in Phase 3. Two permanently
maintained AMF paths would double the driver-matrix burden, and the one we'd keep "for
safety" is precisely the one with the wedge/latency pathology.
## 3. Architecture
New module `crates/punktfunk-host/src/encode/windows/amf.rs` implementing
`crate::encode::Encoder`, compiled unconditionally on Windows (**no new build-time
dependency and no new cargo feature**): the AMF runtime is loaded at runtime from the
driver-installed `amfrt64.dll`, exactly as `nvenc.rs` loads `nvEncodeAPI64.dll`
(`load_api`). A box without an AMD driver simply fails the open and the dispatch falls
through. In-tree FFI decl module (`amf_sys` submodule or `#[repr(C)]` blocks in-file,
mirroring the small interface subset we use) — model it on how `ffmpeg_win.rs` mirrors
`AVD3D11VADeviceContext` rather than pulling a binding crate (none is maintained).
### 3.1 FFI strategy (the load-bearing detail)
The AMF public headers (GPUOpen `AMF/amf/public/include`) define **C-compatible vtable
structs** for every interface (`AMFFactoryVtbl`, `AMFContextVtbl`, `AMFComponentVtbl`,
`AMFSurfaceVtbl`, `AMFDataVtbl`, `AMFBufferVtbl`, `AMFVariantStruct`, …) — this is not a
guess: FFmpeg's `amfenc.c` is plain C and drives AMF exclusively through those vtables, so
the C ABI is the stable, supported surface. Mirror only what we call:
- Entry points: `GetProcAddress("AMFQueryVersion")` (gate: runtime ≥ the pinned
`AMF_FULL_VERSION` we mirror headers from) and `GetProcAddress("AMFInit")`
`AMFFactory*`.
- `factory->CreateContext``context->InitDX11(capturer_device, AMF_DX11_1)` — **the
capturer's own `ID3D11Device`**, same-device requirement as every other backend (the
capture textures are not shared-handle; see the `ensure_inner_d3d11` rebind logic in
`ffmpeg_win.rs` for the device-change lifecycle to replicate).
- `factory->CreateComponent(context, name)` with `AMFVideoEncoderVCE_AVC` /
`AMFVideoEncoder_HEVC` / `AMFVideoEncoder_AV1` → `encoder->Init(AMF_SURFACE_NV12|P010,
w, h)`.
- Per-frame: `context->CreateSurfaceFromDX11Native(texture, &surface, observer)`
per-surface properties (pts via `SetPts`, forced-IDR picture type) →
`encoder->SubmitInput(surface)`; retrieve via `encoder->QueryOutput(&data)`
(`AMF_REPEAT` = not ready), `AMFBuffer::GetNative/GetSize``EncodedFrame`.
- Every mirrored struct/call carries a `// SAFETY:` proof — the whole encode module tree is
under `#![deny(clippy::undocumented_unsafe_blocks)]` (unsafe-proof program).
Reference implementations to crib from (read both before writing FFI): FFmpeg `amfenc.c`
(the C vtbl usage, property plumbing, result-code handling) and OBS
`plugins/obs-ffmpeg/texture-amf.cpp` (D3D11 texture submission + low-latency streaming
config, C++ but the call sequence is what matters). **Verify every property name against
the pinned SDK headers** — names below are from those references and must not be trusted
blind.
### 3.2 Input path (zero-copy by construction)
Own a small D3D11 texture ring (NV12 or P010, `D3D11_BIND_RENDER_TARGET |
SHADER_RESOURCE`, size = `pipeline_depth + 2`), `CopySubresourceRegion` the captured
texture into the next slot (GPU-local, same pattern as `ZeroCopyInner::submit`), wrap the
slot with `CreateSurfaceFromDX11Native`, submit. The copy decouples the encoder from the
capturer's rotating IDD ring; do NOT wrap the capturer's texture directly. This makes
`PUNKTFUNK_ZEROCOPY` irrelevant for native AMF — there is no readback path to fall back
to. Handle the capturer's video-processor format fallback (`Bgra`/`Rgb10a2` instead of
NV12/P010 — see `pool_mismatch` in `ffmpeg_win.rs`) by returning an open/submit error in
Phase 1 so dispatch falls back to the ffmpeg path; an AMFVideoConverter front-end is a
Phase 2 option if that fallback ever fires in the field. `FramePayload::Cpu` (DDA without
video processor): same treatment — ffmpeg fallback in Phase 1.
### 3.3 Retrieval model
Bounded-blocking poll, the `vaapi.rs::poll` model: after `SubmitInput`, spin
`QueryOutput` with ~250 µs sleeps up to a budget of `min(3/4 frame interval, 12 ms)`; on
expiry return `Ok(None)` (the session loop keeps the frame in flight and the watchdog
arbitrates wedges). VCN encode at streaming settings is ~15 ms, so the AU ships the same
tick — this is where the ~2-frame hold dies. Expected observable — measure **`encode_us`
(submit→AU, in `FrameMsg`/the web-console stats), not `wait_us`**: on the ffmpeg path the
hold hides in `encode_us` (~2 frame periods) because its non-blocking poll returns EAGAIN
in ~2 µs; on the native bounded poll the ASIC wait becomes visible as a few ms of
`wait_us` while `encode_us` collapses to ~1 frame period or less.
### 3.4 Encoder configuration (initial property set)
Mirror the intent of the ffmpeg opts block in `open_win_encoder` (`ffmpeg_win.rs:216-247`).
AVC names given; HEVC/AV1 have `_HEVC_`/`_AV1_` twins — check headers:
| Intent | AMF property (verify!) |
| --- | --- |
| usage preset (keep `PUNKTFUNK_AMF_USAGE` mapping) | `AMF_VIDEO_ENCODER_USAGE` = `ULTRA_LOW_LATENCY` (default) |
| CBR, target==peak | `RATE_CONTROL_METHOD=CBR`, `TARGET_BITRATE`, `PEAK_BITRATE` |
| 1-frame VBV (keep `PUNKTFUNK_VBV_FRAMES`) | `VBV_BUFFER_SIZE` |
| HRD + no filler | `ENFORCE_HRD=true`, `FILLER_DATA_ENABLE=false` |
| latency-first quality | `QUALITY_PRESET=SPEED` |
| no B-frames (AVC) | `B_PIC_PATTERN=0` |
| infinite GOP | `IDR_PERIOD=0` (HEVC: `GOP_SIZE`/`NUM_GOPS_PER_IDR` — check) |
| low-latency submission | `LOWLATENCY_MODE=true` (newer SDKs) |
| in-band VPS/SPS/PPS on IDR (wire contract: `EncodedFrame` doc) | HEVC `HEADER_INSERTION_MODE=IDR_ALIGNED`; AVC `HEADER_INSERTION_SPACING` — check |
| SDR/HDR VUI | `FULL_RANGE_COLOR=false` + color primaries/transfer/matrix props (BT.709 vs BT.2020-PQ, mirroring `open_win_encoder`) |
| 10-bit | `COLOR_BIT_DEPTH=10` + P010 surfaces |
| per-frame forced IDR | on the input surface: `AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE=IDR` |
| intra-refresh wave (Phase 2) | AVC `INTRA_REFRESH_NUM_MBS_PER_SLOT`; HEVC CTB twin — check |
| HDR mastering SEI (Phase 2) | HEVC `INPUT_HDR_METADATA` (`AMFHDRMetadata` buffer) |
`SetProperty` failures on *optional* properties (LOWLATENCY_MODE, intra-refresh) must be
log-and-continue, not fatal — availability varies by VCN generation/driver.
### 3.5 Error + stall semantics (interplay with the 2026-07-06 watchdog)
- `SubmitInput``AMF_INPUT_FULL`: **back-pressure, NOT a wedge — drain and retry, do not
reset.** (Original prescription "return `Err` → in-place reset" was **wrong**, disproven
on-glass 2026-07-06: at throughput-ceiling loads — 5120x1440@240 P010 on the lab iGPU —
`INPUT_FULL` → reset → forced IDR → a bigger keyframe → worse overload → a ~320 ms
reset/IDR cascade, strictly worse than the libavcodec path's 16-deep input queue riding it
out as latency. The log showed dozens of `submit failed … AMF_INPUT_FULL … rebuilt in place`
and **zero** watchdog stalls.) The shipped handling: `submit` bounds in-flight surfaces below
the input ring depth (`pending.len() < RING`) by draining finished AUs (buffered in a `ready`
deque for `poll`, FIFO-preserved) to free a slot *before* reusing it, and treats a stray
`INPUT_FULL` from `SubmitInput` the same way (drain + retry the surface). Only a drain that
makes NO progress for a bounded budget (`INPUT_DRAIN_BUDGET`, 200 ms — well under the
session watchdog's ~2 s) is a genuine wedge that escalates to `Err` → the in-place reset. This
also closed a **latent corruption**: the old path let in-flight grow to AMF's internal input
queue limit (16) against a ring of 4, so surfaces referenced ring slots already overwritten —
the reset masked it. Any other non-OK `SubmitInput` result: `Err`.
- `QueryOutput``AMF_REPEAT`: keep spinning within the poll budget, then `Ok(None)`.
`AMF_EOF`: `Ok(None)` after flush. Anything else: `Err` (the loop's poll-error path
resets).
- Implement `Encoder::reset()` natively: `encoder->Drain/Flush`, `Terminate()`, re-`Init`
on the same context (fall back to full context teardown if re-Init fails). Cheaper and
more targeted than the ffmpeg path's drop-and-lazily-reopen.
- `caps()`: `supports_rfi: false` (AMF has no NVENC-style reference invalidation —
intra-refresh is the substitute), `intra_refresh: true` once Phase 2 lands (this flag is
what makes the session glue rate-limit client keyframe requests — see the `IDR_WINDOW`
logic in punktfunk1.rs), `supports_hdr_metadata: true` once the SEI lands,
`chroma_444: false` **permanently** (VCN hardware does not encode 4:4:4;
`probe_can_encode_444` stays false — this is not an FFmpeg limitation).
### 3.6 Encoder trait contract (do not break)
From `encode.rs` + the punktfunk1 loop: AUs must come out FIFO in submit order
(`inflight.pop_front()` pairs with poll order); `data` is Annex-B with in-band headers on
IDRs (both a playable ES and self-contained wire AUs); `poll` returning `Ok(None)` is
legal and watchdog-arbitrated; `submit` must never block indefinitely; after `flush()`,
`poll` drains remaining AUs then returns `Ok(None)`; single encode thread owns the
encoder (manual `unsafe impl Send` with the same proof shape as `FfmpegWinEncoder`).
## 4. Integration seams (exact)
- `encode.rs::open_video_backend`, `WindowsBackend::Amf` arm: try `amf::AmfEncoder::open`
first; on `Err`, `tracing::warn!` + fall back to `ffmpeg_win` (when the `amf-qsv`
feature is built) — the same graceful-degrade shape as zero-copy→system today. Escape
hatch: `PUNKTFUNK_AMF_FFMPEG=1` skips the native path (field triage). Phase 3 deletes
both the fallback arm and the hatch.
- `resolved_backend_label` / `crate::gpu` session record: new label `"amf"` stays (the
mgmt API shows the same name; add `"amf-ffmpeg"` only if the fallback fires, so field
logs distinguish the paths).
- `probe_can_encode` (GameStream codec advertisement) and `windows_codec_support`: replace
the ffmpeg open-probe with a native factory probe (`CreateComponent` per codec on the
selected adapter) once Phase 2 lands; cache shape stays.
- `can_encode_444`: unchanged (`false`).
- The encode-stall watchdog and `Encoder::reset` (punktfunk1.rs): unchanged — it remains
the backstop for in-FFI hangs the native path can't self-detect.
## 5. Validation plan (this box has an AMD iGPU — use it)
Baseline first, on the ffmpeg path (already deployed 2026-07-06 with the watchdog): a
long session on the iGPU with `PUNKTFUNK_PERF=1`, record `wait_us_p50/p99`,
`encode_us`, client-measured latency, and whether the watchdog ever fires. Then per phase:
1. Open/probe smoke per codec (AVC, HEVC, HEVC-10) on the iGPU. **DONE** — the gated live
tests in `amf.rs` (`amf_encode_live_smoke` AVC+HEVC+AV1-probe, `amf_hdr_encode_live_smoke`,
`amf_native_probe_live`, `amf_intra_refresh_property_live`) pass on the lab Ryzen iGPU
(VCN3/RDNA2): both codecs across a native `reset()`, HEVC Main10 IDR carrying the
mastering(137)+CLL(144) SEIs byte-verified, intra-refresh property accepted, probe honestly
`h264/h265=true, av1=false`.
2. A/B the encode latency: expect `encode_us` p50 ~2 frame periods → ≤ 1 frame period
(see §3.3 for why `wait_us` is the wrong metric on the ffmpeg side). **MEASURED**
2026-07-06 by the gated `amf_latency_ab_bench` (`PUNKTFUNK_AMF_BENCH=1`, 1080p60 HEVC,
180 paced frames, same D3D11 NV12 input to both encoders, lab iGPU, debug build):
native `encode_us` p50 **5.18 ms (0.31 frame periods)** / p99 5.81 ms vs libavcodec-AMF
p50 **16.9 ms (1.01 frame periods)** / p99 17.5 ms — **3.3× lower, ~11.7 ms/frame saved**,
and the native path is decisively sub-frame (the ~2-frame hold that used to live in
`encode_us` is gone). Note the ffmpeg baseline came in at ~1 frame period, not the ~2 this
plan projected: the shipping ffmpeg config already sets AMF `latency=true` (a ~1-frame
hold), so the realized win is 3.3× / ~12 ms rather than the ~30 ms projected against an
un-tuned 2-frame baseline; direction and sub-frame collapse are exactly as §3.3 described.
Release builds should show a lower native number still (debug charges host-side
surface-create + copy-submit into the 5.18 ms). Zero-copy baseline for the input side
already measured 2026-07-06 on the lab iGPU (1080p120 HDR P010): `submit_us` p50 2.72.9 ms
(system readback) → **0.26 ms** (zero-copy D3D11), p99 6.6 ms → 0.5 ms.
3. Behavior: IDR on connect; mode switch mid-stream; HDR session (PQ VUI + 0xCE
convergence); client keyframe-request recovery; encoder `reset()` under an injected
failure; ≥30 min soak for the freeze class (watchdog log line
`encode stall detected` must NOT appear).
4. Driver matrix beyond the lab box is field data: VCN1 (Raven) through VCN4/5 differ in
preset support — the optional-property tolerance in §3.4 is what absorbs this.
## 6. Risks / open questions
- **Vtable mirroring correctness** is the concentrated risk: pin one AMF header version in
a comment, mirror minimally, and unit-test `AMFQueryVersion`/`AMFInit` + a headless
`CreateComponent` probe (skips cleanly on non-AMD boxes, like the NVENC live-gated
tests).
- Per-frame `CreateSurfaceFromDX11Native` allocation churn — if it shows up in profiles,
AMF supports pre-created surface pools; start simple.
- AV1 is RDNA3+; probe, never assume (same rule as everywhere in this codebase).
- Hybrid boxes: context must init on the *selected* adapter's device (the capture
device) — inherited for free by taking the capturer's device, but test with the
web-console GPU preference pointed at each GPU.
- The AMF runtime ships with the AMD driver, not with us — a missing/ancient `amfrt64.dll`
must produce a clean "install/update the AMD driver" error at open, then fall back
(Phase 1) or fail the session with that message (Phase 3).
## 7. Phasing
| Phase | Scope | Exit criterion | Status |
| --- | --- | --- | --- |
| 1 | FFI layer + AVC/HEVC (SDR + 10-bit HDR), bounded poll, native `reset()`, dispatch with ffmpeg fallback + `PUNKTFUNK_AMF_FFMPEG` hatch | §5.25.3 pass on the lab iGPU | **DONE** 2026-07-06 (§5.2 measured; §5.3 on-glass in progress) |
| 2 | Intra-refresh (`caps().intra_refresh`), in-band HDR SEI (`supports_hdr_metadata`), AV1, native codec probe | field-validated on ≥2 VCN generations | **CODE DONE** 2026-07-06 (lab VCN3 only; AV1/RDNA3 + 2nd VCN gen still owed) |
| 3 | Delete the ffmpeg-AMF dispatch arm + hatch; FFmpeg remains QSV-only | one release of field silence on the fallback label | **DONE** 2026-07-06 — see the gate note below |
**Phase 3 gate note (honesty):** the stated exit criterion (one release of field silence on the
fallback label) was **NOT met** — Phase 3 was cut the same day the native path was written, on
explicit direction, alongside a live macOS-client on-glass test. What Phase 3 removed: the
`WindowsBackend::Amf` libavcodec fallback arm, the `PUNKTFUNK_AMF_FFMPEG` hatch, and the
AMF→ffmpeg routes in `windows_codec_support` / `can_encode_444`. AMD dispatch, codec
advertisement, and the 4:4:4 answer are all native-only now; FFmpeg (`ffmpeg_win.rs`) is reached
only for QSV in production (its `WinVendor::Amf` machinery is retained solely as the
`amf_latency_ab_bench` comparator, not deleted — excising it would churn the Intel-unvalidated
QSV code for no gain). **Residual risk this pre-emption carries:** with the ffmpeg readback path
gone, an AMD box whose capturer can't produce video-processor NV12/P010 (falls back to
Bgra/Rgb10a2, or hands DDA CPU frames) now **fails the session** instead of degrading — the
design's answer is the native AMFVideoConverter front-end (§3.2), owed if that fallback is ever
seen in the field. Not observed on lab hardware (the VP yields NV12/P010). Reverting Phase 3 is a
small, localized diff if field data turns up trouble.
+202
View File
@@ -0,0 +1,202 @@
# punktfunk — security posture audit (2026-07-05)
> **Status:** AUDIT COMPLETE (2026-07-05). Whole-project **posture** audit — not a finding hunt
> like [`security-review.md`](security-review.md) (2026-06-21) and
> [`security-review-2026-06-28.md`](security-review-2026-06-28.md) (host follow-up), but an
> assessment of the overall security architecture, the state of the prior reviews' remediations,
> the delta landed since 2026-06-28 (display management Stages 08, WoL wake-until-up,
> `--data-port`, web display-config surface), and the process/supply-chain controls around the
> code. Method: single-reviewer read of the trust-model core (`quic.rs`, `crypto.rs`,
> `session.rs`, `native_pairing.rs`, `punktfunk1.rs`, `mgmt.rs`, `mgmt_token.rs`, the web BFF
> auth stack, `gamestream/{pairing,crypto,control}.rs`, `discovery.rs`, `wol.rs`), plus sweeps
> for TLS-bypass patterns, committed secrets, unsafe density, CI workflow hygiene, and the
> dependency/advisory pipeline.
## Executive summary
**The posture is strong — unusually so for a project of this size.** The core trust
architecture is sound and shows defense-in-depth discipline: a modern, mutually-authenticated
native plane (QUIC + rustls, fingerprint pinning with real `CertificateVerify` verification,
SPAKE2 pairing with cert-fingerprint identity binding); a management plane with a clean
authn/authz split (loopback-confined bearer for admin, deny-by-default read-only allowlist for
paired mTLS certs); a web console that never leaks the admin token to the browser and fails
closed when misconfigured; legacy GameStream compatibility correctly quarantined behind an
explicit opt-in; and a supply chain guarded by scheduled `cargo audit`, a tightly-justified
ignore list, a license allowlist, and an exact-pinned toolchain. The two prior reviews' 18
findings are remediated or accepted-with-rationale, and the fixes are visible (and in several
cases regression-tested) in the code today.
**No new high-severity issue was found.** The residual risk concentrates in the **web console's
password gate** (no brute-force throttling; cookie-sealing key derived from the login password)
and in **second-tier supply-chain gaps** (no advisory scanning for the Bun/Nitro stack;
tag-pinned third-party CI actions holding deploy secrets). All are bounded by the documented
threat model (trusted LAN / VPN, no WAN exposure) but are cheap to close and worth closing.
## Remediation status (2026-07-05)
All findings from this audit were fixed the same day, plus one Windows LPE gap (F-8) surfaced by a
reviewer during remediation. Verification notes are per-item.
| # | Sev | Status |
|---|-----|--------|
| F-1 | Medium | **FIXED** — per-IP exponential-backoff login throttle (`web/server/util/loginThrottle.ts` + `login.post.ts`); 5 free attempts then 1s→5min backoff, global floor, size-capped map. Behaviorally tested (backoff, unlock, IP-independence, success-clear). |
| F-2 | Medium | **FIXED** — cookie-seal key now derived from the CSPRNG mgmt token (`sessionConfig` in `auth.ts`), not the login password; password-derivation kept only as a no-token dev fallback. A captured cookie is no longer an offline password oracle. |
| F-3 | Low | **FIXED** — accept-any-cert scoped to the loopback proxy hop via Bun per-request `tls` (`routes/api/[...].ts`, gated on `isLoopbackUrl`); process-wide `NODE_TLS_REJECT_UNAUTHORIZED=0` removed from all 4 deploy files + 3 docs. A non-loopback mgmt URL now verifies normally. |
| F-4 | Low | **FIXED**`bun audit` job added to `audit.yml` (weekly + on `web/bun.lock` change), same fail-on-vulnerability stance as `cargo audit`. |
| F-5 | Low | **FIXED** — the three secret-touching third-party actions SHA-pinned with version comments: `appleboy/scp-action`, `appleboy/ssh-action` (deploy SSH key + registry token, `docker.yml`), `android-actions/setup-android` (signing keystore + Play SA, `android.yml`/`android-screenshots.yml`). GitHub-owned `actions/*` left on major tags (org controls the tag). |
| F-6 | Info | No change (accepted) — tray loopback accept-any-cert; spoofed-status-only, local-process out of scope. |
| F-7 | Info | No change (documented) — session-layer input replay; native plane rides replay-protected QUIC, GameStream covered by the trusted-LAN caveat. |
| **F-8** | **Medium** | **FIXED** — see below. `driver install` executed/trusted files from `--dir` with no check the dir was admin-only → local EoP if the payload is staged somewhere user-writable. Added a DACL/owner check (`ensure_admin_only_source` in `windows/install.rs`) that fails closed. Verified: Win32 FFI typechecks against `windows 0.62.2` for `x86_64-pc-windows-msvc`. |
### F-8 (Medium) — driver install trusts a non-admin-writable source directory
**Reported by the user during remediation; confirmed.** `punktfunk-host driver install --dir <stage>`
runs **elevated** (the Inno `[Run]` section, or a manual admin invocation) and, from `--dir`,
**executes** `nefconc.exe`, trusts a `.cer` into the machine `Root`/`TrustedPublisher` stores, and
`pnputil /add-driver`s the package — with **no check that `--dir` is writable only by
administrators**. If the staging directory is writable by a non-admin, a local unprivileged user can
plant a malicious `nefconc.exe` (arbitrary code as the elevated installer → SYSTEM) or swap the
`.cer` (poisoning the machine trust store) before the elevated step consumes it — a local elevation
of privilege.
**Not exploitable in the default install** (correctly noted by the reporter): the shipped installer
stages into `{tmp}` under a `DefaultDirName={autopf}` (Program Files), `PrivilegesRequired=admin`
Inno setup, which restricts the staging dir to Administrators/SYSTEM. The gap bites only when the
payload is staged somewhere user-writable — a custom install directory, or a hand-run
`driver install --dir <user-writable>`. *Fix:* `ensure_admin_only_source` reads the directory's
owner + DACL (`GetNamedSecurityInfoW`) and refuses to proceed unless every principal with a
create/write/delete/DACL-rewrite right is SYSTEM/Administrators/TrustedInstaller (or CREATOR-OWNER
under a privileged owner). Fails **closed** — an unreadable ACL is treated as unsafe — and, like
every other step in the best-effort installer, a refusal only degrades the host to a physical
display; it never aborts the install.
## Trust architecture assessment
### Native plane (punktfunk/1) — sound
- **Transport:** QUIC via quinn 0.11.11 / rustls 0.23.41 (ring). Data-plane sessions are
created with `encrypt: true` (`punktfunk1.rs:959`) and per-session random key+salt.
- **Identity & pinning:** hosts serve a persisted self-signed cert; clients pin its SHA-256
(TOFU only as the documented bootstrap special case, `endpoint::client_insecure` =
`client_pinned(None)`). Crucially, the pin verifier still verifies TLS 1.2/1.3 handshake
signatures for real (`quic.rs:2064-2096`) — possession of the pinned cert's key is proven,
the classic pin-without-CertificateVerify hole is explicitly avoided. The host-side
`AcceptAnyClientCert` likewise verifies the handshake signature, so a client fingerprint is
proof of key possession, not just a presented blob.
- **Pairing (SPAKE2):** the ceremony binds *both* certificate fingerprints as SPAKE2
identities, so a MITM presenting different certs to each leg cannot reach a shared key.
Key-confirmation MACs are compared constant-time. The PIN is CSPRNG-minted, **single-use and
consumed before the client's proof is read** (`punktfunk1.rs`, `pair_ceremony`) — an
attacker gets exactly one online guess per operator arming, rate-limited further by the 2s
`PAIRING_COOLDOWN`. Arming windows can be **fingerprint-bound** (prior review #9), and the
delegated-approval pending queue is flood-resistant (per-source-IP cap, parked-knock
eviction protection, TTL — prior review #13, all regression-tested in `native_pairing.rs`).
- **Session AEAD:** AES-128-GCM with a documented nonce-uniqueness contract (per-direction
salt bit, per-session key+salt, zero-key rejected by config validation), sequence bound as
AAD. Wire decoders bound every attacker-controllable header field (`packet.rs:323`,
`ReassemblerLimits`); pairing/control messages are length-checked with exact-trailing-bytes
rejection.
- **Untrusted-input hygiene:** client-supplied device names are sanitized against control
chars, ANSI escapes, and Unicode bidi overrides before storage/log/UI
(`native_pairing.rs::sanitize_device_name`) — approval-UI spoofing is handled.
- **Trust store:** atomic temp+rename writes, owner-only permissions (0600/DACL via
`write_secret_file`), in-memory rollback on persist failure.
### Management plane — sound
- HTTPS always; bearer token always required (env > owner-only persisted file > generated;
`mgmt_token.rs`). Token comparison hashes both sides before comparing (`mgmt.rs::token_eq`),
neutralizing timing.
- The **bearer (full-admin) path is honored only from loopback peers**, even though the
listener binds all interfaces by default; LAN callers must present a **paired** mTLS cert
and are confined to a deny-by-default, GET-only allowlist (`cert_may_access`) — a paired
streaming client cannot administer the host (unpair others, read/arm PINs, stop sessions,
or mutate the library/display config). The new display-management routes stayed off the
cert allowlist, with a regression test (`display_settings_surface stays read-only`).
- `/api/v1/local/summary` (tray status) is unauthenticated but loopback-only and
deliberately non-sensitive.
### Web console (Nitro/Bun BFF) — good design, two hardening gaps (F-1, F-2)
- Single shared password gate; **fails closed** (503) when `PUNKTFUNK_UI_PASSWORD` is unset;
constant-time compare; sealed (AES-GCM) `httpOnly` `SameSite=lax` session cookie with a
7-day TTL; open-redirect guard on the post-login path; `/api/**` always gated; the mgmt
bearer token is injected **server-side only** and browser cookies are stripped from the
upstream request. Public-path allowlisting is by path, not extension (with the reasoning
documented — `openapi.json` exposure via a `*.json` allowlist was anticipated and avoided).
- Gaps: no login throttling (F-1), cookie-sealing key derived from the password when
`PUNKTFUNK_UI_SECRET` is unset (F-2), process-wide `NODE_TLS_REJECT_UNAUTHORIZED=0` for the
loopback proxy hop (F-3).
### GameStream/Moonlight compatibility — correctly quarantined
Off by default; enabled only by `--gamestream`/`--moonlight`, and documented (SECURITY.md,
`--help`) as legacy-crypto, trusted-LAN-only. Within that constraint the implementation is
defensively built: PIN delivery only via the authenticated management API (never nvhttp —
prior #1), parked-handshake caps (#12), one RSA signature per ceremony (Marvin-amplifier
hardening, S7), phase-repeat rejection, RTSP gated on a paired `/launch` bound to the
launching peer's IP (#4). The AES-128-ECB pairing crypto and mostly-plaintext media streams
are wire-compatibility constraints of the Moonlight protocol, not implementation choices, and
the input-replay gap at the raw-UDP session layer is documented in code
(`session.rs:31-35`) and covered by the trusted-LAN assumption.
### Discovery & Wake-on-LAN — appropriately advisory
mDNS TXT records (`fp`, `mac`, `pair`, `mgmt`) are treated as unauthenticated hints
everywhere they're consumed: pinning still gates the actual connection, and a spoofed MAC
only makes a wake fail (magic packets are inert). `wol.rs` detect-and-warn never mutates NIC
state.
### Supply chain & process — mature
- **Rust:** `cargo audit` weekly + on every `Cargo.lock` change + on demand
(`.gitea/workflows/audit.yml`); the `.cargo/audit.toml` ignore list is tight, justified,
and *self-correcting* (the RUSTSEC-2023-0071 entry documents that its own earlier rationale
was wrong and re-justifies the accept on corrected grounds — exemplary). Key crypto deps
are current (rustls 0.23.41, quinn 0.11.11, ring 0.17.14). License allowlist enforced via
`cargo-about` (no copyleft in the linked set). Exact toolchain pin (1.96.0).
- **Secrets hygiene:** no committed key material or real `.env` files (templates only);
secret files routed through `write_secret_file` (0600 + Windows SYSTEM/Admins DACL).
- **Process:** SECURITY.md with private reporting, coordinated disclosure, and safe harbor;
two prior deep reviews with per-finding remediation tracking; findings referenced by number
in code comments and regression tests.
- Gaps: no JS-side advisory scanning (F-4); third-party actions tag-pinned, not SHA-pinned
(F-5).
## Findings (this audit)
No High findings. Severities assume the documented LAN/VPN threat model.
| # | Sev | Component | Finding |
|---|-----|-----------|---------|
| F-1 | **Medium** | web console | **No throttling on `POST /_auth/login`** (`web/server/routes/_auth/login.post.ts`). The constant-time compare stops timing leaks but not volume: a LAN peer can brute-force `PUNKTFUNK_UI_PASSWORD` at network speed against a console that is, by design, LAN-exposed. Every other password/PIN gate in the project is rate-limited (pairing cooldown, single-use PINs) — this is the one unthrottled secret. *Fix:* small in-memory failure counter with exponential backoff (per source IP + global), mirroring `PAIRING_COOLDOWN`'s philosophy. |
| F-2 | **Medium** | web console | **Cookie-sealing key is derived from the login password** when `PUNKTFUNK_UI_SECRET` is unset (`web/server/util/auth.ts::sessionConfig``sha256("punktfunk-session-v1:" + password)`). A captured session cookie (e.g. sniffed over the plain-HTTP dev/LAN mode, where `Secure` is off by default) becomes an **offline dictionary oracle for the password**: an attacker tries candidate passwords by deriving the key and attempting to unseal — no server round-trips, so F-1's fix doesn't help. *Fix:* generate and persist a random 32-byte secret on first start when `PUNKTFUNK_UI_SECRET` is unset (same pattern as `mgmt_token.rs`), instead of deriving from the password. |
| F-3 | Low | web console | **`NODE_TLS_REJECT_UNAUTHORIZED=0` is process-wide**, not scoped to the loopback mgmt hop it's documented for (`web/.env.example:25`, README). Today the BFF makes no other outbound TLS call, so the practical scope holds — but it's a global switch that silently unverifies any *future* fetch (an update check, an art CDN, a webhook). *Fix:* drop the env var and pass the host's own `cert.pem` as a per-fetch CA (Bun `fetch` `tls.ca` / undici `Agent`), which also removes a scary line from the deployment docs. |
| F-4 | Low | supply chain | **No advisory scanning for the web stack.** `audit.yml` covers the Rust tree only; the Bun/Nitro BFF — the component that holds the login gate, session sealing, and the mgmt token — has a `bun.lock` but no scheduled vulnerability scan. *Fix:* add a job running `bun audit` (or `osv-scanner --lockfile web/bun.lock`) on the same weekly + lockfile-change triggers. |
| F-5 | Low | CI | **Third-party actions are tag-pinned, not SHA-pinned.** `appleboy/ssh-action@v1.2.5` / `appleboy/scp-action@v0.1.7` receive deploy SSH credentials, and `actions/checkout@v4` / `cache@v4` run on every job on self-hosted runners; a moved tag = code execution on the runners plus secret exfiltration. Given how rigorous the Rust-side supply chain is, this is the soft spot. *Fix:* pin at least the secret-touching actions to full commit SHAs. |
| F-6 | Info | tray / local | The tray's status fetch (`punktfunk-tray/src/status.rs`) uses the accept-any-cert verifier against `127.0.0.1:47990`. If the host isn't running, any local unprivileged process can squat the port and feed the tray fake status. Impact is spoofed tray UI only (the tray sends no secrets and the summary is non-sensitive) — consistent with the documented "local processes are out of scope" stance. No action needed; recorded for completeness. |
| F-7 | Info | core session | Input-event replay at the raw-UDP session layer remains unfiltered — **already documented** in `session.rs:31-35` with the correct future home (a sliding window keyed on the authenticated sequence, in the GameStream host). The native plane is unaffected (client input rides QUIC datagrams, which are inherently replay-protected). Keep the doc note honest until the GameStream path grows the window; it stays covered by the opt-in/trusted-LAN caveat. |
## Prior-review remediations — spot-verified present
- #1 (PIN only via bearer mgmt API): confirmed — `PinGate` doc + no nvhttp PIN route.
- #2 (mgmt token via `write_secret_file`): confirmed — `mgmt_token.rs:62`.
- #9 (fingerprint-bound arming windows): confirmed — `arm_for`/`pin_for_attempt` + tests.
- #12 (parked-waiter cap): confirmed — `MAX_PARKED_WAITERS` + atomic slot reservation.
- #13 (per-IP pending cap, parked-knock protection): confirmed — constants + tests.
- S7 (rsa Marvin accept): rationale in `.cargo/audit.toml` matches the corrected version;
one-signature-per-ceremony hardening referenced in the ignore justification.
## Priorities
1. **F-1 + F-2 together** (web console password gate): a login backoff plus a
generated-not-derived cookie secret close the only realistic LAN-attacker path to admin
found in this audit. Both are small, isolated changes.
2. **F-4 + F-5** (supply chain parity): one CI job and a handful of SHA pins bring the JS
stack and the workflows up to the standard the Rust tree already meets.
3. **F-3** (scoped CA instead of the global TLS switch): low urgency, high
docs/appearance value.
Nothing found here changes the documented threat model or contradicts SECURITY.md's stated
limits. The next full finding-hunt review should focus on the Windows vdisplay/driver
admission surface (largest post-2026-06-28 delta) and the web console once F-1/F-2 land.
+8
View File
@@ -20,6 +20,14 @@ virtual output primary** — `apply_session_env` defaults `PUNKTFUNK_KWIN_VIRTUA
`PUNKTFUNK_MUTTER_VIRTUAL_PRIMARY` on for the auto desktop path. Both shipped in `3363576`; details in
git history.)
> **Update 2026-07-05:** items **#1** (SIGKILL teardown / keep-warm) and **#7** (restore-guard /
> keep-warm interaction) are picked up by `design/gamemode-and-dedicated-sessions.md` **Part A3**
> — the managed session's restore becomes registry-owned and policy-driven (`keep_alive` replaces
> the hardcoded 5 s debounce; "keep warm" = `keep_alive: forever`), with the SIGKILL-teardown
> experiment folded into that stage's validation. That doc also covers the display-management
> registry's game-mode conflicts (ownership classes, stale-node reuse) and dedicated per-launch
> gamescope game sessions.
## Still parked
### 1. F44 gamescope teardown corrupts the GPU context
+197
View File
@@ -0,0 +1,197 @@
# Zero-copy capture hardening — issue handoff
> **Status: FIXED + validated (2026-07-06).** The fix is implemented and on-glass validated — see
> [`zerocopy-worker-isolation.md`](zerocopy-worker-isolation.md): the GPU import (tiled EGL/GL→CUDA
> *and* LINEAR Vulkan→CUDA) now runs in a per-capture **worker subprocess** (CUDA-IPC frame
> hand-off), so this driver SIGSEGV kills the worker and the host degrades to its capture-loss
> rebuild; plus in-process teardown-order fixes and a poison/latch path replacing the corrupt
> tiled→CPU fallback. Validated on the RTX 5070 Ti/GNOME box: worker path streams at p50 1.30 ms,
> and a `kill -9` of the worker mid-stream is survived + recovered (fresh worker in ~185 ms,
> streaming resumes). The description below is kept as the issue record.
>
> *(Original handoff intro:)* This document describes a reproduced
> host **SIGSEGV** in the Linux zero-copy capture path. It deliberately does **not** prescribe a fix —
> the next agent plans the implementation. Everything below is observed fact + root-cause analysis;
> the "Considerations / open questions" section frames the solution space without committing to one.
>
> **This is a pre-existing capture-layer issue, NOT a regression from the gamemode/dedicated-sessions
> work** (`design/gamemode-and-dedicated-sessions.md`). The crashing code
> (`crates/punktfunk-host/src/linux/zerocopy/{cuda,egl}.rs`, `capture/linux/pipewire.rs`) is untouched
> by that branch; it merely surfaced during on-glass validation of it.
## 1. What happened
On **`.181`** (Bazzite F44, RTX 4090, NVIDIA open driver 610.43.02 — see
`[[gamemode-onglass-181-2026-07-06]]`), streaming a game at the client's mode worked smoothly with
**zero-copy enabled** (`PUNKTFUNK_ZEROCOPY=1`). When the user then **switched the box from Steam Game
Mode to the KDE/Plasma desktop mid-stream**, the host process **crashed (SIGSEGV, core dumped)** and
the client saw "session ended". systemd auto-restarted the host ~3 s later.
The host's own logic on the switch was **correct** up to the crash — the session watcher detected
`Gaming → DesktopKde`, bumped the session epoch, rebuilt the backend to KWin, brought up the virtual
output at 5120×1440@240, set it sole desktop, and began PipeWire capture. The crash came **after**
that, the first time the capture thread mapped a KWin frame into CUDA.
## 2. The crash (backtrace)
`coredumpctl` for the host process, crashing thread:
```
Signal: 11 (SEGV)
#0 libnvidia-eglcore.so.610.43.02 ← SIGSEGV inside the NVIDIA driver
#1 libnvidia-eglcore.so.610.43.02
#2 libnvidia-eglcore.so.610.43.02
#3 libnvidia-eglcore.so.610.43.02
#4 libEGL_nvidia.so.0
#5 libcuda.so.1
#6 libcuda.so.1
#7 cuGraphicsMapResources (libcuda.so.1)
#8 punktfunk_host::zerocopy::cuda::RegisteredTexture::copy_mapped_plane
#9 punktfunk_host::zerocopy::egl::EglImporter::import_inner
#10 punktfunk_host::capture::linux::pipewire::pipewire_thread::{{closure}} (the PipeWire on_process callback)
#11 pipewire::stream::…::on_process
#12 libpipewire-0.3.so.0 (impl_node_process_input → process_node → node_on_fd_events → pw_main_loop_run)
```
The fault is **inside `libnvidia-eglcore`, reached through `cuGraphicsMapResources`** — i.e. CUDA was
asked to map a GL/EGL-imported resource, and the driver dereferenced GPU state that was no longer
valid.
## 3. The code path
This is the **tiled-dmabuf zero-copy path** used for compositors that hand out **tiled** buffers
(KWin, and the NVIDIA tiled path generally): PipeWire dmabuf → **EGL/GL import** (`EglImporter`,
`crates/punktfunk-host/src/linux/zerocopy/egl.rs`) → **register as a CUDA graphics resource** and
**map per frame** (`RegisteredTexture::copy_mapped_plane` / `copy_mapped_to`,
`crates/punktfunk-host/src/linux/zerocopy/cuda.rs:810-848` and the sibling `copy_mapped_plane` below
it). The crash is at the `cuGraphicsMapResources(1, &mut self.resource, …)` call
(`cuda.rs:824-828`), driven per frame from the PipeWire `on_process` callback
(`capture/linux/pipewire.rs`, the closure at backtrace frame #10).
The relevant `// SAFETY:` proof (`cuda.rs:814-823`) reasons that `self.resource` is a valid
`CUgraphicsResource` from a successful `register_gl`, the GL+CUDA contexts are current, and the
map/unmap pair is balanced with the copy synced before unmap. **That reasoning is sound for a
well-behaved compositor that keeps the imported dmabuf alive across the map** — it does not cover the
case where the **producer invalidates the underlying buffer/texture out from under an in-flight
map**.
**Not affected on this box:** the **gamescope** path (LINEAR dmabuf → **Vulkan bridge** → CUDA,
`linux/zerocopy/vulkan.rs`) did **not** crash — game-mode streaming was smooth. And the **non-zero-copy
SHM/CPU path** (`PUNKTFUNK_ZEROCOPY=0`, the NVENC default) has no EGL/CUDA import at all, so there is
nothing for the driver to fault on. So the fault is specific to the **EGL/GL→CUDA tiled import**, not
zero-copy in general.
## 4. Root cause
`cuGraphicsMapResources` faulted **inside the driver** on GPU state that had become invalid — almost
certainly because the **KWin compositor freed/recycled (or crashed with) the dmabuf** that our
`EglImporter` had imported and registered, while our capture thread was mapping it for the next frame.
Strong corroborating evidence: in the **same 8-second window**, the box logged core dumps for
`plasmashell` (×2), `Xwayland`, `gamescope`, and `mangoapp`. So the F44 Game↔Desktop transition on
this box is itself highly unstable, and the KWin buffer our zero-copy path held a handle to almost
certainly went away mid-map.
**Why this is a real host-robustness gap, not just "the box is flaky":** a **SIGSEGV inside a
closed-source driver (`libnvidia-eglcore`) is not catchable from Rust** — it is not a `Result`, not a
Rust panic, not something a `catch_unwind` can contain. So *any* time the producer's buffer can vanish
between "we hold a CUDA graphics resource for it" and "we map it" (compositor crash, buffer-pool
recycle, output/mode teardown, hot-unplug), the driver can take the whole host process down. A capture
pipeline that must survive a compositor going away (which the host already tries to do — it has a
capture-loss → rebuild path) cannot rely on `cuGraphicsMapResources` returning an error on a stale
resource; the driver may just crash instead.
## 5. Trigger conditions (what invalidates the imported buffer)
The observed trigger was a **compositor crash during a Game→Desktop switch**, but the same class of
fault can be reached by anything that frees/recycles the imported dmabuf or its GL texture while a map
is in flight or a `RegisteredTexture` still references it:
- compositor crash / restart (observed);
- normal PipeWire buffer-pool recycle / renegotiation (format change, buffer count change) where a
registered texture outlives the buffer it wrapped;
- virtual-output teardown / mode change (e.g. the mid-stream `Reconfigure`, the session-switch
rebuild) racing an in-flight map;
- output removal / disconnect.
The next agent should establish **which of these are actually reachable** in the current code (the
per-frame registration/lifetime in `EglImporter`/`RegisteredTexture` vs. the PipeWire buffer
lifecycle) versus only the compositor-crash case.
## 6. Scope
- **Affected:** the EGL/GL→CUDA tiled-import path (`zerocopy::egl` + `zerocopy::cuda`), driven from
`capture/linux/pipewire.rs`. On NVIDIA this is used for tiled dmabufs (KWin desktop capture is the
concrete case here; the NVIDIA tiled path in general).
- **Likely also worth auditing (same class):** the **Vulkan bridge** path (`zerocopy::vulkan.rs`, LINEAR
dmabuf → Vulkan → CUDA) — it did not crash here, but it imports external dmabufs into the GPU too and
may have the same "producer freed the buffer mid-use" exposure; confirm whether it validates/owns the
buffer lifetime differently.
- **Not affected:** the SHM/CPU capture path (no GPU import).
## 7. What is NOT the cause (to save the next agent time)
- **Not the gamemode/dedicated-sessions branch.** That branch's switch logic worked correctly
(epoch bump, watcher rebuild to KWin, virtual output up); the crash is downstream in pre-existing
capture code it doesn't touch.
- **Not a `.desktop`/KWin authorization problem.** The KWin virtual output was created and set sole
desktop successfully — auth was fine.
- **Not the gamescope "out of buffers" issue** from the same validation session (that was a separate
gamescope-3.16.19 PipeWire-node limitation on SHM). This is a hard driver SEGV on the GPU-import path.
## 8. Observed mitigation (already available, not the fix)
Setting **`PUNKTFUNK_ZEROCOPY=0`** (SHM/CPU path — the NVENC default anyway; the box had it forced
`=1`) removes the EGL/CUDA import, so this crash cannot occur and a compositor going away degrades to a
graceful capture-loss rebuild. Cost: a CPU copy per frame (higher latency than the zero-copy stream the
user measured). This is an operational workaround, **not** a code fix, and it forfeits zero-copy on the
desktop path.
## 9. Considerations / open questions for the implementation plan (do not treat as a prescription)
These frame the solution space; the next agent decides.
- **A driver SIGSEGV is uncatchable in-process.** Any design that "handles" this by wrapping the FFI
call in error handling will not work — the process is already dead. So the fix has to be about
**never handing the driver a resource that can be stale**, or **isolating the GPU-import work** so a
driver crash doesn't take the streaming host down. Both directions are open:
- *Prevent-the-stale-resource* directions to evaluate: strict per-frame import/register/map/unmap
lifetime tied to the exact PipeWire buffer being processed (so no `RegisteredTexture` outlives its
buffer); detecting compositor/output teardown and stopping capture **before** the next map;
reconciling the EGL texture / CUDA resource lifetime with PipeWire's buffer-recycle events.
Establish whether the current code can ever map a resource whose buffer PipeWire has already
recycled/removed.
- *Isolate-the-crash* directions to evaluate: whether the GPU import belongs in a **separate process**
(like the Windows two-process/DDA isolation model) so a driver SEGV is contained and the session
can rebuild — heavier, but the only thing that truly survives an unpreventable driver fault.
- **Per-backend / per-buffer-type routing.** The Vulkan-bridge path did not crash; the SHM path is
safe. A plan might route tiled dmabufs (KWin) away from the fragile EGL/CUDA path, or only enable the
EGL/CUDA path where the producer's buffer lifetime is guaranteed. Decide whether the fix is
"harden the EGL/CUDA path" vs. "don't use it for producers that can pull buffers."
- **Interaction with the existing capture-loss rebuild.** The host already rebuilds on capture loss;
the goal is to reach that path on producer teardown **instead of** the driver crash. Understand why
the crash beats the capture-loss detection today (the map happens in the PipeWire `on_process`
callback before any loss is observed).
- **Reproducibility.** This was observed on a box whose KDE was *itself* crashing (F44 plasmashell/
Xwayland). To isolate "our zero-copy path is fragile" from "the compositor crashed," reproduce on a
**stable** KDE/NVIDIA box — force a buffer invalidation (output teardown / renegotiation / a scripted
compositor restart) mid-capture and confirm the same `cuGraphicsMapResources` fault without the
surrounding compositor chaos. That also tells you whether the non-crash triggers in §5 are real.
- **Keep the SAFETY-proof discipline.** `zerocopy/{cuda,egl,vulkan}.rs` are part of the unsafe-audited
set (`#![deny(clippy::undocumented_unsafe_blocks)]`, every `unsafe` carries a `// SAFETY:`). Any fix
updates those proofs to reflect the new lifetime/validity guarantees.
## 10. Reproduction environment / artifacts
- Box: `bazzite@192.168.1.181` (sudo `bazzite`), Bazzite F44 (`bazzite-deck-nvidia:testing`), RTX 4090,
NVIDIA open 610.43.02. See `[[gamemode-onglass-181-2026-07-06]]` for deploy/access details.
- Trigger: stream a game-mode session with `PUNKTFUNK_ZEROCOPY=1`, then switch the box Game Mode →
KDE desktop mid-stream (the session watcher rebuilds to KWin → tiled EGL/CUDA capture → crash).
- The coredump was present under `coredumpctl` on the box at the time of writing (may age out); the
backtrace in §2 is captured above.
## 11. Related
- `design/gamemode-and-dedicated-sessions.md` (the branch this surfaced under — not the cause).
- `design/session-aware-host-followups.md` (Game↔Desktop switch behavior; F44 GPU instability #1).
- `design/gpu-contention-investigation.md` / `design/host-latency-plan.md` (zero-copy path context).
- `crates/punktfunk-host/src/linux/zerocopy/{egl,cuda,vulkan}.rs`, `capture/linux/pipewire.rs`.
- CLAUDE.md: "GPU **zero-copy** on all paths (tiled dmabuf → EGL/GL → CUDA; LINEAR dmabuf → **Vulkan
bridge** → CUDA)" and the `PUNKTFUNK_ZEROCOPY` semantics (ON for VAAPI/AMD/Intel with a CPU downgrade;
OFF/opt-in for NVENC).
+163
View File
@@ -0,0 +1,163 @@
# Zero-copy capture hardening — GPU-import worker isolation
> **Status: IMPLEMENTED + on-glass validated (2026-07-06).** This is the implementation
> plan + decision record for the crash described in
> [`zerocopy-hardening-handoff.md`](zerocopy-hardening-handoff.md) (host SIGSEGV inside
> `libnvidia-eglcore` via `cuGraphicsMapResources` when the compositor invalidated an imported
> dmabuf mid-map, observed on the Bazzite F44 Game→Desktop switch). Validated on the RTX 5070 Ti /
> GNOME box (.21): the isolated worker carries frames at **p50 1.30 ms** end-to-end (NV12, 1800
> frames 0-mismatched), and a `kill -9` of the worker mid-stream is survived by the host and
> recovered — poison → `capture lost — rebuilding pipeline in place` → a fresh worker in **~185 ms**
> → streaming resumes (2385 frames, 0 mismatched, one 33 ms blip at the rebuild seam). See §6.
## 1. The decision: isolate, don't (only) prevent
The handoff's §9 framed two directions — *prevent the stale resource* vs *isolate the crash*.
The audit (§3 below) shows our per-frame lifetime discipline is already correct: the `EGLImage`
is created and destroyed strictly inside the PipeWire `on_process` callback while the buffer is
held (not requeued), and the CUDA-registered textures are **our own GL render targets**, never
wrappers around producer buffers. The invalidation that crashed the host is **external**
a compositor crash (or GPU channel wreckage from the surrounding plasmashell/Xwayland core dumps)
yanked the dmabuf's GPU-side state while the driver executed our in-flight GL sampling + CUDA map.
No in-process ordering fix can close that race, and a driver SIGSEGV is not catchable.
So the fix is **process isolation**: the entire `EglImporter` (tiled dmabuf → EGL/GL → CUDA *and*
LINEAR dmabuf → Vulkan bridge → CUDA) moves into a small per-capture **worker subprocess**. If the
driver faults, the *worker* dies; the host observes a dead socket, fails the frame/capture cleanly,
and the existing capture-loss rebuild path (`gamestream/stream.rs`, `punktfunk1.rs`) takes over —
which is exactly what already happens today on the safe SHM path when a compositor goes away.
What is deliberately **not** isolated:
- **SHM/CPU capture** — no GPU import, nothing to contain.
- **VAAPI passthrough** (AMD/Intel) — capture only `dup`s the dmabuf fd; the GPU import happens in
the encoder (Mesa VA, which reports errors rather than faulting; no observed crashes). Out of
scope here.
- **NVENC itself** — libavcodec/NVENC surface errors as return codes; if the GPU is globally
wedged the encoder errors and the session rebuilds. Isolating encode would mean shipping a
session-wide media-pipeline process, far beyond this fix.
## 2. Architecture
```
host process worker process (punktfunk-host zerocopy-worker)
──────────── ───────────────────────────────────────────────
PipeWire on_process EGLDisplay + GL ctx + CUDA ctx + VkBridge
│ dmabuf fd (held, fence-waited) │
├── IMPORT{key,geometry} + fd ──────────────▶│ eglCreateImage → GL blit/NV12 convert
│ (SCM_RIGHTS, first sight per key) │ → cuGraphicsMapResources → copy → unmap
│ │ → pooled CUDA buffer (cuMemAllocPitch)
│◀────────── FRAME{id [, ipc desc]} ─────────┤ exported ONCE via cuIpcGetMemHandle
│ host opens the IPC handle once, │
│ wraps it as DeviceBuffer │
▼ │
encode thread (NVENC) reads the device ptr │ keeps the DeviceBuffer in-flight
│ DeviceBuffer drop │
└── RELEASE{id} ────────────────────────────▶│ returns the buffer to its pool
```
- **Transport**: a `socketpair(AF_UNIX, SOCK_SEQPACKET)` created before spawn; the child end is
`dup2`'d to fd 3 (`zerocopy-worker --fd 3`). SEQPACKET gives reliable, ordered, message-framed
delivery; dmabuf fds ride as `SCM_RIGHTS`. Messages are small serde_json bodies (~200 B/frame;
negligible at 240 fps).
- **Frame data never crosses the socket.** The worker's `BufferPool` allocations are exported once
each via `cuIpcGetMemHandle`; the host `cuIpcOpenMemHandle`s each exactly once (cached by buffer
id) and reuses the mapping as the pool recycles. Per frame the reply is just `{id}` — the copy
was already synced (`copy_blocking`) worker-side before the reply, so the host reads complete
pixels. The result is the same zero-CPU-touch path as before, plus one socket RTT (~tens of µs).
- **fd caching**: the host keys each PipeWire buffer by its dmabuf `st_ino` (unique per dma-buf
object) and sends the fd only on first sight; the worker keeps the received dup (tiled: for the
per-frame `eglCreateImage`; LINEAR: for the Vulkan `src_cache`). A format renegotiation
(`param_changed`) sends `CLEAR_CACHE`, dropping both sides' caches — this also fixes the
pre-existing LINEAR-path bug where `VkBridge::src_cache` was keyed by raw fd number and never
invalidated across pool recycles (§3, trigger b). Cache desync is self-healing: a worker that no
longer holds a key's fd (LRU eviction) answers `NeedFd` and the host retries once with the fd.
- **Lifetimes**: the worker holds each exported frame as a real `DeviceBuffer` in an in-flight map
until `RELEASE{id}` arrives, so the existing pool `Arc` machinery keeps device memory alive
across pool replacement while the host still reads it. Host-side, every remote `DeviceBuffer`
holds an `Arc` of the client's shared state (socket + IPC-mapping cache), so mappings are closed
only after the last in-flight frame drops.
- **Worker lifetime**: one worker per capture (per `pipewire_thread`), spawned from
`/proc/self/exe`. It exits on socket EOF; the host reaps children via a global sweep list (no
zombies). Host death ⇒ EOF ⇒ worker exit.
### Failure semantics (the point of the exercise)
| event | behavior |
|---|---|
| worker init fails (no GPU, EGL error) | handshake reports `init_err` → capture falls back to the CPU/SHM offer, same as `EglImporter::new()` failure today |
| driver SIGSEGV in the worker (the observed crash) | socket EOF → import fails with a *dead-worker* error → the capturer is **poisoned**`next_frame`/`try_latest` return an error → the session's capture-loss rebuild runs (new capturer, new worker). **The host process survives.** |
| tiled import fails but worker alive (e.g. `EGL_BAD_MATCH` on one frame) | frame dropped; after 3 consecutive failures the capturer poisons → rebuild. It must **never** fall through to the CPU mmap path — mmap of a *tiled* dmabuf de-pads scrambled bytes (a pre-existing fallback bug; the CPU fallback was only ever correct for LINEAR). |
| LINEAR import fails | unchanged: fall back to the CPU mmap path in-stream (a LINEAR dmabuf is mappable), degraded not dead |
| repeated worker deaths | a process-wide latch (`note_gpu_import_death`, 3 consecutive deaths without a successful import between them) disables the GPU importer for the rest of the process — rebuilds renegotiate the SHM offer. Stops a wedged GPU stack from crash-looping the worker while still streaming (CPU path). A successful import resets the streak. |
### Escape hatch
`PUNKTFUNK_ZEROCOPY_INPROC=1` keeps the importer in-process (the pre-isolation behavior) for
debugging and A/B latency comparison. Default is the worker.
## 3. Audit answers for handoff §5 (which triggers are actually reachable)
- **Compositor crash / restart** — reachable (observed). Contained by the worker.
- **PipeWire buffer-pool recycle / renegotiation**:
- *Tiled EGL path*: **not reachable in code** — the `EGLImage` lives strictly inside
`on_process` while the buffer is held; the CUDA registrations wrap our own persistent GL
textures, not producer buffers.
- *LINEAR Vulkan path*: **reachable**`VkBridge::src_cache` keyed by raw fd, never
invalidated: a pool teardown + fd-number reuse could serve a stale imported buffer (wrong
frame or driver fault), and old entries leaked. Fixed by st_ino keys + `CLEAR_CACHE` on
renegotiation + an LRU cap.
- **Virtual-output teardown / mode change racing an in-flight map** — same class as compositor
crash (external invalidation, another thread); contained by the worker.
- **Output removal** — ditto.
## 4. In-process lifetime fixes (also shipped, they harden the worker itself)
- `Nv12Blit::drop` deleted its GL textures **before** the struct fields dropped, i.e. while
`y_tex`/`uv_tex` were still CUDA-registered. Now `RegisteredTexture::release()` runs first
(unregister → delete), removing a driver-state hazard of exactly the class that crashed.
- `GlBlit` had **no** `Drop` — its GL program/VAO/FBO/textures leaked on every size change and on
importer teardown. Now mirrors `Nv12Blit` (release registrations, then delete GL objects).
## 5. Residual risks, accepted
- A worker death while the encode thread still holds an IPC-mapped frame: the exporting process is
gone; the host-side mapping stays open until the `DeviceBuffer` drops. CUDA surfaces this as a
copy error at worst (encode error → session rebuild), not a host fault.
- The VAAPI encoder's in-host VA dmabuf import (Mesa) keeps its current exposure; no NVIDIA-class
faults observed there.
- `cuIpcOpenMemHandle` requires same-device, different-process — both hold by construction.
## 6. Validation
- **GPU-less (CI / dev VM)**: protocol unit tests (framing, fd round-trip over a socketpair,
error propagation, dead-worker detection against a mock server, latch behavior); worker-spawn
failure path (spawning a non-worker exe ⇒ clean fallback).
- **On-glass (NVIDIA RTX 5070 Ti + GNOME/Mutter, .21, 2026-07-06)** — steps 12 **PASSED**:
1. streamed `PUNKTFUNK_ZEROCOPY=1` through the worker (`zerocopy import worker ready`
`zero-copy GPU import isolated in a worker process``dmabuf imported to CUDA … nv12=true`),
end-to-end **p50 1.30 ms** (1800 frames, 0 mismatched) — parity with the pre-isolation path;
2. `kill -9` the worker mid-stream → host **survived**; the next import logged
`tiled GPU import lost — failing this capture for rebuild … Broken pipe … dead=true`, then
`capture lost — rebuilding pipeline in place, rebuild=1`, a **fresh worker (new pid) in
~185 ms**, and streaming resumed (2385 frames, 0 mismatched; single 33 ms frame at the seam).
The `worker-ready` count was 2 (original + rebuild), confirming the respawn.
Still pending: 3. a real compositor kill/restart mid-stream on a KWin box (the exact original
trigger — a `kill -9` of the worker is a strictly harsher event, so this is corroboration not a
gap); 4. `nv12-selftest` (in-process path untouched). *Note: on a static virtual desktop the
dead-worker detection only fires once a new frame triggers an import — realistic (a running game
produces continuous frames) but it means an idle desktop can sit poisoned-but-quiet briefly.*
## 7. Files
- `crates/punktfunk-host/src/linux/zerocopy/proto.rs` — message types + SEQPACKET/SCM_RIGHTS I/O.
- `crates/punktfunk-host/src/linux/zerocopy/worker.rs` — worker main loop (`zerocopy-worker`),
backend trait (testable), EGL/CUDA backend.
- `crates/punktfunk-host/src/linux/zerocopy/client.rs``RemoteImporter` (spawn, handshake, IPC
mapping cache, release plumbing, reaping) + the `Importer` enum (Remote | InProc).
- `crates/punktfunk-host/src/linux/zerocopy/cuda.rs` — CUDA IPC entry points; remote-release
`DeviceBuffer`s.
- `crates/punktfunk-host/src/linux/zerocopy/egl.rs` — teardown-order fixes (§4).
- `crates/punktfunk-host/src/capture/linux/mod.rs``Importer` wiring, tiled-failure poisoning,
death latch, `CLEAR_CACHE` on renegotiation.
- `crates/punktfunk-host/src/main.rs` — the hidden `zerocopy-worker` subcommand.
+23 -35
View File
@@ -58,25 +58,30 @@ tuning, and example configs. Updates later are just `sudo pacman -Syu`.
## 4. Configure and run
The host runs as a systemd **`--user`** service — it needs your session's PipeWire and D-Bus.
Copy a starting config, enable the service, and enable linger so it starts at boot without a login:
The host runs as a systemd **`--user`** service — it needs your session's PipeWire and D-Bus. Copy a
starting config:
```sh
mkdir -p ~/.config/punktfunk
cp /usr/share/punktfunk/host.env.example ~/.config/punktfunk/host.env # then edit
cp /usr/share/punktfunk/host.env.example ~/.config/punktfunk/host.env
```
How the host creates its virtual display and injects input depends on your desktop, not your distro —
edit `host.env` for the desktop you run, following its page for the exact settings and any quirks:
- [KDE Plasma (KWin)](/docs/kde)
- [GNOME (Mutter)](/docs/gnome)
- [Steam / gamescope](/docs/gamescope)
- [Sway / wlroots](/docs/sway)
Then enable the service and turn on linger so it starts at boot without a login:
```sh
systemctl --user daemon-reload
systemctl --user enable --now punktfunk-host
sudo loginctl enable-linger "$USER"
```
Which compositor the host captures depends on your desktop — it drives a per-client virtual output
via KWin (Plasma), Mutter (GNOME), or wlroots (Sway), or spawns a headless **gamescope** session
per connect. For a headless appliance, the package also ships `punktfunk-kde-session.service`
(a dedicated `kwin --virtual` session, same as the [Fedora KDE](/docs/fedora-kde#3-kwin-streaming-session)
guide — `cp /usr/share/punktfunk/host.env.kde ~/.config/punktfunk/host.env` and enable it alongside
the host). See [Configuration](/docs/configuration) for every knob and
[Running as a Service](/docs/running-as-a-service) for the service model.
Check it came up:
```sh
@@ -84,27 +89,10 @@ systemctl --user status punktfunk-host # active
journalctl --user -u punktfunk-host -f # watch a client connect
```
### Web console
The console (status, paired devices, arm pairing) ships as `punktfunk-web` — enable it, then open
`http://<host-ip>:47992`:
```sh
systemctl --user enable --now punktfunk-web
```
#### Console login password
On first start `punktfunk-web-init` generates a random login password and saves it to
`~/.config/punktfunk/web-password` (as `PUNKTFUNK_UI_PASSWORD=…`). Read it back at any time:
```sh
journalctl --user -u punktfunk-web-init | sed -n 's/.*password generated: //p'
sed -n 's/^PUNKTFUNK_UI_PASSWORD=//p' ~/.config/punktfunk/web-password
```
To set your own, edit that file and `systemctl --user restart punktfunk-web`. Forgot it? See
[Forgot your Password?](/docs/forgot-password).
Enable the browser console, find your login password, and arm PIN pairing from
[The Web Console](/docs/web-console). For a headless KWin appliance that streams at boot with no
graphical login, see [KDE → Headless session](/docs/kde#headless-session). Full reference:
[Configuration](/docs/configuration) · [Running as a Service](/docs/running-as-a-service).
## 5. Open the firewall (if you have one)
@@ -147,9 +135,9 @@ opened. Full port lists (`nftables`, explicit ports) are in
## 6. Connect a client
From any [client](/docs/clients), `--discover` finds the host on the LAN. On first connect, complete
the **PIN pairing** arm it from the host's web console, which displays a 4-digit PIN to type into
the client. (Pairing is required by default; pass `serve --open` only if you deliberately want to
disable it.) See [Clients](/docs/clients) and [Pairing](/docs/pairing).
the **PIN pairing**: arm it from [The Web Console](/docs/web-console#arm-pairing), which displays a
4-digit PIN to type into the client. (Pairing is required by default; pass `serve --open` only if
you deliberately want to disable it.) See [Clients](/docs/clients) for per-platform setup.
## Appendix — build from source (PKGBUILD)
+20 -34
View File
@@ -16,8 +16,8 @@ mid-stream. You flip between Gaming Mode and Desktop with Bazzite's normal Steam
`host.env` forces a mode.
> Ideal for a dedicated game-streaming box that you also occasionally want as a remote desktop. For a
> pure desktop machine, [Ubuntu/Fedora KDE](/docs/ubuntu-kde) or [GNOME](/docs/ubuntu-gnome) are
> simpler.
> pure desktop machine, install on [Ubuntu](/docs/ubuntu) or [Fedora](/docs/fedora) and configure the
> [KDE](/docs/kde) or [GNOME](/docs/gnome) desktop directly — simpler.
> New here? Read [Security & Safe Use](/docs/security) first — a streaming host is remote control of
> the machine, so keep it on a trusted LAN or VPN and require pairing.
@@ -60,7 +60,7 @@ For a fully baked appliance image there's also a **bootc** Containerfile that in
from the registry at image-build time — see `packaging/bootc/` in the repo. Plain `rpm-ostree`
layering from the [RPM registry](https://git.unom.io/unom/-/packages) keeps working too (see
`packaging/bazzite/README.md`), but the sysext is the supported default. Building from source
also works (Bazzite is Fedora Atomic underneath — same steps as [Fedora KDE](/docs/fedora-kde)).
also works (Bazzite is Fedora Atomic underneath — same steps as [Fedora](/docs/fedora)).
## Allow controller input
@@ -85,13 +85,13 @@ cp /usr/share/punktfunk/host.env.bazzite ~/.config/punktfunk/host.env
The template is deliberately minimal — it does **not** force a compositor, because the host
auto-detects Gaming Mode (gamescope) vs Desktop (KWin) on every connect and follows the switch
mid-stream. The only settings that matter are the session anchors plus zero-copy:
mid-stream. The only settings that matter are the session anchors (GPU zero-copy is on by default):
```sh
XDG_RUNTIME_DIR=/run/user/1000
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
PUNKTFUNK_VIDEO_SOURCE=virtual
PUNKTFUNK_ZEROCOPY=1 # GPU zero-copy (dmabuf → CUDA → NVENC); auto-falls back to CPU
# GPU zero-copy (dmabuf → CUDA → NVENC) is ON by default; auto-falls back to CPU. Set =0 to force CPU.
PUNKTFUNK_GAMESCOPE_ATTACH=1 # Gaming Mode = attach to the box's own session (see below)
```
@@ -99,15 +99,14 @@ PUNKTFUNK_GAMESCOPE_ATTACH=1 # Gaming Mode = attach to the box's own session
For Gaming Mode there are two models (pick one; the shipped default is **attach**):
- **Attach** (`PUNKTFUNK_GAMESCOPE_ATTACH=1`, the default) — the **box** owns its gamescope session
and decides Gaming vs Desktop via the normal Steam UI. The host just attaches to whatever's live
and never tears it down, so switching Desktop ↔ Game is rock-solid and disconnecting leaves the box
where it was. The streamed game-mode resolution is the box's gamescope mode
(`SCREEN_WIDTH/HEIGHT` in `/etc/gamescope-session-plus/sessions.d/steam`), not the client's.
- **Managed** (`PUNKTFUNK_GAMESCOPE_MANAGED=1`, and remove the attach line) — the host tears the
box's gamescope down on connect and launches its **own** at the *client's* exact resolution and
refresh, restoring on idle. Client-mode-following, but it can't coexist with a box-owned game-mode
session, and there must be **no physical gaming session already running**.
- **Attach** (`PUNKTFUNK_GAMESCOPE_ATTACH=1`, the default) — the **box** owns its gamescope session,
the host attaches to whatever's live and never tears it down, and the streamed game-mode resolution
is the box's own gamescope mode. Switching Desktop ↔ Game is rock-solid.
- **Managed** (`PUNKTFUNK_GAMESCOPE_MANAGED=1`, and remove the attach line) — the host launches its
**own** gamescope at the *client's* exact resolution and refresh. Client-mode-following, but there
must be no physical gaming session already running.
Full treatment: [Steam / gamescope → Attach vs managed](/docs/gamescope#attach-vs-managed).
Mid-stream Gaming ↔ Desktop following (`PUNKTFUNK_SESSION_WATCH`) is **on by default** on
Bazzite/SteamOS. See [Configuration](/docs/configuration) for the full list of knobs.
@@ -116,8 +115,8 @@ Bazzite/SteamOS. See [Configuration](/docs/configuration) for the full list of k
The **virtual output** (video) for the Desktop session needs no config — the host package ships an
`io.unom.Punktfunk.Host.desktop` file whose `X-KDE-Wayland-Interfaces` grants the host KWin's
restricted screencast protocol on a normal interactive Plasma session (least-privilege, the same
mechanism krfb/krdp use). After a **fresh host install, log out and back into the Desktop session
restricted screencast protocol on a normal interactive Plasma session (background:
[KDE Plasma](/docs/kde)). After a **fresh host install, log out and back into the Desktop session
once** so KWin re-reads that grant.
The one thing a normal KDE login lacks is the RemoteDesktop grant for headless **input** injection.
@@ -138,26 +137,11 @@ Desktop; it follows whichever the box is in.
```sh
systemctl --user enable --now punktfunk-host
# Web console (pairing + status) — enable it and read the auto-generated login password,
# then open http://<host-ip>:47992:
systemctl --user enable --now punktfunk-web
journalctl --user -u punktfunk-web-init | sed -n 's/.*password generated: //p'
systemctl --user enable --now punktfunk-web # web console: pairing + status
```
### Console login password
The console is password-protected. On first start `punktfunk-web-init` generates a random login
password and saves it to `~/.config/punktfunk/web-password` (as `PUNKTFUNK_UI_PASSWORD=…`). Read it
back at any time — from the init service's journal, or straight from the file:
```sh
journalctl --user -u punktfunk-web-init | sed -n 's/.*password generated: //p'
sed -n 's/^PUNKTFUNK_UI_PASSWORD=//p' ~/.config/punktfunk/web-password
```
To set your own password, edit that file (`PUNKTFUNK_UI_PASSWORD=<your-password>`) and restart the
console: `systemctl --user restart punktfunk-web`. Forgot it? This is the recovery path linked from
the console login screen — see [Forgot your Password?](/docs/forgot-password).
Then open [The Web Console](/docs/web-console) for the login password and to
[arm pairing](/docs/web-console#arm-pairing).
## Good to know
@@ -170,5 +154,7 @@ These apply to the **Gaming Mode (gamescope)** path; the KDE Desktop path is una
- **HDR isn't supported yet** on the gamescope path — gamescope's capture output is 8-bit. SDR streams
normally.
Canonical list: [gamescope → Known limits](/docs/gamescope#known-limits).
Then [connect a client](/docs/clients) — Moonlight works great for couch gaming, and the Apple app for
Apple TV / iPad.
+5 -3
View File
@@ -36,7 +36,7 @@ On Linux the host **rewrites `WAYLAND_DISPLAY` / `XDG_CURRENT_DESKTOP` / `XDG_RU
|---|---|---|
| `PUNKTFUNK_COMPOSITOR` | `kwin` · `mutter` · `gamescope` · `wlroots` (aliases: `kde`/`plasma`, `gnome`, `sway`/`hyprland`) | Which backend creates the virtual display. **Leave unset to auto-detect;** set only to force one. |
| `PUNKTFUNK_VIDEO_SOURCE` | `virtual` · `portal` | `virtual` creates a per-client display at the client's exact mode (the normal choice). `portal` captures an existing monitor instead. |
| `PUNKTFUNK_ZEROCOPY` | `1` · `0` | GPU zero-copy capture→encode (dmabuf → CUDA → NVENC, or D3D11 on Windows). Leave on; it falls back to a CPU path automatically. |
| `PUNKTFUNK_ZEROCOPY` | `1` · `0` *(default on)* | GPU zero-copy capture→encode (dmabuf → CUDA → NVENC, or D3D11 on Windows). **On by default** — no need to set it; it falls back to a CPU path automatically. Set `0` to force the CPU path. One exception: Windows **Intel/QSV** keeps the CPU path by default until zero-copy is validated on Intel hardware — set `1` to try it there. |
| `PUNKTFUNK_INPUT_BACKEND` | `libei` · `gamescope` · `wlr` · `uinput` | How input is injected. `libei` for GNOME/KDE, `gamescope` for Bazzite/gamescope, `wlr` for Sway/wlroots. Auto-detected with the compositor. |
| `PUNKTFUNK_ENCODER` | `auto` · `nvenc` · `vaapi` (Linux) · `amf` · `qsv` (Windows) · `software` | Encoder backend. `auto` (default) detects the GPU vendor: NVIDIA→NVENC, AMD→VAAPI/AMF, Intel→VAAPI/QSV. `software` (aliases `sw`/`openh264`) is the GPU-less H.264 path on both platforms — on Windows `auto` falls back to it when no GPU is found; on Linux it is **explicit-only** (`auto` never picks it). |
| `PUNKTFUNK_RENDER_NODE` | path | Linux DRM render node for zero-copy (default `/dev/dri/renderD128`). Set on multi-GPU boxes to pick the right GPU. |
@@ -48,8 +48,8 @@ let you pick a mode or default to the device's display.)
## gamescope / session following (Linux, Bazzite/SteamOS)
Two mutually-exclusive models for a Steam/gamescope box. See [Bazzite](/docs/bazzite) for the full
picture.
Two mutually-exclusive models for a Steam/gamescope box. See [Steam / gamescope](/docs/gamescope) for
the full picture (and [Bazzite](/docs/bazzite) for that distro's specifics).
| Setting | Values | Meaning |
|---|---|---|
@@ -62,6 +62,8 @@ picture.
## Compositor-specific (Linux)
See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set these.
> **Managing virtual displays** — keep-alive after disconnect, exclusive vs. extend, and (on
> Windows/KDE) persistent per-client scaling — now has its own settings surface in the web console
> and `display-settings.json`. See [Virtual displays](/docs/virtual-displays). The two
@@ -1,24 +1,25 @@
---
title: Fedora — KDE Plasma
description: Reproducible punktfunk host setup on Fedora KDE (KWin) via the RPM.
title: Fedora
description: Install the punktfunk host on Fedora from the RPM registry.
---
Set up a punktfunk host on **Fedora KDE** (the KDE Plasma spin). The host runs as an RPM-managed
systemd service and uses KWin to create per-client virtual displays, captured zero-copy
(dmabuf → CUDA → NVENC) on NVIDIA.
> Validated live on **Fedora 44 KDE Plasma** with an RTX 4090: KWin virtual output + full
> zero-copy capture. Everything below is the reproducible flow — paste it on a fresh box.
Install a punktfunk host on **Fedora** from the self-hosted RPM registry. The host installs as an
RPM-managed systemd **`--user`** service and updates with `dnf upgrade` like the rest of your
system — no building required. It works with either **KDE Plasma** or **GNOME**; the
desktop-specific setup (which compositor captures, headless sessions, quirks) lives on the
[desktop configure pages](#3-configure-your-desktop). Host encode is **NVENC on NVIDIA** and **VAAPI on
AMD/Intel** (`PUNKTFUNK_ENCODER=auto` picks per GPU).
> New here? Read [Security & Safe Use](/docs/security) first — a streaming host is remote control of
> the machine, so keep it on a trusted LAN or VPN and require pairing.
The setup has three parts: **NVIDIA driver****host RPM****KWin streaming session**.
Install is two parts: **GPU driver****host RPM**. Then point the host at your desktop from the
[desktop configure pages](#3-configure-your-desktop).
## 1. NVIDIA driver (RPM Fusion akmod)
Enable RPM Fusion (free + nonfree), then install the akmod driver + CUDA. RPM Fusion's nonfree
NVIDIA repo is sometimes pre-enabled on the KDE spin; the full free/nonfree repos below are still
NVIDIA repo is sometimes pre-enabled on some spins; the full free/nonfree repos below are still
needed (they carry the NVENC ffmpeg in the next step).
```sh
@@ -55,6 +56,11 @@ ffmpeg -hide_banner -encoders | grep nvenc
(Or disable Secure Boot in firmware to skip the MOK step — fine for a dedicated test box.)
**AMD / Intel (VAAPI).** No akmod needed — the Mesa stack provides the VAAPI encoder. Install the
freeworld VAAPI drivers for full codec support (`mesa-va-drivers-freeworld` for AMD from RPM Fusion,
`intel-media-driver` for Intel); on a desktop these are usually already present. The host auto-picks
VAAPI on these GPUs.
## 2. Install the host (RPM)
The host is published to the self-hosted Gitea RPM registry, in a per-Fedora-release group (an RPM
@@ -85,71 +91,37 @@ udev rule, the UDP socket-buffer sysctl tuning, and example configs.
> `docker build --build-arg FEDORA_VERSION=NN -f ci/fedora-rpm.Dockerfile -t pf-rpm ci` then run
> `packaging/rpm/build-rpm.sh` inside it — or build from source (appendix below).
## 3. KWin streaming session
## 3. Configure your desktop
KWin's virtual-output capture uses its **privileged** `zkde_screencast` protocol, which an
*interactive* Plasma session will not hand to an external client. So the host streams from a
**dedicated headless KWin session** (`kwin --virtual` launched with
`KWIN_WAYLAND_NO_PERMISSION_CHECKS=1`) — shipped as `punktfunk-kde-session.service`. This also
makes the box a self-contained appliance: it streams at boot with no graphical login.
How the host creates its virtual display and injects input depends on your desktop, not your distro.
Continue on the page for the desktop you run — it covers your `host.env`, any compositor quirks, and
starting the host:
```sh
# KWin appliance config (ships with the package):
mkdir -p ~/.config/punktfunk
cp /usr/share/punktfunk/host.env.kde ~/.config/punktfunk/host.env
- [KDE Plasma (KWin)](/docs/kde)
- [GNOME (Mutter)](/docs/gnome)
- [Steam / gamescope](/docs/gamescope)
- [Sway / wlroots](/docs/sway)
# Start the headless KWin session + the host, and start user units at boot without a login:
systemctl --user daemon-reload
systemctl --user enable --now punktfunk-kde-session punktfunk-host
sudo loginctl enable-linger "$USER"
```
Enable the browser management console (status, paired devices, arm pairing) — see
[Web Console](/docs/web-console).
Check it came up:
For a headless KWin appliance that streams at boot with no graphical login, see
[KDE → Headless session](/docs/kde#headless-session).
```sh
systemctl --user status punktfunk-host # active
journalctl --user -u punktfunk-host -f # watch a client connect
```
The host now listens on `9777` (native punktfunk/1) + the GameStream ports, and advertises over
mDNS. It requires **PIN pairing** by default (secure on a LAN); pair once from your client.
### Web console
The console (status, paired devices, arm pairing) ships as `punktfunk-web` — enable it, then open
`http://<host-ip>:47992`:
```sh
systemctl --user enable --now punktfunk-web
```
#### Console login password
The console is password-protected. On first start `punktfunk-web-init` generates a random login
password and saves it to `~/.config/punktfunk/web-password` (as `PUNKTFUNK_UI_PASSWORD=…`). Read it
back at any time — from the init service's journal, or straight from the file:
```sh
journalctl --user -u punktfunk-web-init | sed -n 's/.*password generated: //p'
sed -n 's/^PUNKTFUNK_UI_PASSWORD=//p' ~/.config/punktfunk/web-password
```
To set your own password, edit that file (`PUNKTFUNK_UI_PASSWORD=<your-password>`) and restart the
console: `systemctl --user restart punktfunk-web`. Forgot it? This is the recovery path linked from
the console login screen — see [Forgot your Password?](/docs/forgot-password).
Full config reference: [Configuration](/docs/configuration). Service model:
[Running as a Service](/docs/running-as-a-service).
## 4. Connect a client
From any [client](/docs/clients) `punktfunk-client --discover` finds the host on the LAN. On
first connect, complete the PIN pairing — **arm it from the host's web console / mgmt API**, which
makes the host display a 4-digit PIN to type into the client. (Pairing is required by default; pass
`serve --open` only if you deliberately want to disable the requirement.) See
[Clients](/docs/clients) and [Running as a Service](/docs/running-as-a-service).
From any [client](/docs/clients), `--discover` finds the host on the LAN. On first connect, complete
the **PIN pairing** — arm it from the host's [web console](/docs/web-console#arm-pairing), which
displays a 4-digit PIN to type into the client. See [Clients](/docs/clients) and
[Pairing](/docs/pairing).
## Appendix — build from source
If there's no RPM for your Fedora release and you don't want to build one, compile the host
directly (no clean updates / no packaged units — you wire those up by hand):
If there's no RPM for your Fedora release and you don't want to build one, compile the host directly
(no clean updates / no packaged units — you wire those up by hand):
```sh
sudo dnf install gcc gcc-c++ make cmake clang clang-devel nasm git \
@@ -162,4 +134,5 @@ cargo build --release -p punktfunk-host
```
Then write `~/.config/punktfunk/host.env` (as in `/usr/share/punktfunk/host.env.kde`, but the host
binary is `target/release/punktfunk-host`) and run it inside the KWin session from step 3.
binary is `target/release/punktfunk-host`) and run it inside your desktop session — for a headless
KWin appliance see [KDE → Headless session](/docs/kde#headless-session).
+6 -7
View File
@@ -8,21 +8,20 @@ password. That password is generated — or, on Windows, chosen — when the con
it lives on the **host**. So if you can't get past the login screen, you recover or change it on the
host machine itself, not from the browser.
New to the console? See [The Web Console](/docs/web-console) to enable it and arm pairing.
> This is **only** the web console login. It is **not** your client/device pairing — if a client
> won't connect, that's [Pairing](/docs/pairing), not this password.
## Find your host
Jump to your host platform for exactly where the password lives and how to read or reset it:
Find your host platform for exactly where the password lives, then read or reset it below:
| Host | Where the password lives | Section |
|------|--------------------------|---------|
| **Ubuntu — GNOME** | `~/.config/punktfunk/web-password` | [Console login password](/docs/ubuntu-gnome#console-login-password) |
| **Ubuntu — KDE Plasma** | `~/.config/punktfunk/web-password` | [Console login password](/docs/ubuntu-kde#console-login-password) |
| **Fedora — KDE Plasma** | `~/.config/punktfunk/web-password` | [Console login password](/docs/fedora-kde#console-login-password) |
| **Bazzite — gamescope** | `~/.config/punktfunk/web-password` | [Console login password](/docs/bazzite#console-login-password) |
| **SteamOS (host)** | `~/.config/punktfunk/web.env` | [Console login password](/docs/steamos-host#console-login-password) |
| **Windows host** | `%ProgramData%\punktfunk\web-password` | [Console login password](/docs/windows-host#console-login-password) |
| **Linux packages (apt / RPM / Bazzite)** | `~/.config/punktfunk/web-password` | [Login password](/docs/web-console#login-password) |
| **SteamOS (host)** | `~/.config/punktfunk/web.env` | [Login password](/docs/web-console#login-password) |
| **Windows host** | `%ProgramData%\punktfunk\web-password` | [Login password](/docs/web-console#login-password) · [Windows Host](/docs/windows-host) |
## The short version
+77
View File
@@ -0,0 +1,77 @@
---
title: Steam / gamescope
description: Configure a gamescope/Steam host — attach vs managed, session following, and limits.
---
gamescope is the compositor behind Steam **Gaming Mode** — the couch/handheld game UI on Bazzite,
SteamOS, or any distro running a gamescope session. The host **auto-detects** gamescope from your
live session, so you rarely need to set anything here. It also **follows a Gaming ↔ Desktop switch
mid-stream** — flip between Gaming Mode and the desktop with Steam's normal UI and the host
re-targets whatever's running without a reconnect.
This page covers the gamescope-specific choices. To get a host running on an appliance box, start
from the install guide for your OS: [Bazzite](/docs/bazzite) or [SteamOS (Host)](/docs/steamos-host).
> New here? Read [Security & Safe Use](/docs/security) first — a streaming host is remote control of
> the machine, so keep it on a trusted LAN or VPN and require pairing.
## Attach vs managed
There are two mutually-exclusive models for a gamescope box; pick one. The shipped default is
**attach**.
- **Attach** (`PUNKTFUNK_GAMESCOPE_ATTACH=1`, the default) — the **box** owns its gamescope session
and decides Gaming vs Desktop via the normal Steam UI. The host just attaches to whatever's live
and never tears it down, so switching Desktop ↔ Game is rock-solid and disconnecting leaves the box
where it was. The streamed game-mode resolution is the box's gamescope mode
(`SCREEN_WIDTH/HEIGHT` in `/etc/gamescope-session-plus/sessions.d/steam`), not the client's.
- **Managed** (`PUNKTFUNK_GAMESCOPE_MANAGED=1`, and remove the attach line) — the host tears the
box's gamescope down on connect and launches its **own** at the *client's* exact resolution and
refresh, restoring on idle. Client-mode-following, but it can't coexist with a box-owned game-mode
session, and there must be **no physical gaming session already running**.
## Session following
`PUNKTFUNK_SESSION_WATCH` follows a Gaming ↔ Desktop switch **mid-stream** — the host rebuilds the
backend in place, with no reconnect. It is **on by default** on Bazzite/SteamOS; set `0` to disable.
One host service covers both faces of the box: it streams Gaming Mode over gamescope and the desktop
over its own compositor, and re-targets whichever is live on each switch.
## Start the host
On an appliance box (Bazzite, SteamOS) the install guide already enables the host service for you. On
any other distro running a gamescope session, start it from your session — the default attach model
just latches onto whatever gamescope session is live:
```sh
systemctl --user enable --now punktfunk-host
```
Then bring up [The Web Console](/docs/web-console) to arm pairing.
## gamescope knobs
The gamescope-specific settings in `host.env`. Leave them unset to auto-detect; set one only to force
a model. See the full [Configuration reference](/docs/configuration) for every other knob.
| Setting | Values | Meaning |
|---|---|---|
| `PUNKTFUNK_GAMESCOPE_ATTACH` | `1` | **Attach** model: the box owns its gamescope session; the host captures whatever's live and never tears it down. Streamed resolution is the box's gamescope mode. The default. |
| `PUNKTFUNK_GAMESCOPE_MANAGED` | `1` | **Managed** model: the host tears the box's gamescope down on connect and launches its own at the client's exact mode, restoring on idle. Doesn't coexist with a box-owned game-mode session. |
| `PUNKTFUNK_GAMESCOPE_SESSION` | `steam` | The host owns a `gamescope-session-plus` (Steam) session at the client's mode — a headless appliance with no physical session running. |
| `PUNKTFUNK_GAMESCOPE_NODE` | `auto` · node id | Discover and capture a **running** gamescope's PipeWire node at a fixed mode. Do **not** combine with `SESSION`. |
| `PUNKTFUNK_GAMESCOPE_APP` | command | For an ad-hoc bare-gamescope session, the nested command to run (e.g. `vkcube`). |
| `PUNKTFUNK_SESSION_WATCH` | `1` · `0` | Follow a Gaming ↔ Desktop switch mid-stream (rebuild in place, no reconnect). On by default on Bazzite/SteamOS; set `0` to disable. |
## Known limits
These apply to the **Gaming Mode (gamescope)** path only; the desktop path is unaffected.
- **gamescope 3.16.22 or newer is required.** Older versions can deadlock during capture. Bazzite's
and SteamOS's current gamescope is fine; this only bites if you've pinned an old one.
- **The mouse cursor isn't included in the captured image** — a gamescope limitation for now.
- **HDR isn't supported on the gamescope path** — gamescope's capture output is 8-bit. SDR streams
normally.
To stream the KDE Plasma desktop of a Steam box instead, see [KDE Plasma](/docs/kde). To bring up the
web console and pair a client, see [The Web Console](/docs/web-console).
+96
View File
@@ -0,0 +1,96 @@
---
title: GNOME (Mutter)
description: Configure a punktfunk host for GNOME — host.env, the EGL/lock traps, and a headless session.
---
Configure a host running **GNOME**. The host drives GNOME's Mutter compositor to create a per-client
virtual display over D-Bus (`RecordVirtual`), zero-copy. This page assumes the host is already
installed — see [Ubuntu](/docs/ubuntu), [Fedora](/docs/fedora), or [Arch](/docs/arch).
> New here? Read [Security & Safe Use](/docs/security) first — a streaming host is remote control of
> the machine, so keep it on a trusted LAN or VPN and require pairing.
## host.env
Write `~/.config/punktfunk/host.env` with the GNOME settings. The host auto-detects the compositor
from your session, so the explicit `PUNKTFUNK_COMPOSITOR` is belt-and-braces:
```ini
# ~/.config/punktfunk/host.env
WAYLAND_DISPLAY=wayland-0
XDG_CURRENT_DESKTOP=GNOME
PUNKTFUNK_COMPOSITOR=mutter
PUNKTFUNK_VIDEO_SOURCE=virtual
# GPU zero-copy (dmabuf → CUDA → NVENC) is ON by default; auto-falls back to CPU. Set =0 to force CPU.
PUNKTFUNK_INPUT_BACKEND=libei
```
You must be on a **Wayland** session (not X11), and Mutter must be **≥ 48**. See the
[Configuration reference](/docs/configuration) for every option.
## The GL/EGL userspace
On NVIDIA, gnome-shell fails to start — or the host logs **"GPU … not supported by EGL"** — when the
NVIDIA GL/EGL userspace is missing. The base driver package doesn't always pull it in. Install your
distro's NVIDIA GL/EGL userspace package — on **Ubuntu/Debian** it's `libnvidia-gl-<version>` matching
your driver; on **Fedora/Arch** it ships with the RPM Fusion / repo driver — then confirm the glvnd
vendor file exists:
```sh
ls /usr/share/glvnd/egl_vendor.d/10_nvidia.json # must exist
```
Installing the driver itself is covered on your distro's install page
([Ubuntu](/docs/ubuntu), [Fedora](/docs/fedora), [Arch](/docs/arch)).
## Do not lock the session
A **locked** GNOME session blocks screen capture — the host fails with
**"Session creation inhibited"**. On an always-on or headless host there's no one to unlock it, so
disable the lock:
```sh
gsettings set org.gnome.desktop.screensaver lock-enabled false
gsettings set org.gnome.desktop.session idle-delay 0
```
## Start the host
With `host.env` in place, start the host from **inside your GNOME session**:
```sh
systemctl --user enable --now punktfunk-host
journalctl --user -u punktfunk-host -f # watch it come up and print its identity fingerprint
```
Then bring up [The Web Console](/docs/web-console) to arm pairing and connect a
[client](/docs/clients). For an always-on box, see the [headless session](#headless-session) below.
## Headless session
To run with no monitor and no login, keep a GNOME Wayland session up at all times and start the host
without a login. Have GDM auto-login your user:
```ini
# /etc/gdm3/custom.conf (Ubuntu) · /etc/gdm/custom.conf (Fedora)
[daemon]
AutomaticLoginEnable = true
AutomaticLogin = your-user
```
Disable the lock (see [above](#do-not-lock-the-session)), then enable the host user service and let it
linger past logout:
```sh
systemctl --user enable --now punktfunk-host
sudo loginctl enable-linger "$USER"
```
Reboot and the host comes up on the auto-login session. Full walkthrough:
[Running as a Service](/docs/running-as-a-service).
## Troubleshooting
More fixes — black screen, discovery, pairing — in [Troubleshooting](/docs/troubleshooting).
Once the host is up, bring the console up and pair — see [The Web Console](/docs/web-console).
+1 -1
View File
@@ -68,4 +68,4 @@ LAN.
## Multiple devices at once
A host can stream to several clients simultaneously — your laptop and your TV both viewing (and
controlling) the desktop, each at its own resolution. See [Multiple devices](/docs/configuration#multiple-devices).
controlling) the desktop, each at its own resolution. See [Multiple devices](/docs/configuration#multiple-devices-at-once).
+2 -2
View File
@@ -29,7 +29,7 @@ It's built for the things that make streaming feel native:
<Cards>
<Card title="How It Works" href="/docs/how-it-works" description="The ideas behind punktfunk in a few minutes — virtual displays, the two protocols, pairing." />
<Card title="Quick Start" href="/docs/quickstart" description="From nothing to streaming: set up a host and connect your first client." />
<Card title="Host Setup" href="/docs/requirements" description="Install the host on Ubuntu (GNOME or KDE), Fedora (KDE), or Bazzite." />
<Card title="Host Setup" href="/docs/requirements" description="Install on Ubuntu, Fedora, Arch, Bazzite, SteamOS, or Windows." />
<Card title="Connect a Client" href="/docs/clients" description="Stream with the native app for your device — macOS, Linux, Windows, Android — or any Moonlight client." />
<Card title="API Reference" href="/api" description="Interactive OpenAPI reference for the host's management REST API — status, devices, pairing, library." />
</Cards>
@@ -37,7 +37,7 @@ It's built for the things that make streaming feel native:
## What you need
- A **host** with a supported GPU — either a **Linux** machine running one of the
[supported setups](/docs/requirements) (**Ubuntu** GNOME or KDE, **Fedora** KDE, or **Bazzite**), or
[supported setups](/docs/requirements) (**Ubuntu**, **Fedora**, **Arch**, or **Bazzite**), or
a **[Windows](/docs/windows-host) PC**.
- A **client device** to stream to — there are native apps for **macOS, iOS/iPadOS, tvOS, Linux,
Windows, and Android**, plus any device that runs **Moonlight**.
+13 -4
View File
@@ -16,9 +16,9 @@ On **Windows**, the host ships as a signed installer instead — see [Windows](#
| Distro | Package manager | One-command happy path | Guide |
|--------|-----------------|------------------------|-------|
| **Ubuntu / Debian** | apt | `sudo apt install punktfunk-host` | [Ubuntu — GNOME](/docs/ubuntu-gnome) · [Ubuntu — KDE](/docs/ubuntu-kde) · [packaging/debian](https://git.unom.io/unom/punktfunk/src/branch/main/packaging/debian/README.md) |
| **Ubuntu / Debian** | apt | `sudo apt install punktfunk-host` | [Ubuntu / Debian](/docs/ubuntu) · [packaging/debian](https://git.unom.io/unom/punktfunk/src/branch/main/packaging/debian/README.md) |
| **Bazzite / Fedora Atomic** | systemd-sysext | `sudo bash punktfunk-sysext.sh install` (no layering, no reboot) | [Bazzite](/docs/bazzite) · [packaging/bazzite](https://git.unom.io/unom/punktfunk/src/branch/main/packaging/bazzite/README.md) |
| **Fedora (dnf)** | dnf / rpm-ostree | `dnf install punktfunk punktfunk-web` | [Fedora — KDE](/docs/fedora-kde) · [packaging/rpm](https://git.unom.io/unom/punktfunk/src/branch/main/packaging/rpm/README.md) |
| **Fedora (dnf)** | dnf / rpm-ostree | `dnf install punktfunk punktfunk-web` | [Fedora](/docs/fedora) · [packaging/rpm](https://git.unom.io/unom/punktfunk/src/branch/main/packaging/rpm/README.md) |
| **Arch** | pacman | `pacman -Sy punktfunk-host` (binary repo) | [Arch Linux](/docs/arch) · [packaging/arch](https://git.unom.io/unom/punktfunk/src/branch/main/packaging/arch/README.md) |
| **SteamOS (host)** | on-device script | `bash scripts/steamdeck/install.sh` | [SteamOS (Host)](/docs/steamos-host) |
@@ -79,13 +79,22 @@ fallback without one. More detail — including the CLI `punktfunk-host service
Bare `serve` is the secure native-only default (native `punktfunk/1` + the web console). On a
trusted LAN, add `--gamestream` to also serve stock [Moonlight](/docs/moonlight) clients.
3. Enable the web console and read its login password, then open `http://<host-ip>:47992`:
3. Enable the web console:
```sh
systemctl --user enable --now punktfunk-web
journalctl --user -u punktfunk-web-init | sed -n 's/.*password generated: //p'
```
Then open `http://<host-ip>:47992`. Reading its [login password](/docs/web-console#login-password)
and [arming PIN pairing](/docs/web-console#arm-pairing) are covered in
[The Web Console](/docs/web-console).
### Configure your desktop
How the virtual display and input work depends on your desktop — see [KDE](/docs/kde),
[GNOME](/docs/gnome), [Steam / gamescope](/docs/gamescope), or [Sway](/docs/sway) for the
compositor-specific setup.
From there, follow the [Quick Start](/docs/quickstart) to pair your first client. To run the host
automatically at boot, see [Running as a Service](/docs/running-as-a-service).
+113
View File
@@ -0,0 +1,113 @@
---
title: KDE Plasma (KWin)
description: Configure a punktfunk host for KDE — host.env, quirks, and a headless KWin session.
---
Configure a punktfunk host on **KDE Plasma**. The host uses KDE's KWin compositor to create a
per-client virtual display, captured zero-copy on NVIDIA. This page assumes the package is already
installed — see [Ubuntu](/docs/ubuntu), [Fedora](/docs/fedora), [Arch](/docs/arch), or the
[Bazzite](/docs/bazzite) appliance.
> New here? Read [Security & Safe Use](/docs/security) first — a streaming host is remote control of
> the machine, so keep it on a trusted LAN or VPN and require pairing.
## host.env
A KDE starter `~/.config/punktfunk/host.env`:
```ini
WAYLAND_DISPLAY=wayland-0
XDG_CURRENT_DESKTOP=KDE
PUNKTFUNK_COMPOSITOR=kwin
PUNKTFUNK_VIDEO_SOURCE=virtual
# GPU zero-copy (dmabuf → CUDA → NVENC) is ON by default; auto-falls back to CPU. Set =0 to force CPU.
PUNKTFUNK_INPUT_BACKEND=libei
```
The host auto-detects the running compositor on every connect, so most of this is optional — the
values above are just what it resolves to on a KWin session. See the
[Configuration reference](/docs/configuration) for every option.
## Use a Wayland session
KDE must run on **Wayland**, not X11 — pick the Wayland session from the picker on the login screen.
The virtual-display path is Wayland-only and will not come up under X11.
KWin must be **6.5.6 or newer** (virtual outputs land there). Check with:
```sh
kwin_wayland --version
```
## Streaming the interactive desktop
To stream a logged-in Plasma desktop (rather than a headless session, below), KWin has to hand the
host its restricted screencast protocol. The host package ships an `io.unom.Punktfunk.Host.desktop`
file whose `X-KDE-Wayland-Interfaces` grants exactly that on a normal interactive session
(least-privilege, the same mechanism krfb/krdp use). After a **fresh install, log out and back into
the Desktop session once** so KWin re-reads the grant.
A normal KDE login still lacks the RemoteDesktop grant that **input** injection needs — without it the
host pops an "Allow remote control?" dialog no headless box can answer. **Fedora and Bazzite** ship a
one-shot helper that seeds it (run once as the streaming user, no root):
```sh
bash /usr/share/punktfunk/bazzite/kde-desktop-setup.sh # Fedora / Bazzite
```
The `.deb` and Arch packages don't include that wrapper. Seed the grant by hand instead — copy the
shipped `kde-authorized` file into the portal store (the share dir is `/usr/share/punktfunk-host` on
Debian/Ubuntu, `/usr/share/punktfunk` on Arch), then log out and back in:
```sh
mkdir -p ~/.local/share/flatpak/db
cp /usr/share/punktfunk*/headless/kde-authorized ~/.local/share/flatpak/db/kde-authorized
```
A login-less appliance skips all of this — its headless session (below) needs none of these grants.
## Start the host
With `host.env` in place, start the host from **inside your Plasma session**:
```sh
systemctl --user enable --now punktfunk-host
journalctl --user -u punktfunk-host -f # watch it come up and print its identity fingerprint
```
Then bring up [The Web Console](/docs/web-console) to arm pairing and connect a
[client](/docs/clients). To start at boot — including fully headless — see the
[headless session](#headless-session) below or [Running as a Service](/docs/running-as-a-service).
## Persistent per-client scaling
KWin round-trips per-client display scale: it names each session's virtual output per client, so a
scale you set for one client (150 %, 125 %, …) is reapplied on that client's next connect. See
[Virtual displays](/docs/virtual-displays).
## Headless session
For a login-less appliance — a box that streams at boot with no graphical login — the host brings up a
**dedicated headless KWin session** rather than relying on an interactive one. It runs its own
`kwin --virtual` session (shipped as the `punktfunk-kde-session.service` unit) with permission checks
relaxed, so it needs none of the interactive grants above.
```sh
mkdir -p ~/.config/punktfunk
cp /usr/share/punktfunk/host.env.kde ~/.config/punktfunk/host.env # Debian/Ubuntu: /usr/share/punktfunk-host/host.env.kde
systemctl --user daemon-reload
systemctl --user enable --now punktfunk-kde-session punktfunk-host
sudo loginctl enable-linger "$USER"
```
The session unit brings up headless KWin; the host unit follows it and starts listening. See
[Running as a Service](/docs/running-as-a-service) for the full headless setup.
## Troubleshooting
- **KWin too old:** virtual outputs need KWin **≥ 6.5.6**. Check with `kwin_wayland --version`.
- **Black screen / no picture:** confirm you're on a Wayland session (not X11) and the NVIDIA GL
userspace is installed. More in [Troubleshooting](/docs/troubleshooting).
To bring the console up and pair, see [The Web Console](/docs/web-console).
+13 -9
View File
@@ -5,27 +5,31 @@
"how-it-works",
"security",
"quickstart",
"install",
"---Host Setup---",
"---Install the host---",
"requirements",
"ubuntu-gnome",
"ubuntu-kde",
"fedora-kde",
"install",
"ubuntu",
"fedora",
"arch",
"bazzite",
"steamos-host",
"windows-host",
"web-console",
"---Configure your desktop---",
"configuration",
"kde",
"gnome",
"gamescope",
"sway",
"running-as-a-service",
"virtual-displays",
"host-cli",
"---Connecting---",
"clients",
"install-client",
"steam-deck",
"moonlight",
"pairing",
"---Configuration---",
"configuration",
"virtual-displays",
"host-cli",
"---Troubleshooting---",
"troubleshooting",
"stats",
+14 -10
View File
@@ -11,15 +11,19 @@ This is the shortest path to a working stream. Each step links to the details.
## 1. Set up the host
On your Linux gaming machine (NVIDIA, AMD, or Intel GPU), follow the guide for your system:
On your gaming machine (NVIDIA, AMD, or Intel GPU), follow the install guide for your system:
- [Ubuntu — GNOME](/docs/ubuntu-gnome)
- [Ubuntu — KDE Plasma](/docs/ubuntu-kde)
- [Fedora — KDE Plasma](/docs/fedora-kde)
- [Bazzite — gamescope / Steam](/docs/bazzite)
- [Ubuntu / Debian](/docs/ubuntu)
- [Fedora](/docs/fedora)
- [Arch](/docs/arch)
- [Bazzite](/docs/bazzite)
- [SteamOS](/docs/steamos-host)
- [Windows host](/docs/windows-host)
Each one covers the GPU driver, the dependencies, and how to build and run the host. Check the
[Requirements](/docs/requirements) first if you're not sure your machine is a fit.
Each one covers the GPU driver, the dependencies, and how to install and run the host. After
installing, configure for your desktop ([KDE](/docs/kde) / [GNOME](/docs/gnome) /
[gamescope](/docs/gamescope) / [Sway](/docs/sway)). Check the [Requirements](/docs/requirements)
first if you're not sure your machine is a fit.
## 2. Start the host
@@ -44,9 +48,9 @@ latency, or any Moonlight client:
the list of hosts found on your network. Select it, and when prompted, **pair**.
- **Anything with Moonlight:** add the host (it should be discovered automatically), then pair.
To pair, the host needs to show a PIN. Arm pairing from the host's web console — the host displays a
4-digit PIN, you type it into the client, and they trust each other from then on. Pairing is required
by default. Full details: [Pairing & Trust](/docs/pairing).
To pair, the host needs to show a PIN. [Arm pairing](/docs/web-console#arm-pairing) from the host's
web console — the host displays a 4-digit PIN, you type it into the client, and they trust each other
from then on. Pairing is required by default. Full details: [Pairing & Trust](/docs/pairing).
## 4. Stream
+28 -20
View File
@@ -6,19 +6,31 @@ description: What you need to run a punktfunk host — GPU, driver, desktop, and
## Supported setups
A punktfunk host runs primarily on a Linux machine with a dedicated GPU — NVIDIA (NVENC) is the
most-exercised path, and AMD/Intel GPUs work via VAAPI (a native
[Windows host](/docs/windows-host) is also available — see below). These are the Linux desktop
environments it supports today, each with its own guide:
most-exercised path, and AMD/Intel GPUs work via VAAPI. A native [Windows host](/docs/windows-host)
is also available. Setup splits along two axes: you **install** the package per distro, then
**configure** the host — and learn its quirks — per desktop/compositor.
| Setup | Desktop / compositor | Guide |
|---|---|---|
| **Ubuntu** (Desktop or Server) | GNOME (Mutter) | [Ubuntu — GNOME](/docs/ubuntu-gnome) |
| **Ubuntu** (Desktop or Server) | KDE Plasma (KWin) | [Ubuntu — KDE](/docs/ubuntu-kde) |
| **Fedora** | KDE Plasma (KWin) | [Fedora — KDE](/docs/fedora-kde) |
| **Bazzite** | gamescope (Steam) | [Bazzite](/docs/bazzite) |
> New here? Read [Security & Safe Use](/docs/security) first — a streaming host is remote control of
> the machine, so keep it on a trusted LAN or VPN and require pairing.
Other wlroots compositors (Sway/Hyprland) also work but aren't a primary target. If your desktop isn't
listed, the host still needs one of these compositor backends to create a virtual display.
**Distros — install the package:**
- [Ubuntu / Debian](/docs/ubuntu)
- [Fedora](/docs/fedora)
- [Arch](/docs/arch)
- [Bazzite](/docs/bazzite)
- [SteamOS](/docs/steamos-host)
**Desktops — configure and quirks:**
- [KDE Plasma (KWin)](/docs/kde)
- [GNOME (Mutter)](/docs/gnome)
- [Steam / gamescope](/docs/gamescope)
- [Sway / wlroots](/docs/sway)
Pick your distro to install, then your desktop to configure — the two are independent. Other
wlroots compositors (Hyprland) work but aren't a primary target; the host still needs one of these
compositor backends to create a virtual display.
> **Windows host:** punktfunk also runs as a native host on **Windows 11 22H2 or newer (x64)** — a
> signed installer that registers a service and bundles a virtual-display driver (whose driver-
@@ -32,8 +44,8 @@ listed, the host still needs one of these compositor backends to create a virtua
encodes the video in hardware.
- **NVIDIA driver 535 or newer** (550+ recommended). The driver must include the **GL/EGL userspace**,
not just `nvidia-utils` — without it the compositor can't initialise the GPU and capture fails. Each
setup guide installs the right package (e.g. `libnvidia-gl-<version>` on Ubuntu).
- **`nvidia-drm modeset=1`** must be enabled (Wayland on NVIDIA needs it). The setup guides cover this.
install guide installs the right package (e.g. `libnvidia-gl-<version>` on Ubuntu).
- **`nvidia-drm modeset=1`** must be enabled (Wayland on NVIDIA needs it). The install guides cover this.
- **AMD / Intel GPUs** encode via **VAAPI** instead (install `mesa-va-drivers` or
`intel-media-driver`; validated live on AMD RDNA3). The NVIDIA-specific notes above don't apply
there. On modern Intel (Gen12/Tiger Lake and newer, including Arc) the driver only offers the
@@ -57,9 +69,9 @@ needs to be running for the user the host runs as. This can be:
Minimum compositor versions (newer is fine):
- **KWin ≥ 6.5.6** (KDE Plasma) — headless virtual outputs.
- **GNOME ≥ 48** (Mutter) — virtual-monitor screen-cast.
- **gamescope ≥ 3.16.22** (Bazzite/Steam) — older versions deadlock during capture.
- **KWin ≥ 6.5.6** ([KDE Plasma](/docs/kde)) — headless virtual outputs.
- **GNOME ≥ 48** ([Mutter](/docs/gnome)) — virtual-monitor screen-cast.
- **gamescope ≥ 3.16.22** ([Bazzite/Steam](/docs/gamescope)) — older versions deadlock during capture.
## Network
@@ -70,10 +82,6 @@ Minimum compositor versions (newer is fine):
- For best results, a wired or fast Wi-Fi link. The host can run a built-in **speed test** to pick a
bitrate for your link (see [Configuration](/docs/configuration)).
> **Before you set up a host, read [Security & Safe Use](/docs/security).** A streaming host is
> remote control of the machine — it's important to understand what that exposes, why to keep it on a
> trusted network, and how pairing protects you.
## A client
You also need something to stream *to* — see [Connect a Client](/docs/clients). There are native
+9 -38
View File
@@ -37,50 +37,21 @@ Start by making the host service start at boot even when nobody logs in:
sudo loginctl enable-linger "$USER"
```
Then bring up a session automatically, depending on your desktop:
Then bring up a session automatically. How you do that is desktop-specific — auto-login, lock
disable, and the session unit differ per compositor, so each is documented on its own page:
### Headless GNOME
- GNOME: [GNOME → Headless session](/docs/gnome#headless-session).
- KDE Plasma: [KDE → Headless session](/docs/kde#headless-session).
- Steam / gamescope: [gamescope](/docs/gamescope) — the host launches its own session per client, so
there's no separate session unit.
Have GDM auto-login your user, so a GNOME Wayland session is always running:
```ini
# /etc/gdm3/custom.conf (Ubuntu) · /etc/gdm/custom.conf (Fedora)
[daemon]
AutomaticLoginEnable = true
AutomaticLogin = your-user
```
Then **disable the screen lock** — a locked GNOME session blocks screen capture, and there's no one to
unlock a headless box:
```sh
gsettings set org.gnome.desktop.screensaver lock-enabled false
gsettings set org.gnome.desktop.session idle-delay 0
```
Enable the host user service (section A) and reboot. The host comes up on the auto-login session.
### Headless KDE
punktfunk ships a unit that brings up a headless KWin/Plasma session with no display manager, so the
host has a desktop to stream even with no monitor attached:
```sh
cp scripts/punktfunk-kde-session.service scripts/punktfunk-host.service ~/.config/systemd/user/
# host.env: PUNKTFUNK_COMPOSITOR=kwin, WAYLAND_DISPLAY=wayland-kde
systemctl --user daemon-reload
systemctl --user enable punktfunk-kde-session punktfunk-host
sudo loginctl enable-linger "$USER"
reboot
```
The session unit starts headless KWin; the host unit follows it and starts listening. (KWin only needs
to be up by the time a client connects, so the ordering is soft.)
Once a session comes up at boot, enable the host user service (section A) and reboot. The host comes up
on that session.
### Headless Bazzite
On Bazzite, the host launches its own gamescope/Steam session per client, so you don't need a separate
session unit — see [Bazzite](/docs/bazzite).
session unit — see [Bazzite](/docs/bazzite) and [gamescope](/docs/gamescope).
## Windows
+7 -15
View File
@@ -86,9 +86,8 @@ When it finishes it prints the web-console URL and how to pair.
By default the host **requires PIN pairing** (secure). Two ways to pair:
- **Web console** (printed at the end of step 2): open `http://<device-ip>:47992`, log in with the
generated password (in `~/.config/punktfunk/web.env`), go to **Devices → arm pairing**, and enter
the PIN on your client.
- **Web console** (printed at the end of step 2): open `http://<device-ip>:47992`,
[arm pairing](/docs/web-console#arm-pairing), and enter the PIN on your client.
- **From the client directly**: pick this host (it advertises over mDNS as `_punktfunk._udp`) and
enter the PIN the host shows.
@@ -96,17 +95,9 @@ On a trusted home LAN you can instead install with `--open` and skip pairing ent
### Console login password
The installer generates a random console login password and writes it to
`~/.config/punktfunk/web.env` (as `PUNKTFUNK_UI_PASSWORD=…`); it's also printed at the end of the
install run (step 2). Read it back with:
```sh
sed -n 's/^PUNKTFUNK_UI_PASSWORD=//p' ~/.config/punktfunk/web.env
```
To set your own password, edit that file and restart the console:
`systemctl --user restart punktfunk-web`. Forgot it? This is the recovery path linked from the
console login screen — see [Forgot your Password?](/docs/forgot-password).
The installer generates a random console login password (printed at the end of step 2) and writes it
to `~/.config/punktfunk/web.env`. To read it back or set your own, see
[The Web Console](/docs/web-console#login-password).
## 4. Verify
@@ -118,7 +109,8 @@ journalctl --user -u punktfunk-host -f # watch a client connect
Connect from a [native client](/docs/clients), or from [Moonlight](/docs/moonlight) (unless you
installed with `--no-gamestream`). In Game Mode the host attaches to the running gamescope session and
streams it at your client's resolution; in Desktop Mode it streams the KDE desktop. The host
auto-detects which session is live per connection.
auto-detects which session is live per connection. See [Steam / gamescope](/docs/gamescope) for the
attach-vs-managed detail and known limits.
## Updating
+68
View File
@@ -0,0 +1,68 @@
---
title: Sway / wlroots
description: Configure a punktfunk host on a wlroots compositor (Sway, Hyprland).
---
The wlroots family can host — but **Sway is the only validated path.** The host adds a per-client
headless output at the client's exact mode and captures it through the xdg-desktop-portal-wlr (xdpw)
ScreenCast portal, injecting input via the wlroots virtual pointer/keyboard protocols. Hyprland and
other wlroots compositors are best-effort (see [How it works](#how-it-works) for the caveat).
This is **not a primary target.** It works and is validated live on **sway 1.11** (zero-copy), but it
sees far less testing than the KDE and GNOME paths — expect rougher edges. If you have a choice,
[KDE](/docs/kde) or [GNOME](/docs/gnome) are the better-exercised desktops.
This page assumes the package is already installed — see [Arch](/docs/arch), [Ubuntu](/docs/ubuntu),
or [Fedora](/docs/fedora).
> New here? Read [Security & Safe Use](/docs/security) first — a streaming host is remote control of
> the machine, so keep it on a trusted LAN or VPN and require pairing.
## host.env
The host auto-detects a wlroots session, so you usually need nothing here. To force the backend, set
these in `~/.config/punktfunk/host.env`:
```ini
PUNKTFUNK_COMPOSITOR=wlroots # aliases: sway, hyprland
PUNKTFUNK_INPUT_BACKEND=wlr
PUNKTFUNK_VIDEO_SOURCE=virtual
# GPU zero-copy capture→encode is ON by default; auto-falls back to CPU. Set PUNKTFUNK_ZEROCOPY=0 to force CPU.
```
See [Configuration](/docs/configuration) for the full reference.
## How it works
- **Video** — the host adds a headless output at the client's exact mode with `swaymsg create_output`.
This uses Sway's IPC specifically; other wlroots compositors (Hyprland, …) don't expose an
equivalent, so virtual-output creation isn't wired up for them yet — Sway is the supported wlroots
path today.
- **Capture** — it captures that output through the **xdg-desktop-portal-wlr (xdpw)** ScreenCast
portal. The host writes a managed chooser config so the output pick is automatic — no interactive
picker dialog to answer.
- **Input** — mouse and keyboard are injected via the wlroots **virtual pointer** and **virtual
keyboard** protocols.
For how long the virtual output lives, and extend-vs-exclusive topology, see
[Virtual displays](/docs/virtual-displays).
## Requirements
- A running wlroots session (Sway, Hyprland, …).
- **xdg-desktop-portal-wlr (xdpw)** installed and running — the host captures through its ScreenCast
portal. Without it there is no video.
## Start the host
With the backend selected, start the host from **inside your Sway session**:
```sh
systemctl --user enable --now punktfunk-host
journalctl --user -u punktfunk-host -f
```
## Bring up the console and pair
Enable the web console, read its login password, and arm PIN pairing — see
[The Web Console](/docs/web-console). Then [connect a client](/docs/clients).

Some files were not shown because too many files have changed in this diff Show More