13 Commits

Author SHA1 Message Date
enricobuehler 3947d5b07a fix(host/audio): drive the Linux virtual mic with RT_PROCESS (was silent)
apple / swift (push) Successful in 1m1s
ci / rust (push) Successful in 4m36s
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 55s
apple / screenshots (push) Successful in 5m9s
ci / bench (push) Successful in 4m36s
windows-host / package (push) Successful in 7m8s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m19s
release / apple (push) Successful in 9m52s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m16s
android / android (push) Successful in 3m21s
decky / build-publish (push) Successful in 11s
deb / build-publish (push) Successful in 2m45s
flatpak / build-publish (push) Successful in 4m11s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 8m48s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m35s
docker / deploy-docs (push) Successful in 18s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 8s
The punktfunk-mic PipeWire source connected without RT_PROCESS, so it ran as an
async/main-loop node. In the host's busy multi-stream graph (desktop audio + video
capture + the session) it never acquired a driver, stayed suspended, and its
process() callback never fired — every recorder reading the remote mic heard pure
silence (the long-standing "Linux host mic broken"). Connect the mic stream with
RT_PROCESS so it is a synchronous node that joins its consumer's driver group and
is actually driven.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 12:46:06 +00:00
enricobuehler 238501597e feat(host/gamestream): follow Desktop<->Game session switches
android / android (push) Successful in 4m49s
ci / web (push) Successful in 55s
apple / swift (push) Successful in 59s
ci / rust (push) Successful in 4m52s
ci / docs-site (push) Successful in 56s
apple / screenshots (push) Successful in 5m16s
windows-host / package (push) Successful in 7m1s
deb / build-publish (push) Successful in 2m30s
decky / build-publish (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 42s
ci / bench (push) Successful in 4m41s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m7s
docker / deploy-docs (push) Successful in 19s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m59s
The GameStream/Moonlight video plane is a separate encode loop that lacked the
session-following the native punktfunk/1 plane has, so a mid-stream Desktop<->Game
switch killed the stream ("video stream failed") instead of following it.

* Normalize the session env like the native plane: extract open_gs_virtual_source,
  which detects the LIVE compositor + apply_session_env/apply_input_env (gamescope
  ATTACH default -> resize-on-attach to the box's own game-mode session at the
  client mode; KWin/Mutter retargeting). GameStream previously ran a bare detect()
  against raw process env, so in game mode it bare-spawned a COMPETING gamescope
  instead of attaching to the box's session.

* In-place capture-loss rebuild: replace the `?` that ended the stream with a
  bounded rebuild (re-detect the live compositor via the same factory, build the
  new source BEFORE dropping the old, reopen the encoder, force an IDR) — keeping
  the send thread + packetizer + socket + RTP clock. A same-resolution
  Desktop<->Game toggle is now FOLLOWED with no Moonlight reconnect.

Protocol limit (unchanged): a mid-stream RESOLUTION change is impossible on
GameStream (WxH locked at ANNOUNCE; no Reconfigure) — a session toggle keeps the
negotiated mode, so this isn't hit. The portal/synthetic source passes no rebuild
closure (propagates as before).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 12:22:12 +00:00
enricobuehler 04dd3e3a19 docs: refresh Windows host page for new users; drop stale Status/NVIDIA-only/SudoVDA
Rewrite the Windows host docs page for first-time setup, on par with the
other host guides: remove the standout "Status:" banner, restructure into
Requirements / Install (web console + pairing + configure) / How it works /
Notes & limits.

Bring the content up to date with the shipping host:
- encode is all-vendor (NVENC/AMF/QSV + software fallback), not NVIDIA-only
- virtual display is punktfunk's own pf-vdisplay IDD (SudoVDA removed)
- gamepads need no prerequisite — UMDF drivers bundled; ViGEmBus is gone
- add HDR10 + Vulkan-game HDR layer coverage

Fix the same stale claims where other pages cross-reference the Windows host
(requirements, running-as-a-service, install, roadmap, status).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:22:50 +00:00
enricobuehler 61aa1053e7 feat(host/gamescope): headless game mode that follows the box + matches the client
apple / swift (push) Successful in 1m2s
android / android (push) Successful in 4m43s
ci / rust (push) Successful in 4m53s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
apple / screenshots (push) Successful in 5m6s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
windows-host / package (push) Successful in 9m2s
ci / bench (push) Successful in 4m41s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m6s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m43s
Make Steam game mode work on a display-less streaming host and stream it at the
client's resolution:

* Ship /etc/gamescope-session-plus/sessions.d/steam (packaging/bazzite/
  gamescope-headless-session, installed by the RPM + Arch PKGBUILD): fall back to
  gamescope's headless backend when no display is connected, so "Switch to Game
  Mode" boots offscreen instead of crashing on the missing panel (and 5-striking
  back to desktop). No-op on display-attached boxes; only sets unset values so
  the host's per-client mode still wins.

* Default Bazzite/SteamOS to ATTACH (PUNKTFUNK_GAMESCOPE_ATTACH=1 in host.env):
  the box owns its session (Desktop<->Game, persistent), the host follows +
  captures it and never tears it down — so switching is rock-solid and a
  disconnect leaves the box in its mode (reconnect returns there).

* Resize-on-attach (gamescope.rs): on connect, ensure the box's own game-mode
  session runs at the CLIENT's resolution — reuse it when already matching (fast
  path, no restart), else reconfigure + restart the box's own autologin
  gamescope-session-plus@<client> at the client mode (cooperative: no competing
  unit, so no autologin-respawn fight). Detect the live gamescope's -W/-H via
  argv[0] in /proc (its /proc/<pid>/exe is unreadable for that process).

Validated live on a headless bazzite-deck-nvidia box: game mode boots headless +
stable (0 strikes); the host attaches + streams video/audio/EIS input; a
5120x1440 client reuses the matching session and streams at 5120x1440.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 11:09:45 +00:00
enricobuehler 50e17b3508 fix(host/capture): hold the session through a slow compositor switch
apple / swift (push) Successful in 1m1s
ci / docs-site (push) Successful in 54s
apple / screenshots (push) Successful in 5m14s
deb / build-publish (push) Successful in 2m30s
decky / build-publish (push) Successful in 11s
android / android (push) Successful in 4m41s
ci / rust (push) Successful in 4m52s
ci / web (push) Successful in 49s
windows-host / package (push) Successful in 7m54s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m34s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m10s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m7s
A Bazzite/SteamOS Gaming↔Desktop switch tears the old compositor down and can
take 15s+ to bring the new one up — longer than the capture-loss rebuild's
~10s window, so the session failed mid-switch ("disconnect — session failed")
and forced the client to cold-reconnect. Retry the rebuild within a 40s budget
instead of giving up after one round, and re-detect the live compositor on
each attempt so the stream follows the box to whatever session comes up (a new
instance of the same compositor, or a different one — the kind-change case).
The QUIC keepalive runs on its own thread, so the client stays connected
(frozen on the last frame) and the stream resumes when the new output appears,
with no reconnect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 09:31:47 +00:00
enricobuehler 94c556f0e3 fix(host/capture): recover from compositor loss instead of freezing
apple / screenshots (push) Successful in 5m7s
apple / swift (push) Successful in 1m1s
windows-host / package (push) Successful in 7m26s
android / android (push) Successful in 4m50s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 54s
decky / build-publish (push) Successful in 11s
ci / rust (push) Successful in 4m51s
deb / build-publish (push) Successful in 2m29s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 5s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m37s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m1s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m47s
When the compositor is torn down mid-stream (a Gaming↔Desktop switch removes
the virtual output), its PipeWire stream leaves Streaming for Paused rather
than disconnecting. try_latest treated that as Ok(None) ("static desktop —
repeat the last frame"), so the stream froze on the last frame forever and
neither recovery path fired: the capture-loss rebuild keys on Err, and the
session watcher keys on a session-KIND change (a desktop→desktop new KWin
instance is the same kind).

Track the PipeWire stream state via state_changed (a `streaming` flag) and,
in try_latest, surface a sustained non-Streaming state (1.5s grace for a
transient renegotiation blip) as a capture-loss Err — which the encode loop
already handles by rebuilding the pipeline in place. A static desktop stays
Streaming, so no false trigger. Complements the now-default session watcher.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 09:00:35 +00:00
enricobuehler 32c1929948 feat(host/session-watch): default Gaming↔Desktop follow on for Bazzite/SteamOS
apple / swift (push) Successful in 1m2s
android / android (push) Successful in 4m52s
ci / rust (push) Successful in 5m3s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 54s
decky / build-publish (push) Successful in 22s
windows-host / package (push) Successful in 9m7s
ci / bench (push) Successful in 4m40s
apple / screenshots (push) Successful in 5m20s
deb / build-publish (push) Successful in 2m31s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 32s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m40s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m39s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m24s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 47s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m19s
docker / deploy-docs (push) Successful in 22s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m29s
The mid-stream session watcher (rebuild the backend in place when the box
flips Gaming↔Desktop) was opt-in via PUNKTFUNK_SESSION_WATCH, so it never
ran on a stock Bazzite/SteamOS box — switching modes froze the stream on the
now-dead compositor. Default it ON when os-release ID/ID_LIKE is
bazzite/steamos (the platforms that flip sessions); still off on plain
desktops. Also parse the env properly so PUNKTFUNK_SESSION_WATCH=0 actually
disables it (was: any value, including "0", enabled it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 08:43:27 +00:00
enricobuehler 3915a82780 fix(host/input): route KWin auto-detect to the fake_input backend
apple / swift (push) Successful in 1m1s
apple / screenshots (push) Successful in 5m2s
windows-host / package (push) Successful in 6m56s
android / android (push) Successful in 4m42s
ci / rust (push) Successful in 4m52s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 56s
deb / build-publish (push) Successful in 2m31s
decky / build-publish (push) Successful in 13s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 4s
ci / bench (push) Successful in 4m29s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m4s
docker / deploy-docs (push) Successful in 6s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m0s
apply_input_env() hard-pinned PUNKTFUNK_INPUT_BACKEND=libei for KWin, and
default_backend() reads that env first — so the auto-detecting host (the
normal `serve` service) ignored the new KwinFakeInput backend and fell back
to the RemoteDesktop portal path that needs a user to approve. Route KWin to
"kwin" (org_kde_kwin_fake_input); GNOME/Mutter stay on libei (no fake_input
there).

Validated live on a Bazzite KDE box via the auto-detect path:
backend=KwinFakeInput, "KWin fake_input ready (no portal)", input events
forwarded with no errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:52:02 +00:00
enricobuehler a4833e4780 feat(android/touch): trackpad-relative cursor (default), with a direct-touch toggle
apple / swift (push) Successful in 1m10s
android / android (push) Successful in 4m53s
ci / rust (push) Successful in 5m1s
ci / web (push) Successful in 58s
ci / docs-site (push) Successful in 55s
apple / screenshots (push) Successful in 5m28s
deb / build-publish (push) Successful in 2m30s
windows-host / package (push) Successful in 8m41s
decky / build-publish (push) Successful in 29s
ci / bench (push) Successful in 4m27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 2m43s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 2m35s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m25s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 48s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 9m46s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m1s
docker / deploy-docs (push) Successful in 24s
One-finger touch was absolute "direct pointing" — the host cursor jumped to the
finger and was recomputed from each touch-start, so you couldn't precisely reach a
target. Now a relative trackpad: the cursor stays put on touch-down and moves by the
finger delta (host MouseMove via nativeSendPointerMove, already supported — no
protocol change), with mild pointer acceleration and sub-pixel remainder
accumulation so slow precise moves aren't lost to Int truncation. Swipe, lift, and
re-swipe to walk it across; tap = left-click at the cursor's current position.
Two-finger scroll / right-click, three-finger HUD toggle, and tap-then-hold-drag are
preserved unchanged; finger-id re-anchoring keeps multi-touch transitions jump-free.

Added Settings → Pointer → "Trackpad mode" (default on); turning it off restores the
old direct-pointing path verbatim.

:app:compileDebugKotlin green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:34:03 +00:00
enricobuehler 4e79e6cdad fix(android/audio): kill the AAudio crackle (RT-safe ring + deeper buffer + XRun sizing)
The jitter ring was a port of the Linux client's, but Linux runs on PipeWire
(adaptive resampling masks host↔DAC drift + a shallow buffer); AAudio hands us a
raw realtime callback and we own the buffer, so the same code crackled only on
Android. Three converging causes, all fixed:

- Heap free on the realtime audio thread every quantum (Android's Scudo free() has
  unbounded tail latency → XRun → click). Decoded buffers are now recycled back to
  the producer via a free-list instead of freed on the audio thread; the ring is
  pre-reserved so extend() never reallocates there.
- The ring collapsed to ~15 ms on the tiny LowLatency burst and re-primed (a fresh
  silence) on every single empty callback. Now ~40 ms prime / ~150 ms hard cap,
  decoupled from the burst size, with de-prime hysteresis (re-prime only after a
  sustained drain).
- AAudio's anti-glitch knobs were unused: prime the HW buffer above its 2-burst
  default and grow it on getXRunCount(). The post-open log now reports
  perf/sharing/buffer so a fall to a resampled legacy path is visible.

Steady-state audio latency ~15 → ~40 ms (within lip-sync tolerance; matches the
Moonlight/Sunshine operating point). cargo-ndk build both ABIs + fmt + clippy green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:33:51 +00:00
enricobuehler f74bc4a3f1 feat(host/input): headless KDE input via org_kde_kwin_fake_input
Desktop-mode (KWin) streaming had no input: the path was libei via the
RemoteDesktop portal, which (a) isn't reachable from the host service env
and (b) requires a human to approve "Allow remote control?" — a
non-starter on a headless box. KWin's own headless RDP server (krdpserver)
solves this with org_kde_kwin_fake_input, authorized by the exact same
.desktop X-KDE-Wayland-Interfaces grant we already ship
(org_kde_kwin_fake_input is listed alongside zkde_screencast_unstable_v1).

Add a fake_input injector: vendor the protocol XML, bind the global as an
ordinary Wayland client, authenticate (auto-accepted for an
interface-authorized client — no dialog), and translate pointer (rel/abs),
button, scroll, keyboard (raw evdev keycodes resolved by KWin's own keymap)
and touch. Select it for KWin (compositor=="kwin" or XDG_CURRENT_DESKTOP
KDE); GNOME stays on libei (it has neither fake_input nor the wlr
protocols). PUNKTFUNK_INPUT_BACKEND=kwin forces it.

cargo check + clippy + fmt green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:26:04 +00:00
enricobuehler 8e18d01af5 fix(host/kwin): authorize Desktop-mode streaming via a shipped .desktop
Streaming the KDE *Desktop* (KWin) session failed on a real interactive
Plasma session with "KWin does not expose zkde_screencast_unstable_v1":
KWin treats the screencast/virtual-output and fake_input globals as
restricted and advertises them only to a client whose installed .desktop
lists them under X-KDE-Wayland-Interfaces (matched by /proc/<pid>/exe ->
Exec, and cached per-executable on first connect). The host shipped no
.desktop, so it was permanently denied; it only ever worked on the
headless dev box via KWIN_WAYLAND_NO_PERMISSION_CHECKS=1.

Ship packaging/linux/io.unom.Punktfunk.Host.desktop (least-privilege:
only the host, only zkde_screencast_unstable_v1 + org_kde_kwin_fake_input)
and install it from the RPM/.deb/Arch host packaging so it is present
before the host first connects. Drop the blunt session-wide
NO_PERMISSION_CHECKS hack from kde-desktop-setup.sh (it now only seeds the
RemoteDesktop input grant) and fix the now-misleading kwin.rs docs/errors.

Validated live on a Bazzite Kinoite box (KWin 6.6.4): probe-compositor +
spike --source kwin-virtual succeed against a KWin running WITHOUT the
permission bypass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 11:15:39 +00:00
enricobuehler 3477cbe7ce fix(audio/windows): stop the client mic echoing back through the loopback
The Windows virtual mic fakes a capture endpoint by writing the client's
uplinked PCM into a virtual device's *render* endpoint, while the
desktop-audio plane loopback-captures the *default render* endpoint — with
no mutual exclusion between the two. WASAPI loopback captures the mixed
output of an endpoint (everything any app renders to it, including our mic
writes), so when both resolve to the same device — VB-CABLE used for both,
or the auto-installed Steam Streaming Microphone being the default render on
a headless box — the injected mic is captured straight back into the
host->client audio stream: an infinite echo.

find_device() now resolves the loopback's endpoint id (default render) and
skips any candidate matching it, scanning on to the next non-loopback match,
so the mic can never land on the device the loopback reads. The auto-install
path now provisions the full Steam pair (Streaming Microphone + Streaming
Speakers) so a bare host gets two distinct devices instead of one shared
one. Errors distinguish "no device" from "only candidate is the loopback
device". Linux was already immune (its mic is a dedicated Audio/Source node,
structurally separate from the monitored sink).

Windows-only (#[cfg(windows)]); rustfmt-clean, compile-checked in
windows-host CI, needs on-glass validation on the RTX box. Does not force
the system default playback onto Steam Streaming Speakers (IPolicyConfig) —
not required to break the echo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 23:51:46 +00:00
29 changed files with 1329 additions and 212 deletions
@@ -19,6 +19,12 @@ data class Settings(
val micEnabled: Boolean = false, val micEnabled: Boolean = false,
/** Show the live stats overlay (FPS / throughput / latency) during a stream. */ /** Show the live stats overlay (FPS / throughput / latency) during a stream. */
val statsHudEnabled: Boolean = true, val statsHudEnabled: Boolean = true,
/**
* Touch input model. `true` (default) = trackpad: the cursor stays put on touch-down and moves
* by the finger's relative delta (swipe to nudge, lift and re-swipe to walk it across), tap to
* click where it is. `false` = direct pointing: the cursor jumps to the finger (the old behaviour).
*/
val trackpadMode: Boolean = true,
) )
/** Loads/saves [Settings] in the app-private `punktfunk_settings` prefs. */ /** Loads/saves [Settings] in the app-private `punktfunk_settings` prefs. */
@@ -35,6 +41,7 @@ class SettingsStore(context: Context) {
gamepad = prefs.getInt(K_GAMEPAD, 0), gamepad = prefs.getInt(K_GAMEPAD, 0),
micEnabled = prefs.getBoolean(K_MIC, false), micEnabled = prefs.getBoolean(K_MIC, false),
statsHudEnabled = prefs.getBoolean(K_HUD, true), statsHudEnabled = prefs.getBoolean(K_HUD, true),
trackpadMode = prefs.getBoolean(K_TRACKPAD, true),
) )
fun save(s: Settings) { fun save(s: Settings) {
@@ -47,6 +54,7 @@ class SettingsStore(context: Context) {
.putInt(K_GAMEPAD, s.gamepad) .putInt(K_GAMEPAD, s.gamepad)
.putBoolean(K_MIC, s.micEnabled) .putBoolean(K_MIC, s.micEnabled)
.putBoolean(K_HUD, s.statsHudEnabled) .putBoolean(K_HUD, s.statsHudEnabled)
.putBoolean(K_TRACKPAD, s.trackpadMode)
.apply() .apply()
} }
@@ -59,6 +67,7 @@ class SettingsStore(context: Context) {
const val K_GAMEPAD = "gamepad" const val K_GAMEPAD = "gamepad"
const val K_MIC = "mic_enabled" const val K_MIC = "mic_enabled"
const val K_HUD = "stats_hud_enabled" const val K_HUD = "stats_hud_enabled"
const val K_TRACKPAD = "trackpad_mode"
} }
} }
@@ -119,6 +119,16 @@ fun SettingsScreen(initial: Settings, onChange: (Settings) -> Unit, onBack: () -
) )
} }
SettingsGroup("Pointer") {
ToggleRow(
title = "Trackpad mode",
subtitle = "Relative cursor like a laptop touchpad — swipe to nudge, tap to click. " +
"Off = the cursor jumps to your finger.",
checked = s.trackpadMode,
onCheckedChange = { on -> update(s.copy(trackpadMode = on)) },
)
}
SettingsGroup("Overlay") { SettingsGroup("Overlay") {
ToggleRow( ToggleRow(
title = "Stats overlay", title = "Stats overlay",
@@ -41,6 +41,7 @@ import io.unom.punktfunk.kit.NativeBridge
import java.util.concurrent.atomic.AtomicBoolean import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.coroutines.delay import kotlinx.coroutines.delay
import kotlin.math.abs import kotlin.math.abs
import kotlin.math.hypot
import kotlin.math.roundToInt import kotlin.math.roundToInt
// Touch-gesture tuning (px / ms). TAP_SLOP: movement under this still counts as a tap, not a drag. // Touch-gesture tuning (px / ms). TAP_SLOP: movement under this still counts as a tap, not a drag.
@@ -50,6 +51,15 @@ private const val TAP_SLOP = 12f
private const val TAP_DRAG_MS = 250L private const val TAP_DRAG_MS = 250L
private const val SCROLL_DIV = 4f private const val SCROLL_DIV = 4f
// Trackpad-mode pointer ballistics (relative one-finger motion). POINTER_SENS: base finger-px →
// host-px gain (~1:1, never twitchy). The rest is mild acceleration so a flick crosses the screen
// while a slow drag stays precise: above ACCEL_SPEED_FLOOR px/ms the gain ramps by ACCEL_GAIN per
// px/ms, capped at ACCEL_MAX (so a fast swipe can't fling the cursor uncontrollably).
private const val POINTER_SENS = 1.3f
private const val ACCEL_GAIN = 0.6f
private const val ACCEL_SPEED_FLOOR = 0.3f
private const val ACCEL_MAX = 3.0f
@Composable @Composable
fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) { fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val context = LocalContext.current val context = LocalContext.current
@@ -68,8 +78,11 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Live decode stats for the HUD. Poll once a second for the whole stream (cheap, and each call // Live decode stats for the HUD. Poll once a second for the whole stream (cheap, and each call
// drains+resets the native window so it never grows unbounded even while the overlay is hidden); // drains+resets the native window so it never grows unbounded even while the overlay is hidden);
// `showStats` only gates rendering. A 3-finger tap toggles it live; the default comes from Settings. // `showStats` only gates rendering. A 3-finger tap toggles it live; the default comes from Settings.
val initialSettings = remember { SettingsStore(context).load() }
var stats by remember { mutableStateOf<DoubleArray?>(null) } var stats by remember { mutableStateOf<DoubleArray?>(null) }
var showStats by remember { mutableStateOf(SettingsStore(context).load().statsHudEnabled) } var showStats by remember { mutableStateOf(initialSettings.statsHudEnabled) }
// Touch model is fixed per session (re-keys the gesture handler below if it ever changes).
val trackpad = initialSettings.trackpadMode
LaunchedEffect(handle) { LaunchedEffect(handle) {
while (true) { while (true) {
delay(1000) delay(1000)
@@ -145,13 +158,18 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
if (showStats) { if (showStats) {
stats?.let { StatsOverlay(it, Modifier.align(Alignment.TopStart).padding(12.dp)) } stats?.let { StatsOverlay(it, Modifier.align(Alignment.TopStart).padding(12.dp)) }
} }
// Touch → mouse, absolute "direct pointing" like the Apple client: the host cursor follows // Touch → mouse. Two models, chosen by the Trackpad-mode setting:
// your finger (MouseMoveAbs, host-normalized against the overlay size — which fills the video, // • trackpad (default): the cursor STAYS where it is on touch-down and moves by the finger's
// so finger position maps straight onto the remote screen). Gestures: tap = left click; // relative delta (MouseMove) with mild pointer acceleration — swipe to nudge, lift and
// two-finger tap = right click; two-finger drag = scroll; tap-then-press-and-drag = left-drag // re-swipe to walk it across, tap to click where it is. This is what makes the cursor
// (text selection / moving windows); three-finger tap = toggle the stats HUD. // reachable on a small screen.
// • direct (opt-out): the cursor jumps to the finger and follows it (MouseMoveAbs,
// host-normalized against the overlay size), the old "direct pointing" behaviour.
// Both share the same gesture vocabulary: tap = left click; two-finger tap = right click;
// two-finger drag = scroll; tap-then-press-and-drag = left-drag (text selection / moving
// windows); three-finger tap = toggle the stats HUD.
Box( Box(
Modifier.fillMaxSize().pointerInput(handle) { Modifier.fillMaxSize().pointerInput(handle, trackpad) {
var lastTapUp = 0L var lastTapUp = 0L
var lastTapX = 0f var lastTapX = 0f
var lastTapY = 0f var lastTapY = 0f
@@ -176,7 +194,9 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
val isDrag = down.uptimeMillis - lastTapUp < TAP_DRAG_MS && val isDrag = down.uptimeMillis - lastTapUp < TAP_DRAG_MS &&
abs(startX - lastTapX) < TAP_SLOP && abs(startY - lastTapY) < TAP_SLOP abs(startX - lastTapX) < TAP_SLOP && abs(startY - lastTapY) < TAP_SLOP
lastTapUp = 0L // consume the arming either way lastTapUp = 0L // consume the arming either way
moveAbs(startX, startY) // cursor jumps to the finger immediately // Direct mode jumps the cursor to the finger; trackpad mode leaves it put (the
// whole point — you nudge it with swipes instead).
if (!trackpad) moveAbs(startX, startY)
if (isDrag) NativeBridge.nativeSendPointerButton(handle, 1, true) if (isDrag) NativeBridge.nativeSendPointerButton(handle, 1, true)
var moved = false var moved = false
@@ -185,6 +205,14 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
var prevCx = startX var prevCx = startX
var prevCy = startY var prevCy = startY
var upTime = down.uptimeMillis var upTime = down.uptimeMillis
// Trackpad relative-motion state: the tracked finger, its last position/time, and
// the sub-pixel remainder so a slow drag isn't lost to Int truncation.
var trackId = down.id
var prevX = startX
var prevY = startY
var prevT = down.uptimeMillis
var accX = 0f
var accY = 0f
while (true) { while (true) {
val ev = awaitPointerEvent() val ev = awaitPointerEvent()
@@ -217,15 +245,46 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
moved = true moved = true
} }
} else if (!scrolling) { } else if (!scrolling) {
// One finger → the cursor follows it (skipped once a gesture turned into // One finger (skipped once a gesture turned into a scroll, so dropping
// a scroll, so dropping back to one finger doesn't jerk the cursor). // back to one finger doesn't jerk the cursor).
val p = pressed.firstOrNull { it.id == down.id } ?: pressed.first() val p = pressed.firstOrNull { it.id == down.id } ?: pressed.first()
if (abs(p.position.x - startX) > TAP_SLOP || if (abs(p.position.x - startX) > TAP_SLOP ||
abs(p.position.y - startY) > TAP_SLOP abs(p.position.y - startY) > TAP_SLOP
) { ) {
moved = true moved = true
} }
moveAbs(p.position.x, p.position.y) if (trackpad) {
// Relative: move by the finger delta × (sensitivity × acceleration),
// carrying the sub-pixel remainder. Re-anchor (zero delta this frame)
// if the tracked finger changed, so lifting one of several fingers
// never jumps the cursor.
if (p.id != trackId) {
trackId = p.id
prevX = p.position.x
prevY = p.position.y
prevT = p.uptimeMillis
}
val dx = p.position.x - prevX
val dy = p.position.y - prevY
val dt = (p.uptimeMillis - prevT).coerceAtLeast(1L)
prevX = p.position.x
prevY = p.position.y
prevT = p.uptimeMillis
val speed = hypot(dx, dy) / dt // finger px per ms
val accel = (1f + ACCEL_GAIN * (speed - ACCEL_SPEED_FLOOR).coerceAtLeast(0f))
.coerceAtMost(ACCEL_MAX)
accX += dx * POINTER_SENS * accel
accY += dy * POINTER_SENS * accel
val outX = accX.toInt() // truncates toward zero → remainder kept w/ sign
val outY = accY.toInt()
if (outX != 0 || outY != 0) {
NativeBridge.nativeSendPointerMove(handle, outX, outY)
accX -= outX
accY -= outY
}
} else {
moveAbs(p.position.x, p.position.y) // direct: cursor follows the finger
}
} }
ev.changes.forEach { it.consume() } ev.changes.forEach { it.consume() }
} }
@@ -239,7 +298,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
NativeBridge.nativeSendPointerButton(handle, 3, true) NativeBridge.nativeSendPointerButton(handle, 3, true)
NativeBridge.nativeSendPointerButton(handle, 3, false) NativeBridge.nativeSendPointerButton(handle, 3, false)
} }
else -> { // tap → left click, and arm tap-and-drag else -> { // tap → left click (at the cursor's current spot), arm tap-drag
NativeBridge.nativeSendPointerButton(handle, 1, true) NativeBridge.nativeSendPointerButton(handle, 1, true)
NativeBridge.nativeSendPointerButton(handle, 1, false) NativeBridge.nativeSendPointerButton(handle, 1, false)
lastTapUp = upTime lastTapUp = upTime
+117 -15
View File
@@ -1,8 +1,17 @@
//! Android audio playback (android-only): pull Opus packets from the connector, decode to //! Android audio playback (android-only): pull Opus packets from the connector, decode to
//! interleaved f32 stereo, and feed AAudio (LowLatency) via its realtime data callback through a //! interleaved f32 stereo, and feed AAudio (LowLatency) via its realtime data callback through a
//! jitter ring. Mirrors [`crate::decode`]: one thread we own (the Opus decode producer) plus a //! jitter ring. Mirrors [`crate::decode`]: one thread we own (the Opus decode producer) plus a
//! shutdown flag; the realtime callback thread is owned by AAudio. Ring logic ported from //! shutdown flag; the realtime callback thread is owned by AAudio.
//! `punktfunk-client-linux/src/audio.rs` (prime ~3 quanta, drop-oldest cap, re-prime on drain). //!
//! The ring started as a port of `punktfunk-client-linux/src/audio.rs`, but AAudio — unlike
//! PipeWire, which adaptively rate-matches the stream and absorbs a shallow buffer — hands us a raw
//! realtime callback and makes us own the buffer. So this client diverges deliberately to stop the
//! Android-only crackle: (1) the callback is allocation/free-free — decoded buffers are recycled to
//! the producer via a free-list instead of being freed on the audio thread (Android's Scudo `free`
//! has unbounded tail latency); (2) the jitter ring is deeper (~40 ms prime / ~150 ms hard cap) and
//! decoupled from the tiny LowLatency burst size, with de-prime hysteresis so a transient drain
//! doesn't manufacture a silence; (3) the AAudio HW buffer is primed above its 2-burst default and
//! grown on XRuns (Google's anti-glitch technique).
use ndk::audio::{ use ndk::audio::{
AudioCallbackResult, AudioDirection, AudioFormat, AudioPerformanceMode, AudioSharingMode, AudioCallbackResult, AudioDirection, AudioFormat, AudioPerformanceMode, AudioSharingMode,
@@ -13,7 +22,7 @@ use punktfunk_core::error::PunktfunkError;
use std::collections::VecDeque; use std::collections::VecDeque;
use std::ffi::c_void; use std::ffi::c_void;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::mpsc::{sync_channel, SyncSender, TrySendError}; use std::sync::mpsc::{sync_channel, Receiver, SyncSender, TrySendError};
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::Duration;
@@ -24,6 +33,29 @@ const RING_CHUNKS: usize = 64;
/// Opus decode scratch: worst-case 120 ms stereo frame (5760 samples/ch × 2 ch). /// Opus decode scratch: worst-case 120 ms stereo frame (5760 samples/ch × 2 ch).
const PCM_SCRATCH: usize = 5760 * CHANNELS; const PCM_SCRATCH: usize = 5760 * CHANNELS;
// --- Jitter-ring depths, in interleaved-f32 samples (all expressed in ms via `MS`). -----------
// Unlike the Linux client (PipeWire adaptively rate-matches the stream to the graph clock, masking
// host↔DAC drift + a shallow ring), AAudio hands us a raw callback and we own the buffer: drift and
// WiFi power-save bunching land as underruns/overflows = crackle. So Android runs a deliberately
// deeper, smoothly-managed ring than Linux — keep the two clients' depths intentionally divergent.
/// Interleaved f32 samples per millisecond (48 kHz × 2 ch).
const MS: usize = (SAMPLE_RATE as usize / 1000) * CHANNELS; // 96
/// Prime/target floor: fill to ~40 ms before playing (and after a sustained drain). Deep enough to
/// ride out WiFi arrival jitter + clock drift; the dominant Android-only anti-crackle lever.
const PRIME_FLOOR: usize = 40 * MS;
/// Ceiling for the burst-scaled target (so a large quantum can't push the prime depth too high).
const PRIME_CEIL: usize = 80 * MS;
/// Drop-oldest headroom above the target before trimming — a ~80 ms band swallows an arrival burst
/// without overflowing.
const JITTER_HEADROOM: usize = 80 * MS;
/// Hard latency bound: never let the ring exceed ~150 ms (the only thing that caps added latency).
const HARD_CAP: usize = 150 * MS;
/// Re-prime (go silent to refill) only after this many CONSECUTIVE empty callbacks, so one transient
/// drain doesn't manufacture a fresh 40 ms silence (the old `if ring.is_empty()` re-primed instantly).
const DEPRIME_AFTER_CALLBACKS: u32 = 5;
/// Throttle the AAudio XRun-driven HW-buffer grow check (cheap, but no need to poll every quantum).
const XRUN_CHECK_EVERY: u32 = 128;
/// Diagnostics — written by the decode thread + the realtime callback, logged periodically. The /// Diagnostics — written by the decode thread + the realtime callback, logged periodically. The
/// audio analogue of the video `fed`/`rendered` counters (we can't "screenshot" sound). /// audio analogue of the video `fed`/`rendered` counters (we can't "screenshot" sound).
#[derive(Default)] #[derive(Default)]
@@ -47,22 +79,41 @@ impl AudioPlayback {
pub fn start(client: Arc<NativeClient>) -> Option<AudioPlayback> { pub fn start(client: Arc<NativeClient>) -> Option<AudioPlayback> {
let counters = Arc::new(Counters::default()); let counters = Arc::new(Counters::default());
let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS); let (tx, rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Recycle free-list: drained PCM buffers go BACK to the decode thread to be refilled, so the
// realtime callback never frees heap (Android's Scudo allocator has unbounded free() tail
// latency — a free on the audio thread is an XRun = a click) and the decode thread rarely
// allocates. Same depth as the data channel.
let (free_tx, free_rx) = sync_channel::<Vec<f32>>(RING_CHUNKS);
// Realtime consumer state, owned by the callback (FnMut) — no lock: AAudio calls it from a // Realtime consumer state, owned by the callback (FnMut) — no lock: AAudio calls it from a
// single high-priority thread, and the decode thread only touches `tx`. // single high-priority thread, and the decode thread only touches `tx`/`free_rx`.
let cb_counters = counters.clone(); let cb_counters = counters.clone();
let mut ring: VecDeque<f32> = VecDeque::with_capacity(PCM_SCRATCH); // Pre-reserve the ring so `extend` never reallocates on the realtime thread. Worst transient
// before the trim below = the hard cap plus one full channel of 5 ms (480-f32) frames — the
// punktfunk protocol always sends 5 ms Opus frames (host `audio_thread`); a larger frame
// would force a one-time realloc, asserted (not silently corrupted) in `decode_loop`.
let mut ring: VecDeque<f32> = VecDeque::with_capacity(HARD_CAP + RING_CHUNKS * 5 * MS);
let mut primed = false; let mut primed = false;
let callback = move |_s: &AudioStream, data: *mut c_void, num_frames: i32| { let mut empties: u32 = 0; // consecutive empty callbacks (de-prime hysteresis)
let mut cb_count: u32 = 0; // callbacks since open (throttles the XRun grow check)
let mut last_xrun: i32 = 0; // last AAudio XRun count we grew the buffer for
let callback = move |s: &AudioStream, data: *mut c_void, num_frames: i32| {
let want = num_frames as usize * CHANNELS; let want = num_frames as usize * CHANNELS;
// SAFETY: AAudio provides `num_frames * channel_count` F32 slots at `data`. // SAFETY: AAudio provides `num_frames * channel_count` F32 slots at `data`.
let out = unsafe { std::slice::from_raw_parts_mut(data as *mut f32, want) }; let out = unsafe { std::slice::from_raw_parts_mut(data as *mut f32, want) };
while let Ok(chunk) = rx.try_recv() { // Drain decoded chunks into the ring WITHOUT freeing on the RT thread: `drain(..)` empties
ring.extend(chunk); // each Vec but keeps its capacity, then the empty buffer is handed back for reuse. The
// only RT-thread free is the rare case where the recycle channel is momentarily full.
while let Ok(mut chunk) = rx.try_recv() {
ring.extend(chunk.drain(..));
let _ = free_tx.try_send(chunk);
} }
// Prime to ~3 quanta (15 ms; floor 15 ms / ceiling 200 ms); drop OLDEST above the cap. // Jitter buffer: prime to ~40 ms (PRIME_FLOOR) before playing and after a sustained drain;
let target = (3 * want).clamp(720 * CHANNELS, 9600 * CHANNELS); // drop-oldest only above a wide ~120 ms band. Decoupled from the AAudio burst `want` (tiny
while ring.len() > target.max(want) + want { // on the LowLatency MMAP path) so the depth doesn't collapse to a single quantum.
let target = (3 * want).clamp(PRIME_FLOOR, PRIME_CEIL);
let hard_cap = (target + JITTER_HEADROOM).min(HARD_CAP);
while ring.len() > hard_cap {
ring.pop_front(); ring.pop_front();
} }
if !primed && ring.len() >= target { if !primed && ring.len() >= target {
@@ -79,12 +130,34 @@ impl AudioPlayback {
out.fill(0.0); out.fill(0.0);
cb_counters.underruns.fetch_add(1, Ordering::Relaxed); cb_counters.underruns.fetch_add(1, Ordering::Relaxed);
} }
// Re-prime only after a RUN of empty callbacks, not a single transient one — otherwise
// every momentary drain costs a fresh 40 ms silence (the old behaviour, self-inflicted
// crackle on any jitter spike).
if ring.is_empty() { if ring.is_empty() {
primed = false; // re-prime after a genuine drain (avoids sustained crackle on loss) empties += 1;
if empties >= DEPRIME_AFTER_CALLBACKS {
primed = false;
}
} else {
empties = 0;
} }
cb_counters cb_counters
.ring_depth .ring_depth
.store(ring.len() as u64, Ordering::Relaxed); .store(ring.len() as u64, Ordering::Relaxed);
// Google's AAudio anti-glitch technique: when the device reports new XRuns, grow the HW
// buffer by one burst (up to capacity). getXRunCount + setBufferSizeInFrames are both
// callback-safe / non-blocking, and set clamps to capacity so it self-limits. Throttled.
cb_count = cb_count.wrapping_add(1);
if cb_count % XRUN_CHECK_EVERY == 0 {
let xr = s.x_run_count();
if xr > last_xrun {
last_xrun = xr;
let burst = s.frames_per_burst().max(1);
let grown =
(s.buffer_size_in_frames() + burst).min(s.buffer_capacity_in_frames());
let _ = s.set_buffer_size_in_frames(grown);
}
}
AudioCallbackResult::Continue AudioCallbackResult::Continue
}; };
@@ -109,19 +182,31 @@ impl AudioPlayback {
log::error!("audio: request_start: {e}"); log::error!("audio: request_start: {e}");
return None; return None;
} }
// Lift the AAudio HW buffer off its brittle ~2-burst LowLatency default so a single late
// callback doesn't immediately underrun; the in-callback XRun loop grows it further if the
// device still glitches. set_buffer_size_in_frames clamps to capacity.
let burst = stream.frames_per_burst().max(1);
let _ =
stream.set_buffer_size_in_frames((burst * 3).min(stream.buffer_capacity_in_frames()));
// perf != LowLatency or rate != 48000 means AAudio silently fell to a resampled legacy path
// (different burst behaviour) — surface it so the field can tell that apart from plain jitter.
log::info!( log::info!(
"audio: AAudio started rate={} ch={} fmt={:?} burst={}", "audio: AAudio started rate={} ch={} fmt={:?} perf={:?} share={:?} burst={} buf={}/{}",
stream.sample_rate(), stream.sample_rate(),
stream.channel_count(), stream.channel_count(),
stream.format(), stream.format(),
stream.performance_mode(),
stream.sharing_mode(),
stream.frames_per_burst(), stream.frames_per_burst(),
stream.buffer_size_in_frames(),
stream.buffer_capacity_in_frames(),
); );
let shutdown = Arc::new(AtomicBool::new(false)); let shutdown = Arc::new(AtomicBool::new(false));
let sd = shutdown.clone(); let sd = shutdown.clone();
let join = std::thread::Builder::new() let join = std::thread::Builder::new()
.name("pf-audio".into()) .name("pf-audio".into())
.spawn(move || decode_loop(client, tx, sd, counters)) .spawn(move || decode_loop(client, tx, free_rx, sd, counters))
.ok(); .ok();
Some(AudioPlayback { Some(AudioPlayback {
@@ -143,9 +228,12 @@ impl Drop for AudioPlayback {
} }
/// Producer: `next_audio` → Opus `decode_float` → push interleaved f32 into the ring channel. /// Producer: `next_audio` → Opus `decode_float` → push interleaved f32 into the ring channel.
/// Buffers come from (and return to) the realtime callback's recycle free-list so the steady state
/// is allocation-free on both threads.
fn decode_loop( fn decode_loop(
client: Arc<NativeClient>, client: Arc<NativeClient>,
tx: SyncSender<Vec<f32>>, tx: SyncSender<Vec<f32>>,
free_rx: Receiver<Vec<f32>>,
shutdown: Arc<AtomicBool>, shutdown: Arc<AtomicBool>,
counters: Arc<Counters>, counters: Arc<Counters>,
) { ) {
@@ -166,8 +254,22 @@ fn decode_loop(
for &s in &pcm[..n] { for &s in &pcm[..n] {
window_peak = window_peak.max(s.abs()); window_peak = window_peak.max(s.abs());
} }
// The ring's pre-reservation in `start` assumes the protocol's 5 ms (≤480-f32)
// frames; a larger frame would force a one-time realloc on the RT thread. Catch a
// future host frame-size change here in debug, not as a silent audio glitch.
debug_assert!(
n <= 5 * MS,
"audio frame {n} f32 exceeds the 5 ms ring reserve"
);
let count = counters.opus_decoded.fetch_add(1, Ordering::Relaxed) + 1; let count = counters.opus_decoded.fetch_add(1, Ordering::Relaxed) + 1;
match tx.try_send(pcm[..n].to_vec()) { // Reuse a recycled buffer if the callback handed one back; only allocate when the
// free-list is momentarily empty (startup / after a backpressure drop).
let mut buf = free_rx
.try_recv()
.unwrap_or_else(|_| Vec::with_capacity(PCM_SCRATCH));
buf.clear();
buf.extend_from_slice(&pcm[..n]);
match tx.try_send(buf) {
Ok(()) | Err(TrySendError::Full(_)) => {} // drop-newest under backpressure Ok(()) | Err(TrySendError::Full(_)) => {} // drop-newest under backpressure
Err(TrySendError::Disconnected(_)) => break, Err(TrySendError::Disconnected(_)) => break,
} }
@@ -0,0 +1,73 @@
<?xml version="1.0" encoding="UTF-8"?>
<protocol name="fake_input">
<copyright>
SPDX-FileCopyrightText: 2015 Martin Gräßlin
SPDX-License-Identifier: LGPL-2.1-or-later
</copyright>
<interface name="org_kde_kwin_fake_input" version="4">
<description summary="Fake input manager">
This interface allows other processes to provide fake input events.
Purpose is on the one hand side to provide testing facilities like XTest
on X11, but also to support use cases like remote control (a remote
desktop server). The compositor gates the interface: it is only exposed
to clients authorized through their .desktop X-KDE-Wayland-Interfaces, so
binding it is the authorization — no per-event confirmation dialog.
</description>
<request name="authenticate">
<description summary="Information about the application requesting fake input">
A FakeInput is required to authenticate itself by providing the
application name and the reason for fake input. The compositor may use
this information to decide whether to allow or deny the request.
</description>
<arg name="application" type="string" summary="user visible name of the application requesting fake input"/>
<arg name="reason" type="string" summary="reason of why fake input is requested"/>
</request>
<request name="pointer_motion">
<description summary="pointer motion event"/>
<arg name="delta_x" type="fixed" summary="X delta of the relative pointer motion"/>
<arg name="delta_y" type="fixed" summary="Y delta of the relative pointer motion"/>
</request>
<request name="button">
<description summary="pointer button event"/>
<arg name="button" type="uint" summary="evdev button code"/>
<arg name="state" type="uint" summary="button state, 0 released, 1 pressed"/>
</request>
<request name="axis">
<description summary="pointer axis (scroll) event"/>
<arg name="axis" type="uint" summary="wl_pointer.axis (0 vertical, 1 horizontal)"/>
<arg name="value" type="fixed" summary="axis value"/>
</request>
<request name="touch_down" since="2">
<description summary="touch down event"/>
<arg name="id" type="uint" summary="unique id of this touch point; must not be reused until up"/>
<arg name="x" type="fixed" summary="x coordinate in global compositor space"/>
<arg name="y" type="fixed" summary="y coordinate in global compositor space"/>
</request>
<request name="touch_motion" since="2">
<description summary="touch motion event"/>
<arg name="id" type="uint" summary="unique id of an existing touch point"/>
<arg name="x" type="fixed" summary="x coordinate in global compositor space"/>
<arg name="y" type="fixed" summary="y coordinate in global compositor space"/>
</request>
<request name="touch_up" since="2">
<description summary="touch up event"/>
<arg name="id" type="uint" summary="unique id of an existing touch point"/>
</request>
<request name="touch_cancel" since="2">
<description summary="cancel all current touch points"/>
</request>
<request name="touch_frame" since="2">
<description summary="end a set of touch events (atomic frame)"/>
</request>
<request name="pointer_motion_absolute" since="3">
<description summary="absolute pointer motion event"/>
<arg name="x" type="fixed" summary="x coordinate in global compositor space"/>
<arg name="y" type="fixed" summary="y coordinate in global compositor space"/>
</request>
<request name="keyboard_key" since="4">
<description summary="keyboard key event"/>
<arg name="button" type="uint" summary="evdev key code"/>
<arg name="state" type="uint" summary="key state, 0 released, 1 pressed"/>
</request>
</interface>
</protocol>
+8 -1
View File
@@ -320,11 +320,18 @@ fn mic_pw_thread(
.into_inner(); .into_inner();
let mut params = [Pod::from_bytes(&values).context("mic pod from bytes")?]; let mut params = [Pod::from_bytes(&values).context("mic pod from bytes")?];
// RT_PROCESS: run the producer callback on PipeWire's realtime data loop, so the source is a
// *synchronous* graph node that joins its consumer's driver group and is actually driven. Without
// it the node is async/main-loop and, in the host's busy multi-stream graph (desktop-audio +
// video capture + the session), never acquires a driver — it stays suspended and its process()
// never fires, so every recorder hears pure silence (the long-standing "Linux host mic broken").
stream stream
.connect( .connect(
spa::utils::Direction::Output, // we PRODUCE samples (a source) spa::utils::Direction::Output, // we PRODUCE samples (a source)
None, None,
pw::stream::StreamFlags::AUTOCONNECT | pw::stream::StreamFlags::MAP_BUFFERS, pw::stream::StreamFlags::AUTOCONNECT
| pw::stream::StreamFlags::MAP_BUFFERS
| pw::stream::StreamFlags::RT_PROCESS,
&mut params, &mut params,
) )
.context("pw mic stream connect")?; .context("pw mic stream connect")?;
@@ -106,7 +106,10 @@ fn capture_thread(
} }
let res = (|| -> Result<()> { let res = (|| -> Result<()> {
// Loopback = capture the RENDER endpoint: get the default render device, but open a CAPTURE // Loopback = capture the RENDER endpoint: get the default render device, but open a CAPTURE
// client with loopback=true over it. // client with loopback=true over it. NOTE: the virtual mic (`super::wasapi_mic`) is guarded
// to NEVER target this same endpoint — otherwise the client's injected mic would be captured
// here and streamed back to the client (infinite echo). Keep that guard in sync if this
// device selection ever changes.
let device = DeviceEnumerator::new() let device = DeviceEnumerator::new()
.context("DeviceEnumerator")? .context("DeviceEnumerator")?
.get_default_device(&Direction::Render) .get_default_device(&Direction::Render)
@@ -5,8 +5,18 @@
//! //!
//! Target device, by friendly-name substring (first match wins; override with `PUNKTFUNK_MIC_DEVICE`): //! Target device, by friendly-name substring (first match wins; override with `PUNKTFUNK_MIC_DEVICE`):
//! "Steam Streaming Microphone" (ships with Steam Remote Play — exactly this purpose), VB-Audio //! "Steam Streaming Microphone" (ships with Steam Remote Play — exactly this purpose), VB-Audio
//! "CABLE Input", VoiceMeeter, or anything with "virtual" in the name. If none is present we return an //! "CABLE Input", VoiceMeeter, or anything with "virtual" in the name. If none is present we
//! error with install guidance and the host runs without mic passthrough. //! auto-install the Steam Streaming audio pair (see [`install_steam_audio_pair`]); failing that we
//! return an error with install guidance and the host runs without mic passthrough.
//!
//! **Anti-echo guard (the whole point of this being non-trivial).** The desktop-audio plane
//! ([`super::wasapi_cap`]) loopback-captures the **default render endpoint**. WASAPI loopback
//! captures the *mixed* output of an endpoint — i.e. everything any app renders to it, including
//! what THIS module writes. So if the virtual-mic target is the same device the loopback captures,
//! the client's uplinked mic is captured straight back into the host→client audio stream: an
//! infinite echo. [`find_device`] therefore **excludes the default render endpoint** from the
//! candidates — the mic is guaranteed to land on a different device. (Linux gets this for free: its
//! mic is a dedicated `Audio/Source` node, structurally separate from the monitored sink.)
//! //!
//! `push` enqueues decoded interleaved-f32 PCM into a bounded ring (drop-oldest beyond ~80 ms so mic //! `push` enqueues decoded interleaved-f32 PCM into a bounded ring (drop-oldest beyond ~80 ms so mic
//! latency stays bounded); a dedicated COM-apartment thread renders it event-driven, filling silence //! latency stays bounded); a dedicated COM-apartment thread renders it event-driven, filling silence
@@ -113,8 +123,23 @@ impl VirtualMic for WasapiVirtualMic {
} }
} }
/// Resolve the virtual-mic target among render endpoints by friendly-name. Logs all candidates so a /// The endpoint ID of the device the desktop-audio loopback records (the **default render
/// missing device is diagnosable. /// endpoint**, see [`super::wasapi_cap`]). The virtual mic must never target this device — injecting
/// there echoes the client's mic back into the host→client audio stream. `None` if it can't be
/// resolved (then [`find_device`] can't prove a candidate is safe and falls back to name-only
/// matching — no worse than before the guard existed).
fn default_render_id() -> Option<String> {
wasapi::DeviceEnumerator::new()
.ok()?
.get_default_device(&Direction::Render)
.ok()?
.get_id()
.ok()
}
/// Resolve the virtual-mic target among render endpoints by friendly-name, **excluding the endpoint
/// the loopback captures** (the [`default_render_id`] anti-echo guard). Logs all candidates so a
/// missing/skipped device is diagnosable.
fn find_device() -> Result<wasapi::Device> { fn find_device() -> Result<wasapi::Device> {
let enumerator = wasapi::DeviceEnumerator::new().context("DeviceEnumerator")?; let enumerator = wasapi::DeviceEnumerator::new().context("DeviceEnumerator")?;
let collection = enumerator let collection = enumerator
@@ -124,8 +149,11 @@ fn find_device() -> Result<wasapi::Device> {
let want = std::env::var("PUNKTFUNK_MIC_DEVICE") let want = std::env::var("PUNKTFUNK_MIC_DEVICE")
.ok() .ok()
.map(|s| s.to_lowercase()); .map(|s| s.to_lowercase());
// The device the loopback captures — a name match on it is rejected below (would echo).
let loopback_id = default_render_id();
let mut names = Vec::new(); let mut names = Vec::new();
let mut found = None; let mut found = None;
let mut skipped_loopback = false;
for i in 0..n { for i in 0..n {
let Ok(dev) = collection.get_device_at_index(i) else { let Ok(dev) = collection.get_device_at_index(i) else {
continue; continue;
@@ -137,16 +165,37 @@ fn find_device() -> Result<wasapi::Device> {
None => CANDIDATES.iter().any(|c| lname.contains(c)), None => CANDIDATES.iter().any(|c| lname.contains(c)),
}; };
if hit && found.is_none() { if hit && found.is_none() {
// Anti-echo guard: never inject into the endpoint the loopback captures.
let is_loopback = match (dev.get_id().ok(), loopback_id.as_deref()) {
(Some(id), Some(lb)) => id == lb,
_ => false,
};
if is_loopback {
skipped_loopback = true;
tracing::warn!(device = %name,
"virtual-mic candidate is the loopback (default render) endpoint — skipping; \
injecting there would echo the client's mic into the desktop-audio stream");
} else {
found = Some(dev); found = Some(dev);
} }
}
names.push(name); names.push(name);
} }
found.ok_or_else(|| { found.ok_or_else(|| {
if skipped_loopback {
anyhow!( anyhow!(
"no virtual-mic device among render endpoints {names:?}. Install VB-Audio Virtual Cable \ "the only virtual-mic candidate among render endpoints {names:?} is the default \
or enable Steam Remote Play's microphone (Steam Streaming Microphone), or set \ playback device the host loopback-captures — injecting there would echo the mic \
back to the client. Add a SEPARATE virtual audio device for the mic (e.g. the Steam \
Streaming Microphone) or set a different default playback device, then reconnect."
)
} else {
anyhow!(
"no virtual-mic device among render endpoints {names:?}. Install VB-Audio Virtual \
Cable or enable Steam Remote Play's microphone (Steam Streaming Microphone), or set \
PUNKTFUNK_MIC_DEVICE=<friendly-name substring>." PUNKTFUNK_MIC_DEVICE=<friendly-name substring>."
) )
}
}) })
} }
@@ -156,15 +205,15 @@ fn find_or_install_device() -> Result<wasapi::Device> {
match find_device() { match find_device() {
Ok(d) => Ok(d), Ok(d) => Ok(d),
Err(e) => { Err(e) => {
tracing::info!("no virtual mic device present — attempting auto-install"); tracing::info!("no usable virtual mic device present — attempting auto-install");
// SAFETY: `try_install_virtual_mic` is `unsafe` only because it `LoadLibraryExW`s // SAFETY: `install_steam_audio_pair` is `unsafe` only because it `LoadLibraryExW`s
// `newdev.dll` and calls `DiInstallDriverW` through a `transmute`d function pointer; // `newdev.dll` and calls `DiInstallDriverW` through a `transmute`d function pointer;
// calling it imposes no extra precondition here (it takes no args and aliases nothing). // calling it imposes no extra precondition here (it takes no args and aliases nothing).
// Its internal contract holds: the `DiInstall` type matches the documented // Its internal contract holds: the `DiInstall` type matches the documented
// `BOOL DiInstallDriverW(HWND, PCWSTR, DWORD, PBOOL)` ABI, and it passes a // `BOOL DiInstallDriverW(HWND, PCWSTR, DWORD, PBOOL)` ABI, and it passes a
// NUL-terminated UTF-16 INF path with null/zero optional args. Invoked once on the // NUL-terminated UTF-16 INF path with null/zero optional args. Invoked once on the
// dedicated mic thread. // dedicated mic thread.
if unsafe { try_install_virtual_mic() } { if unsafe { install_steam_audio_pair() } {
find_device() find_device()
} else { } else {
Err(e) Err(e)
@@ -173,13 +222,26 @@ fn find_or_install_device() -> Result<wasapi::Device> {
} }
} }
/// Best-effort: install a virtual mic device so one exists without the user installing anything. /// Best-effort: install BOTH Steam Streaming audio devices (the "Steam pair") so mic passthrough
/// Mirrors Apollo's Steam Streaming Speakers install — Steam Remote Play ships /// works out of the box and the host has a desktop-audio sink distinct from the mic. Steam Remote
/// `SteamStreamingMicrophone.inf` next to the speakers INF, so install it via `DiInstallDriverW` /// Play ships `SteamStreamingMicrophone.inf` + `SteamStreamingSpeakers.inf`: the microphone gives the
/// (loaded from `newdev.dll`, like Apollo, to avoid an extra windows-crate feature). Needs admin (the /// virtual mic a target whose **capture** endpoint apps record from, and the speakers give a
/// host runs as SYSTEM). Returns true on success; false (no-op) if Steam isn't installed (INF absent), /// **render** endpoint a headless box can loopback-capture that is NOT the mic — so the loopback and
/// the install is denied, or `PUNKTFUNK_NO_MIC_INSTALL` is set. /// the mic land on different devices and never echo (see [`find_device`]). Returns true if either
unsafe fn try_install_virtual_mic() -> bool { /// installed. No-op when Steam isn't installed (INFs absent), the install is denied (needs admin —
/// the host runs as SYSTEM), or `PUNKTFUNK_NO_MIC_INSTALL` is set.
unsafe fn install_steam_audio_pair() -> bool {
// Microphone first (the mic's actual target); speakers second (the distinct desktop-audio sink).
let mic = try_install_steam_audio("SteamStreamingMicrophone.inf");
let spk = try_install_steam_audio("SteamStreamingSpeakers.inf");
mic || spk
}
/// Install one Steam Streaming driver INF by filename via `DiInstallDriverW` (loaded from
/// `newdev.dll`, like Apollo, to avoid an extra windows-crate feature). See
/// [`install_steam_audio_pair`] for the contract; `inf_name` is a bare filename under Steam's
/// per-arch `drivers\Windows10\{arch}\` directory.
unsafe fn try_install_steam_audio(inf_name: &str) -> bool {
use windows::core::{s, w, PCWSTR}; use windows::core::{s, w, PCWSTR};
use windows::Win32::Foundation::HWND; use windows::Win32::Foundation::HWND;
use windows::Win32::System::Environment::ExpandEnvironmentStringsW; use windows::Win32::System::Environment::ExpandEnvironmentStringsW;
@@ -197,9 +259,8 @@ unsafe fn try_install_virtual_mic() -> bool {
let subdir = "arm64"; let subdir = "arm64";
#[cfg(not(any(target_arch = "x86_64", target_arch = "aarch64")))] #[cfg(not(any(target_arch = "x86_64", target_arch = "aarch64")))]
let subdir = "x86"; let subdir = "x86";
let template: Vec<u16> = format!( let template: Vec<u16> =
"%CommonProgramFiles(x86)%\\Steam\\drivers\\Windows10\\{subdir}\\SteamStreamingMicrophone.inf" format!("%CommonProgramFiles(x86)%\\Steam\\drivers\\Windows10\\{subdir}\\{inf_name}")
)
.encode_utf16() .encode_utf16()
.chain(std::iter::once(0)) .chain(std::iter::once(0))
.collect(); .collect();
@@ -210,7 +271,7 @@ unsafe fn try_install_virtual_mic() -> bool {
} }
let Ok(newdev) = LoadLibraryExW(w!("newdev.dll"), None, LOAD_LIBRARY_SEARCH_SYSTEM32) else { let Ok(newdev) = LoadLibraryExW(w!("newdev.dll"), None, LOAD_LIBRARY_SEARCH_SYSTEM32) else {
tracing::warn!("could not load newdev.dll — virtual-mic auto-install unavailable"); tracing::warn!("could not load newdev.dll — Steam-audio auto-install unavailable");
return false; return false;
}; };
let Some(addr) = GetProcAddress(newdev, s!("DiInstallDriverW")) else { let Some(addr) = GetProcAddress(newdev, s!("DiInstallDriverW")) else {
@@ -226,13 +287,17 @@ unsafe fn try_install_virtual_mic() -> bool {
std::ptr::null_mut(), std::ptr::null_mut(),
) != 0; ) != 0;
if ok { if ok {
tracing::info!("installed the Steam Streaming Microphone virtual device"); tracing::info!(
inf = inf_name,
"installed a Steam Streaming virtual audio device"
);
std::thread::sleep(Duration::from_secs(5)); // let the audio subsystem register the endpoint std::thread::sleep(Duration::from_secs(5)); // let the audio subsystem register the endpoint
} else { } else {
let err = windows::Win32::Foundation::GetLastError(); let err = windows::Win32::Foundation::GetLastError();
tracing::info!( tracing::info!(
inf = inf_name,
?err, ?err,
"no virtual mic auto-installed (Steam absent / not admin) — see manual-install guidance" "Steam-audio device not auto-installed (Steam absent / not admin) — see install guidance"
); );
} }
ok ok
+52 -1
View File
@@ -40,6 +40,13 @@ pub struct PortalCapturer {
/// branch to tell "format never negotiated" (modifier/format mismatch) apart from "negotiated /// branch to tell "format never negotiated" (modifier/format mismatch) apart from "negotiated
/// but no buffers arrived" (compositor idle/unmapped) — the two black-screen root causes. /// but no buffers arrived" (compositor idle/unmapped) — the two black-screen root causes.
negotiated: Arc<AtomicBool>, negotiated: Arc<AtomicBool>,
/// True only while the PipeWire stream is `Streaming`. [`try_latest`](Self::try_latest) reads it
/// to distinguish a static desktop (alive, no new buffers) from a dead source (left `Streaming`).
streaming: Arc<AtomicBool>,
/// When the stream first dropped out of `Streaming` with no new frame; used to grace a transient
/// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream
/// is `Streaming`.
stall_since: Option<std::time::Instant>,
/// The PipeWire node this capturer consumes — surfaced in error messages for diagnosis. /// The PipeWire node this capturer consumes — surfaced in error messages for diagnosis.
node_id: u32, node_id: u32,
/// Stops the PipeWire loop on teardown (sent in `Drop`). Without it a dropped or failed /// Stops the PipeWire loop on teardown (sent in `Drop`). Without it a dropped or failed
@@ -109,6 +116,7 @@ struct PwHandles {
frames: Receiver<CapturedFrame>, frames: Receiver<CapturedFrame>,
active: Arc<AtomicBool>, active: Arc<AtomicBool>,
negotiated: Arc<AtomicBool>, negotiated: Arc<AtomicBool>,
streaming: Arc<AtomicBool>,
quit: ::pipewire::channel::Sender<()>, quit: ::pipewire::channel::Sender<()>,
join: thread::JoinHandle<()>, join: thread::JoinHandle<()>,
} }
@@ -121,6 +129,8 @@ impl PwHandles {
frames: self.frames, frames: self.frames,
active: self.active, active: self.active,
negotiated: self.negotiated, negotiated: self.negotiated,
streaming: self.streaming,
stall_since: None,
node_id, node_id,
quit: Some(self.quit), quit: Some(self.quit),
join: Some(self.join), join: Some(self.join),
@@ -143,6 +153,8 @@ fn spawn_pipewire(
let active_cb = active.clone(); let active_cb = active.clone();
let negotiated = Arc::new(AtomicBool::new(false)); let negotiated = Arc::new(AtomicBool::new(false));
let negotiated_cb = negotiated.clone(); let negotiated_cb = negotiated.clone();
let streaming = Arc::new(AtomicBool::new(false));
let streaming_cb = streaming.clone();
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the // pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the // sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope. // inner `mod pipewire` shadows the crate name at this scope.
@@ -157,6 +169,7 @@ fn spawn_pipewire(
frame_tx, frame_tx,
active_cb, active_cb,
negotiated_cb, negotiated_cb,
streaming_cb,
zerocopy, zerocopy,
preferred, preferred,
quit_rx, quit_rx,
@@ -169,6 +182,7 @@ fn spawn_pipewire(
frames: frame_rx, frames: frame_rx,
active, active,
negotiated, negotiated,
streaming,
quit: quit_tx, quit: quit_tx,
join, join,
}) })
@@ -219,6 +233,28 @@ impl Capturer for PortalCapturer {
} }
} }
} }
if latest.is_some() || self.streaming.load(Ordering::Relaxed) {
// A frame arrived, or the source is alive but idle (static desktop) — normal. Clear any
// stall and repeat the last frame on `None`, exactly as before.
self.stall_since = None;
return Ok(latest);
}
// No new frame AND the stream has left `Streaming` (Paused/Unconnected/Error). The source
// went away — a compositor torn down on a Gaming↔Desktop switch, a removed virtual output.
// Grace a brief window (a transient mid-stream renegotiation can blip out of Streaming and
// back) before declaring it lost so the encode loop rebuilds in place rather than freezing
// on the last frame forever.
const STALL_GRACE: Duration = Duration::from_millis(1500);
let since = *self.stall_since.get_or_insert_with(std::time::Instant::now);
if since.elapsed() >= STALL_GRACE {
self.stall_since = None;
return Err(anyhow!(
"PipeWire source stalled (node {}): stream left Streaming for >{}ms with no frames \
— the compositor/virtual output went away (session switch?)",
self.node_id,
STALL_GRACE.as_millis()
));
}
Ok(latest) Ok(latest)
} }
@@ -467,6 +503,10 @@ mod pipewire {
/// Set once a video format is agreed (`param_changed`), so a first-frame timeout can tell /// Set once a video format is agreed (`param_changed`), so a first-frame timeout can tell
/// "format never negotiated" apart from "negotiated but no buffers arrived". /// "format never negotiated" apart from "negotiated but no buffers arrived".
negotiated: Arc<AtomicBool>, negotiated: Arc<AtomicBool>,
/// True only while the PipeWire stream is in `Streaming` (the source is alive). Goes false on
/// `Paused`/`Unconnected`/`Error` — the source vanished (compositor torn down on a session
/// switch). Read by [`PortalCapturer::try_latest`] to surface a sustained drop as a loss.
streaming: Arc<AtomicBool>,
/// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer. /// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer.
importer: Option<crate::zerocopy::EglImporter>, importer: Option<crate::zerocopy::EglImporter>,
/// VAAPI zero-copy: hand the raw dmabuf to the encoder (which imports + GPU-CSCs it) instead /// VAAPI zero-copy: hand the raw dmabuf to the encoder (which imports + GPU-CSCs it) instead
@@ -1056,6 +1096,7 @@ mod pipewire {
tx: SyncSender<CapturedFrame>, tx: SyncSender<CapturedFrame>,
active: Arc<AtomicBool>, active: Arc<AtomicBool>,
negotiated: Arc<AtomicBool>, negotiated: Arc<AtomicBool>,
streaming: Arc<AtomicBool>,
zerocopy: bool, zerocopy: bool,
preferred: Option<(u32, u32, u32)>, preferred: Option<(u32, u32, u32)>,
quit_rx: pw::channel::Receiver<()>, quit_rx: pw::channel::Receiver<()>,
@@ -1150,6 +1191,7 @@ mod pipewire {
tx, tx,
active, active,
negotiated, negotiated,
streaming,
importer, importer,
vaapi_passthrough, vaapi_passthrough,
nv12: crate::zerocopy::nv12_enabled(), nv12: crate::zerocopy::nv12_enabled(),
@@ -1174,8 +1216,17 @@ mod pipewire {
let _listener = stream let _listener = stream
.add_local_listener_with_user_data(data) .add_local_listener_with_user_data(data)
.state_changed(|_stream, _ud, old, new| { .state_changed(|_stream, ud, old, new| {
tracing::info!(?old, ?new, "pipewire stream state"); tracing::info!(?old, ?new, "pipewire stream state");
// Track whether the node is actively producing. A live source sits in `Streaming`
// (a static desktop just sends no buffers); anything else — `Paused`/`Unconnected`/
// `Error` — means the source went away (compositor died, virtual output removed on a
// Gaming↔Desktop switch). `try_latest` turns a sustained non-Streaming state into a
// capture-loss so the encode loop rebuilds instead of freezing on the last frame.
ud.streaming.store(
matches!(new, pw::stream::StreamState::Streaming),
Ordering::Relaxed,
);
}) })
.param_changed(|_stream, ud, id, param| { .param_changed(|_stream, ud, id, param| {
let Some(param) = param else { return }; let Some(param) = param else { return };
+134 -36
View File
@@ -114,12 +114,12 @@ fn run(
// `video_cap`, since a reconnect at a different resolution needs a freshly-sized output; the // `video_cap`, since a reconnect at a different resolution needs a freshly-sized output; the
// output is released when this capturer drops at stream end (RAII via its keepalive). // output is released when this capturer drops at stream end (RAII via its keepalive).
if crate::config::config().video_source.as_deref() == Some("virtual") { if crate::config::config().video_source.as_deref() == Some("virtual") {
// The launched app picks the compositor (e.g. gamescope for game entries) and the // Open the virtual-display source: pick the live compositor, normalize the session env
// nested command. // (apply_session_env/apply_input_env — gamescope ATTACH/resize + KWin/Mutter retargeting,
let compositor = app // exactly like the native plane), create a virtual output at the client mode, and capture it.
.and_then(|a| a.compositor) // Re-runnable: the encode loop calls it again on a mid-stream capture loss to FOLLOW a
.map(Ok) // Desktop<->Game switch.
.unwrap_or_else(|| crate::vdisplay::detect().context("detect compositor"))?; let (mut capturer, compositor) = open_gs_virtual_source(cfg, app)?;
tracing::info!( tracing::info!(
?compositor, ?compositor,
app = ?app.map(|a| &a.title), app = ?app.map(|a| &a.title),
@@ -127,31 +127,6 @@ fn run(
h = cfg.height, h = cfg.height,
"video source: virtual display (native client resolution)" "video source: virtual display (native client resolution)"
); );
let mut vd = crate::vdisplay::open(compositor).context("open virtual display")?;
// Carry the resolved launch command on the backend instance (per-session) rather than a
// process-global env var, so concurrent sessions can't stomp each other's launch target.
vd.set_launch_command(app.and_then(|a| a.cmd.clone()));
let vout = vd
.create(punktfunk_core::Mode {
width: cfg.width,
height: cfg.height,
refresh_hz: cfg.fps,
})
.context("create virtual output at client resolution")?;
// `want_hdr=false`: the IDD-push backend (opt-in PUNKTFUNK_IDD_PUSH) has no monitor-HDR
// auto-detection — it converts its always-FP16 ring per this flag — and GameStream HDR is not
// negotiated into StreamConfig here, so an IDD-push GameStream session streams SDR even on an
// HDR desktop. (The default WGC backend DOES auto-detect HDR from the output colorspace, but
// IDD-push bypasses WGC.) Acceptable for the experimental IDD-push A/B path; HDR over IDD-push
// is wired only for punktfunk/1 (want_hdr = negotiated bit_depth >= 10). TODO: derive want_hdr
// from a GameStream HDR flag once StreamConfig carries one.
let mut capturer = capture::capture_virtual_output(
vout,
capture::OutputFormat::resolve(false),
crate::session_plan::CaptureBackend::resolve(),
)
.context("capture virtual output")?;
capturer.set_active(true);
// Launch the app's command now that capture is live, for the backends that DON'T nest it via // Launch the app's command now that capture is live, for the backends that DON'T nest it via
// set_launch_command above: Windows (no gamescope) and Linux kwin/mutter/wlroots (which stream // set_launch_command above: Windows (no gamescope) and Linux kwin/mutter/wlroots (which stream
// the existing desktop, so the app must be spawned into the session to land on the streamed // the existing desktop, so the app must be spawned into the session to land on the streamed
@@ -171,8 +146,14 @@ fn run(
} }
} }
} }
// Rebuild closure: re-open the source on a mid-stream capture loss, RE-DETECTING the live
// compositor — so a Desktop<->Game switch (at the client's fixed mode) is FOLLOWED in place
// without a Moonlight reconnect. (A resolution change can't be followed mid-stream on
// GameStream — WxH is locked at ANNOUNCE — but a session toggle keeps the negotiated mode.)
let rebuild = || open_gs_virtual_source(cfg, app).map(|(c, _)| c);
return stream_body( return stream_body(
&mut *capturer, &mut capturer,
Some(&rebuild),
&sock, &sock,
cfg, cfg,
running, running,
@@ -200,8 +181,10 @@ fn run(
} }
}; };
capturer.set_active(true); capturer.set_active(true);
// Portal/synthetic source: no compositor virtual output to re-detect, so no rebuild closure.
let result = stream_body( let result = stream_body(
&mut *capturer, &mut capturer,
None,
&sock, &sock,
cfg, cfg,
running, running,
@@ -215,6 +198,53 @@ fn run(
result result
} }
/// Open the virtual-display video source for a GameStream session: pick the LIVE compositor + normalize
/// the session env (apply_session_env/apply_input_env — gamescope ATTACH/resize, KWin/Mutter
/// retargeting) exactly like the native plane (punktfunk1.rs resolve_compositor), create a virtual
/// output at the client's mode, and capture it. Returns the capturer (it owns the output's keepalive;
/// the stateless VirtualDisplay factory is dropped here) plus the resolved compositor. An apps.json
/// entry can PIN a compositor (skips the live detect/retarget). Re-run on a mid-stream capture loss to
/// FOLLOW a Desktop<->Game switch: it re-detects the now-live compositor and re-targets at it. Does NOT
/// launch the app (that happens once at stream start; a rebuild must not re-spawn it).
fn open_gs_virtual_source(
cfg: StreamConfig,
app: Option<&super::apps::AppEntry>,
) -> Result<(Box<dyn Capturer>, crate::vdisplay::Compositor)> {
let compositor = if let Some(c) = app.and_then(|a| a.compositor) {
c
} else {
let active = crate::vdisplay::detect_active_session();
crate::vdisplay::apply_session_env(&active);
let c = crate::vdisplay::compositor_for_kind(active.kind)
.map(Ok)
.unwrap_or_else(crate::vdisplay::detect)
.context("detect compositor")?;
crate::vdisplay::apply_input_env(c);
c
};
let mut vd = crate::vdisplay::open(compositor).context("open virtual display")?;
// Carry the resolved launch command on the backend instance (per-session) rather than a
// process-global env var, so concurrent sessions can't stomp each other's launch target.
vd.set_launch_command(app.and_then(|a| a.cmd.clone()));
let vout = vd
.create(punktfunk_core::Mode {
width: cfg.width,
height: cfg.height,
refresh_hz: cfg.fps,
})
.context("create virtual output at client resolution")?;
// want_hdr=false: GameStream HDR is not negotiated into StreamConfig here (the default WGC backend
// still auto-detects HDR from the output colorspace; only the opt-in IDD-push path streams SDR).
let capturer = capture::capture_virtual_output(
vout,
capture::OutputFormat::resolve(false),
crate::session_plan::CaptureBackend::resolve(),
)
.context("capture virtual output")?;
capturer.set_active(true);
Ok((capturer, compositor))
}
/// One frame's packets, handed from the encode thread to the send thread. /// One frame's packets, handed from the encode thread to the send thread.
type PacketBatch = Vec<Vec<u8>>; type PacketBatch = Vec<Vec<u8>>;
@@ -367,7 +397,11 @@ fn percentile(v: &mut [u32], q: f64) -> u32 {
/// (see [`spawn_sender`]) so a send spike can never stall capture/encode. /// (see [`spawn_sender`]) so a send spike can never stall capture/encode.
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
fn stream_body( fn stream_body(
capturer: &mut dyn Capturer, // `&mut Box` (not `&mut dyn`) so a mid-stream capture-loss rebuild can SWAP the capturer in place.
capturer: &mut Box<dyn Capturer>,
// Re-open the video source on capture loss (virtual-display path → follow a Desktop<->Game switch);
// `None` for the portal/synthetic source, which has nothing to re-detect (propagate the error).
rebuild: Option<&dyn Fn() -> Result<Box<dyn Capturer>>>,
sock: &UdpSocket, sock: &UdpSocket,
cfg: StreamConfig, cfg: StreamConfig,
running: &Arc<AtomicBool>, running: &Arc<AtomicBool>,
@@ -459,7 +493,12 @@ fn stream_body(
// RFI capability is fixed for the session (probed at encoder open). Query it once so the // RFI capability is fixed for the session (probed at encoder open). Query it once so the
// recovery path skips the always-`false` invalidate call on encoders without NVENC RFI and // recovery path skips the always-`false` invalidate call on encoders without NVENC RFI and
// forces a keyframe directly instead. // forces a keyframe directly instead.
let supports_rfi = enc.caps().supports_rfi; let mut supports_rfi = enc.caps().supports_rfi;
// Bound consecutive capture-loss rebuilds (a delivered frame clears the counter) so a permanently
// dead source can't loop forever — it ends the stream after the cap, falling back to a reconnect.
const MAX_REBUILDS: u32 = 5;
let mut rebuilds: u32 = 0;
while running.load(Ordering::SeqCst) { while running.load(Ordering::SeqCst) {
let tick = Instant::now(); let tick = Instant::now();
@@ -467,9 +506,68 @@ fn stream_body(
// armed (cheap Relaxed atomic, re-read each frame). // armed (cheap Relaxed atomic, re-read each frame).
let measure = perf || stats.is_armed(); let measure = perf || stats.is_armed();
// Advance to the freshest captured frame if one arrived; otherwise reuse the last. // Advance to the freshest captured frame if one arrived; otherwise reuse the last.
if let Some(f) = capturer.try_latest().context("capture frame")? { match capturer.try_latest() {
Ok(Some(f)) => {
frame = f; frame = f;
uniq += 1; uniq += 1;
rebuilds = 0; // a delivered frame clears the consecutive-loss counter
}
Ok(None) => {} // no new frame — reuse the last (static/idle desktop)
Err(e) => {
// The capture source went away — the compositor was torn down on a Desktop<->Game
// switch, or the virtual output was removed. On the virtual-display path, re-detect the
// now-live compositor and re-attach IN PLACE (the send thread + packetizer + socket +
// RTP clock all survive), then force an IDR so Moonlight resyncs — so the stream FOLLOWS
// the switch with no client reconnect. Build the new source BEFORE dropping the old.
// Bounded by a counter + a ~40s budget; on exhaustion, end the stream (Moonlight
// reconnect). The portal/synthetic path has no rebuild closure → propagate as before.
let Some(rebuild) = rebuild else {
return Err(e).context("capture frame");
};
rebuilds += 1;
if rebuilds > MAX_REBUILDS {
return Err(e).context("capture lost — rebuild attempts exhausted");
}
tracing::warn!(error = %format!("{e:#}"), rebuild = rebuilds,
"gamestream: capture lost — rebuilding source in place (following a session switch)");
let rebuild_deadline = Instant::now() + Duration::from_secs(40);
let new_cap = loop {
match rebuild() {
Ok(c) => break c,
Err(e2) => {
if !running.load(Ordering::SeqCst) || Instant::now() >= rebuild_deadline
{
return Err(e2)
.context("capture lost — no source within the rebuild budget");
}
tracing::warn!(error = %format!("{e2:#}"),
"gamestream: source not up yet — retrying");
std::thread::sleep(Duration::from_millis(500));
}
}
};
*capturer = new_cap;
capturer.set_active(true);
frame = capturer.next_frame().context("first frame after rebuild")?;
// Re-open the encoder for the new source (same negotiated WxH → same SPS profile) and
// force an IDR so Moonlight resyncs on the first emitted AU.
enc = encode::open_video(
cfg.codec,
frame.format,
frame.width,
frame.height,
cfg.fps,
cfg.bitrate_kbps as u64 * 1000,
frame.is_cuda(),
8,
)
.context("reopen encoder after rebuild")?;
supports_rfi = enc.caps().supports_rfi;
enc.request_keyframe();
next_frame = Instant::now();
tracing::info!("gamestream: source rebuilt — stream continues");
continue;
}
} }
let t_cap = tick.elapsed(); let t_cap = tick.elapsed();
// Honor a client recovery request. Prefer reference-frame invalidation (the encoder // Honor a client recovery request. Prefer reference-frame invalidation (the encoder
+40 -8
View File
@@ -24,6 +24,9 @@ pub trait InputInjector {
pub enum Backend { pub enum Backend {
/// wlroots virtual pointer + keyboard Wayland protocols — the headless-Sway path. /// wlroots virtual pointer + keyboard Wayland protocols — the headless-Sway path.
WlrVirtual, WlrVirtual,
/// KWin `org_kde_kwin_fake_input` — direct injection, no RemoteDesktop portal / approval dialog
/// (authorized by the host's `.desktop`). The headless KDE-Desktop path; what krdpserver uses.
KwinFakeInput,
/// libei via `reis` — Wayland-native (RemoteDesktop portal). Not yet implemented. /// libei via `reis` — Wayland-native (RemoteDesktop portal). Not yet implemented.
Libei, Libei,
/// libei directly against gamescope's own EIS socket (no portal): input lands in the /// libei directly against gamescope's own EIS socket (no portal): input lands in the
@@ -47,6 +50,16 @@ pub fn open(backend: Backend) -> Result<Box<dyn InputInjector>> {
anyhow::bail!("wlroots virtual input requires Linux + a Wayland compositor") anyhow::bail!("wlroots virtual input requires Linux + a Wayland compositor")
} }
} }
Backend::KwinFakeInput => {
#[cfg(target_os = "linux")]
{
Ok(Box::new(kwin_fake_input::KwinFakeInjector::open()?))
}
#[cfg(not(target_os = "linux"))]
{
anyhow::bail!("KWin fake_input requires Linux + a KWin Wayland session")
}
}
Backend::Libei => { Backend::Libei => {
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
{ {
@@ -90,12 +103,18 @@ pub fn open(backend: Backend) -> Result<Box<dyn InputInjector>> {
/// Pick the injection backend for the current session. gamescope hosts its own EIS server (no /// Pick the injection backend for the current session. gamescope hosts its own EIS server (no
/// portal), so a gamescope session injects directly into it. wlroots/Sway only implements the /// portal), so a gamescope session injects directly into it. wlroots/Sway only implements the
/// ScreenCast portal (no RemoteDesktop), so libei can't run there — use the wlr virtual-input /// ScreenCast portal (no RemoteDesktop), so libei can't run there — use the wlr virtual-input
/// protocols. KWin and GNOME implement RemoteDesktop but not the wlr protocols, so use libei. /// protocols. **KWin** exposes `org_kde_kwin_fake_input` (direct injection, no portal / approval
/// `PUNKTFUNK_INPUT_BACKEND=wlr|libei|gamescope|uinput` overrides the auto-detection. /// dialog — the only headless-capable path; what krdpserver uses), so prefer it there. **GNOME**
/// has neither fake_input nor the wlr protocols, so it uses libei via the RemoteDesktop portal
/// (which needs a user to approve, or a pre-seeded grant — not truly headless).
/// `PUNKTFUNK_INPUT_BACKEND=wlr|kwin|libei|gamescope|uinput` overrides the auto-detection.
pub fn default_backend() -> Backend { pub fn default_backend() -> Backend {
if let Ok(v) = std::env::var("PUNKTFUNK_INPUT_BACKEND") { if let Ok(v) = std::env::var("PUNKTFUNK_INPUT_BACKEND") {
match v.trim().to_ascii_lowercase().as_str() { match v.trim().to_ascii_lowercase().as_str() {
"wlr" | "wlroots" | "wlrvirtual" => return Backend::WlrVirtual, "wlr" | "wlroots" | "wlrvirtual" => return Backend::WlrVirtual,
"kwin" | "fakeinput" | "fake_input" | "kwin-fake-input" => {
return Backend::KwinFakeInput
}
"libei" | "ei" | "portal" => return Backend::Libei, "libei" | "ei" | "portal" => return Backend::Libei,
"gamescope" | "gamescope-ei" => return Backend::GamescopeEi, "gamescope" | "gamescope-ei" => return Backend::GamescopeEi,
"uinput" => return Backend::Uinput, "uinput" => return Backend::Uinput,
@@ -112,16 +131,26 @@ pub fn default_backend() -> Backend {
} }
#[cfg(not(target_os = "windows"))] #[cfg(not(target_os = "windows"))]
{ {
if crate::config::config() // An explicit compositor pick (set per connect / mid-stream) is the strongest signal.
.compositor let compositor = crate::config::config().compositor.clone();
.as_deref() if let Some(c) = compositor.as_deref() {
.is_some_and(|v| v.trim().eq_ignore_ascii_case("gamescope")) let c = c.trim();
{ if c.eq_ignore_ascii_case("gamescope") {
return Backend::GamescopeEi; return Backend::GamescopeEi;
} }
if c.eq_ignore_ascii_case("kwin") {
return Backend::KwinFakeInput;
}
if c.eq_ignore_ascii_case("wlroots") || c.eq_ignore_ascii_case("sway") {
return Backend::WlrVirtual;
}
// mutter (GNOME) falls through to the XDG_CURRENT_DESKTOP check below.
}
let desktop = std::env::var("XDG_CURRENT_DESKTOP").unwrap_or_default(); let desktop = std::env::var("XDG_CURRENT_DESKTOP").unwrap_or_default();
let d = desktop.to_ascii_uppercase(); let d = desktop.to_ascii_uppercase();
if d.contains("KDE") || d.contains("GNOME") { if d.contains("KDE") {
Backend::KwinFakeInput
} else if d.contains("GNOME") {
Backend::Libei Backend::Libei
} else { } else {
Backend::WlrVirtual Backend::WlrVirtual
@@ -478,6 +507,9 @@ pub mod gamepad {
} }
} }
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
#[path = "inject/linux/kwin_fake_input.rs"]
mod kwin_fake_input;
#[cfg(target_os = "linux")]
#[path = "inject/linux/libei.rs"] #[path = "inject/linux/libei.rs"]
mod libei; mod libei;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
@@ -0,0 +1,209 @@
//! Headless input injection on KWin via the privileged `org_kde_kwin_fake_input` protocol — the
//! exact path KDE's own headless RDP server (`krdpserver`) uses. KWin advertises this restricted
//! global only to a client authorized through its installed `.desktop` `X-KDE-Wayland-Interfaces`
//! (we ship `io.unom.Punktfunk.Host.desktop`, which lists `org_kde_kwin_fake_input` alongside
//! `zkde_screencast_unstable_v1`). Binding the global IS the authorization, so injection needs **no
//! RemoteDesktop portal and no "Allow remote control?" dialog** — it works with no user present,
//! which the libei/portal path cannot. We connect as an ordinary Wayland client on the KWin session's
//! `$WAYLAND_DISPLAY` and translate events into fake-input requests; keyboard keys are raw Linux
//! evdev codes that KWin resolves through the session's own keymap (no keymap upload, unlike the wlr
//! virtual-keyboard path), and absolute pointer/touch coordinates are global compositor space — which
//! on a headless box (single per-session virtual output at the origin, scale 1) equals the streamed
//! output's pixels.
#![allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::{gs_button_to_evdev, vk_to_evdev, InputEvent, InputInjector};
use anyhow::{Context, Result};
use punktfunk_core::input::InputKind;
use wayland_client::protocol::wl_registry::{self, WlRegistry};
use wayland_client::{Connection, Dispatch, EventQueue, Proxy, QueueHandle};
// Generate the client bindings for the vendored protocol XML inline (no build.rs), exactly like the
// KWin virtual-output backend. Path is relative to CARGO_MANIFEST_DIR.
#[allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
pub mod fake {
use wayland_client;
use wayland_client::protocol::*;
pub mod __interfaces {
use wayland_client::protocol::__interfaces::*;
wayland_scanner::generate_interfaces!("protocols/fake-input.xml");
}
use self::__interfaces::*;
wayland_scanner::generate_client_code!("protocols/fake-input.xml");
}
use fake::org_kde_kwin_fake_input::OrgKdeKwinFakeInput as FakeInput;
/// Highest interface version we drive. `keyboard_key` arrived at v4; KWin advertises ≥4.
const MAX_VERSION: u32 = 4;
/// `wl_pointer.axis` values used by `axis`.
const AXIS_VERTICAL: u32 = 0;
const AXIS_HORIZONTAL: u32 = 1;
/// `code` value marking a horizontal scroll event (mirrors `gamestream::input` / the wlr backend).
const SCROLL_HORIZONTAL: u32 = 1;
/// Registry-bound globals (the Wayland dispatch state).
#[derive(Default)]
struct State {
fake: Option<FakeInput>,
}
impl Dispatch<WlRegistry, ()> for State {
fn event(
state: &mut Self,
registry: &WlRegistry,
event: wl_registry::Event,
_: &(),
_: &Connection,
qh: &QueueHandle<Self>,
) {
if let wl_registry::Event::Global {
name,
interface,
version,
} = event
{
if interface == "org_kde_kwin_fake_input" {
state.fake = Some(registry.bind(name, version.min(MAX_VERSION), qh, ()));
}
}
}
}
// fake_input emits no events.
impl Dispatch<FakeInput, ()> for State {
fn event(
_: &mut Self,
_: &FakeInput,
_: <FakeInput as Proxy>::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
pub struct KwinFakeInjector {
conn: Connection,
queue: EventQueue<State>,
state: State,
fake: FakeInput,
}
impl KwinFakeInjector {
pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env()
.context("connect to KWin Wayland (is WAYLAND_DISPLAY set to the KWin socket?)")?;
let mut queue = conn.new_event_queue();
let qh = queue.handle();
let _registry = conn.display().get_registry(&qh, ());
let mut state = State::default();
queue
.roundtrip(&mut state)
.context("Wayland registry roundtrip")?;
let fake = state.fake.clone().context(
"KWin does not expose org_kde_kwin_fake_input to this client — install the host's \
.desktop (io.unom.Punktfunk.Host.desktop, X-KDE-Wayland-Interfaces) and re-login so \
KWin authorizes it (the grant is cached per-exe on first connect), or this is not a \
KWin session",
)?;
// Authenticate (the legacy handshake; for an interface-authorized client KWin accepts it
// without a dialog — same as krdpserver/krfb headless).
fake.authenticate("punktfunk".into(), "remote streaming input".into());
queue
.roundtrip(&mut state)
.context("fake_input authenticate roundtrip")?;
conn.flush().ok();
tracing::info!("KWin fake_input ready (headless keyboard/mouse/touch — no portal)");
Ok(Self {
conn,
queue,
state,
fake,
})
}
}
impl InputInjector for KwinFakeInjector {
fn inject(&mut self, event: &InputEvent) -> Result<()> {
match event.kind {
InputKind::MouseMove => {
self.fake.pointer_motion(event.x as f64, event.y as f64);
}
InputKind::MouseMoveAbs => {
let w = (event.flags >> 16) & 0xffff;
let h = event.flags & 0xffff;
if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64;
let y = event.y.clamp(0, h as i32) as f64;
self.fake.pointer_motion_absolute(x, y);
}
}
InputKind::MouseButtonDown | InputKind::MouseButtonUp => {
if let Some(btn) = gs_button_to_evdev(event.code) {
let st = u32::from(event.kind == InputKind::MouseButtonDown);
self.fake.button(btn, st);
}
}
InputKind::MouseScroll => {
// GameStream sends WHEEL_DELTA(120)-scaled units; a notch ≈ 15px. Vertical flips
// sign on the Wayland axis, horizontal passes through — same as the wlr backend.
let horizontal = event.code == SCROLL_HORIZONTAL;
let axis = if horizontal {
AXIS_HORIZONTAL
} else {
AXIS_VERTICAL
};
let notches = event.x as f64 / 120.0;
let sign = if horizontal { 1.0 } else { -1.0 };
self.fake.axis(axis, sign * notches * 15.0);
}
InputKind::KeyDown | InputKind::KeyUp => {
// Raw evdev keycode; KWin resolves it through the session's own keymap (and tracks
// modifier state itself, so no separate modifiers request is needed).
if let Some(evdev) = vk_to_evdev(event.code as u8) {
let st = u32::from(event.kind == InputKind::KeyDown);
self.fake.keyboard_key(evdev as u32, st);
} else {
tracing::debug!(vk = event.code, "unmapped VK keycode — dropped");
}
}
// Touch: id = event.code, coords in the client surface w×h packed into flags (same
// absolute mapping as MouseMoveAbs). Each event is its own frame.
InputKind::TouchDown | InputKind::TouchMove => {
let w = (event.flags >> 16) & 0xffff;
let h = event.flags & 0xffff;
if w > 0 && h > 0 {
let x = event.x.clamp(0, w as i32) as f64;
let y = event.y.clamp(0, h as i32) as f64;
if event.kind == InputKind::TouchDown {
self.fake.touch_down(event.code, x, y);
} else {
self.fake.touch_motion(event.code, x, y);
}
self.fake.touch_frame();
}
}
InputKind::TouchUp => {
self.fake.touch_up(event.code);
self.fake.touch_frame();
}
// Gamepads are injected through uinput, not the compositor.
InputKind::GamepadButton | InputKind::GamepadAxis => {}
}
// Surface protocol errors / disconnects, then push the batch to the compositor.
self.queue
.dispatch_pending(&mut self.state)
.context("wayland dispatch")?;
self.conn.flush().context("wayland flush")?;
Ok(())
}
}
+105 -5
View File
@@ -2256,6 +2256,45 @@ struct SessionSwitch {
/// read (so no handshake plumbing). Opt-in via `PUNKTFUNK_SESSION_WATCH`; readiness of the new /// read (so no handshake plumbing). Opt-in via `PUNKTFUNK_SESSION_WATCH`; readiness of the new
/// backend is left to the encode thread's `build_pipeline_with_retry` (the watcher never writes /// backend is left to the encode thread's `build_pipeline_with_retry` (the watcher never writes
/// env). Exits when `stop` is set or the channel closes. /// env). Exits when `stop` is set or the channel closes.
/// Whether to run the mid-stream session-switch watcher. An explicit `PUNKTFUNK_SESSION_WATCH` wins
/// (truthy → on; `0`/`false`/`no`/`off`/empty → off). When unset it defaults **on** for Steam HTPC
/// platforms (Bazzite / SteamOS) — which flip Gaming↔Desktop and need the host to follow the switch
/// mid-stream — and **off** elsewhere, preserving the opt-in default for plain desktop hosts.
fn session_watch_enabled() -> bool {
match std::env::var("PUNKTFUNK_SESSION_WATCH") {
Ok(v) => {
let v = v.trim();
!(v.is_empty()
|| v == "0"
|| v.eq_ignore_ascii_case("false")
|| v.eq_ignore_ascii_case("no")
|| v.eq_ignore_ascii_case("off"))
}
Err(_) => is_steam_htpc_platform(),
}
}
/// True on Bazzite or SteamOS (matched against os-release `ID`/`ID_LIKE`) — the platforms that flip
/// between Steam Gaming Mode and a Desktop session, where following a mid-stream switch is the
/// sensible default. Anything else (incl. non-Linux, where the file is absent) → false.
fn is_steam_htpc_platform() -> bool {
let Ok(os) = std::fs::read_to_string("/etc/os-release") else {
return false;
};
os.lines().any(|line| {
let line = line.trim();
let Some(val) = line
.strip_prefix("ID=")
.or_else(|| line.strip_prefix("ID_LIKE="))
else {
return false;
};
val.trim_matches('"')
.split_whitespace()
.any(|tok| tok.eq_ignore_ascii_case("bazzite") || tok.eq_ignore_ascii_case("steamos"))
})
}
fn session_watcher_loop(tx: std::sync::mpsc::Sender<SessionSwitch>, stop: Arc<AtomicBool>) { fn session_watcher_loop(tx: std::sync::mpsc::Sender<SessionSwitch>, stop: Arc<AtomicBool>) {
use crate::vdisplay; use crate::vdisplay;
const DEBOUNCE: std::time::Duration = std::time::Duration::from_secs(3); const DEBOUNCE: std::time::Duration = std::time::Duration::from_secs(3);
@@ -2491,9 +2530,9 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
// place when the box flips Gaming↔Desktop. When not spawned, session_rx just stays empty. // place when the box flips Gaming↔Desktop. When not spawned, session_rx just stays empty.
let mut compositor = compositor; let mut compositor = compositor;
let (session_tx, session_rx) = std::sync::mpsc::channel::<SessionSwitch>(); let (session_tx, session_rx) = std::sync::mpsc::channel::<SessionSwitch>();
let watch = std::env::var_os("PUNKTFUNK_SESSION_WATCH").is_some() let watch = session_watch_enabled() && crate::config::config().compositor.is_none();
&& crate::config::config().compositor.is_none();
let _watcher = if watch { let _watcher = if watch {
tracing::info!("session watcher on — following a mid-stream Gaming↔Desktop switch");
let stop = stop.clone(); let stop = stop.clone();
std::thread::Builder::new() std::thread::Builder::new()
.name("punktfunk1-watcher".into()) .name("punktfunk1-watcher".into())
@@ -2675,15 +2714,76 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
} }
tracing::warn!(error = %format!("{e:#}"), rebuild = capture_rebuilds, tracing::warn!(error = %format!("{e:#}"), rebuild = capture_rebuilds,
"capture lost — rebuilding pipeline in place"); "capture lost — rebuilding pipeline in place");
let (new_cap, new_enc, new_frame, new_interval) = // A Bazzite/SteamOS Gaming↔Desktop switch tears the old compositor down and can take
build_pipeline_with_retry(&mut vd, cur_mode, bitrate_kbps, bit_depth, plan) // 15s+ to bring the new one up. Don't fail the session over that (the client would
.context("rebuild after capture loss")?; // have to cold-reconnect, surfacing a "session failed") — keep retrying within a
// generous budget while the QUIC keepalive (its own thread) holds the connection,
// RE-DETECTING the live compositor each attempt so we follow the box to whatever
// session comes up: a fresh instance of the same compositor, OR a different one
// (the kind-change case the session watcher also handles). The client stays
// connected, frozen on the last frame, and the stream resumes when the new output
// appears — no reconnect.
const REBUILD_BUDGET: std::time::Duration = std::time::Duration::from_secs(40);
let rebuild_deadline = std::time::Instant::now() + REBUILD_BUDGET;
let (new_cap, new_enc, new_frame, new_interval) = loop {
// Follow the active session unless an explicit PUNKTFUNK_COMPOSITOR pin forbids
// retargeting (then we stick to the pinned backend and just rebuild it).
if crate::config::config().compositor.is_none() {
let active = crate::vdisplay::detect_active_session();
if let Some(c) = crate::vdisplay::compositor_for_kind(active.kind) {
crate::vdisplay::apply_session_env(&active);
crate::vdisplay::apply_input_env(c);
if c != compositor {
if matches!(
c,
crate::vdisplay::Compositor::Kwin
| crate::vdisplay::Compositor::Mutter
) {
crate::vdisplay::settle_desktop_portal(c);
}
match crate::vdisplay::open(c) {
Ok(v) => {
tracing::info!(from = compositor.id(), to = c.id(),
"capture loss: active session switched compositor — retargeting");
vd = v;
compositor = c;
}
Err(e2) => tracing::warn!(error = %format!("{e2:#}"),
"capture loss: opening the newly-detected compositor failed — retrying"),
}
}
}
}
match build_pipeline_with_retry(
&mut vd,
cur_mode,
bitrate_kbps,
bit_depth,
plan,
) {
Ok(p) => break p,
Err(e2) => {
if stop.load(Ordering::SeqCst)
|| std::time::Instant::now() >= rebuild_deadline
{
return Err(e2)
.context("capture lost — no compositor came up within the rebuild budget");
}
tracing::warn!(error = %format!("{e2:#}"),
"capture lost — new session not up yet, retrying");
}
}
};
capturer = new_cap; capturer = new_cap;
enc = new_enc; enc = new_enc;
frame = new_frame; frame = new_frame;
interval = new_interval; interval = new_interval;
enc.request_keyframe(); // belt-and-suspenders; a fresh encoder opens on an IDR anyway enc.request_keyframe(); // belt-and-suspenders; a fresh encoder opens on an IDR anyway
next = std::time::Instant::now(); next = std::time::Instant::now();
tracing::info!(
compositor = compositor.id(),
"capture loss: pipeline rebuilt — stream resumes"
);
} }
} }
if perf && diag_at.elapsed() >= std::time::Duration::from_secs(2) { if perf && diag_at.elapsed() >= std::time::Duration::from_secs(2) {
+5 -1
View File
@@ -457,7 +457,11 @@ pub fn settle_desktop_portal(_chosen: Compositor) {}
pub fn apply_input_env(chosen: Compositor) { pub fn apply_input_env(chosen: Compositor) {
let backend = match chosen { let backend = match chosen {
Compositor::Gamescope => "gamescope", Compositor::Gamescope => "gamescope",
Compositor::Kwin | Compositor::Mutter => "libei", // KWin: org_kde_kwin_fake_input — direct injection, no RemoteDesktop portal / approval
// dialog (headless, the krdpserver path), authorized by the host's shipped .desktop.
Compositor::Kwin => "kwin",
// GNOME has neither fake_input nor the wlr protocols → RemoteDesktop portal via libei.
Compositor::Mutter => "libei",
Compositor::Wlroots => "wlr", Compositor::Wlroots => "wlr",
}; };
std::env::set_var("PUNKTFUNK_INPUT_BACKEND", backend); std::env::set_var("PUNKTFUNK_INPUT_BACKEND", backend);
@@ -15,7 +15,7 @@
//! `inject/libei.rs`) — wired and live-validated. //! `inject/libei.rs`) — wired and live-validated.
use super::{Mode, VirtualDisplay, VirtualOutput}; use super::{Mode, VirtualDisplay, VirtualOutput};
use anyhow::{anyhow, Context, Result}; use anyhow::{anyhow, bail, Context, Result};
use std::process::{Child, Command, Stdio}; use std::process::{Child, Command, Stdio};
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
@@ -110,12 +110,11 @@ impl VirtualDisplay for GamescopeDisplay {
// PUNKTFUNK_GAMESCOPE_NODE=<id|auto>; "auto" discovers the gamescope `Video/Source` node. // PUNKTFUNK_GAMESCOPE_NODE=<id|auto>; "auto" discovers the gamescope `Video/Source` node.
if let Ok(id) = std::env::var("PUNKTFUNK_GAMESCOPE_NODE") { if let Ok(id) = std::env::var("PUNKTFUNK_GAMESCOPE_NODE") {
let node_id: u32 = if id.trim().eq_ignore_ascii_case("auto") { let node_id: u32 = if id.trim().eq_ignore_ascii_case("auto") {
find_gamescope_node().ok_or_else(|| { // Attach to the box-owned game-mode session, but FIRST make it run at the connecting
anyhow!( // client's resolution (the box is headless, so its game-mode mode is ours to set).
"PUNKTFUNK_GAMESCOPE_NODE=auto but no running gamescope Video/Source node \ // Reuse if it already matches (fast, no restart); otherwise relaunch the box's own
was found — is the headless gamescope/Steam session up?" // session at the client mode. Without this the client gets the box's default mode.
) ensure_box_gamescope_mode(mode)?
})?
} else { } else {
id.parse() id.parse()
.context("PUNKTFUNK_GAMESCOPE_NODE must be a node id or 'auto'")? .context("PUNKTFUNK_GAMESCOPE_NODE must be a node id or 'auto'")?
@@ -368,6 +367,150 @@ fn create_managed_session_steamos(mode: Mode) -> Result<VirtualOutput> {
}) })
} }
/// ATTACH at the CLIENT's resolution: ensure the box's own game-mode session is running at `mode`'s
/// output size, then return its capture node. Reuses the running session if it already matches (no
/// restart — the rock-solid fast path a stable client always hits); otherwise reconfigures + restarts
/// the box's OWN autologin `gamescope-session-plus@<client>` unit at the client mode. Restarting the
/// box's own unit (rather than spawning a competing one) avoids the autologin-respawn fight the old
/// MANAGED path hit. A headless box has no physical panel, so its game-mode resolution is ours to set;
/// Steam restarts only on an actual resolution CHANGE.
fn ensure_box_gamescope_mode(mode: Mode) -> Result<u32> {
let target = (mode.width, mode.height);
// Fast path: already at the client's resolution — just attach to the live node.
if current_gamescope_output_size() == Some(target) {
if let Some(node) = find_gamescope_node() {
tracing::info!(
w = mode.width,
h = mode.height,
node,
"gamescope: box game-mode session already at the client's resolution — reusing"
);
return Ok(node);
}
}
let Some(unit) = running_autologin_gamescope_unit() else {
// No box-owned autologin session to reconfigure (a bare/foreign gamescope): attach to
// whatever node exists, accepting its resolution.
return find_gamescope_node().ok_or_else(|| {
anyhow!(
"no running gamescope Video/Source node — is the headless game mode up? \
(put the box into Steam Game Mode)"
)
});
};
tracing::info!(
from = ?current_gamescope_output_size(),
to_w = mode.width,
to_h = mode.height,
hz = mode.refresh_hz,
%unit,
"gamescope: relaunching the box game-mode session at the client's resolution"
);
// The session reads SCREEN_WIDTH/HEIGHT (+ CUSTOM_REFRESH_RATES) from the user-manager
// environment; set them and restart the box's own unit.
systemctl_user(&[
"set-environment",
&format!("SCREEN_WIDTH={}", mode.width),
&format!("SCREEN_HEIGHT={}", mode.height),
&format!("CUSTOM_REFRESH_RATES={}", mode.refresh_hz.max(1)),
]);
systemctl_user(&["restart", &unit]);
// Wait for the relaunched session to come up at the new size and publish its capture node. The
// node appears when gamescope is up (well before Steam finishes booting); the caller's
// first-frame retry absorbs Steam's cold start.
let deadline = Instant::now() + Duration::from_secs(45);
loop {
if current_gamescope_output_size() == Some(target) {
if let Some(node) = find_gamescope_node() {
tracing::info!(
node,
w = mode.width,
h = mode.height,
"gamescope: box game-mode session relaunched at the client's resolution"
);
return Ok(node);
}
}
if Instant::now() >= deadline {
bail!(
"box game-mode session did not come up at {}x{} within 45s after relaunch \
(Steam may still be booting)",
mode.width,
mode.height
);
}
std::thread::sleep(Duration::from_millis(500));
}
}
/// Output (capture) resolution `-W <w> -H <h>` of the running `gamescope` binary, parsed from its
/// `/proc/<pid>/cmdline`. `None` if no gamescope is running or the flags aren't present.
fn current_gamescope_output_size() -> Option<(u32, u32)> {
for entry in std::fs::read_dir("/proc").ok()?.flatten() {
let name = entry.file_name();
let Some(pid) = name.to_str() else { continue };
if !pid.bytes().all(|b| b.is_ascii_digit()) {
continue;
}
let Ok(raw) = std::fs::read(format!("/proc/{pid}/cmdline")) else {
continue;
};
let args: Vec<String> = raw
.split(|&b| b == 0)
.filter(|s| !s.is_empty())
.map(|s| String::from_utf8_lossy(s).into_owned())
.collect();
// Match the gamescope BINARY by argv[0]'s basename — NOT /proc/<pid>/exe, which is commonly
// unreadable for the gamescope process (returns empty). The session wrapper scripts run as
// bash/sh (argv[0] != gamescope), so they're excluded; the -W/-H presence check below is the
// final filter.
let is_gamescope = args
.first()
.map(|a0| a0.rsplit('/').next().unwrap_or(a0) == "gamescope")
.unwrap_or(false);
if !is_gamescope {
continue;
}
let flag = |names: &[&str]| -> Option<u32> {
args.iter().enumerate().find_map(|(i, a)| {
names
.contains(&a.as_str())
.then(|| args.get(i + 1).and_then(|v| v.parse().ok()))
.flatten()
})
};
if let (Some(w), Some(h)) = (
flag(&["-W", "--output-width"]),
flag(&["-H", "--output-height"]),
) {
return Some((w, h));
}
}
None
}
/// The running autologin gaming-mode unit (`gamescope-session-plus@<client>.service`), if any — the
/// box's own game-mode session, which [`ensure_box_gamescope_mode`] reconfigures + restarts.
fn running_autologin_gamescope_unit() -> Option<String> {
let out = Command::new("systemctl")
.args([
"--user",
"list-units",
"--type=service",
"--state=running",
"--no-legend",
"--plain",
"gamescope-session-plus@*.service",
])
.output()
.ok()?;
String::from_utf8_lossy(&out.stdout)
.lines()
.filter_map(|l| l.split_whitespace().next())
.find(|u| u.starts_with("gamescope-session-plus@") && u.ends_with(".service"))
.map(|u| u.to_string())
}
/// Stop every running autologin gaming-mode session (`gamescope-session-plus@*.service`) so its /// Stop every running autologin gaming-mode session (`gamescope-session-plus@*.service`) so its
/// single-instance Steam is free for our own host-managed session. Records the units so /// single-instance Steam is free for our own host-managed session. Records the units so
/// [`schedule_restore_tv_session`] can restart them on disconnect. Our own session is the transient /// [`schedule_restore_tv_session`] can restart them on disconnect. Our own session is the transient
@@ -6,8 +6,14 @@
//! node for it. The node lives on the user's default PipeWire daemon, so [`VirtualOutput::remote_fd`] //! node for it. The node lives on the user's default PipeWire daemon, so [`VirtualOutput::remote_fd`]
//! is `None` and capture connects to that daemon directly. //! is `None` and capture connects to that daemon directly.
//! //!
//! Requirements: KWin must expose the privileged `zkde_screencast` global — a real Plasma session //! Requirements: KWin must expose the privileged `zkde_screencast` global. It is a *restricted*
//! authorizes it for its own clients; the headless test exposes it to bare clients via //! protocol — KWin advertises it only to a client whose installed `.desktop` lists it under
//! `X-KDE-Wayland-Interfaces` (KWin maps the connecting client to a `.desktop` by resolving
//! `/proc/<pid>/exe` against `Exec=`, then caches the grant per-executable for the session's life).
//! So an interactive Plasma session does NOT hand it to a bare client — the host packages ship
//! `io.unom.Punktfunk.Host.desktop` (`Exec=/usr/bin/punktfunk-host`,
//! `X-KDE-Wayland-Interfaces=zkde_screencast_unstable_v1,…`) so it is present before the host first
//! connects. The headless test path instead exposes it to bare clients via
//! `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1`. The compositor backend must implement //! `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1`. The compositor backend must implement
//! `createVirtualOutput`: the **DRM backend** (any version) or the **VirtualBackend since KWin //! `createVirtualOutput`: the **DRM backend** (any version) or the **VirtualBackend since KWin
//! 6.5.6** (`kwin_wayland --virtual`); on `--virtual` < 6.5.6 the request fails with //! 6.5.6** (`kwin_wayland --virtual`); on `--virtual` < 6.5.6 the request fails with
@@ -406,9 +412,11 @@ pub fn probe() -> Result<()> {
queue.roundtrip(&mut state).context("registry roundtrip")?; queue.roundtrip(&mut state).context("registry roundtrip")?;
if state.screencast.is_none() { if state.screencast.is_none() {
bail!( bail!(
"KWin is up but does not (yet) expose zkde_screencast_unstable_v1 — needs a real \ "KWin is up but does not expose zkde_screencast_unstable_v1 to this client — KWin gates \
KDE session (or KWIN_WAYLAND_NO_PERMISSION_CHECKS=1), and KWin ≥ 6.5.6 for the \ it on the host's .desktop X-KDE-Wayland-Interfaces (install \
headless virtual output" io.unom.Punktfunk.Host.desktop with Exec=/usr/bin/punktfunk-host, then re-login so KWin \
re-reads it — the grant is cached per-exe on first connect), or set \
KWIN_WAYLAND_NO_PERMISSION_CHECKS=1 for the headless test; needs KWin ≥ 6.5.6"
); );
} }
Ok(()) Ok(())
@@ -437,8 +445,9 @@ fn run(
let screencast = state.screencast.clone().ok_or_else(|| { let screencast = state.screencast.clone().ok_or_else(|| {
anyhow!( anyhow!(
"KWin does not expose zkde_screencast_unstable_v1 (need a real KDE session, or run \ "KWin does not expose zkde_screencast_unstable_v1 to this client — install the host's \
KWin with KWIN_WAYLAND_NO_PERMISSION_CHECKS=1 for the headless test)" .desktop (io.unom.Punktfunk.Host.desktop, X-KDE-Wayland-Interfaces) and re-login so \
KWin authorizes it, or run KWin with KWIN_WAYLAND_NO_PERMISSION_CHECKS=1 (headless test)"
) )
})?; })?;
+3 -3
View File
@@ -43,12 +43,12 @@ signed installer — see [Windows Host](/docs/windows-host) for what it includes
``` ```
3. Run `punktfunk-host-setup-<ver>.exe` (elevated). It installs to `C:\Program Files\punktfunk`, 3. Run `punktfunk-host-setup-<ver>.exe` (elevated). It installs to `C:\Program Files\punktfunk`,
optionally installs the bundled **SudoVDA** virtual-display driver, and registers + starts the installs the bundled **pf-vdisplay** virtual-display driver, and registers + starts the
`LocalSystem` service (`/VERYSILENT` for an unattended install). Upgrades and uninstall go through `LocalSystem` service (`/VERYSILENT` for an unattended install). Upgrades and uninstall go through
Add/Remove Programs. Add/Remove Programs.
You need an NVIDIA GPU + driver (the host is NVENC-only on Windows). More detail — including the CLI For hardware encode you need a GPU — NVIDIA (NVENC), AMD (AMF), or Intel (QSV); there's a software
`punktfunk-host service install` path — is in fallback without one. More detail — including the CLI `punktfunk-host service install` path — is in
[Running as a Service → Windows](/docs/running-as-a-service#windows). [Running as a Service → Windows](/docs/running-as-a-service#windows).
## What the packages are ## What the packages are
+4 -3
View File
@@ -19,9 +19,10 @@ environments it supports today, each with its own guide:
Other wlroots compositors (Sway/Hyprland) also work but aren't a primary target. If your desktop isn't Other wlroots compositors (Sway/Hyprland) also work but aren't a primary target. If your desktop isn't
listed, the host still needs one of these compositor backends to create a virtual display. listed, the host still needs one of these compositor backends to create a virtual display.
> **Windows host:** punktfunk also runs as a native host on **Windows 10/11 (x64) with an NVIDIA GPU** > **Windows host:** punktfunk also runs as a native host on **Windows 10/11 (x64)** — a signed
> — a signed installer that registers a service and bundles a virtual-display driver. It's NVIDIA-only > installer that registers a service and bundles a virtual-display driver. It encodes on NVIDIA
> and newer than the Linux host; see [Windows Host](/docs/windows-host). > (NVENC), AMD (AMF), or Intel (QSV), with a software fallback, and is newer than the Linux host; see
> [Windows Host](/docs/windows-host).
## GPU and driver ## GPU and driver
+3 -3
View File
@@ -35,7 +35,7 @@ see [Status & Progress](/docs/status).
from one process. from one process.
- **Native-resolution virtual displays** on Linux across KWin, GNOME/Mutter, gamescope, and - **Native-resolution virtual displays** on Linux across KWin, GNOME/Mutter, gamescope, and
Sway/wlroots, with a fully zero-copy GPU path to NVENC (stable 240 fps at 5120×1440). Sway/wlroots, with a fully zero-copy GPU path to NVENC (stable 240 fps at 5120×1440).
- **A native Windows host** (NVIDIA, x64) — a signed installer with secure-desktop capture and a - **A native Windows host** (x64; NVIDIA/AMD/Intel encode) — a signed installer with secure-desktop capture and a
bundled virtual-display driver, and the only host that can stream **HDR** (10-bit BT.2020 PQ, bundled virtual-display driver, and the only host that can stream **HDR** (10-bit BT.2020 PQ,
captured from an HDR Windows desktop and encoded as HEVC Main10). See captured from an HDR Windows desktop and encoded as HEVC Main10). See
[Windows Host](/docs/windows-host). *(Beta — newer than the Linux host.)* [Windows Host](/docs/windows-host). *(Beta — newer than the Linux host.)*
@@ -55,8 +55,8 @@ see [Status & Progress](/docs/status).
- **Apple stage-2 presenter as the default.** The lower-latency `VTDecompressionSession` - **Apple stage-2 presenter as the default.** The lower-latency `VTDecompressionSession`
`CAMetalLayer` path is live behind an opt-in flag and graduating to the default. `CAMetalLayer` path is live behind an opt-in flag and graduating to the default.
- **Web console parity.** Surfacing the speed test and bitrate picker the apps already have. - **Web console parity.** Surfacing the speed test and bitrate picker the apps already have.
- **Windows host hardening.** Broader real-world testing, AMD/Intel encode (NVIDIA-only today), and - **Windows host hardening.** Broader real-world testing — especially on-glass validation of the
bundling the ViGEm gamepad driver. AMD (AMF) and Intel (QSV) encode paths, which are CI-green but newer than NVENC.
## 🔭 Planned ## 🔭 Planned
@@ -95,13 +95,14 @@ model Sunshine/Apollo use.
The easy path is the **signed installer**: download `punktfunk-host-setup-<ver>.exe` from the package The easy path is the **signed installer**: download `punktfunk-host-setup-<ver>.exe` from the package
registry ([`punktfunk-host-windows`](https://git.unom.io/unom/-/packages)) and run it. It drops the host registry ([`punktfunk-host-windows`](https://git.unom.io/unom/-/packages)) and run it. It drops the host
into `C:\Program Files\punktfunk`, optionally installs the bundled **SudoVDA** virtual-display driver, into `C:\Program Files\punktfunk`, installs the bundled **pf-vdisplay** virtual-display driver, and
and registers + starts the service for you (`/VERYSILENT` for unattended). Upgrades and uninstall are registers + starts the service for you (`/VERYSILENT` for unattended). Upgrades and uninstall are
handled through Add/Remove Programs. handled through Add/Remove Programs.
Prefer the CLI? Run `punktfunk-host service install` from an elevated prompt — see Prefer the CLI? Run `punktfunk-host service install` from an elevated prompt — see
[Windows service](https://git.unom.io/unom/punktfunk/src/branch/main/docs/windows-service.md). Either [Windows service](https://git.unom.io/unom/punktfunk/src/branch/main/docs/windows-service.md). For
way you need an NVIDIA GPU + driver (the host is NVENC-only on Windows). hardware encode you need a GPU — NVIDIA (NVENC), AMD (AMF), or Intel (QSV); the host falls back to
software H.264 without one.
## Verifying ## Verifying
+1 -1
View File
@@ -14,7 +14,7 @@ A high-level view of where punktfunk stands. The ordered plan of work is on the
| **Core**`punktfunk-core` + C ABI (protocol · FEC · crypto) | ✅ complete & hardened | | **Core**`punktfunk-core` + C ABI (protocol · FEC · crypto) | ✅ complete & hardened |
| **GameStream host** (Moonlight-compatible) | ✅ working end-to-end; HDR/surround-audio polish open | | **GameStream host** (Moonlight-compatible) | ✅ working end-to-end; HDR/surround-audio polish open |
| **Native protocol**`punktfunk/1` (QUIC control + UDP data, GF(2¹⁶) Leopard FEC + AES-GCM) | ✅ full session planes, validated live | | **Native protocol**`punktfunk/1` (QUIC control + UDP data, GF(2¹⁶) Leopard FEC + AES-GCM) | ✅ full session planes, validated live |
| **Windows host** (NVIDIA, x64) | 🟡 implemented & shipping as a signed installer; NVIDIA-only, newer than the Linux host | | **Windows host** (x64) | 🟡 implemented & shipping as a signed installer; NVIDIA/AMD/Intel encode, newer than the Linux host |
| **macOS / iOS / iPadOS / tvOS client** | ✅ full client; on-glass stage-2 presenter behind an opt-in flag, becoming the default | | **macOS / iOS / iPadOS / tvOS client** | ✅ full client; on-glass stage-2 presenter behind an opt-in flag, becoming the default |
| **Linux client** (`punktfunk-client`, GTK4/libadwaita) | ✅ full client; VAAPI zero-copy decode + software fallback | | **Linux client** (`punktfunk-client`, GTK4/libadwaita) | ✅ full client; VAAPI zero-copy decode + software fallback |
| **Windows client** (`punktfunk-client`, WinUI 3) | ✅ stage 1 complete; ships as signed MSIX; on-glass hardware validation pending | | **Windows client** (`punktfunk-client`, WinUI 3) | ✅ stage 1 complete; ships as signed MSIX; on-glass hardware validation pending |
+86 -40
View File
@@ -1,45 +1,78 @@
--- ---
title: "Windows Host" title: "Windows Host"
description: "Run the punktfunk streaming host on a Windows PC — a first-class, virtual-display host." description: "Run the punktfunk streaming host on a Windows PC — a first-class, all-vendor, virtual-display host."
--- ---
Set up a punktfunk host on a **Windows 10/11 PC** and stream its desktop or games to any punktfunk or
[Moonlight](/docs/moonlight) client. A signed installer registers a Windows service that streams at the
client's **exact resolution and refresh** via punktfunk's own **virtual display** — including
**HDR10** (10-bit BT.2020 PQ) when your Windows desktop is in HDR mode. The virtual display is created
on the fly, so you need **no second monitor and no dummy HDMI plug**, and capture keeps working even on
the secure desktop (UAC prompts, the lock screen).
**Status: implemented and shipping — x64-only.** Alongside the Linux host, punktfunk runs as a > New to this? Skim [Requirements](/docs/requirements) first.
first-class native **Windows host**: a signed installer registers a `LocalSystem` service that streams
your Windows desktop or games to any punktfunk or Moonlight client, at the client's exact resolution
via a **virtual display** — including **HDR10** (10-bit BT.2020 PQ) when your Windows desktop is in HDR
mode. punktfunk has its own **indirect display driver (IDD)** that the host pushes finished frames
straight into, so you get a real on-the-fly virtual display with no physical monitor or dummy HDMI
plug — even on the secure desktop (UAC / lock screen). The Windows host is newer and less
battle-tested than the Linux host. (The Linux host is 8-bit only — HDR there is blocked upstream.)
> This page is about the Windows **host** (streaming *from* a Windows PC). To stream *to* a Windows > This page is about the Windows **host** streaming *from* a Windows PC. To stream *to* a Windows PC,
> PC, see the [Windows client](/docs/clients#windows-desktop-client). > see the [Windows client](/docs/clients#windows-desktop-client).
## Requirements ## Requirements
- **Windows 10/11, x64.** ARM64 is not supported — both NVENC and the virtual-display driver are - **Windows 10 or 11, x64.** ARM64 is not built (no ARM64 NVIDIA driver, and the virtual-display
x64-only. driver is x64-only).
- **An NVIDIA GPU + driver.** The host encodes with NVENC (`nvEncodeAPI64.dll`); there is no other - **A GPU for hardware encode** — the host auto-detects the vendor:
encoder backend on Windows. - **NVIDIA** → NVENC
- **(Optional) ViGEmBus** for virtual gamepads — a manual prerequisite for now - **AMD** → AMF
([releases](https://github.com/nefarius/ViGEmBus/releases)). - **Intel** → QSV
No discrete GPU? The host falls back to a **software H.264** encoder (higher CPU use, lower quality —
fine for light desktop use).
- **No gamepad prerequisite.** The virtual gamepad drivers are bundled in the installer — there is
nothing else to download. (Earlier builds needed ViGEmBus; it is no longer used.)
## Install ## Install
Download the signed `punktfunk-host-setup-<ver>.exe` from the package registry and run it — it Download the signed `punktfunk-host-setup-<ver>.exe` from the
installs the host into `C:\Program Files\punktfunk`, optionally installs the bundled **SudoVDA** [package registry](https://git.unom.io/unom/-/packages) and run it. The installer:
virtual-display driver, and registers + starts the service. Full steps (including the silent install
and the CLI `punktfunk-host service install` path) are in - drops the host into `C:\Program Files\punktfunk` and registers + starts the **`PunktfunkHost`**
[Running as a Service → Windows](/docs/running-as-a-service#windows); packaging internals live in service,
- installs the bundled **virtual-display driver** (`pf-vdisplay`) so the host can create per-client
displays,
- installs the bundled **virtual gamepad drivers** (DualSense, DualShock 4, Xbox 360),
- registers the bundled **HDR Vulkan layer** so Vulkan games can enable HDR over the virtual display,
- sets up the **web management console** (see below).
For an unattended install, append `/VERYSILENT`. Upgrades and uninstall go through **Add/Remove
Programs**; your config and pairings are kept across upgrades. Prefer the CLI, or want the full
service/firewall details? See [Running as a Service → Windows](/docs/running-as-a-service#windows).
Packaging internals live in
[`packaging/windows`](https://git.unom.io/unom/punktfunk/src/branch/main/packaging/windows/README.md). [`packaging/windows`](https://git.unom.io/unom/punktfunk/src/branch/main/packaging/windows/README.md).
### Web console & pairing
The installer also sets up the **web management console** (status, paired devices, the PIN pairing The installer also sets up the **web management console** (status, paired devices, the PIN pairing
flow): it bundles the console plus its own bun runtime and runs it as the **`PunktfunkWeb`** service flow): it bundles the console plus its own runtime and runs it as the **`PunktfunkWeb`** service on
on **`http://<this-PC>:3000`**, starting at boot. During setup you choose the console **login **`http://<this-PC>:3000`**, starting at boot. During setup you choose the console **login password**
password** (pre-filled with a secure random default and shown again on the final page); change it (pre-filled with a secure random default and shown again on the final page); change it later in
later in `%ProgramData%\punktfunk\web-password`. Open the console from any browser on the LAN and log `%ProgramData%\punktfunk\web-password`.
in — no extra install, and the host's management API stays loopback-only behind it.
The host **requires PIN pairing** by default (secure on a LAN). To connect the first time, open the
console from any browser on the LAN, log in, go to **Devices → arm pairing**, and enter the PIN on
your [client](/docs/clients). The host's own management API stays loopback-only behind the console.
### Configure
The service reads `%ProgramData%\punktfunk\host.env`. The defaults work out of the box; common knobs:
- `PUNKTFUNK_ENCODER=auto``auto` picks NVENC/AMF/QSV by GPU vendor. Force one with `nvenc`, `amf`,
`qsv`, or `sw` (software).
- `PUNKTFUNK_HOST_CMD` — the service runs `serve --gamestream` by default (native punktfunk/1 **plus**
the GameStream/Moonlight-compat planes). Set it to `serve` for a **secure native-only** host with no
GameStream surface (GameStream pairs over plain HTTP and uses weaker legacy encryption — trusted LAN
only).
Edit the file, then restart: `punktfunk-host service stop` / `punktfunk-host service start`. See the
[Configuration reference](/docs/configuration) for every option.
## How it works ## How it works
@@ -58,23 +91,36 @@ pipeline orchestration are all shared with the Linux host. The Windows host is a
| Subsystem | Linux backend | Windows backend | | Subsystem | Linux backend | Windows backend |
|---|---|---| |---|---|---|
| **Capture** | xdg ScreenCast portal → PipeWire (dmabuf) | **Windows.Graphics.Capture** (+ Desktop Duplication for the secure desktop) → D3D11 texture; FP16/10-bit when the desktop is HDR | | **Capture** | xdg ScreenCast portal → PipeWire (dmabuf) | **Windows.Graphics.Capture** + **Desktop Duplication** (secure desktop), with a zero-copy path straight from the virtual-display driver; FP16/10-bit when the desktop is HDR |
| **Virtual display** | KWin / Mutter / Sway / gamescope | **SudoVDA** signed IDD — create a `WxH@Hz` monitor per session, capture it, tear it down | | **Virtual display** | KWin / Mutter / Sway / gamescope | **pf-vdisplay** signed IDD — create a `WxH@Hz` monitor per session, capture it, tear it down |
| **Encode** | `ffmpeg-next` NVENC (CUDA hwframes) | **NVENC** with a D3D11 device (`--features nvenc`); HEVC Main10 / BT.2020 PQ for HDR | | **Encode** | NVENC (CUDA) / VAAPI (AMD·Intel) / software | **NVENC** (NVIDIA) · **AMF** (AMD) · **QSV** (Intel) · software H.264; HEVC Main10 / BT.2020 PQ for HDR |
| **Input — mouse/keyboard** | libei / wlr protocols | **SendInput** (Win32 VK + absolute mouse) | | **Input — mouse/keyboard** | libei / wlr protocols | **SendInput** (Win32 VK + absolute mouse) |
| **Input — gamepads** | uinput Xbox 360 pad + rumble | **ViGEm** virtual pad + rumble back-channel | | **Input — gamepads** | uinput Xbox 360 + UHID DualSense/DS4 | **UMDF** virtual pads — DualSense, DualShock 4, Xbox 360 (XUSB) + rumble |
| **Audio capture** | PipeWire sink-monitor | **WASAPI loopback** | | **Audio capture** | PipeWire sink-monitor | **WASAPI loopback** |
| **Virtual mic** | PipeWire `Audio/Source` | WASAPI virtual mic | | **Virtual mic** | PipeWire `Audio/Source` | WASAPI virtual mic |
The virtual display uses **[SudoVDA](https://github.com/VirtualDrivers)** (the Sunshine Virtual The virtual display uses **pf-vdisplay**, punktfunk's own all-Rust **Indirect Display Driver (IDD)**
Display Adapter) — a pre-built, signed Indirect Display Driver — so there is **no kernel driver to the host pushes finished frames straight into it, so you get a real virtual display with no physical
author or WHQL-sign**. The installer bundles and stages it; if it's absent, the host falls back to monitor or dummy plug. The installer bundles and stages the (self-signed) driver; if it isn't
capturing an existing monitor (losing the per-client native-resolution output). installed, the host falls back to capturing an existing monitor, losing the per-client native-resolution
output.
## Limitations ### HDR
- **NVIDIA-only.** NVENC is the only encoder backend — there is no AMD / Intel / software encode path When your Windows desktop is in **HDR** mode, the host captures it as 10-bit, encodes **HEVC Main10 /
on Windows. BT.2020 PQ**, and the client auto-detects HDR from the stream. A small always-on **Vulkan layer**
- **x64-only.** No ARM64 build (no ARM64 NVIDIA driver, and SudoVDA is x64-only). (bundled and registered by the installer) also lets **Vulkan games** enable HDR over the virtual
display — something the NVIDIA/AMD drivers otherwise refuse on an indirect display. The layer is
self-gating: it's a no-op on SDR and on real monitors. HDR is **Windows-only** (the Linux host is
8-bit, blocked upstream).
## Notes & limits
- **AMD / Intel encode is newer.** The NVENC path is the most exercised; AMF (AMD) and QSV (Intel) are
built and tested in CI but less battle-tested on real hardware. Software H.264 is the GPU-less
fallback.
- **x64-only.** No ARM64 build — no ARM64 NVIDIA driver, and the virtual-display driver is x64-only.
- **Newer than the Linux host.** The Linux host is the most battle-tested path; the Windows host is - **Newer than the Linux host.** The Linux host is the most battle-tested path; the Windows host is
more recent, with the virtual-mic and gamepad backends the youngest pieces. more recent, with the virtual-mic and AMD/Intel encode backends the youngest pieces.
Trouble? See [Troubleshooting](/docs/troubleshooting) and [Pairing](/docs/pairing).
+13
View File
@@ -72,6 +72,8 @@ package_punktfunk-host() {
'xdg-desktop-portal-wlr: portal for the headless Sway session helper' 'xdg-desktop-portal-wlr: portal for the headless Sway session helper'
'punktfunk-web: browser management console (device pairing + status)') 'punktfunk-web: browser management console (device pairing + status)')
install=punktfunk-host.install install=punktfunk-host.install
# User-editable config: the headless game-mode drop-in (see below) — don't clobber local edits.
backup=('etc/gamescope-session-plus/sessions.d/steam')
local R; R="$(_repo)"; local T="$srcdir/target/release" local R; R="$(_repo)"; local T="$srcdir/target/release"
install -Dm0755 "$T/punktfunk-host" "$pkgdir/usr/bin/punktfunk-host" install -Dm0755 "$T/punktfunk-host" "$pkgdir/usr/bin/punktfunk-host"
@@ -86,6 +88,12 @@ package_punktfunk-host() {
install -Dm0644 "$R/scripts/punktfunk-kde-session.service" "$pkgdir/usr/lib/systemd/user/punktfunk-kde-session.service" install -Dm0644 "$R/scripts/punktfunk-kde-session.service" "$pkgdir/usr/lib/systemd/user/punktfunk-kde-session.service"
sed -i 's#%h/punktfunk/scripts/headless/run-headless-kde.sh#/usr/share/punktfunk/headless/run-headless-kde.sh#' \ sed -i 's#%h/punktfunk/scripts/headless/run-headless-kde.sh#/usr/share/punktfunk/headless/run-headless-kde.sh#' \
"$pkgdir/usr/lib/systemd/user/punktfunk-kde-session.service" "$pkgdir/usr/lib/systemd/user/punktfunk-kde-session.service"
# KWin Desktop-mode authorization: non-launcher .desktop whose X-KDE-Wayland-Interfaces lets the
# host bind KWin's restricted zkde_screencast (virtual output) + fake_input globals on an
# interactive Plasma session. Must ship with the host (KWin caches the per-exe grant on first
# connect). See the file's header comment.
install -Dm0644 "$R/packaging/linux/io.unom.Punktfunk.Host.desktop" \
"$pkgdir/usr/share/applications/io.unom.Punktfunk.Host.desktop"
# headless session helpers + env templates + OpenAPI doc # headless session helpers + env templates + OpenAPI doc
install -Dm0755 "$R/scripts/headless/run-headless-kde.sh" "$pkgdir/usr/share/punktfunk/headless/run-headless-kde.sh" install -Dm0755 "$R/scripts/headless/run-headless-kde.sh" "$pkgdir/usr/share/punktfunk/headless/run-headless-kde.sh"
install -Dm0755 "$R/scripts/headless/run-headless-sway.sh" "$pkgdir/usr/share/punktfunk/headless/run-headless-sway.sh" install -Dm0755 "$R/scripts/headless/run-headless-sway.sh" "$pkgdir/usr/share/punktfunk/headless/run-headless-sway.sh"
@@ -94,6 +102,11 @@ package_punktfunk-host() {
install -Dm0644 "$R/scripts/host.env.example" "$pkgdir/usr/share/punktfunk/host.env.example" install -Dm0644 "$R/scripts/host.env.example" "$pkgdir/usr/share/punktfunk/host.env.example"
install -Dm0644 "$R/packaging/bazzite/host.env" "$pkgdir/usr/share/punktfunk/host.env.bazzite" install -Dm0644 "$R/packaging/bazzite/host.env" "$pkgdir/usr/share/punktfunk/host.env.bazzite"
install -Dm0644 "$R/packaging/kde/host.env" "$pkgdir/usr/share/punktfunk/host.env.kde" install -Dm0644 "$R/packaging/kde/host.env" "$pkgdir/usr/share/punktfunk/host.env.kde"
# Headless GAME-mode fix: gamescope-session-plus drop-in that uses the headless backend when no
# display is connected (so SteamOS/Bazzite "Switch to Game Mode" works on a display-less streaming
# host). No-op on display-attached boxes; sourced as /etc/gamescope-session-plus/sessions.d/steam.
install -Dm0644 "$R/packaging/bazzite/gamescope-headless-session" \
"$pkgdir/etc/gamescope-session-plus/sessions.d/steam"
install -Dm0644 "$R/api/openapi.json" "$pkgdir/usr/share/punktfunk/openapi.json" install -Dm0644 "$R/api/openapi.json" "$pkgdir/usr/share/punktfunk/openapi.json"
install -Dm0644 "$R/LICENSE-MIT" "$pkgdir/usr/share/licenses/punktfunk-host/LICENSE-MIT" install -Dm0644 "$R/LICENSE-MIT" "$pkgdir/usr/share/licenses/punktfunk-host/LICENSE-MIT"
install -Dm0644 "$R/LICENSE-APACHE" "$pkgdir/usr/share/licenses/punktfunk-host/LICENSE-APACHE" install -Dm0644 "$R/LICENSE-APACHE" "$pkgdir/usr/share/licenses/punktfunk-host/LICENSE-APACHE"
@@ -0,0 +1,22 @@
# punktfunk: headless game-mode fallback for gamescope-session-plus.
#
# Installed as /etc/gamescope-session-plus/sessions.d/steam. The gamescope-session-plus launcher
# SOURCES this (shell, with `set -a` so assignments auto-export) AFTER its /usr/share defaults, so it
# can override the session's gamescope flags.
#
# Why: on a box with NO connected display (a dedicated streaming host), the stock Steam game mode runs
# gamescope's DRM backend against a physical panel (`--prefer-output *,eDP-1`). With nothing to scan
# out, gamescope crashes on launch; after 5 strikes Bazzite/SteamOS force-selects the desktop session
# and "Switch to Game Mode" appears broken. Falling back to gamescope's HEADLESS backend makes game
# mode render entirely offscreen and expose a PipeWire node, which the punktfunk host captures and
# streams — full gamescope game mode (per-game res / FSR / HDR / VRR / frame-limit), no monitor needed.
#
# Safe by construction:
# * NO-OP when any display is connected -> the normal DRM game mode runs unchanged.
# * Only sets values that are still unset (`: "${VAR:=...}"`), so the punktfunk host's per-client
# mode (SCREEN_WIDTH/SCREEN_HEIGHT injected via systemd-run for a managed session) still wins.
if ! grep -qx connected /sys/class/drm/*/status 2>/dev/null; then
: "${BACKEND:=headless}"
: "${SCREEN_WIDTH:=1920}"
: "${SCREEN_HEIGHT:=1080}"
fi
+21 -8
View File
@@ -20,12 +20,25 @@ PUNKTFUNK_ZEROCOPY=1
# PUNKTFUNK_COMPOSITOR=kwin|mutter|wlroots|gamescope # PUNKTFUNK_COMPOSITOR=kwin|mutter|wlroots|gamescope
# PUNKTFUNK_INPUT_BACKEND=libei|wlr|gamescope|uinput # PUNKTFUNK_INPUT_BACKEND=libei|wlr|gamescope|uinput
# #
# In Gaming Mode the host MANAGES a gamescope-session-plus at the CLIENT's resolution by default # GAME MODE = ATTACH (the box owns its session; the host follows). The box decides whether it's in
# (tears the TV's autologin down on connect; restores it on a debounced idle, reused on a quick # Steam Gaming Mode or a Desktop — you switch with the normal Steam UI / "Switch to Desktop". The
# reconnect). To instead ATTACH to the running TV session at its own mode (couch-on-TV — gaming # host just ATTACHES to whatever's live and captures it; it never tears the session down or relaunches
# stays live on the panel, no Steam restart), set: # it. So switching Desktop<->Game is rock-solid, and when you disconnect the box STAYS in its current
# PUNKTFUNK_GAMESCOPE_ATTACH=1 # mode — reconnecting drops you right back where you were. The streamed resolution in game mode is the
# PUNKTFUNK_GAMESCOPE_APP=steam -gamepadui # only for an ad-hoc bare-spawn fallback # box's gamescope mode (see SCREEN_WIDTH/HEIGHT in /etc/gamescope-session-plus/sessions.d/steam).
PUNKTFUNK_GAMESCOPE_ATTACH=1
# #
# Follow a Gaming<->Desktop switch MID-STREAM (rebuild the backend in place, no reconnect): # Opt OUT to the MANAGED model instead (host tears the box's gamescope down on connect and launches
# PUNKTFUNK_SESSION_WATCH=1 # its OWN at the CLIENT's exact resolution; restores on a debounced idle). Client-mode-following, but
# it does not coexist with a box-owned game-mode session — pick one:
# PUNKTFUNK_GAMESCOPE_MANAGED=1 # (and remove PUNKTFUNK_GAMESCOPE_ATTACH above)
#
# Follow a Gaming<->Desktop switch MID-STREAM (rebuild the backend in place, no reconnect). This is
# ON BY DEFAULT on Bazzite/SteamOS (the host detects the platform); set =0 to disable it:
# PUNKTFUNK_SESSION_WATCH=0
#
# HEADLESS GAME MODE: on a box with no display attached, Bazzite's "Switch to Game Mode" normally
# crashes (gamescope's DRM backend has no panel to drive). The host package ships
# /etc/gamescope-session-plus/sessions.d/steam, which auto-falls-back to gamescope's HEADLESS backend
# when no display is connected — so game mode boots offscreen and streams, with no config here. It's a
# no-op on display-attached boxes. (The host then auto-detects Gaming and streams it.)
+24 -23
View File
@@ -1,35 +1,36 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# One-shot setup so the punktfunk host can stream the Bazzite KDE *Desktop* session (KWin virtual # One-shot setup so the punktfunk host can INJECT INPUT while streaming the Bazzite KDE *Desktop*
# output at the client's resolution). Run ONCE as the streaming user (no root needed). Gaming Mode # session. Run ONCE as the streaming user (no root needed). Gaming Mode (gamescope) needs none of
# (gamescope) needs none of this — it auto-attaches. Idempotent: safe to re-run. # this — it auto-attaches. Idempotent: safe to re-run.
# #
# bash /usr/share/punktfunk/bazzite/kde-desktop-setup.sh # bash /usr/share/punktfunk/bazzite/kde-desktop-setup.sh
# #
# Two things a normal KDE login lacks that the headless host needs: # The VIRTUAL OUTPUT (video) needs no setup: the host package ships io.unom.Punktfunk.Host.desktop,
# 1. KWIN_WAYLAND_NO_PERMISSION_CHECKS=1 — so KWin exposes the privileged `zkde_screencast` # whose X-KDE-Wayland-Interfaces grants the host KWin's restricted zkde_screencast protocol on a
# virtual-output protocol to the host (an external client) at all. # normal interactive Plasma session — least-privilege (only the host, only that interface), the same
# 2. The `kde-authorized` RemoteDesktop grant — so libei input setup auto-approves instead of # mechanism krfb/krdp use. No session-wide KWIN_WAYLAND_NO_PERMISSION_CHECKS hack is needed. KWin
# popping an "Allow remote control?" dialog the headless host can't answer. # caches the grant per-executable on first connect, so after a FRESH host install log out + back into
# After running, log out + back into the KDE Desktop session once (or reboot) so KWin restarts # the Desktop session once so KWin re-reads the file.
# with the flag. Gaming Mode is unaffected. #
# The one thing a normal KDE login still lacks is the `kde-authorized` RemoteDesktop grant — so the
# host's libei input setup auto-approves instead of popping an "Allow remote control?" dialog the
# headless host can't answer. That's what this script seeds.
set -euo pipefail set -euo pipefail
GRANT_SRC="${PUNKTFUNK_GRANT_SRC:-/usr/share/punktfunk/headless/kde-authorized}" GRANT_SRC="${PUNKTFUNK_GRANT_SRC:-/usr/share/punktfunk/headless/kde-authorized}"
ENVD="$HOME/.config/environment.d/10-punktfunk-kwin.conf"
GRANT_DST="$HOME/.local/share/flatpak/db/kde-authorized" GRANT_DST="$HOME/.local/share/flatpak/db/kde-authorized"
# Older versions of this script wrote a session-wide KWIN_WAYLAND_NO_PERMISSION_CHECKS=1 env file to
# unlock screencast. The shipped .desktop replaces it; remove the stale, over-broad override.
STALE_ENVD="$HOME/.config/environment.d/10-punktfunk-kwin.conf"
echo "punktfunk: KDE Desktop-mode setup" echo "punktfunk: KDE Desktop-mode input setup"
# 1. KWin permission-check bypass (persistent, picked up by the next KDE session via systemd). if [[ -f "$STALE_ENVD" ]] && grep -q KWIN_WAYLAND_NO_PERMISSION_CHECKS "$STALE_ENVD" 2>/dev/null; then
mkdir -p "$(dirname "$ENVD")" rm -f "$STALE_ENVD"
cat > "$ENVD" <<'EOF' echo " removed stale $STALE_ENVD (screencast is now granted via the shipped .desktop)"
# punktfunk: let the streaming host bind KWin's privileged zkde_screencast (virtual output). fi
# A dedicated streaming box; this relaxes KWin's Wayland permission checks for the desktop path.
KWIN_WAYLAND_NO_PERMISSION_CHECKS=1
EOF
echo " wrote $ENVD"
# 2. RemoteDesktop portal grant for headless libei input (never clobber an existing one). # RemoteDesktop portal grant for headless libei input (never clobber an existing one).
if [[ -s "$GRANT_DST" ]]; then if [[ -s "$GRANT_DST" ]]; then
echo " grant DB already present ($GRANT_DST) — leaving it" echo " grant DB already present ($GRANT_DST) — leaving it"
elif [[ -s "$GRANT_SRC" ]]; then elif [[ -s "$GRANT_SRC" ]]; then
@@ -44,5 +45,5 @@ else
echo " WARN: grant source not found at $GRANT_SRC — input will need a manual portal approval" >&2 echo " WARN: grant source not found at $GRANT_SRC — input will need a manual portal approval" >&2
fi fi
echo "punktfunk: done. Log out + back into the KDE Desktop session (or reboot) so KWin restarts" echo "punktfunk: done. On a fresh host install, log out + back into the KDE Desktop session once"
echo " with the flag, then connect a client while in Desktop Mode." echo " (so KWin authorizes the host's virtual output), then connect a client in Desktop Mode."
+7
View File
@@ -50,6 +50,13 @@ sed -i 's#%h/punktfunk/target/release/punktfunk-host#/usr/bin/punktfunk-host#' \
install -Dm0644 scripts/punktfunk-kde-session.service "$STAGE/usr/lib/systemd/user/punktfunk-kde-session.service" install -Dm0644 scripts/punktfunk-kde-session.service "$STAGE/usr/lib/systemd/user/punktfunk-kde-session.service"
sed -i 's#%h/punktfunk/scripts/headless/run-headless-kde.sh#/usr/share/punktfunk-host/headless/run-headless-kde.sh#' \ sed -i 's#%h/punktfunk/scripts/headless/run-headless-kde.sh#/usr/share/punktfunk-host/headless/run-headless-kde.sh#' \
"$STAGE/usr/lib/systemd/user/punktfunk-kde-session.service" "$STAGE/usr/lib/systemd/user/punktfunk-kde-session.service"
# KWin Desktop-mode authorization: non-launcher .desktop whose X-KDE-Wayland-Interfaces lets the
# host bind KWin's restricted zkde_screencast (virtual output) + fake_input globals on an
# interactive Plasma session. Must ship with the host — KWin caches the per-exe grant on first
# connect, so it has to be present before the host ever connects. See the file's header comment.
install -Dm0644 packaging/linux/io.unom.Punktfunk.Host.desktop \
"$STAGE/usr/share/applications/io.unom.Punktfunk.Host.desktop"
install -Dm0755 scripts/headless/run-headless-kde.sh "$SHAREDIR/headless/run-headless-kde.sh" install -Dm0755 scripts/headless/run-headless-kde.sh "$SHAREDIR/headless/run-headless-kde.sh"
install -Dm0755 scripts/headless/run-headless-sway.sh "$SHAREDIR/headless/run-headless-sway.sh" install -Dm0755 scripts/headless/run-headless-sway.sh "$SHAREDIR/headless/run-headless-sway.sh"
install -Dm0644 scripts/headless/kde-authorized "$SHAREDIR/headless/kde-authorized" install -Dm0644 scripts/headless/kde-authorized "$SHAREDIR/headless/kde-authorized"
@@ -0,0 +1,19 @@
[Desktop Entry]
Type=Application
Name=Punktfunk Host
Comment=punktfunk streaming host — KWin virtual-output / input authorization
Exec=/usr/bin/punktfunk-host
Terminal=false
NoDisplay=true
# This file is NOT a launcher — it exists so KWin authorizes the host to bind its restricted
# Wayland globals when streaming the *Desktop* (KWin) session. KWin maps a connecting client to a
# .desktop by resolving /proc/<pid>/exe against `Exec` (hence the absolute /usr/bin path), then
# grants only the interfaces listed here (the same mechanism krfb-virtualmonitor / krdpserver use):
# * zkde_screencast_unstable_v1 — create the per-session virtual output at the client's mode.
# * org_kde_kwin_fake_input — inject input directly (no RemoteDesktop portal dialog).
# Comma-separated, per KWin's parser. Without this file KWin never advertises these to the host and
# desktop-mode streaming fails with "KWin does not expose zkde_screencast_unstable_v1". Gaming Mode
# (gamescope) does not use this path. NOTE: KWin caches the per-executable grant on first connect,
# so this must be installed *before* the host first connects (a package install satisfies that; an
# already-running KWin session needs a re-login to pick it up).
X-KDE-Wayland-Interfaces=zkde_screencast_unstable_v1,org_kde_kwin_fake_input
+21 -1
View File
@@ -196,6 +196,14 @@ sed -i 's#%h/punktfunk/target/release/punktfunk-host#%{_bindir}/punktfunk-host#'
install -Dm0644 scripts/punktfunk-kde-session.service %{buildroot}%{_userunitdir}/punktfunk-kde-session.service install -Dm0644 scripts/punktfunk-kde-session.service %{buildroot}%{_userunitdir}/punktfunk-kde-session.service
sed -i 's#%h/punktfunk/scripts/headless/run-headless-kde.sh#%{_datadir}/%{name}/headless/run-headless-kde.sh#' %{buildroot}%{_userunitdir}/punktfunk-kde-session.service sed -i 's#%h/punktfunk/scripts/headless/run-headless-kde.sh#%{_datadir}/%{name}/headless/run-headless-kde.sh#' %{buildroot}%{_userunitdir}/punktfunk-kde-session.service
# KWin authorization for Desktop-mode (KWin) streaming: a non-launcher .desktop whose
# X-KDE-Wayland-Interfaces grants the host the restricted zkde_screencast (virtual output) +
# fake_input globals on an interactive Plasma session. Must ship with the host so it is present
# before the host first connects (KWin caches the per-exe grant). Replaces the old manual
# KWIN_WAYLAND_NO_PERMISSION_CHECKS hack for the screencast permission.
install -Dm0644 packaging/linux/io.unom.Punktfunk.Host.desktop \
%{buildroot}%{_datadir}/applications/io.unom.Punktfunk.Host.desktop
# --- client subpackage --- # --- client subpackage ---
install -Dm0755 target/release/punktfunk-client %{buildroot}%{_bindir}/punktfunk-client install -Dm0755 target/release/punktfunk-client %{buildroot}%{_bindir}/punktfunk-client
install -Dm0644 packaging/linux/io.unom.Punktfunk.desktop \ install -Dm0644 packaging/linux/io.unom.Punktfunk.desktop \
@@ -221,9 +229,17 @@ install -Dm0644 scripts/headless/punktfunk-sink.conf %{buildroot}%{_datadir}/%
install -Dm0644 scripts/host.env.example %{buildroot}%{_datadir}/%{name}/host.env.example install -Dm0644 scripts/host.env.example %{buildroot}%{_datadir}/%{name}/host.env.example
install -Dm0644 packaging/bazzite/host.env %{buildroot}%{_datadir}/%{name}/host.env.bazzite install -Dm0644 packaging/bazzite/host.env %{buildroot}%{_datadir}/%{name}/host.env.bazzite
install -Dm0644 packaging/kde/host.env %{buildroot}%{_datadir}/%{name}/host.env.kde install -Dm0644 packaging/kde/host.env %{buildroot}%{_datadir}/%{name}/host.env.kde
# Bazzite KDE Desktop-mode one-shot setup (KWIN_WAYLAND_NO_PERMISSION_CHECKS + RemoteDesktop grant). # Bazzite KDE Desktop-mode one-shot setup (seeds the RemoteDesktop grant for libei input; the
# screencast/virtual-output grant ships as io.unom.Punktfunk.Host.desktop, installed above).
install -d %{buildroot}%{_datadir}/%{name}/bazzite install -d %{buildroot}%{_datadir}/%{name}/bazzite
install -Dm0755 packaging/bazzite/kde-desktop-setup.sh %{buildroot}%{_datadir}/%{name}/bazzite/kde-desktop-setup.sh install -Dm0755 packaging/bazzite/kde-desktop-setup.sh %{buildroot}%{_datadir}/%{name}/bazzite/kde-desktop-setup.sh
# Headless GAME-mode fix: a gamescope-session-plus sessions.d drop-in that falls back to gamescope's
# headless backend when no display is connected (so "Switch to Game Mode" works on a display-less
# streaming host instead of crashing + 5-striking back to desktop). No-op on display-attached boxes.
# Sourced by gamescope-session-plus as /etc/gamescope-session-plus/sessions.d/steam (after its
# /usr/share defaults). Harmless on non-gamescope systems (the file is simply never read).
install -Dm0644 packaging/bazzite/gamescope-headless-session \
%{buildroot}/etc/gamescope-session-plus/sessions.d/steam
install -Dm0644 api/openapi.json %{buildroot}%{_datadir}/%{name}/openapi.json install -Dm0644 api/openapi.json %{buildroot}%{_datadir}/%{name}/openapi.json
%if %{with web} %if %{with web}
@@ -252,6 +268,10 @@ install -Dm0644 web/web.env.example %{buildroot}%{_datadir}/punkt
%{_prefix}/lib/sysctl.d/99-punktfunk-net.conf %{_prefix}/lib/sysctl.d/99-punktfunk-net.conf
%{_userunitdir}/punktfunk-host.service %{_userunitdir}/punktfunk-host.service
%{_userunitdir}/punktfunk-kde-session.service %{_userunitdir}/punktfunk-kde-session.service
%{_datadir}/applications/io.unom.Punktfunk.Host.desktop
%dir /etc/gamescope-session-plus
%dir /etc/gamescope-session-plus/sessions.d
%config(noreplace) /etc/gamescope-session-plus/sessions.d/steam
%dir %{_datadir}/%{name} %dir %{_datadir}/%{name}
%{_datadir}/%{name}/* %{_datadir}/%{name}/*