Compare commits

..
13 Commits
Author SHA1 Message Date
enricobuehlerandClaude Fable 5 c90343c22f feat(android): shared clipboard (text) — device↔host sync while streaming
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m13s
apple / swift (push) Successful in 1m21s
android / android (push) Failing after 1m43s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
decky / build-publish (push) Successful in 33s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 8m18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m49s
deb / build-publish (push) Successful in 10m26s
release / apple (push) Successful in 9m22s
flatpak / build-publish (push) Failing after 8m23s
docker / deploy-docs (push) Successful in 28s
deb / build-publish-host (push) Successful in 12m44s
arch / build-publish (push) Successful in 15m51s
windows-host / package (push) Successful in 16m14s
apple / screenshots (push) Successful in 6m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m6s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m18s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
ci / rust (push) Successful in 24m48s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m21s
The desktop clients' clipboard protocol, adopted on Android (text-only v1):
opt-in ClipControl at stream start when the host advertises HOST_CAP_CLIPBOARD
and the new "Shared clipboard" setting (default on) allows.

Device → host: local copies (primary-clip listener + a probe at start) are
announced as lazy format-list offers; the text crosses only when the host
actually pastes (FetchRequest → served from the live clipboard). Host →
device: a host copy's offer is fetched eagerly and lands in the system
clipboard (Android has no practical lazy-paste provider), with an echo guard
so the resulting clip-changed callback doesn't bounce it back as a new offer.

Native side: session/clipboard.rs JNI shims over NativeClient's clip_*
surface; events cross to Kotlin as compact strings from a blocking
nativeNextClip poll on a dedicated thread (the nativeNextRumble pattern),
joined before the session handle is freed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 19:27:19 +02:00
enricobuehlerandClaude Fable 5 abe6228b42 feat(core+host+android): committed-text input plane (IME path) — InputKind::TextInput
The VK key-event vocabulary cannot express text an input method COMMITS
(autocorrect, gesture typing, non-Latin scripts, emoji). Add a first-class
text event and negotiate it:

- punktfunk-core: InputKind::TextInput (= 15) carries one Unicode scalar per
  event in `code`; HOST_CAP_TEXT_INPUT (0x04) in Welcome::host_caps.
- Host advertises the cap only where the session's inject backend can type
  text: Windows SendInput (KEYEVENTF_UNICODE, surrogate-pair aware) and the
  Linux wlroots backend — a dedicated second zwp_virtual_keyboard whose xkb
  keymap grows Unicode keysyms on demand (the wtype model), so keymap
  re-uploads never disturb the main device's layout/modifier state. The
  KWin-fake-input/libei/gamescope backends can only press layout keycodes, so
  those sessions don't set the bit and clients keep the VK fallback.
- GameStream plane: Moonlight's UTF-8 text packet (MAGIC_UTF8, previously
  recognized-and-dropped) now decodes to the same TextInput events.
- Android: KeyCaptureView picks a real editable InputConnection when the host
  has the cap — the IME runs its full machinery, mirrored to the host live via
  common-prefix diffs of the composition (backspaces + new suffix), with
  setComposingRegion adopting committed text so autocorrect-revert flows diff
  instead of retyping; newline→Enter, deleteSurroundingText→Backspace/Delete.
  Older hosts keep the TYPE_NULL raw-key path unchanged.
- keymap: media VKs (0xB0-0xB3) → evdev so the Android media keys land on
  Linux hosts too.

Verified: punktfunk-core + host gamestream + pf-inject tests green on Linux
(Ubuntu box), clippy clean; Android app+native builds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 19:19:11 +02:00
enricobuehlerandClaude Fable 5 ddb72edcba feat(android): physical mouse + TV remote-as-pointer + keyboard hardening
Phase A — hardware mouse: new MouseForwarder routes every SOURCE_MOUSE event
while streaming. Uncaptured (default): hover/drag positions forward as absolute
cursor moves, wheel with fractional accumulation (high-res wheels), all five
buttons incl. back/forward as X1/X2; the local cursor is hidden over the stream
(TYPE_NULL pointer icon) so the host's composited cursor is the visible one.
Captured ("Capture pointer for games" setting, or Ctrl+Alt+Shift+Q live): the
OS pointer is grabbed via requestPointerCapture on the (repurposed) key-capture
view and raw relative deltas forward as mouse-look; held buttons are tracked
and flushed on capture loss/teardown, and the engaging click is swallowed
(desktop parity). Mouse clicks are intercepted before Compose so they can never
be misread as trackpad taps; a mouse back-button's FALLBACK BACK is dropped so
it can't yank the user out of the stream.

Phase B — keyboard: media/consumer keys (play/pause/next/prev/stop) now map to
their VKs instead of falling to the system; TV-remote SELECT maps to Enter.

Phase D — Android TV: RemotePointer turns a D-pad-only remote into a pointer
(Siri-remote analogue): hold SELECT ~0.8s toggles, D-pad glides the host cursor
with ramped acceleration (Choreographer-paced, diagonal-normalized), SELECT
taps click, play/pause right-clicks, holding it toggles the on-screen keyboard,
BACK leaves pointer mode; an overlay hint teaches the vocabulary.

Settings: "Capture pointer for games" + "Invert scroll direction" (applies
to the wheel and two-finger touch scrolling).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 18:56:38 +02:00
enricobuehlerandClaude Fable 5 5e088af7ed fix(pf-win-display): unpin mode indexes on paths the CCD isolate deactivates
ci / web (push) Successful in 49s
apple / swift (push) Successful in 1m24s
ci / docs-site (push) Successful in 55s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
apple / screenshots (push) Successful in 6m33s
ci / bench (push) Successful in 6m41s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m23s
android / android (push) Successful in 12m50s
docker / deploy-docs (push) Successful in 25s
deb / build-publish-host (push) Successful in 10m10s
arch / build-publish (push) Successful in 17m18s
deb / build-publish (push) Successful in 12m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m3s
ci / rust (push) Successful in 27m3s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m32s
Field report: with Exclusive Topology, the physical monitor sometimes stayed
lit — every isolate attempt re-committed and failed SetDisplayConfig with
0x57 ERROR_INVALID_PARAMETER. Clearing DISPLAYCONFIG_PATH_ACTIVE alone leaves
the deactivated path's source/target modeInfoIdx pinned to the queried mode
entries, and per the SetDisplayConfig contract a path being turned off must
have BOTH mode indexes set to DISPLAYCONFIG_PATH_MODE_IDX_INVALID; some
driver/topology combinations (notably when the doomed path shares a source
with the kept virtual path) reject the supplied config outright, so the
retry loop could never converge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 12:26:44 +02:00
enricobuehlerandClaude Fable 5 b7a00137eb feat(host): hold a logind sleep/idle inhibitor while clients stream
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 1m0s
android / android (push) Failing after 1m6s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 48s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 52s
ci / bench (push) Successful in 8m13s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m40s
deb / build-publish (push) Successful in 11m0s
release / apple (push) Successful in 9m50s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10m19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m57s
docker / deploy-docs (push) Successful in 35s
deb / build-publish-host (push) Successful in 14m3s
apple / screenshots (push) Successful in 6m27s
arch / build-publish (push) Successful in 17m49s
windows-host / package (push) Successful in 18m15s
flatpak / build-publish (push) Failing after 8m25s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m24s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m37s
ci / rust (push) Successful in 25m51s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m48s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m52s
Remote INPUT resets the compositor's idle timers, but a passive
(video-only) viewer sends none — observed live: a SteamOS Game-Mode
host s2idled mid-day and dropped off the network (in a VM with GPU
passthrough it never woke again). Refcounted across planes: the native
LiveSessionGuard and the GameStream video worker each hold a share;
first hold takes the logind sleep:idle block inhibitor on a dedicated
plain thread (zbus blocking must not run on a tokio worker), last drop
releases. Best-effort: no logind → warn once and stream on. No-op off
Linux (macOS/Windows manage their own assertions).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 12:09:47 +02:00
enricobuehlerandClaude Fable 5 acce43ebbf feat(steamdeck): self-healing reliability — post-OS-update rebuild check + script/docs polish
- rebuild-check.sh + punktfunk-rebuild-check.service (enabled, ordered
  Before=punktfunk-host): ldd-probes the host binary at session start —
  milliseconds when healthy, a full update.sh rebuild only when a
  SteamOS update actually broke its library links. update.sh restarts
  go --no-block so the check → update.sh → restart chain can't deadlock
  against the unit ordering. update.sh retrofits the unit.
- installer summary: web console is https (the unit serves TLS).
- docs: steamos-host.md (runner in the build step, keep-list + auto
  rebuild = updates survive hands-free), gamescope.md (Gaming Mode
  touch = single-finger pointer, exact taps, no multi-touch),
  plugins.mdx (SteamOS runner note), scripts/steamdeck/README.md
  (rebuild-check, runner payload, keep list, honest packaging
  trade-off + the CI-prebuilt future direction).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 11:56:09 +02:00
enricobuehlerandClaude Fable 5 35923080fb fix(host/linux): scale degenerate-region absolute coords by the session's real output size
The raw-client-pixels fallback is exact only while the managed session
runs at the client's mode. When they diverge — foreign-gamescope
attach at its own resolution, supersample/under-render, transitions —
raw pixels drift or land out of range. The EIS relay file now carries
the session's current output size as a second "WxH" line (from
current_gamescope_output_size()); the injector scales normalized
client positions into it, keeping raw pixels only as the last resort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:53:14 +02:00
enricobuehlerandClaude Fable 5 0c9461242c fix(host/linux): don't normalize absolute coords into gamescope's degenerate INT32_MAX EIS region
The previous commit's no-region fallback never fired: gamescope DOES
advertise a region — (0,0,INT32_MAX,INT32_MAX), meaning "coordinates
are raw". Mapping normalized positions into it explodes a center tap
to x≈1e9, clamped by gamescope to the far corner — the cursor pinned
at (1279,799) and every degraded-touch tap landing there. Treat a
region as an output geometry only when plausibly sized (≤16384 px);
otherwise emit raw client pixels.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:40:02 +02:00
enricobuehlerandClaude Fable 5 cc6b37fef0 fix(host/linux): emit absolute pointer/touch on region-less EIS devices + abs input-test probe
gamescope's "Gamescope Virtual Input" advertises pointer_abs but no
region, so every MouseMoveAbs (and the degraded-touch moves built on
it) was silently dropped (emitted=false) — clicks then landed at a
stale cursor position. With no region, the managed session runs at the
client's mode, so client pixels are output pixels: emit them raw.
Also: log region count/dims at device-resume, and add
PUNKTFUNK_INPUT_TEST_ABS=WxH to `input-test` (corners + center, 1s
apart) so the degraded-touch path is verifiable with xdotool alone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:36:05 +02:00
enricobuehlerandClaude Fable 5 1927077cbe fix(host/linux): degrade touch to absolute pointer when the EIS has no touchscreen device
gamescope's EIS ("Gamescope Virtual Input") advertises pointer/
pointer_abs/button but never a touchscreen — so in Game Mode every
remote TouchDown/Move/Up was dropped on the floor (Desktop Mode works
via KwinFakeInput, which does touch). Headless KWin's libei has the
same gap. Instead of dropping: degrade to a single-finger absolute
pointer — down = abs-move + left press, move = abs-move, up = release —
synthesized through the normal Mouse* inject paths so region mapping,
held-state tracking, and release_all apply. First finger drives the
pointer; later fingers are ignored (a pinch degrades to a drag). A
touchscreen device appearing later takes over on the next touch —
checked per event, never latched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:24:28 +02:00
enricobuehlerandClaude Fable 5 068da456f5 fix(steamdeck): survive SteamOS A/B updates — register system tuning on the atomic-update keep list
Verified live on a SteamOS 3.8.14→3.8.16 update: the A/B partset switch
rebuilds /etc and drops everything not on Valve's keep list — the udev
rule (uhid access: virtual pads silently degrade to Xbox 360), the
vhci-hcd autoload (native Deck pad transport), and the UDP buffer sysctl
(back to 208 KB → stutter). Valve's sanctioned extension point is a
drop-in under /etc/atomic-update.conf.d/ — itself on the stock keep
list, so it self-preserves. install.sh §4 and update.sh now register
scripts/punktfunk-atomic-keep.conf there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:07:19 +02:00
enricobuehlerandClaude Fable 5 256aa5f7f2 fix(steamdeck): install the plugin runner + harden Deck pad typing + repair update.sh
- scripts/steamdeck/install.sh (§2b) + update.sh: build and install the
  scripting runner user-scoped (~/.local wrapper + pinned bun + bundle) —
  SteamOS's read-only /usr can't take the .deb layout, so the console's
  plugin store reported "the plugin runner isn't installed" on every
  SteamOS host. update.sh retrofits it onto existing installs.
- plugins.rs runner discovery: check the ~/.local layout after the /usr
  ones; mention the SteamOS path in the not-installed error.
- update.sh: define warn() — it was used but never defined, so under
  `set -e` the first warn aborted the update before services restarted.
- pf-client-core gamepad: a Steam Input virtual pad forwarded on a real
  Deck (the only-pad case) now declares the DECK kind in its per-pad
  arrival instead of the wrapper's Xbox 360 identity — the session-level
  auto_pref already resolved SteamDeck, but the arrival overrides it on
  current hosts.
- reject tests: cover SetupFailed; move the foreign-code probe off 0x68.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:01:54 +02:00
enricobuehlerandClaude Fable 5 2562663fc6 fix(host/linux): survive and self-heal dead gamescope sessions on headless SteamOS boxes
Three fixes for the "most connects simply fail" trap on a SteamOS host
with no physical display (VM testbox, panel-less mini PC):

- restore path: skip the restore-to-physical-panel when no DRM connector
  reports a connected display — removing the headless drop-in and
  restarting gamescope-session.target on such a box just crash-loops
  gamescope and strands every later connect on "no usable compositor".
  Keep the headless session (and the takeover state) instead.
- connect path: when no live graphical session exists but the MANAGED
  gamescope infra is present (SteamOS gamescope-session / Bazzite
  session-plus), route to the gamescope takeover — which rebuilds the
  session at the client's mode — instead of failing the connect.
- error delivery: a session-setup failure used to just drop the QUIC
  connection, reaching the client as "control stream finished mid-frame"
  (indistinguishable from transport trouble). Close with a typed
  setup-failed code (0x68, RejectedSetupFailed = -29) carrying the error
  text, and give the client handshake/pairing error paths a short grace
  for the CONNECTION_CLOSE to beat the stream error it caused.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 09:50:30 +02:00
50 changed files with 2190 additions and 103 deletions
@@ -0,0 +1,107 @@
package io.unom.punktfunk
import android.content.ClipData
import android.content.ClipboardManager
import android.content.Context
import android.os.Handler
import android.os.Looper
import io.unom.punktfunk.kit.NativeBridge
/**
* Text clipboard sync for the active session (the desktop-client model, text-only v1):
* * **Device → host**: a local copy (the primary-clip listener, plus one probe at start) is
* announced as a lazy offer — the text crosses only when the host actually pastes (a
* `fetch:` event, answered with the clipboard's current content).
* * **Host → device**: a host copy arrives as an `offer:` event and is fetched eagerly into
* the system clipboard (Android apps can't lazily materialize a paste from the network
* without a content-provider round-trip that isn't worth it here).
*
* Loop guard: text set from a host fetch is remembered ([lastFromHost]) so the resulting
* primary-clip-changed callback doesn't bounce it straight back as a new offer. Clipboard reads
* happen while the stream is foreground (Android only allows focused-app reads). The native
* events are drained on a dedicated thread and applied on the main thread; [stop] joins it.
*/
class ClipboardSync(
private val context: Context,
private val handle: Long,
) {
private val main = Handler(Looper.getMainLooper())
private val cm = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
@Volatile private var running = true
private var seq = 0
private var lastOffered: String? = null
private var lastFromHost: String? = null
private var pendingFetch = -1
private var thread: Thread? = null
private val clipListener = ClipboardManager.OnPrimaryClipChangedListener { offerLocal() }
fun start() {
NativeBridge.nativeClipControl(handle, true)
cm.addPrimaryClipChangedListener(clipListener)
thread = Thread({ pollLoop() }, "pf-clipboard").also { it.start() }
offerLocal() // whatever is already on the clipboard is pasteable host-side right away
}
fun stop() {
running = false
cm.removePrimaryClipChangedListener(clipListener)
thread?.join(600) // one poll timeout (250 ms) + slack
thread = null
}
/** Announce the current local text (if it's new and not an echo of a host copy). */
private fun offerLocal() {
if (!running) return
val text = currentClipText() ?: return
if (text == lastOffered || text == lastFromHost) return
lastOffered = text
seq += 1
NativeBridge.nativeClipOfferText(handle, seq)
}
private fun currentClipText(): String? = runCatching {
cm.primaryClip?.takeIf { it.itemCount > 0 }?.getItemAt(0)
?.coerceToText(context)?.toString()?.takeIf { it.isNotEmpty() }
}.getOrNull()
private fun pollLoop() {
while (running) {
val ev = NativeBridge.nativeNextClip(handle) ?: continue
if (ev == "closed") return
main.post { handleEvent(ev) }
}
}
private fun handleEvent(ev: String) {
if (!running) return
val parts = ev.split(":", limit = 3)
when (parts[0]) {
"offer" -> {
val offerSeq = parts.getOrNull(1)?.toIntOrNull() ?: return
if (parts.getOrNull(2) == "1") {
pendingFetch = NativeBridge.nativeClipFetchText(handle, offerSeq)
}
}
"fetch" -> {
val req = parts.getOrNull(1)?.toIntOrNull() ?: return
val text = currentClipText()
if (text != null) {
NativeBridge.nativeClipServeText(handle, req, text)
} else {
NativeBridge.nativeClipCancel(handle, req)
}
}
"data" -> {
val xfer = parts.getOrNull(1)?.toIntOrNull() ?: return
if (xfer != pendingFetch) return // stale/unknown transfer
pendingFetch = -1
val text = parts.getOrNull(2)?.takeIf { it.isNotEmpty() } ?: return
lastFromHost = text
runCatching { cm.setPrimaryClip(ClipData.newPlainText("Punktfunk", text)) }
}
// "state"/"cancel"/"error": nothing to drive in the text-only v1.
}
}
}
@@ -54,6 +54,21 @@ class MainActivity : ComponentActivity() {
var padKeyProbe: ((KeyEvent) -> Boolean)? = null var padKeyProbe: ((KeyEvent) -> Boolean)? = null
var padMotionProbe: ((MotionEvent) -> Boolean)? = null var padMotionProbe: ((MotionEvent) -> Boolean)? = null
/**
* Physical-mouse forwarder for the active session (built/released by StreamScreen, like
* [gamepadRouter]): uncaptured hover/click/wheel forwards as absolute cursor input, captured
* ([android.view.View.requestPointerCapture]) raw deltas as relative mouse-look. The dispatch
* overrides below route every SOURCE_MOUSE event here while streaming. Null while not streaming.
*/
var mouseForwarder: MouseForwarder? = null
/**
* TV remote-as-pointer for the active session (StreamScreen builds it on TV devices only):
* hold SELECT to toggle, then the D-pad glides the host cursor. Consulted first for
* non-gamepad keys while streaming. Null while not streaming or not a TV.
*/
var remotePointer: RemotePointer? = null
/** /**
* Set by [StreamScreen] to its disconnect action. The emergency-exit chord (below) invokes it so a * Set by [StreamScreen] to its disconnect action. The emergency-exit chord (below) invokes it so a
* couch user with no keyboard/Back can always leave a stream. * couch user with no keyboard/Back can always leave a stream.
@@ -324,9 +339,29 @@ class MainActivity : ComponentActivity() {
return true // consumed return true // consumed
} }
} }
// TV remote-as-pointer sees non-gamepad keys first (SELECT long-press toggles it;
// while active it owns the D-pad/SELECT/PLAY-PAUSE/BACK).
if (!event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
remotePointer?.let { if (it.onKey(event)) return true }
}
// Ctrl+Alt+Shift+Q — the cross-client pointer-capture toggle chord. Swallow both
// edges of the Q (the modifiers already went over the wire, exactly like desktop).
if (event.keyCode == KeyEvent.KEYCODE_Q &&
event.isCtrlPressed && event.isAltPressed && event.isShiftPressed
) {
if (event.action == KeyEvent.ACTION_DOWN && event.repeatCount == 0) {
mouseForwarder?.toggleCapture()
}
return true
}
when (event.keyCode) { when (event.keyCode) {
// A mouse back/forward button whose BUTTON_* press went unconsumed makes the
// framework synthesize a FALLBACK BACK — the button already went over the wire
// as X1/X2, and it must never yank the user out of the stream.
KeyEvent.KEYCODE_BACK ->
if (event.flags and KeyEvent.FLAG_FALLBACK != 0) return true
// Leave these to the system even while streaming. // Leave these to the system even while streaming.
KeyEvent.KEYCODE_BACK, // → BackHandler leaves the stream // (BACK above → BackHandler leaves the stream.)
KeyEvent.KEYCODE_VOLUME_UP, KeyEvent.KEYCODE_VOLUME_UP,
KeyEvent.KEYCODE_VOLUME_DOWN, KeyEvent.KEYCODE_VOLUME_DOWN,
KeyEvent.KEYCODE_VOLUME_MUTE, KeyEvent.KEYCODE_VOLUME_MUTE,
@@ -394,6 +429,10 @@ class MainActivity : ComponentActivity() {
override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean { override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean {
if (streamHandle != 0L) { if (streamHandle != 0L) {
if (gamepadRouter?.onMotion(event) == true) return true if (gamepadRouter?.onMotion(event) == true) return true
// Physical mouse (uncaptured): hover motion, wheel, button edges.
if (event.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onGenericMotion(event)) return true }
}
return super.dispatchGenericMotionEvent(event) return super.dispatchGenericMotionEvent(event)
} }
// The Controllers debug screen sees pad motion before the stick→D-pad synthesis below. // The Controllers debug screen sees pad motion before the stick→D-pad synthesis below.
@@ -431,6 +470,24 @@ class MainActivity : ComponentActivity() {
return super.dispatchGenericMotionEvent(event) return super.dispatchGenericMotionEvent(event)
} }
/**
* Mouse clicks/drags ride the TOUCH stream (the pointer is "down"). While streaming they
* belong to the mouse forwarder, never to the Compose touch-gesture layer — a physical
* mouse click must be a real click at the cursor, not a synthesized trackpad tap.
*/
override fun dispatchTouchEvent(ev: MotionEvent): Boolean {
if (streamHandle != 0L && ev.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onTouchEvent(ev)) return true }
}
return super.dispatchTouchEvent(ev)
}
/** The OS is the source of truth for pointer capture (it releases on focus loss). */
override fun onPointerCaptureChanged(hasCapture: Boolean) {
super.onPointerCaptureChanged(hasCapture)
mouseForwarder?.onCaptureChanged(hasCapture)
}
/** Keys that drive the console UI — D-pad + face buttons; used to classify the last input source. */ /** Keys that drive the console UI — D-pad + face buttons; used to classify the last input source. */
private fun isConsoleNavKey(kc: Int): Boolean = when (kc) { private fun isConsoleNavKey(kc: Int): Boolean = when (kc) {
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN, KeyEvent.KEYCODE_DPAD_LEFT, KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN, KeyEvent.KEYCODE_DPAD_LEFT,
@@ -0,0 +1,206 @@
package io.unom.punktfunk
import android.view.InputDevice
import android.view.MotionEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.roundToInt
/** True when any connected input device is a pointer (USB/BT mouse, or a touchpad driving one). */
fun hasPhysicalMouse(): Boolean = InputDevice.getDeviceIds().any { id ->
InputDevice.getDevice(id)?.supportsSource(InputDevice.SOURCE_MOUSE) == true
}
/**
* Physical mouse → wire, in two modes (the iPadOS/desktop model):
* * **uncaptured** (default): hover/drag positions forward as absolute cursor moves
* (`MouseMoveAbs`, host-normalized against the window size) — desktop-style pointing. The
* local cursor is hidden over the stream (StreamScreen sets a TYPE_NULL pointer icon); the
* host's own cursor, composited into the video, is the one you see.
* * **captured**: the OS pointer is grabbed ([android.view.View.requestPointerCapture]) and raw
* relative deltas forward as `MouseMove` — FPS mouse-look. Engaged at stream start / by
* clicking into the stream when the "Capture pointer for games" setting is on, and toggled
* any time by Ctrl+Alt+Shift+Q (the cross-client chord). Focus loss releases it (the OS
* guarantees that); a click re-engages.
*
* Buttons ride [MotionEvent.ACTION_BUTTON_PRESS]/RELEASE edges (left/middle/right/back/forward →
* wire 1/2/3/4/5), the wheel rides [MotionEvent.ACTION_SCROLL] with fractional accumulation so
* high-resolution wheels don't lose sub-notch travel. Held buttons are tracked and flushed on
* capture loss / stream exit so nothing sticks on the host. Events reach this class from
* MainActivity's dispatch overrides (uncaptured) and the capture view's captured-pointer listener.
*/
class MouseForwarder(
private val handle: Long,
private val invertScroll: Boolean,
private val captureWanted: Boolean,
private val surfaceSize: () -> Pair<Int, Int>,
) {
/** Capture plumbing, owned by StreamScreen (the focusable capture view). */
var onRequestCapture: (() -> Unit)? = null
var onReleaseCapture: (() -> Unit)? = null
/** Live capture state, updated from [android.app.Activity.onPointerCaptureChanged]. */
var captured = false
private set
/** Chord-released: no auto re-engage (start / click) until the user opts back in. */
private var userReleased = false
private val heldButtons = mutableSetOf<Int>()
private var scrollAccV = 0f
private var scrollAccH = 0f
private var moveAccX = 0f
private var moveAccY = 0f
/** Uncaptured mouse events on the TOUCH stream (position while a button is down). */
fun onTouchEvent(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_DOWN -> {
if (captureWanted && !captured && !userReleased) {
// The engaging click: grab the pointer and swallow the click (desktop
// parity — the click that captures never reaches the host). The paired
// BUTTON_RELEASE is dropped by the held-set guard in [button].
onRequestCapture?.invoke()
return true
}
sendAbs(ev)
}
MotionEvent.ACTION_MOVE -> sendAbs(ev)
// Button edges are documented on the generic stream, but be robust to either.
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
}
return true
}
/** Uncaptured mouse events on the GENERIC stream (hover motion, wheel, button edges). */
fun onGenericMotion(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_HOVER_MOVE -> sendAbs(ev)
MotionEvent.ACTION_SCROLL -> wheel(ev)
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_EXIT -> {}
else -> return false
}
return true
}
/**
* Captured-pointer events (the view holds [android.view.View.requestPointerCapture]): x/y ARE
* the relative deltas ([InputDevice.SOURCE_MOUSE_RELATIVE]), batched samples included. A
* captured touchpad reports absolute finger coordinates instead — not handled (the touch
* gesture layer is the touchpad story); returning false leaves those to the framework.
*/
fun onCapturedPointer(ev: MotionEvent): Boolean {
if (!ev.isFromSource(InputDevice.SOURCE_MOUSE_RELATIVE)) return false
when (ev.actionMasked) {
MotionEvent.ACTION_MOVE -> {
var dx = 0f
var dy = 0f
for (i in 0 until ev.historySize) {
dx += ev.getHistoricalX(i)
dy += ev.getHistoricalY(i)
}
dx += ev.x
dy += ev.y
moveAccX += dx
moveAccY += dy
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept w/ sign
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_SCROLL -> wheel(ev)
}
return true
}
/** Ctrl+Alt+Shift+Q: release the grab, or (re-)engage it — works even when auto-capture is off. */
fun toggleCapture() {
if (captured) {
userReleased = true
onReleaseCapture?.invoke()
} else {
userReleased = false
onRequestCapture?.invoke()
}
}
/** Auto-engage at stream start (setting on + a mouse actually present). */
fun engageFromStart() {
if (captureWanted && !captured && !userReleased && hasPhysicalMouse()) {
onRequestCapture?.invoke()
}
}
/** From [android.app.Activity.onPointerCaptureChanged] — the OS is the source of truth. */
fun onCaptureChanged(has: Boolean) {
captured = has
// Losing the grab (focus loss, chord) must not leave buttons held on the host.
if (!has) flushButtons()
}
/** Stream teardown: lift anything held and let the grab go. */
fun release() {
flushButtons()
if (captured) onReleaseCapture?.invoke()
}
private fun sendAbs(ev: MotionEvent) {
val (w, h) = surfaceSize()
if (w <= 0 || h <= 0) return
NativeBridge.nativeSendPointerAbs(
handle,
ev.x.roundToInt().coerceIn(0, w - 1),
ev.y.roundToInt().coerceIn(0, h - 1),
w,
h,
)
}
private fun wheel(ev: MotionEvent) {
val dir = if (invertScroll) -1f else 1f
// Android: AXIS_VSCROLL + = up/away, AXIS_HSCROLL + = right — the wire's convention too.
scrollAccV += ev.getAxisValue(MotionEvent.AXIS_VSCROLL) * 120f * dir
scrollAccH += ev.getAxisValue(MotionEvent.AXIS_HSCROLL) * 120f * dir
val v = scrollAccV.toInt()
if (v != 0) {
NativeBridge.nativeSendScroll(handle, 0, v)
scrollAccV -= v
}
val h = scrollAccH.toInt()
if (h != 0) {
NativeBridge.nativeSendScroll(handle, 1, h)
scrollAccH -= h
}
}
private fun button(actionButton: Int, down: Boolean) {
val b = when (actionButton) {
MotionEvent.BUTTON_PRIMARY -> 1
MotionEvent.BUTTON_TERTIARY -> 2
MotionEvent.BUTTON_SECONDARY -> 3
MotionEvent.BUTTON_BACK -> 4
MotionEvent.BUTTON_FORWARD -> 5
else -> return
}
if (down) {
heldButtons.add(b)
NativeBridge.nativeSendPointerButton(handle, b, true)
} else if (heldButtons.remove(b)) {
// Only release what we pressed — drops the release of a swallowed engaging click
// and anything that raced a capture transition.
NativeBridge.nativeSendPointerButton(handle, b, false)
}
}
private fun flushButtons() {
heldButtons.forEach { NativeBridge.nativeSendPointerButton(handle, it, false) }
heldButtons.clear()
}
}
@@ -0,0 +1,193 @@
package io.unom.punktfunk
import android.os.Handler
import android.os.Looper
import android.view.Choreographer
import android.view.KeyEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.hypot
// Hold this long on SELECT (pointer-mode toggle) / PLAY-PAUSE (keyboard toggle) for the long-press
// action instead of the tap action.
private const val LONG_PRESS_MS = 800L
// D-pad glide ballistics, in screen-widths per second: start slow enough to hit a close button,
// ramp over RAMP_S seconds of continuous hold so crossing the desktop doesn't take all day.
private const val SPEED_MIN = 0.14f
private const val SPEED_MAX = 0.70f
private const val RAMP_S = 1.2f
/**
* Android TV remote as a pointer — the Android analogue of the Apple client's Siri-remote pointer,
* adapted for D-pad-only remotes (most Android TV remotes have no touch surface). For the
* "TV as a desktop client" use case, where a plain remote is often the only thing in hand.
*
* While streaming on a TV, **hold SELECT ≈ 0.8 s** to toggle pointer mode. While active:
* * D-pad (held) glides the host cursor with ramping acceleration (relative `MouseMove`,
* Choreographer-paced, diagonal-normalized);
* * SELECT tap = left click; PLAY/PAUSE tap = right click (Siri-remote parity);
* * PLAY/PAUSE held = toggle the on-screen keyboard; BACK = leave pointer mode
* (a second BACK then leaves the stream as usual).
* While inactive, everything except the SELECT long-press passes through untouched (D-pad =
* arrow keys, SELECT tap = Enter — synthesized on release, since the down was held back to
* disambiguate the long-press).
*
* Only consulted for non-gamepad key events on TV devices (MainActivity gates the calls); all
* state lives on the main thread.
*/
class RemotePointer(
private val handle: Long,
private val surfaceWidth: () -> Int,
private val onActiveChanged: (Boolean) -> Unit,
private val onKeyboardToggle: () -> Unit,
) {
var active = false
private set
private val handler = Handler(Looper.getMainLooper())
private val held = mutableSetOf<Int>() // D-pad keycodes currently down
private var moveAccX = 0f
private var moveAccY = 0f
private var lastFrameNs = 0L
private var rampSec = 0f
private var tickerRunning = false
private var centerLongFired = false
private var playLongFired = false
private val centerLong = Runnable {
centerLongFired = true
toggle()
}
private val playLong = Runnable {
playLongFired = true
onKeyboardToggle()
}
private val frame = object : Choreographer.FrameCallback {
override fun doFrame(nowNs: Long) {
if (!tickerRunning) return
if (held.isEmpty() || !active) {
tickerRunning = false
return
}
val dt = if (lastFrameNs == 0L) {
1f / 60f
} else {
((nowNs - lastFrameNs) / 1e9f).coerceIn(0.001f, 0.1f)
}
lastFrameNs = nowNs
rampSec += dt
var vx = 0f
var vy = 0f
if (KeyEvent.KEYCODE_DPAD_LEFT in held) vx -= 1f
if (KeyEvent.KEYCODE_DPAD_RIGHT in held) vx += 1f
if (KeyEvent.KEYCODE_DPAD_UP in held) vy -= 1f
if (KeyEvent.KEYCODE_DPAD_DOWN in held) vy += 1f
val mag = hypot(vx, vy)
if (mag > 0f) {
val w = surfaceWidth().coerceAtLeast(640)
val speed = w * (SPEED_MIN + (SPEED_MAX - SPEED_MIN) * (rampSec / RAMP_S).coerceAtMost(1f))
moveAccX += vx / mag * speed * dt
moveAccY += vy / mag * speed * dt
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
Choreographer.getInstance().postFrameCallback(this)
}
}
/** One remote key event; true = consumed. Ignore key repeats — the ticker owns motion. */
fun onKey(event: KeyEvent): Boolean {
val down = event.action == KeyEvent.ACTION_DOWN
when (event.keyCode) {
KeyEvent.KEYCODE_DPAD_CENTER -> {
if (down) {
if (event.repeatCount == 0) {
centerLongFired = false
handler.postDelayed(centerLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(centerLong)
if (!centerLongFired) {
if (active) {
click(1)
} else {
// The down was held back to disambiguate the long-press, so the
// normal path never saw it — synthesize the Enter here instead.
NativeBridge.nativeSendKey(handle, 0x0D, true, 0)
NativeBridge.nativeSendKey(handle, 0x0D, false, 0)
}
}
}
return true
}
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN,
KeyEvent.KEYCODE_DPAD_LEFT, KeyEvent.KEYCODE_DPAD_RIGHT,
-> {
if (!active) return false
if (down) {
if (held.add(event.keyCode) && held.size == 1) startTicker()
} else {
held.remove(event.keyCode)
}
return true
}
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE -> {
if (!active) return false // inactive: the media-key VK path owns it
if (down) {
if (event.repeatCount == 0) {
playLongFired = false
handler.postDelayed(playLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(playLong)
if (!playLongFired) click(3)
}
return true
}
KeyEvent.KEYCODE_BACK -> {
if (!active) return false
if (!down) toggle() // leave pointer mode; the next BACK leaves the stream
return true
}
else -> return false
}
}
/** Stream teardown: stop timers/ticker; nothing wire-held to flush (clicks are edges). */
fun release() {
handler.removeCallbacks(centerLong)
handler.removeCallbacks(playLong)
active = false
held.clear()
tickerRunning = false
}
private fun toggle() {
active = !active
if (!active) {
held.clear()
tickerRunning = false
}
onActiveChanged(active)
}
private fun startTicker() {
rampSec = 0f
lastFrameNs = 0L
if (!tickerRunning) {
tickerRunning = true
Choreographer.getInstance().postFrameCallback(frame)
}
}
private fun click(button: Int) {
NativeBridge.nativeSendPointerButton(handle, button, true)
NativeBridge.nativeSendPointerButton(handle, button, false)
}
}
@@ -109,6 +109,27 @@ data class Settings(
* setup where the OS-level pad (lizard mode) is preferred. * setup where the OS-level pad (lizard mode) is preferred.
*/ */
val sc2Capture: Boolean = true, val sc2Capture: Boolean = true,
/**
* Lock a physical mouse to the stream ([android.view.View.requestPointerCapture]) and forward
* raw relative motion — FPS mouse-look, the iPad "Capture pointer for games" twin. Engages at
* stream start and on a click into the stream; Ctrl+Alt+Shift+Q toggles it live (the chord
* works even with this off). Off (default): a mouse points absolutely, desktop-style.
*/
val pointerCapture: Boolean = false,
/**
* Flip scroll direction — the mouse wheel and the two-finger touch scroll both. Parity with
* the Apple/GTK clients' "Invert scroll direction".
*/
val invertScroll: Boolean = false,
/**
* Sync text copied on this device to the host and vice versa while streaming (the desktop
* clients' shared clipboard, text-only here). Only effective when the host advertises the
* clipboard capability; the protocol is opt-in per session either way.
*/
val clipboardSync: Boolean = true,
) )
/** [Settings.touchMode] values; persisted by name. */ /** [Settings.touchMode] values; persisted by name. */
@@ -172,6 +193,9 @@ class SettingsStore(context: Context) {
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true), autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false), rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false),
sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true), sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true),
pointerCapture = prefs.getBoolean(K_POINTER_CAPTURE, false),
invertScroll = prefs.getBoolean(K_INVERT_SCROLL, false),
clipboardSync = prefs.getBoolean(K_CLIPBOARD_SYNC, true),
) )
fun save(s: Settings) { fun save(s: Settings) {
@@ -195,6 +219,9 @@ class SettingsStore(context: Context) {
.putBoolean(K_AUTO_WAKE, s.autoWakeEnabled) .putBoolean(K_AUTO_WAKE, s.autoWakeEnabled)
.putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone) .putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone)
.putBoolean(K_SC2_CAPTURE, s.sc2Capture) .putBoolean(K_SC2_CAPTURE, s.sc2Capture)
.putBoolean(K_POINTER_CAPTURE, s.pointerCapture)
.putBoolean(K_INVERT_SCROLL, s.invertScroll)
.putBoolean(K_CLIPBOARD_SYNC, s.clipboardSync)
.apply() .apply()
} }
@@ -233,6 +260,9 @@ class SettingsStore(context: Context) {
const val K_AUTO_WAKE = "auto_wake_enabled" const val K_AUTO_WAKE = "auto_wake_enabled"
const val K_RUMBLE_ON_PHONE = "rumble_on_phone" const val K_RUMBLE_ON_PHONE = "rumble_on_phone"
const val K_SC2_CAPTURE = "sc2_capture" const val K_SC2_CAPTURE = "sc2_capture"
const val K_POINTER_CAPTURE = "pointer_capture"
const val K_INVERT_SCROLL = "invert_scroll"
const val K_CLIPBOARD_SYNC = "clipboard_sync"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */ /** Legacy Boolean the enum replaced — read once as the migration default, never written. */
const val K_TRACKPAD = "trackpad_mode" const val K_TRACKPAD = "trackpad_mode"
@@ -412,6 +412,27 @@ private fun ControlsSettings(s: Settings, update: (Settings) -> Unit, onOpenCont
style = MaterialTheme.typography.bodySmall, style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant, color = MaterialTheme.colorScheme.onSurfaceVariant,
) )
ToggleRow(
title = "Capture pointer for games",
subtitle = "Lock a connected mouse to the stream and send raw relative motion " +
"(mouse-look). Ctrl+Alt+Shift+Q toggles it live; click the stream to re-capture. " +
"Off: the mouse points at the desktop directly",
checked = s.pointerCapture,
onCheckedChange = { on -> update(s.copy(pointerCapture = on)) },
)
ToggleRow(
title = "Invert scroll direction",
subtitle = "Flip the mouse wheel and two-finger touch scrolling",
checked = s.invertScroll,
onCheckedChange = { on -> update(s.copy(invertScroll = on)) },
)
ToggleRow(
title = "Shared clipboard",
subtitle = "Text copied here pastes on the host and vice versa (hosts with " +
"clipboard sharing enabled)",
checked = s.clipboardSync,
onCheckedChange = { on -> update(s.copy(clipboardSync = on)) },
)
} }
SettingsCard { SettingsCard {
SettingDropdown( SettingDropdown(
@@ -176,6 +176,14 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// "hold to quit" hint overlay. Set from the router's onExitArmed (main thread). // "hold to quit" hint overlay. Set from the router's onExitArmed (main thread).
var exitArming by remember { mutableStateOf(false) } var exitArming by remember { mutableStateOf(false) }
// True while the TV remote is acting as a pointer (hold SELECT toggles) — drives the mode hint.
var remotePointerOn by remember { mutableStateOf(false) }
// Focus anchor the soft keyboard is summoned onto AND the pointer-capture grab target (a grab
// needs a focusable view; captured-pointer events land on it). Declared before the effect
// below so the capture callbacks can reach the view once it exists.
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) }
DisposableEffect(handle) { DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON) window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
wifiLocks.forEach { lock -> wifiLocks.forEach { lock ->
@@ -221,6 +229,54 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Show a "hold to quit" hint the moment the chord completes (the router debounces the actual // Show a "hold to quit" hint the moment the chord completes (the router debounces the actual
// exit); it clears when the buttons release early or the hold elapses. Runs on the main thread. // exit); it clears when the buttons release early or the hold elapses. Runs on the main thread.
router.onExitArmed = { armed -> exitArming = armed } router.onExitArmed = { armed -> exitArming = armed }
// Physical mouse: uncaptured hover/click/wheel forwards as absolute pointing; captured
// (setting or the Ctrl+Alt+Shift+Q chord) raw deltas forward as relative mouse-look.
// The local cursor is hidden over the stream — the host's own cursor, composited into
// the video, is the one the user sees (twin of the desktop clients' hidden cursor).
val decor = window?.decorView
val priorPointerIcon = decor?.pointerIcon
decor?.pointerIcon = android.view.PointerIcon.getSystemIcon(
context,
android.view.PointerIcon.TYPE_NULL,
)
val mouse = MouseForwarder(
handle,
invertScroll = initialSettings.invertScroll,
captureWanted = initialSettings.pointerCapture,
surfaceSize = { (decor?.width ?: 0) to (decor?.height ?: 0) },
)
mouse.onRequestCapture = {
// The grab needs the (focusable) capture view: focus it, then ask. Posted so a
// request racing view attach/focus settles on the next frame.
keyCapture?.let { v ->
v.post {
v.requestFocus()
v.requestPointerCapture()
}
}
}
mouse.onReleaseCapture = { keyCapture?.releasePointerCapture() }
activity?.mouseForwarder = mouse
// TV remote-as-pointer: hold SELECT ≈ 0.8 s to toggle; the D-pad then glides the host
// cursor (see RemotePointer). TV only — a phone's remote-less keys stay on the VK path.
val remote = if (isTv) {
RemotePointer(
handle,
surfaceWidth = { decor?.width ?: 1920 },
onActiveChanged = { on -> remotePointerOn = on },
onKeyboardToggle = { keyCapture?.let { it.setImeVisible(!it.imeShown) } },
)
} else {
null
}
activity?.remotePointer = remote
// Shared clipboard (text v1): only when the user setting is on AND the host has a
// working clipboard service. Protocol-level opt-in + the poll thread live in the sync.
val clip = if (initialSettings.clipboardSync && NativeBridge.nativeClipSupported(handle)) {
ClipboardSync(context, handle).also { it.start() }
} else {
null
}
activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate
// Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad // Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad
// index via the router; poll threads stopped + joined before the router is released and the // index via the router; poll threads stopped + joined before the router is released and the
@@ -286,6 +342,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
} }
onDispose { onDispose {
closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it
clip?.stop() // stop + join the clipboard poll thread BEFORE the handle is freed
feedback.onHidRaw = null feedback.onHidRaw = null
feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed
sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } } sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
@@ -293,6 +350,12 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down
router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener
activity?.gamepadRouter = null activity?.gamepadRouter = null
// Mouse/remote-pointer teardown: lift held buttons, drop the grab, restore the cursor.
mouse.release()
activity?.mouseForwarder = null
remote?.release()
activity?.remotePointer = null
decor?.pointerIcon = priorPointerIcon
activity?.streamHandle = 0L activity?.streamHandle = 0L
activity?.requestStreamExit = null activity?.requestStreamExit = null
// Back in the menus: the SC2 (if present) resumes driving the console UI. // Back in the menus: the SC2 (if present) resumes driving the console UI.
@@ -320,8 +383,12 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger). // Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() } BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
// Focus anchor the three-finger keyboard swipe summons the IME onto (see KeyCaptureView). // Auto-engage pointer capture at stream start (setting on + a mouse actually present).
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) } // Delayed a beat: the grab needs window focus and the capture view attached.
LaunchedEffect(handle) {
delay(400)
activity?.mouseForwarder?.engageFromStart()
}
Box(modifier = Modifier.fillMaxSize()) { Box(modifier = Modifier.fillMaxSize()) {
AndroidView( AndroidView(
@@ -379,11 +446,26 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
if (exitArming) { if (exitArming) {
ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp)) ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
} }
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe // Remote-pointer mode hint — the remote's keys are remapped while it's on, so say so.
// up in the mouse modes) — it never draws or takes touches, it just owns IME focus. if (remotePointerOn) {
RemotePointerHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
}
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe up
// in the mouse modes) AND the pointer-capture grab target — it never draws or takes
// touches, it just owns IME focus and receives captured-pointer events.
AndroidView( AndroidView(
modifier = Modifier.size(1.dp), modifier = Modifier.size(1.dp),
factory = { ctx -> KeyCaptureView(ctx).also { keyCapture = it } }, factory = { ctx ->
KeyCaptureView(ctx).also { v ->
keyCapture = v
// Real IME text path when the host types committed text (see KeyCaptureView).
v.textHandle =
if (NativeBridge.nativeTextInputSupported(handle)) handle else 0L
v.setOnCapturedPointerListener { _, ev ->
(ctx as? MainActivity)?.mouseForwarder?.onCapturedPointer(ev) ?: false
}
}
},
) )
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture // Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no // vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no
@@ -396,6 +478,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
else -> streamTouchInput( else -> streamTouchInput(
handle, handle,
trackpad = touchMode == TouchMode.TRACKPAD, trackpad = touchMode == TouchMode.TRACKPAD,
invertScroll = initialSettings.invertScroll,
onCycleStats = { statsVerbosity = statsVerbosity.next() }, onCycleStats = { statsVerbosity = statsVerbosity.next() },
onKeyboard = { show -> keyCapture?.setImeVisible(show) }, onKeyboard = { show -> keyCapture?.setImeVisible(show) },
) )
@@ -423,14 +506,35 @@ private fun ExitChordHint(modifier: Modifier = Modifier) {
) )
} }
/**
* The remote-pointer mode cue: while active the remote's keys are remapped (D-pad glides the host
* cursor, SELECT clicks), so the overlay both confirms the toggle and teaches the vocabulary.
*/
@Composable
private fun RemotePointerHint(modifier: Modifier = Modifier) {
Text(
"Remote pointer — SELECT click · play/pause right-click · hold SELECT to exit",
modifier = modifier
.background(Color.Black.copy(alpha = 0.55f), RoundedCornerShape(8.dp))
.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White,
fontSize = 15.sp,
)
}
/** /**
* Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME * Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME
* onto this view. `TYPE_NULL` puts the IME in "dumb keyboard" mode — it delivers raw [KeyEvent]s * onto this view. Two IME models, picked by the host's capabilities:
* (no composing text, no autocorrect), which flow through `MainActivity.dispatchKeyEvent` → * * **Text path** ([textHandle] set — the host advertised `HOST_CAP_TEXT_INPUT`): a real
* `Keymap.toVk` → the host, the exact path a hardware keyboard takes. Text an IME insists on * editable [HostTextConnection], so the IME gives autocorrect, gesture typing, non-Latin
* committing instead still arrives: the non-editable [BaseInputConnection] synthesizes KeyEvents * composition and emoji, all mirrored to the host as committed text + diffs.
* for it via `KeyCharacterMap` (with Shift carried as meta state — see the IME-shift wrap in * * **Fallback** (older host): `TYPE_NULL` puts the IME in "dumb keyboard" mode — raw
* `MainActivity.dispatchKeyEvent`). * [KeyEvent]s flow through `MainActivity.dispatchKeyEvent` → `Keymap.toVk` → the host, the
* exact path a hardware keyboard takes (with the IME-shift wrap documented there).
*
* Doubles as the pointer-capture grab target: a grab needs a focusable view, and captured-pointer
* events are delivered to it (routed to [MouseForwarder.onCapturedPointer] via the listener the
* stream screen installs).
*/ */
private class KeyCaptureView(context: Context) : View(context) { private class KeyCaptureView(context: Context) : View(context) {
init { init {
@@ -438,17 +542,32 @@ private class KeyCaptureView(context: Context) : View(context) {
isFocusableInTouchMode = true isFocusableInTouchMode = true
} }
/** The session handle when the host types committed text; `0` = VK-only fallback. */
var textHandle: Long = 0L
/** Whether [setImeVisible] last showed the IME — for toggle-style callers (remote pointer). */
var imeShown = false
private set
override fun onCheckIsTextEditor(): Boolean = true override fun onCheckIsTextEditor(): Boolean = true
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection { override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection {
outAttrs.inputType = InputType.TYPE_NULL outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or EditorInfo.IME_FLAG_NO_FULLSCREEN EditorInfo.IME_FLAG_NO_FULLSCREEN or EditorInfo.IME_FLAG_NO_ENTER_ACTION
return BaseInputConnection(this, false) return if (textHandle != 0L) {
outAttrs.inputType = InputType.TYPE_CLASS_TEXT or
InputType.TYPE_TEXT_FLAG_AUTO_CORRECT or InputType.TYPE_TEXT_FLAG_MULTI_LINE
HostTextConnection(this, textHandle)
} else {
outAttrs.inputType = InputType.TYPE_NULL
BaseInputConnection(this, false)
}
} }
fun setImeVisible(show: Boolean) { fun setImeVisible(show: Boolean) {
val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager
?: return ?: return
imeShown = show
if (show) { if (show) {
requestFocus() requestFocus()
imm.showSoftInput(this, 0) imm.showSoftInput(this, 0)
@@ -457,3 +576,113 @@ private class KeyCaptureView(context: Context) : View(context) {
} }
} }
} }
/**
* IME → host text bridge (the `HOST_CAP_TEXT_INPUT` path): a real **editable** connection, so
* the IME runs its full machinery (autocorrect, gesture typing, non-Latin composition), mirrored
* to the host as it happens. The one piece of host-side state tracked is *what the host currently
* shows of the active composition* ([sentComposition]): composing updates send a common-prefix
* diff (backspaces + the new suffix) so corrections materialize live on the host; a commit
* settles it. [setComposingRegion] adopts already-committed text as the active composition
* (autocorrect-revert / backspace-into-word flows), so the next update diffs against it instead
* of retyping. Newlines become Enter taps; [deleteSurroundingText] becomes Backspace/Delete taps.
*
* Known approximation: diff lengths are counted in Unicode scalars, assuming one host Backspace
* deletes one scalar — true for the composition text IMEs actually produce (emoji and other
* multi-unit graphemes commit directly rather than composing).
*/
private class HostTextConnection(
view: KeyCaptureView,
private val handle: Long,
) : BaseInputConnection(view, true) {
/** What the host currently shows of the active composition ("" = none). */
private var sentComposition = ""
override fun commitText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
sentComposition = ""
val ok = super.commitText(text, newCursorPosition)
trimEditable()
return ok
}
override fun setComposingText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
return super.setComposingText(text, newCursorPosition)
}
override fun finishComposingText(): Boolean {
// The composition text stands as committed — the host already shows it verbatim.
sentComposition = ""
return super.finishComposingText()
}
override fun setComposingRegion(start: Int, end: Int): Boolean {
val e = editable
if (e != null) {
val a = start.coerceIn(0, e.length)
val b = end.coerceIn(0, e.length)
sentComposition = e.subSequence(minOf(a, b), maxOf(a, b)).toString()
}
return super.setComposingRegion(start, end)
}
override fun deleteSurroundingText(beforeLength: Int, afterLength: Int): Boolean {
repeat(beforeLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_BACK) }
repeat(afterLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_DELETE) }
return super.deleteSurroundingText(beforeLength, afterLength)
}
override fun performEditorAction(actionCode: Int): Boolean {
tapVk(VK_RETURN)
return true
}
/** Replace the host's view of the composition with [text] via a common-prefix diff. */
private fun retype(text: String) {
var common = sentComposition.commonPrefixWith(text)
// Never split a surrogate pair mid-diff — back off to the pair boundary.
if (common.isNotEmpty() && common.last().isHighSurrogate()) {
common = common.dropLast(1)
}
val stale = sentComposition.substring(common.length)
repeat(stale.codePointCount(0, stale.length).coerceAtMost(MAX_TAPS)) { tapVk(VK_BACK) }
sendText(text.substring(common.length))
sentComposition = text
}
/** Forward literal text, turning newlines into Enter taps (control chars never ride text). */
private fun sendText(s: String) {
var chunk = StringBuilder()
for (ch in s) {
if (ch == '\n') {
if (chunk.isNotEmpty()) {
NativeBridge.nativeSendText(handle, chunk.toString())
chunk = StringBuilder()
}
tapVk(VK_RETURN)
} else {
chunk.append(ch)
}
}
if (chunk.isNotEmpty()) NativeBridge.nativeSendText(handle, chunk.toString())
}
private fun tapVk(vk: Int) {
NativeBridge.nativeSendKey(handle, vk, true, 0)
NativeBridge.nativeSendKey(handle, vk, false, 0)
}
/** Bound the mirror buffer: once nothing is composing, old text serves no purpose. */
private fun trimEditable() {
val e = editable ?: return
if (getComposingSpanStart(e) == -1 && e.length > 4000) e.clear()
}
private companion object {
const val VK_BACK = 0x08
const val VK_RETURN = 0x0D
const val VK_DELETE = 0x2E
const val MAX_TAPS = 256
}
}
@@ -99,9 +99,11 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
internal suspend fun PointerInputScope.streamTouchInput( internal suspend fun PointerInputScope.streamTouchInput(
handle: Long, handle: Long,
trackpad: Boolean, trackpad: Boolean,
invertScroll: Boolean,
onCycleStats: () -> Unit, onCycleStats: () -> Unit,
onKeyboard: (show: Boolean) -> Unit, onKeyboard: (show: Boolean) -> Unit,
) { ) {
val scrollDir = if (invertScroll) -1 else 1
var lastTapUp = 0L var lastTapUp = 0L
var lastTapX = 0f var lastTapX = 0f
var lastTapY = 0f var lastTapY = 0f
@@ -184,12 +186,12 @@ internal suspend fun PointerInputScope.streamTouchInput(
val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up
val sx = ((cx - prevCx) / SCROLL_DIV).toInt() val sx = ((cx - prevCx) / SCROLL_DIV).toInt()
if (sy != 0) { if (sy != 0) {
NativeBridge.nativeSendScroll(handle, 0, sy * 120) NativeBridge.nativeSendScroll(handle, 0, sy * 120 * scrollDir)
prevCy = cy prevCy = cy
moved = true moved = true
} }
if (sx != 0) { if (sx != 0) {
NativeBridge.nativeSendScroll(handle, 1, sx * 120) NativeBridge.nativeSendScroll(handle, 1, sx * 120 * scrollDir)
prevCx = cx prevCx = cx
moved = true moved = true
} }
@@ -106,6 +106,17 @@ object Keymap {
KeyEvent.KEYCODE_DPAD_UP -> 0x26 KeyEvent.KEYCODE_DPAD_UP -> 0x26
KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27 KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27
KeyEvent.KEYCODE_DPAD_DOWN -> 0x28 KeyEvent.KEYCODE_DPAD_DOWN -> 0x28
// TV-remote SELECT = Enter (a gamepad's press routes via SOURCE_GAMEPAD before this).
KeyEvent.KEYCODE_DPAD_CENTER -> 0x0D
// Consumer/media keys — forwarded to the host while streaming (volume stays local:
// MainActivity's pass-through list wins before the map is consulted).
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE,
KeyEvent.KEYCODE_MEDIA_PLAY,
KeyEvent.KEYCODE_MEDIA_PAUSE -> 0xB3 // VK_MEDIA_PLAY_PAUSE
KeyEvent.KEYCODE_MEDIA_NEXT -> 0xB0 // VK_MEDIA_NEXT_TRACK
KeyEvent.KEYCODE_MEDIA_PREVIOUS -> 0xB1 // VK_MEDIA_PREV_TRACK
KeyEvent.KEYCODE_MEDIA_STOP -> 0xB2 // VK_MEDIA_STOP
// Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant) // Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant)
KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0 KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0
@@ -287,6 +287,50 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */ /** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int) external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
/**
* Whether the host advertised committed-text injection (`HOST_CAP_TEXT_INPUT`) its inject
* backend can type Unicode text directly. Picks the real IME `InputConnection` (autocorrect,
* gesture typing, non-Latin scripts) over the TYPE_NULL raw-key fallback. False on `0`.
*/
external fun nativeTextInputSupported(handle: Long): Boolean
/**
* Committed IME text one `TextInput` wire event per Unicode scalar, in order. Control
* characters are skipped natively (Enter/Backspace ride [nativeSendKey]). Only meaningful
* when [nativeTextInputSupported] returned true older hosts ignore the events.
*/
external fun nativeSendText(handle: Long, text: String)
// ---- Shared clipboard (text v1): Kotlin drives ClipboardManager, Rust the protocol ----
// Opt-in per session (nativeClipControl). Local copies are announced as lazy offers; bytes
// cross only when the host pastes (a "fetch:" event answered by nativeClipServeText). Host
// copies arrive as "offer:" events, fetched eagerly into the system clipboard.
/** Whether the host advertised a working shared-clipboard service (HOST_CAP_CLIPBOARD). */
external fun nativeClipSupported(handle: Long): Boolean
/** Session-level clipboard opt-in/out; nothing happens until enabled=true crosses. */
external fun nativeClipControl(handle: Long, enabled: Boolean)
/** Announce "this device's clipboard now holds text". [seq]: monotonic, newest wins. */
external fun nativeClipOfferText(handle: Long, seq: Int)
/** Pull the text of the host's offer [seq] → transfer id echoed on "data:"/"error:", or -1. */
external fun nativeClipFetchText(handle: Long, seq: Int): Int
/** Answer a "fetch:" event with the clipboard's current text (the host is pasting). */
external fun nativeClipServeText(handle: Long, reqId: Int, text: String)
/** Abort a clipboard transfer by id (either direction). */
external fun nativeClipCancel(handle: Long, id: Int)
/**
* Block 250 ms for the next clipboard event, as a compact string: `state:<0|1>` ·
* `offer:<seq>:<hasText>` · `fetch:<reqId>` · `data:<xferId>:<text>` · `cancel:<id>` ·
* `error:<id>:<code>` · `closed` (session gone) null on timeout. Dedicated poll thread.
*/
external fun nativeNextClip(handle: Long): String?
// ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ---- // ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ----
// The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0, // The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0,
// so its wire is byte-identical to the old single-pad path. The core folds the per-transition // so its wire is byte-identical to the old single-pad path. The core folds the per-transition
@@ -0,0 +1,182 @@
//! Shared-clipboard plane (text-only v1): Kotlin drives the Android `ClipboardManager`, these
//! shims drive [`punktfunk_core::client::NativeClient`]'s clipboard surface.
//!
//! Model (mirrors the desktop clients): opt-in via `nativeClipControl(true)`; local copies are
//! announced lazily as format-list offers (`nativeClipOfferText`) and the bytes cross only when
//! the host pastes (a `fetch` event answered by `nativeClipServeText`); a host copy arrives as an
//! `offer` event, which the Kotlin side fetches eagerly (Android's clipboard has no lazy provider
//! path worth the complexity) and lands in the system clipboard on the `data` event.
//!
//! Events cross to Kotlin as compact strings from the blocking `nativeNextClip` poll (drained on
//! a dedicated thread, same pattern as `nativeNextRumble`):
//! `state:<0|1>` · `offer:<seq>:<has_text 0|1>` · `fetch:<req_id>` · `data:<xfer_id>:<text>` ·
//! `cancel:<id>` · `error:<id>:<code>` · `closed` — null on a poll timeout. Non-text fetch
//! requests are cancelled natively (only text is ever offered, so they shouldn't occur).
use std::time::Duration;
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jint, jlong, jstring};
use jni::JNIEnv;
use punktfunk_core::clipboard::ClipEventCore;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::quic::{ClipKind, CLIP_FILE_INDEX_NONE, HOST_CAP_CLIPBOARD};
use super::SessionHandle;
/// The portable wire MIME both ends map to their platform text type.
const TEXT_MIME: &str = "text/plain;charset=utf-8";
/// Deref the opaque handle (`0` → `None`).
///
/// SAFETY: live handle per the nativeConnect/nativeClose contract; every method used is `&self`
/// on the `Sync` connector.
fn client(handle: jlong) -> Option<&'static SessionHandle> {
if handle == 0 {
return None;
}
// SAFETY: see the function docs — the Kotlin side guarantees the handle outlives the call.
Some(unsafe { &*(handle as *const SessionHandle) })
}
/// `NativeBridge.nativeClipSupported(handle)` — the host advertised `HOST_CAP_CLIPBOARD`.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
client(handle).map_or(0, |h| {
u8::from(h.client.host_caps() & HOST_CAP_CLIPBOARD != 0)
})
}
/// `NativeBridge.nativeClipControl(handle, enabled)` — session-level opt-in/out. Nothing
/// clipboard-related happens on either side until an `enabled: true` crosses.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipControl(
_env: JNIEnv,
_this: JObject,
handle: jlong,
enabled: jboolean,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_control(enabled != 0, 0);
}
}
/// `NativeBridge.nativeClipOfferText(handle, seq)` — announce "the Android clipboard now holds
/// text" (format list only; bytes cross when the host fetches). `seq` is Kotlin's monotonic
/// counter, newest wins.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipOfferText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_offer(
seq as u32,
vec![ClipKind {
mime: TEXT_MIME.into(),
size_hint: 0,
}],
);
}
}
/// `NativeBridge.nativeClipFetchText(handle, seq)` — pull the text of the host's offer `seq`.
/// Returns the transfer id echoed on the matching `data:`/`error:` event, or 1.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipFetchText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) -> jint {
client(handle)
.and_then(|h| {
h.client
.clip_fetch(seq as u32, TEXT_MIME.into(), CLIP_FILE_INDEX_NONE)
.ok()
})
.map_or(-1, |xfer| xfer as jint)
}
/// `NativeBridge.nativeClipServeText(handle, reqId, text)` — answer a `fetch:` event with the
/// clipboard's current text (the host is pasting our offer).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipServeText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
req_id: jint,
text: JString,
) {
let Some(h) = client(handle) else { return };
let Ok(s) = env.get_string(&text) else {
let _ = h.client.clip_cancel(req_id as u32);
return;
};
let _ = h
.client
.clip_serve(req_id as u32, String::from(s).into_bytes(), true);
}
/// `NativeBridge.nativeClipCancel(handle, id)` — abort a transfer (either direction).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipCancel(
_env: JNIEnv,
_this: JObject,
handle: jlong,
id: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_cancel(id as u32);
}
}
/// `NativeBridge.nativeNextClip(handle)` — block ≤250 ms for the next clipboard event, encoded
/// as a compact string (module docs); null on timeout, `"closed"` once the session is gone.
/// Call from a dedicated poll thread.
///
/// Text payloads ride `data:<xfer_id>:<text>` decoded lossily — safe because the phase-0
/// clipboard task delivers a whole payload in ONE event (`last = true`), so a chunk boundary
/// can never split a UTF-8 sequence.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextClip(
env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jstring {
let Some(h) = client(handle) else {
return std::ptr::null_mut();
};
let msg = match h.client.next_clip(Duration::from_millis(250)) {
Ok(ClipEventCore::State { enabled, .. }) => format!("state:{}", u8::from(enabled)),
Ok(ClipEventCore::RemoteOffer { seq, kinds }) => {
let has_text = kinds.iter().any(|k| k.mime.starts_with("text/plain"));
format!("offer:{seq}:{}", u8::from(has_text))
}
Ok(ClipEventCore::FetchRequest { req_id, mime, .. }) => {
if mime.starts_with("text/plain") {
format!("fetch:{req_id}")
} else {
// We only ever offer text; cancel anything else rather than stall the host.
let _ = h.client.clip_cancel(req_id);
return std::ptr::null_mut();
}
}
Ok(ClipEventCore::Data { xfer_id, bytes, .. }) => {
format!("data:{xfer_id}:{}", String::from_utf8_lossy(&bytes))
}
Ok(ClipEventCore::Cancelled { id }) => format!("cancel:{id}"),
Ok(ClipEventCore::Error { id, code }) => format!("error:{id}:{code}"),
Err(PunktfunkError::NoFrame) => return std::ptr::null_mut(),
Err(_) => "closed".into(),
};
env.new_string(msg)
.map(|s| s.into_raw())
.unwrap_or(std::ptr::null_mut())
}
+41 -2
View File
@@ -6,11 +6,11 @@
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal, //! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side). //! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::{JByteBuffer, JObject}; use jni::objects::{JByteBuffer, JObject, JString};
use jni::sys::{jboolean, jint, jlong}; use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv; use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX}; use punktfunk_core::quic::{RichInput, HID_REPORT_MAX, HOST_CAP_TEXT_INPUT};
use super::SessionHandle; use super::SessionHandle;
@@ -145,6 +145,45 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
send_event(handle, kind, vk as u32, 0, 0, mods as u32); send_event(handle, kind, vk as u32, 0, 0, mods as u32);
} }
/// `NativeBridge.nativeTextInputSupported(handle)` — whether the host advertised
/// `HOST_CAP_TEXT_INPUT` (its inject backend types committed text), so the Kotlin side can pick
/// the real IME `InputConnection` over the TYPE_NULL raw-key fallback. `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_TEXT_INPUT != 0)
}
/// `NativeBridge.nativeSendText(handle, text)` — committed IME text, one `TextInput` event per
/// Unicode scalar (`code` = the scalar; multi-char commits are consecutive events in order).
/// Control characters are skipped — Enter/Backspace/Tab ride the VK key path. Call only when
/// [`Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported`] returned true.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
text: JString,
) {
if handle == 0 {
return;
}
let Ok(s) = env.get_string(&text) else {
return;
};
for ch in String::from(s).chars().filter(|c| !c.is_control()) {
send_event(handle, InputKind::TextInput, ch as u32, 0, 0, 0);
}
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input --------------- // ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried // Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried
// in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a // in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a
@@ -17,6 +17,7 @@
//! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode //! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode
//! renegotiation. Port the remaining orchestration from `clients/linux`. //! renegotiation. Port the remaining orchestration from `clients/linux`.
mod clipboard;
mod connect; mod connect;
mod input; mod input;
mod planes; mod planes;
+10 -4
View File
@@ -876,10 +876,16 @@ impl Worker {
); );
return; return;
}; };
let pref = self let pref = match self.pad_info(id) {
.pad_info(id) // Steam Input's virtual pad standing in front of the Deck's built-in controls (the
.map(|p| p.pref) // only-pad-forwarded case, [`Self::forwarded_ids`]): declare the DECK kind, not the
.unwrap_or(GamepadPref::Xbox360); // wrapper's Xbox 360 identity. [`Self::auto_pref`] already resolves the SESSION
// default this way, but a current host honors the per-pad arrival over the session
// default — so without this the host builds an X-Box 360 pad on a real Deck.
Some(p) if p.steam_virtual && is_steam_deck() => GamepadPref::SteamDeck,
Some(p) => p.pref,
None => GamepadPref::Xbox360,
};
match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) { match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) {
Ok(pad) => { Ok(pad) => {
let mut slot = Slot::new(id, index, pref, pad); let mut slot = Slot::new(id, index, pref, pad);
+5
View File
@@ -325,6 +325,11 @@ pub fn connect_reject_message(reason: punktfunk_core::reject::RejectReason) -> S
"Client and host versions don't match — update both to the same release.".into() "Client and host versions don't match — update both to the same release.".into()
} }
R::Busy => "The host is busy with another session.".into(), R::Busy => "The host is busy with another session.".into(),
R::SetupFailed => {
"The host accepted the connection but couldn't start the stream — the host's log \
(web console Log) has the cause."
.into()
}
} }
} }
+6
View File
@@ -36,6 +36,12 @@ pub fn vk_to_evdev(vk: u8) -> Option<u16> {
0x2D => Some(110), // VK_INSERT -> KEY_INSERT 0x2D => Some(110), // VK_INSERT -> KEY_INSERT
0x2E => Some(111), // VK_DELETE -> KEY_DELETE 0x2E => Some(111), // VK_DELETE -> KEY_DELETE
// --- Consumer/media keys (Android TV remotes, keyboard media rows) ---
0xB0 => Some(163), // VK_MEDIA_NEXT_TRACK -> KEY_NEXTSONG
0xB1 => Some(165), // VK_MEDIA_PREV_TRACK -> KEY_PREVIOUSSONG
0xB2 => Some(166), // VK_MEDIA_STOP -> KEY_STOPCD
0xB3 => Some(164), // VK_MEDIA_PLAY_PAUSE -> KEY_PLAYPAUSE
// --- Generic modifiers --- // --- Generic modifiers ---
0x10 => Some(42), // VK_SHIFT -> KEY_LEFTSHIFT 0x10 => Some(42), // VK_SHIFT -> KEY_LEFTSHIFT
0x11 => Some(29), // VK_CONTROL -> KEY_LEFTCTRL 0x11 => Some(29), // VK_CONTROL -> KEY_LEFTCTRL
@@ -424,6 +424,9 @@ impl InputInjector for KwinFakeInjector {
self.fake.touch_up(event.code); self.fake.touch_up(event.code);
self.fake.touch_frame(); self.fake.touch_frame();
} }
// fake_input can only press host-layout keycodes — no committed-text path (the
// HOST_CAP_TEXT_INPUT cap is not advertised on this backend).
InputKind::TextInput => {}
// Gamepads are injected through uinput, not the compositor. // Gamepads are injected through uinput, not the compositor.
InputKind::GamepadState InputKind::GamepadState
| InputKind::GamepadButton | InputKind::GamepadButton
+176 -30
View File
@@ -109,7 +109,7 @@ async fn session_main(mut rx: UnboundedReceiver<InputEvent>, source: EiSource) {
// Keep `_rd`/`_session` bound for the whole loop — dropping the portal session closes the // Keep `_rd`/`_session` bound for the whole loop — dropping the portal session closes the
// EIS connection. Bound the setup so a headless approval dialog (un-bypassed grant) can't // EIS connection. Bound the setup so a headless approval dialog (un-bypassed grant) can't
// hang the worker forever. // hang the worker forever.
let (_keepalive, context, mut events) = match tokio::time::timeout( let (_keepalive, context, mut events, output_hint) = match tokio::time::timeout(
Duration::from_secs(30), Duration::from_secs(30),
connect(source), connect(source),
) )
@@ -130,6 +130,7 @@ async fn session_main(mut rx: UnboundedReceiver<InputEvent>, source: EiSource) {
tracing::info!("libei: EIS connected — awaiting devices"); tracing::info!("libei: EIS connected — awaiting devices");
let mut state = EiState::new(); let mut state = EiState::new();
state.output_hint = output_hint;
// Watchdog: a healthy EIS server adds + resumes an input device within a beat of the handshake. // Watchdog: a healthy EIS server adds + resumes an input device within a beat of the handshake.
// If none has resumed by this deadline, the connection is dead-on-arrival (stale/half-ready // If none has resumed by this deadline, the connection is dead-on-arrival (stale/half-ready
// gamescope socket the handshake passed but no real server is behind) — exit so the next // gamescope socket the handshake passed but no real server is behind) — exit so the next
@@ -177,21 +178,29 @@ type Connected = (
Box<dyn Send>, Box<dyn Send>,
ei::Context, ei::Context,
reis::tokio::EiConvertEventStream, reis::tokio::EiConvertEventStream,
// The compositor's output size ("WxH" relay-file hint) — the scale target for absolute
// coordinates when the EIS advertises only a degenerate region (gamescope). `None` for
// the portal/Mutter paths, whose regions are real.
Option<(u32, u32)>,
); );
/// Reach an EIS server per `source` and run the EI sender handshake. /// Reach an EIS server per `source` and run the EI sender handshake.
async fn connect(source: EiSource) -> Result<Connected> { async fn connect(source: EiSource) -> Result<Connected> {
let (keepalive, stream): (Box<dyn Send>, UnixStream) = match source { let (keepalive, stream, output_hint): (Box<dyn Send>, UnixStream, Option<(u32, u32)>) =
EiSource::Portal => { match source {
let (rd, session, fd) = connect_portal().await?; EiSource::Portal => {
(Box::new((rd, session)), UnixStream::from(fd)) let (rd, session, fd) = connect_portal().await?;
} (Box::new((rd, session)), UnixStream::from(fd), None)
EiSource::MutterEis => { }
let (keepalive, fd) = connect_mutter().await?; EiSource::MutterEis => {
(keepalive, UnixStream::from(fd)) let (keepalive, fd) = connect_mutter().await?;
} (keepalive, UnixStream::from(fd), None)
EiSource::SocketPathFile(file) => (Box::new(()), connect_socket_file(&file).await?), }
}; EiSource::SocketPathFile(file) => {
let (stream, hint) = connect_socket_file(&file).await?;
(Box::new(()), stream, hint)
}
};
let context = ei::Context::new(stream).map_err(|e| anyhow!("reis EI context: {e}"))?; let context = ei::Context::new(stream).map_err(|e| anyhow!("reis EI context: {e}"))?;
// Bound the handshake. `UnixStream::connect` to a socket *file* succeeds the moment the path // Bound the handshake. `UnixStream::connect` to a socket *file* succeeds the moment the path
// exists, but a stale/half-ready gamescope (its socket created early in startup, or left behind // exists, but a stale/half-ready gamescope (its socket created early in startup, or left behind
@@ -206,7 +215,7 @@ async fn connect(source: EiSource) -> Result<Connected> {
anyhow!("EI handshake timed out (EIS server not responding — stale/half-ready socket?)") anyhow!("EI handshake timed out (EIS server not responding — stale/half-ready socket?)")
})? })?
.map_err(|e| anyhow!("EI handshake: {e}"))?; .map_err(|e| anyhow!("EI handshake: {e}"))?;
Ok((keepalive, context, events)) Ok((keepalive, context, events, output_hint))
} }
/// Open a RemoteDesktop portal session (pointer + keyboard) and obtain the EIS socket fd. /// Open a RemoteDesktop portal session (pointer + keyboard) and obtain the EIS socket fd.
@@ -294,8 +303,10 @@ async fn connect_mutter() -> Result<(Box<dyn Send>, std::os::fd::OwnedFd)> {
/// Poll `file` for the EIS socket path (the gamescope backend relays `LIBEI_SOCKET` there once /// Poll `file` for the EIS socket path (the gamescope backend relays `LIBEI_SOCKET` there once
/// the nested app launches), then connect. A bare name is resolved against `XDG_RUNTIME_DIR`, /// the nested app launches), then connect. A bare name is resolved against `XDG_RUNTIME_DIR`,
/// mirroring libei's own `LIBEI_SOCKET` semantics. /// mirroring libei's own `LIBEI_SOCKET` semantics. Line 2 of the relay file, when present,
async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> { /// carries the compositor's output size as `WxH` — returned as the absolute-coordinate scale
/// hint (gamescope's EIS region is degenerate, so the geometry can't come from the protocol).
async fn connect_socket_file(file: &std::path::Path) -> Result<(UnixStream, Option<(u32, u32)>)> {
// The relay file is rewritten each session with the CURRENT gamescope's `LIBEI_SOCKET`, and the // The relay file is rewritten each session with the CURRENT gamescope's `LIBEI_SOCKET`, and the
// socket may not be `listen()`ing the instant its name appears — or the file may briefly still // socket may not be `listen()`ing the instant its name appears — or the file may briefly still
// hold a prior, now-dead session's name (the host-lifetime injector reconnecting between // hold a prior, now-dead session's name (the host-lifetime injector reconnecting between
@@ -319,7 +330,12 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
)); ));
} }
if let Ok(s) = std::fs::read_to_string(file) { if let Ok(s) = std::fs::read_to_string(file) {
let name = s.trim(); let mut file_lines = s.lines();
let name = file_lines.next().unwrap_or("").trim();
let hint = file_lines.next().and_then(|l| {
let (w, h) = l.trim().split_once('x')?;
Some((w.parse::<u32>().ok()?, h.parse::<u32>().ok()?))
});
if !name.is_empty() { if !name.is_empty() {
let full = if name.starts_with('/') { let full = if name.starts_with('/') {
std::path::PathBuf::from(name) std::path::PathBuf::from(name)
@@ -334,7 +350,7 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
logged = name.to_string(); logged = name.to_string();
} }
match UnixStream::connect(&full) { match UnixStream::connect(&full) {
Ok(stream) => return Ok(stream), Ok(stream) => return Ok((stream, hint)),
// Refused = socket file exists but no listener yet (or a dead session); // Refused = socket file exists but no listener yet (or a dead session);
// NotFound = path not created yet. Both heal once the live gamescope's EIS is // NotFound = path not created yet. Both heal once the live gamescope's EIS is
// up — retry. Anything else (e.g. permission) is a real failure. // up — retry. Anything else (e.g. permission) is a real failure.
@@ -386,6 +402,22 @@ struct EiState {
held_keys: Vec<u32>, held_keys: Vec<u32>,
held_buttons: Vec<u32>, held_buttons: Vec<u32>,
held_touches: Vec<u32>, held_touches: Vec<u32>,
/// The touch id currently degraded to the absolute pointer ([`EiState::degrade_touch`]) —
/// the "primary finger" while the EIS has no touchscreen device. `None` between touches.
degraded_touch: Option<u32>,
/// The compositor's output size (relay-file "WxH" hint) — scale target for absolute
/// coordinates when the device's region is degenerate/absent (gamescope). Without it the
/// fallback is raw client pixels, correct only when the stream runs at the output's size.
output_hint: Option<(u32, u32)>,
}
/// Is this EIS region a plausible OUTPUT geometry — something to map normalized coordinates
/// into? gamescope advertises a degenerate `(0,0,INT32_MAX,INT32_MAX)` "everything" region on
/// its virtual input device, meaning "absolute coordinates are raw"; normalizing into it
/// explodes a center tap to x≈1e9, which the compositor clamps to the far corner. 16384
/// comfortably covers real multi-monitor layouts while rejecting the sentinel.
fn sane_region(r: &reis::event::Region) -> bool {
r.width > 0 && r.height > 0 && r.width <= 16_384 && r.height <= 16_384
} }
/// Stable small index per [`InputKind`] for the `seen_kinds` bitmask. /// Stable small index per [`InputKind`] for the `seen_kinds` bitmask.
@@ -406,6 +438,7 @@ fn kind_bit(kind: InputKind) -> u32 {
InputKind::GamepadState => 12, InputKind::GamepadState => 12,
InputKind::GamepadRemove => 13, InputKind::GamepadRemove => 13,
InputKind::GamepadArrival => 14, InputKind::GamepadArrival => 14,
InputKind::TextInput => 15,
}; };
1 << i 1 << i
} }
@@ -422,6 +455,8 @@ impl EiState {
held_keys: Vec::new(), held_keys: Vec::new(),
held_buttons: Vec::new(), held_buttons: Vec::new(),
held_touches: Vec::new(), held_touches: Vec::new(),
degraded_touch: None,
output_hint: None,
} }
} }
@@ -430,6 +465,10 @@ impl EiState {
/// normal [`EiState::inject`] path so the compositor sees proper key-up / button-up / /// normal [`EiState::inject`] path so the compositor sees proper key-up / button-up /
/// touch-up frames before the devices disappear. /// touch-up frames before the devices disappear.
fn release_all(&mut self, ctx: &ei::Context) { fn release_all(&mut self, ctx: &ei::Context) {
// A degraded touch is held as a synthesized left button (in `held_buttons`), so the
// loop below releases it — but the primary-finger latch must clear too, or the NEXT
// session's first TouchDown reads as a second finger and is ignored.
self.degraded_touch = None;
let (keys, buttons, touches) = ( let (keys, buttons, touches) = (
std::mem::take(&mut self.held_keys), std::mem::take(&mut self.held_keys),
std::mem::take(&mut self.held_buttons), std::mem::take(&mut self.held_buttons),
@@ -506,6 +545,8 @@ impl EiState {
keyboard = dev.has_capability(DeviceCapability::Keyboard), keyboard = dev.has_capability(DeviceCapability::Keyboard),
button = dev.has_capability(DeviceCapability::Button), button = dev.has_capability(DeviceCapability::Button),
scroll = dev.has_capability(DeviceCapability::Scroll), scroll = dev.has_capability(DeviceCapability::Scroll),
regions = dev.regions().len(),
region0 = ?dev.regions().first().map(|r| (r.x, r.y, r.width, r.height)),
"libei: device RESUMED (now emittable)" "libei: device RESUMED (now emittable)"
); );
} }
@@ -537,8 +578,85 @@ impl EiState {
} }
} }
/// Degrade touch to a single-finger ABSOLUTE POINTER on compositors whose EIS never
/// creates a touchscreen device (gamescope's "Gamescope Virtual Input" advertises
/// pointer/pointer_abs/button but no touch — observed live; headless KWin the same).
/// Down = abs-move + left press, move = abs-move, up = left release — synthesized through
/// the normal [`EiState::inject`] Mouse* paths so region mapping, held-state tracking and
/// [`EiState::release_all`] all apply. Only the FIRST finger drives the pointer; later
/// fingers are ignored (a pointer has no second contact — a pinch degrades to a drag).
fn degrade_touch(&mut self, ev: &InputEvent, ctx: &ei::Context) {
const GS_BUTTON_LEFT: u32 = 1;
match ev.kind {
InputKind::TouchDown => {
if self.degraded_touch.is_some() {
return; // secondary finger — single-pointer degradation
}
self.degraded_touch = Some(ev.code);
static NOTED: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(false);
if !NOTED.swap(true, std::sync::atomic::Ordering::Relaxed) {
tracing::info!(
"compositor's EIS has no touchscreen device — degrading touch to a \
single-finger absolute pointer (tap = left click; multi-touch \
gestures unavailable)"
);
}
self.inject(
&InputEvent {
kind: InputKind::MouseMoveAbs,
..*ev
},
ctx,
);
self.inject(
&InputEvent {
kind: InputKind::MouseButtonDown,
code: GS_BUTTON_LEFT,
..*ev
},
ctx,
);
}
InputKind::TouchMove if self.degraded_touch == Some(ev.code) => {
self.inject(
&InputEvent {
kind: InputKind::MouseMoveAbs,
..*ev
},
ctx,
);
}
InputKind::TouchUp if self.degraded_touch == Some(ev.code) => {
self.degraded_touch = None;
self.inject(
&InputEvent {
kind: InputKind::MouseButtonUp,
code: GS_BUTTON_LEFT,
..*ev
},
ctx,
);
}
_ => {}
}
}
/// Translate and emit one client input event, committing it as a single `frame`. /// Translate and emit one client input event, committing it as a single `frame`.
fn inject(&mut self, ev: &InputEvent, ctx: &ei::Context) { fn inject(&mut self, ev: &InputEvent, ctx: &ei::Context) {
// No ei_touchscreen device but an absolute pointer exists → degrade rather than drop
// (the game-mode "touch simply does not work" trap). Checked per event, not latched:
// a touchscreen device appearing later (compositor restart, capability change) takes
// over seamlessly on the next touch.
if matches!(
ev.kind,
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp
) && self.device_for(DeviceCapability::Touch).is_none()
&& self.device_for(DeviceCapability::PointerAbsolute).is_some()
{
self.degrade_touch(ev, ctx);
return;
}
let cap = match ev.kind { let cap = match ev.kind {
InputKind::MouseMove => DeviceCapability::Pointer, InputKind::MouseMove => DeviceCapability::Pointer,
InputKind::MouseMoveAbs => DeviceCapability::PointerAbsolute, InputKind::MouseMoveAbs => DeviceCapability::PointerAbsolute,
@@ -553,6 +671,9 @@ impl EiState {
| InputKind::GamepadAxis | InputKind::GamepadAxis
| InputKind::GamepadRemove | InputKind::GamepadRemove
| InputKind::GamepadArrival => return, // uinput path (later) | InputKind::GamepadArrival => return, // uinput path (later)
// libei presses keycodes against the server's negotiated keymap — no committed-text
// path (the HOST_CAP_TEXT_INPUT cap is not advertised on this backend).
InputKind::TextInput => return,
}; };
self.injected += 1; self.injected += 1;
let n = self.injected; let n = self.injected;
@@ -605,16 +726,31 @@ impl EiState {
InputKind::MouseMoveAbs => { InputKind::MouseMoveAbs => {
let w = ((ev.flags >> 16) & 0xffff) as f32; let w = ((ev.flags >> 16) & 0xffff) as f32;
let h = (ev.flags & 0xffff) as f32; let h = (ev.flags & 0xffff) as f32;
match ( match slot.interface::<ei::PointerAbsolute>() {
slot.interface::<ei::PointerAbsolute>(), Some(p) if w > 0.0 && h > 0.0 => {
slot.regions().first(), // Map the normalized client position into the device's first region —
) { // but only when the region looks like a real output geometry.
(Some(p), Some(region)) if w > 0.0 && h > 0.0 => { // gamescope's "Gamescope Virtual Input" advertises a degenerate
// Map the normalized client position into the device's first region. // (0,0,INT32_MAX,INT32_MAX) region meaning "coordinates are raw":
// normalizing into it explodes a center tap to x≈1e9, which gamescope
// clamps to the far corner (the observed cursor-parked-at-1279,799).
// There the managed session runs at the client's mode, so client
// pixels ARE output pixels: emit them raw.
let nx = (ev.x as f32 / w).clamp(0.0, 1.0); let nx = (ev.x as f32 / w).clamp(0.0, 1.0);
let ny = (ev.y as f32 / h).clamp(0.0, 1.0); let ny = (ev.y as f32 / h).clamp(0.0, 1.0);
let x = region.x as f32 + nx * region.width as f32; let (x, y) = match slot.regions().first().filter(|r| sane_region(r)) {
let y = region.y as f32 + ny * region.height as f32; Some(region) => (
region.x as f32 + nx * region.width as f32,
region.y as f32 + ny * region.height as f32,
),
// Degenerate/absent region: scale into the relay-file output hint
// (correct even when the client streams at a different resolution
// than the session runs); raw client pixels as the last resort.
None => match self.output_hint {
Some((ow, oh)) => (nx * ow as f32, ny * oh as f32),
None => (ev.x as f32, ev.y as f32),
},
};
p.motion_absolute(x, y); p.motion_absolute(x, y);
} }
_ => emitted = false, _ => emitted = false,
@@ -680,12 +816,21 @@ impl EiState {
InputKind::TouchDown | InputKind::TouchMove => { InputKind::TouchDown | InputKind::TouchMove => {
let w = ((ev.flags >> 16) & 0xffff) as f32; let w = ((ev.flags >> 16) & 0xffff) as f32;
let h = (ev.flags & 0xffff) as f32; let h = (ev.flags & 0xffff) as f32;
match (slot.interface::<ei::Touchscreen>(), slot.regions().first()) { match slot.interface::<ei::Touchscreen>() {
(Some(t), Some(region)) if w > 0.0 && h > 0.0 => { Some(t) if w > 0.0 && h > 0.0 => {
let nx = (ev.x as f32 / w).clamp(0.0, 1.0); let nx = (ev.x as f32 / w).clamp(0.0, 1.0);
let ny = (ev.y as f32 / h).clamp(0.0, 1.0); let ny = (ev.y as f32 / h).clamp(0.0, 1.0);
let x = region.x as f32 + nx * region.width as f32; // Same degenerate-region fallback ladder as MouseMoveAbs.
let y = region.y as f32 + ny * region.height as f32; let (x, y) = match slot.regions().first().filter(|r| sane_region(r)) {
Some(region) => (
region.x as f32 + nx * region.width as f32,
region.y as f32 + ny * region.height as f32,
),
None => match self.output_hint {
Some((ow, oh)) => (nx * ow as f32, ny * oh as f32),
None => (ev.x as f32, ev.y as f32),
},
};
if ev.kind == InputKind::TouchDown { if ev.kind == InputKind::TouchDown {
t.down(ev.code, x, y); t.down(ev.code, x, y);
} else { } else {
@@ -703,7 +848,8 @@ impl EiState {
| InputKind::GamepadButton | InputKind::GamepadButton
| InputKind::GamepadAxis | InputKind::GamepadAxis
| InputKind::GamepadRemove | InputKind::GamepadRemove
| InputKind::GamepadArrival => emitted = false, | InputKind::GamepadArrival
| InputKind::TextInput => emitted = false,
} }
if emitted { if emitted {
+96
View File
@@ -98,9 +98,26 @@ pub struct WlrootsInjector {
keyboard: ZwpVirtualKeyboardV1, keyboard: ZwpVirtualKeyboardV1,
xkb_state: xkb::State, xkb_state: xkb::State,
_keymap_file: std::fs::File, // keep the memfd alive for the compositor's mmap _keymap_file: std::fs::File, // keep the memfd alive for the compositor's mmap
/// Dedicated committed-text device ([`InputKind::TextInput`]), created on first use.
text: Option<TextKeyboard>,
start: Instant, start: Instant,
} }
/// Cap on distinct characters the dynamic text keymap holds before it restarts from scratch
/// (keycodes grow upward from 9; xkb tops out at 255, so stay well under).
const TEXT_KEYMAP_MAX: usize = 200;
/// The dedicated **text** virtual keyboard: types committed IME text (`InputKind::TextInput`,
/// one Unicode scalar per event) by growing a keymap of Unicode keysyms on demand and pressing
/// the character's keycode — the `wtype` model. A separate `zwp_virtual_keyboard` so keymap
/// re-uploads never disturb the main device's layout/modifier state that VK key events ride on.
struct TextKeyboard {
keyboard: ZwpVirtualKeyboardV1,
/// Characters in keycode order: `chars[i]` types on wire keycode `i + 1` (xkb `i + 9`).
chars: Vec<char>,
_keymap_file: Option<std::fs::File>, // keep the memfd alive for the compositor's mmap
}
impl WlrootsInjector { impl WlrootsInjector {
pub fn open() -> Result<Self> { pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env() let conn = Connection::connect_to_env()
@@ -171,6 +188,7 @@ impl WlrootsInjector {
keyboard, keyboard,
xkb_state, xkb_state,
_keymap_file: file, _keymap_file: file,
text: None,
start: Instant::now(), start: Instant::now(),
}) })
} }
@@ -179,6 +197,54 @@ impl WlrootsInjector {
self.start.elapsed().as_millis() as u32 self.start.elapsed().as_millis() as u32
} }
/// Type one committed-text Unicode scalar on the dedicated text device (created lazily),
/// growing its keymap when the character is new. Control characters are dropped — Enter,
/// Backspace and Tab ride the VK key-event path.
fn type_text(&mut self, cp: u32) -> Result<()> {
let Some(ch) = char::from_u32(cp) else {
return Ok(()); // lone surrogate / out of range
};
if ch.is_control() {
return Ok(());
}
if self.text.is_none() {
let (Some(mgr), Some(seat)) =
(self.globals.keyboard_mgr.clone(), self.globals.seat.clone())
else {
return Ok(());
};
let kb = mgr.create_virtual_keyboard(&seat, &self.queue.handle(), ());
self.text = Some(TextKeyboard {
keyboard: kb,
chars: Vec::new(),
_keymap_file: None,
});
}
let t = self.now_ms();
let text = self.text.as_mut().expect("created above");
let code = match text.chars.iter().position(|&c| c == ch) {
Some(i) => (i + 1) as u32,
None => {
if text.chars.len() >= TEXT_KEYMAP_MAX {
text.chars.clear(); // restart the map; old codes are re-assigned lazily
}
text.chars.push(ch);
let keymap_str = text_keymap(&text.chars);
let file = memfd_with(&keymap_str)?;
text.keyboard.keymap(
1, /* XKB_V1 */
file.as_fd(),
keymap_str.len() as u32 + 1,
);
text._keymap_file = Some(file);
text.chars.len() as u32
}
};
text.keyboard.key(t, code, 1);
text.keyboard.key(t, code, 0);
Ok(())
}
/// Update xkb state for a key and tell the compositor the resulting modifier mask. /// Update xkb state for a key and tell the compositor the resulting modifier mask.
fn send_modifiers(&mut self, evdev: u16, down: bool) { fn send_modifiers(&mut self, evdev: u16, down: bool) {
let kc = xkb::Keycode::new(evdev as u32 + 8); // evdev -> xkb keycode let kc = xkb::Keycode::new(evdev as u32 + 8); // evdev -> xkb keycode
@@ -254,6 +320,9 @@ impl InputInjector for WlrootsInjector {
tracing::debug!(vk = event.code, "unmapped VK keycode — dropped"); tracing::debug!(vk = event.code, "unmapped VK keycode — dropped");
} }
} }
InputKind::TextInput => {
self.type_text(event.code)?;
}
InputKind::GamepadState InputKind::GamepadState
| InputKind::GamepadButton | InputKind::GamepadButton
| InputKind::GamepadAxis | InputKind::GamepadAxis
@@ -271,6 +340,33 @@ impl InputInjector for WlrootsInjector {
} }
} }
/// Build a minimal xkb keymap whose keycode `i + 9` (wire code `i + 1`) types `chars[i]`, using
/// Unicode keysym names (`U<hex>` — xkbcommon resolves them for any scalar, emoji included).
/// Types/compat `include "complete"` mirrors `wtype`'s generated keymap — proven on wlroots
/// compositors, and the system XKB data is present (the main keymap compiled from it in `open`).
fn text_keymap(chars: &[char]) -> String {
use std::fmt::Write as _;
let mut keycodes = String::new();
let mut symbols = String::new();
for (i, ch) in chars.iter().enumerate() {
let _ = writeln!(keycodes, " <T{i}> = {};", i + 9);
let _ = writeln!(symbols, " key <T{i}> {{ [ U{:04X} ] }};", *ch as u32);
}
format!(
"xkb_keymap {{\n\
xkb_keycodes \"punktfunk-text\" {{\n\
minimum = 8;\n\
maximum = {};\n\
{keycodes}\
}};\n\
xkb_types \"punktfunk-text\" {{ include \"complete\" }};\n\
xkb_compatibility \"punktfunk-text\" {{ include \"complete\" }};\n\
xkb_symbols \"punktfunk-text\" {{\n{symbols} }};\n\
}};\n",
chars.len() + 9,
)
}
/// Create an anonymous in-memory file holding `s` + a trailing NUL (for the keymap fd). /// Create an anonymous in-memory file holding `s` + a trailing NUL (for the keymap fd).
fn memfd_with(s: &str) -> Result<std::fs::File> { fn memfd_with(s: &str) -> Result<std::fs::File> {
let name = b"punktfunk-keymap\0"; let name = b"punktfunk-keymap\0";
@@ -27,10 +27,10 @@ use windows::Win32::System::StationsAndDesktops::{
use windows::Win32::UI::Input::KeyboardAndMouse::{ use windows::Win32::UI::Input::KeyboardAndMouse::{
GetKeyboardLayout, MapVirtualKeyExW, SendInput, HKL, INPUT, INPUT_0, INPUT_KEYBOARD, GetKeyboardLayout, MapVirtualKeyExW, SendInput, HKL, INPUT, INPUT_0, INPUT_KEYBOARD,
INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY, KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE, INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY, KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE,
MAPVK_VK_TO_VSC_EX, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL, MOUSEEVENTF_LEFTDOWN, KEYEVENTF_UNICODE, MAPVK_VK_TO_VSC_EX, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL,
MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP, MOUSEEVENTF_MOVE, MOUSEEVENTF_LEFTDOWN, MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP,
MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK, MOUSEEVENTF_WHEEL, MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK,
MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY, MOUSEEVENTF_WHEEL, MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY,
}; };
use windows::Win32::UI::WindowsAndMessaging::{ use windows::Win32::UI::WindowsAndMessaging::{
GetForegroundWindow, GetSystemMetrics, GetWindowThreadProcessId, SM_CXVIRTUALSCREEN, GetForegroundWindow, GetSystemMetrics, GetWindowThreadProcessId, SM_CXVIRTUALSCREEN,
@@ -297,6 +297,33 @@ impl InputInjector for SendInputInjector {
}; };
self.send(&[key(ki)]) self.send(&[key(ki)])
} }
InputKind::TextInput => {
// Committed IME text: one Unicode scalar per event, injected as
// `KEYEVENTF_UNICODE` packets (wScan = UTF-16 unit, no scancode/layout involved
// — the receiving app gets the character verbatim via WM_CHAR). An astral-plane
// scalar (emoji) is its surrogate pair, each unit down+up in order — exactly how
// Windows expects supplementary characters from unicode injection.
let Some(ch) = char::from_u32(event.code) else {
return Ok(()); // lone surrogate / out of range — drop
};
if ch.is_control() {
return Ok(()); // control chars ride the VK path (Enter/Backspace/Tab)
}
let mut units = [0u16; 2];
let mut inputs: Vec<INPUT> = Vec::with_capacity(4);
for &unit in ch.encode_utf16(&mut units).iter() {
for flags in [KEYEVENTF_UNICODE, KEYEVENTF_UNICODE | KEYEVENTF_KEYUP] {
inputs.push(key(KEYBDINPUT {
wVk: VIRTUAL_KEY(0),
wScan: unit,
dwFlags: flags,
time: 0,
dwExtraInfo: 0,
}));
}
}
self.send(&inputs)
}
// Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op. // Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op.
InputKind::GamepadButton InputKind::GamepadButton
| InputKind::GamepadAxis | InputKind::GamepadAxis
+23
View File
@@ -174,6 +174,29 @@ pub fn default_backend() -> Backend {
Backend::Unsupported Backend::Unsupported
} }
/// Whether the session's inject backend can type **committed text**
/// ([`InputKind::TextInput`] — see `HOST_CAP_TEXT_INPUT`): Windows always (`KEYEVENTF_UNICODE`);
/// Linux only on the wlroots backend (a dedicated virtual keyboard with a dynamically-grown
/// Unicode keymap) — KWin fake-input/libei/gamescope can only press keycodes of the host layout.
/// Consulted at Welcome time to advertise the cap; a mid-session backend switch away from a
/// capable one just degrades to dropped text events (input is lossy by design).
#[cfg(target_os = "windows")]
pub fn text_input_supported() -> bool {
true
}
/// See the Windows variant: Linux types text only through the wlroots virtual-keyboard backend.
#[cfg(target_os = "linux")]
pub fn text_input_supported() -> bool {
matches!(default_backend(), Backend::WlrVirtual)
}
/// No injector ⇒ no text.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn text_input_supported() -> bool {
false
}
#[path = "inject/service.rs"] #[path = "inject/service.rs"]
mod service; mod service;
pub use service::InjectorService; pub use service::InjectorService;
+2 -2
View File
@@ -72,8 +72,8 @@ pub use session::{session_epoch, try_recover_session};
#[path = "vdisplay/routing.rs"] #[path = "vdisplay/routing.rs"]
pub(crate) mod routing; pub(crate) mod routing;
pub use routing::{ pub use routing::{
apply_input_env, restore_managed_session, restore_takeover_on_startup, start_restore_worker, apply_input_env, managed_session_available, restore_managed_session,
wants_dedicated_game_session, restore_takeover_on_startup, start_restore_worker, wants_dedicated_game_session,
}; };
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub use routing::{ pub use routing::{
@@ -1116,6 +1116,26 @@ pub fn schedule_restore_tv_session() {
} }
} }
/// Does any DRM connector report a physically `connected` display? Scans
/// `/sys/class/drm/*/status` — only connector nodes (`card0-eDP-1`, `card0-HDMI-A-1`, …) have a
/// `status` file, so the bare `cardN` device dirs and `renderD*` nodes filter themselves out. A
/// headless box (VM, panel-less mini PC) has none — in which case a "restore to the physical
/// panel" can only fail, gamescope having no output to drive. Errors (no DRM at all, sysfs
/// unreadable) read as headless: the safe direction is keeping the working session.
fn physical_display_connected() -> bool {
connected_connector_under(std::path::Path::new("/sys/class/drm"))
}
/// [`physical_display_connected`] against an arbitrary sysfs root (the unit-testable core).
fn connected_connector_under(base: &std::path::Path) -> bool {
let Ok(entries) = std::fs::read_dir(base) else {
return false;
};
entries.flatten().any(|e| {
std::fs::read_to_string(e.path().join("status")).is_ok_and(|s| s.trim() == "connected")
})
}
/// Tear down our host-managed session (freeing Steam) and restart the autologin gaming session(s) /// Tear down our host-managed session (freeing Steam) and restart the autologin gaming session(s)
/// we stopped on connect — so the TV returns to gaming mode when no one is streaming. Invoked by /// we stopped on connect — so the TV returns to gaming mode when no one is streaming. Invoked by
/// [`start_restore_worker`] once the debounce deadline passes; takes the stopped-unit list so a /// [`start_restore_worker`] once the debounce deadline passes; takes the stopped-unit list so a
@@ -1127,6 +1147,19 @@ fn do_restore_tv_session() {
{ {
let mut took = STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()); let mut took = STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner());
if *took { if *took {
// A box with no physically connected display (a VM, a panel-less mini PC) has no
// "physical gaming session" to restore TO: removing the drop-in and restarting the
// target just crash-loops gamescope (no output to drive) and strands every later
// connect on "no usable compositor". Keep the headless session — and the takeover
// state, so a same-mode reconnect reuses it warm — instead. Checked at restore time
// (not connect time) so plugging a panel in later restores normally.
if !physical_display_connected() {
tracing::info!(
"gamescope (SteamOS): no physical display connected — keeping the headless \
session (nothing to restore to)"
);
return;
}
*took = false; *took = false;
clear_takeover(); // A3: takeover undone — drop the persisted crash-restore marker clear_takeover(); // A3: takeover undone — drop the persisted crash-restore marker
*MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None; *MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None;
@@ -1226,15 +1259,31 @@ pub fn start_restore_worker() -> std::sync::Arc<()> {
/// session). Shared by the attach and host-managed-session paths. /// session). Shared by the attach and host-managed-session paths.
fn point_injector_at_eis() { fn point_injector_at_eis() {
match find_gamescope_eis_socket() { match find_gamescope_eis_socket() {
Some(sock) => match std::fs::write(ei_socket_file(), &sock) { Some(sock) => {
Ok(()) => { // Relay format: line 1 = socket, optional line 2 = the session's CURRENT output
tracing::info!(socket = %sock, "gamescope: pointed injector at the session's EIS socket") // size as "WxH". gamescope's EIS advertises only a degenerate INT32_MAX region, so
// the injector can't learn the output geometry from the protocol — the hint lets
// it scale normalized client positions correctly even when the client streams at
// a different resolution than the session runs (foreign attach, supersample).
let size = current_gamescope_output_size();
let body = match size {
Some((w, h)) => format!("{sock}\n{w}x{h}"),
None => sock.clone(),
};
match std::fs::write(ei_socket_file(), body) {
Ok(()) => {
tracing::info!(
socket = %sock,
output = ?size,
"gamescope: pointed injector at the session's EIS socket"
)
}
Err(e) => tracing::warn!(
error = %e,
"gamescope: could not write the EIS relay file — input may not reach the session"
),
} }
Err(e) => tracing::warn!( }
error = %e,
"gamescope: could not write the EIS relay file — input may not reach the session"
),
},
None => tracing::warn!( None => tracing::warn!(
"gamescope: no connectable gamescope EIS socket found — input won't reach the session" "gamescope: no connectable gamescope EIS socket found — input won't reach the session"
), ),
@@ -1524,7 +1573,35 @@ impl Drop for GamescopeProc {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{cgroup_is_punktfunk_owned, is_steam_launch, shape_dedicated_command}; use super::{
cgroup_is_punktfunk_owned, connected_connector_under, is_steam_launch,
shape_dedicated_command,
};
#[test]
fn connector_status_scan() {
let base = std::env::temp_dir().join(format!("pf-drm-scan-{}", std::process::id()));
let mk = |name: &str, status: Option<&str>| {
let dir = base.join(name);
std::fs::create_dir_all(&dir).unwrap();
if let Some(s) = status {
std::fs::write(dir.join("status"), s).unwrap();
}
};
// Headless layout: device + render nodes only (no status files) → not connected.
mk("card0", None);
mk("renderD128", None);
assert!(!connected_connector_under(&base));
// Connectors present but nothing plugged in → still not connected.
mk("card0-HDMI-A-1", Some("disconnected\n"));
assert!(!connected_connector_under(&base));
// A live panel → connected.
mk("card0-eDP-1", Some("connected\n"));
assert!(connected_connector_under(&base));
// A missing base dir (no DRM at all) reads as headless.
assert!(!connected_connector_under(&base.join("nope")));
std::fs::remove_dir_all(&base).unwrap();
}
#[test] #[test]
fn steam_launch_detection() { fn steam_launch_detection() {
@@ -207,6 +207,20 @@ pub fn cancel_pending_tv_restore() {
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
pub fn cancel_pending_tv_restore() {} pub fn cancel_pending_tv_restore() {}
/// Can the MANAGED gamescope path stand a session up from nothing on this box (SteamOS's
/// `gamescope-session` launcher or Bazzite's `gamescope-session-plus` present)? Lets the connect
/// path route a "no live graphical session" box to the gamescope takeover — which rebuilds the
/// session at the client's mode — instead of failing the connect. Always `false` off Linux.
#[cfg(target_os = "linux")]
pub fn managed_session_available() -> bool {
gamescope::managed_session_available()
}
#[cfg(not(target_os = "linux"))]
pub fn managed_session_available() -> bool {
false
}
/// Call when a client session ends: if the host-managed gamescope path took over a box's autologin /// Call when a client session ends: if the host-managed gamescope path took over a box's autologin
/// gaming session (stopped its single-instance Steam to stream at the client's mode), **schedule** a /// gaming session (stopped its single-instance Steam to stream at the client's mode), **schedule** a
/// debounced restore so the TV returns to gaming mode — unless a client reconnects within the window /// debounced restore so the TV returns to gaming mode — unless a client reconnects within the window
+12 -1
View File
@@ -861,7 +861,18 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
continue; continue;
} }
if p.flags & DISPLAYCONFIG_PATH_ACTIVE != 0 { if p.flags & DISPLAYCONFIG_PATH_ACTIVE != 0 {
p.flags &= !DISPLAYCONFIG_PATH_ACTIVE; // mark this path inactive // Mark the path inactive AND unpin its modes: per the SetDisplayConfig
// contract a path being turned OFF needs BOTH mode indexes marked invalid,
// and leaving them referencing the queried mode entries gets the whole
// supplied config rejected with 0x57 ERROR_INVALID_PARAMETER on some
// driver/topology combinations (field-reported: exclusive mode left the
// physical panel lit, every retry failing 0x57). Writing the all-ones
// sentinel to the whole union is also correct under the virtual-mode-aware
// interpretation (cloneGroupId/sourceModeInfoIdx both become their 0xffff
// INVALID values).
p.flags &= !DISPLAYCONFIG_PATH_ACTIVE;
p.sourceInfo.Anonymous.modeInfoIdx = DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
p.targetInfo.Anonymous.modeInfoIdx = DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
others += 1; others += 1;
} }
} }
+16 -4
View File
@@ -86,10 +86,22 @@ impl NativeClient {
// A typed application close from the host (pairing not armed / armed for a // A typed application close from the host (pairing not armed / armed for a
// different device / rate-limited / version mismatch) beats the generic // different device / rate-limited / version mismatch) beats the generic
// transport error the aborted exchange produced — it is the actual answer. // transport error the aborted exchange produced — it is the actual answer.
Err(e) => Err(match reject_from_close(&conn) { // Same close-vs-stream-error race as the connect handshake: give the
Some(r) => PunktfunkError::Rejected(r), // host's CONNECTION_CLOSE a short grace to be processed before deciding
None => e, // the error was plain transport trouble.
}), Err(e) => {
if conn.close_reason().is_none() {
let _ = tokio::time::timeout(
std::time::Duration::from_millis(300),
conn.closed(),
)
.await;
}
Err(match reject_from_close(&conn) {
Some(r) => PunktfunkError::Rejected(r),
None => e,
})
}
ok => ok, ok => ok,
}; };
// Always tell the host we're done so it never blocks at its read — code 0 on // Always tell the host we're done so it never blocks at its read — code 0 on
@@ -240,9 +240,22 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
negotiated, negotiated,
host_caps, host_caps,
}), }),
Err(e) => Err(match reject_from_close(&conn) { Err(e) => {
Some(r) => PunktfunkError::Rejected(r), // The host's typed close can land a beat AFTER the stream error it caused: the
None => e, // stream reset/FIN and the CONNECTION_CLOSE are in flight together, and quinn can
}), // hand the reader its mid-frame EOF before it processes the close. Give the close a
// short grace to arrive so a host-side setup failure renders as its real reason
// ("the host could not start the stream session") instead of "control stream
// finished mid-frame". No-op when the connection already closed (or never will —
// bounded by the timeout).
if conn.close_reason().is_none() {
let _ = tokio::time::timeout(std::time::Duration::from_millis(300), conn.closed())
.await;
}
Err(match reject_from_close(&conn) {
Some(r) => PunktfunkError::Rejected(r),
None => e,
})
}
} }
} }
+2
View File
@@ -61,6 +61,7 @@ pub enum PunktfunkStatus {
RejectedSuperseded = -26, RejectedSuperseded = -26,
RejectedWireVersion = -27, RejectedWireVersion = -27,
RejectedBusy = -28, RejectedBusy = -28,
RejectedSetupFailed = -29,
Panic = -99, Panic = -99,
} }
@@ -89,6 +90,7 @@ impl PunktfunkError {
R::Superseded => PunktfunkStatus::RejectedSuperseded, R::Superseded => PunktfunkStatus::RejectedSuperseded,
R::WireVersionMismatch => PunktfunkStatus::RejectedWireVersion, R::WireVersionMismatch => PunktfunkStatus::RejectedWireVersion,
R::Busy => PunktfunkStatus::RejectedBusy, R::Busy => PunktfunkStatus::RejectedBusy,
R::SetupFailed => PunktfunkStatus::RejectedSetupFailed,
} }
} }
} }
+31 -2
View File
@@ -73,6 +73,17 @@ pub enum InputKind {
/// [`HOST_CAP_GAMEPAD_STATE`](crate::quic::HOST_CAP_GAMEPAD_STATE); an older host ignores the /// [`HOST_CAP_GAMEPAD_STATE`](crate::quic::HOST_CAP_GAMEPAD_STATE); an older host ignores the
/// unknown tag (every pad then uses the session-default kind — the pre-existing behaviour). /// unknown tag (every pad then uses the session-default kind — the pre-existing behaviour).
GamepadArrival = 14, GamepadArrival = 14,
/// One Unicode scalar of **committed text** — `code` = the scalar value, everything else 0.
///
/// The IME path: the layout-independent VK key events cannot express text an input method
/// *commits* (autocorrect, gesture typing, non-Latin scripts, emoji), so a capable client
/// sends the committed characters verbatim and the host injects them directly (Windows
/// `KEYEVENTF_UNICODE`; Linux wlroots via a dynamically-grown Unicode keymap on a dedicated
/// virtual keyboard). A multi-character commit is consecutive events in order. Sent only when
/// the host advertised [`HOST_CAP_TEXT_INPUT`](crate::quic::HOST_CAP_TEXT_INPUT) — toward an
/// older host (or one whose inject backend can't type text) clients keep the best-effort VK
/// synthesis, and an older host ignores the unknown tag entirely.
TextInput = 15,
} }
/// Pack a [`InputKind::GamepadRemove`] `flags` word (`seq << 24 | pad`) — the same low-byte-pad / /// Pack a [`InputKind::GamepadRemove`] `flags` word (`seq << 24 | pad`) — the same low-byte-pad /
@@ -158,6 +169,7 @@ impl InputKind {
12 => GamepadState, 12 => GamepadState,
13 => GamepadRemove, 13 => GamepadRemove,
14 => GamepadArrival, 14 => GamepadArrival,
15 => TextInput,
_ => return None, _ => return None,
}) })
} }
@@ -392,10 +404,27 @@ mod tests {
}; };
assert_eq!(InputEvent::decode(&e.encode()), Some(e)); assert_eq!(InputEvent::decode(&e.encode()), Some(e));
} }
// GamepadRemove + GamepadArrival are valid kinds; 15 (one past them) is not. // GamepadRemove/GamepadArrival/TextInput are valid kinds; 16 (one past them) is not.
assert_eq!(InputKind::from_u8(13), Some(InputKind::GamepadRemove)); assert_eq!(InputKind::from_u8(13), Some(InputKind::GamepadRemove));
assert_eq!(InputKind::from_u8(14), Some(InputKind::GamepadArrival)); assert_eq!(InputKind::from_u8(14), Some(InputKind::GamepadArrival));
assert_eq!(InputKind::from_u8(15), None); assert_eq!(InputKind::from_u8(15), Some(InputKind::TextInput));
assert_eq!(InputKind::from_u8(16), None);
}
#[test]
fn text_input_roundtrip() {
// One Unicode scalar per event — BMP and astral (emoji) alike.
for cp in ['a' as u32, 'ß' as u32, '語' as u32, 0x1F600 /* 😀 */] {
let e = InputEvent {
kind: InputKind::TextInput,
_pad: [0; 3],
code: cp,
x: 0,
y: 0,
flags: 0,
};
assert_eq!(InputEvent::decode(&e.encode()), Some(e));
}
} }
#[test] #[test]
+10
View File
@@ -71,6 +71,16 @@ pub const HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// trailing `host_caps` byte — no wire-layout change. /// trailing `host_caps` byte — no wire-layout change.
pub const HOST_CAP_CLIPBOARD: u8 = 0x02; pub const HOST_CAP_CLIPBOARD: u8 = 0x02;
/// [`Welcome::host_caps`] bit: the host's active inject backend can type **committed text**
/// ([`InputKind::TextInput`](crate::input::InputKind::TextInput) — one Unicode scalar per event):
/// Windows (`KEYEVENTF_UNICODE`) and Linux wlroots (dynamic Unicode keymap on a dedicated virtual
/// keyboard); the KWin/libei/gamescope backends can only press layout keycodes, so those sessions
/// don't set it. A capable client routes its IME's committed text (autocorrect, gesture typing,
/// non-Latin scripts, emoji) through `TextInput` instead of lossy VK synthesis; absent the bit it
/// keeps the VK fallback. Packs into the existing trailing `host_caps` byte — no wire-layout
/// change; an older host ignores the unknown input tag anyway (input is lossy by design).
pub const HOST_CAP_TEXT_INPUT: u8 = 0x04;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software** /// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST /// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
/// advertise this. /// advertise this.
+16 -2
View File
@@ -33,6 +33,12 @@ pub const PAIR_APPROVAL_TIMEOUT_CLOSE_CODE: u32 = 0x65;
pub const PAIR_SUPERSEDED_CLOSE_CODE: u32 = 0x66; pub const PAIR_SUPERSEDED_CLOSE_CODE: u32 = 0x66;
/// The client's wire (protocol) version does not match the host's — one side needs updating. /// The client's wire (protocol) version does not match the host's — one side needs updating.
pub const WIRE_VERSION_CLOSE_CODE: u32 = 0x67; pub const WIRE_VERSION_CLOSE_CODE: u32 = 0x67;
/// The host admitted the connection but could not stand the stream session up (compositor /
/// capture / encoder setup failed host-side). The close reason bytes carry the specific error
/// text for logs/diagnostics; clients render a stable "host-side failure" sentence. Before this
/// code, a setup failure reached the client as a bare dropped connection ("control stream
/// finished mid-frame") — indistinguishable from transport trouble.
pub const SETUP_FAILED_CLOSE_CODE: u32 = 0x68;
/// Why a host turned a connection away, decoded from the QUIC application close code — the /// Why a host turned a connection away, decoded from the QUIC application close code — the
/// client-side view of [`PAIR_NOT_ARMED_CLOSE_CODE`]..[`WIRE_VERSION_CLOSE_CODE`] plus /// client-side view of [`PAIR_NOT_ARMED_CLOSE_CODE`]..[`WIRE_VERSION_CLOSE_CODE`] plus
@@ -59,6 +65,9 @@ pub enum RejectReason {
WireVersionMismatch, WireVersionMismatch,
/// The host refused admission because a conflicting session is live. /// The host refused admission because a conflicting session is live.
Busy, Busy,
/// The host admitted the connection but failed to start the stream session (host-side
/// setup error — the host log has the specific cause).
SetupFailed,
} }
impl RejectReason { impl RejectReason {
@@ -75,6 +84,7 @@ impl RejectReason {
PAIR_SUPERSEDED_CLOSE_CODE => Self::Superseded, PAIR_SUPERSEDED_CLOSE_CODE => Self::Superseded,
WIRE_VERSION_CLOSE_CODE => Self::WireVersionMismatch, WIRE_VERSION_CLOSE_CODE => Self::WireVersionMismatch,
REJECT_BUSY_CLOSE_CODE => Self::Busy, REJECT_BUSY_CLOSE_CODE => Self::Busy,
SETUP_FAILED_CLOSE_CODE => Self::SetupFailed,
_ => return None, _ => return None,
}) })
} }
@@ -91,6 +101,7 @@ impl RejectReason {
Self::Superseded => PAIR_SUPERSEDED_CLOSE_CODE, Self::Superseded => PAIR_SUPERSEDED_CLOSE_CODE,
Self::WireVersionMismatch => WIRE_VERSION_CLOSE_CODE, Self::WireVersionMismatch => WIRE_VERSION_CLOSE_CODE,
Self::Busy => REJECT_BUSY_CLOSE_CODE, Self::Busy => REJECT_BUSY_CLOSE_CODE,
Self::SetupFailed => SETUP_FAILED_CLOSE_CODE,
} }
} }
@@ -107,6 +118,7 @@ impl RejectReason {
Self::Superseded => "superseded", Self::Superseded => "superseded",
Self::WireVersionMismatch => "wire-version", Self::WireVersionMismatch => "wire-version",
Self::Busy => "busy", Self::Busy => "busy",
Self::SetupFailed => "setup-failed",
} }
} }
} }
@@ -125,6 +137,7 @@ impl std::fmt::Display for RejectReason {
Self::Superseded => "a newer request from this device replaced this one", Self::Superseded => "a newer request from this device replaced this one",
Self::WireVersionMismatch => "client and host versions do not match", Self::WireVersionMismatch => "client and host versions do not match",
Self::Busy => "the host is busy with another session", Self::Busy => "the host is busy with another session",
Self::SetupFailed => "the host could not start the stream session",
}) })
} }
} }
@@ -133,7 +146,7 @@ impl std::fmt::Display for RejectReason {
mod tests { mod tests {
use super::*; use super::*;
const ALL: [RejectReason; 9] = [ const ALL: [RejectReason; 10] = [
RejectReason::PairingNotArmed, RejectReason::PairingNotArmed,
RejectReason::PairingBoundToOtherDevice, RejectReason::PairingBoundToOtherDevice,
RejectReason::PairingRateLimited, RejectReason::PairingRateLimited,
@@ -143,6 +156,7 @@ mod tests {
RejectReason::Superseded, RejectReason::Superseded,
RejectReason::WireVersionMismatch, RejectReason::WireVersionMismatch,
RejectReason::Busy, RejectReason::Busy,
RejectReason::SetupFailed,
]; ];
#[test] #[test]
@@ -164,7 +178,7 @@ mod tests {
fn foreign_codes_stay_untyped() { fn foreign_codes_stay_untyped() {
// Bare closes, the client's own pair-done codes, and the deliberate-end codes must // Bare closes, the client's own pair-done codes, and the deliberate-end codes must
// never read as a host rejection. // never read as a host rejection.
for code in [0u32, 1, 0x41, 0x51, 0x52, 0x5f, 0x68, u32::MAX] { for code in [0u32, 1, 0x41, 0x51, 0x52, 0x5f, 0x69, u32::MAX] {
assert_eq!(RejectReason::from_close_code(code), None); assert_eq!(RejectReason::from_close_code(code), None);
} }
} }
+30
View File
@@ -30,6 +30,36 @@ pub fn input_test() -> Result<()> {
y, y,
flags: 0, flags: 0,
}; };
// `PUNKTFUNK_INPUT_TEST_ABS=WxH` (e.g. 1280x800): exercise ABSOLUTE pointer moves instead —
// steps through the corners + center of the given surface, 1s apart, so an observer
// (`DISPLAY=:0 xdotool getmouselocation`) can verify each jump. This is the degraded-touch
// path (touch → MouseMoveAbs), so it validates game-mode touch without a client.
if let Ok(dims) = std::env::var("PUNKTFUNK_INPUT_TEST_ABS") {
let (w, h) = dims
.split_once('x')
.and_then(|(w, h)| Some((w.parse::<u32>().ok()?, h.parse::<u32>().ok()?)))
.unwrap_or((1280, 800));
let flags = (w << 16) | (h & 0xffff);
let pts = [
(100, 100),
(w as i32 - 100, 100),
(w as i32 - 100, h as i32 - 100),
(100, h as i32 - 100),
(w as i32 / 2, h as i32 / 2),
];
tracing::info!(w, h, "input-test: ABS mode — corners + center, 1s apart");
for (x, y) in pts {
let mut e = ev(InputKind::MouseMoveAbs, 0, x, y);
e.flags = flags;
if let Err(err) = inj.inject(&e) {
tracing::warn!(error = %format!("{err:#}"), "input-test: abs inject failed");
}
tracing::info!(x, y, "input-test: abs move emitted");
std::thread::sleep(Duration::from_secs(1));
}
tracing::info!("input-test: done (abs)");
return Ok(());
}
tracing::info!( tracing::info!(
"input-test: injecting a mouse square + 'A'/click taps for ~8s (watch wev / focused app)" "input-test: injecting a mouse square + 'A'/click taps for ~8s (watch wev / focused app)"
); );
+34 -2
View File
@@ -35,7 +35,24 @@ pub fn decode(plaintext: &[u8]) -> Vec<InputEvent> {
if plaintext.len() < 4 || u16::from_le_bytes([plaintext[0], plaintext[1]]) != INPUT_DATA_TYPE { if plaintext.len() < 4 || u16::from_le_bytes([plaintext[0], plaintext[1]]) != INPUT_DATA_TYPE {
return Vec::new(); return Vec::new();
} }
decode_input_packet(&plaintext[4..]).into_iter().collect() let p = &plaintext[4..];
// UTF-8 text (Moonlight's client-side keyboard commit) expands to one `TextInput` event per
// Unicode scalar — the only magic yielding more than one event, so it's handled before the
// single-event dispatch. Injected the same way as the native plane's IME text.
if p.len() >= 8 && u32::from_le_bytes([p[4], p[5], p[6], p[7]]) == MAGIC_UTF8 {
// NV_INPUT_HEADER.size (BE, excludes itself) counts magic + body.
let size = u32::from_be_bytes([p[0], p[1], p[2], p[3]]) as usize;
let body_len = size.saturating_sub(4).min(p.len() - 8);
return match std::str::from_utf8(&p[8..8 + body_len]) {
Ok(s) => s
.chars()
.filter(|c| !c.is_control())
.map(|c| ev(InputKind::TextInput, c as u32, 0, 0, 0))
.collect(),
Err(_) => Vec::new(),
};
}
decode_input_packet(p).into_iter().collect()
} }
fn decode_input_packet(p: &[u8]) -> Option<InputEvent> { fn decode_input_packet(p: &[u8]) -> Option<InputEvent> {
@@ -89,7 +106,7 @@ fn decode_input_packet(p: &[u8]) -> Option<InputEvent> {
modifiers | crate::inject::KEY_FLAG_SEMANTIC_VK, modifiers | crate::inject::KEY_FLAG_SEMANTIC_VK,
) )
} }
// UTF-8 text, gamepad, pen, touch, haptics — not yet injected. // Gamepad, pen, touch, haptics — not yet injected. (UTF-8 text is handled in `decode`.)
_ => return None, _ => return None,
}) })
} }
@@ -144,6 +161,21 @@ mod tests {
assert_eq!(ev[0].flags, 0x04 | crate::inject::KEY_FLAG_SEMANTIC_VK); assert_eq!(ev[0].flags, 0x04 | crate::inject::KEY_FLAG_SEMANTIC_VK);
} }
#[test]
fn decodes_utf8_text_per_scalar() {
// "aß😀" — ASCII, Latin-1, and an astral scalar; one TextInput event per scalar.
let pt = wrap(MAGIC_UTF8, "aß😀".as_bytes());
let ev = decode(&pt);
assert_eq!(ev.len(), 3);
assert!(ev.iter().all(|e| e.kind == InputKind::TextInput));
assert_eq!(ev[0].code, 'a' as u32);
assert_eq!(ev[1].code, 'ß' as u32);
assert_eq!(ev[2].code, 0x1F600);
// Truncated / invalid UTF-8 decodes to nothing rather than mojibake.
let bad = wrap(MAGIC_UTF8, &[0xff, 0xfe]);
assert!(decode(&bad).is_empty());
}
#[test] #[test]
fn ignores_non_input_type() { fn ignores_non_input_type() {
let mut pt = vec![0x00, 0x02]; // type 0x0200 (keepalive) let mut pt = vec![0x00, 0x02]; // type 0x0200 (keepalive)
@@ -60,6 +60,9 @@ pub fn start(
// Same scheduling posture as the native path's capture/encode thread (Linux nice -10 / // Same scheduling posture as the native path's capture/encode thread (Linux nice -10 /
// Windows HIGHEST + session tuning) — GameStream previously ran unboosted on Linux. // Windows HIGHEST + session tuning) — GameStream previously ran unboosted on Linux.
crate::native::boost_thread_priority(true); crate::native::boost_thread_priority(true);
// A GameStream viewer may be video-only too — hold the suspend/idle inhibitor for
// this stream's lifetime (plane parity with the native LiveSessionGuard).
let _sleep = crate::sleep_inhibit::hold();
tracing::info!(?cfg, "video stream starting"); tracing::info!(?cfg, "video stream starting");
// Lifecycle events + the script-facing marker file, plane parity with the native loop // Lifecycle events + the script-facing marker file, plane parity with the native loop
// (RFC §4): `announce` emits `stream.started`/`stream.stopped` and holds the marker for // (RFC §4): `announce` emits `stream.started`/`stream.stopped` and holds the marker for
+1
View File
@@ -72,6 +72,7 @@ mod send_pacing;
mod service; mod service;
mod session_plan; mod session_plan;
mod session_status; mod session_status;
mod sleep_inhibit;
mod spike; mod spike;
mod stats_recorder; mod stats_recorder;
// The plugin store: signed catalogs, tiered trust, and install/uninstall jobs that run through the // The plugin store: signed catalogs, tiered trust, and install/uninstall jobs that run through the
+21 -1
View File
@@ -423,6 +423,10 @@ pub(crate) async fn serve(
// permit's lifetime: it's released while a knock is parked for delegated approval and // permit's lifetime: it's released while a knock is parked for delegated approval and
// re-acquired on approval, so the hold is no longer a simple closure-scoped binding. // re-acquired on approval, so the hold is no longer a simple closure-scoped binding.
let sem_session = sem.clone(); let sem_session = sem.clone();
// Kept for the error path below: `serve_session` consumes `conn`, but a setup failure
// must still close the connection with a typed reason (quinn connections are cheap
// Arc-handle clones).
let conn_err = conn.clone();
sessions.spawn(async move { sessions.spawn(async move {
match serve_session( match serve_session(
conn, conn,
@@ -445,7 +449,23 @@ pub(crate) async fn serve(
"closed before the control handshake (reachability probe)" "closed before the control handshake (reachability probe)"
), ),
Err(e) => { Err(e) => {
tracing::warn!(%peer, error = %format!("{e:#}"), "session ended with error") // Make the failure legible to the client (the [`close_rejected`] discipline,
// extended to EVERY session error): a setup failure that just drops the
// connection reaches the client as a bare close mid-control-frame ("control
// stream finished mid-frame") — indistinguishable from transport trouble.
// Close with the typed setup-failed code, carrying the error text in the
// reason bytes for client-side logs. When a gate already closed with its own
// typed code, or the peer closed first, this close is a no-op (first wins).
let detail = format!("{e:#}");
let mut cut = detail.len().min(256);
while !detail.is_char_boundary(cut) {
cut -= 1;
}
conn_err.close(
punktfunk_core::reject::SETUP_FAILED_CLOSE_CODE.into(),
&detail.as_bytes()[..cut],
);
tracing::warn!(%peer, error = %detail, "session ended with error")
} }
} }
}); });
+17 -1
View File
@@ -92,8 +92,24 @@ pub(super) fn resolve_compositor(
let available = crate::vdisplay::available(); let available = crate::vdisplay::available();
let chosen = match pick_compositor(pref, &available, detected) { let chosen = match pick_compositor(pref, &available, detected) {
Some(c) => c, Some(c) => c,
// No live session, but the MANAGED gamescope infra exists (SteamOS's
// `gamescope-session`, Bazzite's `gamescope-session-plus`): route to the gamescope
// backend anyway — its managed path stands the session up from nothing at the
// client's mode (drop-in takeover / session relaunch), so a dead gaming session
// self-heals on the next connect instead of bouncing every client until someone
// restarts it by hand. (The trap that motivated this: a headless SteamOS box whose
// gamescope died — every connect failed "no usable compositor" even though the
// takeover could rebuild it.) Not under an operator pin: an explicit
// `PUNKTFUNK_COMPOSITOR` keeps its exact, hand-configured meaning.
None if !overridden && crate::vdisplay::managed_session_available() => {
tracing::info!(
"no live graphical session — managed gamescope infra present; routing to \
the managed takeover to revive the session"
);
Compositor::Gamescope
}
None => { None => {
// No live session — the state a compositor crash leaves behind (gnome-shell // The state a compositor crash leaves behind (gnome-shell
// SIGSEGV → GDM greeter, whose auto-login is once-per-boot). If the operator // SIGSEGV → GDM greeter, whose auto-login is once-per-boot). If the operator
// configured a recovery hook, fire it (debounced) and tell the client to retry: // configured a recovery hook, fire it (debounced) and tell the client to retry:
// its next knock lands in the recovered desktop. // its next knock lands in the recovered desktop.
@@ -444,6 +444,15 @@ pub(super) async fn negotiate(
punktfunk_core::quic::HOST_CAP_CLIPBOARD punktfunk_core::quic::HOST_CAP_CLIPBOARD
} else { } else {
0 0
}
// Committed-text injection (InputKind::TextInput): only where the session's inject
// backend can actually type text — Windows SendInput (KEYEVENTF_UNICODE) and the
// Linux wlroots virtual keyboard (dynamic Unicode keymap). Clients without the bit
// keep their VK-synthesis fallback for IME text.
| if crate::inject::text_input_supported() {
punktfunk_core::quic::HOST_CAP_TEXT_INPUT
} else {
0
}, },
// The negotiated session AEAD (resolved above) + its 32-byte key toward a ChaCha // The negotiated session AEAD (resolved above) + its 32-byte key toward a ChaCha
// client; toward everyone else cipher 0 keeps the Welcome byte-identical to the // client; toward everyone else cipher 0 keeps the Welcome byte-identical to the
+17 -1
View File
@@ -158,9 +158,25 @@ pub(crate) fn runner_command() -> Result<(std::path::PathBuf, Vec<String>)> {
if bun.exists() && runner.exists() { if bun.exists() && runner.exists() {
return Ok((bun, vec![runner.to_string_lossy().into_owned()])); return Ok((bun, vec![runner.to_string_lossy().into_owned()]));
} }
// Immutable-/usr distros (SteamOS): scripts/steamdeck/install.sh lays the SAME payload
// out user-scoped under ~/.local — wrapper, private bun, and bundle mirroring the deb's
// /usr layout — because a system package can't exist there.
if let Ok(home) = std::env::var("HOME") {
let home = std::path::Path::new(&home);
let wrapper = home.join(".local/bin/punktfunk-scripting");
if wrapper.exists() {
return Ok((wrapper, Vec::new()));
}
let bun = home.join(".local/lib/punktfunk-scripting/bun");
let runner = home.join(".local/share/punktfunk-scripting/runner-cli.js");
if bun.exists() && runner.exists() {
return Ok((bun, vec![runner.to_string_lossy().into_owned()]));
}
}
bail!( bail!(
"the plugin runner isn't installed — install it first (Debian/Ubuntu: \ "the plugin runner isn't installed — install it first (Debian/Ubuntu: \
`sudo apt install punktfunk-scripting`)" `sudo apt install punktfunk-scripting`; SteamOS: re-run \
scripts/steamdeck/install.sh)"
) )
} }
} }
+7 -1
View File
@@ -114,12 +114,18 @@ pub fn register(
session: session_ref(&session), session: session_ref(&session),
}); });
registry().lock().unwrap().push(session); registry().lock().unwrap().push(session);
LiveSessionGuard { id } LiveSessionGuard {
id,
_sleep: crate::sleep_inhibit::hold(),
}
} }
/// Removes its session from the registry when dropped (any scope exit of the native video loop). /// Removes its session from the registry when dropped (any scope exit of the native video loop).
pub struct LiveSessionGuard { pub struct LiveSessionGuard {
id: u64, id: u64,
/// While any native session lives, the box must not auto-suspend under a passive viewer
/// ([`crate::sleep_inhibit`]) — refcounted, released with the guard.
_sleep: crate::sleep_inhibit::StreamHold,
} }
impl Drop for LiveSessionGuard { impl Drop for LiveSessionGuard {
+100
View File
@@ -0,0 +1,100 @@
//! Session-scoped suspend/idle inhibition: while at least one client is streaming, the host
//! holds a logind `sleep:idle` BLOCK inhibitor so the box doesn't auto-suspend out from under a
//! passive viewer. Remote INPUT resets the compositor's idle timers, but a video-only viewer
//! sends none — observed live on a SteamOS Game-Mode host, which s2idled mid-stream-day and
//! dropped off the network (and, in a VM with GPU passthrough, never woke again). Refcounted
//! across planes (native sessions + GameStream media): the first hold acquires, the last drop
//! releases. Best-effort — no logind (containers, non-systemd boxes) logs once and streams on.
//! Off Linux this is a no-op: macOS/Windows hosts manage their own power assertions.
use std::sync::{Mutex, OnceLock};
/// RAII share of the host-wide inhibitor — hold one per live session/stream.
pub struct StreamHold(());
struct State {
count: u32,
/// The logind inhibitor pipe fd — inhibition lasts exactly as long as it stays open.
#[cfg(target_os = "linux")]
fd: Option<ashpd::zbus::zvariant::OwnedFd>,
}
fn state() -> &'static Mutex<State> {
static S: OnceLock<Mutex<State>> = OnceLock::new();
S.get_or_init(|| {
Mutex::new(State {
count: 0,
#[cfg(target_os = "linux")]
fd: None,
})
})
}
/// Take a share; the underlying inhibitor is acquired on the 0→1 edge.
pub fn hold() -> StreamHold {
let mut st = state().lock().unwrap_or_else(|e| e.into_inner());
st.count += 1;
#[cfg(target_os = "linux")]
if st.count == 1 && st.fd.is_none() {
st.fd = acquire();
}
StreamHold(())
}
impl Drop for StreamHold {
fn drop(&mut self) {
let mut st = state().lock().unwrap_or_else(|e| e.into_inner());
st.count = st.count.saturating_sub(1);
#[cfg(target_os = "linux")]
if st.count == 0 && st.fd.take().is_some() {
tracing::info!("released the sleep/idle inhibitor (no live sessions)");
}
}
}
/// One logind `Inhibit` call on a dedicated plain thread — zbus's blocking API must not run on
/// a tokio worker (its internal `block_on` panics there), and callers of [`hold`] may be either.
/// The join blocks the caller for the D-Bus round-trip (~ms), which every call site tolerates.
#[cfg(target_os = "linux")]
fn acquire() -> Option<ashpd::zbus::zvariant::OwnedFd> {
let fd = std::thread::spawn(|| -> Option<ashpd::zbus::zvariant::OwnedFd> {
use ashpd::zbus;
// zbus's blocking API is configured out by ashpd's feature set — drive the async API on
// a private current-thread runtime instead (still on this plain thread; see fn doc).
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.ok()?;
let attempt: zbus::Result<zbus::zvariant::OwnedFd> = rt.block_on(async {
let conn = zbus::Connection::system().await?;
let reply = conn
.call_method(
Some("org.freedesktop.login1"),
"/org/freedesktop/login1",
Some("org.freedesktop.login1.Manager"),
"Inhibit",
&("sleep:idle", "Punktfunk", "a client is streaming", "block"),
)
.await?;
reply.body().deserialize()
});
match attempt {
Ok(fd) => Some(fd),
Err(e) => {
tracing::warn!(
error = %e,
"could not take a logind sleep/idle inhibitor — the box may auto-suspend \
under a passive (video-only) viewer"
);
None
}
}
})
.join()
.ok()
.flatten();
if fd.is_some() {
tracing::info!("holding a logind sleep/idle inhibitor while clients stream");
}
fd
}
+4
View File
@@ -70,6 +70,10 @@ These apply to the **Gaming Mode (gamescope)** path only; the desktop path is un
- **gamescope 3.16.22 or newer is required.** Older versions can deadlock during capture. Bazzite's - **gamescope 3.16.22 or newer is required.** Older versions can deadlock during capture. Bazzite's
and SteamOS's current gamescope is fine; this only bites if you've pinned an old one. and SteamOS's current gamescope is fine; this only bites if you've pinned an old one.
- **The mouse cursor isn't included in the captured image** — a gamescope limitation for now. - **The mouse cursor isn't included in the captured image** — a gamescope limitation for now.
- **Touch arrives as a single-finger pointer.** gamescope's virtual input device has no
touchscreen, so the host maps a client's touchscreen to an absolute pointer: taps click exactly
where you touch and drags work, but multi-touch gestures (pinch) aren't available in Gaming
Mode. The desktop path has full multi-touch.
- **HDR isn't supported on the gamescope path** — gamescope's capture output is 8-bit. SDR streams - **HDR isn't supported on the gamescope path** — gamescope's capture output is 8-bit. SDR streams
normally. normally.
+4
View File
@@ -69,6 +69,10 @@ punktfunk-host plugins add playnite # or: rom-manager
punktfunk-host plugins enable # turn the runner on (once) punktfunk-host plugins enable # turn the runner on (once)
``` ```
On **SteamOS** the [host installer](/docs/steamos-host) ships the runner automatically (user-scoped
under `~/.local` — the read-only `/usr` can't take the package). If the console reports the runner
isn't installed on an older setup, re-run `scripts/steamdeck/update.sh` once.
</Tab> </Tab>
<Tab value="Windows"> <Tab value="Windows">
+13 -5
View File
@@ -56,14 +56,17 @@ bash ~/punktfunk/scripts/steamdeck/install.sh
It is idempotent — safe to re-run. In one pass it: It is idempotent — safe to re-run. In one pass it:
1. creates the `pf2` Debian-trixie distrobox and installs the build toolchain, 1. creates the `pf2` Debian-trixie distrobox and installs the build toolchain,
2. builds `punktfunk-host` (and the web console), 2. builds `punktfunk-host`, the web console, and the **plugin/script runner** (so the console's
[plugin store](/docs/plugins) works out of the box — the runner service itself stays opt-in),
3. writes config to `~/.config/punktfunk/` (a generated web-console login password), 3. writes config to `~/.config/punktfunk/` (a generated web-console login password),
4. raises the UDP socket buffers to 32 MB, installs the gamepad udev rule + the `vhci-hcd` autoload 4. raises the UDP socket buffers to 32 MB, installs the gamepad udev rule + the `vhci-hcd` autoload
and adds you to the `input` group (virtual gamepads / **native Steam Deck controller passthrough**), and adds you to the `input` group (virtual gamepads / **native Steam Deck controller passthrough**),
and seeds the KDE RemoteDesktop grant for Desktop-mode input — this step **prompts for your `sudo` seeds the KDE RemoteDesktop grant for Desktop-mode input, and **registers all of it on SteamOS's
atomic-update keep list** so OS updates carry it over — this step **prompts for your `sudo`
password** (a stock Steam Deck requires one; without it gamepad passthrough and the UDP tuning are skipped), password** (a stock Steam Deck requires one; without it gamepad passthrough and the UDP tuning are skipped),
5. installs + starts the `punktfunk-host` and `punktfunk-web` **systemd user services** (with linger, 5. installs + starts the `punktfunk-host` and `punktfunk-web` **systemd user services** (with linger,
so they run without a login session). so they run without a login session) plus a boot-time **rebuild check** that repairs the host
automatically if a SteamOS update ever breaks its library links.
Useful flags: Useful flags:
@@ -144,8 +147,13 @@ bash ~/punktfunk/scripts/steamdeck/update.sh
degrades to a generic Xbox 360 controller (still fully playable). If you're streaming *to* another degrades to a generic Xbox 360 controller (still fully playable). If you're streaming *to* another
Steam Deck, also set Steam Input to **Off** for Punktfunk on that Deck — see Steam Deck, also set Steam Input to **Off** for Punktfunk on that Deck — see
[Stream to a Steam Deck](/docs/steam-deck). [Stream to a Steam Deck](/docs/steam-deck).
- **It survives OS updates**, but a major SteamOS bump can move library versions; if the host fails to - **It survives OS updates — automatically.** SteamOS A/B updates rebuild `/etc` and can move
start after an update, just re-run `update.sh` to rebuild against the new base. library versions; the installer defends both sides. The system tuning (gamepad udev rule,
`vhci-hcd`, UDP buffers) is registered on SteamOS's own atomic-update keep list
(`/etc/atomic-update.conf.d/`), so updates carry it over; and a boot-time check
(`punktfunk-rebuild-check`) probes the host binary and re-runs the build only if the new OS
actually broke its library links — you should never need to intervene. (Re-running `update.sh`
by hand still works and is harmless.)
- Deeper reference (services, container, manual steps): [`scripts/steamdeck/README.md`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/steamdeck/README.md). - Deeper reference (services, container, manual steps): [`scripts/steamdeck/README.md`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/steamdeck/README.md).
Trouble? See [Troubleshooting](/docs/troubleshooting) and [Pairing](/docs/pairing). Trouble? See [Troubleshooting](/docs/troubleshooting) and [Pairing](/docs/pairing).
+31
View File
@@ -498,6 +498,18 @@
#define HOST_CAP_CLIPBOARD 2 #define HOST_CAP_CLIPBOARD 2
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host's active inject backend can type **committed text**
// ([`InputKind::TextInput`](crate::input::InputKind::TextInput) — one Unicode scalar per event):
// Windows (`KEYEVENTF_UNICODE`) and Linux wlroots (dynamic Unicode keymap on a dedicated virtual
// keyboard); the KWin/libei/gamescope backends can only press layout keycodes, so those sessions
// don't set it. A capable client routes its IME's committed text (autocorrect, gesture typing,
// non-Latin scripts, emoji) through `TextInput` instead of lossy VK synthesis; absent the bit it
// keeps the VK fallback. Packs into the existing trailing `host_caps` byte — no wire-layout
// change; an older host ignores the unknown input tag anyway (input is lossy by design).
#define HOST_CAP_TEXT_INPUT 4
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software** // [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST // encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
@@ -977,6 +989,13 @@
// The client's wire (protocol) version does not match the host's — one side needs updating. // The client's wire (protocol) version does not match the host's — one side needs updating.
#define WIRE_VERSION_CLOSE_CODE 103 #define WIRE_VERSION_CLOSE_CODE 103
// The host admitted the connection but could not stand the stream session up (compositor /
// capture / encoder setup failed host-side). The close reason bytes carry the specific error
// text for logs/diagnostics; clients render a stable "host-side failure" sentence. Before this
// code, a setup failure reached the client as a bare dropped connection ("control stream
// finished mid-frame") — indistinguishable from transport trouble.
#define SETUP_FAILED_CLOSE_CODE 104
// Minimum supported multiplier (renders under native, upscaled on present). // Minimum supported multiplier (renders under native, upscaled on present).
#define MIN_SCALE 0.5 #define MIN_SCALE 0.5
@@ -1010,6 +1029,7 @@ enum PunktfunkStatus
PUNKTFUNK_STATUS_REJECTED_SUPERSEDED = -26, PUNKTFUNK_STATUS_REJECTED_SUPERSEDED = -26,
PUNKTFUNK_STATUS_REJECTED_WIRE_VERSION = -27, PUNKTFUNK_STATUS_REJECTED_WIRE_VERSION = -27,
PUNKTFUNK_STATUS_REJECTED_BUSY = -28, PUNKTFUNK_STATUS_REJECTED_BUSY = -28,
PUNKTFUNK_STATUS_REJECTED_SETUP_FAILED = -29,
PUNKTFUNK_STATUS_PANIC = -99, PUNKTFUNK_STATUS_PANIC = -99,
}; };
#ifndef __cplusplus #ifndef __cplusplus
@@ -1086,6 +1106,17 @@ enum PunktfunkInputKind
// [`HOST_CAP_GAMEPAD_STATE`](crate::quic::HOST_CAP_GAMEPAD_STATE); an older host ignores the // [`HOST_CAP_GAMEPAD_STATE`](crate::quic::HOST_CAP_GAMEPAD_STATE); an older host ignores the
// unknown tag (every pad then uses the session-default kind — the pre-existing behaviour). // unknown tag (every pad then uses the session-default kind — the pre-existing behaviour).
PUNKTFUNK_INPUT_KIND_GAMEPAD_ARRIVAL = 14, PUNKTFUNK_INPUT_KIND_GAMEPAD_ARRIVAL = 14,
// One Unicode scalar of **committed text** — `code` = the scalar value, everything else 0.
//
// The IME path: the layout-independent VK key events cannot express text an input method
// *commits* (autocorrect, gesture typing, non-Latin scripts, emoji), so a capable client
// sends the committed characters verbatim and the host injects them directly (Windows
// `KEYEVENTF_UNICODE`; Linux wlroots via a dynamically-grown Unicode keymap on a dedicated
// virtual keyboard). A multi-character commit is consecutive events in order. Sent only when
// the host advertised [`HOST_CAP_TEXT_INPUT`](crate::quic::HOST_CAP_TEXT_INPUT) — toward an
// older host (or one whose inject backend can't type text) clients keep the best-effort VK
// synthesis, and an older host ignores the unknown tag entirely.
PUNKTFUNK_INPUT_KIND_TEXT_INPUT = 15,
}; };
#ifndef __cplusplus #ifndef __cplusplus
#if __STDC_VERSION__ >= 202311L #if __STDC_VERSION__ >= 202311L
+17
View File
@@ -0,0 +1,17 @@
# punktfunk — SteamOS atomic-update preserve list (drop-in for /etc/atomic-update.conf.d/).
#
# SteamOS's A/B updates rebuild /etc from scratch and carry over ONLY the files matched by
# /usr/lib/rauc/atomic-update-keep.conf plus these operator drop-ins (the drop-in dir itself is
# on Valve's keep list, so this file is self-preserving). Without it, every OS update silently
# strips the host's system integration: /dev/uhid access (virtual pads degrade to Xbox 360),
# the vhci-hcd autoload (no native Steam Deck pad), and the UDP buffer sizing (stream stutter).
# Same wildcard rules as the stock list: '*' matches within a path segment, '**' across.
## Virtual-gamepad device access (uinput/uhid/vhci for the input group)
/etc/udev/rules.d/60-punktfunk.rules
## vhci-hcd autoload (usbip transport for the native Steam Deck pad)
/etc/modules-load.d/punktfunk.conf
## UDP socket buffer sizing (32 MB, matches the deb's sysctl drop-in)
/etc/sysctl.d/99-punktfunk-net.conf
+28 -7
View File
@@ -22,6 +22,15 @@ is only the build environment; `punktfunk-host` is launched directly, not via `d
rebuild always matches the running OS. Encode is **VAAPI** on the Deck's AMD GPU (NVENC on NVIDIA), rebuild always matches the running OS. Encode is **VAAPI** on the Deck's AMD GPU (NVENC on NVIDIA),
auto-selected by `PUNKTFUNK_ENCODER=auto`. auto-selected by `PUNKTFUNK_ENCODER=auto`.
The honest trade-off: on-device building costs a slow first install (~1015 min, ~1 GB of image +
toolchain) and adds moving parts (apt mirrors, rustup, bun) — the price of an install that can
always chase the OS. Both failure modes of that chase are automated away now:
`punktfunk-rebuild-check` rebuilds when an OS update breaks the binary's links, and the
atomic-update keep list preserves the `/etc` tuning. The eventual lighter-weight alternative is a
CI-prebuilt bundle with the volatile libraries (FFmpeg et al.) vendored under an `$ORIGIN` rpath —
OS-update-proof without a toolchain on the device — worth it once SteamOS host volume justifies
per-release artifact signing/hosting; the from-source path would stay as the dev/fallback route.
The web console is the one part that stays in the container at runtime: it's a Nitro **`bun`** The web console is the one part that stays in the container at runtime: it's a Nitro **`bun`**
build (`bun` both builds **and runs** it — the bun-preset output uses `Bun.serve` with TLS, build (`bun` both builds **and runs** it — the bun-preset output uses `Bun.serve` with TLS,
serving HTTPS (HTTP/1.1 over TLS) with the host's identity cert), so its service does serving HTTPS (HTTP/1.1 over TLS) with the host's identity cert), so its service does
@@ -31,8 +40,9 @@ serving HTTPS (HTTP/1.1 over TLS) with the host's identity cert), so its service
| Script | What it does | | Script | What it does |
|--------|--------------| |--------|--------------|
| `install.sh` | Idempotent installer: ensure the `pf2` distrobox + toolchain → build host (+web) → write config → tune sysctl + `input` group (sudo) → install + start `punktfunk-host` / `punktfunk-web` systemd **user** services with linger. | | `install.sh` | Idempotent installer: ensure the `pf2` distrobox + toolchain → build host + web + **plugin runner** → write config → tune sysctl + udev + `vhci-hcd` + `input` group and **register it on SteamOS's atomic-update keep list** (sudo) → install + start `punktfunk-host` / `punktfunk-web` systemd **user** services with linger, plus the **rebuild check** below. |
| `update.sh` | Rebuild from the current source and restart the services (config + pairings persist). `--pull` does `git pull` first. | | `update.sh` | Rebuild everything from the current source and restart the services (config + pairings persist). `--pull` does `git pull` first. Also retrofits anything a newer install.sh writes (runner, keep-list registration, rebuild check) onto older installs. |
| `rebuild-check.sh` | The post-OS-update self-heal (run by `punktfunk-rebuild-check.service` before the host at session start): `ldd`-probes the binary — milliseconds when healthy, a full `update.sh` rebuild only when a SteamOS update actually broke its library links. |
```sh ```sh
git clone https://git.unom.io/unom/punktfunk ~/punktfunk git clone https://git.unom.io/unom/punktfunk ~/punktfunk
@@ -57,10 +67,19 @@ default `pf2`), `PUNKTFUNK_MGMT_PORT` (47990), `PUNKTFUNK_WEB_PORT` (47992).
here too and persists across updates. here too and persists across updates.
- **Services:** `~/.config/systemd/user/punktfunk-host.service` (runs `serve --gamestream --mgmt-bind - **Services:** `~/.config/systemd/user/punktfunk-host.service` (runs `serve --gamestream --mgmt-bind
0.0.0.0:47990`, `+ --open` if chosen — `--gamestream` adds the Moonlight-compat planes so the Deck's 0.0.0.0:47990`, `+ --open` if chosen — `--gamestream` adds the Moonlight-compat planes so the Deck's
Game Mode also streams to stock Moonlight; the native `punktfunk/1` plane is always on) and Game Mode also streams to stock Moonlight; the native `punktfunk/1` plane is always on),
`punktfunk-web.service`. Linger is enabled so they run without a login session. `punktfunk-web.service`, `punktfunk-rebuild-check.service` (post-OS-update self-heal, enabled), and
`punktfunk-scripting.service` (plugin runner, **opt-in** — enable it once you use plugins/scripts).
Linger is enabled so they run without a login session.
- **Plugin runner:** the deb's payload laid out user-scoped (read-only `/usr` can't take the
package): wrapper `~/.local/bin/punktfunk-scripting`, pinned `bun` in
`~/.local/lib/punktfunk-scripting/`, bundle in `~/.local/share/punktfunk-scripting/`.
- **System tuning (sudo):** `/etc/sysctl.d/99-punktfunk-net.conf` (32 MB UDP buffers — the #1 - **System tuning (sudo):** `/etc/sysctl.d/99-punktfunk-net.conf` (32 MB UDP buffers — the #1
high-bitrate lever), `/etc/udev/rules.d/60-punktfunk.rules`, and `$USER` in the `input` group. high-bitrate lever), `/etc/udev/rules.d/60-punktfunk.rules` (`uinput`/`uhid` access),
`/etc/modules-load.d/punktfunk.conf` (`vhci-hcd` for the native Deck pad), `$USER` in the `input`
group — and `/etc/atomic-update.conf.d/punktfunk.conf`, which registers the three files on
SteamOS's atomic-update keep list so A/B OS updates carry them over (verified: without it an
update silently strips them — pads degrade to Xbox 360, buffers drop to 208 KB).
## Operating ## Operating
@@ -83,5 +102,7 @@ host advertises over mDNS as `_punktfunk._udp`, so clients discover it automatic
for a headless host. for a headless host.
- **WiFi tx ceiling** ≈ 250 Mbps goodput (a Deck hardware/driver packet-rate limit, band-independent); - **WiFi tx ceiling** ≈ 250 Mbps goodput (a Deck hardware/driver packet-rate limit, band-independent);
fine for 1080p/1440p60. A wired dock lifts it. fine for 1080p/1440p60. A wired dock lifts it.
- **After a major SteamOS update**, if the host won't start, run `update.sh` to rebuild against the new - **After a SteamOS update** nothing should be needed: the `/etc` tuning survives via the
base libraries. atomic-update keep list, and `punktfunk-rebuild-check` rebuilds the binary automatically if the
new base actually broke its library links (first session start after the update takes the build's
few minutes in that case). A manual `update.sh` remains harmless.
+67 -1
View File
@@ -153,6 +153,35 @@ cd '$SRC/web' && bun install --frozen-lockfile && bun run build
ok "web console built" ok "web console built"
fi fi
# --- 2b. plugin runner (scripting) -----------------------------------------
# The console's plugin store and `punktfunk-host plugins …` shell out to the scripting runner —
# the SDK's runner CLI bundled to one self-contained JS, run on a pinned bun (it import()s the
# operator's .ts plugins; see packaging/debian/build-scripting-deb.sh). The .deb lays it out under
# /usr, which is read-only here, so ship the SAME payload user-scoped — wrapper + private bun +
# bundle under ~/.local, where the host's runner discovery looks after the /usr layouts.
log "Building the plugin runner (scripting)"
mkdir -p "$HOME/.local/bin" "$HOME/.local/lib/punktfunk-scripting" "$HOME/.local/share/punktfunk-scripting"
distrobox enter "$BOX" -- bash -lc "
set -e
export PATH=\$HOME/.bun/bin:\$PATH
cd '$SRC/sdk'
bun install --frozen-lockfile --ignore-scripts
bun build src/runner-cli.ts --target=bun --outfile \"\$HOME/.local/share/punktfunk-scripting/runner-cli.js\"
# Pin the runtime: copy the box's bun next to the bundle so a bun self-update (or a box rebuild)
# never changes what the runner executes under.
install -m0755 \"\$(command -v bun)\" \"\$HOME/.local/lib/punktfunk-scripting/bun\"
"
grep -q 'attempt=' "$HOME/.local/share/punktfunk-scripting/runner-cli.js" \
|| die "runner bundle missing the dynamic plugin import — wrong build"
cat > "$HOME/.local/bin/punktfunk-scripting" <<'WRAP'
#!/bin/sh
# Generated by scripts/steamdeck/install.sh — user-scoped punktfunk-scripting (the .deb's /usr/bin
# wrapper, relocated): the runner bundle on its private pinned bun.
exec "$HOME/.local/lib/punktfunk-scripting/bun" "$HOME/.local/share/punktfunk-scripting/runner-cli.js" "$@"
WRAP
chmod 0755 "$HOME/.local/bin/punktfunk-scripting"
ok "plugin runner: ~/.local/bin/punktfunk-scripting"
# --- 3. config ------------------------------------------------------------- # --- 3. config -------------------------------------------------------------
log "Configuration ($CONFIG)" log "Configuration ($CONFIG)"
mkdir -p "$CONFIG" mkdir -p "$CONFIG"
@@ -245,6 +274,14 @@ if [ "$SUDO_OK" = 1 ]; then
NEED_RELOGIN=1 NEED_RELOGIN=1
warn "added $USER to the 'input' group (applies on next login)" warn "added $USER to the 'input' group (applies on next login)"
fi fi
# SteamOS A/B updates rebuild /etc and DROP everything not on Valve's keep list — verified
# live: an OS update stripped the udev rule + vhci autoload + UDP sysctl (gamepads silently
# degrade to Xbox 360, buffers back to 208 KB). The sanctioned fix is a preserve drop-in in
# /etc/atomic-update.conf.d/ (itself on the stock keep list, so it self-preserves).
if [ -f "$SRC/scripts/punktfunk-atomic-keep.conf" ]; then
sudo install -Dm644 "$SRC/scripts/punktfunk-atomic-keep.conf" /etc/atomic-update.conf.d/punktfunk.conf
ok "system tuning registered to survive SteamOS updates (atomic-update.conf.d)"
fi
else else
warn "no usable sudo — SKIPPED system tuning. Gamepad passthrough + clean streaming need root (udev" warn "no usable sudo — SKIPPED system tuning. Gamepad passthrough + clean streaming need root (udev"
warn "rule, 'input' group, vhci-hcd, UDP buffers) — there is no user-space way to do these." warn "rule, 'input' group, vhci-hcd, UDP buffers) — there is no user-space way to do these."
@@ -306,6 +343,35 @@ EOF
ok "punktfunk-web.service (port $WEB_PORT)" ok "punktfunk-web.service (port $WEB_PORT)"
fi fi
# The runner's user unit (OPT-IN, matching the .deb: installed but NOT enabled — the runner is
# inert until you add scripts/plugins). ExecStart is rewritten from the packaged /usr wrapper to
# the user-scoped one section 2b installed.
sed 's|^ExecStart=.*|ExecStart=%h/.local/bin/punktfunk-scripting|' \
"$SRC/scripts/punktfunk-scripting.service" > "$UNITS/punktfunk-scripting.service"
ok "punktfunk-scripting.service (opt-in: systemctl --user enable --now punktfunk-scripting)"
# Post-OS-update self-heal: SteamOS A/B updates can bump library sonames the host binary links
# (FFmpeg/PipeWire/libva) — this oneshot probes the binary with ldd before punktfunk-host starts
# and re-runs update.sh only when it actually stopped loading. Milliseconds on a normal boot.
cat > "$UNITS/punktfunk-rebuild-check.service" <<EOF
# Generated by scripts/steamdeck/install.sh — rebuild the host if a SteamOS update broke its libs.
[Unit]
Description=punktfunk SteamOS post-update rebuild check
Before=punktfunk-host.service
[Service]
Type=oneshot
ExecStart=$SRC/scripts/steamdeck/rebuild-check.sh
# A cold-ish rebuild is minutes, not seconds.
TimeoutStartSec=1800
[Install]
WantedBy=default.target
EOF
chmod +x "$SRC/scripts/steamdeck/rebuild-check.sh" 2>/dev/null || true
systemctl --user enable punktfunk-rebuild-check.service 2>/dev/null || true
ok "punktfunk-rebuild-check.service (auto-rebuild after SteamOS updates)"
systemctl --user daemon-reload systemctl --user daemon-reload
loginctl show-user "$USER" 2>/dev/null | grep -q 'Linger=yes' || { sudo loginctl enable-linger "$USER" 2>/dev/null && ok "enabled linger (services run without login)" || warn "could not enable linger — services stop when you log out (sudo loginctl enable-linger $USER)"; } loginctl show-user "$USER" 2>/dev/null | grep -q 'Linger=yes' || { sudo loginctl enable-linger "$USER" 2>/dev/null && ok "enabled linger (services run without login)" || warn "could not enable linger — services stop when you log out (sudo loginctl enable-linger $USER)"; }
# enable + restart (not `enable --now`): restart picks up unit-file changes on a re-run, where # enable + restart (not `enable --now`): restart picks up unit-file changes on a re-run, where
@@ -327,7 +393,7 @@ echo
log "Done — punktfunk host is running on this Steam Deck" log "Done — punktfunk host is running on this Steam Deck"
echo " • Host status: systemctl --user status punktfunk-host" echo " • Host status: systemctl --user status punktfunk-host"
if [ "$WITH_WEB" = 1 ]; then if [ "$WITH_WEB" = 1 ]; then
echo " • Web console: http://${IP:-steamdeck.local}:$WEB_PORT (login: see $CONFIG/web.env)" echo " • Web console: https://${IP:-steamdeck.local}:$WEB_PORT (login: see $CONFIG/web.env)"
echo " • Pair a device: open the web console → Devices → arm pairing → enter the PIN on the client" echo " • Pair a device: open the web console → Devices → arm pairing → enter the PIN on the client"
fi fi
if [ "$OPEN" = 1 ]; then if [ "$OPEN" = 1 ]; then
+31
View File
@@ -0,0 +1,31 @@
#!/usr/bin/env bash
# punktfunk — SteamOS post-OS-update self-heal (runs before punktfunk-host at session start).
#
# The host binary links SteamOS system libraries (FFmpeg, PipeWire, libva, …). A SteamOS A/B
# update that bumps a soname leaves the binary unable to load — a silently dead host until
# someone remembers to re-run update.sh. This probe is the reliability backstop:
# * healthy binary → exit in milliseconds (every normal boot);
# * loader breakage → run scripts/steamdeck/update.sh (rebuild host + web + runner against
# the new library tree, restart services). The build container, source, and cargo caches
# all live under /home, which SteamOS updates never touch — so the rebuild is warm.
#
# Root is NOT needed: the /etc system tuning survives updates via the atomic-update keep list
# (see punktfunk-atomic-keep.conf); only the binary has to chase the OS libraries.
set -euo pipefail
# systemd user units get a minimal PATH — distrobox commonly lives in ~/.local/bin.
export PATH="$HOME/.local/bin:$PATH"
SRC="${PUNKTFUNK_SRC:-$HOME/punktfunk}"
BIN="$SRC/target-steamos/release/punktfunk-host"
if [ ! -x "$BIN" ]; then
echo "punktfunk-host binary missing at $BIN — running a full rebuild" >&2
elif ! ldd "$BIN" 2>/dev/null | grep -q "not found"; then
exit 0 # every library resolves — nothing to do
else
echo "punktfunk-host no longer loads after a SteamOS update — its missing libraries:" >&2
ldd "$BIN" 2>/dev/null | grep "not found" >&2 || true
echo "rebuilding against the new OS tree (this takes a few minutes; streaming resumes after)" >&2
fi
exec bash "$SRC/scripts/steamdeck/update.sh"
+59 -3
View File
@@ -8,6 +8,9 @@
set -euo pipefail set -euo pipefail
log() { printf '\033[1;36m==>\033[0m %s\n' "$*"; } log() { printf '\033[1;36m==>\033[0m %s\n' "$*"; }
ok() { printf '\033[1;32m ok\033[0m %s\n' "$*"; } ok() { printf '\033[1;32m ok\033[0m %s\n' "$*"; }
# warn was USED below but never defined — under `set -e` the first warn call ("command not
# found") aborted the whole update before the service restarts.
warn() { printf '\033[1;33m !!\033[0m %s\n' "$*" >&2; }
die() { printf '\033[1;31merror:\033[0m %s\n' "$*" >&2; exit 1; } die() { printf '\033[1;31merror:\033[0m %s\n' "$*" >&2; exit 1; }
SRC="${PUNKTFUNK_SRC:-$HOME/punktfunk}" SRC="${PUNKTFUNK_SRC:-$HOME/punktfunk}"
@@ -30,6 +33,48 @@ if [ "$WEB" = 1 ]; then
ok "web rebuilt" ok "web rebuilt"
fi fi
# Plugin runner (scripting): rebuild the user-scoped runner payload (install.sh §2b) — also
# RETROFITS it onto older installs that predate it (the "plugin runner isn't installed" console
# state on SteamOS).
log "Rebuilding plugin runner (scripting)"
mkdir -p "$HOME/.local/bin" "$HOME/.local/lib/punktfunk-scripting" "$HOME/.local/share/punktfunk-scripting"
distrobox enter "$BOX" -- bash -lc "set -e; export PATH=\$HOME/.bun/bin:\$PATH; cd '$SRC/sdk' && bun install --frozen-lockfile --ignore-scripts && bun build src/runner-cli.ts --target=bun --outfile \"\$HOME/.local/share/punktfunk-scripting/runner-cli.js\" && install -m0755 \"\$(command -v bun)\" \"\$HOME/.local/lib/punktfunk-scripting/bun\""
grep -q 'attempt=' "$HOME/.local/share/punktfunk-scripting/runner-cli.js" \
|| die "runner bundle missing the dynamic plugin import — wrong build"
cat > "$HOME/.local/bin/punktfunk-scripting" <<'WRAP'
#!/bin/sh
# Generated by scripts/steamdeck/update.sh — user-scoped punktfunk-scripting (see install.sh §2b).
exec "$HOME/.local/lib/punktfunk-scripting/bun" "$HOME/.local/share/punktfunk-scripting/runner-cli.js" "$@"
WRAP
chmod 0755 "$HOME/.local/bin/punktfunk-scripting"
sed 's|^ExecStart=.*|ExecStart=%h/.local/bin/punktfunk-scripting|' \
"$SRC/scripts/punktfunk-scripting.service" > "$HOME/.config/systemd/user/punktfunk-scripting.service"
systemctl --user daemon-reload
ok "plugin runner rebuilt (opt-in service: systemctl --user enable --now punktfunk-scripting)"
# Retrofit the post-OS-update rebuild check (install.sh §5) onto older installs: probes the host
# binary with ldd at session start and re-runs this script when a SteamOS update broke its links.
if [ ! -f "$HOME/.config/systemd/user/punktfunk-rebuild-check.service" ]; then
cat > "$HOME/.config/systemd/user/punktfunk-rebuild-check.service" <<EOF
# Generated by scripts/steamdeck/update.sh — rebuild the host if a SteamOS update broke its libs.
[Unit]
Description=punktfunk SteamOS post-update rebuild check
Before=punktfunk-host.service
[Service]
Type=oneshot
ExecStart=$SRC/scripts/steamdeck/rebuild-check.sh
TimeoutStartSec=1800
[Install]
WantedBy=default.target
EOF
chmod +x "$SRC/scripts/steamdeck/rebuild-check.sh" 2>/dev/null || true
systemctl --user daemon-reload
systemctl --user enable punktfunk-rebuild-check.service 2>/dev/null || true
ok "punktfunk-rebuild-check.service installed (auto-rebuild after SteamOS updates)"
fi
# Retrofit config that install.sh now writes but older installs predate (both idempotent): # Retrofit config that install.sh now writes but older installs predate (both idempotent):
# RADV_PERFTEST — Van Gogh RADV still gates VK_KHR_video_encode_* behind it; without it the # RADV_PERFTEST — Van Gogh RADV still gates VK_KHR_video_encode_* behind it; without it the
# Vulkan backend can't open and sessions silently fall back to libav VAAPI. The KWin .desktop — # Vulkan backend can't open and sessions silently fall back to libav VAAPI. The KWin .desktop —
@@ -77,6 +122,13 @@ if [ "$SUDO_OK" = 1 ]; then
sudo usermod -aG input "$USER" sudo usermod -aG input "$USER"
warn "added $USER to the 'input' group — REBOOT (or log out/in) for it to apply" warn "added $USER to the 'input' group — REBOOT (or log out/in) for it to apply"
fi fi
# Register the tuning on Valve's atomic-update preserve list (see install.sh §4): without
# this, every SteamOS A/B update strips the three files above again (verified live —
# gamepads silently degrade to Xbox 360, UDP buffers back to 208 KB).
if [ -f "$SRC/scripts/punktfunk-atomic-keep.conf" ]; then
sudo install -Dm644 "$SRC/scripts/punktfunk-atomic-keep.conf" /etc/atomic-update.conf.d/punktfunk.conf
ok "system tuning registered to survive SteamOS updates (atomic-update.conf.d)"
fi
else else
warn "no usable sudo — SKIPPED gamepad/udev/vhci/UDP tuning (all root-only; no user-space alternative)." warn "no usable sudo — SKIPPED gamepad/udev/vhci/UDP tuning (all root-only; no user-space alternative)."
warn "A stock SteamOS 'deck' account has NO password — set one with 'passwd', then re-run. Gamepads stay" warn "A stock SteamOS 'deck' account has NO password — set one with 'passwd', then re-run. Gamepads stay"
@@ -94,8 +146,12 @@ if [ ! -s "$GRANT_DST" ] && [ -s "$GRANT_SRC" ]; then
fi fi
log "Restarting services" log "Restarting services"
systemctl --user restart punktfunk-host.service # --no-block: when this script runs INSIDE punktfunk-rebuild-check.service (ordered
ok "punktfunk-host restarted" # Before=punktfunk-host), a blocking restart would deadlock — the restart job waits for the
if [ "$WEB" = 1 ]; then systemctl --user restart punktfunk-web.service; ok "punktfunk-web restarted"; fi # check unit, which waits for this script, which waits for the restart. Enqueue and move on;
# systemd starts the service the moment the ordering allows.
systemctl --user restart --no-block punktfunk-host.service
ok "punktfunk-host restart queued"
if [ "$WEB" = 1 ]; then systemctl --user restart --no-block punktfunk-web.service; ok "punktfunk-web restart queued"; fi
echo echo
log "Updated. Status: systemctl --user status punktfunk-host" log "Updated. Status: systemctl --user status punktfunk-host"