perf(host/windows): SendInput retry-on-failure model (two-process step 2)
apple / swift (push) Successful in 54s
android / android (push) Failing after 0s
ci / rust (push) Failing after 0s
ci / docs-site (push) Failing after 0s
ci / bench (push) Failing after 0s
deb / build-publish (push) Failing after 0s
ci / web (push) Failing after 1s
decky / build-publish (push) Failing after 0s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Failing after 1s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Failing after 0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Failing after 1s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Failing after 0s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 1s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 0s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Failing after 0s
docker / deploy-docs (push) Has been skipped

The injector reattached the input desktop (OpenInputDesktop + SetThreadDesktop,
two syscalls) before EVERY event. Now it stays bound to its desktop and only
reattaches on a SendInput short write (the input desktop switched into UAC/lock)
+ retries once — Sunshine's model. No steady-state per-event overhead; still
follows the desktop across the secure boundary, serving both desktops.

Validated on the RTX 4090 (host as SYSTEM): client-rs --input-test injected for
~6s with no "blocked desktop" errors. Completes all 6 steps of the two-process
secure-desktop build; only a real-UAC user smoke test remains.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-16 08:30:49 +00:00
parent 1e8f210948
commit ec2907fc32
2 changed files with 37 additions and 20 deletions
+22 -11
View File
@@ -1,8 +1,9 @@
//! Windows input injection via `SendInput` (Win32 KeyboardAndMouse) — the Windows analogue of
//! [`super::wlr`]: absolute mouse normalized to the virtual desktop, relative mouse for games,
//! scancode keyboard, scroll, buttons. The client already sends Windows VK codes, so there is no
//! keycode table. Survives UAC/lock desktop switches by reattaching the thread to the current
//! input desktop before each event (`OpenInputDesktop`/`SetThreadDesktop`).
//! keycode table. Survives UAC/lock desktop switches with Sunshine's retry-on-failure model: the
//! thread stays bound to its desktop and only reattaches (`OpenInputDesktop`/`SetThreadDesktop`) when
//! `SendInput` reports a short write (the input desktop switched) — no per-event reattach overhead.
use anyhow::Result;
use punktfunk_core::input::{InputEvent, InputKind};
@@ -68,11 +69,20 @@ impl SendInputInjector {
}
}
fn send(inputs: &[INPUT]) -> Result<()> {
/// Inject with Sunshine's retry-on-failure model: the thread stays bound to whatever desktop it
/// last attached to (no per-event `OpenInputDesktop`/`SetThreadDesktop` — two syscalls saved on
/// every mouse move), and only when `SendInput` reports a short write (0 = the input desktop
/// switched out from under us, e.g. into UAC/lock) do we reattach to the now-current input desktop
/// and retry once. This serves both the normal and secure desktops with no steady-state overhead.
fn send(&mut self, inputs: &[INPUT]) -> Result<()> {
let n = unsafe { SendInput(inputs, size_of::<INPUT>() as i32) };
if n as usize == inputs.len() {
return Ok(());
}
// Short write → the input desktop likely changed. Reattach + retry once.
self.reattach_input_desktop();
let n = unsafe { SendInput(inputs, size_of::<INPUT>() as i32) };
if n as usize != inputs.len() {
// 0 = blocked (different/secure desktop). Surface as Err so the host service drops +
// reopens the injector (which reattaches the input desktop).
anyhow::bail!(
"SendInput injected {n}/{} events (blocked desktop?)",
inputs.len()
@@ -94,7 +104,8 @@ impl Drop for SendInputInjector {
impl InputInjector for SendInputInjector {
fn inject(&mut self, event: &InputEvent) -> Result<()> {
self.reattach_input_desktop();
// No per-event desktop reattach — `send` reattaches lazily only on a short write (desktop
// switch). The injector is bound to the input desktop at open() and follows switches on demand.
match event.kind {
InputKind::MouseMove => {
let mi = MOUSEINPUT {
@@ -105,7 +116,7 @@ impl InputInjector for SendInputInjector {
time: 0,
dwExtraInfo: 0,
};
Self::send(&[mouse(mi)])
self.send(&[mouse(mi)])
}
InputKind::MouseMoveAbs => {
let w = (event.flags >> 16) & 0xffff;
@@ -128,7 +139,7 @@ impl InputInjector for SendInputInjector {
time: 0,
dwExtraInfo: 0,
};
Self::send(&[mouse(mi)])
self.send(&[mouse(mi)])
}
InputKind::MouseButtonDown | InputKind::MouseButtonUp => {
let down = event.kind == InputKind::MouseButtonDown;
@@ -183,7 +194,7 @@ impl InputInjector for SendInputInjector {
time: 0,
dwExtraInfo: 0,
};
Self::send(&[mouse(mi)])
self.send(&[mouse(mi)])
}
InputKind::MouseScroll => {
// GameStream WHEEL_DELTA(120) units. Windows WHEEL positive=up (matches GameStream —
@@ -201,7 +212,7 @@ impl InputInjector for SendInputInjector {
time: 0,
dwExtraInfo: 0,
};
Self::send(&[mouse(mi)])
self.send(&[mouse(mi)])
}
InputKind::KeyDown | InputKind::KeyUp => {
let down = event.kind == InputKind::KeyDown;
@@ -226,7 +237,7 @@ impl InputInjector for SendInputInjector {
time: 0,
dwExtraInfo: 0,
};
Self::send(&[key(ki)])
self.send(&[key(ki)])
}
// Gamepad goes through ViGEm (separate backend). Touch: no SendInput equivalent -> no-op.
InputKind::GamepadButton
+15 -9
View File
@@ -1,11 +1,12 @@
# Windows secure-desktop capture — two-process design
Status: **steps 1, 3, 4, 5, 6 implemented and live-validated on the RTX 4090 (2026-06-16).** The
Status: **all steps (16) implemented and live-validated on the RTX 4090 (2026-06-16).** The
two-process path works end to end (host as SYSTEM): the user-session WGC helper relays video, the mux
switches to the host's DDA on the secure desktop, and a dead helper is rebuilt automatically. Only the
SendInput retry-refactor (step 2) and a *real* UAC/lock smoke test remain. The earlier user-mode WGC
animation fix still ships; this is the SYSTEM-mode design that adds secure-desktop (UAC/lock/login)
coverage, since WGC and the secure desktop need conflicting process tokens.
switches to the host's DDA on the secure desktop, a dead helper is rebuilt automatically, and the
SendInput injector follows desktop switches lazily. Only a *real* UAC/lock smoke test remains (can't
be triggered headless over SSH). The earlier user-mode WGC animation fix still ships; this is the
SYSTEM-mode design that adds secure-desktop (UAC/lock/login) coverage, since WGC and the secure desktop
need conflicting process tokens.
Implemented so far:
- **Step 1 — DesktopWatcher** (`capture/desktop_watch.rs`): polls the input-desktop name → atomic
@@ -36,11 +37,16 @@ Implemented so far:
/ 0 early-ends / 465 frames decoded. (Recovery rebuilds the whole output, not a same-target respawn,
which storm-failed with "no DXGI output for target N yet" after an abrupt kill.)
Remaining: **step 2** (SendInput retry-on-failure refactor — input works today via the existing path;
this hardens it across the desktop boundary) and a **final user-driven smoke test**: trigger a *real*
UAC/lock on the box during a session and confirm the dialog appears on the client (the box's UAC
- Step 2: SendInput now uses the retry-on-failure model (`inject/sendinput.rs`) — the thread stays
bound to its desktop and only reattaches (`OpenInputDesktop`/`SetThreadDesktop`) on a `SendInput`
short write (desktop switched), instead of two syscalls per event. Validated: `client-rs --input-test`
injected for ~6s with no `blocked desktop` errors (steady-state path); the reattach-on-switch path
is the same `OpenInputDesktop` call the old per-event code used, now lazy.
Remaining: a **final user-driven smoke test** — trigger a *real* UAC/lock on the box during a session
and confirm the dialog appears on the client AND that clicking/typing on it lands (the box's UAC
auto-elevates admins, so a real prompt can't be triggered headless over SSH; the mux switch itself is
proven by the timed toggle, and DDA-on-Winlogon capture by the single-process secure path).
proven by the timed toggle, and DDA-on-Winlogon capture + input by the single-process secure path).
> **Note:** the two-process path requires the host to run as SYSTEM (`run.cmd.sysbak` → `-s -i 1`).
> As SYSTEM, WASAPI loopback audio (session 0) does not capture the user session's audio — a known