perf(host/windows): SendInput retry-on-failure model (two-process step 2)
apple / swift (push) Successful in 54s
android / android (push) Failing after 0s
ci / rust (push) Failing after 0s
ci / docs-site (push) Failing after 0s
ci / bench (push) Failing after 0s
deb / build-publish (push) Failing after 0s
ci / web (push) Failing after 1s
decky / build-publish (push) Failing after 0s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Failing after 1s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Failing after 0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Failing after 1s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Failing after 0s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 1s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 0s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Failing after 0s
docker / deploy-docs (push) Has been skipped
apple / swift (push) Successful in 54s
android / android (push) Failing after 0s
ci / rust (push) Failing after 0s
ci / docs-site (push) Failing after 0s
ci / bench (push) Failing after 0s
deb / build-publish (push) Failing after 0s
ci / web (push) Failing after 1s
decky / build-publish (push) Failing after 0s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Failing after 1s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Failing after 0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Failing after 1s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Failing after 0s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 1s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 0s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Failing after 0s
docker / deploy-docs (push) Has been skipped
The injector reattached the input desktop (OpenInputDesktop + SetThreadDesktop, two syscalls) before EVERY event. Now it stays bound to its desktop and only reattaches on a SendInput short write (the input desktop switched into UAC/lock) + retries once — Sunshine's model. No steady-state per-event overhead; still follows the desktop across the secure boundary, serving both desktops. Validated on the RTX 4090 (host as SYSTEM): client-rs --input-test injected for ~6s with no "blocked desktop" errors. Completes all 6 steps of the two-process secure-desktop build; only a real-UAC user smoke test remains. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,8 +1,9 @@
|
||||
//! Windows input injection via `SendInput` (Win32 KeyboardAndMouse) — the Windows analogue of
|
||||
//! [`super::wlr`]: absolute mouse normalized to the virtual desktop, relative mouse for games,
|
||||
//! scancode keyboard, scroll, buttons. The client already sends Windows VK codes, so there is no
|
||||
//! keycode table. Survives UAC/lock desktop switches by reattaching the thread to the current
|
||||
//! input desktop before each event (`OpenInputDesktop`/`SetThreadDesktop`).
|
||||
//! keycode table. Survives UAC/lock desktop switches with Sunshine's retry-on-failure model: the
|
||||
//! thread stays bound to its desktop and only reattaches (`OpenInputDesktop`/`SetThreadDesktop`) when
|
||||
//! `SendInput` reports a short write (the input desktop switched) — no per-event reattach overhead.
|
||||
|
||||
use anyhow::Result;
|
||||
use punktfunk_core::input::{InputEvent, InputKind};
|
||||
@@ -68,11 +69,20 @@ impl SendInputInjector {
|
||||
}
|
||||
}
|
||||
|
||||
fn send(inputs: &[INPUT]) -> Result<()> {
|
||||
/// Inject with Sunshine's retry-on-failure model: the thread stays bound to whatever desktop it
|
||||
/// last attached to (no per-event `OpenInputDesktop`/`SetThreadDesktop` — two syscalls saved on
|
||||
/// every mouse move), and only when `SendInput` reports a short write (0 = the input desktop
|
||||
/// switched out from under us, e.g. into UAC/lock) do we reattach to the now-current input desktop
|
||||
/// and retry once. This serves both the normal and secure desktops with no steady-state overhead.
|
||||
fn send(&mut self, inputs: &[INPUT]) -> Result<()> {
|
||||
let n = unsafe { SendInput(inputs, size_of::<INPUT>() as i32) };
|
||||
if n as usize == inputs.len() {
|
||||
return Ok(());
|
||||
}
|
||||
// Short write → the input desktop likely changed. Reattach + retry once.
|
||||
self.reattach_input_desktop();
|
||||
let n = unsafe { SendInput(inputs, size_of::<INPUT>() as i32) };
|
||||
if n as usize != inputs.len() {
|
||||
// 0 = blocked (different/secure desktop). Surface as Err so the host service drops +
|
||||
// reopens the injector (which reattaches the input desktop).
|
||||
anyhow::bail!(
|
||||
"SendInput injected {n}/{} events (blocked desktop?)",
|
||||
inputs.len()
|
||||
@@ -94,7 +104,8 @@ impl Drop for SendInputInjector {
|
||||
|
||||
impl InputInjector for SendInputInjector {
|
||||
fn inject(&mut self, event: &InputEvent) -> Result<()> {
|
||||
self.reattach_input_desktop();
|
||||
// No per-event desktop reattach — `send` reattaches lazily only on a short write (desktop
|
||||
// switch). The injector is bound to the input desktop at open() and follows switches on demand.
|
||||
match event.kind {
|
||||
InputKind::MouseMove => {
|
||||
let mi = MOUSEINPUT {
|
||||
@@ -105,7 +116,7 @@ impl InputInjector for SendInputInjector {
|
||||
time: 0,
|
||||
dwExtraInfo: 0,
|
||||
};
|
||||
Self::send(&[mouse(mi)])
|
||||
self.send(&[mouse(mi)])
|
||||
}
|
||||
InputKind::MouseMoveAbs => {
|
||||
let w = (event.flags >> 16) & 0xffff;
|
||||
@@ -128,7 +139,7 @@ impl InputInjector for SendInputInjector {
|
||||
time: 0,
|
||||
dwExtraInfo: 0,
|
||||
};
|
||||
Self::send(&[mouse(mi)])
|
||||
self.send(&[mouse(mi)])
|
||||
}
|
||||
InputKind::MouseButtonDown | InputKind::MouseButtonUp => {
|
||||
let down = event.kind == InputKind::MouseButtonDown;
|
||||
@@ -183,7 +194,7 @@ impl InputInjector for SendInputInjector {
|
||||
time: 0,
|
||||
dwExtraInfo: 0,
|
||||
};
|
||||
Self::send(&[mouse(mi)])
|
||||
self.send(&[mouse(mi)])
|
||||
}
|
||||
InputKind::MouseScroll => {
|
||||
// GameStream WHEEL_DELTA(120) units. Windows WHEEL positive=up (matches GameStream —
|
||||
@@ -201,7 +212,7 @@ impl InputInjector for SendInputInjector {
|
||||
time: 0,
|
||||
dwExtraInfo: 0,
|
||||
};
|
||||
Self::send(&[mouse(mi)])
|
||||
self.send(&[mouse(mi)])
|
||||
}
|
||||
InputKind::KeyDown | InputKind::KeyUp => {
|
||||
let down = event.kind == InputKind::KeyDown;
|
||||
@@ -226,7 +237,7 @@ impl InputInjector for SendInputInjector {
|
||||
time: 0,
|
||||
dwExtraInfo: 0,
|
||||
};
|
||||
Self::send(&[key(ki)])
|
||||
self.send(&[key(ki)])
|
||||
}
|
||||
// Gamepad goes through ViGEm (separate backend). Touch: no SendInput equivalent -> no-op.
|
||||
InputKind::GamepadButton
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
# Windows secure-desktop capture — two-process design
|
||||
|
||||
Status: **steps 1, 3, 4, 5, 6 implemented and live-validated on the RTX 4090 (2026-06-16).** The
|
||||
Status: **all steps (1–6) implemented and live-validated on the RTX 4090 (2026-06-16).** The
|
||||
two-process path works end to end (host as SYSTEM): the user-session WGC helper relays video, the mux
|
||||
switches to the host's DDA on the secure desktop, and a dead helper is rebuilt automatically. Only the
|
||||
SendInput retry-refactor (step 2) and a *real* UAC/lock smoke test remain. The earlier user-mode WGC
|
||||
animation fix still ships; this is the SYSTEM-mode design that adds secure-desktop (UAC/lock/login)
|
||||
coverage, since WGC and the secure desktop need conflicting process tokens.
|
||||
switches to the host's DDA on the secure desktop, a dead helper is rebuilt automatically, and the
|
||||
SendInput injector follows desktop switches lazily. Only a *real* UAC/lock smoke test remains (can't
|
||||
be triggered headless over SSH). The earlier user-mode WGC animation fix still ships; this is the
|
||||
SYSTEM-mode design that adds secure-desktop (UAC/lock/login) coverage, since WGC and the secure desktop
|
||||
need conflicting process tokens.
|
||||
|
||||
Implemented so far:
|
||||
- **Step 1 — DesktopWatcher** (`capture/desktop_watch.rs`): polls the input-desktop name → atomic
|
||||
@@ -36,11 +37,16 @@ Implemented so far:
|
||||
/ 0 early-ends / 465 frames decoded. (Recovery rebuilds the whole output, not a same-target respawn,
|
||||
which storm-failed with "no DXGI output for target N yet" after an abrupt kill.)
|
||||
|
||||
Remaining: **step 2** (SendInput retry-on-failure refactor — input works today via the existing path;
|
||||
this hardens it across the desktop boundary) and a **final user-driven smoke test**: trigger a *real*
|
||||
UAC/lock on the box during a session and confirm the dialog appears on the client (the box's UAC
|
||||
- Step 2: SendInput now uses the retry-on-failure model (`inject/sendinput.rs`) — the thread stays
|
||||
bound to its desktop and only reattaches (`OpenInputDesktop`/`SetThreadDesktop`) on a `SendInput`
|
||||
short write (desktop switched), instead of two syscalls per event. Validated: `client-rs --input-test`
|
||||
injected for ~6s with no `blocked desktop` errors (steady-state path); the reattach-on-switch path
|
||||
is the same `OpenInputDesktop` call the old per-event code used, now lazy.
|
||||
|
||||
Remaining: a **final user-driven smoke test** — trigger a *real* UAC/lock on the box during a session
|
||||
and confirm the dialog appears on the client AND that clicking/typing on it lands (the box's UAC
|
||||
auto-elevates admins, so a real prompt can't be triggered headless over SSH; the mux switch itself is
|
||||
proven by the timed toggle, and DDA-on-Winlogon capture by the single-process secure path).
|
||||
proven by the timed toggle, and DDA-on-Winlogon capture + input by the single-process secure path).
|
||||
|
||||
> **Note:** the two-process path requires the host to run as SYSTEM (`run.cmd.sysbak` → `-s -i 1`).
|
||||
> As SYSTEM, WASAPI loopback audio (session 0) does not capture the user session's audio — a known
|
||||
|
||||
Reference in New Issue
Block a user