perf(host/windows): SendInput retry-on-failure model (two-process step 2)
apple / swift (push) Successful in 54s
android / android (push) Failing after 0s
ci / rust (push) Failing after 0s
ci / docs-site (push) Failing after 0s
ci / bench (push) Failing after 0s
deb / build-publish (push) Failing after 0s
ci / web (push) Failing after 1s
decky / build-publish (push) Failing after 0s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Failing after 1s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Failing after 0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Failing after 1s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Failing after 0s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 1s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 0s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Failing after 0s
docker / deploy-docs (push) Has been skipped
apple / swift (push) Successful in 54s
android / android (push) Failing after 0s
ci / rust (push) Failing after 0s
ci / docs-site (push) Failing after 0s
ci / bench (push) Failing after 0s
deb / build-publish (push) Failing after 0s
ci / web (push) Failing after 1s
decky / build-publish (push) Failing after 0s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Failing after 1s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Failing after 0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Failing after 1s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Failing after 0s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 1s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 0s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Failing after 0s
docker / deploy-docs (push) Has been skipped
The injector reattached the input desktop (OpenInputDesktop + SetThreadDesktop, two syscalls) before EVERY event. Now it stays bound to its desktop and only reattaches on a SendInput short write (the input desktop switched into UAC/lock) + retries once — Sunshine's model. No steady-state per-event overhead; still follows the desktop across the secure boundary, serving both desktops. Validated on the RTX 4090 (host as SYSTEM): client-rs --input-test injected for ~6s with no "blocked desktop" errors. Completes all 6 steps of the two-process secure-desktop build; only a real-UAC user smoke test remains. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,8 +1,9 @@
|
|||||||
//! Windows input injection via `SendInput` (Win32 KeyboardAndMouse) — the Windows analogue of
|
//! Windows input injection via `SendInput` (Win32 KeyboardAndMouse) — the Windows analogue of
|
||||||
//! [`super::wlr`]: absolute mouse normalized to the virtual desktop, relative mouse for games,
|
//! [`super::wlr`]: absolute mouse normalized to the virtual desktop, relative mouse for games,
|
||||||
//! scancode keyboard, scroll, buttons. The client already sends Windows VK codes, so there is no
|
//! scancode keyboard, scroll, buttons. The client already sends Windows VK codes, so there is no
|
||||||
//! keycode table. Survives UAC/lock desktop switches by reattaching the thread to the current
|
//! keycode table. Survives UAC/lock desktop switches with Sunshine's retry-on-failure model: the
|
||||||
//! input desktop before each event (`OpenInputDesktop`/`SetThreadDesktop`).
|
//! thread stays bound to its desktop and only reattaches (`OpenInputDesktop`/`SetThreadDesktop`) when
|
||||||
|
//! `SendInput` reports a short write (the input desktop switched) — no per-event reattach overhead.
|
||||||
|
|
||||||
use anyhow::Result;
|
use anyhow::Result;
|
||||||
use punktfunk_core::input::{InputEvent, InputKind};
|
use punktfunk_core::input::{InputEvent, InputKind};
|
||||||
@@ -68,11 +69,20 @@ impl SendInputInjector {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn send(inputs: &[INPUT]) -> Result<()> {
|
/// Inject with Sunshine's retry-on-failure model: the thread stays bound to whatever desktop it
|
||||||
|
/// last attached to (no per-event `OpenInputDesktop`/`SetThreadDesktop` — two syscalls saved on
|
||||||
|
/// every mouse move), and only when `SendInput` reports a short write (0 = the input desktop
|
||||||
|
/// switched out from under us, e.g. into UAC/lock) do we reattach to the now-current input desktop
|
||||||
|
/// and retry once. This serves both the normal and secure desktops with no steady-state overhead.
|
||||||
|
fn send(&mut self, inputs: &[INPUT]) -> Result<()> {
|
||||||
|
let n = unsafe { SendInput(inputs, size_of::<INPUT>() as i32) };
|
||||||
|
if n as usize == inputs.len() {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
// Short write → the input desktop likely changed. Reattach + retry once.
|
||||||
|
self.reattach_input_desktop();
|
||||||
let n = unsafe { SendInput(inputs, size_of::<INPUT>() as i32) };
|
let n = unsafe { SendInput(inputs, size_of::<INPUT>() as i32) };
|
||||||
if n as usize != inputs.len() {
|
if n as usize != inputs.len() {
|
||||||
// 0 = blocked (different/secure desktop). Surface as Err so the host service drops +
|
|
||||||
// reopens the injector (which reattaches the input desktop).
|
|
||||||
anyhow::bail!(
|
anyhow::bail!(
|
||||||
"SendInput injected {n}/{} events (blocked desktop?)",
|
"SendInput injected {n}/{} events (blocked desktop?)",
|
||||||
inputs.len()
|
inputs.len()
|
||||||
@@ -94,7 +104,8 @@ impl Drop for SendInputInjector {
|
|||||||
|
|
||||||
impl InputInjector for SendInputInjector {
|
impl InputInjector for SendInputInjector {
|
||||||
fn inject(&mut self, event: &InputEvent) -> Result<()> {
|
fn inject(&mut self, event: &InputEvent) -> Result<()> {
|
||||||
self.reattach_input_desktop();
|
// No per-event desktop reattach — `send` reattaches lazily only on a short write (desktop
|
||||||
|
// switch). The injector is bound to the input desktop at open() and follows switches on demand.
|
||||||
match event.kind {
|
match event.kind {
|
||||||
InputKind::MouseMove => {
|
InputKind::MouseMove => {
|
||||||
let mi = MOUSEINPUT {
|
let mi = MOUSEINPUT {
|
||||||
@@ -105,7 +116,7 @@ impl InputInjector for SendInputInjector {
|
|||||||
time: 0,
|
time: 0,
|
||||||
dwExtraInfo: 0,
|
dwExtraInfo: 0,
|
||||||
};
|
};
|
||||||
Self::send(&[mouse(mi)])
|
self.send(&[mouse(mi)])
|
||||||
}
|
}
|
||||||
InputKind::MouseMoveAbs => {
|
InputKind::MouseMoveAbs => {
|
||||||
let w = (event.flags >> 16) & 0xffff;
|
let w = (event.flags >> 16) & 0xffff;
|
||||||
@@ -128,7 +139,7 @@ impl InputInjector for SendInputInjector {
|
|||||||
time: 0,
|
time: 0,
|
||||||
dwExtraInfo: 0,
|
dwExtraInfo: 0,
|
||||||
};
|
};
|
||||||
Self::send(&[mouse(mi)])
|
self.send(&[mouse(mi)])
|
||||||
}
|
}
|
||||||
InputKind::MouseButtonDown | InputKind::MouseButtonUp => {
|
InputKind::MouseButtonDown | InputKind::MouseButtonUp => {
|
||||||
let down = event.kind == InputKind::MouseButtonDown;
|
let down = event.kind == InputKind::MouseButtonDown;
|
||||||
@@ -183,7 +194,7 @@ impl InputInjector for SendInputInjector {
|
|||||||
time: 0,
|
time: 0,
|
||||||
dwExtraInfo: 0,
|
dwExtraInfo: 0,
|
||||||
};
|
};
|
||||||
Self::send(&[mouse(mi)])
|
self.send(&[mouse(mi)])
|
||||||
}
|
}
|
||||||
InputKind::MouseScroll => {
|
InputKind::MouseScroll => {
|
||||||
// GameStream WHEEL_DELTA(120) units. Windows WHEEL positive=up (matches GameStream —
|
// GameStream WHEEL_DELTA(120) units. Windows WHEEL positive=up (matches GameStream —
|
||||||
@@ -201,7 +212,7 @@ impl InputInjector for SendInputInjector {
|
|||||||
time: 0,
|
time: 0,
|
||||||
dwExtraInfo: 0,
|
dwExtraInfo: 0,
|
||||||
};
|
};
|
||||||
Self::send(&[mouse(mi)])
|
self.send(&[mouse(mi)])
|
||||||
}
|
}
|
||||||
InputKind::KeyDown | InputKind::KeyUp => {
|
InputKind::KeyDown | InputKind::KeyUp => {
|
||||||
let down = event.kind == InputKind::KeyDown;
|
let down = event.kind == InputKind::KeyDown;
|
||||||
@@ -226,7 +237,7 @@ impl InputInjector for SendInputInjector {
|
|||||||
time: 0,
|
time: 0,
|
||||||
dwExtraInfo: 0,
|
dwExtraInfo: 0,
|
||||||
};
|
};
|
||||||
Self::send(&[key(ki)])
|
self.send(&[key(ki)])
|
||||||
}
|
}
|
||||||
// Gamepad goes through ViGEm (separate backend). Touch: no SendInput equivalent -> no-op.
|
// Gamepad goes through ViGEm (separate backend). Touch: no SendInput equivalent -> no-op.
|
||||||
InputKind::GamepadButton
|
InputKind::GamepadButton
|
||||||
|
|||||||
@@ -1,11 +1,12 @@
|
|||||||
# Windows secure-desktop capture — two-process design
|
# Windows secure-desktop capture — two-process design
|
||||||
|
|
||||||
Status: **steps 1, 3, 4, 5, 6 implemented and live-validated on the RTX 4090 (2026-06-16).** The
|
Status: **all steps (1–6) implemented and live-validated on the RTX 4090 (2026-06-16).** The
|
||||||
two-process path works end to end (host as SYSTEM): the user-session WGC helper relays video, the mux
|
two-process path works end to end (host as SYSTEM): the user-session WGC helper relays video, the mux
|
||||||
switches to the host's DDA on the secure desktop, and a dead helper is rebuilt automatically. Only the
|
switches to the host's DDA on the secure desktop, a dead helper is rebuilt automatically, and the
|
||||||
SendInput retry-refactor (step 2) and a *real* UAC/lock smoke test remain. The earlier user-mode WGC
|
SendInput injector follows desktop switches lazily. Only a *real* UAC/lock smoke test remains (can't
|
||||||
animation fix still ships; this is the SYSTEM-mode design that adds secure-desktop (UAC/lock/login)
|
be triggered headless over SSH). The earlier user-mode WGC animation fix still ships; this is the
|
||||||
coverage, since WGC and the secure desktop need conflicting process tokens.
|
SYSTEM-mode design that adds secure-desktop (UAC/lock/login) coverage, since WGC and the secure desktop
|
||||||
|
need conflicting process tokens.
|
||||||
|
|
||||||
Implemented so far:
|
Implemented so far:
|
||||||
- **Step 1 — DesktopWatcher** (`capture/desktop_watch.rs`): polls the input-desktop name → atomic
|
- **Step 1 — DesktopWatcher** (`capture/desktop_watch.rs`): polls the input-desktop name → atomic
|
||||||
@@ -36,11 +37,16 @@ Implemented so far:
|
|||||||
/ 0 early-ends / 465 frames decoded. (Recovery rebuilds the whole output, not a same-target respawn,
|
/ 0 early-ends / 465 frames decoded. (Recovery rebuilds the whole output, not a same-target respawn,
|
||||||
which storm-failed with "no DXGI output for target N yet" after an abrupt kill.)
|
which storm-failed with "no DXGI output for target N yet" after an abrupt kill.)
|
||||||
|
|
||||||
Remaining: **step 2** (SendInput retry-on-failure refactor — input works today via the existing path;
|
- Step 2: SendInput now uses the retry-on-failure model (`inject/sendinput.rs`) — the thread stays
|
||||||
this hardens it across the desktop boundary) and a **final user-driven smoke test**: trigger a *real*
|
bound to its desktop and only reattaches (`OpenInputDesktop`/`SetThreadDesktop`) on a `SendInput`
|
||||||
UAC/lock on the box during a session and confirm the dialog appears on the client (the box's UAC
|
short write (desktop switched), instead of two syscalls per event. Validated: `client-rs --input-test`
|
||||||
|
injected for ~6s with no `blocked desktop` errors (steady-state path); the reattach-on-switch path
|
||||||
|
is the same `OpenInputDesktop` call the old per-event code used, now lazy.
|
||||||
|
|
||||||
|
Remaining: a **final user-driven smoke test** — trigger a *real* UAC/lock on the box during a session
|
||||||
|
and confirm the dialog appears on the client AND that clicking/typing on it lands (the box's UAC
|
||||||
auto-elevates admins, so a real prompt can't be triggered headless over SSH; the mux switch itself is
|
auto-elevates admins, so a real prompt can't be triggered headless over SSH; the mux switch itself is
|
||||||
proven by the timed toggle, and DDA-on-Winlogon capture by the single-process secure path).
|
proven by the timed toggle, and DDA-on-Winlogon capture + input by the single-process secure path).
|
||||||
|
|
||||||
> **Note:** the two-process path requires the host to run as SYSTEM (`run.cmd.sysbak` → `-s -i 1`).
|
> **Note:** the two-process path requires the host to run as SYSTEM (`run.cmd.sysbak` → `-s -i 1`).
|
||||||
> As SYSTEM, WASAPI loopback audio (session 0) does not capture the user session's audio — a known
|
> As SYSTEM, WASAPI loopback audio (session 0) does not capture the user session's audio — a known
|
||||||
|
|||||||
Reference in New Issue
Block a user