Pen, touch, and absolute mouse arrive normalized to the STREAMED display's
frame, but pointer_windows::to_screen and sendinput's MouseMoveAbs mapped
them over the whole virtual desktop — correct only when the virtual display
is the sole active display (Exclusive topology). In Extend — a physical
monitor kept on beside the virtual output, or an Exclusive isolate degraded
to the 0x57 keep-physicals fallback — the streamed output sits at a non-zero
origin, so every sample landed shifted and mis-scaled. The pen exposed it
first (field report 2026-07-24): a stylus is strictly absolute, with no
closed-loop correction onto the target like a cursor.
New pf-inject::stream_target (Windows): the host publishes the streamed
output's CCD target id at capture bring-up (one central site —
capture_virtual_output covers the native and GameStream planes); mapping
sites resolve its current desktop rect through pf-win-display's
source_desktop_rect — the same resolver the cursor-readback poller uses, so
inject and readback always agree — TTL-cached (250 ms) because a
group-layout re-arrange moves a live output's origin mid-session. No target
set / never resolved falls back to the whole virtual desktop (the historical
mapping, still right for Exclusive topology and devtest).
One change in to_screen covers pen + touch; MouseMoveAbs converts the same
desktop pixel into the 0..65535 MOUSEEVENTF_VIRTUALDESK coordinate, closing
the identical latent absolute-mouse bug (apollo-comparison open item #14/#30).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Same field report as the display-write fix, other half of the symptom: with a
UAC consent prompt up — one the user could SEE in the stream, because capture
already renders the secure desktop (326d6e17) — pen and touch did nothing,
while mouse and keyboard kept working.
The split was exactly which paths knew the input desktop can move. sendinput.rs
reattaches, so mouse/keyboard reached the prompt. pointer_windows.rs had no
desktop handling at all, so every InjectSyntheticPointerInput came back
ERROR_ACCESS_DENIED:
22:20:31 virtual pen created (PT_PEN)
22:20:31 pen inject failed error=Zugriff verweigert (0x80070005)
22:20:42 touch inject failed error=Zugriff verweigert (0x80070005) contacts=1
Measured on glass before fixing, to find out what it actually takes:
device on Default, thread on Default -> 0x80070005 (the field failure)
device on Default, thread on INPUT -> OK
device on INPUT, thread on INPUT -> OK
The middle row is load-bearing: the synthetic pointer device is NOT
desktop-affine, so rebinding the thread suffices and the device is never
destroyed and recreated across a desktop switch — which would have dropped
in-flight contacts and the pen's in-range state mid-stroke.
Injection now retries once bound to the input desktop. The binding is scoped,
not persistent like sendinput's: inject_pen/inject_touch_frame run on TWO
threads (the caller's apply_batch and the refresh threads), and the batch caller
is a shared task thread that must not be left parked on a Winlogon desktop that
disappears when the prompt is dismissed.
The first-failure WARN now carries the rejected sample (flags, pen flags/mask,
pressure, rotation, tilt, position). A 0x80070057 INVALID_PARAMETER was seen
once BEFORE any prompt existed and is still unexplained; the ranges all look
sound on inspection (coordinates clamped to the virtual screen, roll/azimuth
u16 so the modulo cannot go negative, tilt bounded 0..=90), so catching the
actual offending sample is the way to find it.
Verified on glass: with a consent prompt up, pen and touch now reach it — zero
inject failures where every prior session failed immediately. A UAC prompt can
now be dismissed from an iPad with the Pencil.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On-glass report (iPad Moonlight vs the Windows host): pen inked, native touch
did nothing. Cause: POINTER_INFO.pointerId was the client's raw wire id —
Moonlight sends arbitrary large pointerIds, and synthetic-pointer injection
rejects them (Apollo compacts ids into dense slots for exactly this reason);
the failure was logged at trace, i.e. invisibly. Wire ids now map to the
lowest free slot for the contact's lifetime, and the FIRST injection failure
of a device logs at WARN (pen too) so an inert input plane is never silent
again.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Windows leg of design/pen-tablet-input.md §6, following Apollo's recipe:
a per-session PT_PEN device (pressure rescaled 0..1024, polar tilt→tiltX/Y,
barrel roll on rotation — Windows Ink renders Pencil Pro roll natively, barrel
button, eraser via INVERTED/ERASER flags, hover) with a 40ms refresh thread
against the ~100ms synthetic-pointer staleness auto-lift, plus a PT_TOUCH
device closing the historical SendInput wire-touch no-op (full active-contact
frames, per-id DOWN/UP edges, self-healing lost-DOWN synthesis). pen_supported
now probes PT_PEN creation on Windows (1809+), which lights up HOST_CAP_PEN
and the GameStream featureFlags there — Moonlight iPad + Pencil can ink on a
Windows host. Frame grouping mirrors the Linux uinput backend.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Linux injection leg of design/pen-tablet-input.md: 0xCC/0x05 pen batches now
route through the per-session PenTracker into a lazily-created 'Punktfunk Pen'
uinput tablet (BTN_TOOL_PEN/RUBBER, pressure, tilt-from-polar, ABS_Z barrel
roll, hover distance, INPUT_PROP_DIRECT) — compositors pick it up via libinput
and hand apps zwp_tablet_v2 with full fidelity. The host now advertises
HOST_CAP_PEN when /dev/uinput is accessible (PUNKTFUNK_PEN=0 kill-switch);
transitions group into SYN frames so proximity-enter carries its position.
Stroke failsafe: clients heartbeat ≤100ms while in range (documented wire
contract — capture APIs are silent for a stationary pen); 200ms of silence
force-releases. 'punktfunk-host pen-test' draws a pressure-ramped sine stroke
through the real tracker→uinput chain, no client needed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The KWin/Phase-A/B commits were built but never clippy-checked (that distrobox
had no clippy component), so they left TEMP on-glass probes and lint debt in the
tree. With clippy now runnable (fedora rust 1.96.1 = CI parity):
- drop the `fec424ee`/`8cff30d5` TEMP probes: the `update_cursor_meta` SPA_META
diagnostic logs (also un-detaches the `// SAFETY:` comment from its `unsafe`
block → fixes `undocumented_unsafe_blocks`) and the KWin composite-arm probe in
the encode loop.
- `#[allow(clippy::too_many_arguments)]` on `spawn_pipewire` (8 params since the
KWin leg added `expect_exact_dims`; mirrors `from_virtual_output`).
clippy `-p pf-capture -p pf-vdisplay -p punktfunk-host --locked --features
nvenc,vulkan-encode -- -D warnings` is now green. (--all-targets additionally
trips a pre-existing env mismatch: the fedora libspa binding lacks
`SPA_VIDEO_TRANSFER_SMPTE2084`, referenced only by a `#[cfg(test)]` guard-test —
not a code issue; CI's pinned pipewire has it.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The libei absolute device advertises one region per logical monitor, and
both MouseMoveAbs and touch mapped into regions().first() — whichever
output the compositor announced first. Next to a physical monitor (on-glass:
GNOME with a dummy HDMI beside the virtual primary) that put the pointer and
every click on the WRONG output: the seat cursor never entered the streamed
monitor, so neither embedded nor metadata cursor capture could ever see it,
and the cursor channel had nothing to forward. Pick the region whose logical
size matches the streamed mode (the wire flags already carry it); fall back
to first() for the single-monitor case. Region geometry now rides the
device-RESUMED log line for diagnosability; matching the screencast
mapping_id instead of the size is the follow-up for same-sized-monitor
ambiguity.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The VK key-event vocabulary cannot express text an input method COMMITS
(autocorrect, gesture typing, non-Latin scripts, emoji). Add a first-class
text event and negotiate it:
- punktfunk-core: InputKind::TextInput (= 15) carries one Unicode scalar per
event in `code`; HOST_CAP_TEXT_INPUT (0x04) in Welcome::host_caps.
- Host advertises the cap only where the session's inject backend can type
text: Windows SendInput (KEYEVENTF_UNICODE, surrogate-pair aware) and the
Linux wlroots backend — a dedicated second zwp_virtual_keyboard whose xkb
keymap grows Unicode keysyms on demand (the wtype model), so keymap
re-uploads never disturb the main device's layout/modifier state. The
KWin-fake-input/libei/gamescope backends can only press layout keycodes, so
those sessions don't set the bit and clients keep the VK fallback.
- GameStream plane: Moonlight's UTF-8 text packet (MAGIC_UTF8, previously
recognized-and-dropped) now decodes to the same TextInput events.
- Android: KeyCaptureView picks a real editable InputConnection when the host
has the cap — the IME runs its full machinery, mirrored to the host live via
common-prefix diffs of the composition (backspaces + new suffix), with
setComposingRegion adopting committed text so autocorrect-revert flows diff
instead of retyping; newline→Enter, deleteSurroundingText→Backspace/Delete.
Older hosts keep the TYPE_NULL raw-key path unchanged.
- keymap: media VKs (0xB0-0xB3) → evdev so the Android media keys land on
Linux hosts too.
Verified: punktfunk-core + host gamestream + pf-inject tests green on Linux
(Ubuntu box), clippy clean; Android app+native builds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The raw-client-pixels fallback is exact only while the managed session
runs at the client's mode. When they diverge — foreign-gamescope
attach at its own resolution, supersample/under-render, transitions —
raw pixels drift or land out of range. The EIS relay file now carries
the session's current output size as a second "WxH" line (from
current_gamescope_output_size()); the injector scales normalized
client positions into it, keeping raw pixels only as the last resort.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The previous commit's no-region fallback never fired: gamescope DOES
advertise a region — (0,0,INT32_MAX,INT32_MAX), meaning "coordinates
are raw". Mapping normalized positions into it explodes a center tap
to x≈1e9, clamped by gamescope to the far corner — the cursor pinned
at (1279,799) and every degraded-touch tap landing there. Treat a
region as an output geometry only when plausibly sized (≤16384 px);
otherwise emit raw client pixels.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gamescope's "Gamescope Virtual Input" advertises pointer_abs but no
region, so every MouseMoveAbs (and the degraded-touch moves built on
it) was silently dropped (emitted=false) — clicks then landed at a
stale cursor position. With no region, the managed session runs at the
client's mode, so client pixels are output pixels: emit them raw.
Also: log region count/dims at device-resume, and add
PUNKTFUNK_INPUT_TEST_ABS=WxH to `input-test` (corners + center, 1s
apart) so the degraded-touch path is verifiable with xdotool alone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gamescope's EIS ("Gamescope Virtual Input") advertises pointer/
pointer_abs/button but never a touchscreen — so in Game Mode every
remote TouchDown/Move/Up was dropped on the floor (Desktop Mode works
via KwinFakeInput, which does touch). Headless KWin's libei has the
same gap. Instead of dropping: degrade to a single-finger absolute
pointer — down = abs-move + left press, move = abs-move, up = release —
synthesized through the normal Mouse* inject paths so region mapping,
held-state tracking, and release_all apply. First finger drives the
pointer; later fingers are ignored (a pinch degrades to a drag). A
touchscreen device appearing later takes over on the next touch —
checked per event, never latched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`ci.yml`'s clippy gate has been failing on main since the pre-0.16.0 sweep landed, and
because clippy stops at the first crate it can't compile, the visible error was only
ever the first of four. `ci.yml` is not tag-triggered, so v0.16.0 cut and shipped over
a red main; none of this reaches the release artifacts (the two E0308s are in a test
binary and a dev tool, and the rest are lints) but the gate has been blind since.
Fixed, in the order clippy surfaced them:
- clients/probe failed to COMPILE (E0308, x2). `810d918d` moved `clock_sync` onto the
resumable `io::MsgReader` but only updated the client pump, leaving the probe passing
a bare `&mut RecvStream`. The probe now wraps the control stream in a `MsgReader` at
`open_bi` and threads that everywhere — Welcome, the --remode and --bitrate watchers,
and the speed-test result read — which is also what the refactor was for: those reads
sit behind timeouts and `select!`, exactly where a straddling frame would desync the
stream for the rest of the run.
- pf-capture: `SPA_META_Cursor as u32` is a `u32 -> u32` no-op
(`clippy::unnecessary_cast`). Line 1274 already passes the same constant uncast, so
the type is not in question.
- pf-inject: `noop as usize` on the SIGUSR1 wake handler added in `986402f7` trips
`clippy::function_casts_as_integer`; goes via `*const ()` as the lint asks. Same value
in the `usize`-typed `sa_sigaction` slot.
- punktfunk-core: the `ctrl_framing` test module's `use super::*` is unused, which
`-D warnings` promotes to an error.
Verified with CI's own commands on a Linux box (this is all Linux-gated code, so a Mac
cannot check it): `cargo clippy --workspace --all-targets --locked -- -D warnings`
finishes clean, `cargo build --workspace --locked` succeeds, and
`cargo test --workspace --locked` exits 0 with no failures — including punktfunk-core's
196-test suite, which covers the control-stream framing the probe change touches.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two medium findings from the round-1 sweep, each applied to both siblings.
- create_swdevice stack-allocated the SwCreateCtx that the async PnP completion
callback writes through (result + up to 127 u16 of instance id) and then
SetEvents. The wait is bounded at 10s, so on a wedged-PnP timeout the callback
can still be PENDING: the frame is popped, the input thread reuses that stack,
and a late callback corrupts it and SetEvents an already-closed (possibly
recycled) handle. The context is now heap-allocated and reclaimed only where
the callback provably ran; on the timeout path the box is deliberately leaked
and the event left open, so a late write always targets live memory. Costs a
one-off ~264 B + one HANDLE on that rare path. Applied to the DualSense path
and its XUSB sibling in gamepad_windows.rs.
- Ds4WinPad::open swallowed a create_swdevice failure into a WARN and returned
Ok with no devnode. PadSlots::ensure then stored Some(pad) AND called
gate.on_success(), so the slot short-circuited on is_some() forever and the
capped-backoff retry that exists precisely to self-heal a transient PnP failure
never ran — the game saw no controller for the rest of the session unless the
client unplugged the pad. Now propagates, matching the XUSB sibling. Same fix
applied to steam_deck_windows.rs.
Windows .173: pf-inject 53/0. Linux .21: pf-inject 74/0 (8 ignored).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The three high-severity defects from the round-1 sweep of pf-inject /
pf-zerocopy / pf-capture (adjudicated against source — all 7 reported criticals
in these crates downgraded; these were the real highs).
- pf-inject steam_gadget: `SteamDeckGadget::drop` set `running=false` then joined
the control thread, which spends steady state parked in a blocking, no-timeout
`EVENT_FETCH` ioctl that only tests `running` at its loop top. The flag never
reaches it, closing the fd can't wake an in-flight ioctl (the syscall holds a
file reference, and the fd is shared via Arc by the very threads being joined),
so the join hung — and it runs on the session input thread via
`PadSlots::sweep`, driven by the client's `active_mask`, so a remote peer
clearing its pad bit could freeze all session input. Now wakes the parked
threads with SIGUSR1 (no-op, non-SA_RESTART handler → the ioctl returns EINTR
and the loop exits), retried until each reports done and bounded (~1s).
- pf-zerocopy cuda: the GL→CUDA "sync point" was never established for the copy.
`cuGraphicsMapResources`/`UnmapResources` were issued on the NULL stream, but
the D2D copy runs on `copy_stream()`, a `CU_STREAM_NON_BLOCKING` stream exempt
from implicit NULL-stream ordering — and the GL de-tile/CSC that produced the
texture ends with only `glFlush` (no fence). So the copy could race ahead of
the not-yet-retired GL draw: intermittent stale/torn frames under GPU load, on
the default NVIDIA capture→encode path. Map, copy, and unmap now share
`copy_stream()`, so map's device-side guarantee orders the GL work before the
copy. Zero-copy preserved (no GPU→CPU→GPU roundtrip).
- pf-capture cursor meta: `update_cursor_meta` trusted three producer-written
fields (bitmap_offset, pixel offset, stride) with no bound against the metadata
region, driving OOB pointer arithmetic and an oversized `from_raw_parts` — an
OOB read that SIGSEGVs inside the PipeWire `.process` callback (uncatchable by
the surrounding `catch_unwind`). Switched to `spa_buffer_find_meta` to obtain
the region's real `size` and validate every offset with checked arithmetic
before each deref/slice, mirroring the fd-length guard the main frame path
already applies.
Compile + existing tests green on Linux .21 (real RTX 5070 Ti): pf-inject 74/0,
pf-zerocopy 17/0, pf-capture 1/0. The gadget deadlock path only executes on a
SteamOS host with raw_gadget/dummy_hcd (not reproducible on the CachyOS box), so
that fix is reasoned + compile-verified, not runtime-exercised.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
B: PadFeedback.game_drove -> rumble_drove, keyed on vibration-asserting reports — an
LED/adaptive-trigger stream can no longer feed the abandoned-rumble force-off while a
coalesced stop never re-asserts (the confirmed unbounded stuck-ON path). C: Linux parity —
every UHID backend now arms the shared watchdog (Steam Input drives these pads over hidraw
with Windows abandonment semantics) and the uinput mixer force-stops abandoned
infinite-replay FF effects (FfState, unit-tested). Shared PUNKTFUNK_RUMBLE_IDLE_MS hatch
(0 = off; non-zero floored above SDL's ~2 s rumble resend).
A: PadShm v2.1 — a 1024 B tail extension carrying an 8-slot lossless output-report ring,
feature-negotiated via zeroed reserved fields (out_ring_ver; deliberately NO
GAMEPAD_PROTO_VERSION bump — mixed generations degrade to the legacy latest-report slot
instead of failing closed). The pf-dualsense driver dual-writes both planes
(publish_output); the host's shared OutputDrain drains oldest->newest with a torn-read
recheck and an overflow->resync path (PadFeedback.resync force-stops + re-arms dedups).
pf-umdf-util grows a min_data_size map fallback. Ds*Feedback.fresh removed (dead).
design/rumble-root-fix.md par. A-C. Verified: pf-inject tests+clippy Linux+Windows (53/53
on winbox incl. the stop-coalesce repro); drivers ws check+clippy on the CI runner.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
inject.rs + inject/* (the per-OS injectors — wlroots virtual-input, KWin
fake_input, libei/reis, gamescope-EI on Linux; SendInput on Windows — plus the
virtual-gamepad HID stack: DualSense/DualShock4/Switch Pro/Steam Controller/Deck
over uhid/usbip and the Windows UMDF drivers, the proto codecs, the injector
service, and the uhid manager) move into crates/pf-inject behind the
InputInjector trait (plan §W6). It consumes punktfunk_core::input (the neutral
GamepadEvent/InputEvent vocabulary, moved to core in W5) + the pf-driver-proto
wire contract, and reaches pf-capture only for the Windows gamepad-channel
WUDFHost check + the resident-mouse compose-kick hook.
The one inject->vdisplay coupling (the libei gamescope-EI backend needs the EIS
relay socket path) is broken via a leaf: gamescope_ei_socket_file moves to
pf-paths as the shared contract — the gamescope producer (host vdisplay) keeps
its session-env-lock wrapper around it, the libei consumer (pf-inject) reads it
directly post-retarget. The host keeps a `mod inject { pub use pf_inject::* }`
shim so every crate::inject::* path (the native/gamestream input planes + devtest)
is unchanged; the heavy input deps (wayland/reis/xkbcommon/usbip + the KWin
fake-input protocol XML) moved with the crate.
Verified: Linux clippy -D warnings (pf-inject + host nvenc,vulkan-encode,pyrowave
--all-targets) + pf-inject 69/69 + host 230/230 tests; Windows clippy -D warnings
(pf-inject --all-targets + host nvenc,amf-qsv --all-targets) Finished exit 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
