Compare commits

...
33 Commits
Author SHA1 Message Date
enricobuehlerandClaude Opus 4.8 ebddffa4c8 chore(release): bump workspace version to 0.19.1
apple / swift (push) Successful in 1m30s
android-screenshots / screenshots (push) Successful in 3m1s
audit / cargo-audit (push) Successful in 2m14s
audit / bun-audit (push) Successful in 15s
ci / web (push) Successful in 1m8s
ci / docs-site (push) Successful in 1m11s
release / apple (push) Successful in 9m36s
android / android (push) Successful in 14m52s
ci / bench (push) Successful in 6m31s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 14m40s
apple / screenshots (push) Successful in 6m40s
ci / rust (push) Successful in 19m21s
deb / build-publish-host (push) Successful in 10m36s
deb / build-publish (push) Successful in 9m18s
linux-client-screenshots / screenshots (push) Successful in 7m40s
docker / deploy-docs (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
web-screenshots / screenshots (push) Successful in 2m59s
windows-host / package (push) Successful in 9m59s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m8s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m58s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m7s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m17s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m43s
flatpak / build-publish (push) Failing after 8m0s
An all-fixes patch on top of 0.19.0. The headline is session teardown — the
native stop flag made enforceable and bounded (ac3dc432) and GameStream ending
on the control peer's ENet disconnect (41fa25c4), so a session can no longer
outlive the client that started it on either plane.

Also: the Windows input-desktop pair (display writes b5fa878b + pen/touch
injection 24a24734 both surviving a UAC prompt), the Apple display-sleep guard
(bc5f6a38), the gamescope corner-parked cursor (4b2d2d1e), the KWin custom-mode
selection (3a33a694), the tvOS glob warnings (cc8cccbe), and the Linux
starter/env docs revamp (66172753).

Nothing moved on the wire (protocol 2) or at the C ABI (13), so no embedder
rebuild and 0.18/0.19 peers keep mixing freely.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 02:15:40 +02:00
enricobuehlerandClaude Fable 5 cc8cccbee6 fix(core/client): drop two redundant glob imports that warned on tvOS
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 35s
release / apple (push) Successful in 9m19s
ci / web (push) Successful in 1m12s
ci / docs-site (push) Successful in 1m21s
android / android (push) Successful in 12m19s
arch / build-publish (push) Successful in 12m19s
ci / bench (push) Successful in 6m36s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
apple / screenshots (push) Successful in 6m35s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
windows-host / package (push) Successful in 9m44s
deb / build-publish (push) Failing after 4m47s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m52s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m25s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m14s
deb / build-publish-host (push) Successful in 9m39s
flatpak / build-publish (push) Failing after 8m25s
ci / rust (push) Successful in 23m58s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m36s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m37s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m13s
docker / deploy-docs (push) Successful in 12s
`punktfunk-core` emitted `unused import: super::super::*` for
client/pump/handshake.rs and client/pump/input_task.rs — but only when
built for a tvOS target, where the cfg set leaves the glob contributing
nothing. Nobody saw them because CI never builds tvOS: that needs
BUILD_TVOS=1 and the Tier-3 build-std nightly path, and the Apple
workflow only runs swift build/test (macOS).

Both files also `use super::*`, i.e. the parent `pump` module, which
itself globs `super::*` (`client`) unconditionally — so every name stays
reachable by that path and the inner glob was pure duplication. Their
sibling datagram_task.rs already imports only `use super::*`, so this
also makes the module consistent. control_task.rs and data.rs keep
theirs: those genuinely resolve names through it and never warned.

Verified warning-free AND compiling on every slice the xcframework
ships — aarch64/x86_64-apple-darwin, aarch64-apple-ios,
aarch64-apple-ios-sim, aarch64-apple-tvos, aarch64-apple-tvos-sim,
x86_64-apple-tvos — plus Linux with and without the `quic` feature. A
full xcframework rebuild now reports 0 warnings (was 6: two per tvOS
slice), and swift build, swift test (170 tests), and xcodebuild for the
macOS, iOS and tvOS schemes all pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 02:11:49 +02:00
enricobuehlerandClaude Fable 5 ac3dc4323f fix(native/session): make the stop flag enforceable so a session can't outlive its client
Native sessions could survive long after the client was gone, in two
independent ways.

HOST: `stop` was only advisory. The one thing that ends a session is the
stream thread returning — every teardown (conn.close, the joins, and the
RAII drops of the session permit, admission entry and stream marker)
sits after that await, and nothing forced the issue. The encode loop
checks `stop` between iterations, so any unbounded call INSIDE one never
reaches the check, and one stuck syscall became a permanent zombie: it
held its semaphore slot (four of those and the host stops accepting
QUIC entirely), its admission entry (a later client gets "host busy"
forever), and not even the console's Stop button could clear it — that
button sets this same flag.

  * Bound the wait: once the session has been told to stop, the thread
    gets STREAM_STOP_GRACE (90 s, well past the 40 s capture-rebuild
    budget) to return, then teardown runs anyway. The thread is detached,
    not killed — Rust can't cancel a blocking thread — so it keeps its
    capturer/encoder until the stuck call returns, but the session's slot
    and admission entry come back and the host keeps serving. It logs at
    ERROR as the host wedge it is.
  * Bound the audio/input joins too — the last unbounded await in
    teardown.
  * Take the session permit AFTER the QUIC handshake instead of before
    `accept()`, so a host at its concurrency cap still accepts and the
    waiting client sees a live path instead of a silent dial timeout.
  * Bound the compositor helpers that caused the wedge in the first
    place: new pf-vdisplay `proc::{status_within, output_within}` kill a
    child that outlives its budget. `kscreen-doctor` is a Wayland client
    of the very compositor it configures, so against a wedged KWin it
    never returned; same for systemctl/dbus against a stuck session bus.

CLIENTS: the connection was never closed, so the host was right to keep
the session — it still had a live, keep-alive-answering peer.

  * Android: backgrounding did no teardown at all, and Android doesn't
    suspend the process, so the worker kept answering keep-alives until
    the OS reclaimed it (on a TV box, never). End the session on ON_STOP,
    via the existing onDispose path; a plain close, not a quit, so the
    host lingers the display for a fast return.
  * Apple: the .background arm was iOS-only AND gated on an opt-in that
    defaults off, so backgrounding did nothing — while the `audio`
    background mode kept the app (and its connection) alive indefinitely.
    Act unconditionally, and cover tvOS.
  * Core: `conn.close()` only queues the frame, and run_pump is the body
    of a block_on whose runtime is dropped the instant it returns, so the
    driver could never put it on the wire — a deliberate quit reached the
    host as silence (8 s idle timeout, no quit code, and the linger meant
    for an unwanted disconnect). Carry the endpoint out of the handshake
    and flush with wait_idle(), the same discipline the pairing and probe
    paths already use.

Linux check/clippy/tests green: 262 host, 71 pf-vdisplay (incl. new
bounded-process tests), 231 core.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 02:11:49 +02:00
enricobuehlerandClaude Fable 5 41fa25c440 fix(gamestream/session): end the session when the client disconnects or vanishes
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m6s
apple / swift (push) Successful in 1m24s
decky / build-publish (push) Successful in 31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 18s
ci / bench (push) Successful in 7m2s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m8s
deb / build-publish (push) Successful in 9m35s
arch / build-publish (push) Successful in 12m19s
deb / build-publish-host (push) Successful in 12m46s
android / android (push) Successful in 16m5s
windows-host / package (push) Successful in 16m34s
apple / screenshots (push) Successful in 6m28s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m11s
ci / rust (push) Successful in 27m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m19s
docker / deploy-docs (push) Successful in 27s
GameStream sessions outlived their client: the only complete teardowns
were the explicit ones (RTSP TEARDOWN, nvhttp /cancel, mgmt DELETE
/session), and the only automatic detector was a media-UDP send error —
which needs an ICMP port-unreachable, so a true vanish (Wi-Fi drop,
sleep, power off, crash) left video+audio encoding into the void
forever, and even a plain Moonlight quit (which sends neither TEARDOWN
nor /cancel) leaked the session. The stale state then cascaded: a
lingering launch 503-blocked a different client under
mode_conflict=reject, and streaming=true made a reconnect's PLAY take
its "stream already running" branch — no new threads, old threads still
aimed at the dead endpoint, the reconnect got no media.

ENet already detects all of this — the control peer's reliable-ping
timeout (or clean disconnect) fires Event::Disconnect within ~5-30 s —
but the handler only reset input state. Wire the real teardown into it:

* AppState::end_session — THE compat-plane session teardown: stops both
  media threads (their flags), clears launch + negotiated stream config;
  idempotent. /cancel and mgmt stop_session now share it.
* control.rs Disconnect → end_session. Gated on the TRACKED session
  peer, and Connect only tracks a peer from the /launch owner's IP (the
  same source-IP bind the RTSP/media plane uses), so an unauthenticated
  LAN peer connect+disconnect can't end a live session, and a fast
  reconnect's stale-peer timeout can't kill its successor.
* Client-unreachable UDP send errors now end the whole session via an
  OnSessionLost callback (built at PLAY) instead of stopping only the
  plane that noticed — audio no longer keeps streaming after video
  detects the dead client, and vice versa.

Linux check/clippy/tests green (53 gamestream tests incl. the new
end_session regression test).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 02:10:23 +02:00
enricobuehlerandClaude Opus 4.8 3a33a69401 fix(kwin): select the custom mode KWin actually built, not the one we asked for
apple / swift (push) Successful in 1m19s
apple / screenshots (push) Successful in 6m41s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m3s
ci / bench (push) Successful in 5m33s
android / android (push) Successful in 11m38s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
deb / build-publish (push) Successful in 9m6s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m43s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
deb / build-publish-host (push) Successful in 9m50s
arch / build-publish (push) Successful in 21m2s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m27s
docker / deploy-docs (push) Failing after 15s
ci / rust (push) Successful in 28m29s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m21s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m49s
A KDE host streaming to a phone-shaped client fell back to 60 Hz even though the
120 Hz mode was sitting right there in KDE's display list, one manual click away.

KWin generates every custom mode's timing with libxcvt, whose first step is
`hdisplay_rnd = hdisplay - (hdisplay % 8)`. A 2868x1320@120 request (iPhone 16
Pro Max) therefore becomes 2864x1320@119.92 — 4 px narrower, fractional refresh.
We then asked kscreen-doctor for `mode.2868x1320@120`; its `findMode` matches a
mode's id or its own `WxH@qRound(Hz)` name, so that string matched nothing, the
select silently no-op'd, the output stayed on its sacrificial birth mode, and
`size_applied` came back false → "KWin rejected the custom mode" → 60 Hz.
Widths like 1920/2560/3840 are all multiples of 8, which is why only clients with
phone-shaped panels ever hit this.

Resolve the mode out of the output's OWN list instead and address it by kscreen
mode id: exact height, width at most one cell narrower than asked, refresh within
1 Hz (which excludes the native 60 Hz entry). `set_custom_refresh` now returns the
whole achieved mode, and `create` reports that as the output's `preferred_mode` —
so the capturer's renegotiation gate waits for the size KWin will actually deliver
and the encoder opens against it, rather than starving on the requested one.

Also skip `addCustomMode` when a usable mode is already installed: kscreen-doctor
APPENDS to the list and KWin persists it per output name, so the old code grew the
user's display list by one entry per connect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:55:36 +02:00
enricobuehlerandClaude Opus 4.8 24a24734eb fix(inject/windows): follow the input desktop for pen + touch injection
apple / swift (push) Successful in 1m16s
apple / screenshots (push) Successful in 6m43s
android / android (push) Failing after 7m44s
ci / web (push) Successful in 2m3s
ci / docs-site (push) Successful in 1m14s
arch / build-publish (push) Successful in 12m50s
ci / bench (push) Successful in 6m30s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 16s
deb / build-publish (push) Successful in 9m13s
ci / rust (push) Successful in 19m44s
deb / build-publish-host (push) Successful in 9m30s
docker / deploy-docs (push) Successful in 26s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 25m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 25m10s
Same field report as the display-write fix, other half of the symptom: with a
UAC consent prompt up — one the user could SEE in the stream, because capture
already renders the secure desktop (326d6e17) — pen and touch did nothing,
while mouse and keyboard kept working.

The split was exactly which paths knew the input desktop can move. sendinput.rs
reattaches, so mouse/keyboard reached the prompt. pointer_windows.rs had no
desktop handling at all, so every InjectSyntheticPointerInput came back
ERROR_ACCESS_DENIED:

    22:20:31  virtual pen created (PT_PEN)
    22:20:31  pen inject failed    error=Zugriff verweigert (0x80070005)
    22:20:42  touch inject failed  error=Zugriff verweigert (0x80070005) contacts=1

Measured on glass before fixing, to find out what it actually takes:

    device on Default, thread on Default  -> 0x80070005   (the field failure)
    device on Default, thread on INPUT    -> OK
    device on INPUT,   thread on INPUT    -> OK

The middle row is load-bearing: the synthetic pointer device is NOT
desktop-affine, so rebinding the thread suffices and the device is never
destroyed and recreated across a desktop switch — which would have dropped
in-flight contacts and the pen's in-range state mid-stroke.

Injection now retries once bound to the input desktop. The binding is scoped,
not persistent like sendinput's: inject_pen/inject_touch_frame run on TWO
threads (the caller's apply_batch and the refresh threads), and the batch caller
is a shared task thread that must not be left parked on a Winlogon desktop that
disappears when the prompt is dismissed.

The first-failure WARN now carries the rejected sample (flags, pen flags/mask,
pressure, rotation, tilt, position). A 0x80070057 INVALID_PARAMETER was seen
once BEFORE any prompt existed and is still unexplained; the ranges all look
sound on inspection (coordinates clamped to the virtual screen, roll/azimuth
u16 so the modulo cannot go negative, tilt bounded 0..=90), so catching the
actual offending sample is the way to find it.

Verified on glass: with a consent prompt up, pen and touch now reach it — zero
inject failures where every prior session failed immediately. A UAC prompt can
now be dismissed from an iPad with the Pencil.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:36:51 +02:00
enricobuehlerandClaude Opus 4.8 b5fa878bc6 fix(pf-win-display): follow the input desktop so a UAC prompt can't refuse display writes
Field-reported 2026-07-23: a consent prompt left up on an unattended host made
the box unreachable. Every connect black-screened and failed after ~60 s, and
the host had to be reached over SSH to clear the prompt.

Windows refuses ChangeDisplaySettingsEx/SetDisplayConfig issued from a thread
that is not on the desktop currently receiving input. While UAC (or the lock /
logon screen) is up that desktop is Winlogon, and the host — which the service
launches explicitly onto WinSta0\Default — got DISP_CHANGE_FAILED /
ERROR_ACCESS_DENIED for every write. A session starting in that window could
never set its virtual display's mode, so the capturer sized its ring to a stale
mode, every composed frame was dropped for a size mismatch, and bring-up burned
8 retries before giving up.

Measured on glass (RTX box, SYSTEM host in console session 2, real consent
prompt up, virtual display active):

    INPUT desktop = Winlogon
    UNBOUND  CDS_TEST      -> -1 (DISP_CHANGE_FAILED)
    UNBOUND  SDC_VALIDATE  -> 0x5 (ERROR_ACCESS_DENIED)
    BOUND    CDS_TEST      ->  0 (DISP_CHANGE_SUCCESSFUL)
    BOUND    SDC_VALIDATE  -> 0x0 (ERROR_SUCCESS)

New input_desktop.rs mirrors pf-inject's sendinput.rs retry model: issue the
write, and only when it fails the way a wrong-desktop write fails, rebind this
thread to the current input desktop and retry once. A working write is never
touched, so the normal path is unchanged. The retry predicate stays narrow
(ERROR_ACCESS_DENIED only) so the unrelated 0x57 exclusive-mode topology bug is
not re-issued and mis-attributed. Unlike sendinput's dedicated injector thread,
the binding here is SCOPED — a shared display-write thread left on a Winlogon
desktop that is later destroyed would fail every subsequent write, which is the
wedge this removes.

Applied to all eight SetDisplayConfig sites and both ChangeDisplaySettingsExW
calls in set_active_mode. The isolate site now decides its supplied config once,
outside the write, so a retry re-issues the identical config rather than logging
its escalation twice.

A save is logged, because a silent one is indistinguishable in a field log from
a write that never needed saving — which made the first on-glass verification of
this change inconclusive.

Both ACCESS_DENIED diagnostics now ask which cause applies instead of naming
only the disconnected-RDP one: that phrasing sent this investigation chasing a
phantom RDP session on a host that was already service-launched in the console
session, while a consent prompt was the actual cause.

Verified on glass: connect with a consent prompt up now reaches first frame in
3.0 s (was a black screen then "Connection failed" after ~60 s), with six
"retried bound to it and it applied desktop=Winlogon" lines and rc=0x0
throughout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:36:51 +02:00
enricobuehler 4b2d2d1e14 fix(gamescope/cursor): follow gamescope's own cursor verdict — the corner-parked pointer
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 58s
apple / swift (push) Successful in 1m23s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 6m36s
apple / screenshots (push) Successful in 6m34s
deb / build-publish (push) Successful in 9m41s
deb / build-publish-host (push) Successful in 10m13s
arch / build-publish (push) Successful in 12m43s
android / android (push) Successful in 16m41s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m41s
docker / deploy-docs (push) Successful in 25s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m4s
ci / rust (push) Successful in 29m44s
gamescope hides its pointer by WARPING the X pointer to the root's bottom-right corner pixel;
it does not swap in a transparent X cursor, so XFixesGetCursorImage keeps handing back the last
opaque arrow. The XFixes source picked its display by "whichever pointer moved", and a parked
pointer never moves again — so it froze on the parked Xwayland and composited that arrow at
(w-1, h-1): a sliver of cursor welded to the corner of the stream for the rest of the session
while the real pointer went undrawn. Reported on-glass in every game (a game grabs the pointer,
so the hide is permanent): "part of cursor shows up on bottom right ... isn't where it really is".

Follow GAMESCOPE_CURSOR_VISIBLE_FEEDBACK instead — gamescope publishes it on every nested
Xwayland root: 1 on the server whose pointer it is drawing, 0 on the others and 0 on all of them
once the pointer is hidden. It answers both questions correctly for a STATIC pointer, which
motion cannot: which Xwayland owns it, and whether to draw it at all. Read at connect, re-read
on its root PropertyNotify (event drain now discriminates CursorNotify vs PropertyNotify) with a
250 ms resync as the self-heal. A pointer gamescope draws nowhere is published visible:false,
not dropped — the encode loop overwrites the frame's overlay from this slot and strips invisible
ones, so a None would leave the last visible overlay standing on repeat frames. Honouring the
atom also gives the stream gamescope's own idle auto-hide, which this source never had.

The pointer-motion heuristic stays as the fallback for a gamescope that publishes no verdict
(logged at session start via cursor_feedback=false), so nothing regresses to a cursorless stream.

Measured on a live 1920x1080 Steam Gaming Mode session (RTX 5070 Ti, gamescope c31743d+,
--xwayland-count 2 --hide-cursor-delay 3000): idle 3 s => pointer (1919,1079) + feedback 0;
real evdev motion => pointer live + feedback 1. End-to-end against that session the source now
publishes visible=false while parked and tracks the live pointer within one 250 ms sample of
motion. cargo clippy -p pf-capture --lib -D warnings and the 5 new pick_active tests are green
at cargo 1.96.1 (CI parity); cargo fmt --all --check clean.
2026-07-24 00:31:55 +02:00
enricobuehlerandClaude Opus 4.8 bc5f6a3881 fix(apple/session): keep the display awake for the length of a session
ci / web (push) Successful in 50s
apple / swift (push) Successful in 1m19s
ci / docs-site (push) Successful in 1m58s
decky / build-publish (push) Successful in 33s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 54s
deb / build-publish (push) Failing after 4m47s
ci / bench (push) Successful in 6m1s
docker / deploy-docs (push) Successful in 33s
release / apple (push) Successful in 10m4s
android / android (push) Successful in 13m5s
deb / build-publish-host (push) Successful in 12m52s
arch / build-publish (push) Successful in 16m56s
apple / screenshots (push) Successful in 6m33s
ci / rust (push) Successful in 19m36s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m27s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m15s
A stream is not user activity to the OS, and controller input does not feed
the HID idle timer on any Apple platform — so a gamepad-only session reliably
idles the panel out from under the user mid-play. A keyboard/mouse capture
session masks the bug because those events are real local HID. The Android
client has held FLAG_KEEP_SCREEN_ON while streaming all along; the Apple
clients held nothing.

DisplaySleepGuard is acquired in beginStreaming and released at the top of
disconnect, so it is scoped to the session — every teardown path (user quit,
sessionEnded, the backgrounded keep-alive timeout) funnels through disconnect.

  iOS/iPadOS/tvOS: UIApplication.isIdleTimerDisabled. App-wide and ignored
  while backgrounded, which is what the audio-only keep-alive wants.

  macOS: ProcessInfo.beginActivity(.userInitiated, .idleDisplaySleepDisabled),
  the Foundation wrapper over IOKit power assertions. That defers display
  sleep but NOT the screen saver, which runs off the HID idle timer
  independently — so a 30 s IOPMAssertionDeclareUserActivity heartbeat runs
  alongside it. Intended side effect: an idle-lock that follows the screen
  saver is deferred for the session too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:00:55 +02:00
enricobuehlerandClaude Fable 5 6617275387 docs(env): stop teaching the compositor pin + uid-1000 anchors in starters
ci / docs-site (push) Successful in 54s
ci / web (push) Successful in 59s
apple / swift (push) Successful in 1m21s
decky / build-publish (push) Successful in 28s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 25s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
ci / bench (push) Successful in 6m23s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m43s
apple / screenshots (push) Successful in 6m18s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m9s
arch / build-publish (push) Successful in 13m44s
deb / build-publish (push) Successful in 13m48s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7m39s
deb / build-publish-host (push) Successful in 13m11s
docker / deploy-docs (push) Successful in 32s
android / android (push) Successful in 16m23s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m15s
ci / rust (push) Successful in 24m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m55s
Field triage (Nobara, Discord): the kde.md starter host.env told desktop
users to set PUNKTFUNK_COMPOSITOR=kwin, which PINS the backend — detect()
short-circuits and the capture-loss rebuild never re-detects — so a
mid-stream switch to Game Mode killed the stream instead of following it.
A follow-up hardcoded XDG_RUNTIME_DIR=/run/user/1000 anchor broke PipeWire
for any non-1000 uid (pw audio connect: Creation failed).

Revamp across every starter/example/reference:
- Desktop starters (kde/gnome/hyprland/sway) shrink to
  PUNKTFUNK_VIDEO_SOURCE=virtual + an explicit warning that pinning
  disables session-following; forcing a backend is CI/appliance-only.
- host.env.example: rewritten around auto-detection; anchors demoted to a
  commented ssh/cron-only block with the uid trap spelled out; the
  gamescope ATTACH/MANAGED knobs documented (previously missing);
  case-sensitivity called out.
- packaging/bazzite/host.env + README: drop the uid-1000 anchors (a
  systemctl --user service inherits/derives them); README's stale
  PUNKTFUNK_COMPOSITOR=gamescope-era template synced to the real one.
- packaging/kde/host.env: loud APPLIANCE-ONLY header (it pins on purpose).
- configuration.md: session-anchors section inverted to "leave unset",
  compositor row states the pin consequence, case-sensitivity note.
- troubleshooting.md: new "session fails right after editing host.env"
  section (case, wrong-uid anchors, stale pin, restart-to-apply).
- gamescope.md/bazzite.md: attach/managed descriptions match current
  behavior (managed is the infra-detected default; attach re-modes a
  box-owned session to the client's resolution).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 22:42:59 +02:00
enricobuehlerandClaude Opus 4.8 bda015b101 docs(release): rewrite v0.19.0 notes for end users + set end-user notes voice
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m8s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 36s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 15s
apple / screenshots (push) Successful in 6m29s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m1s
ci / bench (push) Successful in 8m1s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 18s
deb / build-publish-host (push) Successful in 10m19s
deb / build-publish (push) Successful in 12m13s
android / android (push) Successful in 13m20s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m26s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 16m53s
ci / rust (push) Successful in 25m37s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m52s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m8s
Release notes were written for the people who build Punktfunk, not the people who use it —
dense with protocol/ABI/type names that confused even technical readers. Rewrite v0.19.0 in a
benefit-first, plain-language voice (New/Improved/Fixed) with all internal terms removed from the
body and every protocol/ABI/embedder detail moved to a single bottom "Under the hood" section.
docs/releases/README.md now codifies this voice as the format spec for all future notes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:15:08 +02:00
enricobuehlerandClaude Opus 4.8 ec219763a6 fix(ci/release): make _release_notes_path POSIX-sh safe (deb/decky attach runs under dash)
ci / web (push) Successful in 55s
apple / swift (push) Successful in 1m26s
ci / docs-site (push) Successful in 56s
android-screenshots / screenshots (push) Successful in 3m20s
ci / bench (push) Successful in 6m47s
release / apple (push) Successful in 10m49s
android / android (push) Successful in 15m8s
decky / build-publish (push) Successful in 26s
apple / screenshots (push) Successful in 6m40s
deb / build-publish (push) Successful in 9m36s
deb / build-publish-host (push) Successful in 9m39s
flatpak / build-publish (push) Successful in 6m33s
arch / build-publish (push) Successful in 15m57s
ci / rust (push) Successful in 26m4s
docker / deploy-docs (push) Successful in 16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
web-screenshots / screenshots (push) Successful in 3m10s
windows-host / package (push) Successful in 10m33s
linux-client-screenshots / screenshots (push) Successful in 7m51s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m12s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m3s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m57s
The v0.19.0 deb + decky release-attach steps source gitea-release.sh under `sh` (dash) and
died with "Bad substitution" at _release_notes_path's `${BASH_SOURCE[0]:-$0}` — a bash array
subscript dash rejects. So those legs never attached their assets to the release (the bash
legs — apple, android, rpm — were fine, which is why the body still seeded and the DMG/.ipa
attached). CI always sources this from the repo root, so resolve the notes as
docs/releases/<tag>.md relative to CWD and drop $BASH_SOURCE entirely. Verified under dash.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 19:03:01 +02:00
enricobuehlerandClaude Opus 4.8 0dce1cd116 chore(release): bump workspace version to 0.19.0
audit / bun-audit (push) Successful in 11s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 54s
apple / swift (push) Successful in 1m20s
audit / cargo-audit (push) Successful in 2m13s
ci / bench (push) Successful in 6m52s
android-screenshots / screenshots (push) Successful in 3m58s
release / apple (push) Successful in 11m11s
deb / build-publish (push) Failing after 8m56s
decky / build-publish (push) Failing after 24s
deb / build-publish-host (push) Failing after 9m15s
android / android (push) Successful in 13m6s
ci / rust (push) Successful in 25m44s
arch / build-publish (push) Successful in 13m37s
docker / deploy-docs (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
apple / screenshots (push) Successful in 6m32s
web-screenshots / screenshots (push) Successful in 3m2s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 2m43s
flatpak / build-publish (push) Failing after 8m20s
linux-client-screenshots / screenshots (push) Successful in 7m46s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m45s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m4s
windows-host / package (push) Successful in 15m54s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m26s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m49s
Headline: the whole pen/tablet input stack (P0–P5) — a pressure-sensitive stylus plane
across core wire, Linux, GameStream/Moonlight, Windows, iPad Pencil and Android. Plus
touch-injection fixes, cursor-DPI scaling, three Windows-host display/cursor fixes, a
gamepad same-PID conflict fix, and the Android mouse/keyboard (TV) regressions.

Release notes authored in-repo at docs/releases/v0.19.0.md (first release on the new flow;
CI seeds the release body from it). cargo update --workspace syncs the lock (versions only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:26:49 +02:00
enricobuehlerandClaude Opus 4.8 c3c0115a82 fix(core/abi): bump ABI_VERSION to 13 for punktfunk_connection_send_pen
The pen P0 work added a new embeddable C entry point (punktfunk_connection_send_pen)
but left ABI_VERSION at 12 — the value 0.18.0 shipped. Per the project's convention
(0.18.0 bumped to 12 for the additive cursor channel), additive C-surface growth bumps
the ABI so embedders can gate on the new symbol. Regenerated include/punktfunk_core.h
via cbindgen (cargo build -p punktfunk-core); WIRE_VERSION is unchanged (pen is
capability-gated on HOST_CAP_PEN).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:23:38 +02:00
enricobuehlerandClaude Opus 4.8 00c8c12a0f feat(ci/apple): export the iOS .ipa to the release + run artifacts
The iOS leg archives with development signing and re-signs for App Store distribution at
export time, but the existing export uses destination=upload (straight to TestFlight) and
leaves no .ipa on disk. Add an export step that re-exports the SAME archive with
destination=export to produce an App Store distribution-signed .ipa, then:
  * attach it as the `punktfunk-ios-ipa` run artifact on every build (upload-artifact@v3,
    since Gitea's backend rejects @v4), and
  * attach it to the unified Gitea release on vX.Y.Z tags (alongside the DMG), via the
    shared ensure_release/upsert_asset helpers.

Same gate as the archive; warn+skip (never fails the best-effort iOS leg) if the archive is
absent. Note: an App Store-signed .ipa installs only via TestFlight/App Store, not by direct
sideload — it is a release/archival artifact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:18:45 +02:00
enricobuehlerandClaude Fable 5 988b5742ec fix(gamestream): tolerate NaN pressureOrDistance — VoidLink's finger touches were dropped whole
apple / swift (push) Successful in 1m18s
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m6s
apple / screenshots (push) Successful in 6m11s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
windows-host / package (push) Successful in 16m57s
android / android (push) Successful in 15m58s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 17m3s
deb / build-publish (push) Successful in 9m26s
deb / build-publish-host (push) Successful in 10m5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m39s
ci / rust (push) Successful in 25m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m31s
The hex dumps from the on-glass session show textbook SS_TOUCH packets with
pressureOrDistance = 0x7fc00000 (NaN): VoidLink encodes 'unknown finger
pressure' as NaN (iPad fingers have no force sensor), and the anti-forgery
finiteness gate rejected the entire packet — silently disabling the touch
plane while pen (real Pencil force, always finite) worked. Coordinates keep
the strict gate (they feed injector scaling); pressureOrDistance now
sanitizes non-finite to 0.0, the spec's own 'unknown', which the pen path
already maps to full-scale ink on contact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 f938174d86 debug(gamestream): hex-dump the first few undecodable pointer packets
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 40611cd54d debug(gamestream): surface the pointer plane's silent failure modes
An on-glass 'touch does nothing' currently leaves NO trace: log the first
SS_TOUCH per session at info (proves the client sends + we decode), and warn
when a packet carrying a pointer magic fails the body parse (layout mismatch)
instead of vanishing into the unknown-magic drop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 98b97d7d76 fix(android/input): stop mouse back/forward leaking to nav + IME popping on hardware typing
apple / swift (push) Successful in 1m25s
apple / screenshots (push) Successful in 7m4s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m14s
android / android (push) Successful in 11m54s
arch / build-publish (push) Successful in 11m58s
ci / bench (push) Successful in 5m36s
decky / build-publish (push) Successful in 24s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
deb / build-publish (push) Successful in 11m52s
docker / deploy-docs (push) Successful in 11s
deb / build-publish-host (push) Successful in 10m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m15s
ci / rust (push) Successful in 26m1s
Two Android TV M&K regressions from the c90343c2 overhaul, both user-reported:

1. Mouse back/forward buttons doubled as Android navigation while streaming.
   The BUTTON_BACK/FORWARD motion edges already go over the wire as X1/X2, but
   Android's input reader ALSO synthesizes KEYCODE_BACK/FORWARD key events from
   those buttons unconditionally (stamped SOURCE_MOUSE, no FLAG_FALLBACK) — and
   the guard only swallowed the FLAG_FALLBACK variant, so the duplicate reached
   the BackHandler and yanked the user out of the stream. Swallow mouse-sourced
   BACK/FORWARD too; a remote/keyboard BACK is never mouse-sourced and still
   exits.

2. The soft keyboard popped up on physical-keyboard typing. KeyCaptureView (the
   always-focused IME/capture anchor) unconditionally claimed to be a text
   editor with a live editable InputConnection, so the TV IME counted input as
   active and showed its UI the moment hardware keys arrived. Gate the editor
   identity on imeShown (the user actually summoning the keyboard via gesture /
   remote toggle), with restartInput on both edges so the gate is re-read, and
   an onKeyPreIme BACK hook so an IME self-dismissal can't leave the gate
   stale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:07:18 +02:00
enricobuehlerandClaude Fable 5 51792536d1 fix(pf-win-display): escalate CCD isolate to a keep-only supplied config on 0x57
apple / swift (push) Successful in 1m30s
ci / web (push) Successful in 1m8s
apple / screenshots (push) Successful in 6m59s
ci / docs-site (push) Successful in 1m6s
android / android (push) Successful in 14m35s
arch / build-publish (push) Successful in 14m20s
decky / build-publish (push) Successful in 21s
ci / bench (push) Successful in 6m4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 18s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
deb / build-publish (push) Successful in 9m41s
ci / rust (push) Successful in 20m14s
deb / build-publish-host (push) Successful in 11m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m42s
docker / deploy-docs (push) Successful in 12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m2s
Field bug (Steam Deck client, exclusive topology): every isolate retry failed
SetDisplayConfig 0x57 with the physical panel left lit, while iPhone sessions
worked — their virtual monitor's connected-set already had a sole-display
topology persisted, so Windows deactivated the physical on hotplug and the
isolate was a no-op re-commit (deactivated 0). The Deck's monitor identity had
no such entry, forcing the live-deactivate path: supplying the doomed path
(inactive, modes unpinned) plus its orphaned mode entries is rejected 0x57 by
some driver/OS validation combos, so the retry loop could never converge — the
prior mode-index unpin (5e088af7) was necessary but not sufficient on this box.

Attempt 1 keeps the shipped shape; once a survivor is seen, later attempts
supply ONLY the keep paths with the mode table rebuilt to just their referenced
entries (indexes remapped, clone-shared entries deduped). The same host applies
exactly this keep-only shape rc=0 whenever the DB has pre-isolated the virtual
display, so the escalated shape is field-validated on the failing machine. The
final attempt also drops SDC_FORCE_MODE_ENUMERATION — a real path removal
drives COMMIT_MODES on its own.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:50:44 +02:00
enricobuehlerandClaude Fable 5 bf2c4456bb fix(inject/windows): compact wire touch ids into slots — Moonlight native touch injected nothing
apple / swift (push) Successful in 1m28s
ci / web (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m12s
release / apple (push) Successful in 9m24s
ci / bench (push) Successful in 6m28s
android / android (push) Successful in 11m59s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 12m30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
apple / screenshots (push) Successful in 6m53s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 19s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 17s
deb / build-publish (push) Successful in 9m36s
deb / build-publish-host (push) Successful in 11m24s
ci / rust (push) Successful in 25m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m53s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m59s
docker / deploy-docs (push) Successful in 25s
On-glass report (iPad Moonlight vs the Windows host): pen inked, native touch
did nothing. Cause: POINTER_INFO.pointerId was the client's raw wire id —
Moonlight sends arbitrary large pointerIds, and synthetic-pointer injection
rejects them (Apollo compacts ids into dense slots for exactly this reason);
the failure was logged at trace, i.e. invisibly. Wire ids now map to the
lowest free slot for the contact's lifetime, and the FIRST injection failure
of a device logs at WARN (pen too) so an inert input plane is never silent
again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Fable 5 c0f792ee8d feat(android): pen P5 — stylus capture onto the pen plane
The Android leg of design/pen-tablet-input.md §7: against a HOST_CAP_PEN host,
stylus/eraser pointers split out of BOTH touch models (passthrough + gesture)
into StylusStream — state-full samples with pressure, AXIS_TILT, azimuth from
AXIS_ORIENTATION (Android's 0 = away-from-user IS wire north), AXIS_DISTANCE
hover, both stylus barrel buttons, the eraser tool, and historical (coalesced)
samples batched oldest-first. Kotlin heartbeats ≤100ms per the wire contract.
JNI: nativeHostSupportsPen + nativeSendPen (flat 10-float stride, sentinels
<0). No barrel-roll axis exists on Android — roll stays unknown here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Fable 5 c8aa3ed48b feat(apple/ios): pen P4 — Apple Pencil (Pro) capture onto the stylus plane
The flagship leg of design/pen-tablet-input.md §7: against a HOST_CAP_PEN host
the Pencil splits out of the finger path (independent of touch-input mode) into
PencilStream — state-full samples with force→pressure, altitude→polar tilt,
azimuth (+90° to the wire's north-clockwise convention), Pencil Pro rollAngle→
barrel roll (17.5+), coalesced touches batched ≤8/send for full 240Hz fidelity,
and Pencil hover (zOffset>0 distinguishes it from trackpad hover) as in-range
samples with distance. Squeeze holds barrel 1, double-tap clicks barrel 2 (the
Pencil has no hardware buttons/eraser — these are how host apps get those
affordances). Implements the ≤100ms heartbeat wire contract (80ms timer) so a
stationary held stroke survives the host's 200ms dead-client failsafe. Toward a
pen-less host nothing changes (Pencil stays a finger).

Also fixes hostSupportsCursor testing the STALE 0x04 bit — HOST_CAP_CURSOR
moved to 0x08 when TEXT_INPUT claimed 0x04; the old test would mistake a
text-input host (e.g. Windows) for a cursor grant and double-render the pointer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Opus 4.8 15d09a8c10 feat(ci/release): author release notes in-repo + announce stable releases to Discord
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m25s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
ci / bench (push) Successful in 6m55s
apple / screenshots (push) Successful in 6m47s
deb / build-publish (push) Successful in 11m23s
android / android (push) Successful in 12m36s
arch / build-publish (push) Successful in 12m46s
docker / deploy-docs (push) Successful in 23s
deb / build-publish-host (push) Successful in 12m32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m9s
ci / rust (push) Successful in 26m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m30s
Release notes now live in the repo at docs/releases/vX.Y.Z.md, authored during the
version bump (before the tag). ensure_release (gitea-release.sh + .ps1 twin) seeds the
Gitea release body from that file in the create POST, so the release is born WITH its
notes instead of being created empty and PATCHed afterward. canary/rc have no file ->
empty body, unchanged.

New manual `announce` workflow (workflow_dispatch, tag input) re-syncs the notes file
over the live release and posts a violet embed (title, notes lead-in, release-page link)
to the Discord #releases channel via the DISCORD_RELEASE_WEBHOOK secret. Stable-only:
a -rc tag is refused unless allow_prerelease=true. Pressing "go" is the quality gate, so
a half-built or failed release is never announced.

docs/releases/README.md documents the ritual; TEMPLATE.md is the skeleton.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 16:43:18 +02:00
enricobuehlerandClaude Opus 4.8 6cbc3eec8b fix(sdl/cursor): scale the forwarded pointer to the video fit
apple / swift (push) Successful in 1m30s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m3s
android / android (push) Canceled after 1m27s
apple / screenshots (push) Canceled after 1m39s
arch / build-publish (push) Canceled after 1m29s
ci / rust (push) Canceled after 1m29s
ci / bench (push) Canceled after 1m17s
deb / build-publish (push) Canceled after 1m15s
deb / build-publish-host (push) Canceled after 27s
decky / build-publish (push) Canceled after 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 15s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 1m18s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 8s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 18s
flatpak / build-publish (push) Canceled after 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / deploy-docs (push) Canceled after 0s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 17s
Parity with the Apple client (35c2c145): the SDL/Linux presenter built the
host-forwarded cursor as an OS cursor at its native bitmap size, so the
on-screen pointer tracked the HOST's display scaling — a 4K/high-DPI virtual
display forwards a 96 px bitmap that then rendered huge on the client.

SDL cursors are fixed-size from their surface (no draw-time scaling), so
cache the shapes RAW and resample each by the live video-fit factor
(min(window_px / mode) — the same aspect-fit `finger_to_content` uses) at
install time, rebuilding when the serial OR the fit changes (a resize). The
resample area-averages on premultiplied alpha so a transparent pixel's
colour can't bleed into the fringe. The pointer now stays sized to the
streamed desktop at any host scaling.

Not yet eyeballed on a Deck/Linux box — SDL's HiDPI cursor sizing is
version-dependent; validate on glass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 16:28:34 +02:00
enricobuehlerandClaude Fable 5 45c9799aa6 feat(inject/windows): pen P3 — PT_PEN + PT_TOUCH synthetic pointer injection
apple / swift (push) Successful in 1m26s
ci / web (push) Successful in 59s
ci / docs-site (push) Successful in 54s
release / apple (push) Successful in 9m40s
arch / build-publish (push) Successful in 12m4s
ci / bench (push) Successful in 7m5s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 22s
apple / screenshots (push) Successful in 6m50s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
android / android (push) Successful in 14m47s
ci / rust (push) Canceled after 13m52s
deb / build-publish (push) Canceled after 7m26s
deb / build-publish-host (push) Canceled after 5m38s
flatpak / build-publish (push) Canceled after 1m55s
docker / deploy-docs (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 1m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 1m59s
windows-host / package (push) Canceled after 8m32s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
The Windows leg of design/pen-tablet-input.md §6, following Apollo's recipe:
a per-session PT_PEN device (pressure rescaled 0..1024, polar tilt→tiltX/Y,
barrel roll on rotation — Windows Ink renders Pencil Pro roll natively, barrel
button, eraser via INVERTED/ERASER flags, hover) with a 40ms refresh thread
against the ~100ms synthetic-pointer staleness auto-lift, plus a PT_TOUCH
device closing the historical SendInput wire-touch no-op (full active-contact
frames, per-id DOWN/UP edges, self-healing lost-DOWN synthesis). pen_supported
now probes PT_PEN creation on Windows (1809+), which lights up HOST_CAP_PEN
and the GameStream featureFlags there — Moonlight iPad + Pencil can ink on a
Windows host. Frame grouping mirrors the Linux uinput backend.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 b192825869 feat(gamestream): pen P2 — SS_PEN/SS_TOUCH ingest, featureFlags advertised on capable hosts
The Moonlight leg of design/pen-tablet-input.md §4: DESCRIBE now advertises
SS_FF_PEN_TOUCH_EVENTS wherever pen injection exists (Linux+uinput, same gate
as HOST_CAP_PEN — elsewhere the flag stays 0 and Moonlight keeps client-side
mouse emulation exactly as today). SS_PEN packets merge over last-known state
into state-full PenSamples (BUTTON_ONLY per spec carries no position; unknown
contact pressure 0.0 inks at full scale; UP keeps proximity only for clients
that demonstrated hover) and drive the same PenTracker→VirtualPen chain as the
native plane — Moonlight iOS + Apple Pencil now exercises the full injection
path end to end. SS_TOUCH forwards as wire touch on a synthetic 65535² surface
with CANCEL_ALL replayed per tracked contact. No stroke timeout on this plane:
ENet is ordered/reliable and Moonlight doesn't heartbeat a stationary pen.
Packet layouts pinned against moonlight-common-c Input.h/Limelight.h upstream.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 aca5aa9993 feat(host+inject): pen P1 — per-session uinput virtual tablet, HOST_CAP_PEN live
The Linux injection leg of design/pen-tablet-input.md: 0xCC/0x05 pen batches now
route through the per-session PenTracker into a lazily-created 'Punktfunk Pen'
uinput tablet (BTN_TOOL_PEN/RUBBER, pressure, tilt-from-polar, ABS_Z barrel
roll, hover distance, INPUT_PROP_DIRECT) — compositors pick it up via libinput
and hand apps zwp_tablet_v2 with full fidelity. The host now advertises
HOST_CAP_PEN when /dev/uinput is accessible (PUNKTFUNK_PEN=0 kill-switch);
transitions group into SYN frames so proximity-enter carries its position.
Stroke failsafe: clients heartbeat ≤100ms while in range (documented wire
contract — capture APIs are silent for a stationary pen); 200ms of silence
force-releases. 'punktfunk-host pen-test' draws a pressure-ramped sine stroke
through the real tracker→uinput chain, no client needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 248e1cbf08 fix(pf-vdisplay/driver): report cursor_excluded adapter-wide — the declare's exclusion is not per-target
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m20s
windows-drivers / probe-and-proto (push) Successful in 1m2s
decky / build-publish (push) Successful in 28s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
windows-drivers / driver-build (push) Successful in 2m23s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m5s
ci / bench (push) Successful in 6m3s
apple / screenshots (push) Successful in 6m41s
deb / build-publish (push) Successful in 12m20s
arch / build-publish (push) Successful in 12m47s
deb / build-publish-host (push) Successful in 13m7s
android / android (push) Successful in 16m14s
windows-host / package (push) Successful in 15m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m8s
ci / rust (push) Successful in 26m6s
docker / deploy-docs (push) Canceled after 25s
Field report (iPad via GameStream, 2026-07-23): cursor missing from the
stream. The session's fresh target 257 reported cursor_excluded=false
(DECLARED_TARGETS is keyed per target and only 259 had declared), so the
host skipped forced-composite — but DWM's pointer exclusion after an
IddCx hardware-cursor declare reaches EVERY later monitor of the adapter,
not just the declaring target (proven on-glass: declare on 259 under
WUDFHost 2192, then 257's frames streamed pointer-free with CURSOR_SHOWING
set and SM_MOUSEPRESENT true). GameStream/Moonlight clients cannot draw a
forwarded cursor, so those sessions were silently cursor-less.

ADD replies now report any-declare-this-WUDFHost-life (the state's real
scope — it dies with the adapter reset, same as before); per-target ids
stay only as the dbglog audit trail. Channel-less sessions on a declared
adapter then hit the existing forced-composite gate (GDI poller + blend)
and carry the pointer in-video. Host code unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 15:58:54 +02:00
enricobuehlerandClaude Opus 4.8 35c2c1450e fix(apple/cursor): scale the forwarded pointer to the video fit
apple / swift (push) Successful in 1m24s
release / apple (push) Successful in 10m11s
apple / screenshots (push) Successful in 6m42s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m9s
ci / bench (push) Successful in 5m48s
decky / build-publish (push) Successful in 25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 20s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 18s
android / android (push) Successful in 16m24s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
deb / build-publish (push) Successful in 10m11s
arch / build-publish (push) Successful in 12m18s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7m5s
docker / deploy-docs (push) Successful in 11s
deb / build-publish-host (push) Successful in 11m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m4s
ci / rust (push) Successful in 18m39s
The Mac client drew the host-forwarded cursor bitmap at its native pixel
size (pixel-as-points), so the on-screen pointer tracked the HOST's display
scaling: a 4K/high-DPI virtual display renders a 96 px pointer, which then
appeared huge on the client — and on retina every pointer was already
2x-inflated and upscaled-blurry, since the backing scale was ignored.

Cache the shape raw and rebuild the NSCursor at the live video-fit factor
(fit.width / mode.width — the same AVMakeRect fit the input mapping uses)
on each apply, re-fitting on resize / retina moves. The pointer now renders
at its true size relative to the streamed desktop at any host scaling:
crisp and 1:1 with the video at 100%, and proportional (not ballooning)
when the host sits at 200-300% — including a deliberately raised scale.

iOS/tvOS (UIPointerInteraction) and Android (no forwarded-bitmap path)
render the pointer differently and need no change; the SDL/Linux client has
the same native-size issue and will follow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 14:14:41 +02:00
enricobuehlerandClaude Fable 5 326d6e17c8 fix(windows/capture): stand the IddCx hardware cursor down while the secure desktop is up
apple / swift (push) Successful in 1m22s
apple / screenshots (push) Successful in 6m41s
ci / web (push) Successful in 1m7s
ci / docs-site (push) Successful in 1m18s
android / android (push) Successful in 12m44s
arch / build-publish (push) Successful in 12m36s
decky / build-publish (push) Successful in 26s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 20s
ci / bench (push) Successful in 6m14s
windows-host / package (push) Successful in 9m46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 21s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
deb / build-publish (push) Successful in 9m38s
deb / build-publish-host (push) Successful in 9m58s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7m0s
ci / rust (push) Successful in 24m1s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m7s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m55s
docker / deploy-docs (push) Successful in 12s
0.18.0 regression: UAC consent and Winlogon (lock/logon) stopped appearing
in streams. The cursor channel's IddCx hardware-cursor declare — re-issued
by the driver on every swap-chain assign — keeps the path out of the OS's
software-cursor mode, which is the only mode the secure desktop renders
through; DWM then never presents UAC/Winlogon into our swap-chain and the
stream repeats the last normal-desktop frame for the whole interaction.

The GDI cursor poller now classifies the input desktop on its reattach
cadence (UOI_NAME != Default = secure, 250 ms cadence instead of 2 s), and
the capturer edge-triggers the existing-but-unused proto-v6
IOCTL_SET_CURSOR_FORWARD flip: OFF at secure entry (the driver stops its
per-assign re-declare; the host facade forces the same-mode re-commit that
actualises the software-cursor default, under the vdisplay manager lock via
the new force_recommit()), back ON at dismissal for channel sessions.
Channel-session open resets the driver's persisted desired state to ON so
a session that died mid-secure-desktop can't leave the next one adopting
UNdeclared (the §8.6 cross-session composite trap); orderly teardown does
the same. The flip closure is built for every Windows session — a
channel-less session can reuse a driver monitor whose earlier-session
cursor worker still re-declares, and NOT_FOUND from never-declared targets
is ignored.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 14:13:15 +02:00
enricobuehlerandClaude Opus 4.8 588a7077e8 fix(host/gamepad): only degrade a virtual Steam pad on a same-PID conflict
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 1m0s
apple / swift (push) Successful in 1m26s
decky / build-publish (push) Successful in 32s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 23s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 20s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 26s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 16s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
ci / bench (push) Successful in 6m10s
apple / screenshots (push) Successful in 6m38s
deb / build-publish (push) Successful in 9m26s
android / android (push) Successful in 12m27s
deb / build-publish-host (push) Successful in 13m55s
windows-host / package (push) Successful in 16m20s
arch / build-publish (push) Successful in 17m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m2s
ci / rust (push) Successful in 27m20s
docker / deploy-docs (push) Successful in 26s
The Steam-conflict gate matched any physical Valve `28DE` HID device, so a
Steam Machine with a physical Steam Controller 2 (`28DE:1302`) plugged in
degraded a client's requested virtual Steam Deck (`28DE:1205`) to a DualSense
— the reporter's controller passthrough came up as the wrong pad.

Steam Input drives *distinct* Steam controllers side by side; only two
*identical* `28DE` devices confuse it (the Deck-as-host case: built-in
`28DE:1205` + a second virtual Deck, observed live on Bazzite). Narrow the
gate to key on the exact VID+PID instead of the `28DE` vendor alone:

- `steam_backend_product()` maps each virtual Steam backend to the PID it
  enumerates as (Deck 1205, classic SC 1102, SC2 1302, SC2 Puck 1304).
- `physical_steam_controller_present()` -> `physical_steam_product_present(pid)`
  matches `:28DE:{PID}`, keeping the FVPF-serial / `vhci_hcd` / `/virtual/`
  exclusions so our own virtual pads never count.
- `degrade_steam_on_conflict` degrades only on a same-PID duplicate and logs
  the conflicting identity. `PUNKTFUNK_STEAM_FORCE=1` override unchanged.

A physical SC2 no longer blocks a virtual Deck (and vice versa); a genuine
same-identity duplicate (physical Deck + virtual Deck, or physical SC2 +
SC2 passthrough) still degrades. Adds a unit test for the PID mapping.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 14:11:44 +02:00
enricobuehlerandClaude Fable 5 4714235fe6 feat(core): stylus wire P0 — state-full RICH_PEN batches, PenTracker, HOST_CAP_PEN
apple / swift (push) Successful in 1m24s
release / apple (push) Successful in 10m3s
android / android (push) Successful in 13m31s
arch / build-publish (push) Successful in 13m39s
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m12s
windows-host / package (push) Successful in 16m40s
apple / screenshots (push) Successful in 6m38s
ci / bench (push) Successful in 5m26s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m9s
ci / rust (push) Successful in 20m46s
decky / build-publish (push) Successful in 30s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m15s
deb / build-publish (push) Successful in 8m50s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 45s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m6s
deb / build-publish-host (push) Successful in 9m52s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m44s
docker / deploy-docs (push) Successful in 28s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 7m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m45s
flatpak / build-publish (push) Failing after 8m6s
The pen plane from design/pen-tablet-input.md, protocol side only (the P1 uinput
tablet injector will consume it): 0xCC kind 0x05 carries batches of state-full
PenSamples (pressure, polar tilt + azimuth, barrel roll, hover distance, eraser
tool, barrel buttons); PenTracker diffs samples into injector transitions so a
lost datagram self-heals; HOST_CAP_PEN (0x10) gates sending and stays
unadvertised until a real backend exists. C ABI: PunktfunkPenSample +
punktfunk_connection_send_pen, documented in docs/embedding-the-c-abi.md §8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 12:58:31 +02:00
91 changed files with 6623 additions and 601 deletions
+39
View File
@@ -0,0 +1,39 @@
# Announce a stable release to the Discord #releases channel.
#
# This is the deliberate "go" step for a release. Release notes live in the repo at
# docs/releases/<tag>.md and are seeded into the Gitea release body at creation by the build
# workflows (scripts/ci/gitea-release.sh), so the release is never noteless. Once every
# platform's CI is green for a tag, dispatch this workflow with that tag: it re-asserts the notes
# file over the live release and posts a formatted embed to #releases.
#
# Manual on purpose — pressing "go" is the quality gate that says "all platforms built, notes are
# final, tell the community." It is NOT wired to the tag push, so a half-built or failed release
# is never announced. Stable-only: a -rc/pre-release tag is refused unless allow_prerelease=true.
#
# Requires the repo secret DISCORD_RELEASE_WEBHOOK (the #releases channel webhook URL); GITEA auth
# reuses REGISTRY_TOKEN like the other release workflows.
name: announce
on:
workflow_dispatch:
inputs:
tag:
description: "Release tag to announce (e.g. v0.18.0)"
required: true
allow_prerelease:
description: "Announce even if the tag is a pre-release (-rc)"
required: false
default: "false"
jobs:
announce:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Post release announcement to Discord
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
DISCORD_RELEASE_WEBHOOK: ${{ secrets.DISCORD_RELEASE_WEBHOOK }}
ALLOW_PRERELEASE: ${{ inputs.allow_prerelease }}
run: bash scripts/ci/discord-announce.sh "${{ inputs.tag }}"
+70
View File
@@ -393,6 +393,76 @@ jobs:
-authenticationKeyID "${{ secrets.ASC_API_KEY_ID }}" \
-authenticationKeyIssuerID "${{ secrets.ASC_API_ISSUER_ID }}"
- name: iOS — export .ipa (Gitea release + run artifact)
# The TestFlight step above uploads straight to App Store Connect (destination=upload) and
# leaves NO .ipa on disk. Re-export the SAME archive with destination=export to get an
# App Store distribution-signed .ipa for the Gitea release + the run artifacts. Same gate as
# that archive; a warn+skip (never fails the best-effort iOS leg) if the archive is absent,
# e.g. a workflow_dispatch with testflight=false. NOTE: an App Store-signed .ipa installs
# only via TestFlight/App Store, not by direct sideload — it's a release/archival artifact.
if: gitea.event_name != 'workflow_dispatch' || inputs.testflight == 'true'
id: ios_ipa
run: |
ARCHIVE="$RUNNER_TEMP/Punktfunk-ios.xcarchive"
if [ ! -d "$ARCHIVE" ]; then
echo "::warning::iOS archive not found — skipping .ipa export"
exit 0
fi
PROFILE="Punktfunk iOS App Store Distribution"
WIDGET_PROFILE="Punktfunk iOS Widgets App Store Distribution"
# destination=export writes the .ipa to -exportPath; otherwise identical manual signing to
# the upload plist (both profiles, Apple Distribution). No ASC key needed — no network.
cat > "$RUNNER_TEMP/export-appstore-ipa.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>method</key><string>app-store-connect</string>
<key>destination</key><string>export</string>
<key>teamID</key><string>$TEAM_ID</string>
<key>signingStyle</key><string>manual</string>
<key>signingCertificate</key><string>Apple Distribution</string>
<key>provisioningProfiles</key>
<dict>
<key>io.unom.punktfunk</key><string>$PROFILE</string>
<key>io.unom.punktfunk.widgets</key><string>$WIDGET_PROFILE</string>
</dict>
</dict>
</plist>
EOF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild -exportArchive \
-archivePath "$ARCHIVE" \
-exportOptionsPlist "$RUNNER_TEMP/export-appstore-ipa.plist" \
-exportPath "$RUNNER_TEMP/export-ipa"
SRC=$(ls "$RUNNER_TEMP/export-ipa/"*.ipa 2>/dev/null | head -1)
[ -n "$SRC" ] || { echo "::warning::no .ipa was produced by export"; exit 0; }
mkdir -p "$GITHUB_WORKSPACE/dist"
IPA="$GITHUB_WORKSPACE/dist/Punktfunk-$VERSION.ipa"
mv "$SRC" "$IPA"
echo "IPA=$IPA" >> "$GITHUB_ENV"
echo "ipa=dist/Punktfunk-$VERSION.ipa" >> "$GITHUB_OUTPUT"
echo "exported $IPA"
- name: Attach .ipa to the workflow run
if: steps.ios_ipa.outputs.ipa != ''
# v3, not v4: Gitea's artifact backend identifies as GHES, which upload-artifact@v4 refuses
# (same reason as android.yml / apple.yml). Download is a zip of the .ipa.
uses: actions/upload-artifact@v3
with:
name: punktfunk-ios-ipa
path: ${{ steps.ios_ipa.outputs.ipa }}
if-no-files-found: warn
retention-days: 30
- name: Attach .ipa to the Gitea release (stable tags only)
if: startsWith(gitea.ref, 'refs/tags/v') && steps.ios_ipa.outputs.ipa != ''
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
. scripts/ci/gitea-release.sh
RID=$(ensure_release "$GITHUB_REF_NAME" "$GITHUB_REF_NAME" auto)
upsert_asset "$RID" "$IPA" "Punktfunk-$VERSION.ipa"
- name: tvOS — archive + upload to TestFlight
# Canary + stable, the same track as iOS/macOS — the tvOS xcframework slice is now built
# on every apple push (above), so this matches the iOS step's gate exactly.
Generated
+27 -27
View File
@@ -2194,7 +2194,7 @@ dependencies = [
[[package]]
name = "latency-probe"
version = "0.18.0"
version = "0.19.1"
[[package]]
name = "lazy_static"
@@ -2299,7 +2299,7 @@ dependencies = [
[[package]]
name = "libvpl-sys"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"bindgen",
"cmake",
@@ -2334,7 +2334,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]]
name = "loss-harness"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"punktfunk-core",
]
@@ -2823,7 +2823,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pf-capture"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"ashpd",
@@ -2844,7 +2844,7 @@ dependencies = [
[[package]]
name = "pf-client-core"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"ash",
@@ -2868,7 +2868,7 @@ dependencies = [
[[package]]
name = "pf-clipboard"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"ashpd",
@@ -2886,7 +2886,7 @@ dependencies = [
[[package]]
name = "pf-console-ui"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"ash",
@@ -2907,7 +2907,7 @@ dependencies = [
[[package]]
name = "pf-encode"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"ash",
@@ -2931,7 +2931,7 @@ dependencies = [
[[package]]
name = "pf-ffvk"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"ash",
"bindgen",
@@ -2940,7 +2940,7 @@ dependencies = [
[[package]]
name = "pf-frame"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"libc",
@@ -2952,7 +2952,7 @@ dependencies = [
[[package]]
name = "pf-gpu"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"pf-host-config",
@@ -2966,11 +2966,11 @@ dependencies = [
[[package]]
name = "pf-host-config"
version = "0.18.0"
version = "0.19.1"
[[package]]
name = "pf-inject"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"ashpd",
@@ -2998,14 +2998,14 @@ dependencies = [
[[package]]
name = "pf-paths"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"tracing",
]
[[package]]
name = "pf-presenter"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"ash",
@@ -3020,7 +3020,7 @@ dependencies = [
[[package]]
name = "pf-vdisplay"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"ashpd",
@@ -3050,7 +3050,7 @@ dependencies = [
[[package]]
name = "pf-win-display"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"pf-paths",
@@ -3062,7 +3062,7 @@ dependencies = [
[[package]]
name = "pf-zerocopy"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"ash",
@@ -3269,7 +3269,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-android"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"android_logger",
"jni",
@@ -3285,7 +3285,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-linux"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"async-channel",
@@ -3301,7 +3301,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-session"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"pf-client-core",
@@ -3316,7 +3316,7 @@ dependencies = [
[[package]]
name = "punktfunk-client-windows"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"async-channel",
"ffmpeg-next",
@@ -3335,7 +3335,7 @@ dependencies = [
[[package]]
name = "punktfunk-core"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"aes-gcm",
"bytes",
@@ -3367,7 +3367,7 @@ dependencies = [
[[package]]
name = "punktfunk-host"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"aes",
"aes-gcm",
@@ -3451,7 +3451,7 @@ dependencies = [
[[package]]
name = "punktfunk-probe"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"mdns-sd",
@@ -3465,7 +3465,7 @@ dependencies = [
[[package]]
name = "punktfunk-tray"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"anyhow",
"ksni",
@@ -3488,7 +3488,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]]
name = "pyrowave-sys"
version = "0.18.0"
version = "0.19.1"
dependencies = [
"bindgen",
"cmake",
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package]
version = "0.18.0"
version = "0.19.1"
edition = "2021"
rust-version = "1.82"
license = "MIT OR Apache-2.0"
@@ -355,11 +355,19 @@ class MainActivity : ComponentActivity() {
return true
}
when (event.keyCode) {
// A mouse back/forward button whose BUTTON_* press went unconsumed makes the
// framework synthesize a FALLBACK BACK — the button already went over the wire
// as X1/X2, and it must never yank the user out of the stream.
KeyEvent.KEYCODE_BACK ->
if (event.flags and KeyEvent.FLAG_FALLBACK != 0) return true
// A mouse's back/forward buttons already go over the wire as X1/X2 via their
// BUTTON_* motion edges — but Android ALSO delivers them as key events: the input
// reader synthesizes KEYCODE_BACK/FORWARD (stamped SOURCE_MOUSE) unconditionally,
// and a view-level FALLBACK BACK appears when the BUTTON_* press goes unconsumed.
// Swallow every such duplicate or it doubles as Android navigation and yanks the
// user out of the stream. A remote/keyboard BACK is never mouse-sourced, so it
// still falls through to the BackHandler and exits.
KeyEvent.KEYCODE_BACK, KeyEvent.KEYCODE_FORWARD ->
if (event.isFromSource(InputDevice.SOURCE_MOUSE) ||
event.flags and KeyEvent.FLAG_FALLBACK != 0
) {
return true
}
// Leave these to the system even while streaming.
// (BACK above → BackHandler leaves the stream.)
KeyEvent.KEYCODE_VOLUME_UP,
@@ -13,6 +13,7 @@ import android.net.wifi.WifiManager
import android.os.Build
import android.text.InputType
import android.util.Log
import android.view.KeyEvent
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.View
@@ -49,6 +50,9 @@ import androidx.core.content.ContextCompat
import androidx.core.view.WindowCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.LifecycleOwner
import io.unom.punktfunk.kit.GamepadFeedback
import io.unom.punktfunk.kit.GamepadRouter
import io.unom.punktfunk.kit.deviceBodyVibrator
@@ -383,6 +387,26 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
// Leaving the app (Home, task switch, screen off) MUST end the session. Android does not
// suspend a process for going to background, so without this the native worker kept running and
// its QUIC connection kept answering the host's keep-alives — the user was long gone but the
// host still saw a live client and held the session (and its display + encoder) open until the
// OS eventually reclaimed the process, which on a TV box is effectively never.
//
// Route it through `onDisconnect()` so the composable's `onDispose` above runs the one real
// teardown path. Deliberately NOT a `nativeDisconnectQuit`: backgrounding isn't a user "quit",
// so the host should linger the display and make coming straight back a fast reconnect.
DisposableEffect(handle) {
val lifecycle = (context as? LifecycleOwner)?.lifecycle
val obs = LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_STOP) {
onDisconnect()
}
}
lifecycle?.addObserver(obs)
onDispose { lifecycle?.removeObserver(obs) }
}
// Auto-engage pointer capture at stream start (setting on + a mouse actually present).
// Delayed a beat: the grab needs window focus and the capture view attached.
LaunchedEffect(handle) {
@@ -471,12 +495,22 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no
// keyboard gesture: its fingers belong to the host verbatim (a swipe there may BE a
// host-OS gesture), so intercepting three fingers would corrupt real multi-touch.
// Stylus lane (design/pen-tablet-input.md §7): against a HOST_CAP_PEN host a stylus
// splits out of BOTH touch models onto the pen plane; its heartbeat coroutine keeps a
// stationary held stroke alive (and its cancellation lifts everything on teardown).
val stylus = remember(handle) {
if (NativeBridge.nativeHostSupportsPen(handle)) StylusStream(handle) else null
}
if (stylus != null) {
LaunchedEffect(stylus) { stylus.heartbeatLoop() }
}
Box(
Modifier.fillMaxSize().pointerInput(handle, touchMode) {
when (touchMode) {
TouchMode.TOUCH -> streamTouchPassthrough(handle)
TouchMode.TOUCH -> streamTouchPassthrough(handle, stylus)
else -> streamTouchInput(
handle,
stylus,
trackpad = touchMode == TouchMode.TRACKPAD,
invertScroll = initialSettings.invertScroll,
onCycleStats = { statsVerbosity = statsVerbosity.next() },
@@ -549,9 +583,15 @@ private class KeyCaptureView(context: Context) : View(context) {
var imeShown = false
private set
override fun onCheckIsTextEditor(): Boolean = true
override fun onCheckIsTextEditor(): Boolean = imeShown
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection {
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection? {
// Only an editor while the user has SUMMONED the keyboard (gesture / remote toggle).
// This view holds focus for the whole stream (it's the capture anchor), and with an
// always-live editable connection the IME counts input as active on it — TV IMEs then
// pop their UI the moment a PHYSICAL keyboard key arrives. With no connection, hardware
// typing stays on the raw dispatchKeyEvent → Keymap → wire path and no keyboard appears.
if (!imeShown) return null
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or
EditorInfo.IME_FLAG_NO_FULLSCREEN or EditorInfo.IME_FLAG_NO_ENTER_ACTION
return if (textHandle != 0L) {
@@ -570,11 +610,29 @@ private class KeyCaptureView(context: Context) : View(context) {
imeShown = show
if (show) {
requestFocus()
// The view may already be focused from a null-connection state — restart so the
// framework re-queries onCreateInputConnection with the gate now open.
imm.restartInput(this)
imm.showSoftInput(this, 0)
} else {
imm.hideSoftInputFromWindow(windowToken, 0)
imm.restartInput(this) // gate closed — drop the editable connection
}
}
/**
* BACK while the summoned keyboard is up: the IME consumes it pre-IME to dismiss itself, so
* [setImeVisible] never hears about it — sync the gate here or a stale `imeShown` leaves the
* editable connection live and physical typing re-pops the keyboard.
*/
override fun onKeyPreIme(keyCode: Int, event: KeyEvent): Boolean {
if (keyCode == KeyEvent.KEYCODE_BACK && imeShown && event.action == KeyEvent.ACTION_UP) {
imeShown = false
(context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager)
?.restartInput(this)
}
return super.onKeyPreIme(keyCode, event)
}
}
/**
@@ -0,0 +1,195 @@
package io.unom.punktfunk
import android.view.MotionEvent
import androidx.compose.ui.ExperimentalComposeUiApi
import androidx.compose.ui.input.pointer.PointerEvent
import androidx.compose.ui.input.pointer.PointerType
import androidx.compose.ui.unit.IntSize
import io.unom.punktfunk.kit.NativeBridge
import kotlinx.coroutines.delay
// Wire PEN_* state bits (punktfunk_core::quic::pen; mirrored, asserted by the Rust shim's docs).
private const val PEN_IN_RANGE = 1f
private const val PEN_TOUCHING = 2f
private const val PEN_BARREL1 = 4f
private const val PEN_BARREL2 = 8f
private const val STRIDE = 10
private const val MAX_SAMPLES = 8
/**
* Android stylus → the state-full pen plane (design/pen-tablet-input.md §7): pressure, tilt
* (`AXIS_TILT`, radians from the surface normal), azimuth (`AXIS_ORIENTATION` — Android's 0 =
* "pointed away from the user" IS the wire's north, no offset needed), hover with
* `AXIS_DISTANCE`, the eraser tool, both stylus barrel buttons, and historical (coalesced)
* samples batched oldest-first for full capture-rate fidelity. Android has no barrel-roll
* axis — roll stays unknown on this client.
*
* Both touch loops call [intercept] first; stylus/eraser pointers are consumed here (against a
* pen-capable host) and never reach the finger paths, independent of the touch-input mode.
* [heartbeatLoop] implements the ≤100 ms keepalive wire contract: a stationary held stylus is
* silent in Android's input pipeline, and the host force-releases a stroke after 200 ms
* without samples.
*/
internal class StylusStream(private val handle: Long) {
private var inRange = false
private var touching = false
private var sawHover = false
private val last = FloatArray(STRIDE)
private val batch = FloatArray(MAX_SAMPLES * STRIDE)
init {
idle(last)
}
/**
* Consume the event's stylus pointers into pen samples. Returns true when this event
* carried any (the caller's finger/gesture handling must then skip those changes).
*/
@OptIn(ExperimentalComposeUiApi::class)
fun intercept(ev: PointerEvent, size: IntSize): Boolean {
val stylusChanges = ev.changes.filter {
it.type == PointerType.Stylus || it.type == PointerType.Eraser
}
if (stylusChanges.isEmpty()) return false
stylusChanges.forEach { it.consume() }
val me = ev.motionEvent ?: return true
if (size.width <= 0 || size.height <= 0) return true
// At most one stylus exists — find its pointer index by tool type.
val idx = (0 until me.pointerCount).firstOrNull {
me.getToolType(it) == MotionEvent.TOOL_TYPE_STYLUS ||
me.getToolType(it) == MotionEvent.TOOL_TYPE_ERASER
} ?: return true
when (me.actionMasked) {
MotionEvent.ACTION_DOWN, MotionEvent.ACTION_POINTER_DOWN,
MotionEvent.ACTION_MOVE,
-> {
touching = true
inRange = true
emitSamples(me, idx, size)
}
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_MOVE -> {
sawHover = true
inRange = true
touching = false
emitSamples(me, idx, size)
}
MotionEvent.ACTION_UP, MotionEvent.ACTION_POINTER_UP -> {
touching = false
// Hover-capable hardware keeps proximity (HOVER_EXIT owns the leave);
// anything else leaves range on lift — the host never parks a phantom pen.
inRange = sawHover
emitSamples(me, idx, size)
}
MotionEvent.ACTION_HOVER_EXIT, MotionEvent.ACTION_CANCEL -> release()
else -> {}
}
return true
}
/** Session/composition teardown: leave range so the host lifts anything still inked. */
fun reset() {
if (inRange || touching) release()
sawHover = false
}
/** The ≤100 ms keepalive (80 ms leaves headroom for one lost datagram). Runs until
* cancelled; resends the last state-full sample while the pen is in range. */
suspend fun heartbeatLoop() {
try {
while (true) {
delay(80)
if (inRange || touching) {
last[9] = 0f // dt
NativeBridge.nativeSendPen(handle, last, 1)
}
}
} finally {
reset()
}
}
private fun release() {
touching = false
inRange = false
last[0] = 0f // state: out of range
last[4] = 0f // pressure
NativeBridge.nativeSendPen(handle, last, 1)
}
/** Historical (coalesced) samples oldest-first, then the current one — a single batch. */
private fun emitSamples(me: MotionEvent, idx: Int, size: IntSize) {
val history = minOf(me.historySize, MAX_SAMPLES - 1)
var count = 0
var prevT = if (history > 0) me.getHistoricalEventTime(0) else me.eventTime
for (h in (me.historySize - history) until me.historySize) {
val t = me.getHistoricalEventTime(h)
fill(
batch, count * STRIDE, size,
x = me.getHistoricalX(idx, h), y = me.getHistoricalY(idx, h),
pressure = me.getHistoricalPressure(idx, h),
tiltRad = me.getHistoricalAxisValue(MotionEvent.AXIS_TILT, idx, h),
orientRad = me.getHistoricalAxisValue(MotionEvent.AXIS_ORIENTATION, idx, h),
distance = me.getHistoricalAxisValue(MotionEvent.AXIS_DISTANCE, idx, h),
buttons = me.buttonState, tool = me.getToolType(idx),
dtUs = ((t - prevT) * 1000).coerceIn(0, 65535).toFloat(),
)
prevT = t
count++
}
fill(
batch, count * STRIDE, size,
x = me.getX(idx), y = me.getY(idx), pressure = me.getPressure(idx),
tiltRad = me.getAxisValue(MotionEvent.AXIS_TILT, idx),
orientRad = me.getAxisValue(MotionEvent.AXIS_ORIENTATION, idx),
distance = me.getAxisValue(MotionEvent.AXIS_DISTANCE, idx),
buttons = me.buttonState, tool = me.getToolType(idx),
dtUs = ((me.eventTime - prevT) * 1000).coerceIn(0, 65535).toFloat(),
)
count++
batch.copyInto(last, 0, (count - 1) * STRIDE, count * STRIDE)
NativeBridge.nativeSendPen(handle, batch, count)
}
private fun fill(
out: FloatArray,
off: Int,
size: IntSize,
x: Float,
y: Float,
pressure: Float,
tiltRad: Float,
orientRad: Float,
distance: Float,
buttons: Int,
tool: Int,
dtUs: Float,
) {
var state = 0f
if (inRange || touching) state += PEN_IN_RANGE
if (touching) state += PEN_TOUCHING
if (buttons and MotionEvent.BUTTON_STYLUS_PRIMARY != 0) state += PEN_BARREL1
if (buttons and MotionEvent.BUTTON_STYLUS_SECONDARY != 0) state += PEN_BARREL2
out[off + 0] = state
out[off + 1] = if (tool == MotionEvent.TOOL_TYPE_ERASER) 1f else 0f
out[off + 2] = (x / (size.width - 1).coerceAtLeast(1)).coerceIn(0f, 1f)
out[off + 3] = (y / (size.height - 1).coerceAtLeast(1)).coerceIn(0f, 1f)
out[off + 4] = if (touching) pressure.coerceIn(0f, 1f) else 0f
// AXIS_DISTANCE units are device-arbitrary; 0..1 covers real hardware, and 0 while
// hovering legitimately means "at the hover floor".
out[off + 5] = if (touching) 0f else distance.coerceIn(0f, 1f)
out[off + 6] = Math.toDegrees(tiltRad.toDouble()).toFloat().coerceIn(0f, 90f)
// AXIS_ORIENTATION: 0 = pointed away from the user (= wire north), clockwise, −π..π.
out[off + 7] = ((Math.toDegrees(orientRad.toDouble()) + 360.0) % 360.0).toFloat()
out[off + 8] = -1f // no barrel-roll axis on Android
out[off + 9] = dtUs
}
private fun idle(out: FloatArray) {
out.fill(0f)
out[5] = -1f // distance unknown
out[6] = -1f // tilt unknown
out[7] = -1f // azimuth unknown
out[8] = -1f // roll unknown
}
}
@@ -1,9 +1,11 @@
package io.unom.punktfunk
import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.ui.input.pointer.AwaitPointerEventScope
import androidx.compose.ui.input.pointer.PointerId
import androidx.compose.ui.input.pointer.PointerInputChange
import androidx.compose.ui.input.pointer.PointerInputScope
import androidx.compose.ui.input.pointer.PointerType
import androidx.compose.ui.input.pointer.changedToDownIgnoreConsumed
import androidx.compose.ui.input.pointer.changedToUpIgnoreConsumed
import androidx.compose.ui.input.pointer.positionChanged
@@ -56,7 +58,26 @@ private const val ACCEL_MAX = 3.0f
* normalizes and maps into the output). On teardown (stream leaves composition) every still-held
* contact is lifted so nothing stays stuck on the host.
*/
internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
/** Whether this change belongs to the stylus lane (only when a pen-capable host is live). */
private fun isStylus(c: PointerInputChange, stylus: StylusStream?): Boolean =
stylus != null && (c.type == PointerType.Stylus || c.type == PointerType.Eraser)
/** [awaitFirstDown] with the stylus lane split out: pen events feed [stylus] and never start a
* mouse/touch gesture. Toward a pen-less host ([stylus] == null) a stylus stays a finger. */
private suspend fun AwaitPointerEventScope.awaitFirstFingerDown(
stylus: StylusStream?,
): PointerInputChange {
while (true) {
val ev = awaitPointerEvent()
stylus?.intercept(ev, size)
val down = ev.changes.firstOrNull {
it.changedToDownIgnoreConsumed() && !isStylus(it, stylus)
}
if (down != null) return down
}
}
internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long, stylus: StylusStream?) {
val ids = mutableMapOf<PointerId, Int>()
fun alloc(p: PointerId): Int {
var id = 0
@@ -68,10 +89,12 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
awaitPointerEventScope {
while (true) {
val ev = awaitPointerEvent()
stylus?.intercept(ev, size)
val sw = size.width
val sh = size.height
if (sw <= 0 || sh <= 0) continue
for (c in ev.changes) {
if (isStylus(c, stylus)) continue // the pen plane owns it
val x = c.position.x.roundToInt().coerceIn(0, sw - 1)
val y = c.position.y.roundToInt().coerceIn(0, sh - 1)
when {
@@ -98,6 +121,7 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
internal suspend fun PointerInputScope.streamTouchInput(
handle: Long,
stylus: StylusStream?,
trackpad: Boolean,
invertScroll: Boolean,
onCycleStats: () -> Unit,
@@ -120,7 +144,7 @@ internal suspend fun PointerInputScope.streamTouchInput(
)
}
awaitEachGesture {
val down = awaitFirstDown(requireUnconsumed = false)
val down = awaitFirstFingerDown(stylus)
val startX = down.position.x
val startY = down.position.y
// A touch landing just after a quick tap nearby = tap-and-drag: hold the left
@@ -157,7 +181,8 @@ internal suspend fun PointerInputScope.streamTouchInput(
while (true) {
val ev = awaitPointerEvent()
val pressed = ev.changes.filter { it.pressed }
stylus?.intercept(ev, size)
val pressed = ev.changes.filter { it.pressed && !isStylus(it, stylus) }
if (pressed.isEmpty()) {
upTime = ev.changes.firstOrNull()?.uptimeMillis ?: upTime
break
@@ -287,6 +287,22 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
/**
* Whether the host advertised full-fidelity stylus injection (`HOST_CAP_PEN`) — the gate
* for splitting stylus pointers out of the touch path onto the pen plane. False on `0`.
*/
external fun nativeHostSupportsPen(handle: Long): Boolean
/**
* One stylus batch of STATE-FULL samples (the pen plane; design/pen-tablet-input.md §7):
* [count] × 10 floats, oldest first — `[state, tool, x, y, pressure, distance, tilt_deg,
* azimuth_deg, roll_deg, dt_us]`. `state` = the wire in-range/touching/barrel bits; `tool`
* 0=pen 1=eraser; x/y/pressure/distance normalized 0..1; distance/tilt/azimuth/roll < 0 =
* unknown. Send only when [nativeHostSupportsPen]; repeat the last sample ≤100 ms while the
* pen is in range (the host force-releases a silent stroke after 200 ms).
*/
external fun nativeSendPen(handle: Long, samples: FloatArray, count: Int)
/**
* Whether the host advertised committed-text injection (`HOST_CAP_TEXT_INPUT`) — its inject
* backend can type Unicode text directly. Picks the real IME `InputConnection` (autocorrect,
+92 -2
View File
@@ -6,11 +6,14 @@
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::{JByteBuffer, JObject, JString};
use jni::objects::{JByteBuffer, JFloatArray, JObject, JString};
use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX, HOST_CAP_TEXT_INPUT};
use punktfunk_core::quic::{
PenSample, PenTool, RichInput, HID_REPORT_MAX, HOST_CAP_PEN, HOST_CAP_TEXT_INPUT,
PEN_ANGLE_UNKNOWN, PEN_BATCH_MAX, PEN_DISTANCE_UNKNOWN, PEN_TILT_UNKNOWN,
};
use super::SessionHandle;
@@ -162,6 +165,93 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSu
u8::from(h.client.host_caps() & HOST_CAP_TEXT_INPUT != 0)
}
/// `NativeBridge.nativeHostSupportsPen(handle)` — the host advertised `HOST_CAP_PEN`, so the
/// Kotlin side splits stylus pointers out of the touch path onto the pen plane
/// (design/pen-tablet-input.md §7). `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeHostSupportsPen(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_PEN != 0)
}
/// Floats per sample in the `nativeSendPen` flat array.
const PEN_JNI_STRIDE: usize = 10;
/// `NativeBridge.nativeSendPen(handle, samples, count)` — one stylus batch of STATE-FULL
/// samples, `count` × [`PEN_JNI_STRIDE`] floats, oldest first:
/// `[state, tool, x, y, pressure, distance, tilt_deg, azimuth_deg, roll_deg, dt_us]`.
/// `state` = the wire `PEN_*` bits; `tool` 0=pen 1=eraser; `x`/`y`/`pressure`/`distance`
/// normalized 0..1; `distance`/`tilt_deg`/`azimuth_deg`/`roll_deg` < 0 = unknown. Call only
/// against a [`nativeHostSupportsPen`] host; the client heartbeats the last sample ≤100 ms
/// while in range (Kotlin side — see `StylusStream`).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPen(
env: JNIEnv,
_this: JObject,
handle: jlong,
samples: JFloatArray,
count: jint,
) {
if handle == 0 || count <= 0 {
return;
}
let count = (count as usize).min(PEN_BATCH_MAX);
let mut buf = [0f32; PEN_BATCH_MAX * PEN_JNI_STRIDE];
let flat = &mut buf[..count * PEN_JNI_STRIDE];
if env.get_float_array_region(&samples, 0, flat).is_err() {
return; // short array — a bridge bug, never worth a crash on the input path
}
let mut batch = [PenSample::default(); PEN_BATCH_MAX];
for (slot, s) in batch.iter_mut().zip(flat.chunks_exact(PEN_JNI_STRIDE)) {
if !s[2].is_finite() || !s[3].is_finite() {
return; // never forward a NaN coordinate
}
*slot = PenSample {
state: s[0] as u8,
tool: if s[1] as u8 == 1 {
PenTool::Eraser
} else {
PenTool::Pen
},
x: s[2].clamp(0.0, 1.0),
y: s[3].clamp(0.0, 1.0),
pressure: (s[4].clamp(0.0, 1.0) * 65535.0) as u16,
distance: if s[5] < 0.0 {
PEN_DISTANCE_UNKNOWN
} else {
(s[5].clamp(0.0, 1.0) * 65534.0) as u16
},
tilt_deg: if s[6] < 0.0 {
PEN_TILT_UNKNOWN
} else {
(s[6].clamp(0.0, 90.0)) as u8
},
azimuth_deg: if s[7] < 0.0 {
PEN_ANGLE_UNKNOWN
} else {
(s[7] as u16) % 360
},
roll_deg: if s[8] < 0.0 {
PEN_ANGLE_UNKNOWN
} else {
(s[8] as u16) % 360
},
dt_us: s[9].clamp(0.0, 65535.0) as u16,
};
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_pen is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_pen(&batch[..count]);
}
/// `NativeBridge.nativeSendText(handle, text)` — committed IME text, one `TextInput` event per
/// Unicode scalar (`code` = the scalar; multi-char commits are consecutive events in order).
/// Control characters are skipped — Enter/Backspace/Tab ride the VK key path. Call only when
@@ -144,14 +144,26 @@ struct ContentView: View {
// tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a
// parallel session this drives the one `model` ContentView owns.
.onOpenURL { handleDeepLink($0) }
#if os(iOS)
// Background keep-alive driver (opt-in). Only .background/.active matter; .inactive (a
// transient peek) is ignored so the disconnect timer never starts for a Control-Center pull.
#if os(iOS) || os(tvOS)
// Backgrounding driver. Only .background/.active matter; .inactive (a transient peek) is
// ignored so neither branch fires for a Control-Center pull.
//
// Backgrounding MUST end the session one way or the other: the app keeps running while
// streaming (the `audio` background mode plus a live audio session), so its QUIC connection
// keeps answering the host's keep-alives with the user long gone the host has no way to
// tell that apart from someone watching, and the session survived indefinitely. Either hold
// it under the opt-in keep-alive (bounded by that path's own auto-disconnect timer) or end
// it here.
.onChange(of: scenePhase) { _, phase in
switch phase {
case .background:
if backgroundKeepAlive, model.phase == .streaming {
guard model.phase == .streaming else { break }
if backgroundKeepAlive {
model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes)
} else {
// Not deliberate: the user may come straight back, so let the host linger the
// display for a fast reconnect instead of tearing it down.
model.disconnect(deliberate: false)
}
case .active:
model.exitBackground()
@@ -159,7 +171,11 @@ struct ContentView: View {
break
}
}
// Live Activity lifecycle, driven from the model's published state.
#endif
#if os(iOS)
// Live Activity lifecycle, driven from the model's published state. iPhone/iPad only
// ActivityKit (and so `liveActivity`) does not exist on tvOS, which is why this stays in its
// own os(iOS) block rather than riding the backgrounding driver's.
.onChange(of: model.phase) { _, phase in
switch phase {
case .streaming:
@@ -0,0 +1,97 @@
// Keeps the local display awake for the duration of a streaming session.
//
// A stream is not "user activity" to the OS: the pixels arrive over the network and the input that
// drives them is often a game controller, which does NOT feed the HID idle timer on any Apple
// platform. So a controller-only session reliably idles the panel out from under the user the
// same reason the Android client holds FLAG_KEEP_SCREEN_ON while streaming (StreamScreen.kt).
//
// Held by SessionModel from `beginStreaming` to `disconnect`, so it is scoped to the session and
// never leaks past it (including a host-ended or timed-out background session, which both land in
// `disconnect`).
import Foundation
#if os(macOS)
import IOKit.pwr_mgt
#else
import UIKit
#endif
@MainActor
final class DisplaySleepGuard {
#if os(macOS)
/// The `beginActivity` token; non-nil exactly while held.
private var activity: NSObjectProtocol?
/// Re-used across heartbeats so the whole session shares one assertion instead of
/// accumulating one per tick.
private var userActivityAssertion: IOPMAssertionID = IOPMAssertionID(0)
private var heartbeat: Timer?
/// The power assertion defers DISPLAY SLEEP but not the screen saver that runs off the
/// HID idle timer, which a controller-only session never touches. Declaring user activity
/// on an interval well under the shortest selectable screen-saver delay (1 minute) keeps
/// that timer from ever reaching it. Side effect, and the intended one: an idle-lock
/// configured to follow the screen saver is deferred too, for the session only.
private static let heartbeatInterval: TimeInterval = 30
#endif
private(set) var isHeld = false
/// Idempotent a second acquire while held is a no-op.
func acquire() {
guard !isHeld else { return }
isHeld = true
#if os(macOS)
// The high-level Foundation API over IOKit power assertions: `.idleDisplaySleepDisabled`
// is the panel, `.userInitiated` also holds off idle SYSTEM sleep and sudden termination
// for a session the user is watching in real time.
activity = ProcessInfo.processInfo.beginActivity(
options: [.userInitiated, .idleDisplaySleepDisabled],
reason: "Punktfunk streaming session")
declareUserActivity()
let timer = Timer.scheduledTimer(withTimeInterval: Self.heartbeatInterval, repeats: true) {
[weak self] _ in
MainActor.assumeIsolated { self?.declareUserActivity() }
}
// The stream runs under a tracking run-loop mode while a menu or a window resize is up;
// .common keeps the heartbeat ticking through those.
RunLoop.main.add(timer, forMode: .common)
heartbeat = timer
#else
// iOS/iPadOS/tvOS: app-wide, and ignored while backgrounded the background keep-alive
// (audio-only, video dropped) correctly lets the device sleep without touching this.
UIApplication.shared.isIdleTimerDisabled = true
#endif
}
/// Idempotent safe to call when not held (`disconnect` runs on paths that never streamed).
func release() {
guard isHeld else { return }
isHeld = false
#if os(macOS)
heartbeat?.invalidate()
heartbeat = nil
if let activity {
ProcessInfo.processInfo.endActivity(activity)
self.activity = nil
}
if userActivityAssertion != IOPMAssertionID(0) {
IOPMAssertionRelease(userActivityAssertion)
userActivityAssertion = IOPMAssertionID(0)
}
#else
UIApplication.shared.isIdleTimerDisabled = false
#endif
}
#if os(macOS)
/// Resets the HID idle timer (see `heartbeatInterval`). `kIOPMUserActiveLocal` = activity at
/// this Mac's own display, which is what a stream being watched here is.
private func declareUserActivity() {
IOPMAssertionDeclareUserActivity(
"Punktfunk streaming session" as CFString,
kIOPMUserActiveLocal,
&userActivityAssertion)
}
#endif
}
@@ -196,6 +196,11 @@ final class SessionModel: ObservableObject {
/// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`.
private var backgroundTimer: DispatchSourceTimer?
/// Holds off display sleep (and, on macOS, the screen saver) for the life of a session
/// nothing about watching a stream looks like user activity to the OS, least of all a
/// controller-only session. Acquired in `beginStreaming`, released in `disconnect`.
private let displaySleepGuard = DisplaySleepGuard()
/// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true
/// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host
/// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair`
@@ -455,6 +460,8 @@ final class SessionModel: ObservableObject {
func disconnect(deliberate: Bool = true) {
statsTimer?.invalidate()
statsTimer = nil
// No-op when this session never reached `.streaming` (a refused/aborted connect).
displaySleepGuard.release()
// Drop any armed background keep-alive (incl. the timeout that just fired us).
backgroundTimer?.cancel()
backgroundTimer = nil
@@ -550,6 +557,7 @@ final class SessionModel: ObservableObject {
// Input capture itself is owned by StreamView (engaged by the captureEnabled
// flip this phase change causes, released/re-engaged by the user from there).
phase = .streaming
displaySleepGuard.acquire()
// Audio starts with streaming, not during the trust prompt no host sound (or
// mic uplink!) before the user trusted the host. Devices come from Settings;
// "" = system default.
@@ -387,8 +387,16 @@ public final class PunktfunkConnection {
/// The host answered `HOST_CAP_CURSOR`: it stopped compositing the pointer and forwards
/// shape/state on the cursor planes the client MUST draw the cursor locally.
/// `0x08` the bit moved when `HOST_CAP_TEXT_INPUT` claimed `0x04` on main; testing the
/// old bit would mistake a text-input-capable host (e.g. Windows) for a cursor grant.
public var hostSupportsCursor: Bool {
hostCaps & 0x04 != 0
hostCaps & 0x08 != 0
}
/// The host injects full-fidelity stylus input (`HOST_CAP_PEN`) the gate for splitting
/// Apple Pencil out of the touch path onto the pen plane (``sendPen(_:)``).
public var hostSupportsPen: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_PEN) != 0
}
/// One forwarded host-cursor shape (the cursor channel, ABI v11): straight-alpha RGBA,
@@ -1133,6 +1141,19 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_send_input(h, &ev)
}
/// Send one stylus sample batch ( `PUNKTFUNK_PEN_BATCH_MAX`, oldest first) on the pen
/// plane. Gate on ``hostSupportsPen`` the core refuses toward a host without the cap.
/// Thread-safe; silently dropped after close (input is lossy by design).
public func sendPen(_ samples: [PunktfunkPenSample]) {
guard !samples.isEmpty else { return }
abiLock.lock()
defer { abiLock.unlock() }
guard let h = handle, !closeRequested else { return }
samples.withUnsafeBufferPointer { buf in
_ = punktfunk_connection_send_pen(h, buf.baseAddress, UInt32(buf.count))
}
}
/// Signal a **deliberate** user-initiated quit before ``close()``: the connection closes with
/// `QUIT_CLOSE_CODE` (81) so the host tears the session down immediately instead of holding the
/// keep-alive linger for a reconnect. Call only from an explicit "Disconnect" action NOT from a
@@ -0,0 +1,268 @@
// Apple Pencil state-full wire pen samples (design/pen-tablet-input.md §7).
//
// Every sample carries the COMPLETE pen state (in-range/touching/buttons + all axes) the
// host diffs consecutive samples and synthesizes down/up/button transitions itself, so a lost
// datagram self-heals and this file never sends edge events. Three sources feed one stream:
// UITouch contacts (with coalesced samples for full 240 Hz fidelity), the hover gesture
// (zOffset > 0 distinguishes a hovering Pencil from a trackpad pointer), and
// UIPencilInteraction (squeeze held barrel 1, double-tap a momentary barrel 2
// Apple Pencil has no hardware eraser end or barrel buttons, so these mappings are how
// host-side apps get their stylus button/eraser affordances).
//
// HEARTBEAT (wire contract see `PunktfunkPenSample` in punktfunk_core.h): while the pen is
// in range or touching, the last sample repeats every 100 ms even when nothing changed.
// UIKit is silent for a stationary Pencil, and the host force-releases the stroke after
// 200 ms without samples (its dead-client failsafe) the timer keeps a held stroke alive.
#if os(iOS)
import PunktfunkCore
import UIKit
final class PencilStream: NSObject, UIPencilInteractionDelegate {
enum Phase { case down, move, up, cancel }
/// One assembled batch ( `PUNKTFUNK_PEN_BATCH_MAX` samples) ready for the connection.
var send: (([PunktfunkPenSample]) -> Void)?
/// View-space point normalized [0,1] video coordinates (the letterbox mapping the
/// touch path already uses). nil until a mode is negotiated samples are dropped then.
var videoNorm: ((CGPoint) -> (Float, Float)?)?
private var inRange = false
private var touching = false
/// Squeeze held (mapped to wire BARREL1).
private var squeezeHeld = false
/// Whether this device/Pencil pair has demonstrated hover decides what a lift means:
/// hover-capable hardware keeps proximity (the hover recognizer owns the exit), anything
/// else leaves range on lift so the host never parks a phantom hovering pen.
private var sawHover = false
/// A hover gesture is live right now (routes ended-state hover callbacks to us even when
/// the recognizer's final zOffset reads 0).
private(set) var hoverActive = false
private var last = PencilStream.idleSample()
private var heartbeat: Timer?
// MARK: - Contact path (UITouch, `.pencil` only)
func touches(_ touches: Set<UITouch>, event: UIEvent?, phase: Phase, in view: UIView) {
// At most one Pencil exists; a set with several is UIKit batching phases of the same
// stylus the last one carries the freshest state.
guard let touch = touches.max(by: { $0.timestamp < $1.timestamp }) else { return }
switch phase {
case .down, .move:
touching = true
inRange = true
// Coalesced samples restore the Pencil's full capture rate (UIKit delivers at
// display cadence); oldest first, `dt_us` preserving their spacing.
let raw = event?.coalescedTouches(for: touch) ?? [touch]
var batch: [PunktfunkPenSample] = []
var prevTs: TimeInterval?
for t in raw.suffix(Int(PUNKTFUNK_PEN_BATCH_MAX)) {
guard let s = contactSample(t, in: view, prevTs: prevTs) else { continue }
prevTs = t.timestamp
batch.append(s)
}
emit(batch)
case .up:
touching = false
// Hover-capable hardware: lift back to hover, the recognizer exits range later.
// Otherwise a lift IS the range exit (mirror of the host's GameStream heuristic).
inRange = sawHover
var s = last
s.pressure = 0
s.state = stateBits()
if let posSample = contactSample(touch, in: view, prevTs: nil) {
s.x = posSample.x
s.y = posSample.y
}
emit([s])
case .cancel:
release()
}
}
// MARK: - Hover path (forwarded from the view's hover recognizer)
/// Returns whether the event was consumed as Pencil hover; `false` hands it back to the
/// pointer path. A hovering Pencil reports `zOffset > 0`; trackpad/mouse hover is 0.
func maybeHover(_ r: UIHoverGestureRecognizer, in view: UIView) -> Bool {
switch r.state {
case .began, .changed:
guard r.zOffset > 0 || hoverActive else { return false }
hoverActive = true
sawHover = true
inRange = true
touching = false
guard let (x, y) = videoNorm?(r.location(in: view)) else { return true }
var s = PencilStream.idleSample()
s.state = stateBits()
s.x = x
s.y = y
s.distance = UInt16((r.zOffset.clamped(to: 0...1) * 65534).rounded())
s.tilt_deg = Self.tiltDeg(altitude: r.altitudeAngle)
s.azimuth_deg = Self.azimuthDeg(r.azimuthAngle(in: view))
if #available(iOS 17.5, *) { s.roll_deg = Self.rollDeg(r.rollAngle) }
emit([s])
return true
case .ended, .cancelled, .failed:
guard hoverActive else { return false }
hoverActive = false
if !touching { release() }
return true
default:
return hoverActive
}
}
// MARK: - UIPencilInteractionDelegate (squeeze barrel 1 held, tap barrel 2 click)
@available(iOS 17.5, *)
func pencilInteraction(
_ interaction: UIPencilInteraction, didReceiveSqueeze squeeze: UIPencilInteraction.Squeeze
) {
switch squeeze.phase {
case .began:
squeezeHeld = true
case .ended, .cancelled:
squeezeHeld = false
default:
return
}
guard inRange || touching else { return }
var s = last
s.state = stateBits()
emit([s])
}
func pencilInteractionDidTap(_ interaction: UIPencilInteraction) {
guard inRange || touching else { return }
// A momentary barrel-2 click: press + release as two state-full samples in ONE batch
// the host's tracker emits the button press and release in order.
var press = last
press.state = stateBits() | UInt8(PUNKTFUNK_PEN_BARREL2)
var releaseS = last
releaseS.state = stateBits()
emit([press, releaseS])
}
// MARK: - Lifecycle
/// Session stop / view teardown: leave range so the host lifts anything held.
func reset() {
if inRange || touching { release() }
sawHover = false
hoverActive = false
squeezeHeld = false
}
private func release() {
touching = false
inRange = false
var s = last
s.pressure = 0
s.state = 0
emit([s])
}
// MARK: - Sample assembly
private func contactSample(
_ t: UITouch, in view: UIView, prevTs: TimeInterval?
) -> PunktfunkPenSample? {
guard let (x, y) = videoNorm?(t.location(in: view)) else { return nil }
var s = PencilStream.idleSample()
s.state = stateBits()
s.x = x
s.y = y
// maximumPossibleForce is 0 until the system knows the stylus full force then
// (binary-stylus semantics, matching the host's unknown-pressure rule).
let maxForce = t.maximumPossibleForce
s.pressure =
maxForce > 0
? UInt16((Double(t.force / maxForce).clamped(to: 0...1) * 65535).rounded())
: UInt16.max
s.distance = 0
s.tilt_deg = Self.tiltDeg(altitude: t.altitudeAngle)
s.azimuth_deg = Self.azimuthDeg(t.azimuthAngle(in: view))
if #available(iOS 17.5, *) { s.roll_deg = Self.rollDeg(t.rollAngle) }
if let prevTs {
s.dt_us = UInt16(((t.timestamp - prevTs) * 1_000_000).clamped(to: 0...65535))
}
return s
}
private func stateBits() -> UInt8 {
var bits: UInt8 = 0
if inRange || touching { bits |= UInt8(PUNKTFUNK_PEN_IN_RANGE) }
if touching { bits |= UInt8(PUNKTFUNK_PEN_TOUCHING) }
if squeezeHeld { bits |= UInt8(PUNKTFUNK_PEN_BARREL1) }
return bits
}
private func emit(_ batch: [PunktfunkPenSample]) {
guard !batch.isEmpty else { return }
last = batch[batch.count - 1]
last.dt_us = 0
send?(batch)
armHeartbeat()
}
/// The 100 ms keepalive while in range (see the file header). 80 ms leaves headroom
/// under the host's 200 ms failsafe even with one lost datagram.
private func armHeartbeat() {
heartbeat?.invalidate()
guard inRange || touching else {
heartbeat = nil
return
}
heartbeat = Timer.scheduledTimer(withTimeInterval: 0.08, repeats: true) {
[weak self] _ in
guard let self, self.inRange || self.touching else {
self?.heartbeat?.invalidate()
self?.heartbeat = nil
return
}
self.send?([self.last])
}
}
// MARK: - Angle conversions
/// Altitude (π/2 = perpendicular) wire tilt-from-normal in degrees, 0...90.
private static func tiltDeg(altitude: CGFloat) -> UInt8 {
UInt8((90 - altitude * 180 / .pi).rounded().clamped(to: 0...90))
}
/// Apple azimuth (0 along the view's +x axis, clockwise, y-down) wire azimuth
/// (0 = north/up on screen, clockwise): +90° offset.
private static func azimuthDeg(_ apple: CGFloat) -> UInt16 {
let deg = (apple * 180 / .pi + 90).truncatingRemainder(dividingBy: 360)
return UInt16((deg + 360).truncatingRemainder(dividingBy: 360).rounded()) % 360
}
/// Pencil Pro roll (radians, π...π) wire barrel roll 0...359°.
private static func rollDeg(_ roll: CGFloat) -> UInt16 {
let deg = (roll * 180 / .pi).truncatingRemainder(dividingBy: 360)
return UInt16(((deg + 360).truncatingRemainder(dividingBy: 360)).rounded()) % 360
}
/// All-unknown baseline: sentinel angles/distance, tool = pen (the eraser is host-side
/// state driven by the squeeze/tap mappings, not a hardware end).
private static func idleSample() -> PunktfunkPenSample {
PunktfunkPenSample(
x: 0, y: 0, pressure: 0,
distance: UInt16(PUNKTFUNK_PEN_DISTANCE_UNKNOWN),
azimuth_deg: UInt16(PUNKTFUNK_PEN_ANGLE_UNKNOWN),
roll_deg: UInt16(PUNKTFUNK_PEN_ANGLE_UNKNOWN),
dt_us: 0, state: 0,
tool: UInt8(PUNKTFUNK_PEN_TOOL_PEN),
tilt_deg: UInt8(PUNKTFUNK_PEN_TILT_UNKNOWN),
_reserved: (0, 0, 0))
}
}
extension Comparable {
fileprivate func clamped(to range: ClosedRange<Self>) -> Self {
min(max(self, range.lowerBound), range.upperBound)
}
}
#endif
@@ -223,7 +223,19 @@ public final class StreamLayerView: NSView {
/// when the Welcome carried `HOST_CAP_CURSOR` (only sessions that advertised the client
/// cap get it). Shapes cache by serial; state is latest-wins. Main-thread only.
private var cursorChannelActive = false
private var hostCursors: [UInt32: NSCursor] = [:]
/// A forwarded host cursor shape, cached RAW (not as a finished `NSCursor`) so the pointer can be
/// (re)built at the CURRENT video-fit scale see `scaledCursor`. The host forwards the bitmap in
/// host FRAMEBUFFER pixels, whose size tracks the host's display scaling (32 px at 100%, 96 px at
/// 300% DPI); scaling by the video fit keeps the pointer sized to the streamed desktop at any host
/// scaling instead of ballooning on a high-DPI host.
private struct HostCursorShape {
let cg: CGImage
let width: Int
let height: Int
let hotX: Int
let hotY: Int
}
private var hostCursors: [UInt32: HostCursorShape] = [:]
private var cursorState: PunktfunkConnection.CursorStateEvent?
/// Last `CursorRenderMode.clientDraws` told to the host (the §8 mid-stream render flip);
/// nil = nothing sent yet. Edge-detected by [`reconcileCursorRender`] from the live mouse
@@ -500,8 +512,8 @@ public final class StreamLayerView: NSView {
// video); hidden host pointer (or no shape yet) = invisible. Without the channel,
// M1 behavior: invisible local cursor, the composited host cursor is the visible one.
if cursorChannelActive, let st = cursorState, st.visible,
let host = hostCursors[st.serial] {
addCursorRect(bounds, cursor: host)
let shape = hostCursors[st.serial] {
addCursorRect(bounds, cursor: scaledCursor(shape))
} else {
addCursorRect(bounds, cursor: Self.invisibleCursor)
}
@@ -570,12 +582,12 @@ public final class StreamLayerView: NSView {
}
private func applyCursorShape(_ ev: PunktfunkConnection.CursorShapeEvent) {
guard let cursor = Self.makeCursor(ev) else {
guard let shape = Self.makeShape(ev) else {
streamInputLog.warning("cursor shape rejected (\(ev.width)x\(ev.height)) — keeping the previous cursor")
return
}
if hostCursors.count >= 64 { hostCursors.removeAll() } // degenerate host: reset
hostCursors[ev.serial] = cursor
hostCursors[ev.serial] = shape
if cursorState?.serial == ev.serial {
window?.invalidateCursorRects(for: self)
}
@@ -597,8 +609,10 @@ public final class StreamLayerView: NSView {
_ = (lastHint, hintOverride)
}
/// Build an `NSCursor` from a forwarded straight-alpha RGBA shape.
private static func makeCursor(_ ev: PunktfunkConnection.CursorShapeEvent) -> NSCursor? {
/// Decode a forwarded straight-alpha RGBA shape into a CGImage + hotspot. The on-screen SIZE is
/// NOT baked in here it is applied per-use in `scaledCursor` from the live video-fit scale, so
/// the same shape re-fits across window resizes / retina moves without a re-forward.
private static func makeShape(_ ev: PunktfunkConnection.CursorShapeEvent) -> HostCursorShape? {
let (w, h) = (ev.width, ev.height)
guard w > 0, h > 0, ev.rgba.count >= w * h * 4,
let provider = CGDataProvider(data: ev.rgba as CFData),
@@ -609,10 +623,40 @@ public final class StreamLayerView: NSView {
provider: provider, decode: nil, shouldInterpolate: false,
intent: .defaultIntent)
else { return nil }
let image = NSImage(cgImage: cg, size: NSSize(width: w, height: h))
return NSCursor(
image: image,
hotSpot: NSPoint(x: min(ev.hotX, w - 1), y: min(ev.hotY, h - 1)))
return HostCursorShape(
cg: cg, width: w, height: h,
hotX: min(ev.hotX, w - 1), hotY: min(ev.hotY, h - 1))
}
/// Points-per-host-pixel: the exact factor the video frame is aspect-fit into the view (the same
/// `AVMakeRect` fit `hostPoint`/`cgScreenPoint` use). The host forwards the pointer bitmap in host
/// framebuffer pixels the mode we drive is in the client's BACKING pixels, so on retina this is
/// ~1/backingScale and the pointer lands at its TRUE size relative to the streamed desktop
/// (crisp, 1:1 with the video) rather than the 2×-inflated pixel-as-points it used to be. Because
/// the bitmap grows with the host's display scaling (96 px at 300% DPI), scaling by this is what
/// keeps a high-DPI host from forwarding a giant pointer. Falls back to 1 before the first
/// mode/layout.
private func cursorFitScale() -> CGFloat {
guard let connection else { return 1 }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0, bounds.width > 0, bounds.height > 0 else { return 1 }
let fit = AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)), insideRect: bounds)
guard fit.width > 0 else { return 1 }
return fit.width / CGFloat(mode.width)
}
/// Build the `NSCursor` for a cached shape at the CURRENT video-fit scale (see `cursorFitScale`).
/// Both the image size and the hotspot scale together so the click point stays true.
private func scaledCursor(_ shape: HostCursorShape) -> NSCursor {
let scale = cursorFitScale()
let sw = max(1, (CGFloat(shape.width) * scale).rounded())
let sh = max(1, (CGFloat(shape.height) * scale).rounded())
let image = NSImage(cgImage: shape.cg, size: NSSize(width: sw, height: sh))
let hot = NSPoint(
x: min(CGFloat(shape.hotX) * scale, sw - 1),
y: min(CGFloat(shape.hotY) * scale, sh - 1))
return NSCursor(image: image, hotSpot: hot)
}
/// Host video px CG GLOBAL screen coordinates (top-left origin, the
@@ -908,6 +952,11 @@ public final class StreamLayerView: NSView {
matchFollower?.noteSize(
widthPx: Int(px.width.rounded()), heightPx: Int(px.height.rounded()))
}
// The video-fit scale just changed (resize / retina move); rebuild the worn host pointer at
// the new scale so it tracks the video instead of freezing at its build-time size.
if captured, desktopMouse, cursorChannelActive {
window?.invalidateCursorRects(for: self)
}
}
public override func viewDidChangeBackingProperties() {
@@ -333,6 +333,13 @@ public final class StreamViewController: StreamViewControllerBase {
guard self?.captureEnabled == true else { return }
connection?.send(event)
}
// Apple Pencil the stylus plane, only against a pen-capable host (elsewhere the
// Pencil stays a finger, exactly as before). Same trust gate as touch.
streamView.penEnabled = connection.hostSupportsPen
streamView.onPenBatch = { [weak self, weak connection] batch in
guard self?.captureEnabled == true else { return }
connection?.sendPen(batch)
}
// Indirect pointer (mouse/trackpad) WITHOUT a lock absolute cursor + buttons + scroll.
// While the scene is pointer-LOCKED the GCMouse path owns motion AND buttons AND scroll, so
// the whole UIKit indirect path is gated off here (`gcMouseForwarding`). The trackpad and a
@@ -499,6 +506,8 @@ public final class StreamViewController: StreamViewControllerBase {
// onTouchEvent can still deliver the button-up.
streamView.resetTouchInput()
streamView.onTouchEvent = nil
streamView.onPenBatch = nil // after reset the pen's leave-range sample rides it
streamView.penEnabled = false
streamView.onPointerMoveAbs = nil
streamView.onPointerButton = nil
streamView.onScroll = nil
@@ -692,6 +701,12 @@ final class StreamLayerUIView: UIView {
/// Direct fingers / Pencil wire events: real touches in passthrough mode, or the
/// touch-driven mouse events (`TouchMouse`) in the trackpad/pointer modes.
var onTouchEvent: ((PunktfunkInputEvent) -> Void)?
/// Apple Pencil state-full pen sample batches (the stylus plane). Active only while
/// `penEnabled`; without it the Pencil stays on the finger path exactly as before.
var onPenBatch: (([PunktfunkPenSample]) -> Void)?
/// The host advertised `HOST_CAP_PEN`, so Pencil input splits out of the finger path onto
/// the pen plane independent of the touch-input mode (drawing must not depend on it).
var penEnabled = false
/// Indirect pointer (mouse/trackpad with no lock) absolute cursor moves.
var onPointerMoveAbs: ((HostPoint) -> Void)?
/// Indirect-pointer buttons (GameStream ids: 1=left 3=right); `down` = press.
@@ -715,10 +730,21 @@ final class StreamLayerUIView: UIView {
/// The finger route latched at gesture start a Settings change mid-gesture applies to
/// the NEXT touch, so one gesture never splits across input models.
private var fingerRoute: TouchInputMode?
/// The Apple Pencil pipeline (contacts + hover + squeeze/tap pen samples).
private lazy var pencil: PencilStream = {
let stream = PencilStream()
stream.send = { [weak self] batch in self?.onPenBatch?(batch) }
stream.videoNorm = { [weak self] point in
guard let h = self?.hostPoint(from: point) else { return nil }
return (Float(h.x) / Float(max(h.w - 1, 1)), Float(h.y) / Float(max(h.h - 1, 1)))
}
return stream
}()
/// Release anything the touch-driven mouse holds and forget gesture state session stop.
func resetTouchInput() {
touchMouse.reset()
pencil.reset() // leaves range the host lifts anything still inked
fingerRoute = nil
setSoftKeyboardVisible(false) // a stream that's gone takes its keyboard with it
}
@@ -755,6 +781,11 @@ final class StreamLayerUIView: UIView {
scrollPan.allowedScrollTypesMask = .all
scrollPan.allowedTouchTypes = []
addGestureRecognizer(scrollPan)
// Pencil squeeze / double-tap the pen plane's barrel buttons (no-op while
// `penEnabled` is false PencilStream ignores interactions out of range).
let pencilInteraction = UIPencilInteraction()
pencilInteraction.delegate = pencil
addInteraction(pencilInteraction)
#endif
backgroundColor = .black
}
@@ -779,17 +810,32 @@ final class StreamLayerUIView: UIView {
private enum TouchKind { case down, move, up, cancel }
/// Split a touch batch by kind: an INDIRECT POINTER (mouse/trackpad with no lock) drives
/// the host cursor as an absolute mouse; everything else (direct finger, Pencil) is a host
/// touch. Mixed batches are possible, so partition rather than branch on the first touch.
/// the host cursor as an absolute mouse; a Pencil goes to the pen plane when the host
/// supports it; everything else (direct finger and the Pencil toward a pen-less host)
/// is a host touch. Mixed batches are possible, so partition rather than branch on the
/// first touch.
private func route(_ touches: Set<UITouch>, event: UIEvent?, kind: TouchKind) {
var fingers: Set<UITouch> = []
var pencilTouches: Set<UITouch> = []
for touch in touches {
if touch.type == .indirectPointer {
handleIndirectPointer(touch, event: event, kind: kind)
} else if penEnabled, touch.type == .pencil {
pencilTouches.insert(touch)
} else {
fingers.insert(touch)
}
}
if !pencilTouches.isEmpty {
let phase: PencilStream.Phase =
switch kind {
case .down: .down
case .move: .move
case .up: .up
case .cancel: .cancel
}
pencil.touches(pencilTouches, event: event, phase: phase, in: self)
}
if !fingers.isEmpty { forwardFingers(fingers, kind: kind) }
}
@@ -861,8 +907,11 @@ final class StreamLayerUIView: UIView {
}
}
/// Button-less mouse/trackpad movement (no lock) absolute cursor move.
/// Button-less mouse/trackpad movement (no lock) absolute cursor move unless it is a
/// hovering PENCIL (`zOffset > 0`) on a pen-capable host, which becomes in-range pen
/// samples (hover preview with distance/tilt/azimuth) instead of a cursor move.
@objc private func handleHover(_ recognizer: UIHoverGestureRecognizer) {
if penEnabled, pencil.maybeHover(recognizer, in: self) { return }
switch recognizer.state {
case .began, .changed:
if let h = hostPoint(from: recognizer.location(in: self)) { onPointerMoveAbs?(h) }
+12
View File
@@ -405,6 +405,16 @@ pub type CursorChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorChannelRequest) -> Result<()> + Send + Sync,
>;
/// The mid-stream cursor-render flip (`IOCTL_SET_CURSOR_FORWARD`, proto v6) as a host-facade
/// closure — same contract as [`CursorChannelSender`]. `bool` = declare the IddCx hardware
/// cursor (`true`) or stand it down (`false`; the host facade additionally forces the same-mode
/// re-commit that actualises the OS's software-cursor default). The capturer drives this from
/// its secure-desktop watch: UAC/Winlogon render only through the software-cursor path, so a
/// path pinned to the hardware cursor never presents them (the 0.18.0 secure-desktop
/// regression).
#[cfg(target_os = "windows")]
pub type CursorForwardSender = std::sync::Arc<dyn Fn(bool) -> Result<()> + Send + Sync>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")]
pub mod pwinit;
@@ -491,6 +501,7 @@ pub fn open_idd_push(
keepalive: Box<dyn Send>,
sender: FrameChannelSender,
cursor_sender: Option<CursorChannelSender>,
cursor_forward: Option<CursorForwardSender>,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open(
target,
@@ -501,6 +512,7 @@ pub fn open_idd_push(
keepalive,
sender,
cursor_sender,
cursor_forward,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
+234 -33
View File
@@ -10,20 +10,31 @@
//!
//! **Multiple Xwaylands.** gamescope runs one Xwayland per `--xwayland-count` (Steam Gaming Mode
//! uses 2: one for Big Picture, one for the game). The pointer lives on whichever is FOCUSED — an
//! inactive display's pointer is frozen. So the source connects to ALL of them and each tick
//! follows the one whose pointer actually moved (gamescope routes input to the focused surface, so
//! exactly one moves at a time). It reads that display's shape too, since each Xwayland has its own
//! current cursor. This is why a single-display read froze the pointer the moment a game on the
//! OTHER Xwayland took focus.
//! inactive display's pointer is frozen. So the source connects to ALL of them and publishes from
//! the one gamescope is actually drawing the pointer on; it reads that display's shape too, since
//! each Xwayland has its own current cursor. This is why a single-display read froze the pointer
//! the moment a game on the OTHER Xwayland took focus.
//!
//! Two X sources per display, split by cost (Sunshine's split):
//! Three X sources per display, split by cost (Sunshine's split, plus gamescope's own verdict):
//! * **Position** — core `QueryPointer` on the root, polled fast. Cheap (a few-byte reply, no
//! bitmap), so it can out-pace the stream fps and keep the composited pointer smooth. It also
//! doubles as the focus signal (the display whose pointer moves is the active one).
//! * **Shape / hotspot / visibility** — `XFixesGetCursorImage`, refreshed only after an XFixes
//! `CursorNotify` (a real cursor change). A game hiding the pointer IS a cursor change → the
//! image comes back fully transparent → `visible: false`, which the encode loop strips before
//! any blend path draws it (so a grabbed pointer shows nothing, matching native gamescope).
//! bitmap), so it can out-pace the stream fps and keep the composited pointer smooth.
//! * **Shape / hotspot** — `XFixesGetCursorImage`, refreshed only after an XFixes `CursorNotify`
//! (a real cursor change). A fully-transparent image reads as hidden.
//! * **Visibility + focus** — [`GAMESCOPE_CURSOR_VISIBLE_FEEDBACK`](GS_CURSOR_FEEDBACK) on the
//! root, read at connect and re-read on its `PropertyNotify`.
//!
//! **Why the feedback atom and not pointer motion.** gamescope hides its pointer by WARPING the X
//! pointer to the root's bottom-right corner pixel — it does NOT swap in a transparent X cursor, so
//! `XFixesGetCursorImage` keeps handing back the last opaque arrow. The original "follow whichever
//! display's pointer moved" heuristic therefore stuck to the parked display (a parked pointer never
//! moves again) and composited that arrow at `(w-1, h-1)`: a sliver of cursor welded to the corner
//! of the stream for the rest of the session, while the real pointer went undrawn. Reported
//! on-glass as "part of cursor shows up on bottom right … isn't where it really is", in every game
//! (a game grabs the pointer, so the hide is permanent). Measured on a live 1920x1080 Gaming Mode
//! session: real motion ⇒ pointer live + feedback `1`; 3 s idle (`--hide-cursor-delay 3000`) ⇒
//! pointer `(1919, 1079)` + feedback `0`. The atom answers BOTH questions correctly for a static
//! pointer, which motion cannot: which Xwayland owns it, and whether to draw it at all — so
//! honouring it also gives the stream gamescope's own idle auto-hide, which this source never had.
use std::sync::{
atomic::{AtomicBool, Ordering},
@@ -35,7 +46,11 @@ use pf_frame::CursorOverlay;
use x11rb::connection::Connection;
use x11rb::errors::ReplyError;
use x11rb::protocol::xfixes::{self, ConnectionExt as _, GetCursorImageReply};
use x11rb::protocol::xproto::{ConnectionExt as _, QueryPointerReply, Window};
use x11rb::protocol::xproto::{
Atom, AtomEnum, ChangeWindowAttributesAux, ConnectionExt as _, EventMask, QueryPointerReply,
Window,
};
use x11rb::protocol::Event;
use x11rb::rust_connection::RustConnection;
/// Serializes the brief `XAUTHORITY` env swap around a connect (the var is process-global). Only
@@ -47,6 +62,17 @@ static XAUTH_LOCK: Mutex<()> = Mutex::new(());
/// and must out-run a 240 fps session or the pointer stutters.
const POLL: Duration = Duration::from_millis(4);
/// gamescope's own pointer verdict, published on EVERY nested Xwayland's root: `1` on the server
/// whose pointer gamescope is currently drawing, `0` on the others — and `0` on all of them once
/// the pointer is hidden (a game grabbed it, or `--hide-cursor-delay` fired). See the module docs
/// for why this, and not pointer motion, is the signal this source follows.
const GS_CURSOR_FEEDBACK: &str = "GAMESCOPE_CURSOR_VISIBLE_FEEDBACK";
/// Self-heal cadence for the feedback re-read: `PropertyNotify` drives it, this only covers a
/// missed/coalesced event (and a gamescope that publishes the atom after we connected). One
/// `GetProperty` per display per interval is nothing next to the 250 Hz pointer poll.
const FEEDBACK_RESYNC: Duration = Duration::from_millis(250);
/// A running XFixes cursor reader. Dropping it stops the worker thread and joins it, releasing the
/// X connections — so it lives exactly as long as the capturer that owns it.
pub(super) struct XFixesCursorSource {
@@ -68,7 +94,9 @@ impl XFixesCursorSource {
let mut displays = Vec::new();
for (dpy, xauth) in targets {
match connect(&dpy, xauth.as_deref()) {
Ok((conn, root)) => displays.push(XDisplay::new(dpy, conn, root)),
Ok((conn, root, feedback)) => {
displays.push(XDisplay::new(dpy, conn, root, feedback))
}
Err(e) => tracing::warn!(
dpy = %dpy,
error = %e,
@@ -84,9 +112,13 @@ impl XFixesCursorSource {
return None;
}
let names: Vec<&str> = displays.iter().map(|d| d.name.as_str()).collect();
let feedback = displays.iter().any(|d| d.gs_visible.is_some());
tracing::info!(
displays = ?names,
"gamescope cursor: XFixes source live — following the focused Xwayland's pointer"
cursor_feedback = feedback,
"gamescope cursor: XFixes source live — following the Xwayland gamescope draws the \
pointer on (cursor_feedback=false this gamescope publishes no \
GAMESCOPE_CURSOR_VISIBLE_FEEDBACK, degrading to the pointer-motion heuristic)"
);
let stop = Arc::new(AtomicBool::new(false));
@@ -111,10 +143,15 @@ impl Drop for XFixesCursorSource {
}
}
/// Open the X connection, negotiate XFixes, and select cursor-change events — returning the
/// connection + root window. `RustConnection` reads `XAUTHORITY` from the env at connect time only,
/// so set it under the lock (the host isn't a gamescope child), connect, then restore.
fn connect(dpy: &str, xauthority: Option<&str>) -> Result<(RustConnection, Window), String> {
/// Open the X connection, negotiate XFixes, and select cursor-change + root-property events —
/// returning the connection, root window and this display's initial
/// [`GAMESCOPE_CURSOR_FEEDBACK`](GS_CURSOR_FEEDBACK) reading (`(atom, value)`; the value is `None`
/// when gamescope publishes no such property here). `RustConnection` reads `XAUTHORITY` from the
/// env at connect time only, so set it under the lock (the host isn't a gamescope child), connect,
/// then restore.
type Connected = (RustConnection, Window, (Atom, Option<bool>));
fn connect(dpy: &str, xauthority: Option<&str>) -> Result<Connected, String> {
let (conn, screen_num) = {
let _g = XAUTH_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let prev = std::env::var_os("XAUTHORITY");
@@ -148,8 +185,41 @@ fn connect(dpy: &str, xauthority: Option<&str>) -> Result<(RustConnection, Windo
.map_err(ReplyError::from)
.and_then(|c| c.check())
.map_err(|e| format!("SelectCursorInput: {e}"))?;
// …and whenever gamescope republishes its cursor verdict. Interned with `only_if_exists=false`
// so we hold a matchable atom id even on a gamescope that sets the property later; a failure to
// select PROPERTY_CHANGE is NOT fatal — the resync re-read still tracks it, just at 250 ms.
let feedback_atom = conn
.intern_atom(false, GS_CURSOR_FEEDBACK.as_bytes())
.map_err(ReplyError::from)
.and_then(|c| c.reply())
.map(|r| r.atom)
.unwrap_or(0);
if let Ok(c) = conn.change_window_attributes(
root,
&ChangeWindowAttributesAux::new().event_mask(EventMask::PROPERTY_CHANGE),
) {
let _ = c.check();
}
let _ = conn.flush();
Ok((conn, root))
let feedback = read_cursor_feedback(&conn, root, feedback_atom);
Ok((conn, root, (feedback_atom, feedback)))
}
/// Read `GAMESCOPE_CURSOR_VISIBLE_FEEDBACK` off `root`. `None` = the property is absent or
/// unreadable (not a gamescope that publishes it) — the caller then falls back to the pointer-motion
/// heuristic rather than blanking the cursor, so an older gamescope keeps today's behaviour.
fn read_cursor_feedback(conn: &RustConnection, root: Window, atom: Atom) -> Option<bool> {
if atom == 0 {
return None;
}
let reply = conn
.get_property(false, root, atom, AtomEnum::CARDINAL, 0, 1)
.ok()?
.reply()
.ok()?;
let value = reply.value32()?.next()?;
Some(value != 0)
}
/// One gamescope Xwayland the source tracks.
@@ -163,12 +233,22 @@ struct XDisplay {
shape: Shape,
/// A `CursorNotify` (or first read) is pending — fetch the shape when this display is active.
need_shape: bool,
/// Interned [`GS_CURSOR_FEEDBACK`] atom (`0` = intern failed — treated as absent).
feedback_atom: Atom,
/// gamescope's verdict for THIS display: `Some(true)` = it is drawing the pointer here,
/// `Some(false)` = it is not, `None` = this gamescope publishes no verdict at all.
gs_visible: Option<bool>,
/// The X connection died (game/Xwayland exited) — skip it.
dead: bool,
}
impl XDisplay {
fn new(name: String, conn: RustConnection, root: Window) -> Self {
fn new(
name: String,
conn: RustConnection,
root: Window,
(feedback_atom, gs_visible): (Atom, Option<bool>),
) -> Self {
XDisplay {
name,
conn,
@@ -176,9 +256,20 @@ impl XDisplay {
last_pos: None,
shape: Shape::default(),
need_shape: true,
feedback_atom,
gs_visible,
dead: false,
}
}
/// Re-read gamescope's verdict, keeping a previously-seen one if the read fails (a transient
/// failure must not look like "this gamescope has no feedback" and re-arm the motion heuristic).
fn resync_feedback(&mut self) {
let fresh = read_cursor_feedback(&self.conn, self.root, self.feedback_atom);
if fresh.is_some() || self.gs_visible.is_none() {
self.gs_visible = fresh;
}
}
}
/// Cached cursor shape for one display.
@@ -209,20 +300,35 @@ fn run(
let mut out_serial = 0u64;
let mut last_key = (usize::MAX, u64::MAX);
let mut warned_image = false;
let mut last_resync = std::time::Instant::now();
while !stop.load(Ordering::Relaxed) {
// 1) Poll every display's pointer; note which moved since last tick (the focus signal).
// A missed/coalesced PropertyNotify would otherwise strand the verdict — re-read on a slow
// cadence so the source always converges (also picks the atom up if gamescope adds it late).
let resync = last_resync.elapsed() >= FEEDBACK_RESYNC;
if resync {
last_resync = std::time::Instant::now();
}
// 1) Poll every display's pointer; note which moved since last tick (the fallback focus
// signal, used only when this gamescope publishes no cursor verdict).
let mut active_moved = false;
let mut other_moved: Option<usize> = None;
for (i, d) in displays.iter_mut().enumerate() {
if d.dead {
continue;
}
// Drain pending events; the only ones selected are CursorNotify, so ANY event means
// "re-read this display's shape". poll_for_event never blocks.
// Drain pending events. Two kinds are selected: XFixes CursorNotify (the shape
// changed) and root PropertyNotify (gamescope republished its cursor verdict, among
// the many other properties it keeps on the root — hence the atom match).
let mut need_feedback = resync;
loop {
match d.conn.poll_for_event() {
Ok(Some(_)) => d.need_shape = true,
Ok(Some(Event::XfixesCursorNotify(_))) => d.need_shape = true,
Ok(Some(Event::PropertyNotify(ev))) => {
need_feedback |= d.feedback_atom != 0 && ev.atom == d.feedback_atom;
}
Ok(Some(_)) => {}
Ok(None) => break,
Err(_) => {
d.dead = true;
@@ -230,6 +336,9 @@ fn run(
}
}
}
if need_feedback && !d.dead {
d.resync_feedback();
}
match fetch_pointer(&d.conn, d.root) {
Ok(p) if p.same_screen => {
let pos = (i32::from(p.root_x), i32::from(p.root_y));
@@ -248,13 +357,11 @@ fn run(
}
}
// 2) Switch focus: sticky to the active display while it moves (no flapping); otherwise
// follow another display that moved. If the active one died, fall to any live display.
if !active_moved {
if let Some(j) = other_moved {
active = j;
}
}
// 2) Pick the display to publish from, and decide whether a pointer should be drawn at all.
let states: Vec<(bool, Option<bool>)> =
displays.iter().map(|d| (d.dead, d.gs_visible)).collect();
let hidden_by_gamescope;
(active, hidden_by_gamescope) = pick_active(&states, active, active_moved, other_moved);
if displays.get(active).is_none_or(|d| d.dead) {
match displays.iter().position(|d| !d.dead) {
Some(k) => active = k,
@@ -283,7 +390,11 @@ fn run(
// 4) Publish the ACTIVE display's pointer + shape (or clear the slot when it has no cursor
// of its own — so a focus switch never leaves the other display's stale pointer showing).
// A pointer gamescope is not drawing is published `visible: false`, NOT dropped: the
// encode loop overwrites the frame's overlay from this slot and strips invisible ones, so
// a `None` here would leave the last visible overlay standing on repeat frames.
let d = &displays[active];
let drawn = d.shape.visible && !hidden_by_gamescope;
let overlay = match (d.last_pos, d.shape.rgba.is_empty()) {
(Some((px, py)), false) => {
let key = (active, d.shape.serial);
@@ -301,7 +412,7 @@ fn run(
serial: out_serial,
hot_x: d.shape.hot_x,
hot_y: d.shape.hot_y,
visible: d.shape.visible,
visible: drawn,
})
}
_ => None,
@@ -314,6 +425,39 @@ fn run(
}
}
/// Which display to publish from, and whether gamescope is drawing NO pointer right now —
/// `(active, hidden)`. `states` is one `(dead, gs_visible)` per display, in `displays` order.
///
/// PREFERRED: gamescope's own verdict. It is authoritative for a STATIC pointer, which is exactly
/// the case motion cannot read — a game grabs the pointer, gamescope parks it in the bottom-right
/// corner, and "follow whichever moved" then never switches again (see the module docs).
///
/// FALLBACK, only when NO live display publishes the verdict: the original motion heuristic —
/// sticky to the active display while its pointer moves (no flapping), else follow another that
/// moved. `hidden` is never asserted on this path, so an older gamescope keeps today's behaviour.
fn pick_active(
states: &[(bool, Option<bool>)],
active: usize,
active_moved: bool,
other_moved: Option<usize>,
) -> (usize, bool) {
let live = |&(dead, _): &(bool, Option<bool>)| !dead;
if states.iter().filter(|s| live(s)).any(|(_, v)| v.is_some()) {
return match states.iter().position(|s| live(s) && s.1 == Some(true)) {
// gamescope is drawing the pointer here — publish from it.
Some(i) => (i, false),
// Drawing none of them (a game grabbed it, or the idle auto-hide fired). Keep `active`
// so the shape cache and last position stay warm for the re-show; the caller publishes
// the overlay `visible: false` instead of dropping it.
None => (active, true),
};
}
match other_moved {
Some(j) if !active_moved => (j, false),
_ => (active, false),
}
}
/// Update `shape` from a fresh `GetCursorImage` reply. A hidden pointer (all-transparent) keeps the
/// last bitmap (instant re-show) but flips visibility; the serial still bumps so the change shows.
fn update_shape(shape: &mut Shape, img: &GetCursorImageReply) {
@@ -364,3 +508,60 @@ fn argb_premul_to_straight_rgba(argb: &[u32]) -> Vec<u8> {
}
out
}
#[cfg(test)]
mod tests {
use super::pick_active;
/// Steam Gaming Mode shape: display 0 = Big Picture's Xwayland, 1 = the game's.
const BPM: usize = 0;
const GAME: usize = 1;
#[test]
fn follows_the_display_gamescope_draws_on() {
// Verdict beats motion: gamescope says the game's Xwayland owns the pointer, so we publish
// from it even though only BPM's (parked) pointer looks like it moved.
let states = [(false, Some(false)), (false, Some(true))];
assert_eq!(pick_active(&states, BPM, false, Some(BPM)), (GAME, false));
}
/// THE REGRESSION: gamescope hides the pointer by warping it to the root's bottom-right corner
/// and leaves the opaque arrow as the X cursor. Nothing moves ever again, so the motion
/// heuristic stayed on the parked display and composited that arrow at (w-1, h-1) — a sliver of
/// cursor welded to the corner of the stream for the whole session, in every game.
#[test]
fn a_pointer_gamescope_draws_nowhere_is_hidden_not_parked() {
let states = [(false, Some(false)), (false, Some(false))];
// `active` is kept (shape cache + last position stay warm for the re-show) but hidden.
assert_eq!(pick_active(&states, GAME, false, None), (GAME, true));
}
#[test]
fn re_show_returns_to_the_drawing_display() {
let states = [(false, Some(true)), (false, Some(false))];
assert_eq!(pick_active(&states, GAME, false, None), (BPM, false));
}
#[test]
fn a_dead_displays_verdict_is_ignored() {
// The game's Xwayland exited mid-session with a stale `Some(true)`; BPM is the live one.
let states = [(false, Some(true)), (true, Some(true))];
assert_eq!(pick_active(&states, GAME, false, None), (BPM, false));
// …and a dead display's `Some` must not count as "this gamescope publishes a verdict",
// which would blank the cursor forever on a gamescope that publishes none.
let states = [(false, None), (true, Some(false))];
assert_eq!(pick_active(&states, BPM, false, Some(GAME)), (GAME, false));
}
#[test]
fn no_verdict_falls_back_to_the_motion_heuristic() {
let none = [(false, None), (false, None)];
// Sticky while the active display's own pointer moves…
assert_eq!(pick_active(&none, BPM, true, Some(GAME)), (BPM, false));
// …otherwise follow the one that moved…
assert_eq!(pick_active(&none, BPM, false, Some(GAME)), (GAME, false));
// …and never assert `hidden` on this path (that would regress an older gamescope to a
// cursorless stream).
assert_eq!(pick_active(&none, BPM, false, None), (BPM, false));
}
}
+102 -1
View File
@@ -408,6 +408,16 @@ pub struct IddPushCapturer {
/// shared slot) destroys the driver's cursor worker — the section here survives, so the
/// channel is re-delivered on ring recreates.
cursor_sender: Option<crate::CursorChannelSender>,
/// The cursor-render flip sender (`IOCTL_SET_CURSOR_FORWARD`) — the secure-desktop guard's
/// actuator. UAC/Winlogon render only through the OS's software-cursor path (its default on
/// every mode commit); with our hardware cursor declared (and re-declared on every
/// swap-chain assign) that path never comes back, and the secure desktop never presents —
/// the stream freezes on the last normal-desktop frame for the whole UAC/lock interaction.
/// [`Self::poll_secure_desktop`] flips the declare off/on at the secure-desktop edges.
cursor_forward: Option<crate::CursorForwardSender>,
/// The secure-desktop guard's edge state: `true` = the poller reports a secure input
/// desktop and the declare is currently stood down.
secure_active: bool,
/// The CAPTURE mouse model is active — the HOST composites the pointer into the frame
/// (see cursor_blend.rs for why DWM cannot: a declared IddCx hardware cursor is forever).
composite_cursor: bool,
@@ -657,7 +667,6 @@ impl IddPushCapturer {
/// instead of tearing the display down (audit §5.1 — no more 20 s black bail). "Failure" includes the
/// driver not attaching to the ring within a few seconds (e.g. a hybrid-GPU render mismatch).
#[allow(clippy::too_many_arguments)]
#[allow(clippy::too_many_arguments)]
pub fn open(
target: WinCaptureTarget,
preferred: Option<(u32, u32, u32)>,
@@ -667,6 +676,7 @@ impl IddPushCapturer {
keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime.
@@ -679,6 +689,7 @@ impl IddPushCapturer {
pyrowave,
sender,
cursor_sender,
cursor_forward,
) {
Ok(mut me) => {
me._keepalive = keepalive;
@@ -697,6 +708,7 @@ impl IddPushCapturer {
pyrowave: bool,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -720,6 +732,7 @@ impl IddPushCapturer {
luid,
sender.clone(),
cursor_sender.clone(),
cursor_forward.clone(),
) {
Ok(me) => Ok(me),
Err(e) => {
@@ -754,6 +767,7 @@ impl IddPushCapturer {
drv,
sender,
cursor_sender,
cursor_forward,
)
.context("IDD-push rebind to the driver's reported render adapter")
}
@@ -770,6 +784,7 @@ impl IddPushCapturer {
luid: LUID,
sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> Result<Self> {
let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -1046,6 +1061,17 @@ impl IddPushCapturer {
.unwrap_or((0, 0, i32::MAX, i32::MAX));
cursor_poll::CursorPoller::spawn(target.target_id, rect)
});
// Heal the driver's persisted cursor-forward state: a session that died on the
// secure desktop (client drops at the lock screen — the common case) leaves the
// per-target desired state `false`, and the NEXT session's channel delivery would
// adopt UNdeclared (the exact cross-session composite trap of §8.6). A fresh
// session always starts declared; the secure-desktop guard re-disables if the
// secure desktop is (still) up, via its first `poll_secure_desktop` edge.
if let (Some(_), Some(fwd)) = (cursor_shared.as_ref(), cursor_forward.as_ref()) {
if let Err(e) = fwd(true) {
tracing::debug!("cursor-forward reset at open failed (pre-v6 driver?): {e:#}");
}
}
tracing::info!(
target_id = target.target_id,
@@ -1108,6 +1134,8 @@ impl IddPushCapturer {
cursor_shared,
cursor_poll,
cursor_sender,
cursor_forward,
secure_active: false,
composite_cursor: composite_forced,
composite_forced,
cursor_blend: None,
@@ -1883,8 +1911,71 @@ impl IddPushCapturer {
Some((tex, srv))
}
/// The secure-desktop guard (the 0.18.0 UAC/Winlogon regression). UAC consent and Winlogon
/// live on the SECURE desktop, which the OS renders through the software-cursor path — its
/// per-mode-commit default. With this session's IddCx hardware cursor declared (and
/// re-declared by the driver on every swap-chain assign), that path never materialises, the
/// secure desktop never presents into our swap-chain, and the stream freezes on the last
/// normal-desktop frame for the entire UAC/lock interaction. On the poller's secure edge:
/// stand the declare down (`SET_CURSOR_FORWARD` off — the driver stops its per-assign
/// re-declare — plus the host facade's forced same-mode re-commit that actualises the
/// software cursor); on dismissal, re-declare. Runs on the capture/encode thread every tick
/// (it must keep running while frames are stalled — that is exactly the state it exits).
fn poll_secure_desktop(&mut self) {
let Some(fwd) = self.cursor_forward.as_ref() else {
return;
};
// Sessions with a declare possibly in play: the channel session that declared it, and
// the forced-composite session whose (reused) driver monitor may still run an earlier
// session's cursor worker. A plain session on a clean target has no poller — no guard.
if self.cursor_shared.is_none() && !self.composite_forced {
return;
}
let secure = self
.cursor_poll
.as_ref()
.is_some_and(|p| p.secure_desktop());
if secure == self.secure_active {
return;
}
self.secure_active = secure;
if secure {
tracing::info!(
target_id = self.target_id,
"secure desktop (UAC/Winlogon) active — standing the IddCx hardware-cursor \
declare down so the OS software-cursor path can render it"
);
if let Err(e) = fwd(false) {
tracing::warn!(
"secure-desktop cursor-forward stand-down failed (secure content may stay \
invisible this session): {e:#}"
);
}
} else {
tracing::info!(
target_id = self.target_id,
"secure desktop dismissed — restoring the cursor render model"
);
// Re-declare only for the session that RUNS the cursor channel; a forced-composite
// session never wanted the declare (leaving the driver's desired state off also
// stops a reused worker's per-assign re-declares for good — the next channel
// session's open-time reset re-arms it).
if self.cursor_shared.is_some() {
if let Err(e) = fwd(true) {
tracing::warn!(
"secure-desktop cursor-forward re-enable failed (client-drawn cursor \
may double with a composited one): {e:#}"
);
}
}
}
}
fn try_consume(&mut self) -> Result<Option<CapturedFrame>> {
self.log_driver_status_once();
// The secure-desktop guard first: while UAC/Winlogon is up there may be NO fresh frames
// at all — this edge is what brings them back.
self.poll_secure_desktop();
// Follow the display: a "Use HDR" flip recreates the ring at the matching format.
self.poll_display_hdr();
// Recover-or-drop (GB1): if a descriptor change triggered a recreate but no fresh frame has resumed
@@ -2486,6 +2577,16 @@ fn warn_444_hdr_downgrade_once() {
impl Drop for IddPushCapturer {
fn drop(&mut self) {
// A channel session ending while the secure-desktop guard is engaged must not leave the
// driver's per-target desired state off — the next session's channel delivery would
// adopt UNdeclared and silently run the composite model (§8.6's cross-session trap).
// The open-time reset also covers this (host-crash case); this is the orderly-teardown
// belt.
if self.secure_active && self.cursor_shared.is_some() {
if let Some(fwd) = self.cursor_forward.as_ref() {
let _ = fwd(true);
}
}
self.slots.clear();
// The shared header section (`MappedSection`), the frame-ready `event` (`OwnedHandle`) and the
// broker's WUDFHost process handle free themselves via RAII (unmap view, then close handle) —
@@ -29,8 +29,8 @@ use windows::Win32::Graphics::Gdi::{
BI_RGB, DIB_RGB_COLORS, HBITMAP, HDC,
};
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, OpenInputDesktop, SetThreadDesktop, DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS,
HDESK,
CloseDesktop, GetUserObjectInformationW, OpenInputDesktop, SetThreadDesktop,
DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS, HDESK, UOI_NAME,
};
use windows::Win32::UI::HiDpi::{
SetThreadDpiAwarenessContext, DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2,
@@ -63,6 +63,11 @@ struct Shape {
pub(super) struct CursorPoller {
slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>>,
stop: Arc<AtomicBool>,
/// The input desktop is a SECURE desktop (Winlogon — UAC consent / lock / logon). Classified
/// on every reattach; the capturer polls it to stand the IddCx hardware-cursor declare down
/// while the secure desktop needs the software-cursor path to render (see
/// `IddPushCapturer::poll_secure_desktop`).
secure: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
@@ -75,7 +80,11 @@ impl CursorPoller {
const INTERVAL: Duration = Duration::from_millis(4);
/// Unconditional input-desktop reattach cadence — catches secure-desktop (UAC/lock) switches
/// without a failure signal (`GetCursorInfo` on a stale desktop *succeeds* with stale data).
const REATTACH: Duration = Duration::from_secs(2);
/// 250 ms, not the original 2 s: the reattach now also feeds [`Self::secure_desktop`], which
/// gates when the secure desktop becomes VISIBLE in the stream (the hardware-cursor
/// stand-down) — a 2 s freeze at every UAC prompt is user-visible, ~4 `OpenInputDesktop`
/// syscalls/s are not.
const REATTACH: Duration = Duration::from_millis(250);
/// Spawn the poller for the virtual display `target_id`. `rect` = the target's desktop rect
/// (`source_desktop_rect` order: x, y, w, h) — cursor positions are desktop-global; the
@@ -84,15 +93,21 @@ impl CursorPoller {
pub(super) fn spawn(target_id: u32, rect: (i32, i32, i32, i32)) -> Self {
let slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>> = Arc::new(Mutex::new(None));
let stop = Arc::new(AtomicBool::new(false));
let (slot_t, stop_t) = (slot.clone(), stop.clone());
let secure = Arc::new(AtomicBool::new(false));
let (slot_t, stop_t, secure_t) = (slot.clone(), stop.clone(), secure.clone());
let thread = std::thread::Builder::new()
.name("pf-cursor-poll".into())
.spawn(move || run(target_id, rect, &slot_t, &stop_t))
.spawn(move || run(target_id, rect, &slot_t, &stop_t, &secure_t))
.ok();
if thread.is_none() {
tracing::warn!("cursor poller thread spawn failed — cursor falls back to driver shm");
}
Self { slot, stop, thread }
Self {
slot,
stop,
secure,
thread,
}
}
/// The latest overlay snapshot (`None` until the first successful shape rasterisation).
@@ -100,6 +115,12 @@ impl CursorPoller {
self.slot.lock().unwrap_or_else(|p| p.into_inner()).clone()
}
/// Whether the input desktop is currently a SECURE desktop (UAC consent / Winlogon lock or
/// logon). Latched by the poll thread on its reattach cadence (≤ [`Self::REATTACH`] stale).
pub(super) fn secure_desktop(&self) -> bool {
self.secure.load(Ordering::Relaxed)
}
/// Whether the worker thread is (still) alive — `false` degrades the capturer to the shm read.
pub(super) fn alive(&self) -> bool {
self.thread.as_ref().is_some_and(|t| !t.is_finished())
@@ -121,6 +142,7 @@ fn run(
rect: (i32, i32, i32, i32),
slot: &Mutex<Option<pf_frame::CursorOverlay>>,
stop: &AtomicBool,
secure: &AtomicBool,
) {
// Physical-pixel coordinates on this thread regardless of the process's DPI awareness:
// `rect` comes from CCD (always physical), and a DPI-virtualized `GetCursorInfo` position
@@ -130,7 +152,8 @@ fn run(
let _ = unsafe { SetThreadDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) };
let mut desktop = DesktopBinding::default();
desktop.reattach(); // best-effort: already on winsta0\default if this fails
// best-effort: already on winsta0\default if this fails
publish_secure(secure, desktop.reattach());
let mut last_attach = Instant::now();
let mut shape: Option<Shape> = None;
@@ -143,7 +166,7 @@ fn run(
std::thread::sleep(CursorPoller::INTERVAL);
if last_attach.elapsed() >= CursorPoller::REATTACH {
last_attach = Instant::now();
desktop.reattach();
publish_secure(secure, desktop.reattach());
}
let mut ci = CURSORINFO {
@@ -155,7 +178,7 @@ fn run(
if unsafe { GetCursorInfo(&mut ci) }.is_err() {
// Desktop went away under us (secure-desktop switch mid-call) — rebind and retry
// next tick; the slot keeps its last snapshot meanwhile.
desktop.reattach();
publish_secure(secure, desktop.reattach());
last_attach = Instant::now();
continue;
}
@@ -218,13 +241,25 @@ fn run(
}
}
/// Store a reattach's secure-desktop verdict (`None` = classification unavailable — keep the
/// previous state rather than flapping the capturer's hardware-cursor stand-down).
fn publish_secure(secure: &AtomicBool, verdict: Option<bool>) {
if let Some(s) = verdict {
secure.store(s, Ordering::Relaxed);
}
}
/// The thread's owned input-desktop handle — the [`SendInputInjector`] reattach model
/// (`pf-inject` sendinput.rs): keep the current binding, swap on demand, close exactly once.
#[derive(Default)]
struct DesktopBinding(Option<HDESK>);
impl DesktopBinding {
fn reattach(&mut self) {
/// Rebind to the CURRENT input desktop. Returns whether that desktop is a SECURE one
/// (`UOI_NAME` != "Default": "Winlogon" during UAC consent / lock / logon) — `None` when the
/// input desktop could not be opened, in which case the binding (and the caller's secure
/// state) stays put.
fn reattach(&mut self) -> Option<bool> {
const GENERIC_ALL: u32 = 0x1000_0000;
// SAFETY: `OpenInputDesktop`/`SetThreadDesktop`/`CloseDesktop` take only by-value args.
// `OpenInputDesktop` yields an owned `HDESK` only on `Ok`; it is either installed (and the
@@ -238,6 +273,7 @@ impl DesktopBinding {
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
) {
Ok(h) => {
let secure = desktop_is_secure(h);
if SetThreadDesktop(h).is_ok() {
if let Some(old) = self.0.replace(h) {
let _ = CloseDesktop(old);
@@ -245,13 +281,40 @@ impl DesktopBinding {
} else {
let _ = CloseDesktop(h);
}
Some(secure)
}
Err(_) => { /* not privileged for this desktop; stay put */ }
Err(_) => None, // not privileged for this desktop; stay put
}
}
}
}
/// `UOI_NAME` of `h` != "Default" — i.e. the input desktop is Winlogon (UAC consent / lock /
/// logon) or a screen-saver desktop, both of which need the OS's software-cursor render path.
/// Unnameable desktops read as NOT secure: the only in-contract failure is a too-small buffer,
/// and misreading secure-as-normal merely keeps today's behavior for a beat.
fn desktop_is_secure(h: HDESK) -> bool {
let mut name = [0u16; 64]; // "Default"/"Winlogon"/"Screen-saver" all fit with room to spare
let mut needed = 0u32;
// SAFETY: `h` is the live desktop handle the caller just opened; `name`/`needed` are live
// out-params sized exactly as passed; the call writes at most `nlength` bytes.
let ok = unsafe {
GetUserObjectInformationW(
windows::Win32::Foundation::HANDLE(h.0),
UOI_NAME,
Some(name.as_mut_ptr().cast()),
(name.len() * 2) as u32,
Some(&mut needed),
)
};
if ok.is_err() {
return false;
}
let len = name.iter().position(|&c| c == 0).unwrap_or(name.len());
let name = String::from_utf16_lossy(&name[..len]);
!name.eq_ignore_ascii_case("Default")
}
impl Drop for DesktopBinding {
fn drop(&mut self) {
if let Some(h) = self.0.take() {
+11 -9
View File
@@ -80,12 +80,13 @@ pub const fn interface_guid_fields() -> (u32, u16, u16, [u8; 8]) {
/// and the desktop model gets exclusion + forwarding. Nothing existing changed; against a v5
/// driver the unknown IOCTL fails and the host logs + keeps the declared-at-ADD behavior.
/// v6 tail ext (no bump, the `AddRequest` luminance-tail discipline):
/// [`control::AddReply::cursor_excluded`] — the driver reports whether the ADDed monitor's OS
/// target already carries a hardware-cursor declare from an earlier session. A declare is
/// IRREVOCABLE for the target's life (remote-desktop-sweep §8.6, proven on-glass): DWM never
/// composites the software cursor back into that target's frames, and the sticky state survives
/// monitor REMOVE→ADD because the host hands every client a STABLE target id. The host uses the
/// flag to self-composite the pointer (GDI poller + blend) in sessions that never negotiate the
/// [`control::AddReply::cursor_excluded`] — the driver reports whether its ADAPTER already
/// carries a hardware-cursor declare from an earlier session. A declare is IRREVOCABLE
/// (remote-desktop-sweep §8.6, proven on-glass) and its exclusion reaches EVERY later monitor of
/// the adapter, not just the declaring target (on-glass 2026-07-23: a declare on one target left
/// a different client's fresh target cursor-less): DWM never composites the software cursor back
/// into any of the adapter's frames until the adapter resets. The host uses the flag to
/// self-composite the pointer (GDI poller + blend) in sessions that never negotiate the
/// cursor channel — without it those sessions are silently cursor-less. Both skews degrade
/// cleanly: an old driver writes only the 20-byte reply prefix (host reads `0` = unknown/clean),
/// an old host retrieves a 20-byte buffer (driver writes just the prefix).
@@ -217,9 +218,10 @@ pub mod control {
/// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a
/// WUDFHost restart between sessions can never leave the host duplicating into a dead process.
pub wudf_pid: u32,
/// Non-zero = this monitor's OS target already carries an IRREVOCABLE hardware-cursor
/// declare from an earlier session (remote-desktop-sweep §8.6): DWM excludes the pointer
/// from every frame on this target, forever, and a session without the cursor channel must
/// Non-zero = the ADAPTER already carries an IRREVOCABLE hardware-cursor declare from an
/// earlier session (remote-desktop-sweep §8.6; reach is adapter-wide, not per-target —
/// on-glass 2026-07-23): DWM excludes the pointer from every frame on every monitor until
/// the adapter resets, and a session without the cursor channel must
/// self-composite (GDI poller + blend) or stream a cursor-less desktop. Appended after
/// [`ADD_REPLY_LEGACY_SIZE`] under the same dual-size discipline as the `AddRequest`
/// luminance tail: an un-upgraded driver writes only the legacy prefix (the host's
+6 -4
View File
@@ -32,10 +32,12 @@ pub struct WinCaptureTarget {
/// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown
/// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse).
pub wudf_pid: u32,
/// The ADD reply flagged this target as carrying an IRREVOCABLE IddCx hardware-cursor declare
/// from an earlier session (remote-desktop-sweep §8.6): DWM excludes the pointer from its
/// frames forever, so a session WITHOUT the cursor channel must self-composite (the IDD-push
/// capturer's forced-composite gate) or the streamed desktop has no cursor at all.
/// The ADD reply flagged the ADAPTER as carrying an IRREVOCABLE IddCx hardware-cursor declare
/// from an earlier session (remote-desktop-sweep §8.6; reach is adapter-wide, not per-target —
/// on-glass 2026-07-23, a GameStream session's fresh target streamed cursor-less): DWM
/// excludes the pointer from its frames until adapter reset, so a session WITHOUT the cursor
/// channel must self-composite (the IDD-push capturer's forced-composite gate) or the
/// streamed desktop has no cursor at all.
pub cursor_excluded: bool,
}
+5
View File
@@ -61,5 +61,10 @@ windows = { version = "0.62", features = [
"Win32_System_StationsAndDesktops",
"Win32_System_Threading",
"Win32_UI_Input_KeyboardAndMouse",
# Synthetic pointer devices (PT_PEN / PT_TOUCH) — the pen/touch injectors. The device
# create/destroy + POINTER_TYPE_INFO live under Controls in this metadata layout;
# InjectSyntheticPointerInput + POINTER_*_INFO under Input_Pointer.
"Win32_UI_Controls",
"Win32_UI_Input_Pointer",
"Win32_UI_WindowsAndMessaging",
] }
+396
View File
@@ -0,0 +1,396 @@
//! Virtual tablet ("Punktfunk Pen"): a uinput stylus device carrying the pen plane's full
//! fidelity — pressure, tilt, barrel roll, hover distance, eraser, barrel buttons
//! (design/pen-tablet-input.md §5).
//!
//! Deliberately a **uinput device, not a compositor-protocol citizen**: no virtual-tablet
//! protocol exists in EI, the RemoteDesktop portal, KWin `fake_input`, or wlroots — while
//! every compositor consumes evdev tablets via libinput and forwards them to apps over
//! `zwp_tablet_v2` with nothing to configure. udev's `input_id` builtin classifies the device
//! from its capabilities (`BTN_TOOL_PEN` + `ABS_X/Y` ⇒ `ID_INPUT_TABLET`), so Krita/GIMP/
//! Xournal++ see a real pen. Output mapping is the compositor's own tablet-mapping default
//! (single output ⇒ correct; multi-monitor pinning is the documented follow-up, which is why
//! the device carries a stable, distinctive identity to key rules on).
//!
//! The consumer feeds it [`PenTransition`]s straight from the core's
//! [`PenTracker`](punktfunk_core::quic::PenTracker); this file only translates transitions to
//! evdev events and groups them into SYN frames so proximity-enter carries its position in the
//! same frame (libinput would otherwise report an entry at a stale point).
//!
//! ioctl numbers/struct layouts mirror `gamepad.rs` (verified against the same kernel
//! generation); each backend file stays self-contained by convention.
use anyhow::{bail, Result};
use punktfunk_core::quic::{PenSample, PenTool, PenTransition, PEN_BARREL1, PEN_BARREL2};
use std::os::fd::{AsRawFd, OwnedFd};
// ioctls (x86_64).
const UI_DEV_CREATE: libc::c_ulong = 0x5501;
const UI_DEV_DESTROY: libc::c_ulong = 0x5502;
const UI_DEV_SETUP: libc::c_ulong = 0x405c_5503;
const UI_ABS_SETUP: libc::c_ulong = 0x401c_5504;
const UI_SET_EVBIT: libc::c_ulong = 0x4004_5564;
const UI_SET_KEYBIT: libc::c_ulong = 0x4004_5565;
const UI_SET_PROPBIT: libc::c_ulong = 0x4004_556e;
// input-event-codes.h subset.
const EV_SYN: u16 = 0x00;
const EV_KEY: u16 = 0x01;
const EV_ABS: u16 = 0x03;
const SYN_REPORT: u16 = 0;
const ABS_X: u16 = 0x00;
const ABS_Y: u16 = 0x01;
/// Barrel roll rides ABS_Z (the Wacom Art-Pen rotation convention); libinput normalizes the
/// declared min..max onto its 0..360° rotation axis.
const ABS_Z: u16 = 0x02;
const ABS_PRESSURE: u16 = 0x18;
const ABS_DISTANCE: u16 = 0x19;
const ABS_TILT_X: u16 = 0x1a;
const ABS_TILT_Y: u16 = 0x1b;
const BTN_TOOL_PEN: u16 = 0x140;
const BTN_TOOL_RUBBER: u16 = 0x141;
const BTN_TOUCH: u16 = 0x14a;
const BTN_STYLUS: u16 = 0x14b;
const BTN_STYLUS2: u16 = 0x14c;
/// The pen writes on the display it is mapped to (a "screen tablet"), not a desk pad —
/// libinput then maps the full ABS range onto the output rect, exactly the wire's normalized
/// coordinate contract.
const INPUT_PROP_DIRECT: libc::c_int = 0x01;
/// Full-scale wire pressure (u16) → the declared 0..4095 axis.
const PRESSURE_SHIFT: u32 = 4;
/// Wire hover distance (u16, 0xFFFF = unknown) → the declared 0..1023 axis.
const DISTANCE_SHIFT: u32 = 6;
const ABS_RANGE: f32 = 65535.0;
#[repr(C)]
struct InputId {
bustype: u16,
vendor: u16,
product: u16,
version: u16,
}
#[repr(C)]
struct UinputSetup {
id: InputId,
name: [u8; 80],
ff_effects_max: u32,
}
#[repr(C)]
#[derive(Default, Clone, Copy)]
struct AbsInfo {
value: i32,
minimum: i32,
maximum: i32,
fuzz: i32,
flat: i32,
resolution: i32,
}
#[repr(C)]
struct UinputAbsSetup {
code: u16,
_pad: u16,
absinfo: AbsInfo,
}
#[repr(C)]
#[derive(Clone, Copy)]
struct InputEventRaw {
time: libc::timeval,
type_: u16,
code: u16,
value: i32,
}
fn ioctl_int(fd: i32, req: libc::c_ulong, arg: libc::c_int, what: &str) -> Result<()> {
// SAFETY: every caller passes a UI_SET_*/UI_DEV_* request whose argument the kernel reads
// as a plain int; `fd` is a live uinput fd owned by the caller. No memory is handed over.
if unsafe { libc::ioctl(fd, req, arg) } < 0 {
bail!("{what}: {}", std::io::Error::last_os_error());
}
Ok(())
}
fn ioctl_ptr<T>(fd: i32, req: libc::c_ulong, arg: *mut T, what: &str) -> Result<()> {
// SAFETY: every caller passes a pointer to a live, initialized `#[repr(C)]` struct matching
// the request's expected layout (UI_DEV_SETUP/UI_ABS_SETUP); the kernel reads it during the
// call and retains nothing.
if unsafe { libc::ioctl(fd, req, arg) } < 0 {
bail!("{what}: {}", std::io::Error::last_os_error());
}
Ok(())
}
/// The active tool's evdev key, one in proximity at a time (Wacom semantics — the core's
/// [`PenTracker`](punktfunk_core::quic::PenTracker) already re-enters proximity on a tool
/// switch, so this only tracks which key to release on `ProximityOut`).
fn tool_key(tool: PenTool) -> u16 {
match tool {
PenTool::Eraser => BTN_TOOL_RUBBER,
// Unknown = a newer client's future tool — nearest ink-capable behavior is the pen.
PenTool::Pen | PenTool::Unknown => BTN_TOOL_PEN,
}
}
/// One per-session virtual tablet. Created lazily on the first pen batch (a session that never
/// draws never creates a device), destroyed with the session (Drop → `UI_DEV_DESTROY`).
pub struct VirtualPen {
fd: OwnedFd,
/// The tool key currently held in proximity (release target for `ProximityOut`).
tool: u16,
/// Whether the current SYN frame already carries a Motion — the frame-split trigger for
/// consecutive samples in one batch.
frame_has_motion: bool,
/// Whether the current SYN frame has any unflushed events.
frame_dirty: bool,
}
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
use std::os::fd::FromRawFd;
// SAFETY: `c"/dev/uinput"` is a 'static NUL-terminated C string literal; `open` reads it
// as a path, returns a fresh fd (or -1) and retains nothing.
let raw = unsafe {
libc::open(
c"/dev/uinput".as_ptr(),
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if raw < 0 {
bail!(
"open /dev/uinput: {} (install the udev rule granting the 'input' group access \
see scripts/60-punktfunk.rules and add the user to the 'input' group)",
std::io::Error::last_os_error()
);
}
// SAFETY: `raw >= 0` here, a freshly-opened fd owned nowhere else; `OwnedFd` becomes the
// unique owner and closes it exactly once on drop.
let fd = unsafe { OwnedFd::from_raw_fd(raw) };
ioctl_int(raw, UI_SET_EVBIT, EV_KEY as i32, "UI_SET_EVBIT(EV_KEY)")?;
ioctl_int(raw, UI_SET_EVBIT, EV_ABS as i32, "UI_SET_EVBIT(EV_ABS)")?;
for key in [
BTN_TOOL_PEN,
BTN_TOOL_RUBBER,
BTN_TOUCH,
BTN_STYLUS,
BTN_STYLUS2,
] {
ioctl_int(raw, UI_SET_KEYBIT, key as i32, "UI_SET_KEYBIT")?;
}
ioctl_int(
raw,
UI_SET_PROPBIT,
INPUT_PROP_DIRECT,
"UI_SET_PROPBIT(DIRECT)",
)?;
// Position spans the full u16 range; `resolution` (units/mm) only feeds libinput's mm
// math (nothing pen-relevant), but tablets without one trip its missing-resolution
// fixup — 100 declares a plausible ~655 mm drawing surface.
let pos = AbsInfo {
minimum: 0,
maximum: 65535,
resolution: 100,
..Default::default()
};
// Tilt in degrees from vertical, per evdev convention; resolution = units/radian (57
// ⇔ 1 unit = 1°, what the Wacom driver declares).
let tilt = AbsInfo {
minimum: -90,
maximum: 90,
resolution: 57,
..Default::default()
};
for (code, info) in [
(ABS_X, pos),
(ABS_Y, pos),
(
ABS_PRESSURE,
AbsInfo {
minimum: 0,
maximum: 4095,
..Default::default()
},
),
(
ABS_DISTANCE,
AbsInfo {
minimum: 0,
maximum: 1023,
..Default::default()
},
),
(ABS_TILT_X, tilt),
(ABS_TILT_Y, tilt),
(
// Barrel roll: libinput maps the declared range linearly onto 0..360°.
ABS_Z,
AbsInfo {
minimum: 0,
maximum: 359,
..Default::default()
},
),
] {
let mut a = UinputAbsSetup {
code,
_pad: 0,
absinfo: info,
};
ioctl_ptr(raw, UI_ABS_SETUP, &mut a, "UI_ABS_SETUP")?;
}
// A stable, distinctive identity (pid.codes open-source VID) so compositor
// tablet-mapping rules — GNOME's per-`vendor:product` gsettings path, sway
// `map_to_output` — can target exactly this device.
let mut setup = UinputSetup {
id: InputId {
bustype: 0x0006, // BUS_VIRTUAL
vendor: 0x1209,
product: 0x5046, // "PF"
version: 1,
},
name: [0; 80],
ff_effects_max: 0,
};
let name = b"Punktfunk Pen";
setup.name[..name.len()].copy_from_slice(name);
ioctl_ptr(raw, UI_DEV_SETUP, &mut setup, "UI_DEV_SETUP")?;
ioctl_int(raw, UI_DEV_CREATE, 0, "UI_DEV_CREATE")?;
tracing::info!("virtual tablet created (Punktfunk Pen, uinput)");
Ok(VirtualPen {
fd,
tool: BTN_TOOL_PEN,
frame_has_motion: false,
frame_dirty: false,
})
}
fn emit(&self, type_: u16, code: u16, value: i32) {
let ev = InputEventRaw {
time: libc::timeval {
tv_sec: 0,
tv_usec: 0,
},
type_,
code,
value,
};
// SAFETY: `ev` is a live local `#[repr(C)]` all-integer struct (no padding: timeval=16 +
// u16 + u16 + i32 = 24), so every byte is initialized; the slice spans exactly `ev`'s
// bytes and is used immediately below with no concurrent mutation.
let bytes = unsafe {
std::slice::from_raw_parts(
&ev as *const _ as *const u8,
std::mem::size_of::<InputEventRaw>(),
)
};
// Best-effort like the gamepad path: a full kernel queue drops the event; pen samples
// are state-full, so the next frame re-syncs axes (and the tracker re-syncs state).
// SAFETY: `self.fd` stays open for the synchronous call; `write` only reads
// `bytes.len()` bytes from the still-live local and retains nothing.
let _ = unsafe {
libc::write(
self.fd.as_raw_fd(),
bytes.as_ptr() as *const libc::c_void,
bytes.len(),
)
};
}
fn flush(&mut self) {
if self.frame_dirty {
self.emit(EV_SYN, SYN_REPORT, 0);
self.frame_dirty = false;
self.frame_has_motion = false;
}
}
fn motion(&mut self, s: &PenSample) {
self.emit(EV_ABS, ABS_X, (s.x * ABS_RANGE) as i32);
self.emit(EV_ABS, ABS_Y, (s.y * ABS_RANGE) as i32);
self.emit(EV_ABS, ABS_PRESSURE, (s.pressure >> PRESSURE_SHIFT) as i32);
if s.distance != punktfunk_core::quic::PEN_DISTANCE_UNKNOWN {
self.emit(EV_ABS, ABS_DISTANCE, (s.distance >> DISTANCE_SHIFT) as i32);
}
// Polar → tiltX/tiltY needs both angles; azimuth clockwise from north, so east (90°)
// tilts +X and south (180°, toward the user) tilts +Y — the evdev/W3C signs.
if s.tilt_deg != punktfunk_core::quic::PEN_TILT_UNKNOWN
&& s.azimuth_deg != punktfunk_core::quic::PEN_ANGLE_UNKNOWN
{
let az = (s.azimuth_deg as f32).to_radians();
let tilt = s.tilt_deg as f32;
self.emit(EV_ABS, ABS_TILT_X, (tilt * az.sin()).round() as i32);
self.emit(EV_ABS, ABS_TILT_Y, (-tilt * az.cos()).round() as i32);
}
if s.roll_deg != punktfunk_core::quic::PEN_ANGLE_UNKNOWN {
self.emit(EV_ABS, ABS_Z, (s.roll_deg % 360) as i32);
}
self.frame_dirty = true;
self.frame_has_motion = true;
}
/// Apply one decoded batch's transitions (the core tracker's output, in its documented
/// order), grouping them into SYN frames: a frame closes before a `ProximityIn` (an entry
/// is a new instant — and must carry its own position, not inherit the stale frame) and
/// before a second `Motion` (consecutive samples are consecutive instants), plus a final
/// close. So `[ProxIn, Motion, TipDown]` lands as ONE frame — libinput reports the entry
/// already at the right point with contact — while a drag batch's `[Motion, Motion]`
/// stays two.
pub fn apply_batch(&mut self, transitions: &[PenTransition]) {
for t in transitions {
match t {
PenTransition::ProximityIn { tool } => {
self.flush();
self.tool = tool_key(*tool);
self.emit(EV_KEY, self.tool, 1);
self.frame_dirty = true;
}
PenTransition::Motion { sample } => {
if self.frame_has_motion {
self.flush();
}
self.motion(sample);
}
PenTransition::TipDown => {
self.emit(EV_KEY, BTN_TOUCH, 1);
self.frame_dirty = true;
}
PenTransition::ButtonsChanged { pressed, released } => {
for (bit, key) in [(PEN_BARREL1, BTN_STYLUS), (PEN_BARREL2, BTN_STYLUS2)] {
if pressed & bit != 0 {
self.emit(EV_KEY, key, 1);
self.frame_dirty = true;
}
if released & bit != 0 {
self.emit(EV_KEY, key, 0);
self.frame_dirty = true;
}
}
}
PenTransition::TipUp => {
self.emit(EV_KEY, BTN_TOUCH, 0);
self.emit(EV_ABS, ABS_PRESSURE, 0);
self.frame_dirty = true;
}
PenTransition::ProximityOut => {
self.emit(EV_KEY, self.tool, 0);
self.frame_dirty = true;
}
}
}
self.flush();
}
}
impl Drop for VirtualPen {
fn drop(&mut self) {
// SAFETY: `self.fd` is still open (OwnedFd closes only after this body returns);
// UI_DEV_DESTROY takes no pointer argument. Errors are moot on teardown.
let _ = unsafe { libc::ioctl(self.fd.as_raw_fd(), UI_DEV_DESTROY, 0) };
}
}
@@ -0,0 +1,668 @@
//! Windows synthetic-pointer injection (design/pen-tablet-input.md §6): a per-session `PT_PEN`
//! device carrying the pen plane's full fidelity — pressure (rescaled to Windows' 0..1024),
//! polar tilt → tiltX/tiltY, barrel roll on `rotation` (0..359 — Windows Ink renders Pencil
//! Pro roll natively), barrel button, eraser (`PEN_FLAG_INVERTED`/`ERASER`), hover — plus a
//! `PT_TOUCH` device that closes the long-standing SendInput touch no-op for wire touches.
//!
//! Both follow Apollo's proven recipe (`design/apollo-comparison.md`): synthetic pointer state
//! goes STALE if not re-injected (~100 ms), so each device runs a small refresh thread that
//! re-asserts the last frame every [`REFRESH_MS`] while the pen is in range / contacts are
//! held — a stationary stylus must not hover-out (and a held finger must not auto-lift) just
//! because no new samples arrived. `CreateSyntheticPointerDevice` needs Win10 1809+;
//! [`crate::pen_supported`] probes it, so older hosts simply never advertise pen.
//!
//! Frame grouping mirrors the Linux uinput backend: a proximity-enter is injected together
//! with its position (never at a stale point), tip edges get their own DOWN/UP frames, and a
//! range-leave is a final frame without `INRANGE`.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{Context, Result};
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{
PenSample, PenTool, PenTransition, PEN_ANGLE_UNKNOWN, PEN_BARREL1, PEN_TILT_UNKNOWN,
};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use windows::Win32::Foundation::POINT;
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, GetThreadDesktop, OpenInputDesktop, SetThreadDesktop, DESKTOP_ACCESS_FLAGS,
DESKTOP_CONTROL_FLAGS, HDESK,
};
use windows::Win32::System::Threading::GetCurrentThreadId;
use windows::Win32::UI::Controls::{
CreateSyntheticPointerDevice, DestroySyntheticPointerDevice, HSYNTHETICPOINTERDEVICE,
POINTER_FEEDBACK_DEFAULT, POINTER_TYPE_INFO, POINTER_TYPE_INFO_0,
};
use windows::Win32::UI::Input::Pointer::{
InjectSyntheticPointerInput, POINTER_FLAGS, POINTER_FLAG_DOWN, POINTER_FLAG_FIRSTBUTTON,
POINTER_FLAG_INCONTACT, POINTER_FLAG_INRANGE, POINTER_FLAG_NEW, POINTER_FLAG_UP,
POINTER_FLAG_UPDATE, POINTER_INFO, POINTER_PEN_INFO, POINTER_TOUCH_INFO,
};
use windows::Win32::UI::WindowsAndMessaging::{
GetSystemMetrics, PEN_FLAG_BARREL, PEN_FLAG_ERASER, PEN_FLAG_INVERTED, PEN_FLAG_NONE,
PEN_MASK_PRESSURE, PEN_MASK_ROTATION, PEN_MASK_TILT_X, PEN_MASK_TILT_Y, PT_PEN, PT_TOUCH,
SM_CXVIRTUALSCREEN, SM_CYVIRTUALSCREEN, SM_XVIRTUALSCREEN, SM_YVIRTUALSCREEN, TOUCH_FLAG_NONE,
TOUCH_MASK_NONE,
};
/// Re-inject cadence while state is held (in-range pen / touching contacts). Apollo uses
/// 50 ms against the ~100 ms staleness window; 40 ms keeps two refreshes inside it.
const REFRESH_MS: u64 = 40;
/// `GENERIC_ALL` for the desktop open — the `windows` crate models desktop rights as their own
/// flag type and doesn't re-export the generic ones (same constant `sendinput.rs` uses).
const DESKTOP_GENERIC_ALL: u32 = 0x1000_0000;
/// This thread's binding to the input desktop, restored on drop.
///
/// `SendInput` (`sendinput.rs`) solves the same problem by binding its DEDICATED injector thread
/// once and keeping it. That can't be borrowed here: `inject_pen`/`inject_touch_frame` are driven
/// from TWO threads — the caller's `apply_batch` and the `pf-pen-refresh`/`pf-touch-refresh`
/// staleness threads — and the batch caller is a shared task thread, which must not be left parked
/// on a `Winlogon` desktop that disappears when the prompt is dismissed. So the binding is scoped
/// to the retry: `GetThreadDesktop` hands back a BORROWED handle (never closed), and only the
/// `OpenInputDesktop` handle is closed, after the thread has moved back off it.
struct InputDesktopBinding {
previous: HDESK,
input: HDESK,
}
impl InputDesktopBinding {
fn bind() -> Option<Self> {
// SAFETY: FFI calls taking by-value args only. `OpenInputDesktop` yields an owned `HDESK`
// solely on `Ok`; it is either installed by `SetThreadDesktop` (then owned by this guard,
// closed exactly once in `Drop`) or closed here on failure — closed once on every path,
// never used after. `SetThreadDesktop` rebinds only the calling thread, which owns no
// windows or hooks, so it cannot fail on that account.
unsafe {
let previous = GetThreadDesktop(GetCurrentThreadId()).ok()?;
let input = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(DESKTOP_GENERIC_ALL),
)
.ok()?;
if SetThreadDesktop(input).is_err() {
let _ = CloseDesktop(input);
return None;
}
Some(Self { previous, input })
}
}
}
impl Drop for InputDesktopBinding {
fn drop(&mut self) {
// SAFETY: `previous` is the borrowed desktop this thread started on, `input` the handle
// this guard uniquely owns. The thread moves back FIRST, so the handle is not the thread's
// desktop when closed — closed exactly once, never used after.
unsafe {
let _ = SetThreadDesktop(self.previous);
let _ = CloseDesktop(self.input);
}
}
}
/// Inject one synthetic-pointer frame, following the input desktop if Windows refuses it.
///
/// While a UAC consent prompt (or the lock / logon screen) owns input, the secure desktop does, and
/// injection from the host's own `WinSta0\Default` thread comes back `ERROR_ACCESS_DENIED` — which
/// is why pen and touch went dead on a prompt a user could SEE in the stream (capture already
/// renders it, and `SendInput` mouse/keyboard already followed the switch, so only these two were
/// left behind). Field-reported 2026-07-23; the exact rc reproduced in a probe the same night.
///
/// Measured then, with a real consent prompt up and a SYSTEM host in the console session:
///
/// ```text
/// device created on Default, thread on Default -> 0x80070005 (the field failure)
/// device created on Default, thread on INPUT desktop -> OK
/// device created on INPUT, thread on INPUT desktop -> OK
/// ```
///
/// The middle row is the load-bearing one: the synthetic pointer device is NOT desktop-affine, so
/// rebinding the THREAD is sufficient and the device never has to be destroyed and recreated
/// across a desktop switch (which would drop in-flight contacts and the pen's in-range state).
///
/// # Safety
/// `dev` must be a live synthetic-pointer device and `frame` a live slice for the call.
unsafe fn inject_following_desktop(
dev: HSYNTHETICPOINTERDEVICE,
frame: &[POINTER_TYPE_INFO],
) -> windows::core::Result<()> {
// SAFETY: per this fn's contract — `dev` is live and `frame` outlives the call, which only
// reads it. Best-effort, exactly as the direct call was.
match InjectSyntheticPointerInput(dev, frame) {
Ok(()) => Ok(()),
Err(first) => {
// Only a desktop switch is worth a rebind; anything else would just fail identically.
let Some(_binding) = InputDesktopBinding::bind() else {
return Err(first);
};
// SAFETY: same live `dev`/`frame`, re-issued with this thread on the input desktop.
InjectSyntheticPointerInput(dev, frame)
}
}
}
/// Windows pen pressure is 0..1024 (vs the wire's full-scale u16).
const WIN_PEN_PRESSURE_MAX: u32 = 1024;
/// Map a normalized [0,1] coordinate pair onto virtual-desktop pixels — the same surface the
/// SendInput absolute mouse targets, so pen, touch, and pointer all land identically.
fn to_screen(x: f32, y: f32) -> POINT {
// SAFETY: `GetSystemMetrics` takes a constant index and reads global metrics; no memory in.
let (vx, vy, vw, vh) = unsafe {
(
GetSystemMetrics(SM_XVIRTUALSCREEN),
GetSystemMetrics(SM_YVIRTUALSCREEN),
GetSystemMetrics(SM_CXVIRTUALSCREEN).max(1),
GetSystemMetrics(SM_CYVIRTUALSCREEN).max(1),
)
};
POINT {
x: vx + (x.clamp(0.0, 1.0) * (vw - 1) as f32) as i32,
y: vy + (y.clamp(0.0, 1.0) * (vh - 1) as f32) as i32,
}
}
/// An owned `HSYNTHETICPOINTERDEVICE` (destroyed exactly once on drop).
struct Device(HSYNTHETICPOINTERDEVICE);
// SAFETY: the handle is a plain kernel object identifier with no thread affinity —
// `InjectSyntheticPointerInput`/`DestroySyntheticPointerDevice` are documented callable from
// any thread; ownership transfer/sharing does not alias memory.
unsafe impl Send for Device {}
impl Drop for Device {
fn drop(&mut self) {
// SAFETY: `self.0` is the device this wrapper uniquely owns; destroyed once here.
unsafe { DestroySyntheticPointerDevice(self.0) };
}
}
/// Probe: can this Windows build create a synthetic pen device (Win10 1809+)?
pub fn synthetic_pen_available() -> bool {
// SAFETY: FFI create with by-value args; on success the returned handle is destroyed
// immediately by the `Device` wrapper, exactly once.
match unsafe { CreateSyntheticPointerDevice(PT_PEN, 1, POINTER_FEEDBACK_DEFAULT) } {
Ok(h) => {
drop(Device(h));
true
}
Err(_) => false,
}
}
/// The tracked pen state a refresh tick re-asserts.
#[derive(Default)]
struct PenState {
in_range: bool,
touching: bool,
barrel: bool,
eraser: bool,
x: f32,
y: f32,
pressure: u16,
tilt_deg: u8,
azimuth_deg: u16,
roll_deg: u16,
}
struct PenShared {
dev: Device,
state: PenState,
/// First injection failure logs at WARN (see `TouchShared::fail_warned`).
fail_warned: bool,
}
/// One per-session virtual pen (the Windows sibling of the Linux uinput tablet — same
/// [`PenTransition`] consumer API). Wire barrel button 2 has no Windows pen equivalent
/// (one barrel + eraser is the platform model) and is ignored here.
pub struct VirtualPen {
shared: Arc<Mutex<PenShared>>,
stop: Arc<AtomicBool>,
refresher: Option<std::thread::JoinHandle<()>>,
// Batch-local frame grouping (single-threaded within apply_batch).
edge_down: bool,
edge_up: bool,
is_new: bool,
frame_dirty: bool,
frame_has_motion: bool,
}
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
// SAFETY: FFI create with by-value args; the handle's sole owner becomes `Device`.
let dev = unsafe { CreateSyntheticPointerDevice(PT_PEN, 1, POINTER_FEEDBACK_DEFAULT) }
.context("CreateSyntheticPointerDevice(PT_PEN) — needs Windows 10 1809+")?;
let shared = Arc::new(Mutex::new(PenShared {
dev: Device(dev),
state: PenState::default(),
fail_warned: false,
}));
let stop = Arc::new(AtomicBool::new(false));
// The staleness guard: re-assert the last frame while in range so a stationary pen
// (native plane: between 100 ms heartbeats; GameStream: indefinitely) never hovers out.
let refresher = {
let shared = Arc::clone(&shared);
let stop = Arc::clone(&stop);
std::thread::Builder::new()
.name("pf-pen-refresh".into())
.spawn(move || {
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(std::time::Duration::from_millis(REFRESH_MS));
let s = &mut *shared.lock().unwrap();
if s.state.in_range {
inject_pen(s, POINTER_FLAG_UPDATE, false);
}
}
})
.context("spawn pen refresh thread")?
};
tracing::info!("virtual pen created (Windows synthetic pointer, PT_PEN)");
Ok(VirtualPen {
shared,
stop,
refresher: Some(refresher),
edge_down: false,
edge_up: false,
is_new: false,
frame_dirty: false,
frame_has_motion: false,
})
}
/// Apply one batch of tracker transitions — same grouping contract as the Linux backend:
/// `[ProxIn, Motion, TipDown]` is ONE frame (entry lands at its position, in contact),
/// consecutive `Motion`s split, tip edges own their frames, range-leave is a final
/// no-`INRANGE` frame.
pub fn apply_batch(&mut self, transitions: &[PenTransition]) {
for t in transitions {
match t {
PenTransition::ProximityIn { tool } => {
self.flush();
let mut s = self.shared.lock().unwrap();
s.state.in_range = true;
s.state.eraser = *tool == PenTool::Eraser;
drop(s);
self.is_new = true;
self.frame_dirty = true;
}
PenTransition::Motion { sample } => {
if self.frame_has_motion {
self.flush();
}
self.set_axes(sample);
self.frame_dirty = true;
self.frame_has_motion = true;
}
PenTransition::TipDown => {
self.shared.lock().unwrap().state.touching = true;
self.edge_down = true;
self.frame_dirty = true;
}
PenTransition::ButtonsChanged { pressed, released } => {
let mut s = self.shared.lock().unwrap();
if pressed & PEN_BARREL1 != 0 {
s.state.barrel = true;
}
if released & PEN_BARREL1 != 0 {
s.state.barrel = false;
}
drop(s);
self.frame_dirty = true;
}
PenTransition::TipUp => {
self.shared.lock().unwrap().state.touching = false;
self.edge_up = true;
self.frame_dirty = true;
self.flush(); // UP owns its frame (still INRANGE)
}
PenTransition::ProximityOut => {
self.shared.lock().unwrap().state.in_range = false;
self.frame_dirty = true;
self.flush(); // final frame without INRANGE
}
}
}
self.flush();
}
fn set_axes(&mut self, s: &PenSample) {
let mut sh = self.shared.lock().unwrap();
sh.state.x = s.x;
sh.state.y = s.y;
sh.state.pressure = s.pressure;
sh.state.tilt_deg = s.tilt_deg;
sh.state.azimuth_deg = s.azimuth_deg;
sh.state.roll_deg = s.roll_deg;
}
fn flush(&mut self) {
if !self.frame_dirty {
return;
}
let edge = if self.edge_down {
POINTER_FLAG_DOWN
} else if self.edge_up {
POINTER_FLAG_UP
} else {
POINTER_FLAG_UPDATE
};
inject_pen(&mut self.shared.lock().unwrap(), edge, self.is_new);
self.edge_down = false;
self.edge_up = false;
self.is_new = false;
self.frame_dirty = false;
self.frame_has_motion = false;
}
}
impl Drop for VirtualPen {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(h) = self.refresher.take() {
let _ = h.join();
}
// The device itself dies with `shared` (Device::drop) — Windows releases any held
// in-range/contact state when the synthetic device is destroyed.
}
}
/// Build + inject one pen frame from tracked state. `edge` is DOWN/UP/UPDATE;
/// `INRANGE`/`INCONTACT` derive from the state itself.
fn inject_pen(sh: &mut PenShared, edge: POINTER_FLAGS, is_new: bool) {
let st = &sh.state;
let mut flags = edge;
if st.in_range {
flags |= POINTER_FLAG_INRANGE;
}
if st.touching {
flags |= POINTER_FLAG_INCONTACT | POINTER_FLAG_FIRSTBUTTON;
}
if is_new {
flags |= POINTER_FLAG_NEW;
}
let mut pen_flags = PEN_FLAG_NONE;
if st.barrel {
pen_flags |= PEN_FLAG_BARREL;
}
if st.eraser {
pen_flags |= PEN_FLAG_INVERTED;
if st.touching {
pen_flags |= PEN_FLAG_ERASER;
}
}
let mut pen_mask = PEN_MASK_PRESSURE;
// Contact needs a nonzero pressure to ink; hover reports 0 (Windows convention).
let pressure = if st.touching {
((st.pressure as u32 * WIN_PEN_PRESSURE_MAX) / u16::MAX as u32).max(1)
} else {
0
};
let (mut tilt_x, mut tilt_y) = (0i32, 0i32);
if st.tilt_deg != PEN_TILT_UNKNOWN && st.azimuth_deg != PEN_ANGLE_UNKNOWN {
let az = (st.azimuth_deg as f32).to_radians();
let tilt = st.tilt_deg as f32;
tilt_x = (tilt * az.sin()).round() as i32;
tilt_y = (-tilt * az.cos()).round() as i32;
pen_mask |= PEN_MASK_TILT_X | PEN_MASK_TILT_Y;
}
let mut rotation = 0u32;
if st.roll_deg != PEN_ANGLE_UNKNOWN {
rotation = (st.roll_deg % 360) as u32;
pen_mask |= PEN_MASK_ROTATION;
}
let pt = to_screen(st.x, st.y);
let info = POINTER_TYPE_INFO {
r#type: PT_PEN,
Anonymous: POINTER_TYPE_INFO_0 {
penInfo: POINTER_PEN_INFO {
pointerInfo: POINTER_INFO {
pointerType: PT_PEN,
pointerId: 0,
pointerFlags: flags,
ptPixelLocation: pt,
ptPixelLocationRaw: pt,
..Default::default()
},
penFlags: pen_flags,
penMask: pen_mask,
pressure,
rotation,
tiltX: tilt_x,
tiltY: tilt_y,
},
},
};
// SAFETY: `sh.dev.0` is the live device this wrapper owns; the one-element array is a live
// stack value the call only reads. Best-effort like every injector write — a transient
// failure (desktop switch) is healed by the next refresh tick re-asserting state.
if let Err(e) = unsafe { inject_following_desktop(sh.dev.0, &[info]) } {
if !sh.fail_warned {
sh.fail_warned = true;
tracing::warn!(
error = %e,
flags = format!("{:#x}", flags.0),
pen_flags = format!("{pen_flags:#x}"),
pen_mask = format!("{pen_mask:#x}"),
pressure,
rotation,
tilt_x,
tilt_y,
x = pt.x,
y = pt.y,
"pen inject failed"
);
} else {
tracing::trace!(error = %e, "pen inject failed (transient)");
}
} else {
sh.fail_warned = false;
}
}
/// One live wire-touch contact. `slot` is the SMALL, DENSE pointer id handed to Windows —
/// synthetic-pointer injection rejects arbitrary large ids, and clients (Moonlight's
/// `pointerId` especially) send exactly those, so wire ids compact into the lowest free slot
/// for the contact's lifetime (Apollo's slot-contiguity rule; the on-glass symptom of passing
/// wire ids through was pen working while every touch silently failed to inject).
#[derive(Clone, Copy)]
struct Contact {
id: u32,
slot: u32,
x: f32,
y: f32,
}
/// Lowest slot not held by a live contact.
fn free_slot(contacts: &[Contact]) -> u32 {
let mut slot = 0u32;
while contacts.iter().any(|c| c.slot == slot) {
slot += 1;
}
slot
}
/// Windows can inject at most this many simultaneous synthetic touch contacts.
const MAX_CONTACTS: usize = 10;
struct TouchShared {
dev: Device,
contacts: Vec<Contact>,
/// First injection failure logs at WARN (an on-glass "touch does nothing" must be
/// visible in the host log); repeats stay at trace.
fail_warned: bool,
}
/// The `PT_TOUCH` device servicing wire `TouchDown/Move/Up` events — closes the SendInput
/// touch no-op. Contacts are keyed by the wire's finger id; every frame re-injects the FULL
/// active set (the synthetic-pointer contract), and a refresh thread re-asserts held contacts
/// against the ~100 ms staleness auto-lift.
pub struct SyntheticTouch {
shared: Arc<Mutex<TouchShared>>,
stop: Arc<AtomicBool>,
refresher: Option<std::thread::JoinHandle<()>>,
}
impl SyntheticTouch {
pub fn create() -> Result<SyntheticTouch> {
// SAFETY: FFI create with by-value args; the handle's sole owner becomes `Device`.
let dev = unsafe {
CreateSyntheticPointerDevice(PT_TOUCH, MAX_CONTACTS as u32, POINTER_FEEDBACK_DEFAULT)
}
.context("CreateSyntheticPointerDevice(PT_TOUCH) — needs Windows 10 1809+")?;
let shared = Arc::new(Mutex::new(TouchShared {
dev: Device(dev),
contacts: Vec::new(),
fail_warned: false,
}));
let stop = Arc::new(AtomicBool::new(false));
let refresher = {
let shared = Arc::clone(&shared);
let stop = Arc::clone(&stop);
std::thread::Builder::new()
.name("pf-touch-refresh".into())
.spawn(move || {
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(std::time::Duration::from_millis(REFRESH_MS));
let s = &mut *shared.lock().unwrap();
if !s.contacts.is_empty() {
inject_touch_frame(s, None);
}
}
})
.context("spawn touch refresh thread")?
};
tracing::info!("virtual touchscreen created (Windows synthetic pointer, PT_TOUCH)");
Ok(SyntheticTouch {
shared,
stop,
refresher: Some(refresher),
})
}
/// Apply one wire touch event (`code` = finger id, pixel x/y against the
/// `flags = (w << 16) | h` reference extent, exactly like `MouseMoveAbs`).
pub fn apply(&mut self, ev: &InputEvent) {
let (w, h) = ((ev.flags >> 16) as f32, (ev.flags & 0xFFFF) as f32);
if (w < 1.0 || h < 1.0) && ev.kind != InputKind::TouchUp {
return; // the documented zero-extent drop, as for MouseMoveAbs
}
let (x, y) = (ev.x as f32 / w.max(1.0), ev.y as f32 / h.max(1.0));
let s = &mut *self.shared.lock().unwrap();
match ev.kind {
InputKind::TouchDown => {
match s.contacts.iter().position(|c| c.id == ev.code) {
Some(i) => (s.contacts[i].x, s.contacts[i].y) = (x, y),
None if s.contacts.len() < MAX_CONTACTS => {
let slot = free_slot(&s.contacts);
s.contacts.push(Contact {
id: ev.code,
slot,
x,
y,
})
}
None => return, // beyond the platform max — drop, never evict a live finger
}
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_DOWN)));
}
InputKind::TouchMove => {
match s.contacts.iter().position(|c| c.id == ev.code) {
Some(i) => {
(s.contacts[i].x, s.contacts[i].y) = (x, y);
inject_touch_frame(s, None);
}
// A move for an unknown id (its DOWN was dropped/lost): synthesize the
// contact so the stroke self-heals, like the pen plane does.
None if s.contacts.len() < MAX_CONTACTS => {
let slot = free_slot(&s.contacts);
s.contacts.push(Contact {
id: ev.code,
slot,
x,
y,
});
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_DOWN)));
}
None => {}
}
}
InputKind::TouchUp => {
let Some(idx) = s.contacts.iter().position(|c| c.id == ev.code) else {
return;
};
// The UP frame still carries the lifting contact (with UP flags), then it
// leaves the active set.
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_UP)));
s.contacts.remove(idx);
}
_ => {}
}
}
}
impl Drop for SyntheticTouch {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(h) = self.refresher.take() {
let _ = h.join();
}
}
}
/// Inject the full active-contact frame; `edge` marks one contact's DOWN/UP transition (by
/// WIRE id — everyone else is a held UPDATE). Windows sees the compacted `slot` ids only.
fn inject_touch_frame(sh: &mut TouchShared, edge: Option<(u32, POINTER_FLAGS)>) {
let contacts = &sh.contacts;
if contacts.is_empty() {
return;
}
let mut frame: Vec<POINTER_TYPE_INFO> = Vec::with_capacity(contacts.len());
for c in contacts {
let mut flags = POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT | POINTER_FLAG_UPDATE;
if let Some((id, e)) = edge {
if id == c.id {
if e == POINTER_FLAG_DOWN {
flags = POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT | POINTER_FLAG_DOWN;
} else if e == POINTER_FLAG_UP {
flags = POINTER_FLAG_UP; // contact + range end together
}
}
}
let pt = to_screen(c.x, c.y);
frame.push(POINTER_TYPE_INFO {
r#type: PT_TOUCH,
Anonymous: POINTER_TYPE_INFO_0 {
touchInfo: POINTER_TOUCH_INFO {
pointerInfo: POINTER_INFO {
pointerType: PT_TOUCH,
pointerId: c.slot,
pointerFlags: flags,
ptPixelLocation: pt,
ptPixelLocationRaw: pt,
..Default::default()
},
touchFlags: TOUCH_FLAG_NONE,
touchMask: TOUCH_MASK_NONE,
..Default::default()
},
},
});
}
// SAFETY: `sh.dev.0` is the live owned device; `frame` is a live Vec the call only reads.
// Best-effort — a transient failure heals on the next event/refresh re-assertion.
if let Err(e) = unsafe { inject_following_desktop(sh.dev.0, &frame) } {
if !sh.fail_warned {
sh.fail_warned = true;
tracing::warn!(error = %e, contacts = frame.len(), "touch inject failed");
} else {
tracing::trace!(error = %e, "touch inject failed (transient)");
}
} else {
sh.fail_warned = false;
}
}
@@ -46,6 +46,11 @@ const XBUTTON2: u32 = 0x0002;
pub struct SendInputInjector {
desktop: Option<HDESK>,
/// PT_TOUCH synthetic device, created on the first wire-touch event (a session that never
/// touches never creates one). `None` after a failed create (pre-1809) — touch then stays
/// the historical no-op.
touch: Option<crate::pen::SyntheticTouch>,
touch_failed: bool,
}
// SAFETY: `SendInputInjector` holds only an `Option<HDESK>` (a desktop handle). The host creates
@@ -58,7 +63,11 @@ unsafe impl Send for SendInputInjector {}
impl SendInputInjector {
pub fn open() -> Result<Self> {
let mut me = Self { desktop: None };
let mut me = Self {
desktop: None,
touch: None,
touch_failed: false,
};
me.reattach_input_desktop(); // best-effort
tracing::info!("SendInput injector ready (Win32 KeyboardAndMouse)");
Ok(me)
@@ -324,15 +333,33 @@ impl InputInjector for SendInputInjector {
}
self.send(&inputs)
}
// Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op.
// Gamepad goes through the XUSB backend.
InputKind::GamepadButton
| InputKind::GamepadAxis
| InputKind::GamepadState
| InputKind::GamepadRemove
| InputKind::GamepadArrival
| InputKind::TouchDown
| InputKind::TouchMove
| InputKind::TouchUp => Ok(()),
| InputKind::GamepadArrival => Ok(()),
// Wire touch → the PT_TOUCH synthetic pointer device (design/pen-tablet-input.md
// §6; closes the historical SendInput no-op). Lazily created — a session that never
// touches never creates one; a pre-1809 create failure latches back to the no-op.
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp => {
if self.touch.is_none() && !self.touch_failed {
match crate::pen::SyntheticTouch::create() {
Ok(t) => self.touch = Some(t),
Err(e) => {
self.touch_failed = true;
tracing::warn!(
error = %format!("{e:#}"),
"touch: synthetic pointer unavailable — wire touch stays a no-op"
);
}
}
}
if let Some(t) = self.touch.as_mut() {
t.apply(event);
}
Ok(())
}
}
}
}
+68
View File
@@ -197,6 +197,50 @@ pub fn text_input_supported() -> bool {
false
}
/// Whether this host can inject full-fidelity stylus input (`HOST_CAP_PEN` —
/// design/pen-tablet-input.md): Linux only, via the [`pen::VirtualPen`] uinput tablet, so the
/// probe is "can we open /dev/uinput" (the same permission the virtual gamepads need) plus the
/// `PUNKTFUNK_PEN=0` operator kill-switch. Consulted at Welcome time; clients without the bit
/// keep folding pen into touch/pointer. Windows PT_PEN synthetic pointers are the design's P3.
#[cfg(target_os = "linux")]
pub fn pen_supported() -> bool {
if std::env::var("PUNKTFUNK_PEN").as_deref() == Ok("0") {
return false;
}
// SAFETY: 'static NUL-terminated path literal; `open` returns a fresh fd (or -1) and
// retains nothing.
let fd = unsafe {
libc::open(
c"/dev/uinput".as_ptr(),
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if fd < 0 {
return false;
}
// SAFETY: `fd >= 0` is the fd opened above, owned by no one else; closed exactly once here.
unsafe { libc::close(fd) };
true
}
/// Windows: pen (and touch) inject via synthetic pointer devices — available on Win10 1809+,
/// probed by actually creating (and immediately destroying) a PT_PEN device. Same
/// `PUNKTFUNK_PEN=0` kill-switch as Linux. The probe result also stands in for PT_TOUCH
/// (both APIs arrived together in 1809).
#[cfg(target_os = "windows")]
pub fn pen_supported() -> bool {
if std::env::var("PUNKTFUNK_PEN").as_deref() == Ok("0") {
return false;
}
pen::synthetic_pen_available()
}
/// See the Linux/Windows variants — no pen injection elsewhere.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn pen_supported() -> bool {
false
}
#[path = "inject/service.rs"]
mod service;
pub use service::InjectorService;
@@ -362,6 +406,30 @@ pub mod gamepad {
pub fn pump_rumble(&mut self, _send: impl FnMut(u16, u16, u16)) {}
}
}
/// Linux: the "Punktfunk Pen" uinput virtual tablet (design/pen-tablet-input.md §5) — the
/// per-session stylus device the native pen plane injects through.
#[cfg(target_os = "linux")]
#[path = "inject/linux/pen.rs"]
pub mod pen;
/// Windows: PT_PEN/PT_TOUCH synthetic pointer devices (design/pen-tablet-input.md §6).
/// `pen::VirtualPen` here is the PT_PEN device; `pen::SyntheticTouch` backs the SendInput
/// injector's wire-touch path.
#[cfg(target_os = "windows")]
#[path = "inject/windows/pointer_windows.rs"]
pub mod pen;
/// Stub — pen injection needs the Linux uinput tablet or Windows synthetic pointers;
/// `pen_supported()` is false here, so no host advertises the cap and no batches arrive.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub mod pen {
use anyhow::{bail, Result};
pub struct VirtualPen;
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
bail!("no pen injection backend on this platform")
}
pub fn apply_batch(&mut self, _transitions: &[punktfunk_core::quic::PenTransition]) {}
}
}
#[cfg(target_os = "linux")]
#[path = "inject/linux/kwin_fake_input.rs"]
mod kwin_fake_input;
+144 -37
View File
@@ -5,10 +5,19 @@
//! negotiated it (`HOST_CAP_CURSOR` in the Welcome — the host stopped compositing then) and
//! only applied while the DESKTOP mouse model is engaged: under capture the pointer is
//! relative-locked (SDL hides it) and games draw their own cursor in-frame.
//!
//! The host sends the bitmap in host-FRAMEBUFFER pixels, whose size tracks the host virtual
//! display's DPI scaling (32 px at 100%, 96 px at 300%). Drawn 1:1 it balloons on a high-DPI
//! host; instead we scale it by the SAME aspect-fit factor the video is drawn at
//! (`min(window_px/mode)`), so the pointer stays sized to the streamed desktop at any host
//! scaling. SDL cursors are fixed-size from their surface (no draw-time scaling), so we cache
//! shapes RAW and resample per install — rebuilding when the serial OR the fit changes.
use punktfunk_core::client::NativeClient;
use punktfunk_core::quic::{CursorState, HOST_CAP_CURSOR};
use sdl3::mouse::{Cursor, MouseUtil, SystemCursor};
use sdl3::pixels::PixelFormat;
use sdl3::surface::Surface;
use std::collections::HashMap;
use std::time::Duration;
@@ -17,14 +26,27 @@ use std::time::Duration;
/// on the reliable stream via the serial-miss path).
const SHAPE_CACHE_MAX: usize = 64;
/// A forwarded cursor shape held RAW (host-framebuffer-pixel bytes + hotspot), so it can be
/// rebuilt into a scaled OS cursor whenever the video-fit changes (a window resize). Caching a
/// fixed-size `Cursor` instead would freeze the pointer at its build-time size.
struct RawShape {
rgba: Vec<u8>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
}
pub struct CursorChannel {
/// The Welcome carried `HOST_CAP_CURSOR` — the host forwards instead of compositing.
negotiated: bool,
/// Serial → built OS cursor. An SDL `Cursor` must outlive its `set()`, so the cache owns
/// every shape ever applied this session (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, Cursor>,
/// The serial currently installed via `Cursor::set` (`None` = default/system cursor).
installed: Option<u32>,
/// Serial → raw forwarded shape (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, RawShape>,
/// The serial + fit scale the currently-installed OS cursor was built at (`None` =
/// default/system cursor). A change in EITHER forces a rebuild.
installed: Option<(u32, f32)>,
/// Keeps the installed `Cursor` alive — SDL requires it to outlive its `set()`.
installed_cursor: Option<Cursor>,
/// Latest `0xD0` state (latest-wins across a drained batch).
state: Option<CursorState>,
}
@@ -39,6 +61,7 @@ impl CursorChannel {
negotiated,
shapes: HashMap::new(),
installed: None,
installed_cursor: None,
state: None,
}
}
@@ -57,8 +80,16 @@ impl CursorChannel {
/// Drain the two planes and apply the newest state — once per run-loop iteration.
/// `desktop_active` = the desktop mouse model is engaged (captured + desktop): only then
/// do we own the local cursor's shape/visibility; under capture SDL's relative mode owns
/// it, and released the system cursor must look normal.
pub fn pump(&mut self, connector: &NativeClient, mouse: &MouseUtil, desktop_active: bool) {
/// it, and released the system cursor must look normal. `fit_scale` is host-framebuffer
/// pixels → window pixels (the aspect-fit factor the video is drawn at); the shape is
/// resampled by it so the pointer matches the streamed desktop at any host DPI.
pub fn pump(
&mut self,
connector: &NativeClient,
mouse: &MouseUtil,
desktop_active: bool,
fit_scale: f32,
) {
if !self.negotiated {
return;
}
@@ -68,30 +99,25 @@ impl CursorChannel {
self.shapes.clear();
self.installed = None;
}
let mut data = shape.rgba;
let built = sdl3::surface::Surface::from_data(
&mut data,
shape.w as u32,
shape.h as u32,
shape.w as u32 * 4,
sdl3::pixels::PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())
.and_then(|surf| {
Cursor::from_surface(&surf, shape.hot_x as i32, shape.hot_y as i32)
.map_err(|e| e.to_string())
});
match built {
Ok(cursor) => {
// A re-sent serial replaces its entry; force re-install if it's current.
if self.installed == Some(shape.serial) {
self.installed = None;
}
self.shapes.insert(shape.serial, cursor);
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
let (w, h) = (shape.w as u32, shape.h as u32);
if w == 0 || h == 0 || shape.rgba.len() < (w * h * 4) as usize {
tracing::warn!(w, h, "cursor shape malformed — ignored");
continue;
}
// A re-sent serial replaces its entry; force re-install if it's current.
if matches!(self.installed, Some((s, _)) if s == shape.serial) {
self.installed = None;
}
self.shapes.insert(
shape.serial,
RawShape {
rgba: shape.rgba,
w,
h,
hot_x: shape.hot_x as u32,
hot_y: shape.hot_y as u32,
},
);
}
while let Ok(st) = connector.next_cursor_state(Duration::ZERO) {
self.state = Some(st); // latest wins
@@ -101,17 +127,25 @@ impl CursorChannel {
// Capture mode / released: hand the cursor back to the system default so a
// released pointer over the window doesn't wear the host's shape.
if self.installed.take().is_some() {
Cursor::from_system(SystemCursor::Arrow)
.map(|c| c.set())
.ok();
if let Ok(c) = Cursor::from_system(SystemCursor::Arrow) {
c.set();
self.installed_cursor = Some(c); // keep it alive past set()
}
}
return;
}
let Some(st) = self.state else { return };
if st.visible() && self.installed != Some(st.serial) {
if let Some(cursor) = self.shapes.get(&st.serial) {
cursor.set();
self.installed = Some(st.serial);
if st.visible() && self.installed != Some((st.serial, fit_scale)) {
if let Some(shape) = self.shapes.get(&st.serial) {
match build_scaled_cursor(shape, fit_scale) {
Ok(cursor) => {
cursor.set();
self.installed = Some((st.serial, fit_scale));
self.installed_cursor = Some(cursor); // outlive set()
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
}
}
// Serial miss: the (reliable) shape hasn't landed yet — keep the previous
// cursor for the RTT rather than flashing default.
@@ -123,3 +157,76 @@ impl CursorChannel {
}
}
}
/// Resample a raw shape by `fit_scale` and build an SDL color cursor from it. The hotspot scales
/// with the bitmap so the click point stays true. `fit_scale <= 0` (or a degenerate result) is
/// clamped so we always hand SDL a ≥1×1 surface.
fn build_scaled_cursor(shape: &RawShape, fit_scale: f32) -> Result<Cursor, String> {
let scale = if fit_scale.is_finite() && fit_scale > 0.0 {
fit_scale
} else {
1.0
};
let dw = ((shape.w as f32 * scale).round() as u32).max(1);
let dh = ((shape.h as f32 * scale).round() as u32).max(1);
let hot_x = ((shape.hot_x as f32 * scale).round() as u32).min(dw - 1) as i32;
let hot_y = ((shape.hot_y as f32 * scale).round() as u32).min(dh - 1) as i32;
if dw == shape.w && dh == shape.h {
// 1:1 fit (mode == window) — no resample, feed the bytes straight through.
let mut data = shape.rgba.clone();
let surf = Surface::from_data(
&mut data,
shape.w,
shape.h,
shape.w * 4,
PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())?;
return Cursor::from_surface(&surf, hot_x, hot_y).map_err(|e| e.to_string());
}
let mut scaled = resample_rgba(&shape.rgba, shape.w, shape.h, dw, dh);
let surf = Surface::from_data(&mut scaled, dw, dh, dw * 4, PixelFormat::RGBA32)
.map_err(|e| e.to_string())?;
Cursor::from_surface(&surf, hot_x, hot_y).map_err(|e| e.to_string())
}
/// Area-average resample of a straight-alpha RGBA bitmap `(sw×sh) → (dw×dh)`. Averaging is done on
/// PREMULTIPLIED colour (weighting each source texel by its alpha) so transparent-pixel colour
/// can't bleed into the fringe, then un-premultiplied back to straight alpha. Handles both down-
/// and up-scale; for a cursor the common case is a downscale (high-DPI host → smaller pointer).
fn resample_rgba(src: &[u8], sw: u32, sh: u32, dw: u32, dh: u32) -> Vec<u8> {
let mut out = vec![0u8; (dw * dh * 4) as usize];
let fx = sw as f32 / dw as f32;
let fy = sh as f32 / dh as f32;
for dy in 0..dh {
let sy0 = (dy as f32 * fy).floor() as u32;
let sy1 = (((dy + 1) as f32 * fy).ceil() as u32).clamp(sy0 + 1, sh);
for dx in 0..dw {
let sx0 = (dx as f32 * fx).floor() as u32;
let sx1 = (((dx + 1) as f32 * fx).ceil() as u32).clamp(sx0 + 1, sw);
let (mut r, mut g, mut b, mut a_sum, mut n) = (0f32, 0f32, 0f32, 0f32, 0f32);
for sy in sy0..sy1 {
for sx in sx0..sx1 {
let i = ((sy * sw + sx) * 4) as usize;
let a = src[i + 3] as f32 / 255.0;
r += src[i] as f32 * a;
g += src[i + 1] as f32 * a;
b += src[i + 2] as f32 * a;
a_sum += a;
n += 1.0;
}
}
let di = ((dy * dw + dx) * 4) as usize;
if a_sum > 0.0 {
out[di] = (r / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 1] = (g / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 2] = (b / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 3] = (a_sum / n * 255.0).round().clamp(0.0, 255.0) as u8;
}
// else fully transparent — already zero-filled.
}
}
out
}
+9 -1
View File
@@ -773,12 +773,20 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// Cursor channel (M2): drain forwarded shape/state and drive the local OS cursor —
// only meaningful in the desktop mouse model (capture's relative lock hides it).
if let Some(st) = stream.as_mut() {
// Host-framebuffer px → window px: the aspect-fit factor the video is drawn at
// (same `min(surface/content)` as `finger_to_content`). The forwarded pointer is
// resampled by it so a high-DPI host's oversized bitmap lands sized to the streamed
// desktop rather than ballooning. 1:1 until the first frame gives `last_video`.
let fit_scale = st.last_video.map_or(1.0, |(vw, vh)| {
let (pw, ph) = window.size_in_pixels();
(pw as f32 / vw.max(1) as f32).min(ph as f32 / vh.max(1) as f32)
});
if let (Some(chan), Some(c)) = (st.cursor_chan.as_mut(), st.connector.as_ref()) {
let desktop_active = st
.capture
.as_ref()
.is_some_and(|cap| cap.captured() && cap.desktop());
chan.pump(c, &mouse, desktop_active);
chan.pump(c, &mouse, desktop_active, fit_scale);
// §8 mid-stream render flip: tell the host who renders the pointer whenever
// the local model changes. Desktop-active = we draw it (host excludes +
// forwards); anything else — the capture model OR a released pointer — the
+5
View File
@@ -58,6 +58,11 @@ pub(crate) fn emit_display_event(ev: DisplayEvent) {
pub(crate) mod backend;
pub use backend::{DisplayOwnership, VirtualDisplay, VirtualOutput};
/// Time-bounded child-process helpers — every compositor query shells out, and an unbounded one
/// can wedge the calling (session) thread forever.
#[path = "vdisplay/proc.rs"]
pub(crate) mod proc;
/// Live-session detection + session-epoch + env retargeting (plan §W3).
#[path = "vdisplay/session.rs"]
pub(crate) mod session;
+344 -101
View File
@@ -149,11 +149,7 @@ impl VirtualDisplay for KwinDisplay {
return;
};
// kscreen-doctor position syntax: `output.<name-or-id>.position.<x>,<y>`.
let ok = std::process::Command::new("kscreen-doctor")
.arg(format!("output.{output}.position.{x},{y}"))
.status()
.map(|s| s.success())
.unwrap_or(false);
let ok = kscreen_ok(&[format!("output.{output}.position.{x},{y}")]);
if ok {
tracing::info!(output, x, y, "KWin: placed output in the desktop layout");
} else {
@@ -210,14 +206,16 @@ impl VirtualDisplay for KwinDisplay {
// resize handling: when the source's texture size changes while recording, KWin re-runs
// `buildFormats` — picking up the output's CURRENT refresh — and renegotiates the live
// stream via `pw_stream_update_params`. So above 60 Hz the output is born at a
// SACRIFICIAL height: installing + selecting the real `WxH@hz` custom mode (supported on
// virtual outputs since KWin 6.6) then changes the SIZE, and the first buffers recorded
// after the consumer connects trigger KWin's resize → a renegotiation to `WxH@hz`. The
// SACRIFICIAL height: installing + selecting the real high-refresh custom mode (supported
// on virtual outputs since KWin 6.6) then changes the SIZE, and the first buffers recorded
// after the consumer connects trigger KWin's resize → a renegotiation to that mode. The
// capturer holds frames until that lands (`expect_exact_dims`), so the pipeline never
// builds against the birth mode. First cut shells out to kscreen-doctor; the in-process
// kde_output_management_v2 client is a follow-up. `set_custom_refresh` reads back what
// KWin *actually* gave so the encoder paces to the real source rate. At ≤60 Hz there's
// nothing to install — the output is born at the real size and 60 Hz is the offer anyway.
// KWin *actually* gave — both the rate (so the encoder paces to the real source) and the
// size, which KWin's CVT generator may have aligned down (see `CVT_H_GRANULARITY`). At
// ≤60 Hz there's nothing to install — the output is born at the real size and 60 Hz is the
// offer anyway.
let want_high = mode.refresh_hz > 60;
let birth_h = if want_high { height + 16 } else { height };
let (mut node_id, mut stop) = spawn_vout(width, birth_h)?;
@@ -241,36 +239,52 @@ impl VirtualDisplay for KwinDisplay {
let mut addr = resolve_kscreen_addr(&name, width, birth_h);
self.last_name = Some(addr.clone()); // for apply_position (registry-driven §6.2 layout)
let mut expect_exact_dims = false;
// The size the output actually ENDS UP at — the request, unless KWin's CVT generator had to
// shrink the width to the cell grain (see `CVT_H_GRANULARITY`). Reported as the output's
// `preferred_mode`, which is what the capturer's renegotiation gate waits for and what the
// encoder opens against, so a CVT-aligned mode flows end-to-end instead of starving.
let mut final_dims = (width, height);
let achieved_hz = if want_high {
let (achieved, size_applied) =
set_custom_refresh(width, height, mode.refresh_hz, &addr);
if size_applied {
// Real mode selected: the recording stream will renegotiate to it (see above).
expect_exact_dims = true;
achieved
} else {
// Custom-mode install/select rejected (pre-6.6 KWin / stale kscreen-doctor): the
// output is STUCK at the sacrificial birth size — unusable. Recreate plain at the
// real size (the pre-sacrifice behavior: correct size, KWin's native 60 Hz).
tracing::warn!(
"KWin rejected the custom mode — recreating the virtual output at the real \
size (60 Hz ceiling on this KWin)"
);
stop.store(true, Ordering::Relaxed);
// Let KWin retire the doomed output before re-using its name.
std::thread::sleep(Duration::from_millis(300));
let (nid, st) = spawn_vout(width, height)?;
node_id = nid;
stop = st;
addr = resolve_kscreen_addr(&name, width, height);
self.last_name = Some(addr.clone());
tracing::info!(
node_id,
width,
height,
"KWin virtual output ready (fallback)"
);
60
let active = set_custom_refresh(width, height, mode.refresh_hz, &addr);
// Accept only an active mode that IS our custom one: the exact requested height, and a
// width at or just below the request (a CVT alignment). That also proves the output
// left the sacrificial birth size, so the recording stream will renegotiate to it.
match active {
Some((aw, ah, ahz))
if ah == height && aw <= width && width - aw < CVT_H_GRANULARITY =>
{
expect_exact_dims = true;
final_dims = (aw, ah);
ahz
}
other => {
// Custom-mode install/select rejected (pre-6.6 KWin / stale kscreen-doctor): the
// output is STUCK at the sacrificial birth size — unusable. Recreate plain at the
// real size (the pre-sacrifice behavior: correct size, KWin's native 60 Hz).
tracing::warn!(
active = ?other,
requested_w = width,
requested_h = height,
requested_hz = mode.refresh_hz,
"KWin rejected the custom mode — recreating the virtual output at the real \
size (60 Hz ceiling on this KWin)"
);
stop.store(true, Ordering::Relaxed);
// Let KWin retire the doomed output before re-using its name.
std::thread::sleep(Duration::from_millis(300));
let (nid, st) = spawn_vout(width, height)?;
node_id = nid;
stop = st;
addr = resolve_kscreen_addr(&name, width, height);
self.last_name = Some(addr.clone());
tracing::info!(
node_id,
width,
height,
"KWin virtual output ready (fallback)"
);
60
}
}
} else {
mode.refresh_hz
@@ -302,7 +316,7 @@ impl VirtualDisplay for KwinDisplay {
// (it owns the display group, so it computes auto-row / manual placement over the whole group).
let mut out = VirtualOutput::owned(
node_id,
Some((mode.width, mode.height, achieved_hz)),
Some((final_dims.0, final_dims.1, achieved_hz)),
Box::new(StopGuard { stop }),
);
out.expect_exact_dims = expect_exact_dims;
@@ -324,9 +338,7 @@ fn reenable_outputs(outputs: &[(String, String)]) {
.iter()
.map(|(name, _)| format!("output.{name}.enable"))
.collect();
let _ = std::process::Command::new("kscreen-doctor")
.args(&enable_args)
.status();
let _ = kscreen_ok(&enable_args);
// THEN re-assert each captured mode, best-effort — a bare re-enable lets KWin fall back to the
// EDID-preferred mode (a 120 Hz panel returns at ~60 Hz); this restores the exact refresh. The
// output is enabled now, so the mode set is valid; a rejected mode just leaves KWin's default.
@@ -336,9 +348,7 @@ fn reenable_outputs(outputs: &[(String, String)]) {
.map(|(name, mode)| format!("output.{name}.mode.{mode}"))
.collect();
if !mode_args.is_empty() {
let _ = std::process::Command::new("kscreen-doctor")
.args(&mode_args)
.status();
let _ = kscreen_ok(&mode_args);
}
std::thread::sleep(Duration::from_millis(200));
tracing::info!(reenabled = ?outputs, "KWin: restored the physical/bootstrap outputs at their captured modes (group empty)");
@@ -386,13 +396,37 @@ fn resolve_kscreen_addr(name: &str, w: u32, h: u32) -> String {
fallback
}
/// Budget for one `kscreen-doctor` call.
///
/// It is a Wayland client of the very compositor it configures, so against a wedged KWin it blocks
/// in its own connect and never returns — and these calls run on the session's stream thread, whose
/// only way to end a session is to return. Generous next to a healthy call (tens of ms).
const KSCREEN_BUDGET: Duration = Duration::from_secs(5);
/// `kscreen-doctor <args>` run for its exit status, bounded by [`KSCREEN_BUDGET`]. A timeout reads
/// as a failed apply — the same best-effort path a rejected argument already takes.
fn kscreen_ok(args: &[String]) -> bool {
crate::proc::status_within(
std::process::Command::new("kscreen-doctor").args(args),
KSCREEN_BUDGET,
)
.map(|s| s.success())
.unwrap_or(false)
}
/// `kscreen-doctor -j` stdout, bounded by [`KSCREEN_BUDGET`]; `None` on any failure.
fn kscreen_json_bytes() -> Option<Vec<u8>> {
crate::proc::output_within(
std::process::Command::new("kscreen-doctor").arg("-j"),
KSCREEN_BUDGET,
)
.ok()
.map(|o| o.stdout)
}
/// `kscreen-doctor -j` parsed, `None` on any failure.
fn kscreen_json() -> Option<serde_json::Value> {
let out = std::process::Command::new("kscreen-doctor")
.arg("-j")
.output()
.ok()?;
serde_json::from_slice(&out.stdout).ok()
serde_json::from_slice(&kscreen_json_bytes()?).ok()
}
/// The `(width, height)` of an output's CURRENT mode from its `kscreen-doctor -j` entry.
@@ -415,40 +449,184 @@ fn output_active_size(o: &serde_json::Value) -> Option<(u32, u32)> {
))
}
/// Best-effort: install + select the `width`x`height`@`hz` custom mode on the just-created
/// virtual output via `kscreen-doctor` (`output` is the RESOLVED kscreen address — numeric id or
/// name, see [`resolve_kscreen_addr`] — refresh given in mHz), then **read back the active mode**
/// and return `(achieved_hz, size_applied)`. The apply command can report success yet leave the
/// output on its old mode (rejected), and a silent rate mismatch surfaces downstream as judder /
/// duplicated frames — so the caller paces the encoder to the *achieved* rate, not the requested
/// one. `size_applied` tells the sacrificial-birth caller (see `create`) whether the SIZE half of
/// the mode actually landed — that, not the refresh, is what triggers KWin's stream
/// renegotiation.
fn set_custom_refresh(width: u32, height: u32, hz: u32, output: &str) -> (u32, bool) {
/// CVT's horizontal cell granularity. KWin generates every custom mode's timing with **libxcvt**,
/// whose first step is `hdisplay_rnd = hdisplay - (hdisplay % 8)` — so a width that isn't a multiple
/// of 8 comes back NARROWER than asked, and the clock-step rounding that follows lands a fractional
/// refresh. A 2868x1320@120 request (an iPhone 16 Pro Max panel) becomes **2864x1320@119.92**.
///
/// That is why a custom mode must never be selected by the `WxH@Hz` string we *requested*:
/// kscreen-doctor's `findMode` matches a mode's id or its own `WxH@qRound(Hz)` name, so
/// `2868x1320@120` matches nothing, the select silently no-ops, the output stays on its sacrificial
/// birth mode, and the caller falls back to 60 Hz — while KDE's display list shows the perfectly
/// good 2864x1320@119.92 mode sitting there unselected. Widths like 1920/2560/3840 are all
/// multiples of 8, which is why only phone-shaped clients ever hit it.
const CVT_H_GRANULARITY: u32 = 8;
/// One row of an output's mode list, as parsed from `kscreen-doctor -j`.
#[derive(Clone, Debug, PartialEq)]
struct KModeRow {
/// kscreen's mode id — what we address the mode by (never the requested `WxH@Hz` string).
id: String,
w: u32,
h: u32,
hz: f64,
}
/// A kscreen JSON id, which is a string on some KWin versions and a number on others.
fn json_id(v: &serde_json::Value) -> Option<String> {
v.as_str()
.map(|s| s.to_string())
.or_else(|| v.as_u64().map(|n| n.to_string()))
}
/// The full mode list of `output` (a RESOLVED kscreen address — numeric id or name) from a parsed
/// `kscreen-doctor -j` document. Split from the process call so the picker can be tested on
/// captured JSON.
fn modes_from_json(doc: &serde_json::Value, output: &str) -> Vec<KModeRow> {
let Some(o) = doc
.get("outputs")
.and_then(|v| v.as_array())
.and_then(|outs| {
outs.iter().find(|o| {
o.get("name").and_then(|n| n.as_str()) == Some(output)
|| o.get("id").and_then(json_id).as_deref() == Some(output)
})
})
else {
return Vec::new();
};
o.get("modes")
.and_then(|m| m.as_array())
.map(|ms| {
ms.iter()
.filter_map(|m| {
let size = m.get("size")?;
Some(KModeRow {
id: m.get("id").and_then(json_id)?,
w: size.get("width").and_then(|v| v.as_u64())? as u32,
h: size.get("height").and_then(|v| v.as_u64())? as u32,
hz: m.get("refreshRate").and_then(|r| r.as_f64())?,
})
})
.collect()
})
.unwrap_or_default()
}
/// [`modes_from_json`] against a live `kscreen-doctor -j`.
fn output_modes(output: &str) -> Vec<KModeRow> {
kscreen_json()
.map(|doc| modes_from_json(&doc, output))
.unwrap_or_default()
}
/// The mode in `modes` that actually fulfils a `width`x`height`@`hz` request, tolerating the CVT
/// alignment KWin applies when it generates the timing (see [`CVT_H_GRANULARITY`]): the height must
/// match exactly (CVT never touches the vertical active), the width may be up to one cell narrower
/// than asked (never wider — that would be a different mode), and the refresh must land within 1 Hz
/// of the request (which excludes the output's native 60 Hz entry for every rate we install a custom
/// mode for). Widest wins, then fastest — so an exact-width mode always beats an aligned one, and a
/// list carrying duplicate custom modes from earlier sessions still resolves.
fn pick_custom_mode<'a>(
modes: &'a [KModeRow],
width: u32,
height: u32,
hz: u32,
) -> Option<&'a KModeRow> {
modes
.iter()
.filter(|m| {
m.h == height
&& m.w <= width
&& width - m.w < CVT_H_GRANULARITY
&& (m.hz - f64::from(hz)).abs() < 1.0
})
.max_by(|a, b| {
a.w.cmp(&b.w)
.then(a.hz.partial_cmp(&b.hz).unwrap_or(std::cmp::Ordering::Equal))
})
}
/// Best-effort: install + select the `width`x`height`@`hz` custom mode on the just-created virtual
/// output via `kscreen-doctor` (`output` is the RESOLVED kscreen address — numeric id or name, see
/// [`resolve_kscreen_addr`] — refresh given in mHz), then **read back the active mode** and return
/// it as `(width, height, refresh_hz)`. `None` if the read-back failed entirely.
///
/// The apply command can report success yet leave the output on its old mode (rejected), and a
/// silent size/rate mismatch surfaces downstream as a starved capture gate or judder — so the
/// caller drives the pipeline off the *achieved* mode, not the requested one. The mode is selected
/// by kscreen **mode id** resolved from the output's own list, never by the requested `WxH@Hz`
/// string, because KWin's CVT generator may hand back a slightly different one
/// ([`CVT_H_GRANULARITY`]).
fn set_custom_refresh(width: u32, height: u32, hz: u32, output: &str) -> Option<(u32, u32, u32)> {
let output = output.to_string();
let mhz = hz.saturating_mul(1000);
let run = |arg: String| {
std::process::Command::new("kscreen-doctor")
.arg(arg)
.status()
.map(|s| s.success())
.unwrap_or(false)
let run = |arg: String| kscreen_ok(&[arg]);
// Install the mode only if the output doesn't already carry a usable one: kscreen-doctor
// APPENDS to the output's custom-mode list and KWin PERSISTS that list per output name
// (`kwinoutputconfig.json`, which is why the same per-slot name is reused across sessions) — so
// re-adding on every connect would grow the user's display list without bound.
let mut modes = output_modes(&output);
if pick_custom_mode(&modes, width, height, hz).is_none() {
let _ = run(format!(
"output.{output}.addCustomMode.{width}.{height}.{mhz}.full"
));
modes = output_modes(&output);
}
let applied = match pick_custom_mode(&modes, width, height, hz) {
Some(target) => {
if (target.w, target.h) != (width, height) {
tracing::info!(
output,
requested_w = width,
requested_h = height,
mode_w = target.w,
mode_h = target.h,
mode_hz = target.hz,
"KWin aligned the custom mode to the CVT cell grain — streaming at its size"
);
}
// By id first; the human `WxH@Hz` form (built from the mode's OWN size/refresh, not the
// request) is the fallback for builds whose ids don't round-trip through the CLI.
run(format!("output.{output}.mode.{}", target.id))
|| run(format!(
"output.{output}.mode.{}x{}@{}",
target.w,
target.h,
target.hz.round() as u32
))
}
None => {
tracing::warn!(
output,
requested_w = width,
requested_h = height,
requested_hz = hz,
offered = ?modes,
"KWin offers no mode matching the request after addCustomMode — is kscreen-doctor \
up to date, and KWin 6.6 (custom modes on virtual outputs)?"
);
false
}
};
// Add the custom mode (a fresh output has none), then select it.
let _ = run(format!(
"output.{output}.addCustomMode.{width}.{height}.{mhz}.full"
));
let applied = run(format!("output.{output}.mode.{width}x{height}@{hz}"));
match read_active_mode(&output) {
Some((w, h, achieved)) => {
let size_applied = (w, h) == (width, height);
if achieved >= hz && size_applied {
if achieved >= hz && (w, h) == (width, height) {
tracing::info!(
output,
requested = hz,
achieved,
"KWin virtual output: custom refresh applied"
);
} else if achieved >= hz {
tracing::info!(
output,
requested = hz,
achieved,
active_w = w,
active_h = h,
"KWin virtual output: custom refresh applied at a CVT-aligned size"
);
} else {
tracing::warn!(
output,
@@ -461,7 +639,7 @@ fn set_custom_refresh(width: u32, height: u32, hz: u32, output: &str) -> (u32, b
achieved rate (custom-mode install rejected? is kscreen-doctor up to date?)"
);
}
(achieved.max(1), size_applied)
Some((w, h, achieved.max(1)))
}
None => {
tracing::warn!(
@@ -471,7 +649,7 @@ fn set_custom_refresh(width: u32, height: u32, hz: u32, output: &str) -> (u32, b
"could not read back KWin virtual output refresh — assuming 60 Hz (is \
kscreen-doctor installed?)"
);
(60, false)
None
}
}
}
@@ -542,14 +720,11 @@ fn output_current_mode_spec(o: &serde_json::Value) -> Option<String> {
/// session's own output — so a 2nd `exclusive` session (with a distinct per-slot name) never disables
/// the 1st session's live output. Parsed from `kscreen-doctor -j` (same source as [`read_active_mode`]).
fn other_enabled_outputs() -> Vec<(String, String)> {
let out = match std::process::Command::new("kscreen-doctor")
.arg("-j")
.output()
{
Ok(o) => o,
Err(_) => return Vec::new(),
let out = match kscreen_json_bytes() {
Some(o) => o,
None => return Vec::new(),
};
let doc: serde_json::Value = match serde_json::from_slice(&out.stdout) {
let doc: serde_json::Value = match serde_json::from_slice(&out) {
Ok(d) => d,
Err(_) => return Vec::new(),
};
@@ -578,13 +753,10 @@ fn other_enabled_outputs() -> Vec<(String, String)> {
/// then sets itself primary — the pre-group behavior). Recent kscreen marks the primary with
/// `"priority": 1`; older builds used a `"primary": true` bool — accept either.
fn a_managed_output_is_primary() -> bool {
let Ok(out) = std::process::Command::new("kscreen-doctor")
.arg("-j")
.output()
else {
let Some(out) = kscreen_json_bytes() else {
return false;
};
let Ok(doc) = serde_json::from_slice::<serde_json::Value>(&out.stdout) else {
let Ok(doc) = serde_json::from_slice::<serde_json::Value>(&out) else {
return false;
};
doc.get("outputs")
@@ -609,13 +781,7 @@ fn a_managed_output_is_primary() -> bool {
/// the keepalive to re-enable on teardown. Best-effort: on failure, streaming continues (just possibly
/// showing only the wallpaper) rather than failing the session.
fn apply_virtual_primary(ours: &str) -> Vec<(String, String)> {
let kscreen = |args: &[String]| {
std::process::Command::new("kscreen-doctor")
.args(args)
.status()
.map(|s| s.success())
.unwrap_or(false)
};
let kscreen = |args: &[String]| kscreen_ok(args);
// First-slot-wins (§6.1): only grab primary if no managed group member is primary yet — so a 2nd
// exclusive session joins as a secondary monitor of the shared desktop instead of stealing the
// shell off the 1st session's output. KWin usually then re-homes the desktop + disables the
@@ -647,11 +813,7 @@ fn apply_virtual_primary(ours: &str) -> Vec<(String, String)> {
/// (don't disable the bootstrap/physical) — so the shell re-homes onto the streamed surface while a
/// physical screen stays usable. Nothing to restore on teardown (we disabled nothing).
fn apply_virtual_primary_only(ours: &str) {
let ok = std::process::Command::new("kscreen-doctor")
.arg(format!("output.{ours}.primary"))
.status()
.map(|s| s.success())
.unwrap_or(false);
let ok = kscreen_ok(&[format!("output.{ours}.primary")]);
if ok {
tracing::info!("KWin: streamed output set primary (physical outputs kept)");
} else {
@@ -884,7 +1046,88 @@ fn run(
#[cfg(test)]
mod tests {
use super::MANAGED_PREFIX;
use super::{modes_from_json, pick_custom_mode, KModeRow, MANAGED_PREFIX};
fn row(id: &str, w: u32, h: u32, hz: f64) -> KModeRow {
KModeRow {
id: id.to_string(),
w,
h,
hz,
}
}
/// The reported regression: an iPhone 16 Pro Max asks for 2868x1320@120; libxcvt rounds the
/// width down to the 8-pixel cell grain and the clock step lands 119.92, so KWin's list holds
/// 2864x1320@119.92. Selecting by the REQUESTED `2868x1320@120` string matched nothing — the
/// output stayed on its birth mode and the session fell back to 60 Hz. The picker must find it.
#[test]
fn picks_the_cvt_aligned_mode() {
let modes = [
row("1", 2868, 1320, 60.0), // the virtual output's native/birth mode
row("2", 2864, 1320, 119.92), // the custom mode KWin actually generated
];
let got = pick_custom_mode(&modes, 2868, 1320, 120).expect("CVT-aligned mode");
assert_eq!((got.id.as_str(), got.w, got.h), ("2", 2864, 1320));
}
/// A width already on the cell grain (every PC resolution: 1920/2560/3840) round-trips exactly,
/// and an exact-width mode outranks an aligned one when both are offered.
#[test]
fn exact_width_outranks_an_aligned_one() {
let modes = [
row("1", 2560, 1440, 60.0),
row("2", 2552, 1440, 119.93), // a stale narrower custom mode from an earlier session
row("3", 2560, 1440, 119.98),
];
let got = pick_custom_mode(&modes, 2560, 1440, 120).expect("exact mode");
assert_eq!(got.id, "3");
}
/// The picker must never wander onto an unrelated mode: not the 60 Hz native entry (the old
/// fallback the reporter got stuck on), not a different height, not a wider width, and not a
/// mode more than one cell narrower than asked.
#[test]
fn rejects_modes_that_are_not_the_request() {
let modes = [
row("1", 2868, 1320, 60.0), // native — refresh too far off
row("2", 2868, 1080, 119.92), // wrong height
row("3", 2880, 1320, 119.92), // wider than requested
row("4", 2856, 1320, 119.92), // two cells narrower — not a CVT alignment of 2868
row("5", 1920, 1080, 120.0), // unrelated
];
assert!(pick_custom_mode(&modes, 2868, 1320, 120).is_none());
}
/// Mode + output ids come through as JSON strings on some KWin versions and numbers on others;
/// both must parse, and a mode row missing its size/refresh is skipped rather than poisoning
/// the list.
#[test]
fn parses_both_id_encodings() {
let doc: serde_json::Value = serde_json::from_str(
r#"{"outputs":[
{"id":7,"name":"Virtual-punktfunk","modes":[
{"id":"m1","size":{"width":2868,"height":1320},"refreshRate":60.0},
{"id":42,"size":{"width":2864,"height":1320},"refreshRate":119.92},
{"id":"broken","size":{"width":800}}
]},
{"id":1,"name":"eDP-1","modes":[
{"id":"x","size":{"width":2864,"height":1320},"refreshRate":119.92}
]}
]}"#,
)
.expect("fixture parses");
// Addressable by numeric id (how `resolve_kscreen_addr` returns it) and by name.
for addr in ["7", "Virtual-punktfunk"] {
let modes = modes_from_json(&doc, addr);
assert_eq!(modes.len(), 2, "the malformed row is dropped ({addr})");
assert_eq!(modes[1].id, "42", "numeric mode ids stringify ({addr})");
let got = pick_custom_mode(&modes, 2868, 1320, 120).expect("aligned mode");
assert_eq!(got.id, "42");
}
// Never reads another output's list (the eDP-1 entry carries a matching mode).
assert!(modes_from_json(&doc, "Virtual-nope").is_empty());
}
/// Group-aware exclusive (§6.1): with two managed group members + a physical panel enabled,
/// exclusive disables ONLY the non-managed panel — never a sibling session's per-slot output
+114
View File
@@ -0,0 +1,114 @@
//! Time-bounded child-process helpers.
//!
//! Every compositor query this crate makes shells out to a helper (`kscreen-doctor`, `systemctl`,
//! `pw-dump`, …), and most of them are *clients of the very thing being diagnosed*: `kscreen-doctor`
//! is a Wayland client, so against a wedged KWin it blocks in its own connect and **never returns**.
//! `Command::status()` / `Command::output()` have no timeout, so one hung helper pinned the calling
//! thread forever — and on the host that thread is the session's stream thread, whose only way to
//! end a session is to return. A stuck query therefore became a permanently stuck session.
//!
//! These wrappers bound the wait: poll for exit until the budget runs out, then kill the child and
//! report [`std::io::ErrorKind::TimedOut`], so callers see a plain "the helper failed" error and
//! take their existing failure path instead of hanging.
use std::io::{Error, ErrorKind, Result};
use std::process::{Command, ExitStatus, Output};
use std::time::{Duration, Instant};
/// Poll interval while waiting for a child to exit. Short enough that a fast helper (the normal
/// case — `kscreen-doctor` answers in tens of ms) isn't measurably delayed.
const POLL: Duration = Duration::from_millis(20);
/// Run `cmd` to completion, killing it if it outlives `budget`.
///
/// Stdout/stderr are left as the caller configured them (inherited by default), so this is for
/// commands run for their exit status alone — see [`output_within`] when the output is read.
pub(crate) fn status_within(cmd: &mut Command, budget: Duration) -> Result<ExitStatus> {
let mut child = cmd.spawn()?;
let deadline = Instant::now() + budget;
loop {
match child.try_wait()? {
Some(status) => return Ok(status),
None if Instant::now() >= deadline => {
let _ = child.kill();
let _ = child.wait(); // reap it — never leave a zombie behind
return Err(timed_out(cmd, budget));
}
None => std::thread::sleep(POLL),
}
}
}
/// Run `cmd` to completion and capture its stdout/stderr, killing it if it outlives `budget`.
///
/// The output is read only after the child has exited, so a helper that fills the pipe buffer and
/// stalls is caught by the budget rather than deadlocking the reader (these helpers emit at most a
/// few hundred KiB, well under any real pipe pressure).
pub(crate) fn output_within(cmd: &mut Command, budget: Duration) -> Result<Output> {
let mut child = cmd
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()?;
let deadline = Instant::now() + budget;
loop {
match child.try_wait()? {
// Exited: `wait_with_output` now only drains already-buffered pipes.
Some(_) => return child.wait_with_output(),
None if Instant::now() >= deadline => {
let _ = child.kill();
let _ = child.wait();
return Err(timed_out(cmd, budget));
}
None => std::thread::sleep(POLL),
}
}
}
fn timed_out(cmd: &Command, budget: Duration) -> Error {
let program = cmd.get_program().to_string_lossy().to_string();
tracing::warn!(
program,
budget_ms = budget.as_millis() as u64,
"helper did not exit within its budget — killed it (a wedged compositor/session bus is the \
usual cause); treating it as a failed query"
);
Error::new(
ErrorKind::TimedOut,
format!("`{program}` did not exit within {budget:?}"),
)
}
#[cfg(test)]
mod tests {
use super::*;
/// A helper that never exits must be killed at the budget and reported as `TimedOut` — the
/// whole point of the module (an unbounded `status()` here is what wedged a whole session).
#[test]
fn a_hung_child_is_killed_at_the_budget() {
let started = Instant::now();
let err = status_within(Command::new("sleep").arg("30"), Duration::from_millis(150))
.expect_err("must time out");
assert_eq!(err.kind(), ErrorKind::TimedOut);
assert!(
started.elapsed() < Duration::from_secs(5),
"must return at its budget, not the child's lifetime (took {:?})",
started.elapsed()
);
}
/// The normal path is unaffected: a quick command still yields its status and its output.
#[test]
fn a_quick_child_returns_normally() {
let st = status_within(&mut Command::new("true"), Duration::from_secs(5)).expect("ran");
assert!(st.success());
let out = output_within(
Command::new("echo").arg("punktfunk"),
Duration::from_secs(5),
)
.expect("ran");
assert!(out.status.success());
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "punktfunk");
}
}
+40 -19
View File
@@ -6,6 +6,15 @@
use super::*;
/// Budget for one `systemctl --user` / `dbus-update-activation-environment` call.
///
/// These talk to the session bus, and a bus that is itself restarting or wedged answers nothing —
/// unbounded, that pinned the caller (on the host, the session's stream thread) forever. A restart
/// of the portal units is the slowest legitimate case, hence the generous window; missing it just
/// means the portal env settles late, which the callers already treat as best-effort.
#[cfg(target_os = "linux")]
const SYSTEMD_BUDGET: std::time::Duration = std::time::Duration::from_secs(10);
/// The **session epoch** — bumped whenever session detection observes a different compositor
/// *instance*: an [`ActiveKind`] change, **or** a new compositor PID for the same kind (the
/// Desktop→Game→Desktop bounce that brings up a fresh KWin/gamescope with an unrelated node-id space).
@@ -86,9 +95,15 @@ pub fn observe_session_instance(active: &ActiveSession) {
/// via the next [`settle_desktop_portal`], so scrubbing on a bounce is harmless.)
#[cfg(target_os = "linux")]
fn scrub_desktop_manager_env() {
let _ = std::process::Command::new("systemctl")
.args(["--user", "unset-environment", "WAYLAND_DISPLAY", "DISPLAY"])
.status();
let _ = crate::proc::status_within(
std::process::Command::new("systemctl").args([
"--user",
"unset-environment",
"WAYLAND_DISPLAY",
"DISPLAY",
]),
SYSTEMD_BUDGET,
);
}
#[cfg(not(target_os = "linux"))]
@@ -499,40 +514,46 @@ pub fn settle_desktop_portal(chosen: Compositor) {
];
// Push our (correct) env into the systemd --user manager + the D-Bus activation environment so a
// re-activated portal/backend inherits the live session.
let _ = std::process::Command::new("systemctl")
.args(["--user", "import-environment"])
.args(VARS)
.status();
let _ = std::process::Command::new("dbus-update-activation-environment")
.arg("--systemd")
.args(VARS)
.status();
let _ = crate::proc::status_within(
std::process::Command::new("systemctl")
.args(["--user", "import-environment"])
.args(VARS),
SYSTEMD_BUDGET,
);
let _ = crate::proc::status_within(
std::process::Command::new("dbus-update-activation-environment")
.arg("--systemd")
.args(VARS),
SYSTEMD_BUDGET,
);
// KWin input goes through the xdg RemoteDesktop portal; the frontend routes RemoteDesktop to a
// backend by its OWN startup XDG_CURRENT_DESKTOP, so restart it (+ the KDE backend) to re-read
// the now-live session, then let it settle before the injector reopens against it.
if chosen == Compositor::Kwin {
let _ = std::process::Command::new("systemctl")
.args([
let _ = crate::proc::status_within(
std::process::Command::new("systemctl").args([
"--user",
"try-restart",
"xdg-desktop-portal-kde.service",
"xdg-desktop-portal.service",
])
.status();
]),
SYSTEMD_BUDGET,
);
std::thread::sleep(std::time::Duration::from_millis(600));
}
// Hyprland capture rides the xdg ScreenCast portal serviced by xdph (G5): on a mid-stream switch
// xdph may still hold the old session's Wayland/instance env, so restart it (+ the frontend) to
// re-read the now-live session, mirroring the KWin settling above.
if chosen == Compositor::Hyprland {
let _ = std::process::Command::new("systemctl")
.args([
let _ = crate::proc::status_within(
std::process::Command::new("systemctl").args([
"--user",
"try-restart",
"xdg-desktop-portal-hyprland.service",
"xdg-desktop-portal.service",
])
.status();
]),
SYSTEMD_BUDGET,
);
std::thread::sleep(std::time::Duration::from_millis(600));
}
tracing::info!(
@@ -300,6 +300,21 @@ pub fn control_device_handle() -> Option<HANDLE> {
VDM.get().and_then(VirtualDisplayManager::device_handle)
}
/// Re-commit the CURRENT display config under the manager `state` lock (the sole-topology-mutator
/// contract of [`force_mode_reenumeration`]). The secure-desktop guard's actuator: the OS only
/// reverts a path to its software-cursor default ON a mode commit, so standing the hardware-cursor
/// declare down (`IOCTL_SET_CURSOR_FORWARD` off) needs this nudge for UAC/Winlogon to actually
/// render. `false` (no-op) before the first backend open — no monitors exist to re-commit for.
pub fn force_recommit() -> bool {
let Some(m) = VDM.get() else {
return false;
};
let _guard = m.state.lock().unwrap();
// SAFETY: `force_mode_reenumeration`'s contract is "call under the manager `state` lock";
// held above. The call reads + re-applies the current CCD config over owned locals.
unsafe { pf_win_display::win_display::force_mode_reenumeration() }
}
/// Best-effort "is this WUDFHost pid still alive?" — the monitor-liveness probe for the JOIN path.
/// `OpenProcess` failing (pid reaped) or the process being signaled ⇒ dead. Pid reuse could
/// theoretically alias a fresh process and read "alive"; the joining session then just retries into
+3
View File
@@ -31,5 +31,8 @@ windows = { version = "0.62", features = [
"Win32_System_LibraryLoader",
# console_session_mismatch: WTSGetActiveConsoleSessionId + ProcessIdToSessionId + GetCurrentProcessId.
"Win32_System_RemoteDesktop",
# input_desktop: OpenInputDesktop/SetThreadDesktop/GetUserObjectInformationW — display writes
# follow the input desktop so a UAC/lock screen can't refuse them.
"Win32_System_StationsAndDesktops",
"Win32_System_Threading",
] }
+200
View File
@@ -0,0 +1,200 @@
//! Make display-config writes follow the INPUT desktop.
//!
//! Windows refuses `ChangeDisplaySettingsEx` / `SetDisplayConfig` issued from a thread that is not
//! on the desktop currently receiving input. While a UAC consent prompt (or the lock / logon
//! screen) is up, the input desktop is `Winlogon`; a host thread sitting on `WinSta0\Default` — the
//! desktop the service explicitly launches it onto — then gets `DISP_CHANGE_FAILED` /
//! `ERROR_ACCESS_DENIED` for EVERY write. A session starting in that window can never set its
//! virtual display's mode, so the capturer sizes its ring to a stale mode, every composed frame is
//! dropped for a size mismatch, and the client sits on a black screen until bring-up runs out of
//! retries. Field-reported 2026-07-23: a consent prompt left up on an unattended host made the box
//! unreachable until someone walked over to it.
//!
//! Measured on-glass that same day (RTX 4090 box, SYSTEM host in console session 2, a real
//! interactive consent prompt up, virtual display `\\.\DISPLAY42` active):
//!
//! ```text
//! INPUT desktop = Winlogon
//! UNBOUND CDS_TEST -> -1 (DISP_CHANGE_FAILED) <- the field failure, reproduced
//! UNBOUND SDC_VALIDATE -> 0x5 (ERROR_ACCESS_DENIED) <- the field failure, reproduced
//! bound thread desktop -> Winlogon
//! BOUND CDS_TEST -> 0 (DISP_CHANGE_SUCCESSFUL)
//! BOUND SDC_VALIDATE -> 0x0 (ERROR_SUCCESS)
//! ```
//!
//! The retry model mirrors `pf-inject`'s `sendinput.rs`: do NOT pay
//! `OpenInputDesktop`/`SetThreadDesktop` on every write — issue the write, and only when it fails
//! the way a wrong-desktop write fails do we rebind and retry once. That keeps the normal path
//! byte-for-byte as it was (a working write is never touched) and makes this strictly additive.
//!
//! Unlike sendinput's injector — a dedicated thread that KEEPS its binding — these helpers run on
//! shared/task threads, so the binding here is SCOPED: [`InputDesktopBinding`] restores the
//! thread's original desktop on drop. A thread left bound to a `Winlogon` desktop that is later
//! destroyed (prompt dismissed) would fail every subsequent display write for the life of the
//! process — the exact wedge this module exists to remove, so it must not be introduced here.
use windows::Win32::Foundation::HANDLE;
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, GetThreadDesktop, GetUserObjectInformationW, OpenInputDesktop, SetThreadDesktop,
DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS, HDESK, UOI_NAME,
};
use windows::Win32::System::Threading::GetCurrentThreadId;
/// `GENERIC_ALL` for the desktop open. The `windows` crate models desktop access as its own flag
/// type and doesn't export the generic rights, so spell it out (same constant `sendinput.rs` and
/// the cursor poller use).
const GENERIC_ALL: u32 = 0x1000_0000;
/// This thread's binding to the input desktop, restored on drop.
///
/// `GetThreadDesktop` returns a BORROWED handle (documented: it creates no new handle and must not
/// be closed), so only the handle `OpenInputDesktop` produced is closed here — and only after the
/// thread has been moved back off it.
pub(crate) struct InputDesktopBinding {
previous: HDESK,
input: HDESK,
/// `UOI_NAME` of the desktop bound to, for the "this write only landed because we rebound"
/// log. Read once at bind time (the failure path only), never in steady state.
name: String,
}
impl InputDesktopBinding {
/// Bind the calling thread to the desktop currently receiving input. `None` when there is no
/// reachable input desktop (not privileged for `Winlogon` — a host that is not SYSTEM) or the
/// rebind is refused, in which case the caller keeps whatever result it already had rather than
/// changing behaviour.
pub(crate) fn bind() -> Option<Self> {
// SAFETY: all four are FFI calls taking by-value args only. `GetThreadDesktop` yields a
// borrowed handle for THIS thread (never closed here). `OpenInputDesktop` yields an owned
// `HDESK` only on `Ok`; it is either installed by `SetThreadDesktop` (and then owned by the
// returned guard, which closes it exactly once in `Drop`) or closed right here on failure —
// so it is closed exactly once on every path and never used after close. `SetThreadDesktop`
// rebinds only the calling thread, which owns no windows or hooks (these are display-config
// worker threads), so it cannot fail on that account.
unsafe {
let previous = GetThreadDesktop(GetCurrentThreadId()).ok()?;
let input = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
)
.ok()?;
let name = desktop_name(input).unwrap_or_else(|| "<unnamed>".into());
if SetThreadDesktop(input).is_err() {
let _ = CloseDesktop(input);
return None;
}
Some(Self {
previous,
input,
name,
})
}
}
}
impl Drop for InputDesktopBinding {
fn drop(&mut self) {
// SAFETY: `self.previous` is the borrowed desktop this thread started on and `self.input`
// is the handle this guard uniquely owns. The thread is moved back FIRST, so the handle is
// no longer the thread's desktop when it is closed — closed exactly once, never used after.
unsafe {
let _ = SetThreadDesktop(self.previous);
let _ = CloseDesktop(self.input);
}
}
}
/// `UOI_NAME` of the current input desktop — `Some("Winlogon")` while a UAC consent prompt, the
/// lock screen or the logon screen owns input, `Some("Default")` in normal operation, `None` when
/// it cannot be opened at all.
pub(crate) fn input_desktop_name() -> Option<String> {
// SAFETY: `OpenInputDesktop` yields an owned handle only on `Ok`, which is closed exactly once
// below and not used after.
unsafe {
let h = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
)
.ok()?;
let name = desktop_name(h);
let _ = CloseDesktop(h);
name
}
}
/// `UOI_NAME` of an already-open desktop handle. Borrows `h` — closing it stays the caller's job.
///
/// # Safety
/// `h` must be a live desktop handle for the duration of the call.
unsafe fn desktop_name(h: HDESK) -> Option<String> {
let mut name = [0u16; 64]; // "Default"/"Winlogon"/"Screen-saver" all fit with room to spare
let mut needed = 0u32;
// SAFETY: `h` is live per this fn's contract; `name`/`needed` are live out-params and the call
// writes at most `nlength` bytes, exactly the size passed.
GetUserObjectInformationW(
HANDLE(h.0),
UOI_NAME,
Some(name.as_mut_ptr().cast()),
(name.len() * 2) as u32,
Some(&mut needed),
)
.ok()?;
let len = name.iter().position(|&c| c == 0).unwrap_or(name.len());
Some(String::from_utf16_lossy(&name[..len]))
}
/// `true` when the input desktop is a SECURE one (`UOI_NAME` != `Default`: `Winlogon` for UAC
/// consent / lock / logon, or a screen-saver desktop) — i.e. when display writes from the host's
/// own desktop are being refused for that reason. `false` when it is normal OR unreadable: this
/// only ever phrases a diagnostic, so an unknown desktop must not claim the secure one is up.
pub(crate) fn input_desktop_is_secure() -> bool {
input_desktop_name().is_some_and(|n| !n.eq_ignore_ascii_case("Default"))
}
/// `ERROR_ACCESS_DENIED` — exactly how Windows refuses a `SetDisplayConfig` issued from a thread
/// that is not on the input desktop (measured, see the module header). Narrower than "any error" on
/// purpose: the CCD path also returns `ERROR_INVALID_PARAMETER` (0x57) for the unrelated
/// exclusive-mode topology bug, and re-issuing THAT on another desktop would only confuse its
/// diagnosis.
const SDC_ACCESS_DENIED: i32 = 5;
/// [`retry_on_input_desktop`] specialised for the CCD writes, which all return a Win32 error code.
pub(crate) fn retry_set_display_config(write: impl FnMut() -> i32) -> i32 {
retry_on_input_desktop(|rc| *rc == SDC_ACCESS_DENIED, write)
}
/// Run a display-config write; if it comes back the way a write from the wrong desktop comes back,
/// bind this thread to the CURRENT input desktop and run it exactly once more.
///
/// `denied` decides which results are worth a retry — keep it NARROW (the specific
/// wrong-desktop verdict), so a genuinely bad mode or a driver-level refusal is not re-issued and
/// mis-attributed. The binding is dropped before returning, so the calling thread always leaves on
/// the desktop it arrived on.
pub(crate) fn retry_on_input_desktop<T>(
denied: impl Fn(&T) -> bool,
mut write: impl FnMut() -> T,
) -> T {
let first = write();
if !denied(&first) {
return first;
}
// Only worth the rebind when the input desktop is genuinely elsewhere; on a normal desktop the
// refusal means something else entirely and a retry would just repeat it.
let Some(binding) = InputDesktopBinding::bind() else {
return first;
};
let second = write();
if !denied(&second) {
// Say so. A silent save is indistinguishable in a log from a write that never needed
// saving, which makes "did the fix fire?" unanswerable in the field — and made the first
// on-glass verification of this very change inconclusive.
tracing::info!(
desktop = %binding.name,
"display write was refused off the input desktop — retried bound to it and it applied \
(a UAC prompt / lock screen owns input; the session continues normally)"
);
}
second
}
+3
View File
@@ -11,6 +11,9 @@
#[cfg(target_os = "windows")]
pub mod display_events;
/// Bind display-config writes to the input desktop so a UAC / lock screen can't refuse them.
#[cfg(target_os = "windows")]
mod input_desktop;
#[cfg(target_os = "windows")]
pub mod monitor_devnode;
#[cfg(target_os = "windows")]
+195 -67
View File
@@ -44,8 +44,8 @@ use windows::Win32::Devices::Display::{
use windows::Win32::Foundation::POINTL;
use windows::Win32::Graphics::Gdi::{
ChangeDisplaySettingsExW, EnumDisplaySettingsW, CDS_TEST, CDS_UPDATEREGISTRY, DEVMODEW,
DISP_CHANGE_SUCCESSFUL, DM_BITSPERPEL, DM_DISPLAYFREQUENCY, DM_PELSHEIGHT, DM_PELSWIDTH,
ENUM_CURRENT_SETTINGS, ENUM_DISPLAY_SETTINGS_MODE,
DISP_CHANGE_FAILED, DISP_CHANGE_SUCCESSFUL, DM_BITSPERPEL, DM_DISPLAYFREQUENCY, DM_PELSHEIGHT,
DM_PELSWIDTH, ENUM_CURRENT_SETTINGS, ENUM_DISPLAY_SETTINGS_MODE,
};
use punktfunk_core::Mode;
@@ -62,7 +62,9 @@ use punktfunk_core::Mode;
pub unsafe fn force_extend_topology() {
// A topology flag with no supplied path/mode arrays tells the OS to recompute + apply that preset
// for the currently-connected displays (the same code path DisplaySwitch.exe drives).
let rc = SetDisplayConfig(None, None, SDC_APPLY | SDC_TOPOLOGY_EXTEND);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(None, None, SDC_APPLY | SDC_TOPOLOGY_EXTEND)
});
if rc == 0 {
tracing::info!(
"display topology forced to EXTEND (a new IddCx monitor would otherwise be CLONED onto the \
@@ -152,11 +154,13 @@ pub unsafe fn activate_target_path(target_id: u32) -> bool {
// SAVE_TO_DATABASE so Windows remembers the arrangement — the next same-identity ADD (the driver
// reuses the slot's EDID serial/ConnectorIndex) then auto-activates from the persistence DB and
// skips this whole fallback ladder.
let rc = SetDisplayConfig(
Some(supplied.as_slice()),
Some(modes.as_slice()),
SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES | SDC_SAVE_TO_DATABASE,
);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(
Some(supplied.as_slice()),
Some(modes.as_slice()),
SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES | SDC_SAVE_TO_DATABASE,
)
});
if rc == 0 {
tracing::info!(
target_id,
@@ -273,14 +277,16 @@ pub unsafe fn force_mode_reenumeration() -> bool {
let Some((paths, modes)) = query_active_config() else {
return false;
};
let rc = SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
)
});
if rc != 0 {
tracing::debug!("force mode re-enumeration: SetDisplayConfig rc={rc:#x}");
}
@@ -625,9 +631,15 @@ pub fn set_active_mode(gdi_name: &str, mode: Mode) {
// SAFETY: `wname` is a live NUL-terminated UTF-16 device name and `&dm` is a live DEVMODEW describing
// the requested mode; both outlive the call. CDS_TEST only validates the mode (no apply), the two
// trailing args are null, and the API only reads its inputs.
let test = unsafe {
ChangeDisplaySettingsExW(PCWSTR(wname.as_ptr()), Some(&dm), None, CDS_TEST, None)
};
// A CDS write from a thread that is not on the input desktop is refused with DISP_CHANGE_FAILED
// (UAC consent / lock screen up) — retry it bound to that desktop rather than declaring the mode
// unsupported, which is what stranded sessions on a black screen for a whole bring-up.
let test = crate::input_desktop::retry_on_input_desktop(
|rc| *rc == DISP_CHANGE_FAILED,
|| unsafe {
ChangeDisplaySettingsExW(PCWSTR(wname.as_ptr()), Some(&dm), None, CDS_TEST, None)
},
);
if test != DISP_CHANGE_SUCCESSFUL {
tracing::warn!(
result = test.0,
@@ -642,15 +654,20 @@ pub fn set_active_mode(gdi_name: &str, mode: Mode) {
// SAFETY: same inputs as the CDS_TEST call above — `wname` (live NUL-terminated device name) and
// `&dm` (live DEVMODEW) both outlive the call; CDS_UPDATEREGISTRY applies the already-validated mode,
// and the API only reads its inputs.
let apply = unsafe {
ChangeDisplaySettingsExW(
PCWSTR(wname.as_ptr()),
Some(&dm),
None,
CDS_UPDATEREGISTRY,
None,
)
};
// Same wrong-desktop retry as the validate above: the two calls bind independently, so an apply
// still lands even when the secure desktop came up between them.
let apply = crate::input_desktop::retry_on_input_desktop(
|rc| *rc == DISP_CHANGE_FAILED,
|| unsafe {
ChangeDisplaySettingsExW(
PCWSTR(wname.as_ptr()),
Some(&dm),
None,
CDS_UPDATEREGISTRY,
None,
)
},
);
if apply == DISP_CHANGE_SUCCESSFUL {
tracing::info!(
"{gdi_name}: active mode set to {}x{}@{}",
@@ -672,12 +689,20 @@ pub fn set_active_mode(gdi_name: &str, mode: Mode) {
/// Human decode for a failed `ChangeDisplaySettingsExW` result. The two codes worth telling apart
/// in a field log: `BADMODE` (the display's mode list genuinely lacks the mode) vs `FAILED` (the
/// write itself was rejected — on a healthy driver that is the signature of a host process without
/// console-session access, e.g. one trapped in a disconnected RDP session). An earlier revision
/// printed "mode not advertised?" for BOTH, which sent a lid-closed field report chasing the wrong
/// cause.
/// write itself was rejected). An earlier revision printed "mode not advertised?" for BOTH, which
/// sent a lid-closed field report chasing the wrong cause.
///
/// `FAILED` itself has two causes needing opposite fixes — no console-session access (disconnected
/// RDP) versus the right session but the wrong desktop (UAC / lock / logon screen owns input) — so
/// it asks which before naming one. See [`sdc_access_denied_hint`] for the same split on the CCD
/// side.
fn disp_change_reason(rc: i32) -> &'static str {
match rc {
-1 if crate::input_desktop::input_desktop_is_secure() => {
"DISP_CHANGE_FAILED: the SECURE desktop owns input — a UAC consent prompt, the lock \
screen or the logon screen is up, and display writes are refused off it. Dismiss the \
prompt on the host"
}
-1 => {
"DISP_CHANGE_FAILED: the display write was rejected — a host without console-session \
access (disconnected RDP session / non-console session) fails ALL display writes \
@@ -691,15 +716,28 @@ fn disp_change_reason(rc: i32) -> &'static str {
}
}
/// Appended to `SetDisplayConfig` failure logs when rc is `ERROR_ACCESS_DENIED` (0x5) — per MS
/// docs "the caller does not have access to the console session", the field signature of a host
/// running in a disconnected RDP / non-console session. Every other rc gets no hint.
/// Appended to `SetDisplayConfig` failure logs when rc is `ERROR_ACCESS_DENIED` (0x5). Every other
/// rc gets no hint.
///
/// `ERROR_ACCESS_DENIED` has TWO field causes and they need opposite fixes, so ask which one before
/// naming it. MS docs say only "the caller does not have access to the console session", which is
/// the disconnected-RDP / non-console case — but the SAME rc comes back when the host IS in the
/// console session and merely off the input desktop (UAC consent, lock or logon screen up). Naming
/// only the first sent a 2026-07-23 investigation chasing a phantom RDP session while a consent
/// prompt was the actual cause, on a host the message told to "run via the installed service" —
/// which it already was. [`crate::input_desktop`] now retries these bound to the input desktop, so
/// seeing this at all means even that did not help.
fn sdc_access_denied_hint(rc: i32) -> &'static str {
if rc == 5 {
if rc != 5 {
return "";
}
if crate::input_desktop::input_desktop_is_secure() {
" (ERROR_ACCESS_DENIED: the SECURE desktop owns input — a UAC consent prompt, the lock \
screen or the logon screen is up, and display writes are refused off it. Dismiss the \
prompt on the host)"
} else {
" (ERROR_ACCESS_DENIED: the host has no console-session access — disconnected RDP \
session? run via the installed service so it tracks the console session)"
} else {
""
}
}
@@ -928,14 +966,41 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
// never calls ASSIGN_SWAPCHAIN, so the driver receives no frames. SDC_FORCE_MODE_ENUMERATION
// forces the re-commit; SAVE_TO_DATABASE only in the sole-path case (matches prior behavior —
// don't permanently rewrite the user's multi-display layout; the teardown restore handles it).
let mut flags = SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION;
if others == 0 {
flags |= SDC_SAVE_TO_DATABASE;
}
let rc = SetDisplayConfig(Some(paths.as_slice()), Some(modes.as_slice()), flags);
// The supplied shape is decided (and its escalation announced) ONCE, outside the write, so a
// wrong-desktop retry re-issues the identical config instead of logging the escalation twice.
let keep_only = (others > 0 && attempt >= 2).then(|| {
// ESCALATION (attempt 2+): supply ONLY the keep paths. Field-reported (AMD +
// pf-vdisplay): carrying the doomed path in the array — inactive, modes unpinned —
// gets the whole config rejected 0x57 on EVERY retry, so the loop alone never
// converged; the same host applies the keep-only shape rc=0 whenever the topology
// database has already made the virtual display sole. The final attempt also drops
// SDC_FORCE_MODE_ENUMERATION in case the driver rejects it combined with a real
// topology change — an actual path removal drives COMMIT_MODES on its own, so the
// re-commit rationale doesn't need the flag here.
let (kp, km) = keep_only_supplied(&paths, &modes);
let mut esc = SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES;
if attempt < 4 {
esc |= SDC_FORCE_MODE_ENUMERATION;
}
tracing::info!(
"display isolate (CCD): escalating to a keep-only supplied config (attempt {attempt}/4, paths {}→{}, modes {}→{})",
paths.len(), kp.len(), modes.len(), km.len()
);
(kp, km, esc)
});
let rc = crate::input_desktop::retry_set_display_config(|| match &keep_only {
Some((kp, km, esc)) => SetDisplayConfig(Some(kp.as_slice()), Some(km.as_slice()), *esc),
None => {
let mut flags = SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION;
if others == 0 {
flags |= SDC_SAVE_TO_DATABASE;
}
SetDisplayConfig(Some(paths.as_slice()), Some(modes.as_slice()), flags)
}
});
// A failed apply must be VISIBLE even when the verification below passes vacuously (nothing
// else was active to deactivate — the lid-closed laptop case, where the success INFO used to
// swallow rc=0x5): the re-commit above is load-bearing (COMMIT_MODES → ASSIGN_SWAPCHAIN),
@@ -962,6 +1027,63 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
Some(saved)
}
/// Build the ESCALATED supplied config for [`isolate_displays_ccd`]: ONLY the paths still flagged
/// ACTIVE (the keep set — the caller already cleared ACTIVE on every doomed path), with the mode
/// table rebuilt to just the entries those paths reference (indexes remapped). Docs-wise the
/// dropped inactive entries were declared ignored anyway ("Only the paths within this array that
/// have the DISPLAYCONFIG_PATH_ACTIVE flag set are set"), so this shape asks for the identical
/// topology — minus the array contents some driver/OS validation combos reject with 0x57.
unsafe fn keep_only_supplied(
paths: &[DISPLAYCONFIG_PATH_INFO],
modes: &[DISPLAYCONFIG_MODE_INFO],
) -> (Vec<DISPLAYCONFIG_PATH_INFO>, Vec<DISPLAYCONFIG_MODE_INFO>) {
let mut out_paths = Vec::new();
let mut out_modes = Vec::new();
// old mode index → new. Shared entries dedup through here: a clone-style pair references ONE
// source mode, and the docs require each source/target mode to appear in the table only once.
let mut remap = std::collections::HashMap::new();
for p in paths {
if p.flags & DISPLAYCONFIG_PATH_ACTIVE == 0 {
continue;
}
let mut q = *p;
q.sourceInfo.Anonymous.modeInfoIdx = remap_mode_idx(
q.sourceInfo.Anonymous.modeInfoIdx,
modes,
&mut out_modes,
&mut remap,
);
q.targetInfo.Anonymous.modeInfoIdx = remap_mode_idx(
q.targetInfo.Anonymous.modeInfoIdx,
modes,
&mut out_modes,
&mut remap,
);
out_paths.push(q);
}
(out_paths, out_modes)
}
/// Move `modes[old]` into `out` (once — `remap` dedups) and return its new index. INVALID and
/// out-of-range indexes stay INVALID — `SDC_ALLOW_CHANGES` lets best-mode logic fill the gap.
fn remap_mode_idx(
old: u32,
modes: &[DISPLAYCONFIG_MODE_INFO],
out: &mut Vec<DISPLAYCONFIG_MODE_INFO>,
remap: &mut std::collections::HashMap<u32, u32>,
) -> u32 {
if old == DISPLAYCONFIG_PATH_MODE_IDX_INVALID {
return old;
}
let Some(m) = modes.get(old as usize) else {
return DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
};
*remap.entry(old).or_insert_with(|| {
out.push(*m);
(out.len() - 1) as u32
})
}
/// The desktop-space rectangle `(x, y, w, h)` of `target_id`'s SOURCE — where this display's
/// region lives in the desktop coordinate space. `None` while the target isn't an active path.
/// Used by the IDD-push compose kick to dirty THE TARGET display: with parallel displays the
@@ -1056,14 +1178,16 @@ pub unsafe fn apply_source_positions(positions: &[(u32, i32, i32)]) {
if moved == 0 {
return;
}
let rc = SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
)
});
if rc == 0 {
tracing::info!(
?positions,
@@ -1148,14 +1272,16 @@ pub unsafe fn set_virtual_primary_ccd(keep_target_id: u32) -> Option<SavedConfig
}
}
let rc = SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION,
)
});
if rc == 0 {
tracing::info!("display primary (CCD): virtual target {keep_target_id} set PRIMARY at (0,0); {others} other display(s) kept ACTIVE + packed to its right");
} else {
@@ -1175,11 +1301,13 @@ pub unsafe fn restore_displays_ccd(saved: &SavedConfig) {
if paths.is_empty() {
return;
}
let rc = SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES,
);
let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(
Some(paths.as_slice()),
Some(modes.as_slice()),
SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES,
)
});
if rc == 0 {
tracing::info!("display isolate (CCD): restored original topology");
} else {
+152 -3
View File
@@ -870,6 +870,94 @@ impl PunktfunkRichInputEx {
}
}
/// [`PunktfunkPenSample::state`] bit: the pen hovers in range (implied by `TOUCHING`).
pub const PUNKTFUNK_PEN_IN_RANGE: u8 = 0x01;
/// [`PunktfunkPenSample::state`] bit: the tip is in contact.
pub const PUNKTFUNK_PEN_TOUCHING: u8 = 0x02;
/// [`PunktfunkPenSample::state`] bit: primary barrel button (or squeeze mapping) held.
pub const PUNKTFUNK_PEN_BARREL1: u8 = 0x04;
/// [`PunktfunkPenSample::state`] bit: secondary barrel button (or double-tap mapping) held.
pub const PUNKTFUNK_PEN_BARREL2: u8 = 0x08;
/// [`PunktfunkPenSample::tool`]: the pen tip.
pub const PUNKTFUNK_PEN_TOOL_PEN: u8 = 0;
/// [`PunktfunkPenSample::tool`]: the eraser (a client-side mode — Apple Pencil has no
/// hardware eraser end; the squeeze/double-tap mapping usually drives this).
pub const PUNKTFUNK_PEN_TOOL_ERASER: u8 = 1;
/// Most samples one [`punktfunk_connection_send_pen`] call accepts (one wire batch).
pub const PUNKTFUNK_PEN_BATCH_MAX: u32 = 8;
/// [`PunktfunkPenSample::tilt_deg`] sentinel: no tilt reading.
pub const PUNKTFUNK_PEN_TILT_UNKNOWN: u8 = 0xFF;
/// [`PunktfunkPenSample::azimuth_deg`] / `roll_deg` sentinel: no reading.
pub const PUNKTFUNK_PEN_ANGLE_UNKNOWN: u16 = 0xFFFF;
/// [`PunktfunkPenSample::distance`] sentinel: no hover-distance reading.
pub const PUNKTFUNK_PEN_DISTANCE_UNKNOWN: u16 = 0xFFFF;
/// One complete stylus state at one instant ([`punktfunk_connection_send_pen`];
/// design/pen-tablet-input.md). STATE-FULL, never an edge event: fill every field on every
/// sample (unknown axes take their `*_UNKNOWN` sentinel) — the host diffs consecutive samples
/// and synthesizes down/up/button transitions itself, which is what makes a lost datagram
/// self-heal. `x`/`y` are normalized `0.0..=1.0` in VIDEO-FRAME space (map your letterbox
/// before filling, exactly like wire touches).
#[cfg(feature = "quic")]
#[repr(C)]
#[derive(Clone, Copy)]
pub struct PunktfunkPenSample {
/// Normalized `0.0..=1.0` across the video frame. Must be finite.
pub x: f32,
/// Normalized `0.0..=1.0` across the video frame. Must be finite.
pub y: f32,
/// Tip force, `0..=65535` full scale (`0` while hovering).
pub pressure: u16,
/// Hover distance `0..=65534` (0 = at the hover floor), or `PUNKTFUNK_PEN_DISTANCE_UNKNOWN`.
pub distance: u16,
/// Tilt azimuth, degrees `0..=359` clockwise from north, or `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
pub azimuth_deg: u16,
/// Barrel roll (Apple Pencil Pro `rollAngle`), degrees `0..=359`, or
/// `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
pub roll_deg: u16,
/// µs since the previous sample in the same call (`0` for the first) — the coalesced
/// capture spacing.
pub dt_us: u16,
/// Bitfield of `PUNKTFUNK_PEN_*` state bits. Unknown bits are rejected (`InvalidArg`).
pub state: u8,
/// `PUNKTFUNK_PEN_TOOL_PEN` or `PUNKTFUNK_PEN_TOOL_ERASER`.
pub tool: u8,
/// Tilt from the surface normal, degrees `0..=90`, or `PUNKTFUNK_PEN_TILT_UNKNOWN`.
pub tilt_deg: u8,
/// Set to 0.
pub _reserved: [u8; 3],
}
#[cfg(feature = "quic")]
impl PunktfunkPenSample {
/// `None` = invalid field (non-finite coordinate, unknown state bit, unknown tool) —
/// embedder input is validated strictly, unlike the loss-tolerant wire decode.
fn to_sample(self) -> Option<crate::quic::PenSample> {
use crate::quic as q;
let known = q::PEN_IN_RANGE | q::PEN_TOUCHING | q::PEN_BARREL1 | q::PEN_BARREL2;
if !self.x.is_finite() || !self.y.is_finite() || self.state & !known != 0 {
return None;
}
let tool = match self.tool {
PUNKTFUNK_PEN_TOOL_PEN => q::PenTool::Pen,
PUNKTFUNK_PEN_TOOL_ERASER => q::PenTool::Eraser,
_ => return None,
};
Some(q::PenSample {
state: self.state,
tool,
x: self.x,
y: self.y,
pressure: self.pressure,
distance: self.distance,
tilt_deg: self.tilt_deg,
azimuth_deg: self.azimuth_deg,
roll_deg: self.roll_deg,
dt_us: self.dt_us,
})
}
}
/// Read an optional NUL-terminated UTF-8 string parameter; `Err` = invalid pointer/UTF-8.
#[cfg(feature = "quic")]
unsafe fn opt_cstr<'a>(p: *const std::os::raw::c_char) -> std::result::Result<Option<&'a str>, ()> {
@@ -988,6 +1076,12 @@ pub const PUNKTFUNK_HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// Host-capability bit in [`punktfunk_connection_host_caps`]: the host supports the shared
/// clipboard, so a client may offer the toggle. (Mirrors `quic::HOST_CAP_CLIPBOARD`.)
pub const PUNKTFUNK_HOST_CAP_CLIPBOARD: u8 = 0x02;
/// Host-capability bit in [`punktfunk_connection_host_caps`]: the host injects full-fidelity
/// stylus input, so a capable client splits pen contacts out of its touch path and sends them
/// via [`punktfunk_connection_send_pen`]; without the bit that call returns `Unsupported` and
/// the client keeps its pen-as-touch fallback. (Mirrors `quic::HOST_CAP_PEN`;
/// design/pen-tablet-input.md.)
pub const PUNKTFUNK_HOST_CAP_PEN: u8 = 0x10;
// Keep the ABI cap bits in lockstep with the wire constants (compile-time guard against drift).
#[cfg(feature = "quic")]
@@ -1001,6 +1095,15 @@ const _: () = {
assert!(PUNKTFUNK_CODEC_PYROWAVE == crate::quic::CODEC_PYROWAVE);
assert!(PUNKTFUNK_HOST_CAP_GAMEPAD_STATE == crate::quic::HOST_CAP_GAMEPAD_STATE);
assert!(PUNKTFUNK_HOST_CAP_CLIPBOARD == crate::quic::HOST_CAP_CLIPBOARD);
assert!(PUNKTFUNK_HOST_CAP_PEN == crate::quic::HOST_CAP_PEN);
assert!(PUNKTFUNK_PEN_IN_RANGE == crate::quic::PEN_IN_RANGE);
assert!(PUNKTFUNK_PEN_TOUCHING == crate::quic::PEN_TOUCHING);
assert!(PUNKTFUNK_PEN_BARREL1 == crate::quic::PEN_BARREL1);
assert!(PUNKTFUNK_PEN_BARREL2 == crate::quic::PEN_BARREL2);
assert!(PUNKTFUNK_PEN_BATCH_MAX as usize == crate::quic::PEN_BATCH_MAX);
assert!(PUNKTFUNK_PEN_TILT_UNKNOWN == crate::quic::PEN_TILT_UNKNOWN);
assert!(PUNKTFUNK_PEN_ANGLE_UNKNOWN == crate::quic::PEN_ANGLE_UNKNOWN);
assert!(PUNKTFUNK_PEN_DISTANCE_UNKNOWN == crate::quic::PEN_DISTANCE_UNKNOWN);
};
// Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift).
@@ -2763,6 +2866,51 @@ pub unsafe extern "C" fn punktfunk_connection_send_rich_input2(
})
}
/// Send one stylus sample batch — `count` (`1..=PUNKTFUNK_PEN_BATCH_MAX`) state-full
/// [`PunktfunkPenSample`]s, oldest first (a capture callback's coalesced samples) — as one
/// `0xCC/0x05` pen datagram (non-blocking enqueue; design/pen-tablet-input.md). Split longer
/// runs into consecutive calls. Gate on `punktfunk_connection_host_caps() &
/// PUNKTFUNK_HOST_CAP_PEN`: toward a host without the bit this returns
/// [`PunktfunkStatus::Unsupported`] — keep the pen-as-touch fallback there.
/// [`PunktfunkStatus::InvalidArg`] on a bad count or a bad sample (non-finite coordinate,
/// unknown state bit / tool).
///
/// # Safety
/// `c` is a valid connection handle; `samples` is null or points to `count` valid
/// [`PunktfunkPenSample`]s.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_send_pen(
c: *mut PunktfunkConnection,
samples: *const PunktfunkPenSample,
count: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if samples.is_null() {
return PunktfunkStatus::NullPointer;
}
if count == 0 || count > PUNKTFUNK_PEN_BATCH_MAX {
return PunktfunkStatus::InvalidArg;
}
let raw = unsafe { std::slice::from_raw_parts(samples, count as usize) };
let mut batch = [crate::quic::PenSample::default(); crate::quic::PEN_BATCH_MAX];
for (slot, s) in batch.iter_mut().zip(raw) {
match s.to_sample() {
Some(v) => *slot = v,
None => return PunktfunkStatus::InvalidArg,
}
}
match c.inner.send_pen(&batch[..count as usize]) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// The currently active session mode — the Welcome's, until an accepted
/// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect.
///
@@ -2977,9 +3125,10 @@ fn build_clip_event(
}
/// The host capability bitfield the session's `Welcome` carried — a bitfield of
/// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD`. A client tests
/// `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide whether to offer the shared-clipboard toggle.
/// Safe any time after connect.
/// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD` /
/// `PUNKTFUNK_HOST_CAP_PEN`. A client tests `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide
/// whether to offer the shared-clipboard toggle, `caps & PUNKTFUNK_HOST_CAP_PEN` before
/// sending stylus batches. Safe any time after connect.
///
/// # Safety
/// `c` is a valid connection handle; `caps` is writable (NULL is skipped).
+44 -5
View File
@@ -20,7 +20,7 @@ use crate::quic::{
RfiRequest, RichInput,
};
use crate::session::Frame;
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU32, AtomicU64, Ordering};
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU16, AtomicU32, AtomicU64, Ordering};
use std::sync::mpsc::{Receiver, RecvTimeoutError};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
@@ -104,8 +104,11 @@ pub struct NativeClient {
/// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing
/// audio-latency (and memory) without limit — mic is best-effort end to end.
mic_tx: tokio::sync::mpsc::Sender<(u32, u64, Vec<u8>)>,
/// Outbound rich input (DualSense touchpad / motion) → 0xCC datagrams by the worker.
rich_input_tx: tokio::sync::mpsc::UnboundedSender<RichInput>,
/// Outbound 0xCC rich-input plane, PRE-ENCODED datagrams: [`RichInput`] touchpad/motion
/// (encoded in [`NativeClient::send_rich_input`]) and stylus [`crate::quic::PenBatch`]es
/// (encoded in [`NativeClient::send_pen`]) share the channel — the worker's task just
/// forwards bytes, so a new 0xCC kind never touches the pump.
rich_input_tx: tokio::sync::mpsc::UnboundedSender<Vec<u8>>,
/// Outbound control-stream requests (mode switch, speed test) → the worker's control task.
/// Bounded ([`CTRL_QUEUE`]) — the requests are sparse; a full queue means the control task
/// is wedged/dead, and callers treat it like a closed session.
@@ -121,6 +124,9 @@ pub struct NativeClient {
/// Monotonic id for outbound fetches ([`NativeClient::clip_fetch`]); stays below
/// [`crate::clipboard::INBOUND_REQ_FLAG`] so it never collides with an inbound serve `req_id`.
next_xfer_id: AtomicU32,
/// Wrapping per-connection [`crate::quic::PenBatch::seq`] counter, stamped by
/// [`NativeClient::send_pen`] (the host's reorder gate compares it).
pen_seq: AtomicU16,
/// The host capability bitfield ([`crate::quic::Welcome::host_caps`]) — see
/// [`NativeClient::host_caps`].
pub host_caps: u8,
@@ -344,7 +350,7 @@ impl NativeClient {
std::sync::mpsc::sync_channel::<crate::quic::HostTiming>(HOST_TIMING_QUEUE);
let (input_tx, input_rx) = tokio::sync::mpsc::unbounded_channel::<InputEvent>();
let (mic_tx, mic_rx) = tokio::sync::mpsc::channel::<(u32, u64, Vec<u8>)>(MIC_QUEUE);
let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<RichInput>();
let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<Vec<u8>>();
let (ctrl_tx, ctrl_rx) = tokio::sync::mpsc::channel::<CtrlRequest>(CTRL_QUEUE);
let (clip_event_tx, clip_event_rx) =
std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE);
@@ -473,6 +479,7 @@ impl NativeClient {
clip: Mutex::new(clip_event_rx),
clip_cmd_tx,
next_xfer_id: AtomicU32::new(1),
pen_seq: AtomicU16::new(0),
host_caps: negotiated.host_caps,
probe,
shutdown,
@@ -1073,7 +1080,39 @@ impl NativeClient {
/// loss like every datagram. No-op unless the host runs the DualSense gamepad backend.
pub fn send_rich_input(&self, rich: RichInput) -> Result<()> {
self.rich_input_tx
.send(rich)
.send(rich.encode())
.map_err(|_| PunktfunkError::Closed)
}
/// Queue one stylus sample batch for delivery as a `0xCC/0x05` pen datagram
/// (design/pen-tablet-input.md). `samples` are state-full and oldest-first (a capture
/// callback's coalesced samples), at most [`crate::quic::PEN_BATCH_MAX`] per call — split
/// longer runs into consecutive calls so the stamped wrapping `seq` keeps them ordered.
/// Best-effort like every datagram: a lost batch self-heals on the next one (the samples
/// carry full state, the host diffs — see [`crate::quic::PenTracker`]).
///
/// **Heartbeat contract**: while the pen is in range or touching, repeat the last sample
/// at least every ~100 ms even when nothing changed (capture APIs are silent for a
/// stationary pen) — the host force-releases the stroke after
/// [`crate::quic::PEN_TOUCH_TIMEOUT_MS`] of silence as its dead-client failsafe.
///
/// Requires the host to have advertised [`crate::quic::HOST_CAP_PEN`]; toward an older
/// host this returns `Unsupported` (embedders keep their pen-as-touch fallback instead of
/// spraying 240 Hz datagrams the host drops unread).
pub fn send_pen(&self, samples: &[crate::quic::PenSample]) -> Result<()> {
if self.host_caps & crate::quic::HOST_CAP_PEN == 0 {
return Err(PunktfunkError::Unsupported(
"host did not advertise HOST_CAP_PEN",
));
}
if samples.is_empty() || samples.len() > crate::quic::PEN_BATCH_MAX {
return Err(PunktfunkError::InvalidArg(
"pen batch must hold 1..=PEN_BATCH_MAX samples",
));
}
let seq = self.pen_seq.fetch_add(1, Ordering::Relaxed);
self.rich_input_tx
.send(crate::quic::PenBatch::new(seq, samples).encode())
.map_err(|_| PunktfunkError::Closed)
}
+12 -3
View File
@@ -36,6 +36,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
};
let handshake::HandshakeOut {
conn,
ep,
session,
ctrl_send,
ctrl_recv,
@@ -99,11 +100,12 @@ pub(super) async fn run_pump(args: WorkerArgs) {
}
});
// Rich-input task: embedder DualSense touchpad / motion → 0xCC uplink datagrams.
// Rich-input task: pre-encoded 0xCC uplink datagrams (DualSense touchpad / motion, pen
// batches — encoded at the NativeClient surface so new plane kinds never touch the pump).
let rich_conn = conn.clone();
tokio::spawn(async move {
while let Some(rich) = rich_input_rx.recv().await {
let _ = rich_conn.send_datagram(rich.encode().into());
while let Some(d) = rich_input_rx.recv().await {
let _ = rich_conn.send_datagram(d.into());
}
});
@@ -191,4 +193,11 @@ pub(super) async fn run_pump(args: WorkerArgs) {
0
};
conn.close(close_code.into(), b"client closed");
// Flush the CONNECTION_CLOSE before the runtime is dropped (the same discipline as the pairing
// + probe paths). `close` only queues the frame — the endpoint driver puts it on the wire, and
// this fn is the body of a `block_on` whose runtime is dropped the instant it returns, so
// without this the driver could simply never be polled again. The host then saw a deliberate
// quit as silence: no `QUIT_CLOSE_CODE`, an 8 s idle timeout, and the keep-alive linger meant
// for an UNWANTED disconnect. Bounded — a host already gone must not delay the client's exit.
let _ = tokio::time::timeout(std::time::Duration::from_millis(300), ep.wait_idle()).await;
}
@@ -3,12 +3,17 @@
//! up the data-plane [`Session`]. A typed application close from the host surfaces as
//! [`PunktfunkError::Rejected`] instead of the generic transport error.
use super::super::*;
use super::*;
/// Everything [`run_pump`](super::run_pump) needs from a successful connect + handshake.
pub(super) struct HandshakeOut {
pub(super) conn: quinn::Connection,
/// The dialing endpoint, kept alive for the session so the pump can FLUSH its
/// `CONNECTION_CLOSE` before the runtime is dropped ([`super::run_pump`]). Dropping it here
/// left nothing to drive the close onto the wire, so a deliberate quit reached the host as
/// silence — an 8 s idle timeout with no quit code, which the host reads as an unwanted
/// disconnect and lingers the display for.
pub(super) ep: quinn::Endpoint,
pub(super) session: Session,
pub(super) ctrl_send: quinn::SendStream,
pub(super) ctrl_recv: io::MsgReader,
@@ -238,6 +243,7 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
match handshake.await {
Ok((session, send, recv, negotiated, host_caps)) => Ok(HandshakeOut {
conn,
ep,
session,
ctrl_send: send,
ctrl_recv: recv,
@@ -9,7 +9,6 @@
//! Keyboard/mouse/touch events pass through unchanged; an older host (no caps bit) keeps
//! getting the legacy per-transition gamepad events.
use super::super::*;
use super::*;
pub(super) async fn run(
+3 -2
View File
@@ -5,7 +5,7 @@ use crate::clipboard::{ClipCommand, ClipEventCore};
use crate::config::{CompositorPref, GamepadPref, Mode};
use crate::error::Result;
use crate::input::InputEvent;
use crate::quic::{HdrMeta, HidOutput, RichInput};
use crate::quic::{HdrMeta, HidOutput};
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU64};
use std::sync::mpsc::SyncSender;
use std::sync::{Arc, Mutex};
@@ -43,7 +43,8 @@ pub(crate) struct WorkerArgs {
pub(crate) cursor_state_tx: SyncSender<crate::quic::CursorState>,
pub(crate) input_rx: tokio::sync::mpsc::UnboundedReceiver<InputEvent>,
pub(crate) mic_rx: tokio::sync::mpsc::Receiver<(u32, u64, Vec<u8>)>,
pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<RichInput>,
/// Pre-encoded 0xCC datagrams (rich input AND pen batches — see `NativeClient.rich_input_tx`).
pub(crate) rich_input_rx: tokio::sync::mpsc::UnboundedReceiver<Vec<u8>>,
pub(crate) ctrl_rx: tokio::sync::mpsc::Receiver<CtrlRequest>,
pub(crate) ctrl_tx: tokio::sync::mpsc::Sender<CtrlRequest>,
/// Inbound clipboard event plane feed — the control task pushes ClipState/ClipOffer, the
+4 -1
View File
@@ -106,7 +106,10 @@ pub use stats::Stats;
/// (design/remote-desktop-sweep.md §8): the client's mouse-model chord tells the host who
/// renders the pointer. Additive; rides the existing control stream (a new message TYPE, which
/// pre-§8 hosts ignore), so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 12;
/// v13: added `punktfunk_connection_send_pen` — the stylus wire plane
/// (design/pen-tablet-input.md): a client sends `RICH_PEN` sample batches once the host
/// advertises `HOST_CAP_PEN`. Additive and capability-gated, so [`WIRE_VERSION`] is unchanged.
pub const ABI_VERSION: u32 = 13;
/// The punktfunk/1 **wire** version — what `Hello`/`Welcome` carry and hosts equality-check.
/// Deliberately its own constant: [`ABI_VERSION`] tracks the embeddable **C surface**
+12
View File
@@ -100,6 +100,18 @@ pub const CLIENT_CAP_CURSOR: u8 = 0x01;
/// [`HOST_CAP_TEXT_INPUT`], `0x01`/`0x02` are gamepad-state / clipboard.
pub const HOST_CAP_CURSOR: u8 = 0x08;
/// [`Welcome::host_caps`] bit: the host injects full-fidelity stylus input — it routes
/// [`PenBatch`](super::pen::PenBatch) `0xCC/0x05` datagrams (pressure, tilt, azimuth, barrel
/// roll, hover, eraser, barrel buttons) through the [`PenTracker`](super::pen::PenTracker)
/// into a virtual tablet device (design/pen-tablet-input.md). A capable client (Apple Pencil,
/// Android stylus) then splits pen contacts out of its finger/touch path and sends pen
/// batches; absent the bit it keeps folding the pen into touch/pointer like today, and
/// [`NativeClient::send_pen`](crate::client::NativeClient::send_pen) refuses to send. The
/// wire ships ahead of the backend (P0): no host sets this bit until the P1 injector lands —
/// which is exactly why the gate exists. `0x10` — `0x08` is [`HOST_CAP_CURSOR`], `0x04` is
/// [`HOST_CAP_TEXT_INPUT`], `0x01`/`0x02` are gamepad-state / clipboard.
pub const HOST_CAP_PEN: u8 = 0x10;
/// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
/// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
/// advertise this.
+8 -1
View File
@@ -162,6 +162,12 @@ pub(super) const RICH_TOUCHPAD: u8 = 0x01;
pub(super) const RICH_MOTION: u8 = 0x02;
pub(super) const RICH_TOUCHPAD_EX: u8 = 0x03;
pub(super) const RICH_HID_REPORT: u8 = 0x04;
/// Claimed by the stylus plane ([`PenBatch`](super::pen::PenBatch)), which is NOT a
/// [`RichInput`] variant: rich input is pad-indexed controller state consumed by the gamepad
/// backends, while a pen batch routes to the (P1) tablet injector — so it gets its own decoder
/// and [`RichInput::decode`] keeps returning `None` here (= the documented unknown-kind drop
/// on a pre-pen host). Registered in this list so 0xCC kind bytes stay unique.
pub(super) const RICH_PEN: u8 = 0x05;
/// Longest raw HID report a [`RichInput::HidReport`] / [`HidOutput::HidRaw`] can carry — the
/// 64-byte interrupt/feature report size every Valve controller uses (Triton input reports are
@@ -171,7 +177,8 @@ pub const HID_REPORT_MAX: usize = 64;
/// A rich client→host controller input beyond the fixed [`InputEvent`](crate::input::InputEvent):
/// the DualSense touchpad and motion sensors. `pad` is the gamepad index. Wire form is
/// `[0xCC][kind][fields…]` — variable-length and kind-tagged (forward-compatible: an unknown
/// kind decodes to `None` and is dropped).
/// kind decodes to `None` and is dropped). Kind `0x05` on this plane is the stylus batch
/// ([`PenBatch`](super::pen::PenBatch)) with its own decoder — see [`RICH_PEN`].
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum RichInput {
/// One touchpad contact. `x`/`y` are normalized `0..=65535` in SCREEN convention —
+3
View File
@@ -26,6 +26,7 @@
//! positional Hello/Welcome/Start codecs, `caps` the capability/codec-negotiation
//! vocabulary, `control` the typed control + clipboard messages, `pairing` the pairing
//! message codecs with [`pake`] the SPAKE2 itself, `datagram` the 0xC90xCF plane codecs,
//! `pen` the stylus batch (0xCC kind 0x05) + host stroke tracker,
//! [`io`] framed stream IO, `clock` skew estimation + mid-stream re-sync, [`endpoint`] the
//! quinn constructors, [`clipstream`] the per-transfer clipboard fetch streams. Every item
//! is re-exported here, so all existing `crate::quic::X` paths compile unchanged; each
@@ -46,6 +47,7 @@ mod control;
mod datagram;
mod handshake;
mod pairing;
mod pen;
/// quinn endpoint constructors. Host: self-signed identity (fresh, or persisted PEMs via
/// [`endpoint::server_with_identity`]). Client: fingerprint pinning / TOFU via
@@ -72,6 +74,7 @@ pub use control::*;
pub use datagram::*;
pub use handshake::*;
pub use pairing::*;
pub use pen::*;
// Typed rejection close codes + [`RejectReason`] live in `crate::reject` (ungated — the
// error enum references them even in `quic`-less builds) and are re-exported here so the
+691
View File
@@ -0,0 +1,691 @@
//! The stylus plane: full-fidelity pen input on the 0xCC rich-input datagram
//! (design/pen-tablet-input.md).
//!
//! A pen datagram is a batch of **state-full samples**: every sample carries the *complete*
//! pen state — in-range/touching/buttons/tool plus all axes — never an edge ("down"/"up")
//! event. The host diffs each sample against its own tracked state ([`PenTracker`]) and
//! synthesizes the transitions, so a lost datagram self-heals on the next sample: a dropped
//! "first contact" batch becomes a tip-down when the next in-contact sample arrives, and a
//! dropped lift heals on the next hover/out-of-range sample (with the
//! [`PEN_TOUCH_TIMEOUT_MS`] failsafe for a client that dies mid-stroke). That is what makes
//! the lossy datagram plane sound for a 240 Hz stylus without any reliable-delivery
//! machinery — the same idempotent-snapshot argument as [`GamepadSnapshot`]
//! (crate::input::GamepadSnapshot) and [`RichInput::HidReport`](super::RichInput).
//!
//! Batches are ordered by a wrapping `u16` sequence number and dropped **whole** when stale
//! ([`pen_seq_newer`]) — applying a stale state-full sample would rewind the stroke.
//!
//! Clients send this only after the host advertised [`HOST_CAP_PEN`](super::HOST_CAP_PEN);
//! a pre-pen host drops the unknown 0xCC kind by the plane's documented forward-compat rule.
use super::datagram::{RICH_INPUT_MAGIC, RICH_PEN};
/// [`PenSample::state`] bit: the pen is in the hover range of the surface. Implied by
/// [`PEN_TOUCHING`] (decode normalizes, so a client that only sets TOUCHING still produces a
/// coherent contact).
pub const PEN_IN_RANGE: u8 = 0x01;
/// [`PenSample::state`] bit: the tip is in contact with the surface.
pub const PEN_TOUCHING: u8 = 0x02;
/// [`PenSample::state`] bit: the primary barrel button (or the client's squeeze mapping) is held.
pub const PEN_BARREL1: u8 = 0x04;
/// [`PenSample::state`] bit: the secondary barrel button (or the client's double-tap mapping)
/// is held.
pub const PEN_BARREL2: u8 = 0x08;
/// [`PenSample::state`] bit, RESERVED: a predicted (not yet observed) sample. Never sent v1;
/// receivers MUST ignore samples carrying it until a capability negotiates otherwise
/// (design/pen-tablet-input.md §8).
pub const PEN_PREDICTED: u8 = 0x80;
/// The button subset of [`PenSample::state`].
const PEN_BUTTONS_MASK: u8 = PEN_BARREL1 | PEN_BARREL2;
/// [`PenSample::tilt_deg`] sentinel: the client has no tilt sensor / no reading.
pub const PEN_TILT_UNKNOWN: u8 = 0xFF;
/// [`PenSample::azimuth_deg`] / [`PenSample::roll_deg`] sentinel: no reading.
pub const PEN_ANGLE_UNKNOWN: u16 = 0xFFFF;
/// [`PenSample::distance`] sentinel: no hover-distance reading.
pub const PEN_DISTANCE_UNKNOWN: u16 = 0xFFFF;
/// Most samples one [`PenBatch`] can carry. Sized for coalesced capture at video-frame cadence
/// (240 Hz pen ÷ 30 fps = 8); a client producing more splits into consecutive batches.
pub const PEN_BATCH_MAX: usize = 8;
/// Wire length of one encoded [`PenSample`].
pub const PEN_SAMPLE_WIRE_LEN: usize = 21;
/// `[0xCC][0x05][flags][count][u16 seq LE]` — bytes before the first sample.
const PEN_HEADER_LEN: usize = 6;
/// Host-side failsafe (design/pen-tablet-input.md §2): a tracker still in range after this
/// many ms without a sample force-releases ([`PenTracker::force_release`]) — a client that
/// died mid-stroke must not leave the host's virtual pen inked-down forever. This makes the
/// **client heartbeat a wire contract**: capture APIs only fire on change, so a stationary
/// pen is naturally silent — senders MUST repeat the last sample at least every ~100 ms while
/// the pen is in range or touching (it re-decodes as pure Motion, harmless), keeping a live
/// stationary stroke two heartbeats clear of the deadline.
pub const PEN_TOUCH_TIMEOUT_MS: u32 = 200;
/// Which end of the stylus (or which mapped mode) a sample describes. A tool *switch* while in
/// range is a physical re-entry — [`PenTracker`] emits a full release + re-proximity, matching
/// how a real tablet treats each tool as its own proximity session.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
#[repr(u8)]
pub enum PenTool {
#[default]
Pen = 0,
Eraser = 1,
/// An unrecognized wire value (a future tool from a newer client) — injectors treat it as
/// [`PenTool::Pen`]. Inside a proximity session it inherits the session's tool instead of
/// forcing a spurious re-entry.
Unknown = 0xFF,
}
impl PenTool {
fn from_u8(v: u8) -> PenTool {
match v {
0 => PenTool::Pen,
1 => PenTool::Eraser,
_ => PenTool::Unknown,
}
}
}
/// One complete stylus state at one instant. All axes ride every sample (state-full — see the
/// module doc); unknown axes carry their sentinel, never 0.
///
/// `x`/`y` are normalized `0.0..=1.0` in **video-frame space** — the client maps its letterbox
/// / viewport before sending (exactly as its wire touches already do), so the host scales
/// straight to the streamed output. f32 keeps sub-pixel precision at any resolution.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct PenSample {
/// Bitfield of `PEN_*` state bits ([`PEN_IN_RANGE`] … [`PEN_PREDICTED`]).
pub state: u8,
/// Active tool. Meaningful while in range; [`PenTool::Unknown`] inherits (see [`PenTool`]).
pub tool: PenTool,
/// Normalized `0.0..=1.0` across the video frame (decode clamps; see [`PenBatch::decode`]).
pub x: f32,
/// Normalized `0.0..=1.0` across the video frame.
pub y: f32,
/// Tip force, `0..=65535` full scale, `0` while hovering. Injectors rescale (Windows pens
/// are 0..1024, uinput declares its own range) — full u16 keeps every source's precision.
pub pressure: u16,
/// Hover distance, `0..=65534` normalized (0 = touching the hover floor), or
/// [`PEN_DISTANCE_UNKNOWN`].
pub distance: u16,
/// Tilt from the surface normal, degrees `0..=90`, or [`PEN_TILT_UNKNOWN`]. Polar form —
/// what Apple capture and the GameStream wire both produce; injectors needing tiltX/tiltY
/// convert (design/pen-tablet-input.md §2).
pub tilt_deg: u8,
/// Tilt azimuth, degrees `0..=359` clockwise from north, or [`PEN_ANGLE_UNKNOWN`].
pub azimuth_deg: u16,
/// Barrel roll (Apple Pencil Pro `rollAngle`), degrees `0..=359`, or [`PEN_ANGLE_UNKNOWN`].
pub roll_deg: u16,
/// µs since the previous sample in the same batch (`0` for the first) — preserves the
/// coalesced capture spacing for injectors/consumers that pace.
pub dt_us: u16,
}
impl Default for PenSample {
fn default() -> PenSample {
PenSample {
state: 0,
tool: PenTool::Pen,
x: 0.0,
y: 0.0,
pressure: 0,
distance: PEN_DISTANCE_UNKNOWN,
tilt_deg: PEN_TILT_UNKNOWN,
azimuth_deg: PEN_ANGLE_UNKNOWN,
roll_deg: PEN_ANGLE_UNKNOWN,
dt_us: 0,
}
}
}
impl PenSample {
fn encode_into(&self, out: &mut Vec<u8>) {
out.push(self.state);
out.push(self.tool as u8);
out.extend_from_slice(&self.x.to_le_bytes());
out.extend_from_slice(&self.y.to_le_bytes());
out.extend_from_slice(&self.pressure.to_le_bytes());
out.extend_from_slice(&self.distance.to_le_bytes());
out.push(self.tilt_deg);
out.extend_from_slice(&self.azimuth_deg.to_le_bytes());
out.extend_from_slice(&self.roll_deg.to_le_bytes());
out.extend_from_slice(&self.dt_us.to_le_bytes());
}
/// Decode one sample from exactly [`PEN_SAMPLE_WIRE_LEN`] bytes (caller bounds-checks).
/// `None` on a non-finite coordinate — an attacker-forged NaN/∞ must never reach an
/// injector's pixel scaling. Finite out-of-range coordinates clamp to `0.0..=1.0` (a
/// stroke drifting a hair past the letterbox edge is real input, not corruption).
fn decode(b: &[u8]) -> Option<PenSample> {
let f32at = |o: usize| f32::from_le_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]]);
let u16at = |o: usize| u16::from_le_bytes([b[o], b[o + 1]]);
let (x, y) = (f32at(2), f32at(6));
if !x.is_finite() || !y.is_finite() {
return None;
}
Some(PenSample {
state: b[0],
tool: PenTool::from_u8(b[1]),
x: x.clamp(0.0, 1.0),
y: y.clamp(0.0, 1.0),
pressure: u16at(10),
distance: u16at(12),
tilt_deg: b[14],
azimuth_deg: u16at(15),
roll_deg: u16at(17),
dt_us: u16at(19),
})
}
}
/// One pen datagram: `[0xCC][0x05][flags][count][u16 seq LE]` + `count` ×
/// [`PEN_SAMPLE_WIRE_LEN`]-byte samples, oldest first. `flags` is reserved (sent 0, ignored on
/// decode — semantic changes take a new 0xCC kind, never a flag reinterpretation). `seq` is the
/// sender's wrapping batch counter, the reorder gate ([`pen_seq_newer`]).
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct PenBatch {
pub seq: u16,
count: u8,
samples: [PenSample; PEN_BATCH_MAX],
}
impl PenBatch {
/// Build a batch from up to [`PEN_BATCH_MAX`] samples (a longer slice truncates — senders
/// with more coalesced samples split into consecutive batches so nothing is lost).
pub fn new(seq: u16, samples: &[PenSample]) -> PenBatch {
let count = samples.len().min(PEN_BATCH_MAX);
let mut buf = [PenSample::default(); PEN_BATCH_MAX];
buf[..count].copy_from_slice(&samples[..count]);
PenBatch {
seq,
count: count as u8,
samples: buf,
}
}
/// The batch's samples, oldest first.
pub fn samples(&self) -> &[PenSample] {
&self.samples[..self.count as usize]
}
pub fn encode(&self) -> Vec<u8> {
let n = self.count as usize;
let mut out = Vec::with_capacity(PEN_HEADER_LEN + n * PEN_SAMPLE_WIRE_LEN);
out.extend_from_slice(&[RICH_INPUT_MAGIC, RICH_PEN, 0, self.count]);
out.extend_from_slice(&self.seq.to_le_bytes());
for s in self.samples() {
s.encode_into(&mut out);
}
out
}
/// Parse a pen datagram. `None` on bad tag/kind, an empty batch, or a forged coordinate
/// (see [`PenSample::decode`]). Every read is bounded: `count` clamps to the declared
/// value, the fixed maximum, AND what the buffer actually holds — a torn datagram yields
/// the complete samples that arrived, never an over-read (the
/// [`RichInput::HidReport`](super::RichInput) truncation contract).
pub fn decode(b: &[u8]) -> Option<PenBatch> {
if b.len() < PEN_HEADER_LEN || b[0] != RICH_INPUT_MAGIC || b[1] != RICH_PEN {
return None;
}
let count = (b[3] as usize)
.min(PEN_BATCH_MAX)
.min((b.len() - PEN_HEADER_LEN) / PEN_SAMPLE_WIRE_LEN);
if count == 0 {
return None;
}
let mut samples = [PenSample::default(); PEN_BATCH_MAX];
for (i, slot) in samples.iter_mut().enumerate().take(count) {
let o = PEN_HEADER_LEN + i * PEN_SAMPLE_WIRE_LEN;
*slot = PenSample::decode(&b[o..o + PEN_SAMPLE_WIRE_LEN])?;
}
Some(PenBatch {
seq: u16::from_le_bytes([b[4], b[5]]),
count: count as u8,
samples,
})
}
}
/// The batch reorder gate: is `new` strictly newer than `last` on the wrapping u16 circle?
/// `None` (nothing applied yet) always passes. The u16 analog of
/// [`GamepadSnapshot::seq_newer`](crate::input::GamepadSnapshot::seq_newer): newer ⇔ the
/// forward distance is `1..=0x7FFF`, so reordered stale batches drop and a wrap (65535 → 0)
/// still counts as newer.
pub fn pen_seq_newer(new: u16, last: Option<u16>) -> bool {
match last {
None => true,
Some(last) => (new.wrapping_sub(last) as i16) > 0,
}
}
/// One synthesized stroke transition, in the order [`PenTracker`] emits them for a sample:
/// `ProximityIn?` → `Motion` → `TipDown?` → `ButtonsChanged?` → `TipUp?` → `ProximityOut?`.
/// `Motion` precedes `TipDown` so the contact lands where the sample says; a release
/// ([`PenTracker::force_release`] or an out-of-range sample) orders
/// `ButtonsChanged?` → `TipUp?` → `ProximityOut` so nothing is left held.
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum PenTransition {
/// The pen entered hover range. Injectors map [`PenTool::Unknown`] to a plain pen.
ProximityIn { tool: PenTool },
/// Position + all axes moved to this sample's values (emitted for every in-range sample).
Motion { sample: PenSample },
/// The tip made contact.
TipDown,
/// Barrel buttons changed: `pressed` / `released` are disjoint `PEN_BARREL*` subsets.
ButtonsChanged { pressed: u8, released: u8 },
/// The tip lifted.
TipUp,
/// The pen left hover range.
ProximityOut,
}
/// The host-side stroke state machine (one per session): diffs state-full [`PenSample`]s
/// against tracked state and appends the synthesized [`PenTransition`]s. Pure and clock-free —
/// the owner arms its own [`PEN_TOUCH_TIMEOUT_MS`] timer over [`PenTracker::is_active`] and
/// calls [`PenTracker::force_release`] when it fires (and on session teardown).
#[derive(Debug, Default)]
pub struct PenTracker {
last_seq: Option<u16>,
in_range: bool,
touching: bool,
buttons: u8,
tool: PenTool,
}
impl PenTracker {
/// Apply one decoded batch, appending transitions to `out` (callers reuse the buffer). A
/// stale batch ([`pen_seq_newer`]) is dropped whole with no transitions. Samples carrying
/// the reserved [`PEN_PREDICTED`] bit are skipped (never injected — module doc).
pub fn apply(&mut self, batch: &PenBatch, out: &mut Vec<PenTransition>) {
if !pen_seq_newer(batch.seq, self.last_seq) {
return;
}
self.last_seq = Some(batch.seq);
for s in batch.samples() {
if s.state & PEN_PREDICTED != 0 {
continue;
}
self.apply_sample(s, out);
}
}
/// The tracker holds live state a dead client could leave stuck (in range, or mid-stroke).
pub fn is_active(&self) -> bool {
self.in_range || self.touching
}
/// Release everything held (buttons → tip → proximity) — the [`PEN_TOUCH_TIMEOUT_MS`]
/// failsafe and session teardown. Keeps the seq gate armed so a late stale datagram from
/// the dead stroke cannot re-apply after the release.
pub fn force_release(&mut self, out: &mut Vec<PenTransition>) {
if self.buttons != 0 {
out.push(PenTransition::ButtonsChanged {
pressed: 0,
released: self.buttons,
});
self.buttons = 0;
}
if self.touching {
out.push(PenTransition::TipUp);
self.touching = false;
}
if self.in_range {
out.push(PenTransition::ProximityOut);
self.in_range = false;
}
}
fn apply_sample(&mut self, s: &PenSample, out: &mut Vec<PenTransition>) {
let touching = s.state & PEN_TOUCHING != 0;
// Touching implies in-range (a contact IS a proximity) — normalize here once so every
// consumer sees coherent states.
let in_range = touching || s.state & PEN_IN_RANGE != 0;
if !in_range {
self.force_release(out);
return;
}
// Unknown inherits the session's tool (a newer client's future tool must not thrash
// proximity); outside a session it grounds to the default.
let tool = match s.tool {
PenTool::Unknown if self.in_range => self.tool,
t => t,
};
// A tool switch mid-session is a physical re-entry (see [`PenTool`]).
if self.in_range && tool != self.tool {
self.force_release(out);
}
if !self.in_range {
out.push(PenTransition::ProximityIn { tool });
self.in_range = true;
}
self.tool = tool;
out.push(PenTransition::Motion { sample: *s });
if touching && !self.touching {
out.push(PenTransition::TipDown);
self.touching = true;
}
let buttons = s.state & PEN_BUTTONS_MASK;
if buttons != self.buttons {
out.push(PenTransition::ButtonsChanged {
pressed: buttons & !self.buttons,
released: self.buttons & !buttons,
});
self.buttons = buttons;
}
if !touching && self.touching {
out.push(PenTransition::TipUp);
self.touching = false;
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::quic::RichInput;
fn hover(x: f32, y: f32) -> PenSample {
PenSample {
state: PEN_IN_RANGE,
x,
y,
distance: 300,
..Default::default()
}
}
fn touch(x: f32, y: f32, pressure: u16) -> PenSample {
PenSample {
state: PEN_IN_RANGE | PEN_TOUCHING,
x,
y,
pressure,
distance: 0,
tilt_deg: 35,
azimuth_deg: 180,
roll_deg: 90,
..Default::default()
}
}
#[test]
fn pen_batch_roundtrip_and_truncation() {
let samples = [hover(0.25, 0.5), touch(0.26, 0.5, 40000), {
let mut s = touch(0.27, 0.51, 42000);
s.state |= PEN_BARREL1;
s.dt_us = 4167;
s
}];
let b = PenBatch::new(7, &samples);
let d = b.encode();
assert_eq!(d[0], RICH_INPUT_MAGIC);
assert_eq!(d.len(), 6 + 3 * PEN_SAMPLE_WIRE_LEN);
let back = PenBatch::decode(&d).unwrap();
assert_eq!(back.seq, 7);
assert_eq!(back.samples(), &samples);
// A torn datagram yields exactly the complete samples that arrived — never an
// over-read, never a partial sample.
let torn = PenBatch::decode(&d[..6 + 2 * PEN_SAMPLE_WIRE_LEN + 5]).unwrap();
assert_eq!(torn.samples(), &samples[..2]);
// Header-only / empty batches and short buffers are rejected whole.
assert!(PenBatch::decode(&d[..PEN_HEADER_LEN]).is_none());
assert!(PenBatch::decode(&PenBatch::new(0, &[]).encode()).is_none());
// Wrong tag / wrong kind are None before any read.
let mut bad = d.clone();
bad[0] = 0xC8;
assert!(PenBatch::decode(&bad).is_none());
let mut bad = d;
bad[1] = 0x01; // RICH_TOUCHPAD
assert!(PenBatch::decode(&bad).is_none());
}
#[test]
fn pen_batch_oversize_truncates_and_flags_reserved() {
// 10 samples truncate to PEN_BATCH_MAX on construction (senders split instead).
let many: Vec<PenSample> = (0..10).map(|i| hover(i as f32 / 10.0, 0.5)).collect();
let b = PenBatch::new(1, &many);
assert_eq!(b.samples().len(), PEN_BATCH_MAX);
// A declared count larger than the payload clamps to what arrived.
let mut d = b.encode();
d[3] = 200;
assert_eq!(PenBatch::decode(&d).unwrap().samples().len(), PEN_BATCH_MAX);
// A nonzero reserved flags byte still parses (receivers MUST ignore).
d[2] = 0xAA;
assert!(PenBatch::decode(&d).is_some());
}
#[test]
fn pen_batch_rejects_forged_floats_and_clamps_stragglers() {
// NaN / ∞ coordinates kill the whole batch — nothing legitimate produces them.
for forged in [f32::NAN, f32::INFINITY, f32::NEG_INFINITY] {
let mut s = touch(0.5, 0.5, 1);
s.x = forged;
assert!(PenBatch::decode(&PenBatch::new(0, &[s]).encode()).is_none());
}
// A finite coordinate a hair outside the letterbox clamps instead (real input).
let mut s = hover(0.5, 0.5);
s.x = -0.01;
s.y = 1.25;
let back = PenBatch::decode(&PenBatch::new(0, &[s]).encode()).unwrap();
assert_eq!((back.samples()[0].x, back.samples()[0].y), (0.0, 1.0));
}
#[test]
fn pen_plane_is_disjoint_from_rich_input() {
// A pen datagram shares the 0xCC tag but is NOT a RichInput: a pre-pen host takes the
// documented unknown-kind drop, and the pen decoder rejects every RichInput kind.
let d = PenBatch::new(3, &[touch(0.5, 0.5, 100)]).encode();
assert!(RichInput::decode(&d).is_none());
let rich = RichInput::Touchpad {
pad: 0,
finger: 0,
active: true,
x: 1,
y: 2,
}
.encode();
assert!(PenBatch::decode(&rich).is_none());
}
#[test]
fn seq_gate_wraps_and_drops_stale() {
assert!(pen_seq_newer(0, None));
assert!(pen_seq_newer(6, Some(5)));
assert!(!pen_seq_newer(5, Some(5)));
assert!(!pen_seq_newer(4, Some(5)));
assert!(pen_seq_newer(2, Some(0xFFFE))); // wrap
assert!(!pen_seq_newer(0xFFFE, Some(2))); // stale across the wrap
}
/// Drives a tracker and returns the transitions of one batch.
fn run(t: &mut PenTracker, seq: u16, samples: &[PenSample]) -> Vec<PenTransition> {
let mut out = Vec::new();
t.apply(&PenBatch::new(seq, samples), &mut out);
out
}
#[test]
fn tracker_full_stroke_lifecycle() {
let mut t = PenTracker::default();
// Hover in → ProximityIn + Motion.
let h = hover(0.2, 0.2);
assert_eq!(
run(&mut t, 0, &[h]),
vec![
PenTransition::ProximityIn { tool: PenTool::Pen },
PenTransition::Motion { sample: h },
]
);
// Contact: Motion precedes TipDown so ink lands at the sample's position.
let c = touch(0.21, 0.2, 30000);
assert_eq!(
run(&mut t, 1, &[c]),
vec![PenTransition::Motion { sample: c }, PenTransition::TipDown]
);
assert!(t.is_active());
// Drag: motion only.
let m = touch(0.3, 0.25, 45000);
assert_eq!(
run(&mut t, 2, &[m]),
vec![PenTransition::Motion { sample: m }]
);
// Lift back to hover, then leave range: buttons(none) → TipUp, then ProximityOut.
let l = hover(0.3, 0.25);
assert_eq!(
run(&mut t, 3, &[l]),
vec![PenTransition::Motion { sample: l }, PenTransition::TipUp]
);
let gone = PenSample::default(); // state 0 = out of range
assert_eq!(run(&mut t, 4, &[gone]), vec![PenTransition::ProximityOut]);
assert!(!t.is_active());
}
#[test]
fn tracker_self_heals_lost_transitions() {
let mut t = PenTracker::default();
// The hover batch AND the tip-down batch were lost: the first surviving mid-stroke
// sample synthesizes the whole entry.
let m = touch(0.5, 0.5, 20000);
assert_eq!(
run(&mut t, 10, &[m]),
vec![
PenTransition::ProximityIn { tool: PenTool::Pen },
PenTransition::Motion { sample: m },
PenTransition::TipDown,
]
);
// The lift batch was lost; the next out-of-range sample heals it fully.
assert_eq!(
run(&mut t, 11, &[PenSample::default()]),
vec![PenTransition::TipUp, PenTransition::ProximityOut]
);
}
#[test]
fn tracker_drops_stale_batches_whole() {
let mut t = PenTracker::default();
let c = touch(0.5, 0.5, 100);
assert!(!run(&mut t, 5, &[c]).is_empty());
// A reordered older batch (a hover from before the contact) must not rewind the stroke.
assert!(run(&mut t, 4, &[hover(0.4, 0.4)]).is_empty());
assert!(t.is_active());
}
#[test]
fn tracker_buttons_and_eraser_reentry() {
let mut t = PenTracker::default();
let mut held = touch(0.5, 0.5, 100);
held.state |= PEN_BARREL1;
// Buttons apply after TipDown on entry…
assert_eq!(
run(&mut t, 0, &[held]),
vec![
PenTransition::ProximityIn { tool: PenTool::Pen },
PenTransition::Motion { sample: held },
PenTransition::TipDown,
PenTransition::ButtonsChanged {
pressed: PEN_BARREL1,
released: 0
},
]
);
// …swap BARREL1 → BARREL2 in one sample: one delta, both directions.
let mut swapped = held;
swapped.state = (swapped.state & !PEN_BARREL1) | PEN_BARREL2;
assert_eq!(
run(&mut t, 1, &[swapped]),
vec![
PenTransition::Motion { sample: swapped },
PenTransition::ButtonsChanged {
pressed: PEN_BARREL2,
released: PEN_BARREL1
},
]
);
// Tool switch to eraser mid-contact = full release + re-entry as the eraser, with the
// held button released first so nothing sticks across tools.
let mut erase = touch(0.5, 0.5, 200);
erase.tool = PenTool::Eraser;
assert_eq!(
run(&mut t, 2, &[erase]),
vec![
PenTransition::ButtonsChanged {
pressed: 0,
released: PEN_BARREL2
},
PenTransition::TipUp,
PenTransition::ProximityOut,
PenTransition::ProximityIn {
tool: PenTool::Eraser
},
PenTransition::Motion { sample: erase },
PenTransition::TipDown,
]
);
// Unknown tool inside the session inherits (no re-entry thrash from a newer client).
let mut unk = touch(0.51, 0.5, 210);
unk.tool = PenTool::Unknown;
assert_eq!(
run(&mut t, 3, &[unk]),
vec![PenTransition::Motion { sample: unk }]
);
}
#[test]
fn tracker_force_release_and_late_stale_datagram() {
let mut t = PenTracker::default();
let mut held = touch(0.5, 0.5, 100);
held.state |= PEN_BARREL2;
run(&mut t, 100, &[held]);
// The 200 ms failsafe / teardown: buttons → tip → proximity, all released.
let mut out = Vec::new();
t.force_release(&mut out);
assert_eq!(
out,
vec![
PenTransition::ButtonsChanged {
pressed: 0,
released: PEN_BARREL2
},
PenTransition::TipUp,
PenTransition::ProximityOut,
]
);
assert!(!t.is_active());
// Idempotent.
let mut out = Vec::new();
t.force_release(&mut out);
assert!(out.is_empty());
// The seq gate survives the release: a late datagram from the dead stroke is stale.
assert!(run(&mut t, 99, &[held]).is_empty());
assert!(!t.is_active());
// But the stroke after it proceeds normally.
assert!(!run(&mut t, 101, &[held]).is_empty());
assert!(t.is_active());
}
#[test]
fn tracker_skips_reserved_predicted_samples() {
let mut t = PenTracker::default();
let mut p = touch(0.5, 0.5, 100);
p.state |= PEN_PREDICTED;
// A v1 host must never inject a predicted sample — and skipping must not corrupt
// tracking for the real samples around it.
let real = touch(0.6, 0.5, 120);
let out = run(&mut t, 0, &[p, real]);
assert_eq!(
out,
vec![
PenTransition::ProximityIn { tool: PenTool::Pen },
PenTransition::Motion { sample: real },
PenTransition::TipDown,
]
);
}
}
+31
View File
@@ -175,6 +175,36 @@ pub fn capture_virtual_output(
},
) as pf_capture::CursorChannelSender
});
// The secure-desktop guard's actuator (`IOCTL_SET_CURSOR_FORWARD`): the capturer flips the
// driver's hardware-cursor declare off while UAC/Winlogon is up (the secure desktop renders
// only through the OS's software-cursor path) and back on at dismissal. The stand-down needs
// the same-mode re-commit that actualises the software-cursor default — driven here because
// topology commits belong under the vdisplay manager's lock, which pf-capture cannot take.
// Built for EVERY session (not just `want.hw_cursor`): a channel-less session can reuse a
// driver monitor whose cursor worker (an earlier session's) is still live and re-declaring —
// the flip is the only way to stop it; on a never-declared target the driver answers
// NOT_FOUND, which the capturer logs and ignores.
let target_id = target.target_id;
let cursor_forward: Option<pf_capture::CursorForwardSender> = Some({
std::sync::Arc::new(move |enable: bool| {
let req = pf_driver_proto::control::SetCursorForwardRequest {
target_id,
enable: enable as u32,
};
// SAFETY: `control_raw` is the pf-vdisplay control handle resolved above; it is
// never closed for the process lifetime (`send_cursor_forward`'s precondition).
unsafe {
crate::vdisplay::driver::send_cursor_forward(
windows::Win32::Foundation::HANDLE(control_raw as *mut core::ffi::c_void),
&req,
)?;
}
if !enable {
crate::vdisplay::manager::force_recommit();
}
Ok(())
}) as pf_capture::CursorForwardSender
});
pf_capture::open_idd_push(
target,
pref,
@@ -184,6 +214,7 @@ pub fn capture_virtual_output(
keep,
sender,
cursor_sender,
cursor_forward,
)
.map_err(|(e, _keep)| e.context("IDD-push capture open (no fallback)"))
}
+69
View File
@@ -8,6 +8,75 @@
use anyhow::Context;
use anyhow::Result;
/// Draw a scripted stylus stroke through the REAL pen chain — wire-shaped samples → the core
/// [`PenTracker`](punktfunk_core::quic::PenTracker) → the "Punktfunk Pen" uinput tablet — so
/// full-fidelity pen injection is validated without any client (design/pen-tablet-input.md P1):
/// hover in from the left, tip down, a sine stroke across the mapped output with a pressure
/// ramp + tilt sweep, tip up, hover out. Observe in Krita/GIMP with a pressure brush, or
/// `sudo libinput debug-events` (expect `TABLET_TOOL_PROXIMITY/TIP/AXIS`).
#[cfg(target_os = "linux")]
pub fn pen_test() -> Result<()> {
use punktfunk_core::quic::{
PenBatch, PenSample, PenTracker, PenTransition, PEN_IN_RANGE, PEN_TOUCHING,
};
use std::time::Duration;
let mut dev = crate::inject::pen::VirtualPen::create()?;
let mut tracker = PenTracker::default();
let mut out: Vec<PenTransition> = Vec::new();
// Compositors need a beat to enumerate the new evdev node before events count.
std::thread::sleep(Duration::from_secs(2));
let mut seq = 0u16;
let mut send = |tracker: &mut PenTracker, out: &mut Vec<PenTransition>, s: PenSample| {
out.clear();
tracker.apply(&PenBatch::new(seq, &[s]), out);
seq = seq.wrapping_add(1);
dev.apply_batch(out);
};
tracing::info!("pen-test: hover in, then a 3 s pressure-ramped sine stroke");
let hover = |x: f32| PenSample {
state: PEN_IN_RANGE,
x,
y: 0.5,
distance: 300,
..Default::default()
};
for i in 0..20 {
send(&mut tracker, &mut out, hover(0.05 + i as f32 * 0.005));
std::thread::sleep(Duration::from_millis(10));
}
const STEPS: u32 = 360;
for i in 0..=STEPS {
let t = i as f32 / STEPS as f32;
send(
&mut tracker,
&mut out,
PenSample {
state: PEN_IN_RANGE | PEN_TOUCHING,
x: 0.15 + 0.7 * t,
y: 0.5 + 0.2 * (t * std::f32::consts::TAU * 2.0).sin(),
// Ramp 10 % → 100 % so a pressure brush visibly widens along the stroke.
pressure: (6553.0 + 58982.0 * t) as u16,
distance: 0,
tilt_deg: 25 + (20.0 * t) as u8,
azimuth_deg: ((90.0 + 180.0 * t) as u16) % 360,
roll_deg: ((360.0 * t) as u16) % 360,
..Default::default()
},
);
std::thread::sleep(Duration::from_millis(8));
}
for i in 0..10 {
send(&mut tracker, &mut out, hover(0.85 + i as f32 * 0.005));
std::thread::sleep(Duration::from_millis(10));
}
send(&mut tracker, &mut out, PenSample::default()); // state 0 = out of range
tracing::info!("pen-test: done (stroke drawn, pen out of range) — device destroyed on exit");
Ok(())
}
/// Inject a scripted mouse + keyboard pattern through the session's input backend (libei on
/// KWin/GNOME, wlr on Sway). Lets us validate input injection without a Moonlight client.
#[cfg(target_os = "linux")]
+16 -4
View File
@@ -220,12 +220,13 @@ pub fn start(
rikeyid: i32,
params: AudioParams,
audio_cap: AudioCapSlot,
on_lost: super::OnSessionLost,
) {
let _ = std::thread::Builder::new()
.name("punktfunk-audio".into())
.spawn(move || {
tracing::info!(?params, "audio stream starting");
if let Err(e) = run(&running, &gcm_key, rikeyid, params, &audio_cap) {
if let Err(e) = run(&running, &gcm_key, rikeyid, params, &audio_cap, &on_lost) {
tracing::error!(error = %format!("{e:#}"), "audio stream failed");
}
running.store(false, Ordering::SeqCst);
@@ -243,6 +244,7 @@ pub fn start(
_rikeyid: i32,
_params: AudioParams,
_audio_cap: AudioCapSlot,
_on_lost: super::OnSessionLost,
) {
tracing::error!("GameStream audio requires Linux (PipeWire) or Windows (WASAPI) + libopus");
running.store(false, std::sync::atomic::Ordering::SeqCst);
@@ -255,6 +257,7 @@ fn run(
rikeyid: i32,
params: AudioParams,
audio_cap: &std::sync::Mutex<Option<Box<dyn AudioCapturer>>>,
on_lost: &super::OnSessionLost,
) -> Result<()> {
let sock = UdpSocket::bind(("0.0.0.0", AUDIO_PORT)).context("bind audio UDP")?;
// Grow SO_SNDBUF/RCVBUF; the opt-in DSCP/QoS tag happens after connect below (Windows
@@ -296,7 +299,7 @@ fn run(
}
None => audio::open_audio_capture(want).context("open audio capture")?,
};
let result = audio_body(&mut *cap, &sock, gcm_key, rikeyid, params, running);
let result = audio_body(&mut *cap, &sock, gcm_key, rikeyid, params, running, on_lost);
cap.idle(); // parked between sessions — release the routing claim (Linux stream sink)
audio::park_audio_capture(audio_cap, cap); // drop on Windows (restores the default), keep on Linux
result
@@ -355,6 +358,7 @@ impl SessionEncoder {
}
#[cfg(any(target_os = "linux", target_os = "windows"))]
#[allow(clippy::too_many_arguments)]
fn audio_body(
cap: &mut dyn AudioCapturer,
sock: &UdpSocket,
@@ -362,6 +366,9 @@ fn audio_body(
rikeyid: i32,
params: AudioParams,
running: &AtomicBool,
// Whole-session teardown for the client-unreachable send errors below — video would
// otherwise keep streaming at the dead endpoint (see `AppState::end_session`).
on_lost: &super::OnSessionLost,
) -> Result<()> {
let layout = layout_for(&params);
let mut enc = SessionEncoder::new(layout)?;
@@ -427,7 +434,8 @@ fn audio_body(
.encrypt_padded_vec_mut::<Pkcs7>(&out[..n]);
let pkt = build_rtp(seq, timestamp, &ct);
if sock.send(&pkt).is_err() {
tracing::info!(sent, "audio: client unreachable — stopping");
tracing::info!(sent, "audio: client unreachable — ending session");
on_lost();
return Ok(());
}
// Surround FEC: accumulate the encrypted payloads of the aligned 4-packet block;
@@ -449,7 +457,11 @@ fn audio_body(
let fp =
build_fec_rtp(rtp_seq, x as u8, fec_base_seq, fec_base_ts, par);
if sock.send(&fp).is_err() {
tracing::info!(sent, "audio: client unreachable — stopping");
tracing::info!(
sent,
"audio: client unreachable — ending session"
);
on_lost();
return Ok(());
}
}
@@ -71,6 +71,9 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
// from `seq`, so a per-message-type counter would reuse (key, nonce) pairs across
// message types in the host direction.
let mut pads = GamepadManager::new();
// Pen/touch translator (SS_PEN/SS_TOUCH → virtual tablet / wire touch). Sent only
// by clients that saw our SS_FF_PEN_TOUCH_EVENTS feature flag (rtsp.rs).
let mut pointer = super::pen::GsPointer::new();
let mut host_seq: u32 = 0;
// One-shot latch for the HDR-mode control message (0x010e); re-armed on Disconnect.
let mut hdr_sent = false;
@@ -80,18 +83,54 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
match host.service() {
Ok(Some(event)) => match event {
Event::Connect { peer: p, .. } => {
tracing::info!("control: client connected");
peer = Some(p.id());
// Track this peer as THE session peer only if it comes from the
// `/launch` owner's IP (when captured — `None` falls back to
// trusting the connect, the pre-teardown behavior). The tracked
// peer's disconnect now ENDS the session, so an unauthenticated
// LAN peer that connects+disconnects on 47999 must not be able to
// steal the slot and tear a live session down. Same source-IP
// bind the RTSP/media plane uses (security-review #4).
let owner_ip = state.launch.lock().unwrap().and_then(|s| s.peer_ip);
let from = p.address().map(|a| a.ip());
if owner_ip.is_some() && from.is_some() && owner_ip != from {
tracing::warn!(
?from,
"control: peer connected from a non-owner IP — ignoring"
);
} else {
tracing::info!("control: client connected");
peer = Some(p.id());
}
}
Event::Disconnect { .. } => {
Event::Disconnect { peer: p, .. } => {
// Gate on the TRACKED session peer: a stray probe peer (or the
// OLD peer's late timeout after a fast reconnect replaced it in
// the Connect arm) must neither clobber the live session's input
// state nor end its session.
if peer != Some(p.id()) {
tracing::debug!("control: non-session peer disconnected");
continue;
}
tracing::info!("control: client disconnected");
detected = None;
decrypt_fails = 0;
peer = None;
// Re-arm the HDR-mode signal for the next connection.
hdr_sent = false;
// Unplug the session's virtual pads.
// Unplug the session's virtual pads + tablet (destroying the
// uinput pen releases any held tool/tip kernel-side).
pads = GamepadManager::new();
pointer = super::pen::GsPointer::new();
// The control stream is the session's liveness anchor — Moonlight
// holds it for the whole stream, and ENet detects a vanished peer
// via its reliable-ping timeout (~530 s), which ALSO lands here.
// End the session: without this, a client that disconnects without
// an explicit RTSP TEARDOWN / nvhttp `/cancel` (a network drop,
// sleep, crash — or just a plain Moonlight quit, which sends
// neither) left the media threads streaming at the dead endpoint
// forever (a UDP send only errors on an ICMP port-unreachable) and
// the stale launch/streaming state wedged every reconnect.
state.end_session("control stream disconnected");
}
Event::Receive {
channel_id, packet, ..
@@ -104,6 +143,7 @@ pub fn spawn(state: Arc<AppState>) -> Result<()> {
&mut decrypt_fails,
&inj_tx,
&mut pads,
&mut pointer,
);
}
},
@@ -196,6 +236,7 @@ fn decode_rfi_range(pt: &[u8]) -> Option<(i64, i64)> {
/// Handle one received control packet: decrypt it (learning the GCM scheme on the first one),
/// decode any input event, and inject it into the host session.
#[allow(clippy::too_many_arguments)]
fn on_receive(
state: &AppState,
_channel_id: u8,
@@ -204,6 +245,7 @@ fn on_receive(
decrypt_fails: &mut u64,
inj_tx: &Sender<InputEvent>,
pads: &mut GamepadManager,
pointer: &mut super::pen::GsPointer,
) {
let Some(key) = state.launch.lock().unwrap().map(|s| s.gcm_key) else {
return; // control traffic before /launch — no key yet
@@ -274,6 +316,34 @@ fn on_receive(
return;
}
// Pen/touch extension events (Moonlight sends them only after seeing our feature flag):
// pen drives this session's virtual tablet; touch forwards as ordinary wire touches.
if let Some(p) = super::input::decode_pointer(&pt) {
pointer.apply(&p, |ev| {
let _ = inj_tx.send(ev);
});
return;
} else if super::input::is_pointer_magic(&pt) {
// A pointer magic that failed the body parse — a layout mismatch against this
// client, exactly what an on-glass "touch/pen does nothing" needs surfaced. The
// first few dump their bytes so the mismatch is diagnosable from the log alone.
static HEX_DUMPS: std::sync::atomic::AtomicU32 = std::sync::atomic::AtomicU32::new(0);
if HEX_DUMPS.fetch_add(1, std::sync::atomic::Ordering::Relaxed) < 5 {
let hex: String = pt.iter().map(|b| format!("{b:02x}")).collect();
tracing::warn!(
len = pt.len(),
bytes = %hex,
"gamestream: SS_TOUCH/SS_PEN packet failed to decode (malformed/unexpected layout)"
);
} else {
tracing::warn!(
len = pt.len(),
"gamestream: SS_TOUCH/SS_PEN packet failed to decode (malformed/unexpected layout)"
);
}
return;
}
let events = super::input::decode(&pt);
if events.is_empty() {
return; // keepalive / QoS / unhandled input kind
@@ -25,6 +25,8 @@ const MAGIC_MOUSE_BTN_UP: u32 = 0x09;
const MAGIC_SCROLL_GEN5: u32 = 0x0A;
const MAGIC_UTF8: u32 = 0x17;
const MAGIC_HSCROLL: u32 = 0x5500_0001;
const MAGIC_SS_TOUCH: u32 = 0x5500_0002;
const MAGIC_SS_PEN: u32 = 0x5500_0003;
/// `code` value marking a [`InputKind::MouseScroll`] as horizontal (vs `0` = vertical).
pub const SCROLL_HORIZONTAL: u32 = 1;
@@ -111,6 +113,106 @@ fn decode_input_packet(p: &[u8]) -> Option<InputEvent> {
})
}
/// One decoded `SS_PEN_PACKET` body (moonlight-common-c `Input.h`; all fields little-endian,
/// coordinates/pressure as normalized floats). Semantics — `pressure_or_distance` is pressure
/// (0..1, 0 = unknown) for contact events and hover distance (1 = farthest) for hover;
/// `rotation` is the tilt azimuth (0..360, `0xFFFF` unknown); `tilt` is degrees from the
/// surface normal (0..90, `0xFF` unknown) — are interpreted in [`super::pen`].
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct SsPen {
pub event_type: u8,
pub tool: u8,
pub buttons: u8,
pub x: f32,
pub y: f32,
pub pressure_or_distance: f32,
pub rotation: u16,
pub tilt: u8,
}
/// One decoded `SS_TOUCH_PACKET` body (same conventions as [`SsPen`]; contact-area fields are
/// not carried — the wire touch kinds have nowhere to put them yet).
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct SsTouch {
pub event_type: u8,
pub rotation: u16,
pub pointer_id: u32,
pub x: f32,
pub y: f32,
pub pressure_or_distance: f32,
}
/// A Sunshine-extension pointer event (sent only after we advertise
/// `SS_FF_PEN_TOUCH_EVENTS` — see [`super::rtsp`]).
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum SsPointer {
Pen(SsPen),
Touch(SsTouch),
}
/// Whether this control plaintext carries a pointer magic ([`decode_pointer`]'s domain) —
/// lets the caller tell "malformed pointer packet" (worth a warn) apart from "some other
/// message" when `decode_pointer` returns `None`.
pub fn is_pointer_magic(plaintext: &[u8]) -> bool {
plaintext.len() >= 12
&& u16::from_le_bytes([plaintext[0], plaintext[1]]) == INPUT_DATA_TYPE
&& matches!(
u32::from_le_bytes([plaintext[8], plaintext[9], plaintext[10], plaintext[11]]),
MAGIC_SS_TOUCH | MAGIC_SS_PEN
)
}
/// Decode a control plaintext into a pen/touch pointer event, or `None` for every other
/// message (the caller then falls through to [`decode`]). Bounds- and sanity-checked like the
/// rest of the plane: short bodies and non-finite floats (a forged NaN must never reach the
/// injectors' scaling) drop the packet whole.
pub fn decode_pointer(plaintext: &[u8]) -> Option<SsPointer> {
if plaintext.len() < 12 || u16::from_le_bytes([plaintext[0], plaintext[1]]) != INPUT_DATA_TYPE {
return None;
}
let p = &plaintext[4..];
let magic = u32::from_le_bytes([p[4], p[5], p[6], p[7]]);
let b = &p[8..];
// Coordinates must be finite (they feed the injectors' scaling); a forged NaN drops the
// packet whole.
let f32at = |o: usize| -> Option<f32> {
let v = f32::from_le_bytes([*b.get(o)?, *b.get(o + 1)?, *b.get(o + 2)?, *b.get(o + 3)?]);
v.is_finite().then_some(v)
};
// pressureOrDistance is different: real clients ship NaN there to mean "unknown" (iPad
// fingers have no force sensor — observed live from VoidLink, which NaN'd every touch and
// a strict gate silently killed the whole plane). The spec's own unknown value is 0.0, so
// non-finite sanitizes to that instead of poisoning the packet.
let f32_pressure = |o: usize| -> Option<f32> {
let v = f32::from_le_bytes([*b.get(o)?, *b.get(o + 1)?, *b.get(o + 2)?, *b.get(o + 3)?]);
Some(if v.is_finite() { v } else { 0.0 })
};
match magic {
// eventType, zero[1], rotation u16, pointerId u32, x, y, pressureOrDistance, areas.
MAGIC_SS_TOUCH => Some(SsPointer::Touch(SsTouch {
event_type: *b.first()?,
rotation: u16::from_le_bytes([*b.get(2)?, *b.get(3)?]),
pointer_id: u32::from_le_bytes([*b.get(4)?, *b.get(5)?, *b.get(6)?, *b.get(7)?]),
x: f32at(8)?,
y: f32at(12)?,
pressure_or_distance: f32_pressure(16)?,
})),
// eventType, toolType, penButtons, zero[1], x, y, pressureOrDistance, rotation u16,
// tilt, zero2[1], areas.
MAGIC_SS_PEN => Some(SsPointer::Pen(SsPen {
event_type: *b.first()?,
tool: *b.get(1)?,
buttons: *b.get(2)?,
x: f32at(4)?,
y: f32at(8)?,
pressure_or_distance: f32_pressure(12)?,
rotation: u16::from_le_bytes([*b.get(16)?, *b.get(17)?]),
tilt: *b.get(18)?,
})),
_ => None,
}
}
fn ev(kind: InputKind, code: u32, x: i32, y: i32, flags: u32) -> InputEvent {
InputEvent {
kind,
@@ -176,6 +278,78 @@ mod tests {
assert!(decode(&bad).is_empty());
}
#[test]
fn decodes_ss_pen_and_touch_golden_bytes() {
// SS_PEN body per Input.h: DOWN, pen tool, primary button, x=0.5 y=0.25,
// pressure=0.75, rotation=180, tilt=45, then contact areas (present but ignored).
let mut body = vec![0x01, 0x01, 0x01, 0x00];
for f in [0.5f32, 0.25, 0.75] {
body.extend_from_slice(&f.to_le_bytes());
}
body.extend_from_slice(&180u16.to_le_bytes());
body.extend_from_slice(&[45, 0x00]);
for f in [0.0f32, 0.0] {
body.extend_from_slice(&f.to_le_bytes());
}
let pt = wrap(0x5500_0003, &body);
assert_eq!(
decode_pointer(&pt),
Some(SsPointer::Pen(SsPen {
event_type: 0x01,
tool: 0x01,
buttons: 0x01,
x: 0.5,
y: 0.25,
pressure_or_distance: 0.75,
rotation: 180,
tilt: 45,
}))
);
// A pen packet is invisible to the classic decoder (no misparse as mouse/key).
assert!(decode(&pt).is_empty());
// SS_TOUCH body: MOVE, rotation unknown, pointerId 42, x=1.0 y=0.0, pressure 1.0.
let mut body = vec![0x03, 0x00];
body.extend_from_slice(&0xFFFFu16.to_le_bytes());
body.extend_from_slice(&42u32.to_le_bytes());
for f in [1.0f32, 0.0, 1.0, 0.0, 0.0] {
body.extend_from_slice(&f.to_le_bytes());
}
let pt = wrap(0x5500_0002, &body);
assert_eq!(
decode_pointer(&pt),
Some(SsPointer::Touch(SsTouch {
event_type: 0x03,
rotation: 0xFFFF,
pointer_id: 42,
x: 1.0,
y: 0.0,
pressure_or_distance: 1.0,
}))
);
// Truncated bodies and forged NaN coordinates drop the packet whole.
assert_eq!(decode_pointer(&pt[..pt.len() - 18]), None);
let mut nan = body.clone();
nan[8..12].copy_from_slice(&f32::NAN.to_le_bytes());
assert_eq!(decode_pointer(&wrap(0x5500_0002, &nan)), None);
// …but a NaN pressureOrDistance sanitizes to 0.0 ("unknown") instead of killing the
// packet — a REAL client convention, observed live: VoidLink NaN's the pressure of
// every iPad finger touch (no force sensor), and the strict gate silently disabled
// the whole touch plane.
let mut nan_pod = body.clone();
nan_pod[16..20].copy_from_slice(&f32::NAN.to_le_bytes());
match decode_pointer(&wrap(0x5500_0002, &nan_pod)) {
Some(SsPointer::Touch(t)) => assert_eq!(t.pressure_or_distance, 0.0),
other => panic!("NaN pressure must decode with pod=0.0, got {other:?}"),
}
// Non-pointer magics fall through to the classic decoder.
assert_eq!(
decode_pointer(&wrap(MAGIC_MOUSE_REL_GEN5, &[0, 0, 0, 0])),
None
);
}
#[test]
fn ignores_non_input_type() {
let mut pt = vec![0x00, 0x02]; // type 0x0200 (keepalive)
+104
View File
@@ -18,6 +18,8 @@ mod input;
mod mdns;
mod nvhttp;
mod pairing;
/// Moonlight `SS_PEN`/`SS_TOUCH` → the native pen model / wire touch (design/pen-tablet-input.md §4).
mod pen;
mod rtsp;
mod serverinfo;
mod stream;
@@ -180,7 +182,46 @@ pub struct AppState {
pub stats: Arc<crate::stats_recorder::StatsRecorder>,
}
/// Session-lost callback the media threads invoke when they detect the client is unreachable
/// (a UDP send error): ends the WHOLE GameStream session via [`AppState::end_session`], not just
/// the thread that noticed — video and audio otherwise stop independently and leave the launch
/// state behind. Built by the RTSP PLAY handler (the one place with the `Arc<AppState>`).
pub(crate) type OnSessionLost = Arc<dyn Fn() + Send + Sync>;
impl AppState {
/// End the GameStream session as one unit: signal BOTH media threads to stop (they observe
/// their `streaming`/`audio_streaming` flags) and clear the launch + negotiated stream
/// config. Idempotent — safe to call from every "the client is gone" site.
///
/// This is THE teardown for the compat plane. Anything less leaves a stale session behind:
/// a lingering `launch` 503-blocks a different client's `/launch` under
/// `mode_conflict = reject`, and a stale `streaming = true` makes a reconnect's RTSP PLAY
/// take its "stream already running" branch while the old threads still stream at the
/// vanished client's endpoint (no new threads are started — the reconnect gets no media).
/// Returns whether the video stream was live (for the caller's log line).
pub(crate) fn end_session(&self, reason: &str) -> bool {
use std::sync::atomic::Ordering;
let was_streaming = self.streaming.swap(false, Ordering::SeqCst);
let was_audio = self.audio_streaming.swap(false, Ordering::SeqCst);
let had_launch = self
.launch
.lock()
.unwrap_or_else(|e| e.into_inner())
.take()
.is_some();
self.stream.lock().unwrap_or_else(|e| e.into_inner()).take();
if was_streaming || was_audio || had_launch {
tracing::info!(
reason,
was_streaming,
was_audio,
had_launch,
"gamestream: session ended"
);
}
was_streaming
}
/// Fresh control-plane state: no active session; the pairing allow-list is loaded from
/// disk (pairings persist across restarts). `stats` is the shared recorder handed to both the
/// mgmt API and the streaming loops.
@@ -427,6 +468,69 @@ pub(crate) fn save_paired(paired: &[Vec<u8>]) {
}
}
#[cfg(test)]
mod session_tests {
use super::*;
fn test_state() -> AppState {
let host = Host {
hostname: "test-host".into(),
uniqueid: "deadbeef".into(),
local_ip: IpAddr::V4(Ipv4Addr::LOCALHOST),
http_port: HTTP_PORT,
https_port: HTTPS_PORT,
};
let identity = cert::ServerIdentity::ephemeral().expect("ephemeral identity");
let stats = crate::stats_recorder::StatsRecorder::new(std::env::temp_dir().join(format!(
"pf-gs-endsession-{}-{:p}",
std::process::id(),
&0u8 as *const u8
)));
AppState::new(host, identity, stats)
}
/// `end_session` is THE compat-plane teardown: one call must clear the whole session — both
/// media-thread flags, the launch, and the negotiated stream config — and be idempotent.
/// Guards the ENet-Disconnect / client-unreachable paths that previously stopped nothing
/// (the "session stays alive after the client disconnects" bug).
#[test]
fn end_session_clears_the_whole_session() {
use std::sync::atomic::Ordering;
let state = test_state();
state.streaming.store(true, Ordering::SeqCst);
state.audio_streaming.store(true, Ordering::SeqCst);
*state.launch.lock().unwrap() = Some(LaunchSession {
gcm_key: [0; 16],
rikeyid: 0,
width: 1920,
height: 1080,
fps: 60,
appid: 1,
peer_ip: None,
owner_fp: None,
});
*state.stream.lock().unwrap() = Some(stream::StreamConfig {
width: 1920,
height: 1080,
fps: 60,
packet_size: 1024,
bitrate_kbps: 20_000,
codec: crate::encode::Codec::H265,
min_fec: 0,
hdr: false,
});
assert!(state.end_session("test"), "video was live");
assert!(!state.streaming.load(Ordering::SeqCst));
assert!(!state.audio_streaming.load(Ordering::SeqCst));
assert!(state.launch.lock().unwrap().is_none());
assert!(state.stream.lock().unwrap().is_none());
// Idempotent: a second end (e.g. `/cancel` racing the ENet Disconnect) is a no-op.
assert!(!state.end_session("test again"));
}
}
#[cfg(all(test, unix))]
mod tests {
use std::os::unix::fs::PermissionsExt;
@@ -250,14 +250,9 @@ async fn h_cancel(
tracing::warn!("cancel rejected — caller does not own the session");
return xml(error_xml());
}
*st.launch.lock().unwrap() = None;
// Quit semantics: stop the running media threads (they observe these flags) so the session
// actually ends — the virtual output/gamescope teardown follows via the capturer's RAII.
st.streaming
.store(false, std::sync::atomic::Ordering::SeqCst);
st.audio_streaming
.store(false, std::sync::atomic::Ordering::SeqCst);
tracing::info!("cancel — launch session cleared, streams stopping");
// Quit semantics: the shared full teardown (launch cleared + both media threads stop on
// their flags) — the virtual output/gamescope teardown follows via the capturer's RAII.
st.end_session("client /cancel");
xml("<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<root status_code=\"200\"><cancel>1</cancel></root>\n".to_string())
}
+371
View File
@@ -0,0 +1,371 @@
//! Translate Moonlight's edge-based `SS_PEN` / `SS_TOUCH` events (design/pen-tablet-input.md
//! §4) into punktfunk's state-full pen model and wire-touch events.
//!
//! Pen: each packet becomes one complete [`PenSample`] (packet fields merged over the last
//! known state, since e.g. `BUTTON_ONLY` carries no position by spec), fed through the same
//! [`PenTracker`] → [`VirtualPen`] chain the native plane uses — so Moonlight iOS with an
//! Apple Pencil exercises the exact injection path our own clients will.
//!
//! No stroke timeout here, deliberately: the native plane's 200 ms failsafe exists because
//! datagrams are lossy and clients heartbeat; Moonlight's control stream is ordered/reliable
//! ENet and its clients do NOT heartbeat a stationary pen — a timeout would lift a held
//! stroke. Lost-client cleanup is the ENet disconnect (the control loop re-news this
//! translator, and destroying the uinput device releases everything kernel-side).
//!
//! Touch: forwarded as the ordinary wire touch kinds (`TouchDown/Move/Up`, normalized onto a
//! synthetic 65535² surface) through the injector service — pressure/area are dropped v1
//! (the wire touch has no field for them; `RICH_TOUCH` is the design's §8 upgrade).
use super::input::{SsPen, SsPointer, SsTouch};
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{
PenSample, PenTool, PenTracker, PenTransition, PEN_ANGLE_UNKNOWN, PEN_BARREL1, PEN_BARREL2,
PEN_IN_RANGE, PEN_TILT_UNKNOWN, PEN_TOUCHING,
};
// moonlight-common-c Limelight.h event/tool/button vocabulary.
const LI_TOUCH_EVENT_HOVER: u8 = 0x00;
const LI_TOUCH_EVENT_DOWN: u8 = 0x01;
const LI_TOUCH_EVENT_UP: u8 = 0x02;
const LI_TOUCH_EVENT_MOVE: u8 = 0x03;
const LI_TOUCH_EVENT_CANCEL: u8 = 0x04;
const LI_TOUCH_EVENT_BUTTON_ONLY: u8 = 0x05;
const LI_TOUCH_EVENT_HOVER_LEAVE: u8 = 0x06;
const LI_TOUCH_EVENT_CANCEL_ALL: u8 = 0x07;
const LI_TOOL_TYPE_PEN: u8 = 0x01;
const LI_TOOL_TYPE_ERASER: u8 = 0x02;
const LI_PEN_BUTTON_PRIMARY: u8 = 0x01;
const LI_PEN_BUTTON_SECONDARY: u8 = 0x02;
const LI_ROT_UNKNOWN: u16 = 0xFFFF;
const LI_TILT_UNKNOWN: u8 = 0xFF;
/// Synthetic reference extent for forwarded touches: coordinates arrive normalized, so map
/// them onto a full-range surface and let the injector rescale to its output (the
/// `MouseMoveAbs`/touch `flags = (w << 16) | h` contract).
const TOUCH_SURFACE: u32 = 65535;
/// Most concurrently-tracked touch pointers (for `CANCEL_ALL` replay). A flood of distinct
/// ids can't grow memory — untracked ids still forward down/up verbatim, they just miss a
/// later `CANCEL_ALL` (which a real client never has more than ~10 contacts for anyway).
const MAX_TOUCH_IDS: usize = 32;
/// Per-GameStream-session pen + touch translator. Owned by the control loop next to the
/// gamepad manager; re-created on client disconnect (the dropped [`VirtualPen`] destroys the
/// uinput device, which releases any held tool/tip kernel-side).
pub struct GsPointer {
tracker: PenTracker,
dev: Option<crate::inject::pen::VirtualPen>,
create_failed: bool,
seq: u16,
out: Vec<PenTransition>,
/// Last complete pen state — the merge base for packets that omit fields
/// (`BUTTON_ONLY` ignores x/y/pressure/tilt/rotation by spec).
last: PenSample,
/// This client sends hover events, so an `UP` means "back to hover" rather than "gone" —
/// clients without hover support get proximity-out on lift instead of a pen parked
/// forever at the last point.
saw_hover: bool,
/// Active forwarded touch ids, for `CANCEL_ALL` replay as per-id ups.
touch_ids: Vec<u32>,
/// Whether this session's first SS_TOUCH was logged (the one observable breadcrumb an
/// on-glass "touch does nothing" report needs — every later stage logs its own failure).
touch_seen: bool,
}
impl GsPointer {
pub fn new() -> GsPointer {
GsPointer {
tracker: PenTracker::default(),
dev: None,
create_failed: false,
seq: 0,
out: Vec::new(),
last: PenSample::default(),
saw_hover: false,
touch_ids: Vec::new(),
touch_seen: false,
}
}
/// Apply one decoded pointer packet. Touch events forward through `sink` (the injector
/// channel); pen events drive this session's virtual tablet.
pub fn apply(&mut self, p: &SsPointer, sink: impl FnMut(InputEvent)) {
match p {
SsPointer::Pen(pen) => self.apply_pen(pen),
SsPointer::Touch(touch) => self.apply_touch(touch, sink),
}
}
fn apply_pen(&mut self, p: &SsPen) {
let Some(sample) = self.pen_sample(p) else {
return;
};
self.last = sample;
if self.dev.is_none() && !self.create_failed {
match crate::inject::pen::VirtualPen::create() {
Ok(d) => self.dev = Some(d),
Err(e) => {
self.create_failed = true;
tracing::warn!(
error = %format!("{e:#}"),
"gamestream pen: virtual tablet creation failed — dropping pen input"
);
}
}
}
self.out.clear();
let batch = punktfunk_core::quic::PenBatch::new(self.seq, &[sample]);
self.seq = self.seq.wrapping_add(1);
self.tracker.apply(&batch, &mut self.out);
if let Some(dev) = self.dev.as_mut() {
dev.apply_batch(&self.out);
}
}
/// Merge one edge-based packet over the last known state into a complete sample.
/// `None` = nothing to apply (a `BUTTON_ONLY` before any positioned event).
fn pen_sample(&mut self, p: &SsPen) -> Option<PenSample> {
let buttons = (if p.buttons & LI_PEN_BUTTON_PRIMARY != 0 {
PEN_BARREL1
} else {
0
}) | (if p.buttons & LI_PEN_BUTTON_SECONDARY != 0 {
PEN_BARREL2
} else {
0
});
let tool = match p.tool {
LI_TOOL_TYPE_PEN => PenTool::Pen,
LI_TOOL_TYPE_ERASER => PenTool::Eraser,
_ => PenTool::Unknown,
};
// BUTTON_ONLY ignores x/y/pressure/rotation/tilt by spec — reuse the last state.
if p.event_type == LI_TOUCH_EVENT_BUTTON_ONLY {
if !self.tracker.is_active() {
return None;
}
let mut s = self.last;
s.state = (s.state & !(PEN_BARREL1 | PEN_BARREL2)) | buttons;
return Some(s);
}
let mut s = PenSample {
state: buttons,
tool,
x: p.x.clamp(0.0, 1.0),
y: p.y.clamp(0.0, 1.0),
dt_us: 0,
roll_deg: PEN_ANGLE_UNKNOWN, // the GameStream wire has no barrel-roll axis
azimuth_deg: if p.rotation == LI_ROT_UNKNOWN {
PEN_ANGLE_UNKNOWN
} else {
p.rotation % 360
},
tilt_deg: if p.tilt == LI_TILT_UNKNOWN {
PEN_TILT_UNKNOWN
} else {
p.tilt.min(90)
},
..PenSample::default()
};
match p.event_type {
LI_TOUCH_EVENT_DOWN | LI_TOUCH_EVENT_MOVE => {
s.state |= PEN_IN_RANGE | PEN_TOUCHING;
// pressureOrDistance is pressure (0..1) in contact — and 0.0 means UNKNOWN
// there, which must still ink: binary-stylus clients get full pressure.
s.pressure = if p.pressure_or_distance <= 0.0 {
u16::MAX
} else {
(p.pressure_or_distance.clamp(0.0, 1.0) * 65535.0) as u16
};
s.distance = 0;
}
LI_TOUCH_EVENT_HOVER => {
self.saw_hover = true;
s.state |= PEN_IN_RANGE;
// Hovering: pressureOrDistance is distance, 1.0 = farthest.
s.distance = (p.pressure_or_distance.clamp(0.0, 1.0) * 65534.0) as u16;
}
LI_TOUCH_EVENT_UP => {
// A hover-capable client keeps proximity after lift (its HOVER/HOVER_LEAVE
// events own the exit); one that never hovers would otherwise park the pen
// in proximity forever, so lift = leave for those.
if self.saw_hover {
s.state |= PEN_IN_RANGE;
}
}
LI_TOUCH_EVENT_HOVER_LEAVE | LI_TOUCH_EVENT_CANCEL | LI_TOUCH_EVENT_CANCEL_ALL => {
s.state &= !(PEN_BARREL1 | PEN_BARREL2); // out of range releases buttons too
}
_ => return None, // unknown future event type — drop, never guess
}
Some(s)
}
fn apply_touch(&mut self, t: &SsTouch, mut sink: impl FnMut(InputEvent)) {
if !self.touch_seen {
self.touch_seen = true;
tracing::info!(
event_type = t.event_type,
"gamestream: touch plane active (first SS_TOUCH from this client)"
);
}
let ev = |kind: InputKind, id: u32, x: f32, y: f32| InputEvent {
kind,
_pad: [0; 3],
code: id,
x: (x.clamp(0.0, 1.0) * TOUCH_SURFACE as f32) as i32,
y: (y.clamp(0.0, 1.0) * TOUCH_SURFACE as f32) as i32,
flags: (TOUCH_SURFACE << 16) | TOUCH_SURFACE,
};
match t.event_type {
LI_TOUCH_EVENT_DOWN => {
if !self.touch_ids.contains(&t.pointer_id) && self.touch_ids.len() < MAX_TOUCH_IDS {
self.touch_ids.push(t.pointer_id);
}
sink(ev(InputKind::TouchDown, t.pointer_id, t.x, t.y));
}
LI_TOUCH_EVENT_MOVE => sink(ev(InputKind::TouchMove, t.pointer_id, t.x, t.y)),
LI_TOUCH_EVENT_UP | LI_TOUCH_EVENT_CANCEL => {
self.touch_ids.retain(|&id| id != t.pointer_id);
sink(ev(InputKind::TouchUp, t.pointer_id, t.x, t.y));
}
LI_TOUCH_EVENT_CANCEL_ALL => {
for id in self.touch_ids.drain(..) {
sink(ev(InputKind::TouchUp, id, 0.0, 0.0));
}
}
// Touch hover has no wire representation (and BUTTON_ONLY is pen-only in
// practice) — nothing to forward.
_ => {}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn pen(event_type: u8, x: f32, y: f32, pod: f32) -> SsPen {
SsPen {
event_type,
tool: LI_TOOL_TYPE_PEN,
buttons: 0,
x,
y,
pressure_or_distance: pod,
rotation: 270,
tilt: 40,
}
}
#[test]
fn pen_down_move_up_maps_to_state_full_samples() {
let mut g = GsPointer::new();
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_DOWN, 0.25, 0.5, 0.5))
.unwrap();
assert_eq!(s.state, PEN_IN_RANGE | PEN_TOUCHING);
assert_eq!(s.pressure, 32767);
assert_eq!((s.tilt_deg, s.azimuth_deg), (40, 270));
assert_eq!(s.roll_deg, PEN_ANGLE_UNKNOWN);
// Unknown contact pressure (0.0) must still ink — binary-stylus full scale.
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_MOVE, 0.3, 0.5, 0.0))
.unwrap();
assert_eq!(s.pressure, u16::MAX);
// No hover ever seen: UP = fully out of range (no parked proximity).
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_UP, 0.3, 0.5, 0.0))
.unwrap();
assert_eq!(s.state, 0);
}
#[test]
fn hover_capable_client_keeps_proximity_on_lift() {
let mut g = GsPointer::new();
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_HOVER, 0.2, 0.2, 0.5))
.unwrap();
assert_eq!(s.state, PEN_IN_RANGE);
assert_eq!(s.distance, 32767);
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_UP, 0.2, 0.2, 0.0))
.unwrap();
assert_eq!(s.state, PEN_IN_RANGE);
// HOVER_LEAVE exits.
let s = g
.pen_sample(&pen(LI_TOUCH_EVENT_HOVER_LEAVE, 0.2, 0.2, 0.0))
.unwrap();
assert_eq!(s.state, 0);
}
#[test]
fn button_only_merges_over_last_state() {
let mut g = GsPointer::new();
// BUTTON_ONLY before any positioned event: nothing to merge over.
let mut b = pen(LI_TOUCH_EVENT_BUTTON_ONLY, 0.0, 0.0, 0.0);
b.buttons = LI_PEN_BUTTON_PRIMARY;
assert!(g.pen_sample(&b).is_none());
// After a DOWN is applied (through the real tracker path minus the device), the
// button packet reuses the held position/pressure.
g.apply_pen(&pen(LI_TOUCH_EVENT_DOWN, 0.4, 0.6, 0.8));
let s = g.pen_sample(&b).unwrap();
assert_eq!(s.state & (PEN_BARREL1 | PEN_BARREL2), PEN_BARREL1);
assert_eq!(s.state & PEN_TOUCHING, PEN_TOUCHING);
assert!((s.x - 0.4).abs() < 1e-6);
assert_eq!(s.pressure, (0.8f32 * 65535.0) as u16);
}
#[test]
fn eraser_and_unknown_tools_map() {
let mut g = GsPointer::new();
let mut p = pen(LI_TOUCH_EVENT_DOWN, 0.5, 0.5, 1.0);
p.tool = LI_TOOL_TYPE_ERASER;
assert_eq!(g.pen_sample(&p).unwrap().tool, PenTool::Eraser);
p.tool = 0x7F;
assert_eq!(g.pen_sample(&p).unwrap().tool, PenTool::Unknown);
// Unknown sentinel angles pass through as our sentinels.
p.rotation = LI_ROT_UNKNOWN;
p.tilt = LI_TILT_UNKNOWN;
let s = g.pen_sample(&p).unwrap();
assert_eq!(s.azimuth_deg, PEN_ANGLE_UNKNOWN);
assert_eq!(s.tilt_deg, PEN_TILT_UNKNOWN);
}
#[test]
fn touch_forwards_wire_touch_and_cancel_all_replays_ups() {
let mut g = GsPointer::new();
let mut got: Vec<InputEvent> = Vec::new();
let t = |event_type, pointer_id, x: f32| SsTouch {
event_type,
rotation: 0,
pointer_id,
x,
y: 0.5,
pressure_or_distance: 0.5,
};
g.apply_touch(&t(LI_TOUCH_EVENT_DOWN, 7, 0.5), |e| got.push(e));
g.apply_touch(&t(LI_TOUCH_EVENT_DOWN, 9, 0.25), |e| got.push(e));
g.apply_touch(&t(LI_TOUCH_EVENT_MOVE, 7, 0.75), |e| got.push(e));
assert_eq!(got.len(), 3);
assert_eq!(got[0].kind, InputKind::TouchDown);
assert_eq!(got[0].code, 7);
assert_eq!(got[0].x, 32767);
assert_eq!(got[0].flags, (65535 << 16) | 65535);
assert_eq!(got[2].kind, InputKind::TouchMove);
assert_eq!(got[2].x, 49151);
// CANCEL_ALL lifts every tracked contact.
got.clear();
g.apply_touch(&t(LI_TOUCH_EVENT_CANCEL_ALL, 0, 0.0), |e| got.push(e));
assert_eq!(got.len(), 2);
assert!(got.iter().all(|e| e.kind == InputKind::TouchUp));
let mut ids: Vec<u32> = got.iter().map(|e| e.code).collect();
ids.sort_unstable();
assert_eq!(ids, vec![7, 9]);
// And the set is empty now — a second CANCEL_ALL is a no-op.
got.clear();
g.apply_touch(&t(LI_TOUCH_EVENT_CANCEL_ALL, 0, 0.0), |e| got.push(e));
assert!(got.is_empty());
}
}
+29 -2
View File
@@ -190,7 +190,9 @@ fn authorized_launch(state: &AppState, peer: Option<SocketAddr>) -> Option<Launc
}
}
fn handle_request(req: &Request, state: &AppState, peer: Option<SocketAddr>) -> String {
// `&Arc<AppState>` (not `&AppState`): PLAY hands the media threads a `'static` session-lost
// callback, which needs an owned clone of the state.
fn handle_request(req: &Request, state: &Arc<AppState>, peer: Option<SocketAddr>) -> String {
match req.method.as_str() {
"OPTIONS" => response(
&req.cseq,
@@ -254,6 +256,15 @@ fn handle_request(req: &Request, state: &AppState, peer: Option<SocketAddr>) ->
return response_status("401 Unauthorized", &req.cseq, &[], None);
};
let cfg = *state.stream.lock().unwrap();
// Client-unreachable teardown for the media threads: ends the WHOLE session (both
// planes + launch state), so one plane detecting the dead client can't leave the
// other streaming at it — or leave a stale launch to wedge the next connect.
let on_lost: super::OnSessionLost = {
let st = state.clone();
Arc::new(move || {
st.end_session("client unreachable");
})
};
match cfg {
Some(cfg) if !state.streaming.swap(true, Ordering::SeqCst) => {
// Resolve the launched catalog entry (session recipe) for the stream.
@@ -267,6 +278,7 @@ fn handle_request(req: &Request, state: &AppState, peer: Option<SocketAddr>) ->
state.rfi_range.clone(),
state.video_cap.clone(),
state.stats.clone(),
on_lost.clone(),
);
}
Some(_) => tracing::info!("RTSP PLAY — stream already running"),
@@ -283,6 +295,7 @@ fn handle_request(req: &Request, state: &AppState, peer: Option<SocketAddr>) ->
ls.rikeyid,
*state.audio_params.lock().unwrap(),
state.audio_cap.clone(),
on_lost,
);
}
response(&req.cseq, &[("Session", "DEADBEEFCAFE;timeout = 90")], None)
@@ -309,12 +322,26 @@ fn handle_request(req: &Request, state: &AppState, peer: Option<SocketAddr>) ->
}
}
/// moonlight-common-c `LI_FF_PEN_TOUCH_EVENTS`: with this featureFlags bit set, Moonlight
/// clients send native `SS_PEN`/`SS_TOUCH` events (an iPad's Apple Pencil included) instead
/// of synthesizing mouse input client-side.
const SS_FF_PEN_TOUCH_EVENTS: u32 = 0x01;
/// Host capability SDP returned by DESCRIBE. Advertises HEVC + AV1 and no encryption
/// (plaintext streams for now; P1.5 adds the negotiated AES paths).
fn describe_sdp() -> String {
// Pen/touch events are advertised only where we can actually inject them (Linux with
// uinput — the same gate as HOST_CAP_PEN; design/pen-tablet-input.md §4). Elsewhere the
// flag stays 0 so Moonlight keeps its client-side mouse emulation for touch, exactly as
// today. PUNKTFUNK_PEN=0 clears it (the operator kill-switch inside pen_supported).
let feature_flags: u32 = if crate::inject::pen_supported() {
SS_FF_PEN_TOUCH_EVENTS
} else {
0
};
// Line-oriented a=key:value, matching what moonlight-common-c scans for.
let mut lines: Vec<String> = vec![
"a=x-ss-general.featureFlags:0".into(),
format!("a=x-ss-general.featureFlags:{feature_flags}"),
"a=x-ss-general.encryptionSupported:0".into(),
"a=x-ss-general.encryptionRequested:0".into(),
"sprop-parameter-sets=AAAAAU".into(), // HEVC capability indicator
+17 -2
View File
@@ -45,6 +45,7 @@ pub type RfiSlot = Arc<std::sync::Mutex<Option<(i64, i64)>>>;
/// Spawn the video stream thread (idempotent via `running`). Stops when `running` clears.
/// `force_idr` is set by the control stream on a client recovery request; `video_cap` holds
/// the persistent capturer the thread borrows for the stream's duration.
#[allow(clippy::too_many_arguments)]
pub fn start(
cfg: StreamConfig,
app: Option<super::apps::AppEntry>,
@@ -53,6 +54,7 @@ pub fn start(
rfi_range: RfiSlot,
video_cap: CapturerSlot,
stats: Arc<crate::stats_recorder::StatsRecorder>,
on_lost: super::OnSessionLost,
) {
let _ = std::thread::Builder::new()
.name("punktfunk-video".into())
@@ -103,6 +105,7 @@ pub fn start(
&rfi_range,
&video_cap,
&stats,
&on_lost,
);
// A clean return is a stop (RTSP teardown / cancel / client unreachable) → `quit`;
// an error return is `error`. The compat plane can't tell a user stop from an idle
@@ -137,6 +140,8 @@ fn run(
// Shared stats recorder for the web-console capture/graph. Threaded into `stream_body` (the
// encode loop); per-frame sample emission is wired by a later pass.
stats: &Arc<crate::stats_recorder::StatsRecorder>,
// Whole-session teardown for the send thread's client-unreachable detection.
on_lost: &super::OnSessionLost,
) -> Result<()> {
// GameStream capture/encode thread: apply Windows session tuning (no-op off Windows).
pf_frame::session_tuning::on_hot_thread();
@@ -252,6 +257,7 @@ fn run(
rfi_range,
stats,
&client_label,
on_lost,
);
}
@@ -299,6 +305,7 @@ fn run(
rfi_range,
stats,
&client_label,
on_lost,
);
capturer.set_active(false);
*video_cap.lock().unwrap() = Some((capturer, cfg.hdr));
@@ -572,12 +579,15 @@ fn spawn_packetizer(
/// shared [`send_pacing`](crate::send_pacing) policy at the GameStream parameterization: no
/// microburst stage, a BOUNDED step count (≤ 12, chunk ≥ 16, see the policy's docs for the
/// "send queue full" history that bound guards), each step ending in a sleep toward its slice
/// of the fixed budget. On send failure (client gone) it clears `running`.
/// of the fixed budget. On send failure (client gone) it ends the whole session via `on_lost` —
/// not just this thread: audio would otherwise keep streaming at the dead endpoint and the stale
/// launch state would wedge the next connect (see `AppState::end_session`).
fn spawn_sender(
sock: UdpSocket,
rx: std::sync::mpsc::Receiver<PacketBatch>,
frame_interval: Duration,
running: Arc<AtomicBool>,
on_lost: super::OnSessionLost,
) -> Result<()> {
std::thread::Builder::new()
.name("punktfunk-send".into())
@@ -613,8 +623,9 @@ fn spawn_sender(
},
);
if let Err(e) = r {
tracing::info!(error = %e, sent, "video: client unreachable — stopping stream");
tracing::info!(error = %e, sent, "video: client unreachable — ending session");
running.store(false, Ordering::SeqCst);
on_lost();
return;
}
}
@@ -645,6 +656,8 @@ fn stream_body(
stats: &Arc<crate::stats_recorder::StatsRecorder>,
// Short client label (peer IP) seeded into the capture meta on the first armed registration.
client_label: &str,
// Whole-session teardown, handed to the send thread's client-unreachable detection.
on_lost: &super::OnSessionLost,
) -> Result<()> {
// The first frame establishes the authoritative size/format for the encoder.
let mut frame = capturer.next_frame().context("capture first frame")?;
@@ -708,6 +721,7 @@ fn stream_body(
batch_rx,
Duration::from_secs_f64(1.0 / target_fps as f64),
running.clone(),
on_lost.clone(),
)?;
let (raw_tx, raw_rx) = std::sync::mpsc::sync_channel::<RawFrame>(2);
spawn_packetizer(raw_rx, batch_tx, pk, goodput.clone())?;
@@ -1075,6 +1089,7 @@ mod tests {
rx,
Duration::from_millis(8), // ~120fps frame interval
running.clone(),
Arc::new(|| {}),
)
.unwrap();
+5
View File
@@ -307,6 +307,11 @@ fn real_main() -> Result<()> {
// Standalone input-injection smoke test (no client needed): open the session's input
// backend and inject a scripted mouse/keyboard pattern. Watch a focused app / `wev`.
Some("input-test") => devtest::input_test(),
// Standalone stylus smoke test (no client needed): create the "Punktfunk Pen" virtual
// tablet and draw a pressure-ramped stroke through the real tracker→uinput chain.
// Watch in Krita/GIMP (pressure brush) or `libinput debug-events`.
#[cfg(target_os = "linux")]
Some("pen-test") => devtest::pen_test(),
// Zero-copy FFI/GPU probe: init the EGL importer + CUDA context (no capture needed).
#[cfg(target_os = "linux")]
Some("zerocopy-probe") => zerocopy::probe(),
+2 -5
View File
@@ -19,11 +19,8 @@ use std::sync::atomic::Ordering;
)
)]
pub(crate) async fn stop_session(State(st): State<Arc<MgmtState>>) -> StatusCode {
let was_streaming = st.app.streaming.swap(false, Ordering::SeqCst);
st.app.audio_streaming.store(false, Ordering::SeqCst);
*st.app.launch.lock().unwrap_or_else(|e| e.into_inner()) = None;
*st.app.stream.lock().unwrap_or_else(|e| e.into_inner()) = None;
// Native plane: the GameStream flags above don't reach it (it runs its own loops off the shared
let was_streaming = st.app.end_session("management API stop");
// Native plane: the GameStream teardown above doesn't reach it (it runs its own loops off the shared
// session registry), so signal every live native session to tear down too.
let native = crate::session_status::count();
crate::session_status::stop_all();
+87 -13
View File
@@ -392,11 +392,6 @@ pub(crate) async fn serve(
);
loop {
let permit = sem
.clone()
.acquire_owned()
.await
.expect("session semaphore is never closed");
let incoming = match ep.accept().await {
Some(i) => i,
None => break, // endpoint closed
@@ -409,9 +404,18 @@ pub(crate) async fn serve(
Ok(c) => c,
Err(e) => {
tracing::warn!(error = %e, "QUIC accept failed");
continue; // `permit` drops here → slot freed; not counted toward max_sessions
continue; // not counted toward max_sessions
}
};
// Take the session slot only AFTER the handshake, so a full host still ACCEPTS the
// connection and the waiting client sees a live path (quinn's keep-alive holds it) instead
// of a silent dial timeout — previously the loop parked on this await before `accept()`, so
// a host at its concurrency cap looked simply unreachable.
let permit = sem
.clone()
.acquire_owned()
.await
.expect("session semaphore is never closed");
let peer = conn.remote_address();
tracing::info!(%peer, "punktfunk/1 client connected");
let opts = opts.clone();
@@ -486,6 +490,31 @@ pub(crate) async fn serve(
/// connects and never finishes the handshake would otherwise wedge the host for everyone.
const HANDSHAKE_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
/// How long the stream thread may still run AFTER its session was told to stop before
/// [`serve_session`] gives up waiting for it.
///
/// Must exceed every legitimate post-stop path so a slow-but-healthy teardown is never abandoned:
/// the capture-loss rebuild budget is 40 s and one pipeline-build attempt can take ~10 s on a cold
/// compositor, so 90 s leaves generous headroom.
const STREAM_STOP_GRACE: std::time::Duration = std::time::Duration::from_secs(90);
/// How long teardown waits for the audio + input threads once the connection is closed. They exit
/// promptly by construction (the audio loop checks `stop` every ≤5 s; the input thread's channel
/// drops with the connection), so this only catches a genuine wedge.
const SIDE_THREAD_JOIN_GRACE: std::time::Duration = std::time::Duration::from_secs(10);
/// Resolves once `stop` has been set for [`STREAM_STOP_GRACE`] — i.e. the session was told to end
/// and its stream thread *still* hasn't returned.
///
/// Polled rather than notified: `stop` is a plain flag shared with blocking threads, and the poll
/// only runs while a session is live (every 500 ms, one relaxed atomic load).
async fn stop_overdue(stop: &AtomicBool) {
while !stop.load(Ordering::SeqCst) {
tokio::time::sleep(std::time::Duration::from_millis(500)).await;
}
tokio::time::sleep(STREAM_STOP_GRACE).await;
}
/// QUIC application error code the host closes with on a `mode_conflict = reject` admission refusal,
/// carrying the human-readable busy reason (live mode + client label) the client surfaces. A distinct
/// code lets a client tell "host busy" apart from a transport failure. Shared with the clients via
@@ -1066,6 +1095,14 @@ async fn serve_session(
if rich_tx.send(ClientInput::Rich(rich)).is_err() {
break;
}
} else if let Some(pen) = punktfunk_core::quic::PenBatch::decode(&d) {
// 0xCC kind 0x05 — the stylus plane (RichInput::decode returns None for it by
// design; see punktfunk_core::quic::pen). Routed to the same input thread,
// which owns the per-session tracker + virtual tablet.
rich_count += 1;
if rich_tx.send(ClientInput::Pen(pen)).is_err() {
break;
}
} else if let Some(mut ev) = InputEvent::decode(&d) {
input_count += 1;
// Wire hygiene: KEY_FLAG_SEMANTIC_VK is an in-process tag (GameStream ingest
@@ -1322,7 +1359,7 @@ async fn serve_session(
let bringup_dp = bringup.clone();
let resize_ms_dp = resize_ms.clone();
let result: Result<()> = async {
tokio::task::spawn_blocking(move || -> Result<()> {
let stream_thread = tokio::task::spawn_blocking(move || -> Result<()> {
// Bring up the (already-bound) data-plane socket. Default: hole-punch — wait briefly
// for the client's punch, then stream to its OBSERVED source, so video traverses a
// NAT / stateful inter-VLAN firewall (control + side planes ride the client-initiated
@@ -1436,9 +1473,34 @@ async fn serve_session(
}
}
}
})
.await
.context("stream thread")??;
});
// `stop` is only ADVISORY: the stream thread observes it between iterations, so a call that
// blocks without a bound INSIDE one (a compositor CLI that never returns, a D-Bus round-trip
// on a stuck bus, a driver wait on a hung GPU) never reaches the check — and nothing else
// can end the session, because every teardown below runs only once this await resolves. That
// made one stuck syscall a permanent zombie: it kept its semaphore slot (four of them and the
// host stops accepting entirely), its admission entry (a later client gets "host busy"
// forever) and its stream marker, and even the console's Stop button — which just sets this
// same flag — could not clear it.
//
// So bound the wait: once the session HAS been told to stop, give the thread
// `STREAM_STOP_GRACE` to return, then stop waiting for it and let teardown run. The thread is
// detached, not killed (a blocking thread can't be cancelled in Rust) — it keeps its capturer
// and encoder until the stuck call returns, and its own guards unwind if it ever does. That
// is a leak, but a bounded one: the session's slot and admission entry come back, so the rest
// of the host keeps serving.
tokio::select! {
joined = stream_thread => joined.context("stream thread")??,
() = stop_overdue(&stop) => {
tracing::error!(
grace_s = STREAM_STOP_GRACE.as_secs(),
"stream thread has not returned since the session was stopped — abandoning it so \
the session slot is freed. Its capture/encoder stay held until the stuck call \
returns; this is a HOST WEDGE please report it with the log above"
);
anyhow::bail!("stream thread wedged after stop");
}
}
// Give the client a moment to drain before the close.
tokio::time::sleep(std::time::Duration::from_secs(1)).await;
Ok(())
@@ -1454,13 +1516,25 @@ async fn serve_session(
if result.is_ok() { 0u32 } else { 1u32 }.into(),
if result.is_ok() { b"done" } else { b"error" },
);
let _ = tokio::task::spawn_blocking(move || {
// Bounded, for the same reason the stream-thread wait is: the input thread exits only when the
// datagram task drops its channel, which the `conn.close()` above forces — but a join is the
// last unbounded await in teardown, and one stuck side thread must not hold the session's
// permit/admission entry (released when this fn returns) hostage.
let side_threads = tokio::task::spawn_blocking(move || {
if let Some(h) = audio_handle {
let _ = h.join();
}
let _ = input_handle.join();
})
.await;
});
if tokio::time::timeout(SIDE_THREAD_JOIN_GRACE, side_threads)
.await
.is_err()
{
tracing::warn!(
grace_s = SIDE_THREAD_JOIN_GRACE.as_secs(),
"audio/input threads did not exit after the connection closed — detaching them"
);
}
// The capture (and our gamescope session's VirtualOutput) are gone by here. If this was the
// host-managed gamescope path on a box that autologs into gaming mode (Bazzite default), put the
// TV's gaming session back so it's the default when no one is streaming.
+72 -31
View File
@@ -3,7 +3,7 @@
//! the live platform) to a backend this host can actually build, applying the runtime UHID /
//! Steam-conflict degrades. Pure selection ([`pick_gamepad`]) is separated from the env/logging
//! shell ([`resolve_gamepad`]) and the per-pad variant ([`resolve_pad_kind`]); the platform degrades
//! (`degrade_if_no_uhid`, `physical_steam_controller_present`, `degrade_steam_on_conflict`) are
//! (`degrade_if_no_uhid`, `physical_steam_product_present`, `degrade_steam_on_conflict`) are
//! cfg-split linux/other and MUST be re-verified on Windows when touched (Linux clippy can't see the
//! non-linux copies).
@@ -128,31 +128,53 @@ fn degrade_if_no_uhid(chosen: GamepadPref) -> GamepadPref {
chosen
}
/// True if a **physical** Valve Steam controller (`28DE`) is already attached. The host's own Steam
/// Input is then managing a `28DE` device, and presenting a second (virtual) one makes Steam juggle
/// two Decks — confirmed conflict-prone on a Deck-as-host (the physical `28DE:1205` + Steam's
/// `28DE:11FF` XInput output pad are both live). HID device dirs are named `BUS:VID:PID.INST`
/// (uppercase); a UHID virtual device resolves through `/devices/virtual/…`, a real one does not.
///
/// Punktfunk's OWN virtual Decks must never count: the usbip/gadget transports present a real USB
/// device (vhci resolves through `vhci_hcd`, NOT `/devices/virtual/`), so a just-ended session's
/// pad still detaching — or a concurrent session's live one — read as "physical" and degraded
/// every back-to-back Deck session to DualSense (observed live on Bazzite 2026-07-04). Ours are
/// recognizable by the `FVPF…` serial ([`steam_proto::deck_serial`]) in `HID_UNIQ`, with the
/// vhci path as belt and braces.
/// The Valve product id (`28DE:xxxx`) a virtual Steam backend enumerates as, or `None` for a
/// non-Steam backend. This is the identity the conflict gate compares against the *physical* Valve
/// devices attached to the host: only a genuine duplicate (same VID **and** PID) confuses Steam
/// Input, so the gate keys on the PID, not on the `28DE` vendor alone.
#[cfg(target_os = "linux")]
fn physical_steam_controller_present() -> bool {
fn steam_backend_product(pref: GamepadPref) -> Option<u16> {
match pref {
GamepadPref::SteamDeck => Some(0x1205), // built-in Deck controller identity
GamepadPref::SteamController => Some(0x1102), // classic wired Steam Controller
GamepadPref::SteamController2 => Some(0x1302), // Steam Controller 2 (Triton), wired
GamepadPref::SteamController2Puck => Some(0x1304), // Steam Controller 2 via its Puck dongle
_ => None,
}
}
/// True if a **physical** Valve device with this exact product id (`28DE:{product}`) is already
/// attached. The host's own Steam Input is then managing that identity, and presenting a *second,
/// identical* virtual one makes Steam juggle two copies of the same Deck — confirmed conflict-prone
/// on a Deck-as-host (the physical `28DE:1205` + Steam's `28DE:11FF` XInput output pad are both
/// live).
///
/// Crucially this keys on the PID, not just the `28DE` vendor: a *different* Valve product is not a
/// conflict. A physical Steam Controller 2 (`28DE:1302`) must NOT block a virtual Steam Deck
/// (`28DE:1205`) — Steam Input drives distinct Steam controllers side by side, so the wanted pad
/// has to pass through unharmed (this over-broad vendor match degraded such sessions to DualSense).
///
/// HID device dirs are named `BUS:VID:PID.INST` (uppercase); a UHID virtual device resolves through
/// `/devices/virtual/…`, a real one does not. Punktfunk's OWN virtual pads must never count: the
/// usbip/gadget transports present a real USB device (vhci resolves through `vhci_hcd`, NOT
/// `/devices/virtual/`), so a just-ended session's pad still detaching — or a concurrent session's
/// live one — read as "physical" and degraded every back-to-back Deck session to DualSense
/// (observed live on Bazzite 2026-07-04). Ours are recognizable by the `FVPF…` serial
/// ([`steam_proto::deck_serial`]) in `HID_UNIQ`, with the vhci path as belt and braces.
#[cfg(target_os = "linux")]
fn physical_steam_product_present(product: u16) -> bool {
let needle = format!(":28DE:{product:04X}");
let Ok(entries) = std::fs::read_dir("/sys/bus/hid/devices") else {
return false;
};
entries.flatten().any(|e| {
if !e.file_name().to_string_lossy().contains(":28DE:") {
if !e.file_name().to_string_lossy().contains(&needle) {
return false;
}
if std::fs::read_to_string(e.path().join("uevent"))
.is_ok_and(|u| u.lines().any(|l| l.starts_with("HID_UNIQ=FVPF")))
{
return false; // one of our own virtual Decks
return false; // one of our own virtual pads
}
match std::fs::read_link(e.path()) {
Ok(target) => {
@@ -164,28 +186,30 @@ fn physical_steam_controller_present() -> bool {
})
}
/// Gate a virtual Steam pad off when a physical Steam controller is attached (§ conflict). Degrade to
/// DualSense (then the uhid ladder), which Steam treats as an ordinary, distinct pad. Override with
/// `PUNKTFUNK_STEAM_FORCE=1` when the host has no competing Steam Input (e.g. a remote-only box).
/// Gate a virtual Steam pad off when a physical Steam controller *of the same identity* is attached
/// (§ conflict). Degrade to DualSense (then the uhid ladder), which Steam treats as an ordinary,
/// distinct pad. Override with `PUNKTFUNK_STEAM_FORCE=1` when the host has no competing Steam Input
/// (e.g. a remote-only box).
///
/// Only a same-PID duplicate degrades: a physical Steam Controller 2 no longer blocks a virtual
/// Steam Deck (and vice versa), so a Steam Machine with an SC2 plugged in streams the pad the client
/// actually asked for.
#[cfg(target_os = "linux")]
fn degrade_steam_on_conflict(chosen: GamepadPref) -> GamepadPref {
if !matches!(
chosen,
GamepadPref::SteamDeck
| GamepadPref::SteamController
| GamepadPref::SteamController2
| GamepadPref::SteamController2Puck
) {
return chosen;
}
let Some(product) = steam_backend_product(chosen) else {
return chosen; // not a virtual Steam (28DE) pad — nothing to gate
};
let forced = std::env::var("PUNKTFUNK_STEAM_FORCE")
.map(|v| v == "1" || v.eq_ignore_ascii_case("true"))
.unwrap_or(false);
if !forced && physical_steam_controller_present() {
if !forced && physical_steam_product_present(product) {
let conflict = format!("28DE:{product:04X}");
tracing::warn!(
wanted = chosen.as_str(),
"a physical Steam controller is attached — the host's Steam Input would manage two 28DE \
devices; falling back to DualSense (set PUNKTFUNK_STEAM_FORCE=1 to override)"
conflict = conflict.as_str(),
"a physical Steam controller of the same identity is attached — the host's Steam Input \
would manage two identical 28DE devices; falling back to DualSense (set \
PUNKTFUNK_STEAM_FORCE=1 to override)"
);
return degrade_if_no_uhid(GamepadPref::DualSense);
}
@@ -383,4 +407,21 @@ mod tests {
Xbox360
);
}
// The conflict gate keys on the exact Valve PID, so a physical Steam Controller 2 (`28DE:1302`)
// never blocks a virtual Steam Deck (`28DE:1205`) — only a same-identity duplicate degrades.
#[cfg(target_os = "linux")]
#[test]
fn steam_backend_product_ids() {
use super::steam_backend_product;
use GamepadPref::*;
assert_eq!(steam_backend_product(SteamDeck), Some(0x1205));
assert_eq!(steam_backend_product(SteamController), Some(0x1102));
assert_eq!(steam_backend_product(SteamController2), Some(0x1302));
assert_eq!(steam_backend_product(SteamController2Puck), Some(0x1304));
// Non-Steam backends carry no Valve identity, so the gate never fires for them.
assert_eq!(steam_backend_product(DualSense), None);
assert_eq!(steam_backend_product(Xbox360), None);
assert_eq!(steam_backend_product(SwitchPro), None);
}
}
@@ -494,6 +494,15 @@ pub(super) async fn negotiate(
punktfunk_core::quic::HOST_CAP_CURSOR
} else {
0
}
// Full-fidelity stylus (0xCC/0x05 pen batches → the per-session uinput tablet):
// Linux with /dev/uinput access, minus the PUNKTFUNK_PEN=0 kill-switch. Clients
// without the bit keep folding pen into touch/pointer (and NativeClient::send_pen
// refuses toward us if we don't set it).
| if crate::inject::pen_supported() {
punktfunk_core::quic::HOST_CAP_PEN
} else {
0
},
// The negotiated session AEAD (resolved above) + its 32-byte key toward a ChaCha
// client; toward everyone else cipher 0 keeps the Welcome byte-identical to the
+100 -1
View File
@@ -523,6 +523,87 @@ pub(super) enum ClientInput {
Event(InputEvent),
/// The 0xCC plane: touchpad contacts + motion samples.
Rich(punktfunk_core::quic::RichInput),
/// The 0xCC/0x05 stylus plane: state-full pen sample batches, diffed into a per-session
/// virtual tablet (design/pen-tablet-input.md).
Pen(punktfunk_core::quic::PenBatch),
}
/// The per-session stylus lane: the core [`PenTracker`](punktfunk_core::quic::PenTracker)
/// diffs state-full batches into transitions, applied to a lazily-created uinput tablet
/// ([`crate::inject::pen::VirtualPen`]) — a session that never draws never creates a device,
/// and the device dies with the session (kernel removes the tablet, apps see the pen unplug).
struct PenSession {
tracker: punktfunk_core::quic::PenTracker,
dev: Option<crate::inject::pen::VirtualPen>,
/// Creation failed once — don't retry per batch (240 Hz of ioctl churn + log spam); the
/// tracker still consumes batches so its state stays coherent.
create_failed: bool,
last_rx: std::time::Instant,
/// Reused transition buffer (a batch yields at most a few).
out: Vec<punktfunk_core::quic::PenTransition>,
}
impl PenSession {
fn new() -> PenSession {
PenSession {
tracker: punktfunk_core::quic::PenTracker::default(),
dev: None,
create_failed: false,
last_rx: std::time::Instant::now(),
out: Vec::new(),
}
}
fn apply(&mut self, batch: &punktfunk_core::quic::PenBatch) {
self.last_rx = std::time::Instant::now();
if self.dev.is_none() && !self.create_failed {
match crate::inject::pen::VirtualPen::create() {
Ok(d) => self.dev = Some(d),
Err(e) => {
// Shouldn't happen when the Welcome advertised HOST_CAP_PEN off the same
// uinput probe — but permissions can change between then and first ink.
self.create_failed = true;
tracing::warn!(
error = %format!("{e:#}"),
"pen: virtual tablet creation failed — dropping pen input this session"
);
}
}
}
self.out.clear();
self.tracker.apply(batch, &mut self.out);
if let Some(dev) = self.dev.as_mut() {
dev.apply_batch(&self.out);
}
}
/// The dead-client failsafe ([`PEN_TOUCH_TIMEOUT_MS`](punktfunk_core::quic::PEN_TOUCH_TIMEOUT_MS)):
/// clients repeat the last sample (≤100 ms) while the pen is in range — a stationary
/// touching pen included — so silence past the timeout means the client is gone, and the
/// host must not leave the stroke inked-down. Called every input-loop wake; the loop caps
/// its recv timeout at 100 ms while the pen is active so this actually gets to run.
fn check_timeout(&mut self) {
if self.tracker.is_active()
&& self.last_rx.elapsed().as_millis()
>= punktfunk_core::quic::PEN_TOUCH_TIMEOUT_MS as u128
{
tracing::debug!("pen: sample stream went silent — force-releasing the stroke");
self.release_all();
}
}
/// Lift buttons/tip/proximity in order (a no-op when idle). Session end + the timeout.
fn release_all(&mut self) {
self.out.clear();
self.tracker.force_release(&mut self.out);
if let Some(dev) = self.dev.as_mut() {
dev.apply_batch(&self.out);
}
}
fn active(&self) -> bool {
self.tracker.is_active()
}
}
/// Default TTL stamped on a non-zero rumble envelope (0xCA v2): how long the client renders the
@@ -640,8 +721,17 @@ pub(super) fn input_thread(
const MAX_HELD: usize = 256;
let mut held_buttons: std::collections::HashSet<u32> = std::collections::HashSet::new();
let mut held_keys: std::collections::HashSet<u32> = std::collections::HashSet::new();
let mut pen = PenSession::new();
loop {
match rx.recv_timeout(pads.feedback_poll_interval()) {
// While a pen is in range/touching, wake at least every 100 ms so the stroke failsafe
// (PenSession::check_timeout) has real granularity against its 200 ms deadline.
let poll = if pen.active() {
pads.feedback_poll_interval()
.min(std::time::Duration::from_millis(100))
} else {
pads.feedback_poll_interval()
};
match rx.recv_timeout(poll) {
// Rich input (touchpad / motion) is applied the moment it arrives; the single channel
// wakes for gyro samples instead of making them wait out the feedback poll interval.
Ok(ClientInput::Rich(rich)) => {
@@ -673,6 +763,9 @@ pub(super) fn input_thread(
}
pads.apply_rich(rich);
}
// Stylus batches apply on arrival like rich input — the tracker synthesizes the
// transitions, the lazily-created virtual tablet renders them.
Ok(ClientInput::Pen(batch)) => pen.apply(&batch),
Ok(ClientInput::Event(ev)) => match ev.kind {
InputKind::GamepadButton | InputKind::GamepadAxis => {
// A bad index / unknown axis just doesn't update a pad — fall through (no
@@ -774,6 +867,9 @@ pub(super) fn input_thread(
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {}
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => break,
}
// Pen stroke failsafe: a silent sample stream past the deadline force-releases (see
// PenSession::check_timeout — clients heartbeat while the pen is down).
pen.check_timeout();
// Service feedback every iteration (≤1 ms for Triton, ≤4 ms otherwise; games block on
// EVIOCSFF, and HID handshakes must be answered promptly). Rumble → the universal 0xCA
// plane; rich/raw HID feedback → 0xCD.
@@ -863,6 +959,9 @@ pub(super) fn input_thread(
}
}
}
// Pen: lift anything still inked (buttons → tip → proximity), then the device dies with
// this thread (VirtualPen::drop → UI_DEV_DESTROY; apps see the tablet unplug).
pen.release_all();
// Session ended (client gone). Release anything still held through the host-lifetime injector —
// its EIS connection (and any implicit grab Mutter holds for our pressed button) outlives this
// session, so without this a button pressed at disconnect stays latched and breaks clicks for
+9 -9
View File
@@ -85,11 +85,10 @@ cp /usr/share/punktfunk/host.env.bazzite ~/.config/punktfunk/host.env
The template is deliberately minimal — it does **not** force a compositor, because the host
auto-detects Gaming Mode (gamescope) vs Desktop (KWin) on every connect and follows the switch
mid-stream. The only settings that matter are the session anchors (GPU zero-copy is on by default):
mid-stream. No session anchors are needed either (a user service inherits the right runtime dir).
The only settings that matter (GPU zero-copy is on by default):
```sh
XDG_RUNTIME_DIR=/run/user/1000
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
PUNKTFUNK_VIDEO_SOURCE=virtual
# GPU zero-copy (dmabuf → CUDA → NVENC) is ON by default; auto-falls back to CPU. Set =0 to force CPU.
PUNKTFUNK_GAMESCOPE_ATTACH=1 # Gaming Mode = attach to the box's own session (see below)
@@ -99,12 +98,13 @@ PUNKTFUNK_GAMESCOPE_ATTACH=1 # Gaming Mode = attach to the box's own session
For Gaming Mode there are two models (pick one; the shipped default is **attach**):
- **Attach** (`PUNKTFUNK_GAMESCOPE_ATTACH=1`, the default) — the **box** owns its gamescope session,
the host attaches to whatever's live and never tears it down, and the streamed game-mode resolution
is the box's own gamescope mode. Switching Desktop ↔ Game is rock-solid.
- **Managed** (`PUNKTFUNK_GAMESCOPE_MANAGED=1`, and remove the attach line) — the host launches its
**own** gamescope at the *client's* exact resolution and refresh. Client-mode-following, but there
must be no physical gaming session already running.
- **Attach** (`PUNKTFUNK_GAMESCOPE_ATTACH=1`, the template's default) — the **box** owns its
gamescope session on its own display, and the host attaches to whatever's live without ever
tearing it down (a box-owned autologin session is restarted at the client's resolution on a
mismatch). Switching Desktop ↔ Game is rock-solid.
- **Managed** (`PUNKTFUNK_GAMESCOPE_MANAGED=1`, and remove the attach line) — the host takes the
box's gamescope over and relaunches it **headless** at the *client's* exact resolution and
refresh — Game Mode on the virtual screen — restoring the box on idle.
Full treatment: [Steam / gamescope → Attach vs managed](/docs/gamescope#attach-vs-managed).
+15 -15
View File
@@ -4,7 +4,8 @@ description: Every host.env setting and PUNKTFUNK_* environment variable — com
---
The host reads its settings from **`~/.config/punktfunk/host.env`** (a simple `KEY=value` file, `#`
starts a comment). On Windows the service reads **`%ProgramData%\punktfunk\host.env`** instead. Your
starts a comment; keys are **case-sensitive**`punktfunk_compositor` sets nothing, use the exact
uppercase names). On Windows the service reads **`%ProgramData%\punktfunk\host.env`** instead. Your
[setup guide](/docs/requirements) gives you a starting `host.env` for your desktop; this page is the
full reference for every setting.
@@ -16,25 +17,24 @@ full reference for every setting.
## Session anchors
These tell the host which desktop session to attach to. Your setup guide sets them for you; they're
required when the host runs outside your interactive session (e.g. as a service).
**Leave these unset on a normal setup.** Running as a `systemctl --user` service the host inherits
the correct `XDG_RUNTIME_DIR` from systemd, derives the session bus from it, and **rewrites
`WAYLAND_DISPLAY` / `XDG_CURRENT_DESKTOP` / `XDG_RUNTIME_DIR` / `DBUS_SESSION_BUS_ADDRESS` on every
connect** to follow the active session (Gaming ↔ Desktop) — a value written here can only be
redundant or stale.
| Setting | What it does |
| Setting | When to set it |
|---|---|
| `XDG_RUNTIME_DIR` | Your session's runtime dir (e.g. `/run/user/1000`). Always needed for a service. |
| `DBUS_SESSION_BUS_ADDRESS` | Your session bus (e.g. `unix:path=/run/user/1000/bus`). Always needed for a service. |
| `WAYLAND_DISPLAY` | The Wayland socket of your session (`wayland-0` for a normal desktop, `wayland-kde` for the headless-KDE unit). |
| `XDG_CURRENT_DESKTOP` | Your desktop (`GNOME`, `KDE`). |
On Linux the host **rewrites `WAYLAND_DISPLAY` / `XDG_CURRENT_DESKTOP` / `XDG_RUNTIME_DIR` /
`DBUS_SESSION_BUS_ADDRESS` on every connect** to follow the active session (Gaming ↔ Desktop). Only
`XDG_RUNTIME_DIR` and `DBUS_SESSION_BUS_ADDRESS` need to be pinned as trustworthy anchors.
| `XDG_RUNTIME_DIR` | Only when the host runs **outside** a user service (ssh, cron): `/run/user/<your uid>` — check `id -u`. A copy-pasted `1000` on a box where that isn't your uid points the host at another user's (nonexistent) PipeWire/D-Bus, and **everything** fails (audio `Creation failed`, no capture, clients report the host unreachable). |
| `DBUS_SESSION_BUS_ADDRESS` | Same cases only: `unix:path=/run/user/<your uid>/bus`. Otherwise derived automatically. |
| `WAYLAND_DISPLAY` | Only the dedicated [headless-KDE appliance](/docs/kde#headless-session) (`wayland-kde`, set by its shipped `host.env.kde`). |
| `XDG_CURRENT_DESKTOP` | Same — appliance-only. |
## Core
| Setting | Values | Meaning |
|---|---|---|
| `PUNKTFUNK_COMPOSITOR` | `kwin` · `mutter` · `gamescope` · `wlroots` · `hyprland` (aliases: `kde`/`plasma`, `gnome`, `sway`/`wlr`) | Which backend creates the virtual display. `wlroots` is sway/River; `hyprland` is its own backend. **Leave unset to auto-detect;** set only to force one. |
| `PUNKTFUNK_COMPOSITOR` | `kwin` · `mutter` · `gamescope` · `wlroots` · `hyprland` (aliases: `kde`/`plasma`, `gnome`, `sway`/`wlr`) | Which backend creates the virtual display. `wlroots` is sway/River; `hyprland` is its own backend. **Leave unset.** Setting it **pins** the backend and turns session-following **off** — per connect *and* mid-stream, so a Desktop ↔ Gaming switch kills the stream instead of being followed. For CI/tests and dedicated single-session appliances only. |
| `PUNKTFUNK_VIDEO_SOURCE` | `virtual` · `portal` | `virtual` creates a per-client display at the client's exact mode (the normal choice). `portal` captures an existing monitor instead. |
| `PUNKTFUNK_ZEROCOPY` | `1` · `0` *(default on)* | GPU zero-copy capture→encode (dmabuf → CUDA → NVENC, or D3D11 on Windows). **On by default** — no need to set it; it falls back to a CPU path automatically. Set `0` to force the CPU path. One exception: Windows **Intel/QSV** keeps the CPU path by default until zero-copy is validated on Intel hardware — set `1` to try it there. |
| `PUNKTFUNK_INPUT_BACKEND` | `libei` · `gamescope` · `wlr` · `uinput` | How input is injected. `libei` for GNOME/KDE, `gamescope` for Bazzite/gamescope, `wlr` for Sway/wlroots **and Hyprland**. Auto-detected with the compositor. |
@@ -53,8 +53,8 @@ the full picture (and [Bazzite](/docs/bazzite) for that distro's specifics).
| Setting | Values | Meaning |
|---|---|---|
| `PUNKTFUNK_GAMESCOPE_ATTACH` | `1` | **Attach** model: the box owns its gamescope session (you switch Gaming ↔ Desktop with the Steam UI); the host just captures whatever's live and never tears it down. Rock-solid; streamed resolution is the box's gamescope mode. |
| `PUNKTFUNK_GAMESCOPE_MANAGED` | `1` | **Managed** model: the host tears the box's gamescope down on connect and launches its **own** at the *client's* exact resolution, restoring on idle. Client-mode-following, but doesn't coexist with a box-owned game-mode session. |
| `PUNKTFUNK_GAMESCOPE_ATTACH` | `1` | **Attach** model: the box owns its gamescope session on its own display (you switch Gaming ↔ Desktop with the Steam UI); the host just captures whatever's live and never tears it down. A box-owned autologin session is restarted at the client's resolution on a mismatch; a foreign/bare gamescope streams at its own mode. |
| `PUNKTFUNK_GAMESCOPE_MANAGED` | `1` | **Managed** model (the default where session infra is detected): the host takes the box's gamescope over and relaunches it **headless** at the *client's* exact resolution — Game Mode on the virtual screen — restoring the box on idle. |
| `PUNKTFUNK_GAMESCOPE_SESSION` | `steam` | The host owns a `gamescope-session-plus` (Steam) session at the client's mode (headless appliance; no physical session running). |
| `PUNKTFUNK_GAMESCOPE_NODE` | `auto` · node id | Discover + capture a **running** gamescope's PipeWire node at a fixed mode. Do **not** combine with `SESSION`. |
| `PUNKTFUNK_GAMESCOPE_APP` | command | For an ad-hoc bare-gamescope session, the nested command to run (e.g. `vkcube`). |
+17 -15
View File
@@ -17,18 +17,20 @@ from the install guide for your OS: [Bazzite](/docs/bazzite) or [SteamOS (Host)]
## Attach vs managed
There are two mutually-exclusive models for a gamescope box; pick one. The shipped default is
**attach**.
There are two mutually-exclusive models for a gamescope box; pick one. With **nothing set**, a box
that has gamescope session infrastructure (Bazzite, SteamOS, Nobara) gets **managed**; the
[Bazzite template](/docs/bazzite) ships with **attach** chosen instead.
- **Attach** (`PUNKTFUNK_GAMESCOPE_ATTACH=1`, the default) — the **box** owns its gamescope session
and decides Gaming vs Desktop via the normal Steam UI. The host just attaches to whatever's live
and never tears it down, so switching Desktop ↔ Game is rock-solid and disconnecting leaves the box
where it was. The streamed game-mode resolution is the box's gamescope mode
(`SCREEN_WIDTH/HEIGHT` in `/etc/gamescope-session-plus/sessions.d/steam`), not the client's.
- **Managed** (`PUNKTFUNK_GAMESCOPE_MANAGED=1`, and remove the attach line) — the host tears the
box's gamescope down on connect and launches its **own** at the *client's* exact resolution and
refresh, restoring on idle. Client-mode-following, but it can't coexist with a box-owned game-mode
session, and there must be **no physical gaming session already running**.
- **Attach** (`PUNKTFUNK_GAMESCOPE_ATTACH=1`) — the **box** owns its gamescope session and decides
Gaming vs Desktop via the normal Steam UI. Game Mode stays on the box's own (physical) display;
the host attaches to whatever's live and never tears it down, so switching Desktop ↔ Game is
rock-solid and disconnecting leaves the box where it was. When the session is the box's own
autologin unit, the host restarts it at the **client's** resolution on a mismatch; a foreign or
bare gamescope is streamed at its own mode.
- **Managed** (the infra-detected default; force with `PUNKTFUNK_GAMESCOPE_MANAGED=1`) — the host
takes the box's gamescope session over and relaunches it **headless** at the *client's* exact
resolution and refresh — Game Mode runs on the virtual screen, physical displays drop out of it —
restoring the box on idle after disconnect.
## Session following
@@ -40,8 +42,8 @@ over its own compositor, and re-targets whichever is live on each switch.
## Start the host
On an appliance box (Bazzite, SteamOS) the install guide already enables the host service for you. On
any other distro running a gamescope session, start it from your session — the default attach model
just latches onto whatever gamescope session is live:
any other distro running a gamescope session, just start it — the host auto-detects the live
gamescope session and picks the model for it:
```sh
systemctl --user enable --now punktfunk-host
@@ -56,8 +58,8 @@ a model. See the full [Configuration reference](/docs/configuration) for every o
| Setting | Values | Meaning |
|---|---|---|
| `PUNKTFUNK_GAMESCOPE_ATTACH` | `1` | **Attach** model: the box owns its gamescope session; the host captures whatever's live and never tears it down. Streamed resolution is the box's gamescope mode. The default. |
| `PUNKTFUNK_GAMESCOPE_MANAGED` | `1` | **Managed** model: the host tears the box's gamescope down on connect and launches its own at the client's exact mode, restoring on idle. Doesn't coexist with a box-owned game-mode session. |
| `PUNKTFUNK_GAMESCOPE_ATTACH` | `1` | **Attach** model: the box owns its gamescope session (on its own display); the host captures whatever's live and never tears it down. A box-owned autologin session is restarted at the client's resolution on a mismatch; a foreign/bare gamescope streams at its own mode. |
| `PUNKTFUNK_GAMESCOPE_MANAGED` | `1` | **Managed** model (the default where session infra is detected): the host takes the box's gamescope over and relaunches it headless at the client's exact mode, restoring on idle. |
| `PUNKTFUNK_GAMESCOPE_SESSION` | `steam` | The host owns a `gamescope-session-plus` (Steam) session at the client's mode — a headless appliance with no physical session running. |
| `PUNKTFUNK_GAMESCOPE_NODE` | `auto` · node id | Discover and capture a **running** gamescope's PipeWire node at a fixed mode. Do **not** combine with `SESSION`. |
| `PUNKTFUNK_GAMESCOPE_APP` | command | For an ad-hoc bare-gamescope session, the nested command to run (e.g. `vkcube`). |
+8 -7
View File
@@ -12,19 +12,20 @@ installed — see [Ubuntu](/docs/ubuntu), [Fedora](/docs/fedora), or [Arch](/doc
## host.env
Write `~/.config/punktfunk/host.env` with the GNOME settings. The host auto-detects the compositor
from your session, so the explicit `PUNKTFUNK_COMPOSITOR` is belt-and-braces:
The host auto-detects the compositor from your live session on every connect, so the starter
`~/.config/punktfunk/host.env` is one line:
```ini
# ~/.config/punktfunk/host.env
WAYLAND_DISPLAY=wayland-0
XDG_CURRENT_DESKTOP=GNOME
PUNKTFUNK_COMPOSITOR=mutter
# ~/.config/punktfunk/host.env (keys are case-sensitive)
PUNKTFUNK_VIDEO_SOURCE=virtual
# GPU zero-copy (dmabuf → CUDA → NVENC) is ON by default; auto-falls back to CPU. Set =0 to force CPU.
PUNKTFUNK_INPUT_BACKEND=libei
```
> **Don't set `PUNKTFUNK_COMPOSITOR`, `WAYLAND_DISPLAY`, or `XDG_CURRENT_DESKTOP` here.** Pinning
> the compositor turns auto-detection **off** — per connect *and* mid-stream — so the host stops
> following session switches, and stale session values point it at dead sockets. Forcing a backend
> is a CI / dedicated-appliance posture, not desktop configuration.
You must be on a **Wayland** session (not X11), and Mutter must be **≥ 48**. See the
[Configuration reference](/docs/configuration) for every option.
+9 -4
View File
@@ -19,14 +19,19 @@ or [Fedora](/docs/fedora).
## host.env
The host auto-detects a Hyprland session, so you usually need nothing here. To force the backend, set
these in `~/.config/punktfunk/host.env`:
The host auto-detects a Hyprland session, so the starter `~/.config/punktfunk/host.env` is one line:
```ini
PUNKTFUNK_VIDEO_SOURCE=virtual
# GPU zero-copy capture→encode is ON by default; auto-falls back to CPU. Set PUNKTFUNK_ZEROCOPY=0 to force CPU.
```
To force the backend (CI/testing — note that pinning turns live-session auto-detection **off**, so
the host stops following session switches):
```ini
PUNKTFUNK_COMPOSITOR=hyprland
PUNKTFUNK_INPUT_BACKEND=wlr
PUNKTFUNK_VIDEO_SOURCE=virtual
# GPU zero-copy capture→encode is ON by default; auto-falls back to CPU. Set PUNKTFUNK_ZEROCOPY=0 to force CPU.
```
See [Configuration](/docs/configuration) for the full reference.
+18 -8
View File
@@ -13,20 +13,30 @@ installed — see [Ubuntu](/docs/ubuntu), [Fedora](/docs/fedora), [Arch](/docs/a
## host.env
A KDE starter `~/.config/punktfunk/host.env`:
The host auto-detects your KWin session on every connect — including a box that switches between
the Plasma desktop and Steam Game Mode — so the starter `~/.config/punktfunk/host.env` is one line:
```ini
WAYLAND_DISPLAY=wayland-0
XDG_CURRENT_DESKTOP=KDE
PUNKTFUNK_COMPOSITOR=kwin
# ~/.config/punktfunk/host.env (keys are case-sensitive)
PUNKTFUNK_VIDEO_SOURCE=virtual
# GPU zero-copy (dmabuf → CUDA → NVENC) is ON by default; auto-falls back to CPU. Set =0 to force CPU.
PUNKTFUNK_INPUT_BACKEND=libei
```
The host auto-detects the running compositor on every connect, so most of this is optional — the
values above are just what it resolves to on a KWin session. See the
[Configuration reference](/docs/configuration) for every option.
> **Don't set `PUNKTFUNK_COMPOSITOR`, `WAYLAND_DISPLAY`, or `XDG_CURRENT_DESKTOP` here.** Pinning
> the compositor turns auto-detection **off** — per connect *and* mid-stream — so a switch to Game
> Mode then kills the stream instead of being followed, and stale session values point the host at
> dead sockets. Forcing a backend is for CI and dedicated appliances (the
> [headless session](#headless-session) below ships a `host.env.kde` that pins on purpose).
If the box switches between the desktop and Game Mode, also enable lingering — the host is a user
service, and without linger the logout moment of a session switch tears it (and PipeWire) down
mid-stream:
```sh
sudo loginctl enable-linger "$USER"
```
See the [Configuration reference](/docs/configuration) for every option.
## Use a Wayland session
+9 -4
View File
@@ -23,16 +23,21 @@ or [Fedora](/docs/fedora).
## host.env
The host auto-detects a wlroots session, so you usually need nothing here. To force the backend, set
these in `~/.config/punktfunk/host.env`:
The host auto-detects a wlroots session, so the starter `~/.config/punktfunk/host.env` is one line:
```ini
PUNKTFUNK_COMPOSITOR=wlroots # aliases: sway, wlr, hyprland (all the wlroots family; the exact backend is auto-detected)
PUNKTFUNK_INPUT_BACKEND=wlr
PUNKTFUNK_VIDEO_SOURCE=virtual
# GPU zero-copy capture→encode is ON by default; auto-falls back to CPU. Set PUNKTFUNK_ZEROCOPY=0 to force CPU.
```
To force the backend (CI/testing — note that pinning turns live-session auto-detection **off**, so
the host stops following session switches):
```ini
PUNKTFUNK_COMPOSITOR=wlroots # aliases: sway, wlr (the wlroots-proper family)
PUNKTFUNK_INPUT_BACKEND=wlr
```
See [Configuration](/docs/configuration) for the full reference.
## How it works
+16 -1
View File
@@ -103,7 +103,22 @@ See [GNOME](/docs/gnome) for the GL/EGL userspace details.
- KWin must be **≥ 6.5.6** (`kwin_wayland --version`); GNOME **≥ 48**; gamescope **≥ 3.16.22**. See
[KDE](/docs/kde) for the KWin/Wayland requirement and [gamescope](/docs/gamescope) for the
gamescope one.
- Confirm `PUNKTFUNK_COMPOSITOR` in [`host.env`](/docs/configuration) matches your desktop.
- If [`host.env`](/docs/configuration) sets `PUNKTFUNK_COMPOSITOR`, **remove it** — the host
auto-detects the live compositor, and the pin points it at one backend even when a different
session is live (it also disables Gaming ↔ Desktop following).
## Session fails right after editing host.env
- Keys are **case-sensitive**: `punktfunk_gamescope_attach=1` sets nothing — use the exact
uppercase names.
- Hardcoded session anchors with the wrong uid (`XDG_RUNTIME_DIR=/run/user/1000` when `id -u`
isn't 1000) point the host at another user's PipeWire/D-Bus: audio errors like
`pw audio connect … Creation failed`, no capture, and clients reporting the host as
unreachable or asleep. **Delete both anchor lines** — a `systemctl --user` service doesn't need
them — or fix the uid.
- `PUNKTFUNK_COMPOSITOR` pins the backend and disables Gaming ↔ Desktop following — remove it on
any box that switches sessions.
- The env file is read at service start: `systemctl --user restart punktfunk-host` after edits.
## Capture fails: "Session creation inhibited" (GNOME)
+36
View File
@@ -437,6 +437,42 @@ punktfunk_connection_send_input(c, &ev);
- `punktfunk_connection_send_rich_input2(c, &richEx)` — the forward-compatible superset (Steam
trackpads, signed coords, pressure); set `struct_size = sizeof(PunktfunkRichInputEx)`.
### Stylus / pen
Full-fidelity pen (pressure, tilt, azimuth, barrel roll, hover, eraser, barrel buttons) has its
own plane. **Gate on the capability first** — only send when
`punktfunk_connection_host_caps(c, &caps)` shows `PUNKTFUNK_HOST_CAP_PEN`; without it the call
returns `UNSUPPORTED` and you keep your pen-as-touch fallback (what every client does today).
Samples are **state-full**: every sample carries the complete pen state (in-range / touching /
buttons in `state`, plus all axes — unknown axes take their `PUNKTFUNK_PEN_*_UNKNOWN` sentinel,
never 0). There are no down/up events to send; the host diffs consecutive samples and
synthesizes the transitions, so a lost datagram self-heals on the next one. Batch a capture
callback's coalesced samples (oldest first, `dt_us` = spacing) into one call, at most
`PUNKTFUNK_PEN_BATCH_MAX` per call — split longer runs into consecutive calls.
```c
// Example: an in-contact Apple Pencil sample, mapped into video-frame space
PunktfunkPenSample s; memset(&s, 0, sizeof s);
s.state = PUNKTFUNK_PEN_IN_RANGE | PUNKTFUNK_PEN_TOUCHING;
s.tool = PUNKTFUNK_PEN_TOOL_PEN;
s.x = 0.5f; s.y = 0.5f; // normalized 0..1 across the video frame
s.pressure = (uint16_t)(force_norm * 65535); // 0 while hovering
s.tilt_deg = 90 - altitude_deg; // polar tilt-from-normal
s.azimuth_deg = azimuth_deg; // 0..359, clockwise from north
s.roll_deg = PUNKTFUNK_PEN_ANGLE_UNKNOWN; // Pencil Pro: rollAngle in degrees
s.distance = PUNKTFUNK_PEN_DISTANCE_UNKNOWN;
punktfunk_connection_send_pen(c, &s, 1);
```
While hovering, send `PUNKTFUNK_PEN_IN_RANGE` samples (with `distance` if you have it); when the
pen leaves range, send one final sample with `state = 0` so the host releases proximity.
**Heartbeat**: while the pen is in range or touching, repeat the last sample at least every
~100 ms even when nothing changed — pen capture APIs are silent for a stationary pen, and the
host force-releases the stroke after 200 ms without samples (its dead-client failsafe). Run a
timer alongside your touch callbacks.
---
## 9. Feedback planes (rumble, HID, HDR)
+58
View File
@@ -0,0 +1,58 @@
# Release notes
One file per **stable** release: `docs/releases/vX.Y.Z.md`. Its contents become the Gitea release
body **verbatim** and are the source of the Discord `#releases` announcement.
## Why this exists
Releases used to be created by CI with an **empty body**; the notes were pasted in by hand
afterward. That left a window where the release — and anything announcing it — carried no notes.
Now the notes are authored **before the tag is pushed**, as part of the version bump, so the
release is born complete and the announcement always has something to say.
## The flow
1. **Write the notes.** Add `docs/releases/vX.Y.Z.md` in the same commit (or PR) as the version
bump. Copy `TEMPLATE.md` and fill it in. This file is the single source of truth for the body.
2. **Tag & push.** `git tag -a vX.Y.Z … && git push origin vX.Y.Z` fans out to the build
workflows. Whichever one wins the create race seeds the release body from this file
(`scripts/ci/gitea-release.sh``ensure_release`, and its PowerShell twin). The release page
shows the notes immediately.
3. **Wait for green.** Let every platform's CI finish and go green.
4. **Announce.** Dispatch the `announce` workflow (`.gitea/workflows/announce.yml`) with the tag.
It re-asserts this file over the live release (so any late edit wins) and posts an embed to
Discord `#releases`. Pressing "go" is the quality gate — a half-built release is never
announced. Stable-only; a `-rc` tag is refused unless `allow_prerelease=true`.
Editing the notes after the tag is fine: update this file, then re-run step 4 (or PATCH the body
via the API) — the announce step always re-syncs from the file, so the file stays authoritative
even across a tag re-point.
Canary / `-rc` builds have **no** file here on purpose: they get no curated body and are not
announced.
## Voice & format
**Write for the people who USE Punktfunk to stream their games and desktops — not for the people who
build it.** A non-engineer should finish knowing what's new and whether it affects them; an engineer
should never be confused or forced to decode internals. (See any recent `vX.Y.Z.md` for the target.)
1. **Lead with the benefit.** Each entry = what the user can now *do*, what now *works*, or what
stopped *going wrong* — in their words. Implementation is not the story.
2. **No internal vocabulary in the body.** No protocol/message names, code type names, hex codes or
hardware IDs, crate/component names, or API symbols. Translate any essential detail to plain
language. Name things users recognize (iPad, Apple Pencil, Steam Deck, Android TV, the Windows
sign-in screen) — not subsystems.
3. **Group as New / Improved / Fixed**, each a bold one-line lead-in + a tight plain explanation.
Skimmable. The lead-in text before the first `##` is what the Discord announcement shows, so make
it a real, plain-language summary.
4. **Be specific and honest** — no vague "various improvements"; a reader should know exactly what
changed.
5. **Compatibility line up top, in plain terms:** can they update one side at a time? does their
existing setup keep working? No version numbers in the lead.
6. **All protocol / ABI / driver / embedder detail goes in ONE `## Under the hood (for developers)`
section at the very bottom** — the only place internal names and version numbers belong, clearly
optional. The old dense engineering style survives only there.
The short annotated-**tag** message stays separate and short (a headline + a paragraph); it is the
tag object's message, not this file.
+20
View File
@@ -0,0 +1,20 @@
Wire-compatible with X.Y.x — existing pairings and clients keep working. <one sentence on how
older/newer clients negotiate or fall back, so nobody fears updating.>
<Optional: one or two sentences naming the headline change of this release — this whole lead-in
(everything above the first `##`) is what the Discord #releases embed shows, so make it read as a
standalone summary.>
## Highlights
- **<Lead-in>.** <What changed and why it matters, concretely.>
- **<Lead-in>.** <…>
## Fixes
- **<Lead-in>.** <…>
## Platform notes
- **<Platform>.** <Anything platform-specific: env vars, ABI/protocol versions, hardware that was
verified on-glass.>
+33
View File
@@ -0,0 +1,33 @@
Update whenever it suits you — you can update your app and the machine you stream from one at a time, the new and old versions work together, and everything you've already paired stays paired. The new pen features simply switch on once both ends are updated. Nothing you already use changes.
## New: draw and write with a pen or stylus
You can now use a real stylus while streaming, and it behaves like a real pen on the machine you're controlling — **pressure, tilt, and the pen's side buttons all come through**, not just a plain tap. Use your **iPad's Apple Pencil** (including Pencil Pro), an **Android phone or tablet's S Pen** or other active stylus, or a pen from a Moonlight app.
It's great for drawing apps, handwriting and note-taking, signing documents, and photo editing over the stream. Windows and Linux hosts receive it as genuine pen input. If the machine you're streaming from is still on an older version, your pen keeps working as an ordinary touch.
## Fixed: your Steam Deck controller shows up as the right controller
If a Steam controller was plugged into the computer you stream from, a streamed **Steam Deck** controller could be misread by games as a **PlayStation (DualSense)** controller — so button prompts and layouts came out wrong. It now stays a Steam Deck. Punktfunk only steps in to avoid a clash when two genuinely identical controllers are present.
## Improved: a sharper, correctly-sized mouse pointer
- **Right-sized pointer on high-resolution screens.** On Linux and Mac, a pointer coming from a high-resolution host could show up about twice as big as it should. It now matches the video exactly.
- **Cleaner pointer on Windows security screens.** The mouse pointer no longer duplicates or gets stuck on Windows sign-in and User Account Control (admin permission) prompts.
- **More reliable multi-monitor streaming on Windows.** Switching between monitor layouts on the host could occasionally drop the stream — that's fixed.
## Fixed: touch input
- **Multi-touch from Moonlight apps now works on Windows hosts.** It previously registered nothing; pinch, zoom, and multi-finger gestures now come through.
- **Touch is no longer silently dropped** from some devices that report finger pressure in an unusual way.
## Fixed: Android (and Android TV)
- **Your mouse's back/forward buttons stay in the stream** instead of bouncing you out to the Android home screen — a relief on Android TV in particular.
- **The on-screen keyboard stays out of the way** when you're typing on a physical keyboard.
## Under the hood (for developers)
- The streaming protocol is unchanged, so 0.18 and 0.19 hosts and apps mix freely; the pen is negotiated and older peers simply fall back to touch.
- The embeddable core library adds one new call for sending pen input (the C ABI moves to **13**) — rebuild any embedders against 0.19. The Windows virtual-display driver is unchanged: pen goes through Windows' normal pen system, not the display driver.
- iOS builds are now attached to each release as a downloadable file (for TestFlight/archival).
+60
View File
@@ -0,0 +1,60 @@
A maintenance update — nothing changes in how Punktfunk works, and you can update your app and the machine you stream from one at a time. Everything you've already paired stays paired.
This one is entirely about things that used to go wrong, and the big one is stuck sessions: a machine you streamed from could go on believing you were still connected long after you'd closed the app, walked out of Wi-Fi range or put the device to sleep — leaving it busy, refusing the next connection, and in the worst case unable to accept anyone at all until it was restarted. That's fixed on both sides. Also fixed: an admin prompt no longer locks you out of a Windows machine, your Apple device's screen no longer sleeps while you're playing with a controller, the mouse pointer behaves in Steam's Gaming Mode, and phone-shaped screens finally get the high refresh rate they asked for from a KDE Plasma machine.
## Fixed: sessions no longer outlive the device that started them
If a session ended in any way other than pressing Disconnect, the machine you were streaming from often never found out. Quitting the app, closing your laptop, losing Wi-Fi, a device running out of battery, or simply switching to another app on a phone or TV box — all of these could leave a session running forever, encoding video and audio to nobody.
That had real consequences, and all of them are fixed:
- **You can reconnect and actually get a picture.** A machine that still thought you were connected would accept your next connection and then show you nothing, because it believed it was already streaming to you.
- **Someone else can use the machine.** A stuck session made it look busy, so the next person to connect was refused — indefinitely.
- **A machine no longer stops accepting connections entirely.** A session that got wedged used to hold onto its slot permanently, and a handful of those would make the machine unreachable until it was restarted. Even the Stop button in the web console couldn't clear one. Stuck sessions are now cleaned up regardless, and the Stop button works.
- **Closing the app on your phone or TV box actually ends the session.** On Android and on iPhone, iPad and Apple TV, switching away from Punktfunk now ends the stream instead of quietly keeping it alive in the background. Coming straight back is still quick.
- **Quitting deliberately is heard.** Pressing Disconnect now reliably reaches the other machine, rather than looking like a device that vanished.
- **Streaming from Moonlight ends properly too.** Moonlight sessions had the same problem — quitting Moonlight left the machine streaming into the void. It now notices a client that has gone away, typically within half a minute.
- **A busy machine tells you it's busy.** If it's already at its limit, you now get a clear answer instead of a connection that hangs and eventually times out.
## Fixed: an admin prompt on Windows no longer locks you out
If a Windows machine was left showing a **User Account Control prompt** (the "do you want to allow this app to make changes?" dialog), or was sitting on the lock or sign-in screen, connecting to it broke in two ways. Both are fixed.
- **Connecting works again.** Every attempt used to show a black screen and then give up after about a minute, which meant an unattended machine had to be reached some other way just to click one button. A connection now comes up in about **three seconds** with the prompt still on screen.
- **Pen and touch reach the prompt.** Mouse and keyboard already worked, but pen and touch input did nothing — so you could see the prompt but not dismiss it from a tablet. You can now dismiss a Windows admin prompt with an **Apple Pencil from an iPad**, or with a finger.
## Fixed: your screen stays awake on iPhone, iPad, Apple TV and Mac
Playing with a controller, the screen would dim and sleep out from under you mid-session — because a game controller isn't something the operating system counts as "using the device". The display now stays awake for as long as the session lasts, and goes back to normal the moment you disconnect. On a Mac, the screen saver and the idle lock that follows it are held off for the session too. (The Android app has always done this; the Apple apps now match.)
## Fixed: the stuck pointer in Steam's Gaming Mode
On a machine streaming from Steam's Gaming Mode — a Steam Deck, or a desktop in the same session — a **fragment of the mouse pointer got welded to the bottom-right corner** of the stream and stayed there for the rest of the session, while the real pointer went undrawn. It showed up in every game, because a game takes over the pointer.
The stream now follows what the machine itself decides to draw: the pointer appears where it really is, disappears when the machine hides it, and picks up Gaming Mode's own "hide the pointer after a few seconds of stillness" behaviour, which the stream never had before.
## Fixed: high refresh rate on phone-shaped screens from KDE Plasma
Streaming from a **KDE Plasma** desktop to a screen with an unusual width — an **iPhone 16 Pro Max**, for instance — dropped to **60 Hz**, even though the 120 Hz option was sitting right there in the machine's own display settings. Punktfunk was asking for the resolution it wanted, while the desktop had quietly built a mode a few pixels narrower, so the two never matched.
Punktfunk now picks whichever mode the desktop actually created and streams at its real refresh rate. Screens with common widths (1080p, 1440p, 4K) were never affected.
Also fixed: connecting no longer leaves a **new leftover entry in your display settings every time**. Previously each connection appended one more custom resolution to the list.
## Improved: setup guides for Linux desktops
Following a field report, the Linux setup guides no longer tell you to lock Punktfunk to a specific desktop environment. That setting **stops the stream from following you** when you switch to Steam's Gaming Mode — the stream would end instead of coming with you. The example configuration also no longer hard-codes a user account ID, which broke audio for anyone who wasn't the first user created on the machine.
**If you copied a Punktfunk configuration from the docs before this release, it's worth re-checking it** — the current guides for KDE, GNOME, Hyprland and Sway are down to a single setting, and the troubleshooting page now has a section for sessions that break right after a config edit.
## Under the hood (for developers)
- **Nothing changed on the wire or at the API boundary.** The streaming protocol stays at version 2 and the embeddable core library stays at C ABI **13** — no embedder rebuild is required, and 0.18/0.19 hosts and clients still mix freely. The Windows virtual-display driver protocol is unchanged.
- **Native session teardown is now bounded and enforceable.** `stop` was advisory: every teardown (`conn.close`, the joins, and the RAII drops of the session permit, admission entry and stream marker) sat after the stream thread's await, and the encode loop only checks the flag *between* iterations — so one unbounded syscall inside an iteration became a permanent zombie holding its semaphore slot and admission entry. The thread now gets `STREAM_STOP_GRACE` (90 s, past the 40 s capture-rebuild budget) and teardown runs regardless; the thread is detached rather than killed, since Rust cannot cancel a blocking thread. Audio/input joins are bounded too. The session permit is taken *after* the QUIC handshake rather than before `accept()`, so a host at its cap still accepts and the client sees a live path instead of a silent dial timeout. New `pf-vdisplay` `proc::{status_within, output_within}` kill a child that outruns its budget — `kscreen-doctor` is a Wayland client of the very compositor it configures and never returns against a wedged KWin.
- **Client-side close actually reaches the wire.** `conn.close()` only queues the frame, and `run_pump` is the body of a `block_on` whose runtime is dropped the moment it returns, so a deliberate quit arrived as silence (8 s idle timeout, no quit code). The endpoint is now carried out of the handshake and flushed with `wait_idle()`. Android tears down on `ON_STOP` via the existing `onDispose` path; the Apple `.background` arm was iOS-only *and* gated on an opt-in defaulting off while the `audio` background mode kept the connection alive indefinitely — it now acts unconditionally and covers tvOS.
- **GameStream sessions end on ENet disconnect.** The only automatic detector was a media-UDP send error, which needs an ICMP port-unreachable — so a true vanish left both planes encoding forever, and a plain Moonlight quit (no RTSP `TEARDOWN`, no nvhttp `/cancel`) leaked the session. The control peer's reliable-ping timeout fires `Event::Disconnect` within ~530 s and now runs the shared, idempotent `AppState::end_session`. Peer tracking is gated on the `/launch` owner's source IP so an unauthenticated LAN peer cannot end a live session and a stale peer cannot kill its successor. Unreachable-client UDP errors end the whole session via an `OnSessionLost` callback built at PLAY, instead of stopping only the plane that noticed.
- **Windows display and pointer-injection calls now follow the input desktop.** `SetDisplayConfig`/`ChangeDisplaySettingsEx` and `InjectSyntheticPointerInput` are rejected with `ERROR_ACCESS_DENIED` when issued from a thread that isn't on the desktop currently receiving input — Winlogon, whenever UAC or the lock screen is up. Both paths retry once bound to the input desktop, with the binding scoped rather than persistent so a shared thread is never left parked on a desktop that disappears. The retry predicate stays narrow (`ERROR_ACCESS_DENIED` only) so the unrelated exclusive-mode `0x57` topology failure is not re-issued under a different name.
- **The X11 cursor source honours `GAMESCOPE_CURSOR_VISIBLE_FEEDBACK`.** gamescope hides the pointer by warping it to the root window's bottom-right pixel rather than swapping in a transparent cursor, so `XFixesGetCursorImage` kept returning the last opaque arrow and the "whichever pointer moved last" heuristic froze on the parked Xwayland. The atom is read at connect and re-read on its root `PropertyNotify`, with a 250 ms resync; the motion heuristic remains the fallback for a gamescope that publishes no verdict (logged as `cursor_feedback=false`).
- **KWin custom modes are resolved from the output's own mode list and selected by mode id.** libxcvt rounds the requested width down to a multiple of 8 (2868×1320@120 → 2864×1320@119.92), so the `WxH@Hz` name passed to kscreen-doctor's `findMode` matched nothing and the select silently no-op'd. `set_custom_refresh` now returns the achieved mode and it is reported as the output's preferred mode, so the renegotiation gate waits for the geometry KWin will really deliver. `addCustomMode` is skipped when a usable mode already exists, since kscreen-doctor appends and KWin persists per output name.
- **Apple display-sleep guard.** `UIApplication.isIdleTimerDisabled` on iOS/iPadOS/tvOS; on macOS `ProcessInfo.beginActivity(.idleDisplaySleepDisabled)` plus a 30 s `IOPMAssertionDeclareUserActivity` heartbeat, because the screen saver runs off the HID idle timer independently of display sleep. Acquired in `beginStreaming`, released at the top of `disconnect`, so every teardown path releases it.
- **The xcframework builds warning-free again.** Two redundant `super::super::*` globs in the client pump warned only on tvOS targets, which CI never builds (that needs `BUILD_TVOS=1` and the Tier-3 build-std nightly path).
+180 -4
View File
@@ -49,7 +49,10 @@
// (design/remote-desktop-sweep.md §8): the client's mouse-model chord tells the host who
// renders the pointer. Additive; rides the existing control stream (a new message TYPE, which
// pre-§8 hosts ignore), so [`WIRE_VERSION`] is unchanged.
#define ABI_VERSION 12
// v13: added `punktfunk_connection_send_pen` — the stylus wire plane
// (design/pen-tablet-input.md): a client sends `RICH_PEN` sample batches once the host
// advertises `HOST_CAP_PEN`. Additive and capability-gated, so [`WIRE_VERSION`] is unchanged.
#define ABI_VERSION 13
// The punktfunk/1 **wire** version — what `Hello`/`Welcome` carry and hosts equality-check.
// Deliberately its own constant: [`ABI_VERSION`] tracks the embeddable **C surface**
@@ -88,6 +91,37 @@
// `punktfunk_connection_send_rich_input2` (added with client capture).
#define PUNKTFUNK_RICH_TOUCHPAD_EX 3
// [`PunktfunkPenSample::state`] bit: the pen hovers in range (implied by `TOUCHING`).
#define PUNKTFUNK_PEN_IN_RANGE 1
// [`PunktfunkPenSample::state`] bit: the tip is in contact.
#define PUNKTFUNK_PEN_TOUCHING 2
// [`PunktfunkPenSample::state`] bit: primary barrel button (or squeeze mapping) held.
#define PUNKTFUNK_PEN_BARREL1 4
// [`PunktfunkPenSample::state`] bit: secondary barrel button (or double-tap mapping) held.
#define PUNKTFUNK_PEN_BARREL2 8
// [`PunktfunkPenSample::tool`]: the pen tip.
#define PUNKTFUNK_PEN_TOOL_PEN 0
// [`PunktfunkPenSample::tool`]: the eraser (a client-side mode — Apple Pencil has no
// hardware eraser end; the squeeze/double-tap mapping usually drives this).
#define PUNKTFUNK_PEN_TOOL_ERASER 1
// Most samples one [`punktfunk_connection_send_pen`] call accepts (one wire batch).
#define PUNKTFUNK_PEN_BATCH_MAX 8
// [`PunktfunkPenSample::tilt_deg`] sentinel: no tilt reading.
#define PUNKTFUNK_PEN_TILT_UNKNOWN 255
// [`PunktfunkPenSample::azimuth_deg`] / `roll_deg` sentinel: no reading.
#define PUNKTFUNK_PEN_ANGLE_UNKNOWN 65535
// [`PunktfunkPenSample::distance`] sentinel: no hover-distance reading.
#define PUNKTFUNK_PEN_DISTANCE_UNKNOWN 65535
// Compositor preference for [`punktfunk_connect_ex`] (`compositor` arg). `AUTO` lets the host
// pick (auto-detect from its running desktop); a concrete value is honored only if that backend
// is available on the host right now, else the host falls back to auto-detect. The resolved
@@ -219,6 +253,13 @@
// clipboard, so a client may offer the toggle. (Mirrors `quic::HOST_CAP_CLIPBOARD`.)
#define PUNKTFUNK_HOST_CAP_CLIPBOARD 2
// Host-capability bit in [`punktfunk_connection_host_caps`]: the host injects full-fidelity
// stylus input, so a capable client splits pen contacts out of its touch path and sends them
// via [`punktfunk_connection_send_pen`]; without the bit that call returns `Unsupported` and
// the client keeps its pen-as-touch fallback. (Mirrors `quic::HOST_CAP_PEN`;
// design/pen-tablet-input.md.)
#define PUNKTFUNK_HOST_CAP_PEN 16
// [`punktfunk_connect_ex9`] `client_caps` bit: render the host cursor locally (the cursor
// channel, `design/remote-desktop-sweep.md` M2).
#define PUNKTFUNK_CLIENT_CAP_CURSOR 1
@@ -541,6 +582,20 @@
#define HOST_CAP_CURSOR 8
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host injects full-fidelity stylus input — it routes
// [`PenBatch`](super::pen::PenBatch) `0xCC/0x05` datagrams (pressure, tilt, azimuth, barrel
// roll, hover, eraser, barrel buttons) through the [`PenTracker`](super::pen::PenTracker)
// into a virtual tablet device (design/pen-tablet-input.md). A capable client (Apple Pencil,
// Android stylus) then splits pen contacts out of its finger/touch path and sends pen
// batches; absent the bit it keeps folding the pen into touch/pointer like today, and
// [`NativeClient::send_pen`](crate::client::NativeClient::send_pen) refuses to send. The
// wire ships ahead of the backend (P0): no host sets this bit until the P1 injector lands —
// which is exactly why the gate exists. `0x10` — `0x08` is [`HOST_CAP_CURSOR`], `0x04` is
// [`HOST_CAP_TEXT_INPUT`], `0x01`/`0x02` are gamepad-state / clipboard.
#define HOST_CAP_PEN 16
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_codecs`] bit: the client can decode H.264 / AVC. The GPU-less **software**
// encode path (openh264) emits H.264, so a client that wants to stream from a software host MUST
@@ -985,6 +1040,73 @@
#define MSG_PAIR_RESULT 19
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::state`] bit: the pen is in the hover range of the surface. Implied by
// [`PEN_TOUCHING`] (decode normalizes, so a client that only sets TOUCHING still produces a
// coherent contact).
#define PEN_IN_RANGE 1
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::state`] bit: the tip is in contact with the surface.
#define PEN_TOUCHING 2
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::state`] bit: the primary barrel button (or the client's squeeze mapping) is held.
#define PEN_BARREL1 4
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::state`] bit: the secondary barrel button (or the client's double-tap mapping)
// is held.
#define PEN_BARREL2 8
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::state`] bit, RESERVED: a predicted (not yet observed) sample. Never sent v1;
// receivers MUST ignore samples carrying it until a capability negotiates otherwise
// (design/pen-tablet-input.md §8).
#define PEN_PREDICTED 128
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::tilt_deg`] sentinel: the client has no tilt sensor / no reading.
#define PEN_TILT_UNKNOWN 255
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::azimuth_deg`] / [`PenSample::roll_deg`] sentinel: no reading.
#define PEN_ANGLE_UNKNOWN 65535
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`PenSample::distance`] sentinel: no hover-distance reading.
#define PEN_DISTANCE_UNKNOWN 65535
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Most samples one [`PenBatch`] can carry. Sized for coalesced capture at video-frame cadence
// (240 Hz pen ÷ 30 fps = 8); a client producing more splits into consecutive batches.
#define PEN_BATCH_MAX 8
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Wire length of one encoded [`PenSample`].
#define PEN_SAMPLE_WIRE_LEN 21
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Host-side failsafe (design/pen-tablet-input.md §2): a tracker still in range after this
// many ms without a sample force-releases ([`PenTracker::force_release`]) — a client that
// died mid-stroke must not leave the host's virtual pen inked-down forever. This makes the
// **client heartbeat a wire contract**: capture APIs only fire on change, so a stationary
// pen is naturally silent — senders MUST repeat the last sample at least every ~100 ms while
// the pen is in range or touching (it re-decodes as pure Motion, harmless), keeping a live
// stationary stroke two heartbeats clear of the deadline.
#define PEN_TOUCH_TIMEOUT_MS 200
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Stream-kind byte: a clipboard fetch (request/response of one format). Future stream kinds
// (e.g. a bulk file-content push) mux under the same [`STREAM_MAGIC`] with a different byte.
@@ -1489,6 +1611,41 @@ typedef struct {
} PunktfunkRichInputEx;
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// One complete stylus state at one instant ([`punktfunk_connection_send_pen`];
// design/pen-tablet-input.md). STATE-FULL, never an edge event: fill every field on every
// sample (unknown axes take their `*_UNKNOWN` sentinel) — the host diffs consecutive samples
// and synthesizes down/up/button transitions itself, which is what makes a lost datagram
// self-heal. `x`/`y` are normalized `0.0..=1.0` in VIDEO-FRAME space (map your letterbox
// before filling, exactly like wire touches).
typedef struct {
// Normalized `0.0..=1.0` across the video frame. Must be finite.
float x;
// Normalized `0.0..=1.0` across the video frame. Must be finite.
float y;
// Tip force, `0..=65535` full scale (`0` while hovering).
uint16_t pressure;
// Hover distance `0..=65534` (0 = at the hover floor), or `PUNKTFUNK_PEN_DISTANCE_UNKNOWN`.
uint16_t distance;
// Tilt azimuth, degrees `0..=359` clockwise from north, or `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
uint16_t azimuth_deg;
// Barrel roll (Apple Pencil Pro `rollAngle`), degrees `0..=359`, or
// `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
uint16_t roll_deg;
// µs since the previous sample in the same call (`0` for the first) — the coalesced
// capture spacing.
uint16_t dt_us;
// Bitfield of `PUNKTFUNK_PEN_*` state bits. Unknown bits are rejected (`InvalidArg`).
uint8_t state;
// `PUNKTFUNK_PEN_TOOL_PEN` or `PUNKTFUNK_PEN_TOOL_ERASER`.
uint8_t tool;
// Tilt from the surface normal, degrees `0..=90`, or `PUNKTFUNK_PEN_TILT_UNKNOWN`.
uint8_t tilt_deg;
// Set to 0.
uint8_t _reserved[3];
} PunktfunkPenSample;
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// One advertised clipboard format passed to [`punktfunk_connection_clipboard_offer`].
typedef struct {
@@ -2310,6 +2467,24 @@ PunktfunkStatus punktfunk_connection_send_rich_input2(PunktfunkConnection *c,
const PunktfunkRichInputEx *rich);
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Send one stylus sample batch — `count` (`1..=PUNKTFUNK_PEN_BATCH_MAX`) state-full
// [`PunktfunkPenSample`]s, oldest first (a capture callback's coalesced samples) — as one
// `0xCC/0x05` pen datagram (non-blocking enqueue; design/pen-tablet-input.md). Split longer
// runs into consecutive calls. Gate on `punktfunk_connection_host_caps() &
// PUNKTFUNK_HOST_CAP_PEN`: toward a host without the bit this returns
// [`PunktfunkStatus::Unsupported`] — keep the pen-as-touch fallback there.
// [`PunktfunkStatus::InvalidArg`] on a bad count or a bad sample (non-finite coordinate,
// unknown state bit / tool).
//
// # Safety
// `c` is a valid connection handle; `samples` is null or points to `count` valid
// [`PunktfunkPenSample`]s.
PunktfunkStatus punktfunk_connection_send_pen(PunktfunkConnection *c,
const PunktfunkPenSample *samples,
uint32_t count);
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// The currently active session mode — the Welcome's, until an accepted
// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect.
@@ -2335,9 +2510,10 @@ PunktfunkStatus punktfunk_connection_gamepad(const PunktfunkConnection *c, uint3
#if defined(PUNKTFUNK_FEATURE_QUIC)
// The host capability bitfield the session's `Welcome` carried — a bitfield of
// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD`. A client tests
// `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide whether to offer the shared-clipboard toggle.
// Safe any time after connect.
// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD` /
// `PUNKTFUNK_HOST_CAP_PEN`. A client tests `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide
// whether to offer the shared-clipboard toggle, `caps & PUNKTFUNK_HOST_CAP_PEN` before
// sending stylus batches. Safe any time after connect.
//
// # Safety
// `c` is a valid connection handle; `caps` is writable (NULL is skipped).
+17 -24
View File
@@ -234,33 +234,25 @@ cp /usr/share/punktfunk/host.env.bazzite ~/.config/punktfunk/host.env
The Bazzite template (`packaging/bazzite/host.env`) contains:
```sh
XDG_RUNTIME_DIR=/run/user/1000
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
# gamescope backend: spawned per session, no compositor login required.
PUNKTFUNK_COMPOSITOR=gamescope
PUNKTFUNK_VIDEO_SOURCE=virtual
PUNKTFUNK_GAMESCOPE_APP=steam -gamepadui
# gamescope hosts its own EIS input socket — input lands in the nested session.
PUNKTFUNK_INPUT_BACKEND=gamescope
# GPU zero-copy capture (dmabuf -> CUDA -> NVENC) is ON by default and auto-falls back to CPU if
# unavailable. No need to set it. Set to 0 only to force the CPU path.
# PUNKTFUNK_ZEROCOPY=0
#RUST_LOG=info
# Gaming Mode = ATTACH: the box owns its gamescope session; the host captures + follows it.
PUNKTFUNK_GAMESCOPE_ATTACH=1
```
**What each knob means and why these are the Bazzite defaults:**
| Knob | Value | Meaning |
|---|---|---|
| `XDG_RUNTIME_DIR` / `DBUS_SESSION_BUS_ADDRESS` | `…/user/1000` | Session bus / runtime dir. **`1000` assumes your user is UID 1000** — change both if `id -u` says otherwise. |
| `PUNKTFUNK_COMPOSITOR` | `gamescope` | **The Bazzite default.** The host spawns a **headless gamescope per session** at the client's exact resolution/refresh and captures its PipeWire node — so you need **no graphical desktop login** to stream. Bazzite ships gamescope, so this "just works." |
| *(no compositor / no anchors)* | — | The host **auto-detects** the live session per connect (Gaming Mode gamescope vs the KDE desktop) and follows switches mid-stream; a `systemctl --user` service inherits the right `XDG_RUNTIME_DIR` and the host derives the bus itself. Pinning `PUNKTFUNK_COMPOSITOR` or hardcoding uid-1000 anchors only breaks this — leave them out. |
| `PUNKTFUNK_VIDEO_SOURCE` | `virtual` | Create a per-client virtual output at the client's exact WxH@Hz (the flagship "native resolution, no scaling" mode), vs. `portal` which captures an existing monitor. |
| `PUNKTFUNK_GAMESCOPE_APP` | `steam -gamepadui` | The command launched **inside** the nested gamescope — here, a SteamOS-style couch UI. Set it to whatever you want the session to run. |
| `PUNKTFUNK_INPUT_BACKEND` | `gamescope` | Inject mouse/keyboard/gamepad into the nested gamescope via its own EIS socket. |
| `PUNKTFUNK_GAMESCOPE_ATTACH` | `1` | Gaming Mode model: the **box** owns its gamescope session; the host attaches to whatever's live and never tears it down. Swap for `PUNKTFUNK_GAMESCOPE_MANAGED=1` to have the host relaunch the gaming session headless at the **client's** exact mode instead (see the template's comments). |
| `PUNKTFUNK_ZEROCOPY` | `on` *(default)* | GPU zero-copy capture (dmabuf → CUDA → NVENC), on by default. Falls back to CPU automatically if unavailable; set `0` to force the CPU path. |
| `RUST_LOG` | (commented) | Uncomment `RUST_LOG=info` for verbose logs while debugging. |
@@ -268,16 +260,14 @@ PUNKTFUNK_INPUT_BACKEND=gamescope
games a virtual Sony DualSense (lightbar, adaptive triggers, touchpad, motion) instead of the
default X-Box-360 pad. The feedback flows back to a real DualSense on the client.
**Alternative — drive the full Plasma/GNOME desktop** instead of a nested gamescope (per the
template's footer comment): switch to `PUNKTFUNK_COMPOSITOR=kwin` and
`PUNKTFUNK_INPUT_BACKEND=libei`, and run the host **inside** a KDE session with `WAYLAND_DISPLAY` /
`XDG_CURRENT_DESKTOP` set. The full knob list (FEC %, per-stage timing, etc.) is in
`scripts/host.env.example` / `/usr/share/punktfunk/host.env.example`.
**The Plasma desktop needs no extra config:** the same auto-detection streams the KDE Desktop
session whenever that's what's live — no compositor pin, no `WAYLAND_DISPLAY` /
`XDG_CURRENT_DESKTOP` (the host retargets those per connect). The full knob list (FEC %, per-stage
timing, etc.) is in `scripts/host.env.example` / `/usr/share/punktfunk/host.env.example`.
> The gamescope default is what makes Bazzite the easy path: it's a **headless, per-session**
> compositor — no desktop login, no display manager, no `--drm` scanout. You don't need any of the
> headless-KDE bring-up scripts (`scripts/headless/run-headless-kde.sh`) on Bazzite unless you
> deliberately switch to the KWin backend.
> Auto-detection is what makes Bazzite the easy path: the host follows the box between Gaming Mode
> and the Desktop — even mid-stream — with a one-line config. You don't need any of the
> headless-KDE bring-up scripts (`scripts/headless/run-headless-kde.sh`) on Bazzite.
---
@@ -474,8 +464,11 @@ desktop viewer.
NVIDIA driver. The code falls back to CPU automatically; check the log for the fallback line and
verify the `-nvidia` image / driver is healthy.
- **Wrong UID in `host.env`.** `XDG_RUNTIME_DIR=/run/user/1000` and the bus path assume UID 1000. Run
`id -u`; if it's different, fix both lines or the host can't reach your session's PipeWire/D-Bus.
- **Session anchors in `host.env`.** The template no longer sets `XDG_RUNTIME_DIR` /
`DBUS_SESSION_BUS_ADDRESS` — a `systemctl --user` service inherits the right values. If an older
config hardcodes them with the wrong uid (`/run/user/1000` when `id -u` isn't 1000), the host
points at another user's PipeWire/D-Bus and everything fails (`pw audio connect … Creation
failed`, no capture). Delete both lines, or fix the uid.
- **Service `ExecStart` points at a missing path in `$HOME`.** The dev unit references
`%h/punktfunk/target/release/...`. The RPM binary is `/usr/bin/punktfunk-host`. Override
+8 -8
View File
@@ -3,10 +3,9 @@
# The compositor + input backend are AUTO-DETECTED per connect from the ACTIVE session: the host
# follows the box as you flip between Steam Gaming Mode (gamescope — a managed session at the
# CLIENT's resolution) and a KDE/GNOME Desktop (KWin/Mutter virtual output at the client's mode).
# So nothing here forces a backend — only the trustworthy anchors stay.
XDG_RUNTIME_DIR=/run/user/1000
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
# So nothing here forces a backend, and no session anchors are needed: a `systemctl --user`
# service inherits the correct XDG_RUNTIME_DIR and the host derives the bus from it. (Keys are
# CASE-SENSITIVE — use the exact uppercase names.)
PUNKTFUNK_VIDEO_SOURCE=virtual
@@ -23,10 +22,11 @@ PUNKTFUNK_VIDEO_SOURCE=virtual
#
# GAME MODE = ATTACH (the box owns its session; the host follows). The box decides whether it's in
# Steam Gaming Mode or a Desktop — you switch with the normal Steam UI / "Switch to Desktop". The
# host just ATTACHES to whatever's live and captures it; it never tears the session down or relaunches
# it. So switching Desktop<->Game is rock-solid, and when you disconnect the box STAYS in its current
# mode — reconnecting drops you right back where you were. The streamed resolution in game mode is the
# box's gamescope mode (see SCREEN_WIDTH/HEIGHT in /etc/gamescope-session-plus/sessions.d/steam).
# host just ATTACHES to whatever's live and captures it; it never tears the session down. So
# switching Desktop<->Game is rock-solid, and when you disconnect the box STAYS in its current
# mode — reconnecting drops you right back where you were. On a resolution mismatch the host
# restarts the box's own game-mode session at the CLIENT's resolution (a foreign/bare gamescope
# instead streams at its own mode).
PUNKTFUNK_GAMESCOPE_ATTACH=1
#
# Opt OUT to the MANAGED model instead (host tears the box's gamescope down on connect and launches
+7
View File
@@ -1,4 +1,11 @@
# punktfunk host config for a Fedora/Ubuntu KDE Plasma appliance (kwin backend).
#
# APPLIANCE-ONLY: this file deliberately PINS the backend (PUNKTFUNK_COMPOSITOR) and the session
# env (WAYLAND_DISPLAY/XDG_CURRENT_DESKTOP) at the dedicated headless KWin session — which also
# turns OFF the host's live-session auto-detection and Desktop<->Game following. On a normal
# desktop (or any box that switches to Steam Game Mode) do NOT use this file; start from
# host.env.example instead, whose defaults auto-detect and follow the live session.
#
# Copy to ~/.config/punktfunk/host.env. Pairs with punktfunk-kde-session.service, which brings
# up a headless `kwin --virtual` on wayland-kde (with KWIN_WAYLAND_NO_PERMISSION_CHECKS=1 so the
# host can bind KWin's privileged zkde_screencast protocol — an interactive Plasma session will
+4 -2
View File
@@ -168,14 +168,16 @@ Everything the RPM's `%install` + `%post` do, declaratively:
### Headless / appliance
Set `autoStart = true`, enable lingering, and pick a backend in `settings`:
Set `autoStart = true`, enable lingering, and — for a **dedicated single-session appliance**
pin a backend in `settings` (pinning `PUNKTFUNK_COMPOSITOR` disables live-session auto-detection,
so leave it out on any box that switches between a desktop and Game Mode):
```nix
services.punktfunk.host = {
enable = true;
autoStart = true;
users = [ "streamer" ];
settings = { PUNKTFUNK_COMPOSITOR = "gamescope"; }; # or kwin/mutter/wlroots
settings = { PUNKTFUNK_COMPOSITOR = "gamescope"; }; # appliance-only; omit to auto-detect
};
users.users.streamer.linger = true;
# For the gamescope/KWin backends extend the service PATH, e.g.:
@@ -165,9 +165,11 @@ unsafe fn add(request: WDFREQUEST) {
// This WUDFHost's pid — where the host duplicates the sealed frame channel's handles INTO
// (`ProcessSharingDisabled`: this process is exclusively ours and dies with the device).
wudf_pid: std::process::id(),
// The target already carries an irrevocable hardware-cursor declare from an earlier
// session — a channel-less session must self-composite the pointer (§8.6 gap).
cursor_excluded: crate::monitor::target_declared(target_id) as u32,
// The ADAPTER already carries an irrevocable hardware-cursor declare from an earlier
// session — DWM's exclusion reaches every later monitor, not just the declaring target
// (on-glass 2026-07-23: declare on 259 left a fresh GameStream 257 cursor-less) — so a
// channel-less session must self-composite the pointer (§8.6 gap, adapter-wide).
cursor_excluded: crate::monitor::any_declared() as u32,
};
// Dual-size reply (the `cursor_excluded` tail ext): an un-upgraded host retrieves only the
// legacy 20-byte buffer — write the prefix it asked for instead of failing its ADD.
@@ -200,15 +200,18 @@ fn cursor_forward_desired(target_id: u32) -> bool {
}
/// OS target ids on which `IddCxMonitorSetupHardwareCursor` ever SUCCEEDED. A declare is
/// IRREVOCABLE for the target's life (no un-declare DDI; empty-caps re-setup is rejected; even a
/// successful same-mode re-commit never brings DWM's composited pointer back — all proven
/// on-glass, remote-desktop-sweep §8.6) and it survives monitor REMOVE→ADD because the host hands
/// each client a STABLE target id. Reported back on every ADD
/// ([`AddReply::cursor_excluded`](pf_driver_proto::control::AddReply)) so a session that never
/// negotiates the cursor channel knows it must self-composite the pointer instead of streaming a
/// cursor-less desktop. Never cleared: the state's true scope is this WUDFHost's life
/// (`ProcessSharingDisabled` — the process, and with it the adapter and every sticky declare,
/// dies on adapter reset), which is exactly when this static resets too. Bounded: ≤ 16 ids.
/// IRREVOCABLE (no un-declare DDI; empty-caps re-setup is rejected; even a successful same-mode
/// re-commit never brings DWM's composited pointer back — all proven on-glass,
/// remote-desktop-sweep §8.6) and its EXCLUSION REACH IS THE WHOLE ADAPTER, not the declaring
/// target: a declare on one target leaves every LATER monitor's frames pointer-free too, even a
/// fresh never-declared target id (proven on-glass 2026-07-23: declare on 259, then a GameStream
/// session's fresh 257 streamed a cursor-less desktop). ADD replies therefore report
/// [`AddReply::cursor_excluded`](pf_driver_proto::control::AddReply) from [`any_declared`] —
/// a session that never negotiates the cursor channel must self-composite the pointer instead
/// of streaming a cursor-less desktop. Never cleared: the state's true scope is this WUDFHost's
/// life (`ProcessSharingDisabled` — the process, and with it the adapter and every sticky
/// declare, dies on adapter reset), which is exactly when this static resets too. Per-target
/// ids are kept purely for the dbglog audit trail. Bounded: ≤ 16 ids.
static DECLARED_TARGETS: Mutex<Vec<u32>> = Mutex::new(Vec::new());
/// Record a SUCCESSFUL hardware-cursor declare on `target_id` (see [`DECLARED_TARGETS`]).
@@ -220,12 +223,13 @@ fn mark_declared(target_id: u32) {
}
}
/// True if `target_id` ever had a hardware cursor declared (see [`DECLARED_TARGETS`]).
pub fn target_declared(target_id: u32) -> bool {
DECLARED_TARGETS
/// True if ANY target ever had a hardware cursor declared in this WUDFHost's life — the
/// adapter-wide exclusion reach (see [`DECLARED_TARGETS`]).
pub fn any_declared() -> bool {
!DECLARED_TARGETS
.lock()
.unwrap_or_else(|e| e.into_inner())
.contains(&target_id)
.is_empty()
}
/// Record a departing monitor's advertised list for its id ([`MODE_HISTORY`]).
+114
View File
@@ -0,0 +1,114 @@
#!/usr/bin/env bash
# Announce a stable Punktfunk release to the Discord #releases channel.
#
# Model: release notes live in the repo at docs/releases/<tag>.md (authored during the version
# bump, BEFORE the tag is pushed — see docs/releases/README.md). CI seeds the Gitea release body
# from that file at creation, so the release is never noteless. This script is the deliberate
# "go" step (workflow_dispatch, .gitea/workflows/announce.yml): once every platform's CI is green
# it re-asserts the notes file over the live release, then posts a formatted embed to Discord.
#
# Usage: scripts/ci/discord-announce.sh <tag> # e.g. v0.18.0
# Env:
# GITHUB_SERVER_URL https://git.unom.io (Gitea Actions sets it)
# GITHUB_REPOSITORY unom/punktfunk (Gitea Actions sets it)
# GITEA_TOKEN PAT with repo write scope (from secrets.REGISTRY_TOKEN)
# DISCORD_RELEASE_WEBHOOK the #releases channel webhook URL (from secrets)
# ALLOW_PRERELEASE "1" to permit announcing a -rc/pre-release tag (default: refuse)
#
# Requires: curl + python3 (both present on the ubuntu-24.04 runner).
set -euo pipefail
TAG="${1:?usage: discord-announce.sh <tag>}"
: "${GITHUB_SERVER_URL:?}" "${GITHUB_REPOSITORY:?}" "${GITEA_TOKEN:?}" "${DISCORD_RELEASE_WEBHOOK:?}"
# Stable-only guard: a `-` in the tag marks a pre-release (v0.2.0-rc1). Refuse unless explicitly
# allowed, so an rc dispatched by accident never pings the community.
case "$TAG" in
*-*)
case "$(printf '%s' "${ALLOW_PRERELEASE:-0}" | tr '[:upper:]' '[:lower:]')" in
1|true|yes) : ;;
*) echo "discord-announce: '$TAG' looks like a pre-release; set ALLOW_PRERELEASE=1 to override" >&2
exit 1 ;;
esac ;;
esac
_here="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)"
API="$GITHUB_SERVER_URL/api/v1/repos/$GITHUB_REPOSITORY"
# Resolve the release object created by the build workflows (id, html_url, published_at).
release_json="$(curl -fsS "$API/releases/tags/$TAG" -H "Authorization: token $GITEA_TOKEN")"
RID="$(printf '%s' "$release_json" | python3 -c 'import json,sys;print(json.load(sys.stdin).get("id",""))')"
[ -n "$RID" ] || { echo "discord-announce: no release found for tag '$TAG' — has CI created it yet?" >&2; exit 1; }
# Re-assert the notes file over the live release (source of truth), covering an edit after the
# release object was first created. No-op if docs/releases/<tag>.md is absent.
# shellcheck source=scripts/ci/gitea-release.sh
. "$_here/gitea-release.sh"
apply_release_notes "$RID" "$TAG"
# Build and post the Discord embed. The description is the notes' lead-in (everything before the
# first `## ` section header), trimmed to a readable length, with a link to the full release page
# for the sections + downloads. Falls back to the release's own body if there is no notes file.
NOTES_PATH="$(_release_notes_path "$TAG" || true)"
payload_file="$(mktemp)"
trap 'rm -f "$payload_file"' EXIT
RELEASE_JSON="$release_json" NOTES_PATH="$NOTES_PATH" TAG="$TAG" python3 - "$payload_file" <<'PY'
import json, os, sys
rel = json.loads(os.environ["RELEASE_JSON"])
tag = os.environ["TAG"]
version = tag[1:] if tag.startswith("v") else tag
url = rel.get("html_url") or ""
published = rel.get("published_at") or rel.get("created_at") or ""
notes_path = os.environ.get("NOTES_PATH") or ""
body = ""
if notes_path and os.path.isfile(notes_path):
with open(notes_path, encoding="utf-8") as f:
body = f.read()
else:
body = rel.get("body") or ""
# Lead-in = text before the first `## ` section header (the intro paragraphs); fall back to the
# whole body when the notes open straight into a section.
lead = body
idx = body.find("\n## ")
if idx != -1:
lead = body[:idx]
lead = lead.strip()
if not lead:
lead = body.strip()
LIMIT = 1800
if len(lead) > LIMIT:
cut = lead.rfind("\n\n", 0, LIMIT)
if cut < LIMIT // 2:
cut = lead.rfind(" ", 0, LIMIT)
if cut <= 0:
cut = LIMIT
lead = lead[:cut].rstrip() + " …"
description = lead
if url:
description = (description + "\n\n" if description else "") + \
f"**[⬇️ Download & full release notes →]({url})**"
embed = {
"title": f"Punktfunk {version}",
"color": 0x6C5BF3, # --pf-brand deep violet
"description": description,
"footer": {"text": "Punktfunk • git.unom.io"},
}
if url:
embed["url"] = url
if published:
embed["timestamp"] = published
with open(sys.argv[1], "w", encoding="utf-8") as f:
json.dump({"embeds": [embed]}, f)
PY
curl -fsS -o /dev/null -X POST "$DISCORD_RELEASE_WEBHOOK" \
-H 'Content-Type: application/json' --data-binary @"$payload_file"
echo "discord-announce: posted Punktfunk $TAG to #releases (release $RID)"
+5
View File
@@ -40,6 +40,11 @@ function Ensure-GiteaRelease {
$api = _GiteaApi; $h = _GiteaHeaders
$payload = @{ tag_name = $Tag; name = $Name; prerelease = $pre }
if ($TargetCommitish) { $payload.target_commitish = $TargetCommitish }
# Seed the body from docs/releases/<Tag>.md (the source of truth) so a Windows job winning the
# create race is born WITH its notes, exactly like the bash twin. Missing file (canary/rc) -> no
# body key. $PSScriptRoot is scripts/ci; walk up two levels to the repo root.
$notes = Join-Path (Split-Path (Split-Path $PSScriptRoot -Parent) -Parent) "docs/releases/$Tag.md"
if (Test-Path $notes) { $payload.body = Get-Content -Raw -Encoding utf8 $notes }
try {
$r = Invoke-RestMethod -Method Post -Uri "$api/releases" -Headers $h `
-ContentType 'application/json' -Body ($payload | ConvertTo-Json -Compress)
+55 -6
View File
@@ -35,6 +35,22 @@ for a in json.load(sys.stdin):
}
_urlencode() { python3 -c 'import urllib.parse,sys;print(urllib.parse.quote(sys.argv[1],safe=""))' "$1"; }
# _release_notes_path TAG
# Print the path of the in-repo release notes for TAG (docs/releases/<TAG>.md) IFF it exists,
# else print nothing. This file is the single source of truth for a stable release's body
# (authored as part of the version bump, before the tag is pushed — see docs/releases/README.md),
# so the Gitea release is born WITH its notes instead of being PATCHed noteless-then-late.
# canary/rc tags have no such file, which is intended (they get no curated body).
#
# Resolved relative to CWD: every caller sources this as `. scripts/ci/gitea-release.sh` from the
# repo root, so the notes are always docs/releases/<tag>.md from here. Do NOT use ${BASH_SOURCE[0]}
# — the deb + decky attach steps run under POSIX sh (dash), where an array subscript is a fatal
# "Bad substitution" (it silently broke the v0.19.0 deb/decky release-attach).
_release_notes_path() {
local notes="docs/releases/$1.md"
[ -f "$notes" ] && printf '%s' "$notes"
}
# ensure_release TAG NAME PRERELEASE [TARGET_COMMITISH]
# Idempotently create (or fetch) the release for TAG; prints its numeric id on stdout.
# PRERELEASE is "true", "false", or "auto" — auto marks it a prerelease iff TAG carries a
@@ -49,12 +65,25 @@ ensure_release() {
case "$tag" in *-*) prerelease=true ;; *) prerelease=false ;; esac
fi
api="$(_gitea_api)"
if [ -n "$target" ]; then
body=$(printf '{"tag_name":"%s","name":"%s","prerelease":%s,"target_commitish":"%s"}' \
"$tag" "$name" "$prerelease" "$target")
else
body=$(printf '{"tag_name":"%s","name":"%s","prerelease":%s}' "$tag" "$name" "$prerelease")
fi
# Build the create payload with python3 so the (multi-line, quote-bearing) release body from
# docs/releases/<tag>.md is JSON-escaped correctly. Whichever workflow wins the create race thus
# sets the body ATOMICALLY at creation; the losers 409 and fall through to the fetch-by-tag path
# below (which never touches the body). No notes file (canary/rc) -> no body key -> empty body.
local notes; notes="$(_release_notes_path "$tag")"
body=$(TAG="$tag" NAME="$name" PRERELEASE="$prerelease" TARGET="$target" NOTES_FILE="$notes" \
python3 - <<'PY'
import json, os
d = {"tag_name": os.environ["TAG"], "name": os.environ["NAME"],
"prerelease": os.environ["PRERELEASE"] == "true"}
if os.environ.get("TARGET"):
d["target_commitish"] = os.environ["TARGET"]
nf = os.environ.get("NOTES_FILE") or ""
if nf:
with open(nf, encoding="utf-8") as f:
d["body"] = f.read()
print(json.dumps(d))
PY
)
# Try to create. On any failure (almost always "release already exists"), fall back to
# fetching it by tag. Either path MUST yield an id, or we error loudly — so a 401/scope
# problem can't masquerade as a successful no-op.
@@ -93,3 +122,23 @@ upsert_asset() {
-F "attachment=@$file"
echo "gitea-release: uploaded '$name' -> release $rid"
}
# apply_release_notes RELEASE_ID TAG
# Force the release body to match docs/releases/<TAG>.md (the source of truth), if that file
# exists — a no-op otherwise. PATCHes ONLY the body, so name/prerelease/assets are preserved
# (Gitea has no partial-update footgun here; sending just {"body":...} leaves everything else).
# ensure_release already seeds the body at creation; this exists so the announce step can
# re-assert the file over the live release right before publishing — covering the case where the
# notes file was edited after the release object was first created (e.g. a tag re-point).
apply_release_notes() {
local rid="${1:?release id}" tag="${2:?tag}" api notes payload
notes="$(_release_notes_path "$tag")"
[ -n "$notes" ] || { echo "gitea-release: no docs/releases/$tag.md — leaving body as-is"; return 0; }
api="$(_gitea_api)"
payload=$(NOTES_FILE="$notes" python3 -c \
'import json,os;print(json.dumps({"body":open(os.environ["NOTES_FILE"],encoding="utf-8").read()}))')
curl -fsS -o /dev/null -X PATCH "$api/releases/$rid" \
-H "Authorization: token ${GITEA_TOKEN:?}" -H 'Content-Type: application/json' \
-d "$payload"
echo "gitea-release: synced release $rid body from $notes"
}
+44 -35
View File
@@ -1,16 +1,17 @@
# punktfunk host configuration (~/.config/punktfunk/host.env) — consumed by punktfunk-host.service.
#
# The compositor + input backend are AUTO-DETECTED per connect from the live session (the host
# probes which compositor is actually running and retargets WAYLAND_DISPLAY/XDG_CURRENT_DESKTOP/
# DBUS at it), so a box that flips between Steam Gaming Mode and a KDE/GNOME desktop is followed
# automatically. The blocks below are OPTIONAL OVERRIDES — uncomment one only to force a backend
# (this also skips the per-connect env retargeting). The anchors XDG_RUNTIME_DIR + DBUS stay.
# Session / compositor environment (headless KWin example).
XDG_RUNTIME_DIR=/run/user/1000
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
WAYLAND_DISPLAY=wayland-kde
XDG_CURRENT_DESKTOP=KDE
# YOU BARELY NEED THIS FILE. The host AUTO-DETECTS the live session per connect — which compositor
# is running (KWin / Mutter / sway / Hyprland / gamescope), its Wayland socket, session bus, and the
# matching input backend — and FOLLOWS the box when it switches between a desktop and Steam Gaming
# Mode, even mid-stream. Everything below except PUNKTFUNK_VIDEO_SOURCE is an optional override.
#
# Two rules that save debugging sessions:
# * Keys are CASE-SENSITIVE. `punktfunk_gamescope_attach=1` sets nothing — use the exact
# uppercase names.
# * On a desktop you actually use, do NOT set PUNKTFUNK_COMPOSITOR / WAYLAND_DISPLAY /
# XDG_CURRENT_DESKTOP. Pinning the compositor DISABLES session-following (a switch to Game
# Mode mid-stream then kills the stream instead of being followed), and stale session vars
# point detection at dead sockets. Those knobs are for CI and dedicated appliances (below).
# Video source: `virtual` creates a per-client virtual output at the client's exact
# resolution+refresh (the flagship mode); `portal` captures an existing monitor.
@@ -20,33 +21,41 @@ PUNKTFUNK_VIDEO_SOURCE=virtual
# CPU automatically. No need to set it. Set to 0 only to force the CPU path.
# PUNKTFUNK_ZEROCOPY=0
# --- Bazzite / SteamOS-like host: host-managed Steam-Deck-UI session -----------------------
# The host LAUNCHES gamescope-session-plus headless AT THE CLIENT'S mode (so games see the
# client's exact resolution + refresh, not the box's TV), and relaunches it when the mode
# changes. Requires the headless-appliance prereqs (linger + multi-user.target — see
# punktfunk-steam-session.service header) and NO physical gaming session running.
#PUNKTFUNK_COMPOSITOR=gamescope
#PUNKTFUNK_GAMESCOPE_SESSION=steam # host owns a gamescope-session-plus session at the client mode
#PUNKTFUNK_INPUT_BACKEND=gamescope
# Mutually exclusive with the above: ATTACH to a gamescope session something ELSE owns (fixed mode):
#PUNKTFUNK_GAMESCOPE_NODE=auto # discover + capture a running gamescope (do NOT combine with SESSION)
# --- Session anchors (rarely needed) -------------------------------------------------------
# As a `systemctl --user` service the host inherits the correct XDG_RUNTIME_DIR from systemd and
# derives the bus (`unix:path=$XDG_RUNTIME_DIR/bus`) itself. Set these ONLY when running the host
# outside a user service (ssh, cron) — and with YOUR uid (`id -u`), never a copy-pasted 1000: a
# wrong uid points the host at another user's (nonexistent) PipeWire/D-Bus, and every session
# fails with errors like "pw audio connect … Creation failed".
#XDG_RUNTIME_DIR=/run/user/<uid>
#DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/<uid>/bus
# --- GNOME / Mutter host (e.g. an Ubuntu desktop) -----------------------------------------
# Attach to a running GNOME (Wayland) session — its default socket is wayland-0, not wayland-kde.
# Mutter creates the per-client virtual output via its `RecordVirtual` D-Bus API (a virtual
# monitor alongside any real one), and input goes through the RemoteDesktop portal (libei). On a
# real desktop the host runs as the logged-in user; headless GNOME also works (gnome-shell
# --headless). Needs GNOME ≥ 48 for the zero-copy RecordVirtual path.
#WAYLAND_DISPLAY=wayland-0
#XDG_CURRENT_DESKTOP=GNOME
#PUNKTFUNK_COMPOSITOR=mutter
#PUNKTFUNK_VIDEO_SOURCE=virtual
#PUNKTFUNK_INPUT_BACKEND=libei
# --- Steam Gaming Mode (Linux boxes with gamescope session infra: Bazzite/SteamOS/Nobara) ---
# Game Mode is auto-handled; two models decide WHERE it runs when a client streams:
# * MANAGED (the default where session infra is detected) — the host relaunches the gaming
# session HEADLESS at the CLIENT's exact mode ("game mode on the virtual screen"); physical
# displays drop out of it, and the box is restored on a debounced idle after disconnect.
# * ATTACH — the BOX owns its session: Game Mode stays on the physical screen and the host
# captures/follows it, never tearing it down. Reconnects land wherever the box is.
#PUNKTFUNK_GAMESCOPE_ATTACH=1 # pick the ATTACH model
#PUNKTFUNK_GAMESCOPE_MANAGED=1 # force MANAGED even where infra detection wouldn't pick it
#PUNKTFUNK_GAMESCOPE_SESSION=steam # host owns a gamescope-session-plus session at the client mode
#PUNKTFUNK_GAMESCOPE_NODE=auto # raw attach: discover + capture a running gamescope's node
# # (do NOT combine with SESSION)
#PUNKTFUNK_GAMESCOPE_APP=vkcube # nested command for ad-hoc bare-gamescope sessions
#PUNKTFUNK_SESSION_WATCH=0 # disable mid-stream Desktop<->Game following (on by default
# # on gamescope-infra boxes)
# --- Force a backend (CI / tests / dedicated single-session appliances ONLY) ---------------
# PINS the backend: the host stops following the live session entirely — per connect AND
# mid-stream. Fine for a dedicated headless appliance (punktfunk-kde-session.service, a pure
# gamescope box) or a CI run; wrong for any box that switches sessions.
#PUNKTFUNK_COMPOSITOR=kwin # kwin | mutter | gamescope | wlroots | hyprland
#PUNKTFUNK_INPUT_BACKEND=libei # wlr | libei | gamescope | uinput (auto-routed per connect)
#WAYLAND_DISPLAY=wayland-kde # headless-KDE appliance socket; retargeted per connect otherwise
#XDG_CURRENT_DESKTOP=KDE
# Optional overrides (apps.json is the primary mechanism for per-app settings):
#PUNKTFUNK_COMPOSITOR=kwin # kwin | mutter | gamescope | wlroots
#PUNKTFUNK_GAMESCOPE_APP=vkcube # nested command for ad-hoc bare-gamescope sessions
#PUNKTFUNK_INPUT_BACKEND=libei # wlr | libei | gamescope | uinput
#PUNKTFUNK_FEC_PCT=20 # video FEC overhead percent
#PUNKTFUNK_PERF=1 # per-stage timing logs
#PUNKTFUNK_MDNS=0 # disable the mDNS adverts (native + GameStream) — for multicast-
+8 -5
View File
@@ -2,15 +2,18 @@
# GameStream/Moonlight-compat planes). For a SECURE native-only host (no plain-HTTP pairing / legacy
# GCM nonce reuse — security-review #5/#9; native clients only), drop `--gamestream` from ExecStart.
#
# Install (against an already-running compositor session):
# Install (against an already-running compositor session — the host auto-detects and follows it,
# so host.env needs no backend config):
# mkdir -p ~/.config/systemd/user && cp scripts/punktfunk-host.service ~/.config/systemd/user/
# cp scripts/host.env.example ~/.config/punktfunk/host.env # then edit for your backend
# cp scripts/host.env.example ~/.config/punktfunk/host.env # defaults are right for a desktop
# systemctl --user daemon-reload && systemctl --user enable --now punktfunk-host
#
# Self-contained boot appliance (no login, no manual steps after boot):
# Self-contained boot appliance (no login, no manual steps after boot). These routes PIN the
# backend via PUNKTFUNK_COMPOSITOR — correct for a dedicated single-session box, but it turns off
# live-session auto-detection, so never do it on a desktop that switches sessions (Game Mode etc.):
# - kwin backend (stream the Plasma desktop): also install + enable
# punktfunk-kde-session.service (it brings up the headless KWin session this After=s), and set
# PUNKTFUNK_COMPOSITOR=kwin + WAYLAND_DISPLAY=wayland-kde in host.env.
# punktfunk-kde-session.service (it brings up the headless KWin session this After=s), and use
# the shipped packaging/kde/host.env (pins kwin + WAYLAND_DISPLAY=wayland-kde on purpose).
# - gamescope backend (stream a nested app, no desktop): set PUNKTFUNK_COMPOSITOR=gamescope in
# host.env — the host spawns gamescope per session, so no kde-session unit is needed.
# Then `sudo loginctl enable-linger "$USER"` so user units start at boot, and reboot.