Compare commits

..
Author SHA1 Message Date
enricobuehlerandClaude Opus 4.8 f4fe4d0792 chore(release): bump workspace version to 0.19.2
android-screenshots / screenshots (push) Successful in 2m59s
windows-host / package (push) Successful in 9m45s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m48s
android / android (push) Successful in 12m37s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m20s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m5s
arch / build-publish (push) Successful in 12m2s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m45s
audit / cargo-audit (push) Successful in 2m10s
audit / bun-audit (push) Successful in 14s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m6s
apple / swift (push) Successful in 1m57s
ci / bench (push) Successful in 5m41s
ci / rust (push) Successful in 18m52s
decky / build-publish (push) Successful in 20s
deb / build-publish (push) Successful in 8m50s
deb / build-publish-host (push) Successful in 9m24s
web-screenshots / screenshots (push) Successful in 3m7s
flatpak / build-publish (push) Successful in 6m27s
docker / deploy-docs (push) Successful in 12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
release / apple (push) Successful in 1h4m50s
apple / screenshots (push) Successful in 23m21s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m40s
linux-client-screenshots / screenshots (push) Successful in 7m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m0s
An all-fixes patch on top of 0.19.1, all host-side:

- KDE Plasma display takeover moved off kscreen-doctor to in-process
  kde_output_management_v2 (86d5d666), with the >60 Hz custom-mode install
  ported too (64511af4) — a KWin host whose kscreen-doctor wedges now disables
  its physicals and hands over the streamed output in ~2.4 ms instead of hanging
  ~26 s and never taking over.
- The gamescope managed takeover is display-manager-flavor-aware (e35dad52) so
  switching a non-SDDM Linux desktop (Nobara / plasmalogin) into Game Mode no
  longer start-limits the display manager into a permanent black screen, and
  in-stream 'Switch to Desktop' is honored under it (a3e21d92).
- Windows display isolation anchors the kept sources at the desktop origin
  (9b241d9d) so an origin-less desktop no longer fails 0x57, with restore
  guaranteed never to leave the desktop all-dark.
- Windows pen/touch/absolute-mouse map over the streamed output's rect rather
  than the whole virtual desktop (087e7d04), so absolute input lands correctly
  when a physical monitor is kept on beside the streamed display.

Nothing moved on the wire (protocol 2) or at the C ABI (13), so no embedder
rebuild and 0.18/0.19 peers keep mixing freely. The Windows virtual-display
driver protocol is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 17:56:08 +02:00
enricobuehlerandClaude Fable 5 087e7d0405 fix(inject/windows): map absolute input over the streamed output's rect, not the whole desktop
audit / cargo-audit (push) Successful in 2m13s
audit / bun-audit (push) Successful in 15s
arch / build-publish (push) Successful in 12m46s
windows-host / package (push) Successful in 12m53s
android / android (push) Successful in 13m48s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m9s
apple / swift (push) Successful in 15m13s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m9s
ci / bench (push) Successful in 5m28s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m10s
ci / rust (push) Successful in 19m38s
decky / build-publish (push) Successful in 27s
deb / build-publish (push) Successful in 8m37s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 44s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 1m36s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 1m39s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Failing after 1m15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m5s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m54s
docker / deploy-docs (push) Skipped
deb / build-publish-host (push) Successful in 10m7s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m26s
flatpak / build-publish (push) Successful in 6m33s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m41s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m42s
release / apple (push) Successful in 56m19s
apple / screenshots (push) Successful in 23m11s
Pen, touch, and absolute mouse arrive normalized to the STREAMED display's
frame, but pointer_windows::to_screen and sendinput's MouseMoveAbs mapped
them over the whole virtual desktop — correct only when the virtual display
is the sole active display (Exclusive topology). In Extend — a physical
monitor kept on beside the virtual output, or an Exclusive isolate degraded
to the 0x57 keep-physicals fallback — the streamed output sits at a non-zero
origin, so every sample landed shifted and mis-scaled. The pen exposed it
first (field report 2026-07-24): a stylus is strictly absolute, with no
closed-loop correction onto the target like a cursor.

New pf-inject::stream_target (Windows): the host publishes the streamed
output's CCD target id at capture bring-up (one central site —
capture_virtual_output covers the native and GameStream planes); mapping
sites resolve its current desktop rect through pf-win-display's
source_desktop_rect — the same resolver the cursor-readback poller uses, so
inject and readback always agree — TTL-cached (250 ms) because a
group-layout re-arrange moves a live output's origin mid-session. No target
set / never resolved falls back to the whole virtual desktop (the historical
mapping, still right for Exclusive topology and devtest).

One change in to_screen covers pen + touch; MouseMoveAbs converts the same
desktop pixel into the 0..65535 MOUSEEVENTF_VIRTUALDESK coordinate, closing
the identical latent absolute-mouse bug (apollo-comparison open item #14/#30).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 17:38:36 +02:00
enricobuehlerandClaude Opus 4.8 64511af4d3 fix(kwin): install >60Hz custom modes in-process too — KWin path is now kscreen-free
audit / bun-audit (push) Successful in 35s
audit / cargo-audit (push) Successful in 2m34s
decky / build-publish (push) Successful in 17s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 47s
deb / build-publish-host (push) Successful in 9m56s
deb / build-publish (push) Successful in 11m17s
android / android (push) Successful in 12m31s
docker / deploy-docs (push) Successful in 29s
arch / build-publish (push) Successful in 15m14s
apple / swift (push) Successful in 1m21s
apple / screenshots (push) Successful in 4m50s
release / apple (push) Successful in 12m45s
flatpak / build-publish (push) Successful in 7m14s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m58s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m18s
windows-host / package (push) Successful in 16m32s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m19s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m14s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m28s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m25s
ci / rust (push) Successful in 17m55s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 59s
ci / bench (push) Successful in 5m45s
Follow-up to the previous commit: the only KWin topology op still shelling out to
kscreen-doctor was the >60 Hz custom-mode INSTALL (`set_custom_refresh`). Port it
to `kde_output_management_v2` so the whole KWin path is kscreen-free on modern KWin.

`kwin_output_mgmt::set_custom_mode` builds a one-entry `kde_mode_list_v2`
(set_resolution / set_refresh_rate / set_reduced_blanking=full / add_mode), applies
it via `kde_output_configuration_v2.set_custom_modes` (since 18), waits for KWin to
generate the mode (its CVT generator may align the width down — matched with the
same height-exact / width-within-8 / refresh-within-1Hz gate as before), then
selects it — which changes the output size and drives the sacrificial-birth stream
renegotiation. `kwin::create`'s want_high branch tries this first and only resolves
a kscreen id + calls `set_custom_refresh` if it returns None (pre-6.6 KWin without
set_custom_modes, or the compositor not answering) — so a >60 Hz session on a box
where kscreen-doctor wedges no longer eats a 5 s resolve timeout either.

Bonus: `set_custom_modes` REPLACES the custom list, so reconnects are idempotent —
no more one-custom-mode-per-connect growth of the user's display list.

Verified: cargo test -p pf-vdisplay (73 pass), clippy -D warnings clean, fmt clean.
The set_custom_modes round-trip is proven live on KWin 6.6.4: apply=applied, and a
1648x928@75 request generates 1648x928@74.901 (CVT fractional refresh, caught by the
1 Hz tolerance). The mode SELECT + full >60 Hz stream is still owed on-glass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 12:47:34 +02:00
enricobuehlerandClaude Opus 4.8 86d5d66660 fix(kwin): drive display topology over kde_output_management_v2, not kscreen-doctor
A KDE host on Nobara stopped disabling its physical screens and creating the
virtual output the moment it updated: streaming still came up, but bring-up took
~26 s and the streamed output never became the desktop (`also_disabled=[]`).

Root cause is not our topology logic — it's that every `kscreen-doctor` call on
the reporter's session wedges. kscreen-doctor drives libkscreen, which (per
setup) waits on the kscreen KDED module over D-Bus; when that layer is stuck it
blocks in its own connect and never returns, so all five topology queries hit
their 5 s budget and got killed (host log). Reproduced live on a Nobara / KWin
6.6.4 box: `kscreen-doctor -j` there times out at 8 s, every time.

But the compositor's OWN Wayland is fully responsive on that same session — the
host just created a virtual output over it via zkde_screencast. So drive the
topology (resolve our output, take primary, disable the physical/bootstrap
outputs, capture their modes, re-enable on teardown, position) directly over
`kde_output_management_v2` + `kde_output_device_v2` instead of shelling out.
On that same Nobara box the in-process path binds management (v19) and
enumerates the outputs in 2.4 ms — ~3400x faster than the 8 s hang, and immune
to whatever wedges the standalone tool.

- vendor kde-output-management-v2 / kde-output-device-v2 (KWin advertises mgmt
  v19, device v20); generate client bindings inline (the interdependent-protocol
  module split from the wayland-protocols crate; needs the `bitflags` dep for the
  device protocol's bitfield enums).
- new `kwin_output_mgmt`: bounded enumerate-then-apply over one Wayland
  connection; every wait is time-bounded so a genuinely wedged compositor
  degrades to `handled = false` and the old kscreen-doctor path still runs.
- `kwin::create` topology + `apply_position` prefer the in-process path (address
  our output by its stable device UUID, supersede-robust) with kscreen-doctor as
  the fallback. The 60 Hz path now makes ZERO kscreen-doctor calls; only the
  >60 Hz custom-mode install still shells out (its in-process port is a follow-up).

Verified on Linux: cargo test -p pf-vdisplay (73 pass), clippy --all-targets
-D warnings clean, fmt clean, punktfunk-host checks clean. In-process enumeration
proven live against KWin 6.6.4 (the reporter's env); the disable/apply path is
not yet on-glass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 12:47:33 +02:00
enricobuehlerandClaude Fable 5 a3e21d9227 feat(vdisplay/gamescope): honor in-stream 'Switch to Desktop' under the managed DM takeover
ci / web (push) Successful in 59s
ci / docs-site (push) Successful in 1m7s
apple / swift (push) Successful in 1m21s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
ci / bench (push) Successful in 6m2s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m32s
apple / screenshots (push) Successful in 6m21s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m27s
deb / build-publish-host (push) Successful in 9m39s
docker / deploy-docs (push) Successful in 29s
deb / build-publish (push) Successful in 13m6s
android / android (push) Successful in 17m28s
arch / build-publish (push) Successful in 18m8s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m9s
windows-host / package (push) Successful in 24m2s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m12s
ci / rust (push) Successful in 26m49s
First on-glass test of the DM-flavor takeover confirmed the black-screen
fix, but exposed that the managed model could not switch back: Steam's
in-stream session-select is a silent no-op while the display manager is
stopped (every config-write branch in os-session-select requires the DM
to be running), so the capture loss it causes just relaunched game mode.

The user pass of the switch script does leave one durable trace — the
~/.config/steamos-session-select sentinel, written before any DM check.
The managed launch now baselines its mtime; a capture loss with the
sentinel advanced reads as the switch request and is honored by
replaying the switch with the DM up: restore the display manager, run
the distro's own os-session-select (its internal pkexec is authorized
allow_any by the distro policy, so it works from the host's sessionless
context), then stop the autologin game-mode unit so Relogin lands in
the newly selected desktop. The capture-loss re-detection then follows
KWin as it comes up; a 120 s post-honor grace stops the rebuild loop
from racing the booting desktop back into game mode (superseded early
if the box's own game-mode unit reappears — the desktop→game leg stays
fast). The baseline is re-recorded on crash-restore so a pre-existing
sentinel never reads as a fresh request. Every verb in the sequence was
live-validated on the Nobara repro VM.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 11:49:24 +02:00
enricobuehlerandClaude Fable 5 e35dad529b fix(vdisplay/gamescope): DM-flavor-aware session takeover — stop masking Nobara's plasmalogin to death
Field report (Nobara 43 HTPC): switching the host to Steam Game Mode
mid-stream permanently black-screened the box. Live-proven root cause on
a Nobara repro VM: our takeover masks the box's gamescope-session-plus
unit, plasmalogin's Relogin=true then fails its session Exec repeatedly
and trips systemd's start limit within ~1 s — the display manager dies,
and our restore verb (unmask + user start) cannot bring a seatless
gamescope back. Only 'reset-failed + restart' of the DM recovers.

The takeover is now display-manager-flavor-aware:
- SDDM (Bazzite/SteamOS) keeps the proven mask+SIGKILL path unchanged.
- plasmalogin/unknown DMs never mask: with privilege (root or an
  operator polkit rule scoped to the DM unit — documented) the host
  stops the DM for the stream and restores it with reset-failed +
  restart (PUNKTFUNK_RECOVER_SESSION_CMD as fallback), recorded in the
  persisted takeover state so a host crash still restores; without
  privilege the managed takeover degrades to ATTACH and mirrors the
  box's live Game Mode instead of destabilizing the seat. Both legs of
  the privileged cycle live-verified on the repro VM (headless managed
  session works with zero login sessions; render nodes are 0666).
  A loaded-but-inactive leftover instance never triggers the DM stop.

Companion fixes from the same triage:
- ensure_box_gamescope_mode gains the attach-only rebuild-probe guard
  both managed paths already had (stale post-capture-loss detection
  restarted the box's unit), and no longer re-modes a box that drives a
  physical display — attach mirrors on-glass; re-mode is the
  headless-box model.
- Capture-loss rebuilds targeting gamescope get a 100 s budget: the
  40 s budget expired inside the first 45 s Steam-cold-start launch
  attempt, a guaranteed single-shot failure.
- A PUNKTFUNK_COMPOSITOR pin now WARNs once per capture loss when the
  live session no longer matches it (the pin disables
  session-following — the reporter's original stream-death trigger).
- A managed session that took nothing over (client gamescope pin beside
  a live desktop) is stopped on disconnect instead of being orphaned
  forever.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 11:49:24 +02:00
enricobuehlerandClaude Fable 5 9b241d9d7b fix(pf-win-display): anchor the kept sources at the desktop origin when isolating
ci / docs-site (push) Successful in 1m0s
ci / web (push) Successful in 1m4s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 34s
ci / bench (push) Successful in 6m36s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 46s
apple / screenshots (push) Successful in 6m35s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m51s
deb / build-publish (push) Successful in 12m42s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
deb / build-publish-host (push) Successful in 13m31s
android / android (push) Successful in 15m54s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7m18s
arch / build-publish (push) Successful in 16m19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9m34s
docker / deploy-docs (push) Successful in 29s
ci / rust (push) Successful in 20m38s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m34s
Round 3 of the field-reported exclusive-topology 0x57, with the reporter's
retest logs finally separating the variable: the SAME isolate call converged
rc=0 whenever a kept member already sat at (0,0), and failed 0x57 on every
shape — doomed-path-carried AND keep-only escalation — whenever the doomed
physical held the origin. A committable config must still contain a primary
(a source pinned exactly at (0,0)); deactivating the display that held it
while the kept virtual stays at its EXTEND offset supplies an origin-less
desktop, which Windows rejects wholesale. Translate the kept sources rigidly
so the top-left-most lands on the origin; sets already covering (0,0) are
untouched so plain re-commits stay byte-identical.

Also from the same logs:
- restore_displays_ccd now guarantees the desk is never left all-dark: the
  saved snapshot can be unappliable (0x64a — it pinned a virtual target
  incarnation removed before teardown) or apply cleanly yet re-light nothing
  (snapshotted while an earlier failed teardown had the physicals off). If no
  external physical is active after the apply while one is connected, fall
  back to the database EXTEND preset. Internal panels don't count — a closed
  lid must not be forced back on.
- the final isolate failure now names the surviving targets instead of
  asserting 'a non-virtual display stayed active' — the logs showed a sibling
  VIRTUAL display as the survivor (linger-expiry shrink), and the wording sent
  triage the wrong way.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 10:21:47 +02:00
enricobuehlerandClaude Opus 4.8 ebddffa4c8 chore(release): bump workspace version to 0.19.1
apple / swift (push) Successful in 1m30s
android-screenshots / screenshots (push) Successful in 3m1s
audit / cargo-audit (push) Successful in 2m14s
audit / bun-audit (push) Successful in 15s
ci / web (push) Successful in 1m8s
ci / docs-site (push) Successful in 1m11s
release / apple (push) Successful in 9m36s
android / android (push) Successful in 14m52s
ci / bench (push) Successful in 6m31s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 14m40s
apple / screenshots (push) Successful in 6m40s
ci / rust (push) Successful in 19m21s
deb / build-publish-host (push) Successful in 10m36s
deb / build-publish (push) Successful in 9m18s
linux-client-screenshots / screenshots (push) Successful in 7m40s
docker / deploy-docs (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
web-screenshots / screenshots (push) Successful in 2m59s
windows-host / package (push) Successful in 9m59s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m8s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m58s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m7s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m17s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m43s
flatpak / build-publish (push) Failing after 8m0s
An all-fixes patch on top of 0.19.0. The headline is session teardown — the
native stop flag made enforceable and bounded (ac3dc432) and GameStream ending
on the control peer's ENet disconnect (41fa25c4), so a session can no longer
outlive the client that started it on either plane.

Also: the Windows input-desktop pair (display writes b5fa878b + pen/touch
injection 24a24734 both surviving a UAC prompt), the Apple display-sleep guard
(bc5f6a38), the gamescope corner-parked cursor (4b2d2d1e), the KWin custom-mode
selection (3a33a694), the tvOS glob warnings (cc8cccbe), and the Linux
starter/env docs revamp (66172753).

Nothing moved on the wire (protocol 2) or at the C ABI (13), so no embedder
rebuild and 0.18/0.19 peers keep mixing freely.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 02:15:40 +02:00
enricobuehlerandClaude Fable 5 cc8cccbee6 fix(core/client): drop two redundant glob imports that warned on tvOS
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 35s
release / apple (push) Successful in 9m19s
ci / web (push) Successful in 1m12s
ci / docs-site (push) Successful in 1m21s
android / android (push) Successful in 12m19s
arch / build-publish (push) Successful in 12m19s
ci / bench (push) Successful in 6m36s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
apple / screenshots (push) Successful in 6m35s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
windows-host / package (push) Successful in 9m44s
deb / build-publish (push) Failing after 4m47s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m52s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m25s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m14s
deb / build-publish-host (push) Successful in 9m39s
flatpak / build-publish (push) Failing after 8m25s
ci / rust (push) Successful in 23m58s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m36s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m37s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m13s
docker / deploy-docs (push) Successful in 12s
`punktfunk-core` emitted `unused import: super::super::*` for
client/pump/handshake.rs and client/pump/input_task.rs — but only when
built for a tvOS target, where the cfg set leaves the glob contributing
nothing. Nobody saw them because CI never builds tvOS: that needs
BUILD_TVOS=1 and the Tier-3 build-std nightly path, and the Apple
workflow only runs swift build/test (macOS).

Both files also `use super::*`, i.e. the parent `pump` module, which
itself globs `super::*` (`client`) unconditionally — so every name stays
reachable by that path and the inner glob was pure duplication. Their
sibling datagram_task.rs already imports only `use super::*`, so this
also makes the module consistent. control_task.rs and data.rs keep
theirs: those genuinely resolve names through it and never warned.

Verified warning-free AND compiling on every slice the xcframework
ships — aarch64/x86_64-apple-darwin, aarch64-apple-ios,
aarch64-apple-ios-sim, aarch64-apple-tvos, aarch64-apple-tvos-sim,
x86_64-apple-tvos — plus Linux with and without the `quic` feature. A
full xcframework rebuild now reports 0 warnings (was 6: two per tvOS
slice), and swift build, swift test (170 tests), and xcodebuild for the
macOS, iOS and tvOS schemes all pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 02:11:49 +02:00
enricobuehlerandClaude Fable 5 ac3dc4323f fix(native/session): make the stop flag enforceable so a session can't outlive its client
Native sessions could survive long after the client was gone, in two
independent ways.

HOST: `stop` was only advisory. The one thing that ends a session is the
stream thread returning — every teardown (conn.close, the joins, and the
RAII drops of the session permit, admission entry and stream marker)
sits after that await, and nothing forced the issue. The encode loop
checks `stop` between iterations, so any unbounded call INSIDE one never
reaches the check, and one stuck syscall became a permanent zombie: it
held its semaphore slot (four of those and the host stops accepting
QUIC entirely), its admission entry (a later client gets "host busy"
forever), and not even the console's Stop button could clear it — that
button sets this same flag.

  * Bound the wait: once the session has been told to stop, the thread
    gets STREAM_STOP_GRACE (90 s, well past the 40 s capture-rebuild
    budget) to return, then teardown runs anyway. The thread is detached,
    not killed — Rust can't cancel a blocking thread — so it keeps its
    capturer/encoder until the stuck call returns, but the session's slot
    and admission entry come back and the host keeps serving. It logs at
    ERROR as the host wedge it is.
  * Bound the audio/input joins too — the last unbounded await in
    teardown.
  * Take the session permit AFTER the QUIC handshake instead of before
    `accept()`, so a host at its concurrency cap still accepts and the
    waiting client sees a live path instead of a silent dial timeout.
  * Bound the compositor helpers that caused the wedge in the first
    place: new pf-vdisplay `proc::{status_within, output_within}` kill a
    child that outlives its budget. `kscreen-doctor` is a Wayland client
    of the very compositor it configures, so against a wedged KWin it
    never returned; same for systemctl/dbus against a stuck session bus.

CLIENTS: the connection was never closed, so the host was right to keep
the session — it still had a live, keep-alive-answering peer.

  * Android: backgrounding did no teardown at all, and Android doesn't
    suspend the process, so the worker kept answering keep-alives until
    the OS reclaimed it (on a TV box, never). End the session on ON_STOP,
    via the existing onDispose path; a plain close, not a quit, so the
    host lingers the display for a fast return.
  * Apple: the .background arm was iOS-only AND gated on an opt-in that
    defaults off, so backgrounding did nothing — while the `audio`
    background mode kept the app (and its connection) alive indefinitely.
    Act unconditionally, and cover tvOS.
  * Core: `conn.close()` only queues the frame, and run_pump is the body
    of a block_on whose runtime is dropped the instant it returns, so the
    driver could never put it on the wire — a deliberate quit reached the
    host as silence (8 s idle timeout, no quit code, and the linger meant
    for an unwanted disconnect). Carry the endpoint out of the handshake
    and flush with wait_idle(), the same discipline the pairing and probe
    paths already use.

Linux check/clippy/tests green: 262 host, 71 pf-vdisplay (incl. new
bounded-process tests), 231 core.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 02:11:49 +02:00
enricobuehlerandClaude Fable 5 41fa25c440 fix(gamestream/session): end the session when the client disconnects or vanishes
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m6s
apple / swift (push) Successful in 1m24s
decky / build-publish (push) Successful in 31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 18s
ci / bench (push) Successful in 7m2s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m8s
deb / build-publish (push) Successful in 9m35s
arch / build-publish (push) Successful in 12m19s
deb / build-publish-host (push) Successful in 12m46s
android / android (push) Successful in 16m5s
windows-host / package (push) Successful in 16m34s
apple / screenshots (push) Successful in 6m28s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m11s
ci / rust (push) Successful in 27m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m19s
docker / deploy-docs (push) Successful in 27s
GameStream sessions outlived their client: the only complete teardowns
were the explicit ones (RTSP TEARDOWN, nvhttp /cancel, mgmt DELETE
/session), and the only automatic detector was a media-UDP send error —
which needs an ICMP port-unreachable, so a true vanish (Wi-Fi drop,
sleep, power off, crash) left video+audio encoding into the void
forever, and even a plain Moonlight quit (which sends neither TEARDOWN
nor /cancel) leaked the session. The stale state then cascaded: a
lingering launch 503-blocked a different client under
mode_conflict=reject, and streaming=true made a reconnect's PLAY take
its "stream already running" branch — no new threads, old threads still
aimed at the dead endpoint, the reconnect got no media.

ENet already detects all of this — the control peer's reliable-ping
timeout (or clean disconnect) fires Event::Disconnect within ~5-30 s —
but the handler only reset input state. Wire the real teardown into it:

* AppState::end_session — THE compat-plane session teardown: stops both
  media threads (their flags), clears launch + negotiated stream config;
  idempotent. /cancel and mgmt stop_session now share it.
* control.rs Disconnect → end_session. Gated on the TRACKED session
  peer, and Connect only tracks a peer from the /launch owner's IP (the
  same source-IP bind the RTSP/media plane uses), so an unauthenticated
  LAN peer connect+disconnect can't end a live session, and a fast
  reconnect's stale-peer timeout can't kill its successor.
* Client-unreachable UDP send errors now end the whole session via an
  OnSessionLost callback (built at PLAY) instead of stopping only the
  plane that noticed — audio no longer keeps streaming after video
  detects the dead client, and vice versa.

Linux check/clippy/tests green (53 gamestream tests incl. the new
end_session regression test).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 02:10:23 +02:00
enricobuehlerandClaude Opus 4.8 3a33a69401 fix(kwin): select the custom mode KWin actually built, not the one we asked for
apple / swift (push) Successful in 1m19s
apple / screenshots (push) Successful in 6m41s
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m3s
ci / bench (push) Successful in 5m33s
android / android (push) Successful in 11m38s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
deb / build-publish (push) Successful in 9m6s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m43s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
deb / build-publish-host (push) Successful in 9m50s
arch / build-publish (push) Successful in 21m2s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m27s
docker / deploy-docs (push) Failing after 15s
ci / rust (push) Successful in 28m29s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m21s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m49s
A KDE host streaming to a phone-shaped client fell back to 60 Hz even though the
120 Hz mode was sitting right there in KDE's display list, one manual click away.

KWin generates every custom mode's timing with libxcvt, whose first step is
`hdisplay_rnd = hdisplay - (hdisplay % 8)`. A 2868x1320@120 request (iPhone 16
Pro Max) therefore becomes 2864x1320@119.92 — 4 px narrower, fractional refresh.
We then asked kscreen-doctor for `mode.2868x1320@120`; its `findMode` matches a
mode's id or its own `WxH@qRound(Hz)` name, so that string matched nothing, the
select silently no-op'd, the output stayed on its sacrificial birth mode, and
`size_applied` came back false → "KWin rejected the custom mode" → 60 Hz.
Widths like 1920/2560/3840 are all multiples of 8, which is why only clients with
phone-shaped panels ever hit this.

Resolve the mode out of the output's OWN list instead and address it by kscreen
mode id: exact height, width at most one cell narrower than asked, refresh within
1 Hz (which excludes the native 60 Hz entry). `set_custom_refresh` now returns the
whole achieved mode, and `create` reports that as the output's `preferred_mode` —
so the capturer's renegotiation gate waits for the size KWin will actually deliver
and the encoder opens against it, rather than starving on the requested one.

Also skip `addCustomMode` when a usable mode is already installed: kscreen-doctor
APPENDS to the list and KWin persists it per output name, so the old code grew the
user's display list by one entry per connect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:55:36 +02:00
enricobuehlerandClaude Opus 4.8 24a24734eb fix(inject/windows): follow the input desktop for pen + touch injection
apple / swift (push) Successful in 1m16s
apple / screenshots (push) Successful in 6m43s
android / android (push) Failing after 7m44s
ci / web (push) Successful in 2m3s
ci / docs-site (push) Successful in 1m14s
arch / build-publish (push) Successful in 12m50s
ci / bench (push) Successful in 6m30s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 16s
deb / build-publish (push) Successful in 9m13s
ci / rust (push) Successful in 19m44s
deb / build-publish-host (push) Successful in 9m30s
docker / deploy-docs (push) Successful in 26s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 25m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 25m10s
Same field report as the display-write fix, other half of the symptom: with a
UAC consent prompt up — one the user could SEE in the stream, because capture
already renders the secure desktop (326d6e17) — pen and touch did nothing,
while mouse and keyboard kept working.

The split was exactly which paths knew the input desktop can move. sendinput.rs
reattaches, so mouse/keyboard reached the prompt. pointer_windows.rs had no
desktop handling at all, so every InjectSyntheticPointerInput came back
ERROR_ACCESS_DENIED:

    22:20:31  virtual pen created (PT_PEN)
    22:20:31  pen inject failed    error=Zugriff verweigert (0x80070005)
    22:20:42  touch inject failed  error=Zugriff verweigert (0x80070005) contacts=1

Measured on glass before fixing, to find out what it actually takes:

    device on Default, thread on Default  -> 0x80070005   (the field failure)
    device on Default, thread on INPUT    -> OK
    device on INPUT,   thread on INPUT    -> OK

The middle row is load-bearing: the synthetic pointer device is NOT
desktop-affine, so rebinding the thread suffices and the device is never
destroyed and recreated across a desktop switch — which would have dropped
in-flight contacts and the pen's in-range state mid-stroke.

Injection now retries once bound to the input desktop. The binding is scoped,
not persistent like sendinput's: inject_pen/inject_touch_frame run on TWO
threads (the caller's apply_batch and the refresh threads), and the batch caller
is a shared task thread that must not be left parked on a Winlogon desktop that
disappears when the prompt is dismissed.

The first-failure WARN now carries the rejected sample (flags, pen flags/mask,
pressure, rotation, tilt, position). A 0x80070057 INVALID_PARAMETER was seen
once BEFORE any prompt existed and is still unexplained; the ranges all look
sound on inspection (coordinates clamped to the virtual screen, roll/azimuth
u16 so the modulo cannot go negative, tilt bounded 0..=90), so catching the
actual offending sample is the way to find it.

Verified on glass: with a consent prompt up, pen and touch now reach it — zero
inject failures where every prior session failed immediately. A UAC prompt can
now be dismissed from an iPad with the Pencil.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:36:51 +02:00
enricobuehlerandClaude Opus 4.8 b5fa878bc6 fix(pf-win-display): follow the input desktop so a UAC prompt can't refuse display writes
Field-reported 2026-07-23: a consent prompt left up on an unattended host made
the box unreachable. Every connect black-screened and failed after ~60 s, and
the host had to be reached over SSH to clear the prompt.

Windows refuses ChangeDisplaySettingsEx/SetDisplayConfig issued from a thread
that is not on the desktop currently receiving input. While UAC (or the lock /
logon screen) is up that desktop is Winlogon, and the host — which the service
launches explicitly onto WinSta0\Default — got DISP_CHANGE_FAILED /
ERROR_ACCESS_DENIED for every write. A session starting in that window could
never set its virtual display's mode, so the capturer sized its ring to a stale
mode, every composed frame was dropped for a size mismatch, and bring-up burned
8 retries before giving up.

Measured on glass (RTX box, SYSTEM host in console session 2, real consent
prompt up, virtual display active):

    INPUT desktop = Winlogon
    UNBOUND  CDS_TEST      -> -1 (DISP_CHANGE_FAILED)
    UNBOUND  SDC_VALIDATE  -> 0x5 (ERROR_ACCESS_DENIED)
    BOUND    CDS_TEST      ->  0 (DISP_CHANGE_SUCCESSFUL)
    BOUND    SDC_VALIDATE  -> 0x0 (ERROR_SUCCESS)

New input_desktop.rs mirrors pf-inject's sendinput.rs retry model: issue the
write, and only when it fails the way a wrong-desktop write fails, rebind this
thread to the current input desktop and retry once. A working write is never
touched, so the normal path is unchanged. The retry predicate stays narrow
(ERROR_ACCESS_DENIED only) so the unrelated 0x57 exclusive-mode topology bug is
not re-issued and mis-attributed. Unlike sendinput's dedicated injector thread,
the binding here is SCOPED — a shared display-write thread left on a Winlogon
desktop that is later destroyed would fail every subsequent write, which is the
wedge this removes.

Applied to all eight SetDisplayConfig sites and both ChangeDisplaySettingsExW
calls in set_active_mode. The isolate site now decides its supplied config once,
outside the write, so a retry re-issues the identical config rather than logging
its escalation twice.

A save is logged, because a silent one is indistinguishable in a field log from
a write that never needed saving — which made the first on-glass verification of
this change inconclusive.

Both ACCESS_DENIED diagnostics now ask which cause applies instead of naming
only the disconnected-RDP one: that phrasing sent this investigation chasing a
phantom RDP session on a host that was already service-launched in the console
session, while a consent prompt was the actual cause.

Verified on glass: connect with a consent prompt up now reaches first frame in
3.0 s (was a black screen then "Connection failed" after ~60 s), with six
"retried bound to it and it applied desktop=Winlogon" lines and rc=0x0
throughout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:36:51 +02:00
enricobuehler 4b2d2d1e14 fix(gamescope/cursor): follow gamescope's own cursor verdict — the corner-parked pointer
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 58s
apple / swift (push) Successful in 1m23s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
ci / bench (push) Successful in 6m36s
apple / screenshots (push) Successful in 6m34s
deb / build-publish (push) Successful in 9m41s
deb / build-publish-host (push) Successful in 10m13s
arch / build-publish (push) Successful in 12m43s
android / android (push) Successful in 16m41s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m41s
docker / deploy-docs (push) Successful in 25s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m4s
ci / rust (push) Successful in 29m44s
gamescope hides its pointer by WARPING the X pointer to the root's bottom-right corner pixel;
it does not swap in a transparent X cursor, so XFixesGetCursorImage keeps handing back the last
opaque arrow. The XFixes source picked its display by "whichever pointer moved", and a parked
pointer never moves again — so it froze on the parked Xwayland and composited that arrow at
(w-1, h-1): a sliver of cursor welded to the corner of the stream for the rest of the session
while the real pointer went undrawn. Reported on-glass in every game (a game grabs the pointer,
so the hide is permanent): "part of cursor shows up on bottom right ... isn't where it really is".

Follow GAMESCOPE_CURSOR_VISIBLE_FEEDBACK instead — gamescope publishes it on every nested
Xwayland root: 1 on the server whose pointer it is drawing, 0 on the others and 0 on all of them
once the pointer is hidden. It answers both questions correctly for a STATIC pointer, which
motion cannot: which Xwayland owns it, and whether to draw it at all. Read at connect, re-read
on its root PropertyNotify (event drain now discriminates CursorNotify vs PropertyNotify) with a
250 ms resync as the self-heal. A pointer gamescope draws nowhere is published visible:false,
not dropped — the encode loop overwrites the frame's overlay from this slot and strips invisible
ones, so a None would leave the last visible overlay standing on repeat frames. Honouring the
atom also gives the stream gamescope's own idle auto-hide, which this source never had.

The pointer-motion heuristic stays as the fallback for a gamescope that publishes no verdict
(logged at session start via cursor_feedback=false), so nothing regresses to a cursorless stream.

Measured on a live 1920x1080 Steam Gaming Mode session (RTX 5070 Ti, gamescope c31743d+,
--xwayland-count 2 --hide-cursor-delay 3000): idle 3 s => pointer (1919,1079) + feedback 0;
real evdev motion => pointer live + feedback 1. End-to-end against that session the source now
publishes visible=false while parked and tracks the live pointer within one 250 ms sample of
motion. cargo clippy -p pf-capture --lib -D warnings and the 5 new pick_active tests are green
at cargo 1.96.1 (CI parity); cargo fmt --all --check clean.
2026-07-24 00:31:55 +02:00
enricobuehlerandClaude Opus 4.8 bc5f6a3881 fix(apple/session): keep the display awake for the length of a session
ci / web (push) Successful in 50s
apple / swift (push) Successful in 1m19s
ci / docs-site (push) Successful in 1m58s
decky / build-publish (push) Successful in 33s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 54s
deb / build-publish (push) Failing after 4m47s
ci / bench (push) Successful in 6m1s
docker / deploy-docs (push) Successful in 33s
release / apple (push) Successful in 10m4s
android / android (push) Successful in 13m5s
deb / build-publish-host (push) Successful in 12m52s
arch / build-publish (push) Successful in 16m56s
apple / screenshots (push) Successful in 6m33s
ci / rust (push) Successful in 19m36s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m27s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m15s
A stream is not user activity to the OS, and controller input does not feed
the HID idle timer on any Apple platform — so a gamepad-only session reliably
idles the panel out from under the user mid-play. A keyboard/mouse capture
session masks the bug because those events are real local HID. The Android
client has held FLAG_KEEP_SCREEN_ON while streaming all along; the Apple
clients held nothing.

DisplaySleepGuard is acquired in beginStreaming and released at the top of
disconnect, so it is scoped to the session — every teardown path (user quit,
sessionEnded, the backgrounded keep-alive timeout) funnels through disconnect.

  iOS/iPadOS/tvOS: UIApplication.isIdleTimerDisabled. App-wide and ignored
  while backgrounded, which is what the audio-only keep-alive wants.

  macOS: ProcessInfo.beginActivity(.userInitiated, .idleDisplaySleepDisabled),
  the Foundation wrapper over IOKit power assertions. That defers display
  sleep but NOT the screen saver, which runs off the HID idle timer
  independently — so a 30 s IOPMAssertionDeclareUserActivity heartbeat runs
  alongside it. Intended side effect: an idle-lock that follows the screen
  saver is deferred for the session too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 00:00:55 +02:00
enricobuehlerandClaude Fable 5 6617275387 docs(env): stop teaching the compositor pin + uid-1000 anchors in starters
ci / docs-site (push) Successful in 54s
ci / web (push) Successful in 59s
apple / swift (push) Successful in 1m21s
decky / build-publish (push) Successful in 28s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 25s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
ci / bench (push) Successful in 6m23s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m43s
apple / screenshots (push) Successful in 6m18s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m9s
arch / build-publish (push) Successful in 13m44s
deb / build-publish (push) Successful in 13m48s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7m39s
deb / build-publish-host (push) Successful in 13m11s
docker / deploy-docs (push) Successful in 32s
android / android (push) Successful in 16m23s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m15s
ci / rust (push) Successful in 24m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m55s
Field triage (Nobara, Discord): the kde.md starter host.env told desktop
users to set PUNKTFUNK_COMPOSITOR=kwin, which PINS the backend — detect()
short-circuits and the capture-loss rebuild never re-detects — so a
mid-stream switch to Game Mode killed the stream instead of following it.
A follow-up hardcoded XDG_RUNTIME_DIR=/run/user/1000 anchor broke PipeWire
for any non-1000 uid (pw audio connect: Creation failed).

Revamp across every starter/example/reference:
- Desktop starters (kde/gnome/hyprland/sway) shrink to
  PUNKTFUNK_VIDEO_SOURCE=virtual + an explicit warning that pinning
  disables session-following; forcing a backend is CI/appliance-only.
- host.env.example: rewritten around auto-detection; anchors demoted to a
  commented ssh/cron-only block with the uid trap spelled out; the
  gamescope ATTACH/MANAGED knobs documented (previously missing);
  case-sensitivity called out.
- packaging/bazzite/host.env + README: drop the uid-1000 anchors (a
  systemctl --user service inherits/derives them); README's stale
  PUNKTFUNK_COMPOSITOR=gamescope-era template synced to the real one.
- packaging/kde/host.env: loud APPLIANCE-ONLY header (it pins on purpose).
- configuration.md: session-anchors section inverted to "leave unset",
  compositor row states the pin consequence, case-sensitivity note.
- troubleshooting.md: new "session fails right after editing host.env"
  section (case, wrong-uid anchors, stale pin, restart-to-apply).
- gamescope.md/bazzite.md: attach/managed descriptions match current
  behavior (managed is the infra-detected default; attach re-modes a
  box-owned session to the client's resolution).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 22:42:59 +02:00
enricobuehlerandClaude Opus 4.8 bda015b101 docs(release): rewrite v0.19.0 notes for end users + set end-user notes voice
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m8s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 36s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 15s
apple / screenshots (push) Successful in 6m29s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m1s
ci / bench (push) Successful in 8m1s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 18s
deb / build-publish-host (push) Successful in 10m19s
deb / build-publish (push) Successful in 12m13s
android / android (push) Successful in 13m20s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m26s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 16m53s
ci / rust (push) Successful in 25m37s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m52s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m8s
Release notes were written for the people who build Punktfunk, not the people who use it —
dense with protocol/ABI/type names that confused even technical readers. Rewrite v0.19.0 in a
benefit-first, plain-language voice (New/Improved/Fixed) with all internal terms removed from the
body and every protocol/ABI/embedder detail moved to a single bottom "Under the hood" section.
docs/releases/README.md now codifies this voice as the format spec for all future notes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:15:08 +02:00
enricobuehlerandClaude Opus 4.8 ec219763a6 fix(ci/release): make _release_notes_path POSIX-sh safe (deb/decky attach runs under dash)
ci / web (push) Successful in 55s
apple / swift (push) Successful in 1m26s
ci / docs-site (push) Successful in 56s
android-screenshots / screenshots (push) Successful in 3m20s
ci / bench (push) Successful in 6m47s
release / apple (push) Successful in 10m49s
android / android (push) Successful in 15m8s
decky / build-publish (push) Successful in 26s
apple / screenshots (push) Successful in 6m40s
deb / build-publish (push) Successful in 9m36s
deb / build-publish-host (push) Successful in 9m39s
flatpak / build-publish (push) Successful in 6m33s
arch / build-publish (push) Successful in 15m57s
ci / rust (push) Successful in 26m4s
docker / deploy-docs (push) Successful in 16s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
web-screenshots / screenshots (push) Successful in 3m10s
windows-host / package (push) Successful in 10m33s
linux-client-screenshots / screenshots (push) Successful in 7m51s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m12s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m3s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m18s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m57s
The v0.19.0 deb + decky release-attach steps source gitea-release.sh under `sh` (dash) and
died with "Bad substitution" at _release_notes_path's `${BASH_SOURCE[0]:-$0}` — a bash array
subscript dash rejects. So those legs never attached their assets to the release (the bash
legs — apple, android, rpm — were fine, which is why the body still seeded and the DMG/.ipa
attached). CI always sources this from the repo root, so resolve the notes as
docs/releases/<tag>.md relative to CWD and drop $BASH_SOURCE entirely. Verified under dash.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 19:03:01 +02:00
enricobuehlerandClaude Opus 4.8 0dce1cd116 chore(release): bump workspace version to 0.19.0
audit / bun-audit (push) Successful in 11s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 54s
apple / swift (push) Successful in 1m20s
audit / cargo-audit (push) Successful in 2m13s
ci / bench (push) Successful in 6m52s
android-screenshots / screenshots (push) Successful in 3m58s
release / apple (push) Successful in 11m11s
deb / build-publish (push) Failing after 8m56s
decky / build-publish (push) Failing after 24s
deb / build-publish-host (push) Failing after 9m15s
android / android (push) Successful in 13m6s
ci / rust (push) Successful in 25m44s
arch / build-publish (push) Successful in 13m37s
docker / deploy-docs (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
apple / screenshots (push) Successful in 6m32s
web-screenshots / screenshots (push) Successful in 3m2s
windows / build (x86_64-pc-windows-msvc) (push) Failing after 2m43s
flatpak / build-publish (push) Failing after 8m20s
linux-client-screenshots / screenshots (push) Successful in 7m46s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m45s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m4s
windows-host / package (push) Successful in 15m54s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m26s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m49s
Headline: the whole pen/tablet input stack (P0–P5) — a pressure-sensitive stylus plane
across core wire, Linux, GameStream/Moonlight, Windows, iPad Pencil and Android. Plus
touch-injection fixes, cursor-DPI scaling, three Windows-host display/cursor fixes, a
gamepad same-PID conflict fix, and the Android mouse/keyboard (TV) regressions.

Release notes authored in-repo at docs/releases/v0.19.0.md (first release on the new flow;
CI seeds the release body from it). cargo update --workspace syncs the lock (versions only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:26:49 +02:00
enricobuehlerandClaude Opus 4.8 c3c0115a82 fix(core/abi): bump ABI_VERSION to 13 for punktfunk_connection_send_pen
The pen P0 work added a new embeddable C entry point (punktfunk_connection_send_pen)
but left ABI_VERSION at 12 — the value 0.18.0 shipped. Per the project's convention
(0.18.0 bumped to 12 for the additive cursor channel), additive C-surface growth bumps
the ABI so embedders can gate on the new symbol. Regenerated include/punktfunk_core.h
via cbindgen (cargo build -p punktfunk-core); WIRE_VERSION is unchanged (pen is
capability-gated on HOST_CAP_PEN).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:23:38 +02:00
enricobuehlerandClaude Opus 4.8 00c8c12a0f feat(ci/apple): export the iOS .ipa to the release + run artifacts
The iOS leg archives with development signing and re-signs for App Store distribution at
export time, but the existing export uses destination=upload (straight to TestFlight) and
leaves no .ipa on disk. Add an export step that re-exports the SAME archive with
destination=export to produce an App Store distribution-signed .ipa, then:
  * attach it as the `punktfunk-ios-ipa` run artifact on every build (upload-artifact@v3,
    since Gitea's backend rejects @v4), and
  * attach it to the unified Gitea release on vX.Y.Z tags (alongside the DMG), via the
    shared ensure_release/upsert_asset helpers.

Same gate as the archive; warn+skip (never fails the best-effort iOS leg) if the archive is
absent. Note: an App Store-signed .ipa installs only via TestFlight/App Store, not by direct
sideload — it is a release/archival artifact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 18:18:45 +02:00
enricobuehlerandClaude Fable 5 988b5742ec fix(gamestream): tolerate NaN pressureOrDistance — VoidLink's finger touches were dropped whole
apple / swift (push) Successful in 1m18s
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m6s
apple / screenshots (push) Successful in 6m11s
ci / bench (push) Successful in 5m34s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
decky / build-publish (push) Successful in 27s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
windows-host / package (push) Successful in 16m57s
android / android (push) Successful in 15m58s
docker / deploy-docs (push) Successful in 24s
arch / build-publish (push) Successful in 17m3s
deb / build-publish (push) Successful in 9m26s
deb / build-publish-host (push) Successful in 10m5s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m39s
ci / rust (push) Successful in 25m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m31s
The hex dumps from the on-glass session show textbook SS_TOUCH packets with
pressureOrDistance = 0x7fc00000 (NaN): VoidLink encodes 'unknown finger
pressure' as NaN (iPad fingers have no force sensor), and the anti-forgery
finiteness gate rejected the entire packet — silently disabling the touch
plane while pen (real Pencil force, always finite) worked. Coordinates keep
the strict gate (they feed injector scaling); pressureOrDistance now
sanitizes non-finite to 0.0, the spec's own 'unknown', which the pen path
already maps to full-scale ink on contact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 f938174d86 debug(gamestream): hex-dump the first few undecodable pointer packets
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 40611cd54d debug(gamestream): surface the pointer plane's silent failure modes
An on-glass 'touch does nothing' currently leaves NO trace: log the first
SS_TOUCH per session at info (proves the client sends + we decode), and warn
when a packet carrying a pointer magic fails the body parse (layout mismatch)
instead of vanishing into the unknown-magic drop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:48:17 +02:00
enricobuehlerandClaude Fable 5 98b97d7d76 fix(android/input): stop mouse back/forward leaking to nav + IME popping on hardware typing
apple / swift (push) Successful in 1m25s
apple / screenshots (push) Successful in 7m4s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m14s
android / android (push) Successful in 11m54s
arch / build-publish (push) Successful in 11m58s
ci / bench (push) Successful in 5m36s
decky / build-publish (push) Successful in 24s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
deb / build-publish (push) Successful in 11m52s
docker / deploy-docs (push) Successful in 11s
deb / build-publish-host (push) Successful in 10m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m15s
ci / rust (push) Successful in 26m1s
Two Android TV M&K regressions from the c90343c2 overhaul, both user-reported:

1. Mouse back/forward buttons doubled as Android navigation while streaming.
   The BUTTON_BACK/FORWARD motion edges already go over the wire as X1/X2, but
   Android's input reader ALSO synthesizes KEYCODE_BACK/FORWARD key events from
   those buttons unconditionally (stamped SOURCE_MOUSE, no FLAG_FALLBACK) — and
   the guard only swallowed the FLAG_FALLBACK variant, so the duplicate reached
   the BackHandler and yanked the user out of the stream. Swallow mouse-sourced
   BACK/FORWARD too; a remote/keyboard BACK is never mouse-sourced and still
   exits.

2. The soft keyboard popped up on physical-keyboard typing. KeyCaptureView (the
   always-focused IME/capture anchor) unconditionally claimed to be a text
   editor with a live editable InputConnection, so the TV IME counted input as
   active and showed its UI the moment hardware keys arrived. Gate the editor
   identity on imeShown (the user actually summoning the keyboard via gesture /
   remote toggle), with restartInput on both edges so the gate is re-read, and
   an onKeyPreIme BACK hook so an IME self-dismissal can't leave the gate
   stale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 17:07:18 +02:00
enricobuehlerandClaude Fable 5 51792536d1 fix(pf-win-display): escalate CCD isolate to a keep-only supplied config on 0x57
apple / swift (push) Successful in 1m30s
ci / web (push) Successful in 1m8s
apple / screenshots (push) Successful in 6m59s
ci / docs-site (push) Successful in 1m6s
android / android (push) Successful in 14m35s
arch / build-publish (push) Successful in 14m20s
decky / build-publish (push) Successful in 21s
ci / bench (push) Successful in 6m4s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 18s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 18s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
deb / build-publish (push) Successful in 9m41s
ci / rust (push) Successful in 20m14s
deb / build-publish-host (push) Successful in 11m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m42s
docker / deploy-docs (push) Successful in 12s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m2s
Field bug (Steam Deck client, exclusive topology): every isolate retry failed
SetDisplayConfig 0x57 with the physical panel left lit, while iPhone sessions
worked — their virtual monitor's connected-set already had a sole-display
topology persisted, so Windows deactivated the physical on hotplug and the
isolate was a no-op re-commit (deactivated 0). The Deck's monitor identity had
no such entry, forcing the live-deactivate path: supplying the doomed path
(inactive, modes unpinned) plus its orphaned mode entries is rejected 0x57 by
some driver/OS validation combos, so the retry loop could never converge — the
prior mode-index unpin (5e088af7) was necessary but not sufficient on this box.

Attempt 1 keeps the shipped shape; once a survivor is seen, later attempts
supply ONLY the keep paths with the mode table rebuilt to just their referenced
entries (indexes remapped, clone-shared entries deduped). The same host applies
exactly this keep-only shape rc=0 whenever the DB has pre-isolated the virtual
display, so the escalated shape is field-validated on the failing machine. The
final attempt also drops SDC_FORCE_MODE_ENUMERATION — a real path removal
drives COMMIT_MODES on its own.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:50:44 +02:00
enricobuehlerandClaude Fable 5 bf2c4456bb fix(inject/windows): compact wire touch ids into slots — Moonlight native touch injected nothing
apple / swift (push) Successful in 1m28s
ci / web (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m12s
release / apple (push) Successful in 9m24s
ci / bench (push) Successful in 6m28s
android / android (push) Successful in 11m59s
decky / build-publish (push) Successful in 22s
arch / build-publish (push) Successful in 12m30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 15s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
apple / screenshots (push) Successful in 6m53s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 19s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 17s
deb / build-publish (push) Successful in 9m36s
deb / build-publish-host (push) Successful in 11m24s
ci / rust (push) Successful in 25m11s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m53s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m59s
docker / deploy-docs (push) Successful in 25s
On-glass report (iPad Moonlight vs the Windows host): pen inked, native touch
did nothing. Cause: POINTER_INFO.pointerId was the client's raw wire id —
Moonlight sends arbitrary large pointerIds, and synthetic-pointer injection
rejects them (Apollo compacts ids into dense slots for exactly this reason);
the failure was logged at trace, i.e. invisibly. Wire ids now map to the
lowest free slot for the contact's lifetime, and the FIRST injection failure
of a device logs at WARN (pen too) so an inert input plane is never silent
again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Fable 5 c0f792ee8d feat(android): pen P5 — stylus capture onto the pen plane
The Android leg of design/pen-tablet-input.md §7: against a HOST_CAP_PEN host,
stylus/eraser pointers split out of BOTH touch models (passthrough + gesture)
into StylusStream — state-full samples with pressure, AXIS_TILT, azimuth from
AXIS_ORIENTATION (Android's 0 = away-from-user IS wire north), AXIS_DISTANCE
hover, both stylus barrel buttons, the eraser tool, and historical (coalesced)
samples batched oldest-first. Kotlin heartbeats ≤100ms per the wire contract.
JNI: nativeHostSupportsPen + nativeSendPen (flat 10-float stride, sentinels
<0). No barrel-roll axis exists on Android — roll stays unknown here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Fable 5 c8aa3ed48b feat(apple/ios): pen P4 — Apple Pencil (Pro) capture onto the stylus plane
The flagship leg of design/pen-tablet-input.md §7: against a HOST_CAP_PEN host
the Pencil splits out of the finger path (independent of touch-input mode) into
PencilStream — state-full samples with force→pressure, altitude→polar tilt,
azimuth (+90° to the wire's north-clockwise convention), Pencil Pro rollAngle→
barrel roll (17.5+), coalesced touches batched ≤8/send for full 240Hz fidelity,
and Pencil hover (zOffset>0 distinguishes it from trackpad hover) as in-range
samples with distance. Squeeze holds barrel 1, double-tap clicks barrel 2 (the
Pencil has no hardware buttons/eraser — these are how host apps get those
affordances). Implements the ≤100ms heartbeat wire contract (80ms timer) so a
stationary held stroke survives the host's 200ms dead-client failsafe. Toward a
pen-less host nothing changes (Pencil stays a finger).

Also fixes hostSupportsCursor testing the STALE 0x04 bit — HOST_CAP_CURSOR
moved to 0x08 when TEXT_INPUT claimed 0x04; the old test would mistake a
text-input host (e.g. Windows) for a cursor grant and double-render the pointer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:48:57 +02:00
enricobuehlerandClaude Opus 4.8 15d09a8c10 feat(ci/release): author release notes in-repo + announce stable releases to Discord
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m25s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
ci / bench (push) Successful in 6m55s
apple / screenshots (push) Successful in 6m47s
deb / build-publish (push) Successful in 11m23s
android / android (push) Successful in 12m36s
arch / build-publish (push) Successful in 12m46s
docker / deploy-docs (push) Successful in 23s
deb / build-publish-host (push) Successful in 12m32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m9s
ci / rust (push) Successful in 26m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m30s
Release notes now live in the repo at docs/releases/vX.Y.Z.md, authored during the
version bump (before the tag). ensure_release (gitea-release.sh + .ps1 twin) seeds the
Gitea release body from that file in the create POST, so the release is born WITH its
notes instead of being created empty and PATCHed afterward. canary/rc have no file ->
empty body, unchanged.

New manual `announce` workflow (workflow_dispatch, tag input) re-syncs the notes file
over the live release and posts a violet embed (title, notes lead-in, release-page link)
to the Discord #releases channel via the DISCORD_RELEASE_WEBHOOK secret. Stable-only:
a -rc tag is refused unless allow_prerelease=true. Pressing "go" is the quality gate, so
a half-built or failed release is never announced.

docs/releases/README.md documents the ritual; TEMPLATE.md is the skeleton.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 16:43:18 +02:00
enricobuehlerandClaude Opus 4.8 6cbc3eec8b fix(sdl/cursor): scale the forwarded pointer to the video fit
apple / swift (push) Successful in 1m30s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m3s
android / android (push) Canceled after 1m27s
apple / screenshots (push) Canceled after 1m39s
arch / build-publish (push) Canceled after 1m29s
ci / rust (push) Canceled after 1m29s
ci / bench (push) Canceled after 1m17s
deb / build-publish (push) Canceled after 1m15s
deb / build-publish-host (push) Canceled after 27s
decky / build-publish (push) Canceled after 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 15s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 1m18s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 8s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 18s
flatpak / build-publish (push) Canceled after 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / deploy-docs (push) Canceled after 0s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Canceled after 17s
Parity with the Apple client (35c2c145): the SDL/Linux presenter built the
host-forwarded cursor as an OS cursor at its native bitmap size, so the
on-screen pointer tracked the HOST's display scaling — a 4K/high-DPI virtual
display forwards a 96 px bitmap that then rendered huge on the client.

SDL cursors are fixed-size from their surface (no draw-time scaling), so
cache the shapes RAW and resample each by the live video-fit factor
(min(window_px / mode) — the same aspect-fit `finger_to_content` uses) at
install time, rebuilding when the serial OR the fit changes (a resize). The
resample area-averages on premultiplied alpha so a transparent pixel's
colour can't bleed into the fringe. The pointer now stays sized to the
streamed desktop at any host scaling.

Not yet eyeballed on a Deck/Linux box — SDL's HiDPI cursor sizing is
version-dependent; validate on glass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 16:28:34 +02:00
enricobuehlerandClaude Fable 5 45c9799aa6 feat(inject/windows): pen P3 — PT_PEN + PT_TOUCH synthetic pointer injection
apple / swift (push) Successful in 1m26s
ci / web (push) Successful in 59s
ci / docs-site (push) Successful in 54s
release / apple (push) Successful in 9m40s
arch / build-publish (push) Successful in 12m4s
ci / bench (push) Successful in 7m5s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 14s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 22s
apple / screenshots (push) Successful in 6m50s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
android / android (push) Successful in 14m47s
ci / rust (push) Canceled after 13m52s
deb / build-publish (push) Canceled after 7m26s
deb / build-publish-host (push) Canceled after 5m38s
flatpak / build-publish (push) Canceled after 1m55s
docker / deploy-docs (push) Canceled after 0s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 1m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 1m59s
windows-host / package (push) Canceled after 8m32s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Canceled after 0s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Canceled after 0s
windows / build (aarch64-pc-windows-msvc) (push) Canceled after 0s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 0s
The Windows leg of design/pen-tablet-input.md §6, following Apollo's recipe:
a per-session PT_PEN device (pressure rescaled 0..1024, polar tilt→tiltX/Y,
barrel roll on rotation — Windows Ink renders Pencil Pro roll natively, barrel
button, eraser via INVERTED/ERASER flags, hover) with a 40ms refresh thread
against the ~100ms synthetic-pointer staleness auto-lift, plus a PT_TOUCH
device closing the historical SendInput wire-touch no-op (full active-contact
frames, per-id DOWN/UP edges, self-healing lost-DOWN synthesis). pen_supported
now probes PT_PEN creation on Windows (1809+), which lights up HOST_CAP_PEN
and the GameStream featureFlags there — Moonlight iPad + Pencil can ink on a
Windows host. Frame grouping mirrors the Linux uinput backend.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 b192825869 feat(gamestream): pen P2 — SS_PEN/SS_TOUCH ingest, featureFlags advertised on capable hosts
The Moonlight leg of design/pen-tablet-input.md §4: DESCRIBE now advertises
SS_FF_PEN_TOUCH_EVENTS wherever pen injection exists (Linux+uinput, same gate
as HOST_CAP_PEN — elsewhere the flag stays 0 and Moonlight keeps client-side
mouse emulation exactly as today). SS_PEN packets merge over last-known state
into state-full PenSamples (BUTTON_ONLY per spec carries no position; unknown
contact pressure 0.0 inks at full scale; UP keeps proximity only for clients
that demonstrated hover) and drive the same PenTracker→VirtualPen chain as the
native plane — Moonlight iOS + Apple Pencil now exercises the full injection
path end to end. SS_TOUCH forwards as wire touch on a synthetic 65535² surface
with CANCEL_ALL replayed per tracked contact. No stroke timeout on this plane:
ENet is ordered/reliable and Moonlight doesn't heartbeat a stationary pen.
Packet layouts pinned against moonlight-common-c Input.h/Limelight.h upstream.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 aca5aa9993 feat(host+inject): pen P1 — per-session uinput virtual tablet, HOST_CAP_PEN live
The Linux injection leg of design/pen-tablet-input.md: 0xCC/0x05 pen batches now
route through the per-session PenTracker into a lazily-created 'Punktfunk Pen'
uinput tablet (BTN_TOOL_PEN/RUBBER, pressure, tilt-from-polar, ABS_Z barrel
roll, hover distance, INPUT_PROP_DIRECT) — compositors pick it up via libinput
and hand apps zwp_tablet_v2 with full fidelity. The host now advertises
HOST_CAP_PEN when /dev/uinput is accessible (PUNKTFUNK_PEN=0 kill-switch);
transitions group into SYN frames so proximity-enter carries its position.
Stroke failsafe: clients heartbeat ≤100ms while in range (documented wire
contract — capture APIs are silent for a stationary pen); 200ms of silence
force-releases. 'punktfunk-host pen-test' draws a pressure-ramped sine stroke
through the real tracker→uinput chain, no client needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:18:31 +02:00
enricobuehlerandClaude Fable 5 248e1cbf08 fix(pf-vdisplay/driver): report cursor_excluded adapter-wide — the declare's exclusion is not per-target
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m20s
windows-drivers / probe-and-proto (push) Successful in 1m2s
decky / build-publish (push) Successful in 28s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 16s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
windows-drivers / driver-build (push) Successful in 2m23s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m5s
ci / bench (push) Successful in 6m3s
apple / screenshots (push) Successful in 6m41s
deb / build-publish (push) Successful in 12m20s
arch / build-publish (push) Successful in 12m47s
deb / build-publish-host (push) Successful in 13m7s
android / android (push) Successful in 16m14s
windows-host / package (push) Successful in 15m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m23s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m8s
ci / rust (push) Successful in 26m6s
docker / deploy-docs (push) Canceled after 25s
Field report (iPad via GameStream, 2026-07-23): cursor missing from the
stream. The session's fresh target 257 reported cursor_excluded=false
(DECLARED_TARGETS is keyed per target and only 259 had declared), so the
host skipped forced-composite — but DWM's pointer exclusion after an
IddCx hardware-cursor declare reaches EVERY later monitor of the adapter,
not just the declaring target (proven on-glass: declare on 259 under
WUDFHost 2192, then 257's frames streamed pointer-free with CURSOR_SHOWING
set and SM_MOUSEPRESENT true). GameStream/Moonlight clients cannot draw a
forwarded cursor, so those sessions were silently cursor-less.

ADD replies now report any-declare-this-WUDFHost-life (the state's real
scope — it dies with the adapter reset, same as before); per-target ids
stay only as the dbglog audit trail. Channel-less sessions on a declared
adapter then hit the existing forced-composite gate (GDI poller + blend)
and carry the pointer in-video. Host code unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 15:58:54 +02:00
enricobuehlerandClaude Opus 4.8 35c2c1450e fix(apple/cursor): scale the forwarded pointer to the video fit
apple / swift (push) Successful in 1m24s
release / apple (push) Successful in 10m11s
apple / screenshots (push) Successful in 6m42s
ci / web (push) Successful in 1m3s
ci / docs-site (push) Successful in 1m9s
ci / bench (push) Successful in 5m48s
decky / build-publish (push) Successful in 25s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 20s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 18s
android / android (push) Successful in 16m24s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
deb / build-publish (push) Successful in 10m11s
arch / build-publish (push) Successful in 12m18s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 7m5s
docker / deploy-docs (push) Successful in 11s
deb / build-publish-host (push) Successful in 11m42s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m4s
ci / rust (push) Successful in 18m39s
The Mac client drew the host-forwarded cursor bitmap at its native pixel
size (pixel-as-points), so the on-screen pointer tracked the HOST's display
scaling: a 4K/high-DPI virtual display renders a 96 px pointer, which then
appeared huge on the client — and on retina every pointer was already
2x-inflated and upscaled-blurry, since the backing scale was ignored.

Cache the shape raw and rebuild the NSCursor at the live video-fit factor
(fit.width / mode.width — the same AVMakeRect fit the input mapping uses)
on each apply, re-fitting on resize / retina moves. The pointer now renders
at its true size relative to the streamed desktop at any host scaling:
crisp and 1:1 with the video at 100%, and proportional (not ballooning)
when the host sits at 200-300% — including a deliberately raised scale.

iOS/tvOS (UIPointerInteraction) and Android (no forwarded-bitmap path)
render the pointer differently and need no change; the SDL/Linux client has
the same native-size issue and will follow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 14:14:41 +02:00
enricobuehlerandClaude Fable 5 326d6e17c8 fix(windows/capture): stand the IddCx hardware cursor down while the secure desktop is up
apple / swift (push) Successful in 1m22s
apple / screenshots (push) Successful in 6m41s
ci / web (push) Successful in 1m7s
ci / docs-site (push) Successful in 1m18s
android / android (push) Successful in 12m44s
arch / build-publish (push) Successful in 12m36s
decky / build-publish (push) Successful in 26s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 21s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 20s
ci / bench (push) Successful in 6m14s
windows-host / package (push) Successful in 9m46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 21s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
deb / build-publish (push) Successful in 9m38s
deb / build-publish-host (push) Successful in 9m58s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7m0s
ci / rust (push) Successful in 24m1s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m7s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m55s
docker / deploy-docs (push) Successful in 12s
0.18.0 regression: UAC consent and Winlogon (lock/logon) stopped appearing
in streams. The cursor channel's IddCx hardware-cursor declare — re-issued
by the driver on every swap-chain assign — keeps the path out of the OS's
software-cursor mode, which is the only mode the secure desktop renders
through; DWM then never presents UAC/Winlogon into our swap-chain and the
stream repeats the last normal-desktop frame for the whole interaction.

The GDI cursor poller now classifies the input desktop on its reattach
cadence (UOI_NAME != Default = secure, 250 ms cadence instead of 2 s), and
the capturer edge-triggers the existing-but-unused proto-v6
IOCTL_SET_CURSOR_FORWARD flip: OFF at secure entry (the driver stops its
per-assign re-declare; the host facade forces the same-mode re-commit that
actualises the software-cursor default, under the vdisplay manager lock via
the new force_recommit()), back ON at dismissal for channel sessions.
Channel-session open resets the driver's persisted desired state to ON so
a session that died mid-secure-desktop can't leave the next one adopting
UNdeclared (the §8.6 cross-session composite trap); orderly teardown does
the same. The flip closure is built for every Windows session — a
channel-less session can reuse a driver monitor whose earlier-session
cursor worker still re-declares, and NOT_FOUND from never-declared targets
is ignored.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 14:13:15 +02:00
enricobuehlerandClaude Opus 4.8 588a7077e8 fix(host/gamepad): only degrade a virtual Steam pad on a same-PID conflict
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 1m0s
apple / swift (push) Successful in 1m26s
decky / build-publish (push) Successful in 32s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 23s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 20s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 26s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 16s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 19s
ci / bench (push) Successful in 6m10s
apple / screenshots (push) Successful in 6m38s
deb / build-publish (push) Successful in 9m26s
android / android (push) Successful in 12m27s
deb / build-publish-host (push) Successful in 13m55s
windows-host / package (push) Successful in 16m20s
arch / build-publish (push) Successful in 17m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 17m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m2s
ci / rust (push) Successful in 27m20s
docker / deploy-docs (push) Successful in 26s
The Steam-conflict gate matched any physical Valve `28DE` HID device, so a
Steam Machine with a physical Steam Controller 2 (`28DE:1302`) plugged in
degraded a client's requested virtual Steam Deck (`28DE:1205`) to a DualSense
— the reporter's controller passthrough came up as the wrong pad.

Steam Input drives *distinct* Steam controllers side by side; only two
*identical* `28DE` devices confuse it (the Deck-as-host case: built-in
`28DE:1205` + a second virtual Deck, observed live on Bazzite). Narrow the
gate to key on the exact VID+PID instead of the `28DE` vendor alone:

- `steam_backend_product()` maps each virtual Steam backend to the PID it
  enumerates as (Deck 1205, classic SC 1102, SC2 1302, SC2 Puck 1304).
- `physical_steam_controller_present()` -> `physical_steam_product_present(pid)`
  matches `:28DE:{PID}`, keeping the FVPF-serial / `vhci_hcd` / `/virtual/`
  exclusions so our own virtual pads never count.
- `degrade_steam_on_conflict` degrades only on a same-PID duplicate and logs
  the conflicting identity. `PUNKTFUNK_STEAM_FORCE=1` override unchanged.

A physical SC2 no longer blocks a virtual Deck (and vice versa); a genuine
same-identity duplicate (physical Deck + virtual Deck, or physical SC2 +
SC2 passthrough) still degrades. Adds a unit test for the PID mapping.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 14:11:44 +02:00
enricobuehlerandClaude Fable 5 4714235fe6 feat(core): stylus wire P0 — state-full RICH_PEN batches, PenTracker, HOST_CAP_PEN
apple / swift (push) Successful in 1m24s
release / apple (push) Successful in 10m3s
android / android (push) Successful in 13m31s
arch / build-publish (push) Successful in 13m39s
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m12s
windows-host / package (push) Successful in 16m40s
apple / screenshots (push) Successful in 6m38s
ci / bench (push) Successful in 5m26s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m9s
ci / rust (push) Successful in 20m46s
decky / build-publish (push) Successful in 30s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m15s
deb / build-publish (push) Successful in 8m50s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 45s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m6s
deb / build-publish-host (push) Successful in 9m52s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m44s
docker / deploy-docs (push) Successful in 28s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 7m4s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m45s
flatpak / build-publish (push) Failing after 8m6s
The pen plane from design/pen-tablet-input.md, protocol side only (the P1 uinput
tablet injector will consume it): 0xCC kind 0x05 carries batches of state-full
PenSamples (pressure, polar tilt + azimuth, barrel roll, hover distance, eraser
tool, barrel buttons); PenTracker diffs samples into injector transitions so a
lost datagram self-heals; HOST_CAP_PEN (0x10) gates sending and stays
unadvertised until a real backend exists. C ABI: PunktfunkPenSample +
punktfunk_connection_send_pen, documented in docs/embedding-the-c-abi.md §8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 12:58:31 +02:00
enricobuehlerandClaude Opus 4.8 1a6deeb781 feat(host/library): scan the user's non-Steam shortcuts into the game library
apple / swift (push) Successful in 1m21s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 57s
apple / screenshots (push) Successful in 6m41s
ci / bench (push) Successful in 6m22s
ci / rust (push) Successful in 24m42s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
decky / build-publish (push) Successful in 28s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
android-screenshots / screenshots (push) Successful in 3m10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m41s
windows-host / package (push) Successful in 9m53s
linux-client-screenshots / screenshots (push) Successful in 7m11s
deb / build-publish (push) Successful in 11m1s
docker / deploy-docs (push) Successful in 24s
deb / build-publish-host (push) Successful in 11m55s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m31s
arch / build-publish (push) Successful in 12m57s
web-screenshots / screenshots (push) Successful in 3m12s
android / android (push) Successful in 13m46s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m34s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m0s
release / apple (push) Successful in 10m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m45s
flatpak / build-publish (push) Successful in 5m51s
Steam's "Add a Non-Steam Game to My Library" entries carry no `appmanifest`, so
`scan_manifests` can't see them and they never appear in the streamed library.
Read each Steam account's binary `userdata/<id>/config/shortcuts.vdf` directly
(a binary KeyValues parse) to surface the user's own custom entries, plus their
grid artwork from `userdata/<id>/config/grid`. Best-effort: an unreadable or
absent shortcuts.vdf contributes nothing, and hidden shortcuts / duplicate
appids are dropped. Verified: clippy -D warnings + compile green on Linux.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 00:45:25 +02:00
enricobuehlerandClaude Opus 4.8 8c9baf3214 fix(decky): recreate the library shortcut when its cached appId is stale
The visible "Punktfunk" shortcut's Steam appId is cached in SteamUI
localStorage, which lives in Steam's CEF profile — so it survives a
plugin uninstall/reinstall. Deleting the shortcut left the key dangling;
on the next mount ensure*Shortcut recalled the dead appId, took the reuse
branch, and ran SetShortcut* against a non-existent id (silent no-ops).
AddShortcut was never reached, so the entry never came back. The same
stale-reuse also made the hidden stream shortcut RunGame a dead gameid.

Verify the remembered appId still maps to a live shortcut via
appStore.GetAppOverviewByAppID before reusing it; a confident miss falls
through to AddShortcut. When appStore is unavailable, assume it exists so
a false negative can't spawn a duplicate library entry.

Also add a "Recreate library shortcut" recovery button to the QAM About
section (recreateShortcuts): drops any dead cached keys and re-ensures.
Covers deleting the shortcut without reinstalling, where no mount fires
the self-heal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 00:41:32 +02:00
enricobuehlerandClaude Fable 5 d83eeedde4 ci(flatpak): drop the use-vc TCP-DNS hack — the runner-side cache made it the failure
audit / bun-audit (push) Successful in 11s
ci / web (push) Successful in 46s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m21s
audit / cargo-audit (push) Successful in 2m20s
decky / build-publish (push) Successful in 19s
ci / bench (push) Successful in 6m53s
apple / screenshots (push) Successful in 6m28s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 1m0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m27s
windows-drivers / probe-and-proto (push) Successful in 16s
windows-drivers / driver-build (push) Successful in 1m27s
deb / build-publish-host (push) Successful in 9m37s
android / android (push) Successful in 11m56s
docker / deploy-docs (push) Successful in 24s
deb / build-publish (push) Successful in 11m31s
arch / build-publish (push) Successful in 15m34s
windows-host / package (push) Successful in 11m29s
ci / rust (push) Successful in 24m30s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m37s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m28s
flatpak / build-publish (push) Failing after 8m18s
Every flatpak Tooling failure on 2026-07-22 (4 runs, incl. two AFTER the
runner fleet got its local dnsmasq cache) died in flatpak remote-add with an
instant 'Could not resolve hostname' ×10 while dnf in the SAME container
resolved fine. The one thing only this leg does is force glibc onto TCP DNS
(use-vc) — a 2026-07-11 mitigation for the embedded resolver dropping UDP
under fleet load. With job containers now resolving against the on-box
dnsmasq cache (daemon.json dns → docker0), UDP is reliable again and the TCP
path through the same chain is the flaky one under concurrency. Remove the
hack; keep the nss-resolve drop and retry.sh as the backstop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 00:27:52 +02:00
enricobuehlerandClaude Fable 5 786f6ad28f fix(pf-driver-proto): catch the layout tests up with the cursor-channel proto bump
The cursor-channel sweep grew AddReply by the cursor_excluded u32 tail (24
bytes, legacy prefix 20) and bumped PROTOCOL_VERSION to 6, but left two lib
tests behind: the AddReply byte-layout roundtrip still built the legacy
initializer (a compile error that turned the rust CI leg red on main), and
the version-coherence assertion still pinned v5 (hidden behind that compile
error). Cover the new tail bytes + the legacy-size constant, and widen the
compat-window note to v4–v6 (floor stays 3).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 00:22:34 +02:00
enricobuehlerandClaude Fable 5 1e9957d97b chore(web): clear the 2026-07-22 JS advisory wave — overrides + lock re-resolve
bun-audit went red mid-day on a fresh advisory wave (13 findings across two
snapshots of a moving DB): node-tar DoS trio+1 (critical), sharp/libvips CVEs,
fast-uri host confusion, undici ×7, brace-expansion/linkify-it/js-yaml/postcss
DoS-class, immutable, dompurify. All transitive pins in the management-console
tree; none ship in the streaming stack.

Every fix version is in-range for its consumers, so: pin overrides for the
single-major packages (tar/dompurify/linkify-it/sharp/fast-uri/immutable/
undici/postcss/js-yaml — bun ignores range-scoped override keys, so plain keys)
and a full lockfile re-resolve, which also lifts brace-expansion 2.1.1→2.1.2
in place while its 5.x line (minimatch@10) stays untouched. : No
vulnerabilities found.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 00:20:44 +02:00
enricobuehlerandClaude Opus 4.8 92f38ec3dd chore(cleanup): drop TEMP cursor probes + clear KWin-leg clippy debt
audit / bun-audit (push) Failing after 14s
ci / web (push) Successful in 52s
windows-drivers / probe-and-proto (push) Successful in 54s
ci / docs-site (push) Successful in 1m6s
apple / swift (push) Successful in 1m21s
audit / cargo-audit (push) Successful in 2m49s
windows-drivers / driver-build (push) Successful in 2m0s
decky / build-publish (push) Successful in 22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m47s
ci / rust (push) Failing after 10m23s
release / apple (push) Successful in 9m24s
deb / build-publish-host (push) Successful in 9m51s
apple / swift (pull_request) Successful in 1m21s
apple / screenshots (pull_request) Skipped
android / android (push) Successful in 12m25s
flatpak / build-publish (push) Failing after 8m8s
deb / build-publish (push) Successful in 12m16s
ci / web (pull_request) Successful in 50s
ci / docs-site (pull_request) Successful in 51s
docker / deploy-docs (push) Successful in 26s
arch / build-publish (push) Successful in 14m34s
windows-host / package (push) Successful in 15m5s
apple / screenshots (push) Successful in 6m33s
ci / rust (pull_request) Failing after 6m56s
ci / bench (pull_request) Successful in 6m5s
android / android (pull_request) Successful in 9m56s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m14s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m42s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m13s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m34s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 4m47s
windows-drivers / probe-and-proto (pull_request) Successful in 15s
windows-drivers / driver-build (pull_request) Successful in 1m30s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m3s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 1m54s
The KWin/Phase-A/B commits were built but never clippy-checked (that distrobox
had no clippy component), so they left TEMP on-glass probes and lint debt in the
tree. With clippy now runnable (fedora rust 1.96.1 = CI parity):

- drop the `fec424ee`/`8cff30d5` TEMP probes: the `update_cursor_meta` SPA_META
  diagnostic logs (also un-detaches the `// SAFETY:` comment from its `unsafe`
  block → fixes `undocumented_unsafe_blocks`) and the KWin composite-arm probe in
  the encode loop.
- `#[allow(clippy::too_many_arguments)]` on `spawn_pipewire` (8 params since the
  KWin leg added `expect_exact_dims`; mirrors `from_virtual_output`).

clippy `-p pf-capture -p pf-vdisplay -p punktfunk-host --locked --features
nvenc,vulkan-encode -- -D warnings` is now green. (--all-targets additionally
trips a pre-existing env mismatch: the fedora libspa binding lacks
`SPA_VIDEO_TRANSFER_SMPTE2084`, referenced only by a `#[cfg(test)]` guard-test —
not a code issue; CI's pinned pipewire has it.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:54:56 +02:00
enricobuehlerandClaude Fable 5 7058647264 fix(vdisplay/capture): channel-less sessions composite the pointer on a sticky-declared target
A declared IddCx hardware cursor is IRREVOCABLE for its OS target's life
(§8.6), and the sticky exclusion survives monitor REMOVE→ADD because each
client gets a STABLE target id — so once any desktop-mode session declared,
every later pure-capture session on that target streamed a cursor-less
desktop: DWM excluded the pointer, no channel forwarded it, no blend drew it
(the exact no-regression gap §8.6's per-session cap gate cannot see).

Driver: track every successful SetupHardwareCursor per target
(DECLARED_TARGETS — scoped to the WUDFHost's life, exactly the sticky
state's scope) and report it in a new AddReply::cursor_excluded tail field
(dual-size discipline, both skews degrade cleanly; no proto bump).

Host: the flag rides AddedMonitor → Monitor → WinCaptureTarget; a session
WITHOUT the cursor channel on a flagged target forces composite mode in the
IDD-push capturer — GDI poller + blend for the session's life, pinned on
(set_cursor_forward cannot clear it: with no client drawing, un-compositing
would erase the pointer entirely).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Opus 4.8 d5ae8dcc3e feat(capture): gamescope cursor via XFixes shape + QueryPointer position (Phase C)
gamescope excludes its pointer from the PipeWire node it feeds us and can't embed
one either (`set_hw_cursor` is inert), so every gamescope stream was cursorless.
Read the pointer from gamescope's nested Xwayland instead — XFixesGetCursorImage
for shape/hotspot/visibility, core QueryPointer for position — and publish a
CursorOverlay into the capturer's existing `cursor_live` slot, so the encoder
blend composites it into the video exactly like the SPA_META_Cursor path.

- pf-capture/src/linux/xfixes_cursor.rs (new): the XFixes reader. Connects to
  EVERY nested Xwayland (Gaming Mode runs one per --xwayland-count) and follows
  the focused one each tick (the display whose pointer moves), reading that
  display's own cursor shape. Un-premultiplies ARGB -> straight RGBA. Drop stops
  the thread.
- Capturer::attach_gamescope_cursor + the PortalCapturer override spawn it into
  the same `cursor_live` slot; pf_vdisplay::gamescope_xwayland_cursor_targets
  discovers the (DISPLAY, XAUTHORITY) pairs via the GAMESCOPE_WAYLAND_DISPLAY scan.
- host: SessionPlan.gamescope_cursor (set from the compositor at both resolve
  sites AND on a mid-stream Desktop->Gaming switch); the blend gate now builds the
  encoder blend for gamescope; a sibling composite arm attaches capturer.cursor()
  per tick for capture-mode clients (no channel needed). native NV12 is disabled
  for these sessions — that encode path can't blend the cursor (it assumes
  gamescope embeds the pointer), so we capture RGB and route to the proven CUDA
  VkSlotBlend / compute-CSC blend.
- the pipewire thread no longer clobbers `cursor_live` with None on a buffer that
  carries no SPA_META_Cursor (gamescope) — that raced the XFixes writer and
  strobed the composited pointer on/off.

On-glass (home-bazzite-2, RTX 5070 Ti, Gaming Mode): cursor visible + steady in
Big Picture and CS2 menus, follows focus between the two Xwaylands, hidden in-game.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 925130d1f9 feat(vdisplay): compositor-embedded pointer for sessions without the cursor channel (Phase B)
Since the cursor-channel work, every Linux virtual output was created in
metadata pointer mode — making ALL sessions depend on host-side cursor
compositing, including the ones that can never use the channel (Moonlight/
GameStream, legacy clients, capture-mode starts). Those sessions paid the
blend bring-up per session and, whenever a visible cursor was composited,
the loss of NVENC's stream-ordered submit — for a strictly worse cursor
than the compositor's own.

Mirror the Windows no-regression gate: the already-wired per-session
set_hw_cursor(cursor_forward) now drives the Linux backends too. A
cursor-channel session gets metadata (shapes forwarded, composite flip
blends host-side — today's validated path, unchanged); every other session
gets the pointer compositor-EMBEDDED at creation (KWin zkde pointer=2,
Mutter cursor-mode=1, wlroots/hyprland portal CursorMode::Embedded — their
pre-channel default; the portal pair also gains the Metadata arm for
channel sessions, wired but untested on-glass). SessionPlan.cursor_blend
narrows to cursor-forward sessions and rides into the direct-SDK NVENC
open, which skips the Vulkan slot-blend bring-up entirely when off —
embedded sessions ring on plain CUDA surfaces and pay zero cursor cost,
per-session or per-frame.

The keep-alive registry's reuse key grows the created pointer mode (new
VirtualDisplay::hw_cursor getter): a kept embedded display has no cursor
metadata for a channel session to forward, and a kept metadata display
would leave a channel-less session with no pointer in its frames.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 d2c46eaf3c feat(encode/nvenc): SPIR-V cursor blend over Vulkan-allocated input slots — retire the PTX kernels
A vendored PTX blob is JIT'd against the driver's ISA ceiling, so the
cursor-blend module silently dies on drivers older than the generating
toolkit (CUDA_ERROR_UNSUPPORTED_PTX_VERSION/INVALID_PTX, 222/218 — the
KWin leg's invisible composite cursor on driver 595/CUDA 13.2 vs a
CUDA 13.3 blob). SPIR-V has no such coupling, and the in-tree precedent
already exists twice (vulkan_video's CSC blend, VkBridge's exportable
OPAQUE_FD → cuImportExternalMemory bridge).

New pf_zerocopy::vkslot::VkSlotBlend: the direct-SDK NVENC encoder now
allocates its input ring as exportable Vulkan buffers CUDA-imports (same
contiguous InputSurface layouts, pitch = row bytes rounded to 256), and
the cursor composite is a compute dispatch over the cursor's rectangle
(cursor_blend.comp, vendored .spv; spec-constant selects ARGB/NV12/YUV444;
BT.709 limited, matching the retired .cu). The surface SSBO is uint[] with
every invocation owning whole words — no 8-bit-storage device dependency.
Cursor-bearing frames force the existing CPU-synced submit path so the
CUDA copy → Vulkan dispatch (fence-waited) → NVENC encode ordering is
CPU-established; cursorless frames keep the stream-ordered fast path
untouched. Any bring-up/alloc/registration failure falls back wholesale
to plain pitched CUDA surfaces (never a mixed or short ring): sessions
always encode, composite mode just loses the cursor, warned once.

cursor_blend.cu / cursor_blend.ptx and the CursorBlend PTX loader are
deleted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 33121ece4d chore(kwin-composite): TEMP blend-path probes — DROP BEFORE MERGE
Rate-limited journal logging bracketing the silent segment of the Linux
composite path: what overlay (if any) the encode loop's composite arm
hands the encoder, and whether the NVENC cursor-blend kernel launches and
with what geometry. On-glass (KWin leg) the blend module loads yet the
composite cursor stays invisible — these two probes attribute it to
no-overlay / stripped-en-route / kernel-draws-nothing. The module-loaded
INFO line is permanent (success must be as attributable as failure);
everything else in this commit is temporary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 af87549052 fix(linux/vdisplay): KWin virtual outputs stream above 60 Hz via sacrificial-birth renegotiation
KWin's ScreenCastStream builds its PipeWire format offer — including the
maxFramerate cap it actively throttles delivery to — ONCE at stream
creation, when the virtual output sits at its hardcoded birth 60 Hz. A
kscreen custom-mode change afterwards updates the OUTPUT (readback says
240) but never the offer, so every consumer negotiates max=60 and the
stream delivers 60 (pw-dump-verified on KWin 6.6.4). The only path that
rebuilds the offer is the stream's own resize handling: a source SIZE
change while recording re-runs buildFormats — picking up the output's
current refresh — and renegotiates the live stream.

So above 60 Hz, birth the output at a sacrificial height (+16), then
install + select the real WxH@hz custom mode: the first frame recorded
after the consumer connects triggers KWin's resize path, which renegotiates
the live stream to WxH@hz. The capturer holds (requeues) buffers until the
negotiated size matches — new expect_exact_dims plumbing VirtualOutput →
registry (fresh creates only) → open_virtual_output → a self-disarming
gate in the process callback — bounded by a 3 s deadline that accepts the
producer's dims rather than wedging the session into the first-frame
retry loop. set_custom_refresh reads back the full active mode; a size
reject (pre-6.6 KWin) recreates plain at the real size @60.

Every kscreen operation now addresses the output by its NUMERIC id,
resolved by matching the managed name-prefix AND the just-created birth
size (newest id wins): a mode-switch supersede reuses the per-slot output
NAME (deliberately, for KWin's per-name config persistence) while the
superseded sibling is still alive, and name addressing hit the FIRST
match = the OLD output — on-glass that resized the live session's display
out from under it (wrong-res/black), read the old output back as 'mode
applied', and set the old output primary while the replacement starved at
its birth mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 09113c9899 fix(encode/nvenc): cursor-blend PTX must carry a driver-portable ISA version
The vendored cursor_blend.ptx was regenerated with the CUDA 13.3 toolkit,
which stamps '.version 9.3' — and a driver whose JIT predates that ISA
refuses the whole module with CUDA_ERROR_UNSUPPORTED_PTX_VERSION (222),
silently killing host-side cursor compositing (on-glass: composite-mode
cursor invisible on driver 595.58, KWin leg). The kernels use only baseline
arithmetic, so hand-lower the version directive to 8.0 (CUDA 12.0), which
every supported Turing+ driver JITs. Both files now warn that an nvcc
regeneration re-stamps the version and must be re-lowered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 4436c7fb61 fix(host/stream): composite mode re-blends the LIVE cursor on repeat ticks
The frame-attached overlay is the pointer position at the last damage
frame; repeats re-encoding a static desktop froze the blended cursor
between redraws — on-glass the composite-mode cursor stuttered while
window drags (constant damage) were smooth. Refresh the repeat's overlay
from Capturer::cursor each tick so pointer-only motion re-blends at tick
rate — the bandwidth the pre-channel embedded mode already paid. Linux
only: the Windows capturer composites internally and its frames must
never carry an overlay a blend path would double-draw.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 088004cac4 feat(linux/capture): serve the LIVE cursor overlay through Capturer::cursor
The forwarder read only frame.cursor — the overlay attached to the LAST
EMITTED frame. Mutter's pointer-only buffers update the capture-side cursor
state but are skipped as frames, so between damage frames the forwarder
re-sent a stale snapshot: on-glass the client cursor appeared exactly when
clicking or crossing hover targets (damage) and froze/vanished otherwise —
the same pointer-only-motion gap the Windows IddCx channel fills, which is
why the encode tick already prefers the capturer's LIVE cursor. Publish the
overlay from every dequeued buffer (frames or not) into a shared slot and
implement the Capturer::cursor hook on PortalCapturer.

Host-composite mode on Linux still updates the blended pointer only on
damage frames (no re-encode of a static desktop on cursor motion) — the
regen-on-cursor-change the Windows path has is a follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 63d83217ef fix(linux/capture): a stale id-0 cursor meta is 'no information', not 'hidden'
Mutter only rewrites a buffer's SPA_META_Cursor region when the cursor
changed; recycled buffers between damage frames carry a stale id-0 meta.
Treating id 0 as a hidden pointer flickered the client cursor off between
hovers on-glass — per the SPA contract id 0 is 'invalid/no cursor info',
so keep the last-known state (OBS's consumer does the same). A genuinely
hidden pointer stops producing updates; the M3 relative-mode hint keeps
its Windows CURSOR_SUPPRESSED source.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 9db4841ce4 fix(linux/capture): cursor meta size range must cover Mutter's fixed 384x384 offer
Mutter offers SPA_META_Cursor with a FIXED size pod —
SPA_POD_Int(CURSOR_META_SIZE(384, 384)) in meta-screen-cast-stream-src.c —
while our request capped the range at meta_size(256, 256). The intersection
is empty, so the Meta param silently failed to negotiate and no buffer ever
carried the meta region: the entire Linux cursor pipeline (forward AND
blend) was blind on GNOME, proven by an on-glass probe counting 32k+
buffers with zero metas. KWin's offer fits inside 256², which is why the
same consumer code passed there. Raise the max to 1024² headroom (the
negotiated allocation follows the producer's value, not our max) and align
the bitmap parse guard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 151adc4bcc chore(m2b): TEMP cursor-meta probe — DROP BEFORE MERGE
Rate-limited journal logging of SPA_META_Cursor arrival/absence, reported
id/position/bitmap offset, and bitmap acceptance — the Linux cursor
pipeline is otherwise blind end-to-end during Mutter RecordVirtual
metadata bring-up on .21.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 0619af391e fix(linux/inject): map absolute pointer/touch into the streamed output's region
The libei absolute device advertises one region per logical monitor, and
both MouseMoveAbs and touch mapped into regions().first() — whichever
output the compositor announced first. Next to a physical monitor (on-glass:
GNOME with a dummy HDMI beside the virtual primary) that put the pointer and
every click on the WRONG output: the seat cursor never entered the streamed
monitor, so neither embedded nor metadata cursor capture could ever see it,
and the cursor channel had nothing to forward. Pick the region whose logical
size matches the streamed mode (the wire flags already carry it); fall back
to first() for the single-monitor case. Region geometry now rides the
device-RESUMED log line for diagnosability; matching the screencast
mapping_id instead of the size is the follow-up for same-sized-monitor
ambiguity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 afe120bf34 fix(linux/vdisplay): kwin virtual output ships the cursor as metadata too
Same trap as the Mutter backend: zkde_screencast pointer mode 2 (embedded)
never delivers SPA_META_Cursor, starving both the cursor channel and the
encoder blend. Mode 4 (metadata) feeds both.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 c29186bd1a fix(host+vdisplay): a mode-switch replacement display inherits the group topology
The resize rebuild creates the new virtual display BEFORE retiring the old
one (create-before-drop), so the registry saw a live same-backend sibling —
its own dying predecessor — and told the backend it was not first in group.
Mutter then skipped the Primary/Exclusive apply ("joining an existing
display group — extending") and the retiring owner took the topology with
it: every resize silently demoted the virtual output to an extended,
shell-less desktop. Thread the superseded pool gen through acquire so the
replacement establishes topology, and stop counting kept (Lingering/Pinned)
entries as demoting siblings — no session owns them, so there is no live
desktop to clobber.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 315ffaf144 fix(linux/vdisplay): mutter virtual monitor ships the cursor as SPA_META_Cursor
RecordVirtual was created with cursor-mode EMBEDDED, which never delivers
cursor metadata — the cursor channel had nothing to forward (client-drawn
cursor invisible) and the always-built encoder blend had nothing to
composite (host-drawn cursor invisible too). Metadata mode feeds both:
non-forwarding sessions get the blended pointer (same pixels Mutter would
have embedded), forwarding sessions strip the overlay and send shape/state.
The capturer already negotiates the meta on every stream (meta_param).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 c63ac48c9c fix(windows/capture): actually QUERY the HDR SDR-white scale — the refresh block was silently dropped
faad4b57 shipped the field, shader scale, and plumbing but the
scripted edit inserting the sdr_white_level_scale() refresh in the
scratch-rebuild arm matched nothing and silently no-op'd (no assert) —
sdr_white_scale stayed at its 1.0 init, i.e. 80 nits, i.e. exactly the
old dark cursor. Now queried on every scratch rebuild, with an info
log of queried/applied so an HDR session shows what it uses.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 cecc059830 fix(windows/capture): 250 Hz cursor poll + HDR white-scale observability
- The polled position is also the composite-blend position: at 16 ms a
  240 fps session reused a stale position for ~4 consecutive frames and
  the composited pointer visibly stuttered against the video. 4 ms
  out-paces every session rate; a tick is one GetCursorInfo syscall.
- Log the queried DISPLAYCONFIG_SDR_WHITE_LEVEL scale on each blend-
  scratch rebuild so an HDR session shows what is actually applied
  (the virtual display's own SDR-brightness setting is the suspect
  when the cursor still reads dark).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 d31d41b6b8 fix(windows/capture): match the HDR-composited cursor to the target's SDR white level
sRGB→linear alone lands cursor-white at 80 nits (scRGB 1.0) while DWM
composes the surrounding SDR desktop at the user's SDR-brightness
setting (~200+ nits by default) — the cursor read visibly dark on HDR.
Scale by DISPLAYCONFIG_SDR_WHITE_LEVEL (new
win_display::sdr_white_level_scale, queried on blend-scratch rebuilds —
the CCD query stays off the per-frame path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 6e37da8b4e fix(windows/capture): clippy unnecessary_lazy_evaluations in scratch build
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 0af280a793 feat(windows/capture): host-side cursor compositing for the capture model — the deterministic fix
Root cause, proven on-glass across five driver builds: a declared IddCx
hardware cursor is IRREVOCABLE. There is no un-declare DDI, the
empty-caps re-setup is rejected INVALID_PARAMETER, and after a
successful same-mode re-commit with the driver's re-declare provably
suppressed (sticky per-target flag, zero re-setups logged) DWM still
never composites the software cursor back into the monitor's frames.
Every DWM-based composite flip is therefore unfixable.

The composite (capture) model is now implemented in OUR pipeline:
- the driver keeps its hardware cursor declared for the session's whole
  life (the state that works) — frames stay pointer-free always;
- in composite mode try_consume routes the conversion through a blend
  scratch: slot copy + one alpha-blended quad of the GDI poller's shape
  at its polled position (CursorBlendPass — fullscreen-triangle VS with
  viewport placement, straight-alpha PS, sRGB→linear for FP16/HDR
  rings), covering all four convert paths (NV12, 4:4:4 copy, P010,
  PyroWave) GPU-side under the slot's keyed mutex;
- pointer-only motion produces no driver publish (declared hw cursor),
  so try_consume REGENERATES from the last slot whenever the polled
  cursor state changes — the cursor moves on a static desktop at tick
  rate, without faking freshness for the stall/death watchdogs;
- set_cursor_forward becomes purely host-internal state; the
  SET_CURSOR_FORWARD IOCTL machinery is no longer used by the host
  (driver keeps it, dormant) and its sender plumbing is removed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 89cfef429e fix(windows/vdisplay): cursor-render state sticky per TARGET across monitor generations
On-glass: 're-setup on swap-chain assign -> 0x0' fired AFTER 'enable=0
stored' — duplicate/successor monitor entries (re-arrival churn) kept
their default-true flag and re-declared the hardware cursor right after
the composite flip's re-commit, undoing it.

The desired state now lives in a per-target static
(CURSOR_FORWARD_DESIRED): every arrival inherits it (no generation can
resurrect a declare the session turned off), and the flip stamps EVERY
live entry matching the target, declaring against whichever has the
live worker.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 b50e4767de fix(windows/cursor-flip): flag + forced same-mode re-commit — the working un-declare
The empty-caps un-declare is REJECTED by IddCx (STATUS_INVALID_PARAMETER,
on-glass driver 9.9.0722.1407) — there is no un-declare DDI. The
composite flip now works with the only lever the OS gives us: every
mode COMMIT reverts the path to the software cursor, and the driver
skips its per-commit re-declare while cursor_forward_on is off. So:

- driver: disable stores the flag only (no DDI call); enable still
  declares immediately against the live worker's event.
- host capturer: after a successful disable flip, force a same-mode
  re-commit (win_display::force_mode_reenumeration) — DWM composites
  the pointer from the very next commit; the swap-chain flap is the
  class the driver's preserved-publisher machinery already rides out.
- state now survives re-arrivals end-to-end: the capturer caches the
  APPLIED state (cleared on every channel (re)delivery, which
  re-created driver entries need — their flag defaults to declared),
  and the stream loop re-applies every tick instead of edge-gating
  (steady-state cost: one Option compare).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 79d30fde64 fix(windows/capture): unsafe block for the extracted dup_into_public call
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 6a8df2ba97 fix(windows/cursor-flip): survive driver-side monitor re-arrival — flag-based flip + channel re-delivery
On-glass (.173, match-window session): every window-size convergence
re-creates the driver-side monitor (re-arrival resize / sibling-session
slot churn), destroying the cursor worker the channel was delivered to —
the flip then died NOT_FOUND (no entry with target_id ∧ worker) and the
declared state was lost entirely.

Driver: set_cursor_forward becomes STATE, not an edge on one monitor
generation — find by target_id ∧ hw_cursor, store cursor_forward_on
even without a live worker (it steers the next delivery/re-setup);
declare/un-declare only when a worker exists. Channel delivery honors
the flag: setup_and_spawn(declare=false) adopts + spawns WITHOUT
declaring (composite mode), so a later enable-flip declares against the
worker's event.

Host: retain the SET_CURSOR_CHANNEL sender; extract
deliver_cursor_channel and RE-deliver the surviving section on every
ring recreate and — with one retry — when a flip IOCTL fails
(the re-arrived entry gets a fresh worker, declared per its flag).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 030a779391 feat(windows/vdisplay): the driver leg of the mid-stream cursor-render flip
Belongs to 493f4fae (which shipped only the crates/ side — this
packaging/ half was left unstaged, so the deployed driver silently
lacked the IOCTL and every flip died NOT_FOUND at dispatch):
IOCTL_SET_CURSOR_FORWARD dispatch + monitor::set_cursor_forward
(cursor_forward_on flag, resetup gate) + cursor_worker::
unsetup_hardware_cursor (empty-caps un-declare experiment).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 d86073dc6a feat(apple/cursor): send the mid-stream cursor-render flip — ABI v12
PunktfunkConnection.setCursorRender(clientDraws:) over the new
punktfunk_connection_set_cursor_render export, driven by one
edge-detected reconciler in StreamView (clientDraws = captured &&
desktopMouse) called from every transition — the ⌃⌥⇧M chord,
engage/release, and session start. Capture model and released now get
the host-composited pointer back in the video (full fidelity); the
desktop model keeps the local NSCursor path. xcframework rebuilt v12.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 92761813a9 feat(core+host+driver+client): mid-stream cursor-render flip — host composites in the capture model
The cursor channel excluded the host pointer from the video for the
whole session, but the client only draws it under the DESKTOP mouse
model — flipping ⌃⌥⇧M to capture mid-stream left the session
pointer-blind (user-found). The fix makes render ownership LIVE state:

- wire: CursorRenderMode 0x51 (client→host, control stream) —
  client_draws: true = desktop model (host excludes + forwards),
  false = capture model / released (host composites, full fidelity —
  DWM on Windows incl. real XOR inversion, encoder blend on Linux).
  Sessions start client_draws=true (the pre-message behavior).
- host: control task stores the flag; the encode loop edge-detects it —
  forwarding + frame.cursor strip while the client draws, quiet
  forwarder + overlay-into-blend while composited. SessionPlan now
  grants blend CAPABILITY wherever the capture has a pointer (dropping
  the !cursor_forward gate) so the composite side needs no encoder
  rebuild; per-tick frame.cursor decides what is drawn.
- Windows: Capturer::set_cursor_forward → retained
  IOCTL_SET_CURSOR_FORWARD sender (proto v6) → driver (un)declares the
  IddCx hardware cursor on the LIVE monitor. Un-declare re-issues the
  setup with EMPTY caps (candidate mechanism — the DDI has no
  documented un-setup); resetup-on-commit is skipped while off, so a
  mode commit's software-cursor default is the fallback path. Failure
  against a pre-v6 driver logs and keeps declared-at-ADD behavior.
- SDL client: one edge-detected reconciler at the cursor pump site —
  desktop-active = client draws; capture model or released = host
  composites. Covers the chord, the M3 auto-flip, and engage/release.
- C ABI v12: punktfunk_connection_set_cursor_render (additive; wire
  version unchanged — pre-§8 hosts ignore the unknown message type).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 0420be9fa5 fix(apple/cursor): reset per-session cursor-channel state in stop()
cursorChannelActive/hostCursors/cursorState stayed latched across
sessions — a next session against a host WITHOUT the cursor cap would
wear the previous session's stale shapes via resetCursorRects.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 f2082f8b57 fix(windows/capture): drop redundant unsafe at the poller-spawn CCD call (inside open_on's unsafe region)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 9162e7b451 fix(windows/vdisplay): drop now-unused INFINITE import (poll-loop wait)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 e59634a578 feat(windows/capture): GDI cursor-shape poller — masked/monochrome cursors for the cursor channel
The M2c redesign (design/remote-desktop-sweep.md §8): the IddCx hardware-
cursor query is alpha-only by design — IDDCX_CURSOR_SHAPE_TYPE has no
monochrome value, the OS pre-converts monochrome to masked-color, and
masked-color delivery is dead code on modern builds (proven on-glass at
every ColorXorCursorSupport level). The driver keeps its hardware cursor
declared (XOR FULL) purely so DWM excludes every cursor type from the
IDD frame; the SHAPE now comes from a GDI poller in the capture host —
which runs as SYSTEM inside the interactive session, so no helper
process is needed.

CursorPoller (idd_push/cursor_poll.rs, the RustDesk/WebRTC/OBS pattern):
~60 Hz GetCursorInfo on a dedicated thread; rasterise via
CopyIcon→GetIconInfo→GetDIBits only when the HCURSOR value changes;
monochrome via the WebRTC truth table with invert pixels as opaque black
plus a white outline; AND-mask alpha for alpha-less color cursors;
CURSOR_SUPPRESSED = hidden; per-output visibility via the target's
desktop rect; per-monitor-V2 DPI awareness thread-scoped; input-desktop
reattach on failure + 2 s cadence (GetCursorInfo on a stale desktop
succeeds with stale data). Spawned iff the cursor channel was delivered;
a live poller is the sole overlay source (no serial-namespace mixing) —
the driver shm read remains as fallback if the poller dies.

Not chosen: DXGI Desktop Duplication GetFramePointerShape —
PointerPosition.Visible goes stale under injected-only input on current
Win11 (Sunshine #5293, exactly our topology), it burns one of the
session's four duplication slots, and IDD-output metadata has
conflicting field reports.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 d2cc770938 fix(windows/vdisplay): QueryHardwareCursor3 + per-mode-commit cursor re-setup — M2c findings
Durable results of the M2c on-glass bring-up (.173, 2026-07-22):

- The base IddCxMonitorQueryHardwareCursor DDI slot is stubbed to
  STATUS_NOT_SUPPORTED on WDK 26100 — add the ...QueryHardwareCursor3
  wrapper (wdk-iddcx) and drain it instead; v3 X/Y are only meaningful
  when PositionValid, so a position-invalid tick keeps the prior position.
- Hardware-cursor setup is per-mode-commit: the OS silently reverts to a
  software cursor on every mode commit, so re-issue the setup on each
  swap-chain assignment (monitor::resetup_cursor, called from
  assign_swap_chain) or the query fails NOT_SUPPORTED forever.
- One caps definition for initial setup and re-setup, resting at XOR
  FULL: the query delivers ONLY alpha shapes in every configuration
  (all three ColorXorCursorSupport levels, event-driven and 30 Hz
  polled) — masked/monochrome cursors never arrive. FULL keeps the
  frame cursor-free for ALL cursor types; the full-fidelity shape comes
  from the session-side cursor source in the host
  (design/remote-desktop-sweep.md §8).
- Poll the query at ~30 Hz instead of pure event-wait: masked cursors
  never fire the data event, and the seqlock's position/visibility
  should stay fresh regardless.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 2bbcbc81c6 fix(apple/cursor): disable M3 visibility-based auto-flip — it broke desktop drags
On-glass (Mac ↔ Windows M2c host): the host cursor's VISIBILITY is not a
usable 'a game grabbed the mouse' signal — Windows hides the pointer for
ordinary desktop activity (clicking, typing). The per-frame forwarder
turned those transients into relative_hint=true, so the client flipped
desktop→capture→desktop, which (a) warped the cursor to view-centre and
(b) flushed held buttons via releaseAll — a spurious mouse-up ~200 ms
into every press that broke window dragging/resizing.

Disable the auto-flip; the mouse model stays user-driven (⌃⌥⇧M). The
hint still rides the wire, unused, for a future M3 that keys off a REAL
pointer-lock signal (host-side ClipCursor/raw-input detection) instead
of visibility.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 a99ef7f940 fix(windows): hw_cursor_capable must not assume an initialised display manager
The FIRST session's Welcome runs before any backend construction
(vdisplay::open happens at display prep, after the Welcome), so the
capability probe's vdm() expect panicked the very first handshake of a
fresh service — the client saw a dead connect and auto-wake kicked in.
init() is idempotent and the driver facade is a unit struct, so the
probe now initialises the manager itself. Found on-glass (.173, first
Mac-client connect), fix deployed there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 ec8ca9a535 feat(apple): cursor channel on the Mac client — ABI v11 + NSCursor rendering
The Mac client now exercises the full cursor channel against capable
hosts: desktop-mode sessions advertise CLIENT_CAP_CURSOR, the host
stops compositing the pointer, and StreamView draws the forwarded
shapes as the real NSCursor — plus the M3 host-driven model flip.

- ABI v11: punktfunk_connect_ex9 (adds client_caps; ex7/ex8 pass 0 —
  Android untouched), PunktfunkCursorShape/State + the two
  next_cursor_* poll fns (audio-style borrow contract), and
  PUNKTFUNK_CLIENT_CAP_CURSOR. Fixed in passing: the first insertion
  split next_host_timing's cfg/no_mangle attributes off the fn, which
  silently dropped the symbol from the header AND dylib — caught by
  the Swift build, reattached with its docs.
- PunktfunkConnection: clientCaps connect param, hostSupportsCursor,
  CursorShapeEvent/CursorStateEvent + nextCursorShape/nextCursorState
  (one cursor thread drains both planes; cursorLock joins the close()
  ladder).
- SessionModel: desktop-mode sessions (DefaultsKey.mouseMode) connect
  with the cursor cap on macOS.
- StreamView: shape cache (serial → NSCursor, straight-alpha RGBA via
  CGImage), resetCursorRects wears the HOST shape while the desktop
  model is engaged (hidden host pointer ⇒ invisible), latest-wins
  state pump, and the M3 auto-flip — edge-triggered on relative_hint,
  ⌃⌥⇧M sets an override latch cleared by the next host edge, leaving
  relative warps the pointer to the host position via the inverse
  letterbox mapping (CGWarpMouseCursorPosition, CursorCapture's
  coordinate convention).

Verified: swift build + full test suites green (xcframework rebuilt at
ABI v11, signed); core 218 tests + clippy -D warnings on Linux (.21).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 3a34440a6b feat(windows): IddCx hardware-cursor channel — remote-desktop sweep M2c
Brings the cursor channel to Windows hosts. The pf-vdisplay driver
declares an IddCx hardware cursor for sessions that negotiated
cursor-forward — DWM then EXCLUDES the pointer from the IDD frame and
delivers shape/position out-of-band, into the same CursorOverlay →
forwarder → wire → client pipeline the Linux portal path uses.

- pf-driver-proto v5 (additive, host floor stays 3): AddRequest's spare
  tail becomes hw_cursor (same size/offsets); IOCTL_SET_CURSOR_CHANNEL
  delivers a host-created CursorShm section (64-byte seqlock header +
  256² shape buffer, layout pinned + tested). No event crosses the
  boundary — the host polls at encode-tick pace.
- driver: wdk-iddcx grows the two cursor DDI wrappers; a per-monitor
  cursor worker (event wait → QueryHardwareCursor → seqlock publish)
  starts only when BOTH the ADD asked and the channel arrived, so a
  failed delivery leaves DWM compositing as today. Shape bytes ship raw
  (BGRA/masked + pitch); the host converts.
- host: the section rides the existing sealed-channel broker (least-
  privilege dup, remote reap on failure); IddPushCapturer::cursor()
  seqlock-reads → CursorOverlay (BGRA→RGBA, masked-color approximation,
  desktop→frame origin shift, per-shape conversion cache). New
  Capturer::cursor() trait hook — the encode loop prefers it over the
  frame-attached overlay because hardware-cursor moves produce NO new
  frame on a static desktop. hw_cursor survives the re-arrival resize
  (carried on the manager's Monitor).
- negotiation: cursor_forward grows the Windows arm (client cap ∧
  driver proto ≥ 5, probed once via the control device); SessionPlan
  carries cursor_forward → OutputFormat.hw_cursor.
- drive-by: wdk-probe's two pre-existing same-type casts (clippy) and
  pf-vdisplay's stale spike-test refs (crate::win_display moved to
  pf-win-display; tracing-subscriber was never a dep) repaired.

Verified: proto tests on Mac AND MSVC (15/15 incl. the CursorShm
layout pin); clippy -D warnings for proto/frame/capture/vdisplay/host
on Linux (.21) and native Windows (.173); the DRIVER workspace clippy
-D warnings green against the real WDK 10.0.26100 bindgen (DDI names,
enum variants and IDARG layouts all bind). On-box driver deploy +
on-glass validation follow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 c3cbffe662 feat(host+client): host-driven mouse-model flip — remote-desktop sweep M3
The full Parsec model: launching a game from a desktop session flips the
client into captured relative automatically, and back — no chords.

- Capture overlay now distinguishes 'hidden' from 'no cursor yet':
  CursorOverlay grows a visible flag, overlay() returns Some whenever a
  bitmap is known (the encode loop strips invisible overlays after
  forwarding so no blend path ever draws one; the CPU composite guarded
  on visibility already).
- Host forwarder maps it onto the reserved wire bit: visible ⇒ VISIBLE,
  hidden-but-known ⇒ RELATIVE_HINT (an app grabbed/hid the pointer),
  never any hint before the first bitmap — a cold start can't flip a
  desktop session into capture.
- Presenter: edge-triggered auto-flip on hint changes via the new
  Capture::set_desktop (chord semantics untouched); a manual ⌃⌥⇧M sets
  an override latch that holds until the HOST's intent next changes, so
  the hint never fights the user. Leaving relative warps the local
  cursor to the host's last pointer position through the new
  content_to_window inverse-letterbox mapping (unit-tested against
  finger_to_content) — the hand-back is seamless.

Verified on .21: fmt + clippy -D warnings + tests (presenter 20 incl.
the inverse-mapping roundtrip, host 245).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 4a01bc4463 feat(core+host+client): cursor channel — remote-desktop sweep M2a+M2b
The host cursor stops riding the video and becomes a real OS cursor on
the client (the Parsec/RDP model): pointer feel no longer pays the
capture→encode→network→decode→present round trip.

Wire (M2a):
- Hello grows a client_caps trailing byte (CLIENT_CAP_CURSOR) after the
  fixed display_hdr block — presence disambiguated by remaining length,
  which caps the post-HDR tail at 27 bytes (documented); Welcome answers
  HOST_CAP_CURSOR (capable-and-asked, the 444/clipboard precedent).
- CursorShape (0x50, control stream): serial + dims + hotspot + straight
  RGBA, ≤120px/side so the u16 frame always fits (128² would overshoot);
  client caches by serial — re-showing a known shape costs 14 bytes, not
  a bitmap (RDP pointer-cache for free).
- CursorState (0xD0 datagram): serial + visible/relative_hint flags +
  position, sent once per encode-loop tick — latest-wins, self-healing
  under loss, no refresh timer. relative_hint is reserved for M3.
- Client core: two new planes (control-task + datagram-task arms) →
  next_cursor_shape/next_cursor_state; connect() grows client_caps
  (C ABI passes 0 until the v11 cursor poll fns exist).

Host (M2b, Linux portal only):
- handshake::cursor_forward is THE predicate (client asked ∧ Linux ∧
  compositor ≠ gamescope) — Welcome bit and session wiring both read it.
- SessionPlan.cursor_blend goes false for a forwarding session; the
  encode loop ticks a CursorForwarder every iteration: shape-serial diff
  → control-task bridge (mirrors probe_result), state datagram → conn.
- CursorOverlay/capture CursorState carry the hotspot through
  (nearest-neighbor downscale backstop for XL cursors, unit-tested).

Presenter:
- CursorChannel drains both planes per loop iteration; shapes become
  SDL color cursors (from_surface + hotspot), applied while the desktop
  mouse model is engaged; visibility follows the host; capture/released
  hands back the system cursor. Sessions advertise the cap when they
  START in desktop mode.

Verified on .21: fmt + clippy -D warnings (7 crates) + tests green
(core 218 incl. new wire roundtrips, host 245 incl. e2e + forwarder
downscale tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 97d20e1f0a feat(apple/input): desktop (absolute) mouse mode on macOS — remote-desktop sweep M1
Folds the parked client-side-cursor machinery (cursorMode auto/always/
never, hidden while disabled) into the cross-client mouse model:
MouseInputMode capture|desktop under DefaultsKey.mouseMode, picked in
Settings ▸ Keyboard & mouse (macOS), resolved at session start and
gated off on gamescope hosts (relative-only EIS).

Desktop model = the un-neutered absolute path with the SDL cursor
policy: pointer never disassociated (enters/leaves the stream freely),
monitor forwards letterboxed absolute positions, and the local cursor
hides only while over the view via an invisible-cursor rect (the
host's composited cursor is the one you see; AppKit manages the rect,
so no hide/unhide balancing). ⌘⇧C becomes ⌃⌥⇧M — the same chord as
the SDL clients — flipping the model live with an atomic
release/re-engage.

Verified: swift build + full test suites green (macOS arm64).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:37:32 +02:00
enricobuehlerandClaude Fable 5 df04880273 feat(client/input): desktop (absolute) mouse mode — remote-desktop sweep M1
New physical-mouse model beside capture: Desktop leaves the pointer
uncaptured and sends absolute positions through the letterbox
(MouseMoveAbs — every host injector already consumes it). Capture
stays the default and the game model.

- pf-client-core: MouseMode (capture|desktop) persisted in Settings,
  default capture so existing stores are unchanged.
- pf-presenter: Capture grows a desktop model — latest-wins pending
  abs position coalesced per loop iteration (same 1000 Hz discipline
  as relative), flushed before clicks/keys/wheel so they land where
  the cursor is; Ctrl+Alt+Shift+M flips the model live; local cursor
  stays hidden over the window (the host's composited cursor is the
  one you see until the M2 cursor channel); Windows keyboard grab
  only engages for capture (a desktop stream is something you
  Alt-Tab away from).
- gamescope gating: its EIS is relative-only, so desktop mode is
  pinned off there (resolved_compositor), with a log note.
- Settings surfaces: GTK row (dynamic caption), WinUI combo,
  console-UI row + step test, capture-hint line.

Verified: fmt + clippy -D warnings + tests (33/40/19) on Linux .21;
clippy -D warnings for all five crates incl. punktfunk-client-windows
native on .173.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:37:32 +02:00
enricobuehlerandClaude Fable 5 73fee86337 chore(release): bump workspace version to 0.18.0
audit / bun-audit (push) Failing after 13s
ci / docs-site (push) Failing after 22s
ci / web (push) Successful in 58s
apple / swift (push) Successful in 1m19s
audit / cargo-audit (push) Successful in 3m15s
decky / build-publish (push) Successful in 17s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 55s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 19s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / bench (push) Successful in 8m25s
deb / build-publish (push) Successful in 9m47s
docker / deploy-docs (push) Successful in 25s
release / apple (push) Successful in 10m10s
android / android (push) Successful in 12m56s
deb / build-publish-host (push) Successful in 13m10s
windows-host / package (push) Successful in 16m18s
arch / build-publish (push) Successful in 17m16s
apple / screenshots (push) Successful in 6m39s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m5s
ci / rust (push) Successful in 19m43s
flatpak / build-publish (push) Successful in 6m25s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m10s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m33s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m12s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m32s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m24s
The 0.17.2 placeholder underestimated this cycle: Android mouse & keyboard
(physical mouse + capture, TV remote-as-pointer, IME text, clipboard), the new
committed-text input plane (InputKind::TextInput / HOST_CAP_TEXT_INPUT), and
the ChaCha20-Poly1305 session cipher are feature work — a minor, not a patch.
Wire/driver/C-ABI markers all unchanged (2 / 4 / 10): every addition is
negotiated or ignored by older peers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 22:13:20 +02:00
enricobuehlerandClaude Fable 5 b5ed486243 ci(android): survive fleet-load download truncation in the SDK setup
The android leg failed twice on 2026-07-22 with sdkmanager's "Error on
ZipFile unknown archive" — once in our NDK/build-tools step, once inside
setup-android's default package set. Root cause is the shared runner fleet
dropping packets under parallel-job load (the same class retry.sh documents
for the flatpak leg): sdkmanager unzips while streaming, so a truncated
stream reads as a corrupt archive. A solo re-download on the same runner
verified clean.

- setup-android: install only platform-tools — the action's default legacy
  `tools` package drags in the ~250 MB emulator this job never uses, which
  was the largest and flakiest download in the whole leg.
- NDK/build-tools/CMake: wrap sdkmanager in retry.sh (4 attempts, linear
  backoff) — a failed attempt leaves no partial package, so re-invoking is
  safe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 22:12:02 +02:00
enricobuehlerandClaude Opus 4.8 0e7f6d85d6 feat(gamestream): signal HDR mode to the client via the 0x010e control message
ci / web (push) Successful in 1m14s
apple / swift (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m20s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
ci / bench (push) Successful in 6m41s
apple / screenshots (push) Successful in 6m12s
deb / build-publish (push) Successful in 9m48s
docker / deploy-docs (push) Successful in 25s
deb / build-publish-host (push) Successful in 10m26s
windows-host / package (push) Successful in 16m13s
android / android (push) Successful in 16m27s
arch / build-publish (push) Successful in 17m20s
ci / rust (push) Successful in 20m1s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m43s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m41s
punktfunk negotiates HDR correctly (the stream is BT.2020 PQ 10-bit) but never
told the client to switch its display into HDR picture mode, so HDR streams
stayed in SDR on the TV. Sunshine sends an async control-stream message
(IDX_HDR_MODE, type 0x010e, carrying SS_HDR_METADATA) shortly after the stream
starts; Moonlight only flips the TV into HDR on that cue. We sent exactly one
host→client control message (rumble 0x010B) and never this one. (aurora-tv PR
#53 worked around it client-side by inferring HDR from the negotiated format;
this fixes it host-side so stock Moonlight clients switch too.)

Once the control stream is live and the session negotiated HDR, send a one-shot
0x010e message (enable + 26-byte SS_HDR_METADATA from generic HDR10 defaults),
latched per connection and re-armed on Disconnect. The rumble sequence counter
is unified into a single monotonic `host_seq` shared by both message types —
the GCM nonce derives from `seq`, so a separate per-type counter would reuse
(key, nonce) pairs in the host direction.

The BT.2020 PQ VUI already rides the bitstream (the encoder derives it from the
P010 capture format), so this is purely the missing "switch now" signal; no
encoder change needed. Wire layout locked by unit tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 21:35:16 +02:00
enricobuehlerandClaude Opus 4.8 2dddf0fefc fix(gamestream): accept legacy X.509 v2 Moonlight client certs in mTLS pairing
rustls-webpki 0.103 accepts only X.509 v3 certs and returns
`UnsupportedCertVersion` for anything older, aborting the mTLS handshake at
the `CertificateVerify` step before it ever checks the signature. The
moonlight-embedded client family (moonlight-embedded, aurora-tv, …) still mint
self-signed **v2** certs with no keyUsage extension, so pairing fails against
punktfunk while it works against Sunshine — whose OpenSSL verify callback never
inspects the cert version. (aurora-tv PR #53 worked around this client-side by
switching to v3 + keyUsage; this fixes it host-side for stock clients too.)

The cert's X.509 version and extensions carry no security weight here: the
client cert is self-signed and pinned by SHA-256 (`peer_is_paired`), and the PIN
pairing ceremony is the real proof of identity. The property that DOES matter —
that the peer holds the private key for the cert it presented — is still
enforced: when webpki rejects the cert we re-run exactly that `CertificateVerify`
check with a version-agnostic parser (x509-parser SPKI + the same `rsa` verify
`pairing::verify256` uses). This is a full cryptographic check, never a bypass:
a bad signature still fails, and non-RSA / unsupported schemes fall through to
webpki's original error. Matches Sunshine's leniency without loosening the
pinned trust model.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 21:35:03 +02:00
enricobuehlerandClaude Fable 5 c90343c22f feat(android): shared clipboard (text) — device↔host sync while streaming
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m13s
apple / swift (push) Successful in 1m21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
decky / build-publish (push) Successful in 33s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 8m18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m49s
deb / build-publish (push) Successful in 10m26s
release / apple (push) Successful in 9m22s
docker / deploy-docs (push) Successful in 28s
deb / build-publish-host (push) Successful in 12m44s
arch / build-publish (push) Successful in 15m51s
windows-host / package (push) Successful in 16m14s
apple / screenshots (push) Successful in 6m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m6s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m18s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
ci / rust (push) Successful in 24m48s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m21s
android / android (push) Successful in 11m30s
flatpak / build-publish (push) Successful in 6m16s
The desktop clients' clipboard protocol, adopted on Android (text-only v1):
opt-in ClipControl at stream start when the host advertises HOST_CAP_CLIPBOARD
and the new "Shared clipboard" setting (default on) allows.

Device → host: local copies (primary-clip listener + a probe at start) are
announced as lazy format-list offers; the text crosses only when the host
actually pastes (FetchRequest → served from the live clipboard). Host →
device: a host copy's offer is fetched eagerly and lands in the system
clipboard (Android has no practical lazy-paste provider), with an echo guard
so the resulting clip-changed callback doesn't bounce it back as a new offer.

Native side: session/clipboard.rs JNI shims over NativeClient's clip_*
surface; events cross to Kotlin as compact strings from a blocking
nativeNextClip poll on a dedicated thread (the nativeNextRumble pattern),
joined before the session handle is freed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 19:27:19 +02:00
enricobuehlerandClaude Fable 5 abe6228b42 feat(core+host+android): committed-text input plane (IME path) — InputKind::TextInput
The VK key-event vocabulary cannot express text an input method COMMITS
(autocorrect, gesture typing, non-Latin scripts, emoji). Add a first-class
text event and negotiate it:

- punktfunk-core: InputKind::TextInput (= 15) carries one Unicode scalar per
  event in `code`; HOST_CAP_TEXT_INPUT (0x04) in Welcome::host_caps.
- Host advertises the cap only where the session's inject backend can type
  text: Windows SendInput (KEYEVENTF_UNICODE, surrogate-pair aware) and the
  Linux wlroots backend — a dedicated second zwp_virtual_keyboard whose xkb
  keymap grows Unicode keysyms on demand (the wtype model), so keymap
  re-uploads never disturb the main device's layout/modifier state. The
  KWin-fake-input/libei/gamescope backends can only press layout keycodes, so
  those sessions don't set the bit and clients keep the VK fallback.
- GameStream plane: Moonlight's UTF-8 text packet (MAGIC_UTF8, previously
  recognized-and-dropped) now decodes to the same TextInput events.
- Android: KeyCaptureView picks a real editable InputConnection when the host
  has the cap — the IME runs its full machinery, mirrored to the host live via
  common-prefix diffs of the composition (backspaces + new suffix), with
  setComposingRegion adopting committed text so autocorrect-revert flows diff
  instead of retyping; newline→Enter, deleteSurroundingText→Backspace/Delete.
  Older hosts keep the TYPE_NULL raw-key path unchanged.
- keymap: media VKs (0xB0-0xB3) → evdev so the Android media keys land on
  Linux hosts too.

Verified: punktfunk-core + host gamestream + pf-inject tests green on Linux
(Ubuntu box), clippy clean; Android app+native builds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 19:19:11 +02:00
enricobuehlerandClaude Fable 5 ddb72edcba feat(android): physical mouse + TV remote-as-pointer + keyboard hardening
Phase A — hardware mouse: new MouseForwarder routes every SOURCE_MOUSE event
while streaming. Uncaptured (default): hover/drag positions forward as absolute
cursor moves, wheel with fractional accumulation (high-res wheels), all five
buttons incl. back/forward as X1/X2; the local cursor is hidden over the stream
(TYPE_NULL pointer icon) so the host's composited cursor is the visible one.
Captured ("Capture pointer for games" setting, or Ctrl+Alt+Shift+Q live): the
OS pointer is grabbed via requestPointerCapture on the (repurposed) key-capture
view and raw relative deltas forward as mouse-look; held buttons are tracked
and flushed on capture loss/teardown, and the engaging click is swallowed
(desktop parity). Mouse clicks are intercepted before Compose so they can never
be misread as trackpad taps; a mouse back-button's FALLBACK BACK is dropped so
it can't yank the user out of the stream.

Phase B — keyboard: media/consumer keys (play/pause/next/prev/stop) now map to
their VKs instead of falling to the system; TV-remote SELECT maps to Enter.

Phase D — Android TV: RemotePointer turns a D-pad-only remote into a pointer
(Siri-remote analogue): hold SELECT ~0.8s toggles, D-pad glides the host cursor
with ramped acceleration (Choreographer-paced, diagonal-normalized), SELECT
taps click, play/pause right-clicks, holding it toggles the on-screen keyboard,
BACK leaves pointer mode; an overlay hint teaches the vocabulary.

Settings: "Capture pointer for games" + "Invert scroll direction" (applies
to the wheel and two-finger touch scrolling).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 18:56:38 +02:00
enricobuehlerandClaude Fable 5 5e088af7ed fix(pf-win-display): unpin mode indexes on paths the CCD isolate deactivates
ci / web (push) Successful in 49s
apple / swift (push) Successful in 1m24s
ci / docs-site (push) Successful in 55s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
apple / screenshots (push) Successful in 6m33s
ci / bench (push) Successful in 6m41s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m23s
android / android (push) Successful in 12m50s
docker / deploy-docs (push) Successful in 25s
deb / build-publish-host (push) Successful in 10m10s
arch / build-publish (push) Successful in 17m18s
deb / build-publish (push) Successful in 12m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m3s
ci / rust (push) Successful in 27m3s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m32s
Field report: with Exclusive Topology, the physical monitor sometimes stayed
lit — every isolate attempt re-committed and failed SetDisplayConfig with
0x57 ERROR_INVALID_PARAMETER. Clearing DISPLAYCONFIG_PATH_ACTIVE alone leaves
the deactivated path's source/target modeInfoIdx pinned to the queried mode
entries, and per the SetDisplayConfig contract a path being turned off must
have BOTH mode indexes set to DISPLAYCONFIG_PATH_MODE_IDX_INVALID; some
driver/topology combinations (notably when the doomed path shares a source
with the kept virtual path) reject the supplied config outright, so the
retry loop could never converge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 12:26:44 +02:00
enricobuehlerandClaude Fable 5 b7a00137eb feat(host): hold a logind sleep/idle inhibitor while clients stream
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 1m0s
android / android (push) Failing after 1m6s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 48s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 52s
ci / bench (push) Successful in 8m13s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m40s
deb / build-publish (push) Successful in 11m0s
release / apple (push) Successful in 9m50s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10m19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m57s
docker / deploy-docs (push) Successful in 35s
deb / build-publish-host (push) Successful in 14m3s
apple / screenshots (push) Successful in 6m27s
arch / build-publish (push) Successful in 17m49s
windows-host / package (push) Successful in 18m15s
flatpak / build-publish (push) Failing after 8m25s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m24s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m37s
ci / rust (push) Successful in 25m51s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m48s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m52s
Remote INPUT resets the compositor's idle timers, but a passive
(video-only) viewer sends none — observed live: a SteamOS Game-Mode
host s2idled mid-day and dropped off the network (in a VM with GPU
passthrough it never woke again). Refcounted across planes: the native
LiveSessionGuard and the GameStream video worker each hold a share;
first hold takes the logind sleep:idle block inhibitor on a dedicated
plain thread (zbus blocking must not run on a tokio worker), last drop
releases. Best-effort: no logind → warn once and stream on. No-op off
Linux (macOS/Windows manage their own assertions).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 12:09:47 +02:00
enricobuehlerandClaude Fable 5 acce43ebbf feat(steamdeck): self-healing reliability — post-OS-update rebuild check + script/docs polish
- rebuild-check.sh + punktfunk-rebuild-check.service (enabled, ordered
  Before=punktfunk-host): ldd-probes the host binary at session start —
  milliseconds when healthy, a full update.sh rebuild only when a
  SteamOS update actually broke its library links. update.sh restarts
  go --no-block so the check → update.sh → restart chain can't deadlock
  against the unit ordering. update.sh retrofits the unit.
- installer summary: web console is https (the unit serves TLS).
- docs: steamos-host.md (runner in the build step, keep-list + auto
  rebuild = updates survive hands-free), gamescope.md (Gaming Mode
  touch = single-finger pointer, exact taps, no multi-touch),
  plugins.mdx (SteamOS runner note), scripts/steamdeck/README.md
  (rebuild-check, runner payload, keep list, honest packaging
  trade-off + the CI-prebuilt future direction).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 11:56:09 +02:00
enricobuehlerandClaude Fable 5 35923080fb fix(host/linux): scale degenerate-region absolute coords by the session's real output size
The raw-client-pixels fallback is exact only while the managed session
runs at the client's mode. When they diverge — foreign-gamescope
attach at its own resolution, supersample/under-render, transitions —
raw pixels drift or land out of range. The EIS relay file now carries
the session's current output size as a second "WxH" line (from
current_gamescope_output_size()); the injector scales normalized
client positions into it, keeping raw pixels only as the last resort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:53:14 +02:00
enricobuehlerandClaude Fable 5 0c9461242c fix(host/linux): don't normalize absolute coords into gamescope's degenerate INT32_MAX EIS region
The previous commit's no-region fallback never fired: gamescope DOES
advertise a region — (0,0,INT32_MAX,INT32_MAX), meaning "coordinates
are raw". Mapping normalized positions into it explodes a center tap
to x≈1e9, clamped by gamescope to the far corner — the cursor pinned
at (1279,799) and every degraded-touch tap landing there. Treat a
region as an output geometry only when plausibly sized (≤16384 px);
otherwise emit raw client pixels.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:40:02 +02:00
enricobuehlerandClaude Fable 5 cc6b37fef0 fix(host/linux): emit absolute pointer/touch on region-less EIS devices + abs input-test probe
gamescope's "Gamescope Virtual Input" advertises pointer_abs but no
region, so every MouseMoveAbs (and the degraded-touch moves built on
it) was silently dropped (emitted=false) — clicks then landed at a
stale cursor position. With no region, the managed session runs at the
client's mode, so client pixels are output pixels: emit them raw.
Also: log region count/dims at device-resume, and add
PUNKTFUNK_INPUT_TEST_ABS=WxH to `input-test` (corners + center, 1s
apart) so the degraded-touch path is verifiable with xdotool alone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:36:05 +02:00
enricobuehlerandClaude Fable 5 1927077cbe fix(host/linux): degrade touch to absolute pointer when the EIS has no touchscreen device
gamescope's EIS ("Gamescope Virtual Input") advertises pointer/
pointer_abs/button but never a touchscreen — so in Game Mode every
remote TouchDown/Move/Up was dropped on the floor (Desktop Mode works
via KwinFakeInput, which does touch). Headless KWin's libei has the
same gap. Instead of dropping: degrade to a single-finger absolute
pointer — down = abs-move + left press, move = abs-move, up = release —
synthesized through the normal Mouse* inject paths so region mapping,
held-state tracking, and release_all apply. First finger drives the
pointer; later fingers are ignored (a pinch degrades to a drag). A
touchscreen device appearing later takes over on the next touch —
checked per event, never latched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:24:28 +02:00
enricobuehlerandClaude Fable 5 068da456f5 fix(steamdeck): survive SteamOS A/B updates — register system tuning on the atomic-update keep list
Verified live on a SteamOS 3.8.14→3.8.16 update: the A/B partset switch
rebuilds /etc and drops everything not on Valve's keep list — the udev
rule (uhid access: virtual pads silently degrade to Xbox 360), the
vhci-hcd autoload (native Deck pad transport), and the UDP buffer sysctl
(back to 208 KB → stutter). Valve's sanctioned extension point is a
drop-in under /etc/atomic-update.conf.d/ — itself on the stock keep
list, so it self-preserves. install.sh §4 and update.sh now register
scripts/punktfunk-atomic-keep.conf there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:07:19 +02:00
enricobuehlerandClaude Fable 5 256aa5f7f2 fix(steamdeck): install the plugin runner + harden Deck pad typing + repair update.sh
- scripts/steamdeck/install.sh (§2b) + update.sh: build and install the
  scripting runner user-scoped (~/.local wrapper + pinned bun + bundle) —
  SteamOS's read-only /usr can't take the .deb layout, so the console's
  plugin store reported "the plugin runner isn't installed" on every
  SteamOS host. update.sh retrofits it onto existing installs.
- plugins.rs runner discovery: check the ~/.local layout after the /usr
  ones; mention the SteamOS path in the not-installed error.
- update.sh: define warn() — it was used but never defined, so under
  `set -e` the first warn aborted the update before services restarted.
- pf-client-core gamepad: a Steam Input virtual pad forwarded on a real
  Deck (the only-pad case) now declares the DECK kind in its per-pad
  arrival instead of the wrapper's Xbox 360 identity — the session-level
  auto_pref already resolved SteamDeck, but the arrival overrides it on
  current hosts.
- reject tests: cover SetupFailed; move the foreign-code probe off 0x68.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:01:54 +02:00
enricobuehlerandClaude Fable 5 2562663fc6 fix(host/linux): survive and self-heal dead gamescope sessions on headless SteamOS boxes
Three fixes for the "most connects simply fail" trap on a SteamOS host
with no physical display (VM testbox, panel-less mini PC):

- restore path: skip the restore-to-physical-panel when no DRM connector
  reports a connected display — removing the headless drop-in and
  restarting gamescope-session.target on such a box just crash-loops
  gamescope and strands every later connect on "no usable compositor".
  Keep the headless session (and the takeover state) instead.
- connect path: when no live graphical session exists but the MANAGED
  gamescope infra is present (SteamOS gamescope-session / Bazzite
  session-plus), route to the gamescope takeover — which rebuilds the
  session at the client's mode — instead of failing the connect.
- error delivery: a session-setup failure used to just drop the QUIC
  connection, reaching the client as "control stream finished mid-frame"
  (indistinguishable from transport trouble). Close with a typed
  setup-failed code (0x68, RejectedSetupFailed = -29) carrying the error
  text, and give the client handshake/pairing error paths a short grace
  for the CONNECTION_CLOSE to beat the stream error it caused.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 09:50:30 +02:00
185 changed files with 19560 additions and 2475 deletions
+12 -2
View File
@@ -46,13 +46,23 @@ jobs:
# SHA-pinned: this workflow's release job carries the signing keystore + Play service-account # SHA-pinned: this workflow's release job carries the signing keystore + Play service-account
# secrets, so a moved tag on a third-party action could exfiltrate them. v3 = 9fc6c4e. # secrets, so a moved tag on a third-party action could exfiltrate them. v3 = 9fc6c4e.
uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3 uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3
with:
# Only platform-tools — NOT the action's default legacy `tools`, whose dependency chain
# drags in the ~250 MB emulator nobody here runs (instrumentation tests are deferred).
# That download was the single flakiest piece of this job: the shared runner fleet drops
# packets under parallel-job load and sdkmanager's streamed unzip turns a truncated
# stream into "Error on ZipFile unknown archive" (observed 2026-07-22, twice).
packages: platform-tools
- name: NDK r30 + platform 36 + build-tools + CMake (libopus cross-build) - name: NDK r30 + platform 36 + build-tools + CMake (libopus cross-build)
# cmake;3.22.1 installs cmake + ninja under $ANDROID_SDK/cmake/3.22.1/bin — the exact path # cmake;3.22.1 installs cmake + ninja under $ANDROID_SDK/cmake/3.22.1/bin — the exact path
# kit/build.gradle.kts prepends to PATH for cargo-ndk's audiopus_sys (libopus) CMake build. # kit/build.gradle.kts prepends to PATH for cargo-ndk's audiopus_sys (libopus) CMake build.
# Note: platforms;android-37 is sometimes missing from standard channels; AGP will # Note: platforms;android-37 is sometimes missing from standard channels; AGP will
# auto-download it if needed during the build. # auto-download it if needed during the build.
run: sdkmanager "platform-tools" "platforms;android-36" "build-tools;37.0.0" "ndk;30.0.14904198" "cmake;3.22.1" # retry.sh: sdkmanager is a single-shot multi-hundred-MB fetch, exactly the class the
# helper exists for (fleet-load packet drops truncate the stream mid-unzip); a failed
# attempt leaves no partial package behind, so a plain re-invoke is safe.
run: bash scripts/ci/retry.sh 4 sdkmanager "platform-tools" "platforms;android-36" "build-tools;37.0.0" "ndk;30.0.14904198" "cmake;3.22.1"
- name: Caches (cargo + gradle) - name: Caches (cargo + gradle)
uses: actions/cache@v4 uses: actions/cache@v4
+39
View File
@@ -0,0 +1,39 @@
# Announce a stable release to the Discord #releases channel.
#
# This is the deliberate "go" step for a release. Release notes live in the repo at
# docs/releases/<tag>.md and are seeded into the Gitea release body at creation by the build
# workflows (scripts/ci/gitea-release.sh), so the release is never noteless. Once every
# platform's CI is green for a tag, dispatch this workflow with that tag: it re-asserts the notes
# file over the live release and posts a formatted embed to #releases.
#
# Manual on purpose — pressing "go" is the quality gate that says "all platforms built, notes are
# final, tell the community." It is NOT wired to the tag push, so a half-built or failed release
# is never announced. Stable-only: a -rc/pre-release tag is refused unless allow_prerelease=true.
#
# Requires the repo secret DISCORD_RELEASE_WEBHOOK (the #releases channel webhook URL); GITEA auth
# reuses REGISTRY_TOKEN like the other release workflows.
name: announce
on:
workflow_dispatch:
inputs:
tag:
description: "Release tag to announce (e.g. v0.18.0)"
required: true
allow_prerelease:
description: "Announce even if the tag is a pre-release (-rc)"
required: false
default: "false"
jobs:
announce:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Post release announcement to Discord
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
DISCORD_RELEASE_WEBHOOK: ${{ secrets.DISCORD_RELEASE_WEBHOOK }}
ALLOW_PRERELEASE: ${{ inputs.allow_prerelease }}
run: bash scripts/ci/discord-announce.sh "${{ inputs.tag }}"
+12 -25
View File
@@ -73,33 +73,20 @@ jobs:
# sufficient — the Tooling step's dnf install pulls a systemd package upgrade whose RPM # sufficient — the Tooling step's dnf install pulls a systemd package upgrade whose RPM
# trigger re-runs authselect and regenerates this file, undoing the fix. It's reapplied # trigger re-runs authselect and regenerates this file, undoing the fix. It's reapplied
# there, right before the first `flatpak` network call. # there, right before the first `flatpak` network call.
- name: Fix container DNS (drop nss-resolve, resolve over TCP) - name: Fix container DNS (drop nss-resolve)
run: | run: |
sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf
# Resolve over TCP instead of UDP. The documented root cause of the flathub # History: this step used to ALSO force glibc onto TCP DNS (`options use-vc`) because
# bootstrap failures (investigated 2026-07-11, see the Tooling step) is this box's # the runner fleet's Docker embedded resolver dropped UDP lookups under parallel-job
# Docker embedded resolver at 127.0.0.11 DROPPING UDP lookups while the shared # load (investigated 2026-07-11; v0.15.0/v0.16.0 each burned retry.sh's whole budget).
# runner fleet is saturated — a datagram nobody retransmits, so the lookup just # That root cause is now fixed at the infra level (2026-07-22): the runner host runs a
# times out. The answer then was to widen retry.sh's budget to 10 attempts (~9 min), # local dnsmasq cache on the docker bridge and daemon.json points every job container
# which is enough to outlast a main push's ~8-workflow fan-out but NOT a TAG push's # at it, so lookups terminate on-box instead of crossing the saturated uplink — the
# 13: v0.15.0 (twice) and v0.16.0 each burned all 10 attempts and failed the job, # UDP path is reliable again. The TCP path through the same chain proved FLAKY under
# each needing a manual re-run. # fleet concurrency (flatpak remote-add failed 10/10 with instant NXDOMAIN while dnf
# # in the same container resolved fine), so `use-vc` flipped from mitigation to sole
# `use-vc` makes glibc use TCP, where the kernel retransmits and the query cannot be # cause of this leg's failures — removed. retry.sh (10×) stays as the backstop for
# silently lost under load. Same resolver, same search path — only the transport # genuine upstream blips.
# changes, so internal names (git.unom.io) resolve exactly as before; deliberately
# NO extra nameservers, which would risk answering an internal name from a public
# resolver. Docker's embedded DNS serves TCP on 127.0.0.11:53 as well as UDP.
# retry.sh stays as the backstop for genuine upstream blips.
#
# Non-fatal: Docker bind-mounts /etc/resolv.conf and can present it read-only, and a
# DNS tuning that cannot be applied must not be what fails the release build — that
# would trade an occasional re-run for a hard stop. Falling back to UDP just restores
# today's behaviour, which retry.sh already covers.
if ! grep -q '^options .*use-vc' /etc/resolv.conf 2>/dev/null; then
echo 'options use-vc timeout:3 attempts:3' >> /etc/resolv.conf \
|| echo "::warning::could not set use-vc (read-only resolv.conf?); staying on UDP"
fi
cat /etc/resolv.conf || true cat /etc/resolv.conf || true
# fedora:43 has no node, but actions/checkout (a JS action) needs it. A plain `run:` step # fedora:43 has no node, but actions/checkout (a JS action) needs it. A plain `run:` step
+70
View File
@@ -393,6 +393,76 @@ jobs:
-authenticationKeyID "${{ secrets.ASC_API_KEY_ID }}" \ -authenticationKeyID "${{ secrets.ASC_API_KEY_ID }}" \
-authenticationKeyIssuerID "${{ secrets.ASC_API_ISSUER_ID }}" -authenticationKeyIssuerID "${{ secrets.ASC_API_ISSUER_ID }}"
- name: iOS — export .ipa (Gitea release + run artifact)
# The TestFlight step above uploads straight to App Store Connect (destination=upload) and
# leaves NO .ipa on disk. Re-export the SAME archive with destination=export to get an
# App Store distribution-signed .ipa for the Gitea release + the run artifacts. Same gate as
# that archive; a warn+skip (never fails the best-effort iOS leg) if the archive is absent,
# e.g. a workflow_dispatch with testflight=false. NOTE: an App Store-signed .ipa installs
# only via TestFlight/App Store, not by direct sideload — it's a release/archival artifact.
if: gitea.event_name != 'workflow_dispatch' || inputs.testflight == 'true'
id: ios_ipa
run: |
ARCHIVE="$RUNNER_TEMP/Punktfunk-ios.xcarchive"
if [ ! -d "$ARCHIVE" ]; then
echo "::warning::iOS archive not found — skipping .ipa export"
exit 0
fi
PROFILE="Punktfunk iOS App Store Distribution"
WIDGET_PROFILE="Punktfunk iOS Widgets App Store Distribution"
# destination=export writes the .ipa to -exportPath; otherwise identical manual signing to
# the upload plist (both profiles, Apple Distribution). No ASC key needed — no network.
cat > "$RUNNER_TEMP/export-appstore-ipa.plist" <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>method</key><string>app-store-connect</string>
<key>destination</key><string>export</string>
<key>teamID</key><string>$TEAM_ID</string>
<key>signingStyle</key><string>manual</string>
<key>signingCertificate</key><string>Apple Distribution</string>
<key>provisioningProfiles</key>
<dict>
<key>io.unom.punktfunk</key><string>$PROFILE</string>
<key>io.unom.punktfunk.widgets</key><string>$WIDGET_PROFILE</string>
</dict>
</dict>
</plist>
EOF
DEVELOPER_DIR="$XCODE_DEV_DIR" xcodebuild -exportArchive \
-archivePath "$ARCHIVE" \
-exportOptionsPlist "$RUNNER_TEMP/export-appstore-ipa.plist" \
-exportPath "$RUNNER_TEMP/export-ipa"
SRC=$(ls "$RUNNER_TEMP/export-ipa/"*.ipa 2>/dev/null | head -1)
[ -n "$SRC" ] || { echo "::warning::no .ipa was produced by export"; exit 0; }
mkdir -p "$GITHUB_WORKSPACE/dist"
IPA="$GITHUB_WORKSPACE/dist/Punktfunk-$VERSION.ipa"
mv "$SRC" "$IPA"
echo "IPA=$IPA" >> "$GITHUB_ENV"
echo "ipa=dist/Punktfunk-$VERSION.ipa" >> "$GITHUB_OUTPUT"
echo "exported $IPA"
- name: Attach .ipa to the workflow run
if: steps.ios_ipa.outputs.ipa != ''
# v3, not v4: Gitea's artifact backend identifies as GHES, which upload-artifact@v4 refuses
# (same reason as android.yml / apple.yml). Download is a zip of the .ipa.
uses: actions/upload-artifact@v3
with:
name: punktfunk-ios-ipa
path: ${{ steps.ios_ipa.outputs.ipa }}
if-no-files-found: warn
retention-days: 30
- name: Attach .ipa to the Gitea release (stable tags only)
if: startsWith(gitea.ref, 'refs/tags/v') && steps.ios_ipa.outputs.ipa != ''
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
. scripts/ci/gitea-release.sh
RID=$(ensure_release "$GITHUB_REF_NAME" "$GITHUB_REF_NAME" auto)
upsert_asset "$RID" "$IPA" "Punktfunk-$VERSION.ipa"
- name: tvOS — archive + upload to TestFlight - name: tvOS — archive + upload to TestFlight
# Canary + stable, the same track as iOS/macOS — the tvOS xcframework slice is now built # Canary + stable, the same track as iOS/macOS — the tvOS xcframework slice is now built
# on every apple push (above), so this matches the iOS step's gate exactly. # on every apple push (above), so this matches the iOS step's gate exactly.
Generated
+57 -27
View File
@@ -1459,6 +1459,16 @@ dependencies = [
"version_check", "version_check",
] ]
[[package]]
name = "gethostname"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bd49230192a3797a9a4d6abe9b3eed6f7fa4c8a8a4947977c6f80025f92cbd8"
dependencies = [
"rustix",
"windows-link 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]] [[package]]
name = "getrandom" name = "getrandom"
version = "0.2.17" version = "0.2.17"
@@ -2184,7 +2194,7 @@ dependencies = [
[[package]] [[package]]
name = "latency-probe" name = "latency-probe"
version = "0.17.2" version = "0.19.2"
[[package]] [[package]]
name = "lazy_static" name = "lazy_static"
@@ -2289,7 +2299,7 @@ dependencies = [
[[package]] [[package]]
name = "libvpl-sys" name = "libvpl-sys"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
@@ -2324,7 +2334,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]] [[package]]
name = "loss-harness" name = "loss-harness"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"punktfunk-core", "punktfunk-core",
] ]
@@ -2813,7 +2823,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]] [[package]]
name = "pf-capture" name = "pf-capture"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2829,11 +2839,12 @@ dependencies = [
"tokio", "tokio",
"tracing", "tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)", "windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"x11rb",
] ]
[[package]] [[package]]
name = "pf-client-core" name = "pf-client-core"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2857,7 +2868,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-clipboard" name = "pf-clipboard"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2875,7 +2886,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-console-ui" name = "pf-console-ui"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2896,7 +2907,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-encode" name = "pf-encode"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2920,7 +2931,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-ffvk" name = "pf-ffvk"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"ash", "ash",
"bindgen", "bindgen",
@@ -2929,7 +2940,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-frame" name = "pf-frame"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"libc", "libc",
@@ -2941,7 +2952,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-gpu" name = "pf-gpu"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-host-config", "pf-host-config",
@@ -2955,11 +2966,11 @@ dependencies = [
[[package]] [[package]]
name = "pf-host-config" name = "pf-host-config"
version = "0.17.2" version = "0.19.2"
[[package]] [[package]]
name = "pf-inject" name = "pf-inject"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2970,6 +2981,7 @@ dependencies = [
"pf-driver-proto", "pf-driver-proto",
"pf-host-config", "pf-host-config",
"pf-paths", "pf-paths",
"pf-win-display",
"punktfunk-core", "punktfunk-core",
"reis", "reis",
"tokio", "tokio",
@@ -2987,14 +2999,14 @@ dependencies = [
[[package]] [[package]]
name = "pf-paths" name = "pf-paths"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"tracing", "tracing",
] ]
[[package]] [[package]]
name = "pf-presenter" name = "pf-presenter"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -3009,10 +3021,11 @@ dependencies = [
[[package]] [[package]]
name = "pf-vdisplay" name = "pf-vdisplay"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
"bitflags",
"bytemuck", "bytemuck",
"futures-util", "futures-util",
"hex", "hex",
@@ -3039,7 +3052,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-win-display" name = "pf-win-display"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-paths", "pf-paths",
@@ -3051,7 +3064,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-zerocopy" name = "pf-zerocopy"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -3258,7 +3271,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-android" name = "punktfunk-client-android"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"android_logger", "android_logger",
"jni", "jni",
@@ -3274,7 +3287,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-linux" name = "punktfunk-client-linux"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-channel", "async-channel",
@@ -3290,7 +3303,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-session" name = "punktfunk-client-session"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-client-core", "pf-client-core",
@@ -3305,7 +3318,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-windows" name = "punktfunk-client-windows"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"async-channel", "async-channel",
"ffmpeg-next", "ffmpeg-next",
@@ -3324,7 +3337,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-core" name = "punktfunk-core"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"aes-gcm", "aes-gcm",
"bytes", "bytes",
@@ -3356,7 +3369,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-host" name = "punktfunk-host"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"aes", "aes",
"aes-gcm", "aes-gcm",
@@ -3440,7 +3453,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-probe" name = "punktfunk-probe"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"mdns-sd", "mdns-sd",
@@ -3454,7 +3467,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-tray" name = "punktfunk-tray"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ksni", "ksni",
@@ -3477,7 +3490,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]] [[package]]
name = "pyrowave-sys" name = "pyrowave-sys"
version = "0.17.2" version = "0.19.2"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
@@ -5886,6 +5899,23 @@ version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4" checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4"
[[package]]
name = "x11rb"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9993aa5be5a26815fe2c3eacfc1fde061fc1a1f094bf1ad2a18bf9c495dd7414"
dependencies = [
"gethostname",
"rustix",
"x11rb-protocol",
]
[[package]]
name = "x11rb-protocol"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea6fc2961e4ef194dcbfe56bb845534d0dc8098940c7e5c012a258bfec6701bd"
[[package]] [[package]]
name = "x509-parser" name = "x509-parser"
version = "0.16.0" version = "0.16.0"
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" } ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package] [workspace.package]
version = "0.17.2" version = "0.19.2"
edition = "2021" edition = "2021"
rust-version = "1.82" rust-version = "1.82"
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
+1 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0", "name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0" "identifier": "MIT OR Apache-2.0"
}, },
"version": "0.17.2" "version": "0.18.0"
}, },
"paths": { "paths": {
"/api/v1/clients": { "/api/v1/clients": {
@@ -0,0 +1,107 @@
package io.unom.punktfunk
import android.content.ClipData
import android.content.ClipboardManager
import android.content.Context
import android.os.Handler
import android.os.Looper
import io.unom.punktfunk.kit.NativeBridge
/**
* Text clipboard sync for the active session (the desktop-client model, text-only v1):
* * **Device → host**: a local copy (the primary-clip listener, plus one probe at start) is
* announced as a lazy offer — the text crosses only when the host actually pastes (a
* `fetch:` event, answered with the clipboard's current content).
* * **Host → device**: a host copy arrives as an `offer:` event and is fetched eagerly into
* the system clipboard (Android apps can't lazily materialize a paste from the network
* without a content-provider round-trip that isn't worth it here).
*
* Loop guard: text set from a host fetch is remembered ([lastFromHost]) so the resulting
* primary-clip-changed callback doesn't bounce it straight back as a new offer. Clipboard reads
* happen while the stream is foreground (Android only allows focused-app reads). The native
* events are drained on a dedicated thread and applied on the main thread; [stop] joins it.
*/
class ClipboardSync(
private val context: Context,
private val handle: Long,
) {
private val main = Handler(Looper.getMainLooper())
private val cm = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
@Volatile private var running = true
private var seq = 0
private var lastOffered: String? = null
private var lastFromHost: String? = null
private var pendingFetch = -1
private var thread: Thread? = null
private val clipListener = ClipboardManager.OnPrimaryClipChangedListener { offerLocal() }
fun start() {
NativeBridge.nativeClipControl(handle, true)
cm.addPrimaryClipChangedListener(clipListener)
thread = Thread({ pollLoop() }, "pf-clipboard").also { it.start() }
offerLocal() // whatever is already on the clipboard is pasteable host-side right away
}
fun stop() {
running = false
cm.removePrimaryClipChangedListener(clipListener)
thread?.join(600) // one poll timeout (250 ms) + slack
thread = null
}
/** Announce the current local text (if it's new and not an echo of a host copy). */
private fun offerLocal() {
if (!running) return
val text = currentClipText() ?: return
if (text == lastOffered || text == lastFromHost) return
lastOffered = text
seq += 1
NativeBridge.nativeClipOfferText(handle, seq)
}
private fun currentClipText(): String? = runCatching {
cm.primaryClip?.takeIf { it.itemCount > 0 }?.getItemAt(0)
?.coerceToText(context)?.toString()?.takeIf { it.isNotEmpty() }
}.getOrNull()
private fun pollLoop() {
while (running) {
val ev = NativeBridge.nativeNextClip(handle) ?: continue
if (ev == "closed") return
main.post { handleEvent(ev) }
}
}
private fun handleEvent(ev: String) {
if (!running) return
val parts = ev.split(":", limit = 3)
when (parts[0]) {
"offer" -> {
val offerSeq = parts.getOrNull(1)?.toIntOrNull() ?: return
if (parts.getOrNull(2) == "1") {
pendingFetch = NativeBridge.nativeClipFetchText(handle, offerSeq)
}
}
"fetch" -> {
val req = parts.getOrNull(1)?.toIntOrNull() ?: return
val text = currentClipText()
if (text != null) {
NativeBridge.nativeClipServeText(handle, req, text)
} else {
NativeBridge.nativeClipCancel(handle, req)
}
}
"data" -> {
val xfer = parts.getOrNull(1)?.toIntOrNull() ?: return
if (xfer != pendingFetch) return // stale/unknown transfer
pendingFetch = -1
val text = parts.getOrNull(2)?.takeIf { it.isNotEmpty() } ?: return
lastFromHost = text
runCatching { cm.setPrimaryClip(ClipData.newPlainText("Punktfunk", text)) }
}
// "state"/"cancel"/"error": nothing to drive in the text-only v1.
}
}
}
@@ -54,6 +54,21 @@ class MainActivity : ComponentActivity() {
var padKeyProbe: ((KeyEvent) -> Boolean)? = null var padKeyProbe: ((KeyEvent) -> Boolean)? = null
var padMotionProbe: ((MotionEvent) -> Boolean)? = null var padMotionProbe: ((MotionEvent) -> Boolean)? = null
/**
* Physical-mouse forwarder for the active session (built/released by StreamScreen, like
* [gamepadRouter]): uncaptured hover/click/wheel forwards as absolute cursor input, captured
* ([android.view.View.requestPointerCapture]) raw deltas as relative mouse-look. The dispatch
* overrides below route every SOURCE_MOUSE event here while streaming. Null while not streaming.
*/
var mouseForwarder: MouseForwarder? = null
/**
* TV remote-as-pointer for the active session (StreamScreen builds it on TV devices only):
* hold SELECT to toggle, then the D-pad glides the host cursor. Consulted first for
* non-gamepad keys while streaming. Null while not streaming or not a TV.
*/
var remotePointer: RemotePointer? = null
/** /**
* Set by [StreamScreen] to its disconnect action. The emergency-exit chord (below) invokes it so a * Set by [StreamScreen] to its disconnect action. The emergency-exit chord (below) invokes it so a
* couch user with no keyboard/Back can always leave a stream. * couch user with no keyboard/Back can always leave a stream.
@@ -324,9 +339,37 @@ class MainActivity : ComponentActivity() {
return true // consumed return true // consumed
} }
} }
// TV remote-as-pointer sees non-gamepad keys first (SELECT long-press toggles it;
// while active it owns the D-pad/SELECT/PLAY-PAUSE/BACK).
if (!event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
remotePointer?.let { if (it.onKey(event)) return true }
}
// Ctrl+Alt+Shift+Q — the cross-client pointer-capture toggle chord. Swallow both
// edges of the Q (the modifiers already went over the wire, exactly like desktop).
if (event.keyCode == KeyEvent.KEYCODE_Q &&
event.isCtrlPressed && event.isAltPressed && event.isShiftPressed
) {
if (event.action == KeyEvent.ACTION_DOWN && event.repeatCount == 0) {
mouseForwarder?.toggleCapture()
}
return true
}
when (event.keyCode) { when (event.keyCode) {
// A mouse's back/forward buttons already go over the wire as X1/X2 via their
// BUTTON_* motion edges — but Android ALSO delivers them as key events: the input
// reader synthesizes KEYCODE_BACK/FORWARD (stamped SOURCE_MOUSE) unconditionally,
// and a view-level FALLBACK BACK appears when the BUTTON_* press goes unconsumed.
// Swallow every such duplicate or it doubles as Android navigation and yanks the
// user out of the stream. A remote/keyboard BACK is never mouse-sourced, so it
// still falls through to the BackHandler and exits.
KeyEvent.KEYCODE_BACK, KeyEvent.KEYCODE_FORWARD ->
if (event.isFromSource(InputDevice.SOURCE_MOUSE) ||
event.flags and KeyEvent.FLAG_FALLBACK != 0
) {
return true
}
// Leave these to the system even while streaming. // Leave these to the system even while streaming.
KeyEvent.KEYCODE_BACK, // → BackHandler leaves the stream // (BACK above → BackHandler leaves the stream.)
KeyEvent.KEYCODE_VOLUME_UP, KeyEvent.KEYCODE_VOLUME_UP,
KeyEvent.KEYCODE_VOLUME_DOWN, KeyEvent.KEYCODE_VOLUME_DOWN,
KeyEvent.KEYCODE_VOLUME_MUTE, KeyEvent.KEYCODE_VOLUME_MUTE,
@@ -394,6 +437,10 @@ class MainActivity : ComponentActivity() {
override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean { override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean {
if (streamHandle != 0L) { if (streamHandle != 0L) {
if (gamepadRouter?.onMotion(event) == true) return true if (gamepadRouter?.onMotion(event) == true) return true
// Physical mouse (uncaptured): hover motion, wheel, button edges.
if (event.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onGenericMotion(event)) return true }
}
return super.dispatchGenericMotionEvent(event) return super.dispatchGenericMotionEvent(event)
} }
// The Controllers debug screen sees pad motion before the stick→D-pad synthesis below. // The Controllers debug screen sees pad motion before the stick→D-pad synthesis below.
@@ -431,6 +478,24 @@ class MainActivity : ComponentActivity() {
return super.dispatchGenericMotionEvent(event) return super.dispatchGenericMotionEvent(event)
} }
/**
* Mouse clicks/drags ride the TOUCH stream (the pointer is "down"). While streaming they
* belong to the mouse forwarder, never to the Compose touch-gesture layer — a physical
* mouse click must be a real click at the cursor, not a synthesized trackpad tap.
*/
override fun dispatchTouchEvent(ev: MotionEvent): Boolean {
if (streamHandle != 0L && ev.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onTouchEvent(ev)) return true }
}
return super.dispatchTouchEvent(ev)
}
/** The OS is the source of truth for pointer capture (it releases on focus loss). */
override fun onPointerCaptureChanged(hasCapture: Boolean) {
super.onPointerCaptureChanged(hasCapture)
mouseForwarder?.onCaptureChanged(hasCapture)
}
/** Keys that drive the console UI — D-pad + face buttons; used to classify the last input source. */ /** Keys that drive the console UI — D-pad + face buttons; used to classify the last input source. */
private fun isConsoleNavKey(kc: Int): Boolean = when (kc) { private fun isConsoleNavKey(kc: Int): Boolean = when (kc) {
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN, KeyEvent.KEYCODE_DPAD_LEFT, KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN, KeyEvent.KEYCODE_DPAD_LEFT,
@@ -0,0 +1,206 @@
package io.unom.punktfunk
import android.view.InputDevice
import android.view.MotionEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.roundToInt
/** True when any connected input device is a pointer (USB/BT mouse, or a touchpad driving one). */
fun hasPhysicalMouse(): Boolean = InputDevice.getDeviceIds().any { id ->
InputDevice.getDevice(id)?.supportsSource(InputDevice.SOURCE_MOUSE) == true
}
/**
* Physical mouse → wire, in two modes (the iPadOS/desktop model):
* * **uncaptured** (default): hover/drag positions forward as absolute cursor moves
* (`MouseMoveAbs`, host-normalized against the window size) — desktop-style pointing. The
* local cursor is hidden over the stream (StreamScreen sets a TYPE_NULL pointer icon); the
* host's own cursor, composited into the video, is the one you see.
* * **captured**: the OS pointer is grabbed ([android.view.View.requestPointerCapture]) and raw
* relative deltas forward as `MouseMove` — FPS mouse-look. Engaged at stream start / by
* clicking into the stream when the "Capture pointer for games" setting is on, and toggled
* any time by Ctrl+Alt+Shift+Q (the cross-client chord). Focus loss releases it (the OS
* guarantees that); a click re-engages.
*
* Buttons ride [MotionEvent.ACTION_BUTTON_PRESS]/RELEASE edges (left/middle/right/back/forward →
* wire 1/2/3/4/5), the wheel rides [MotionEvent.ACTION_SCROLL] with fractional accumulation so
* high-resolution wheels don't lose sub-notch travel. Held buttons are tracked and flushed on
* capture loss / stream exit so nothing sticks on the host. Events reach this class from
* MainActivity's dispatch overrides (uncaptured) and the capture view's captured-pointer listener.
*/
class MouseForwarder(
private val handle: Long,
private val invertScroll: Boolean,
private val captureWanted: Boolean,
private val surfaceSize: () -> Pair<Int, Int>,
) {
/** Capture plumbing, owned by StreamScreen (the focusable capture view). */
var onRequestCapture: (() -> Unit)? = null
var onReleaseCapture: (() -> Unit)? = null
/** Live capture state, updated from [android.app.Activity.onPointerCaptureChanged]. */
var captured = false
private set
/** Chord-released: no auto re-engage (start / click) until the user opts back in. */
private var userReleased = false
private val heldButtons = mutableSetOf<Int>()
private var scrollAccV = 0f
private var scrollAccH = 0f
private var moveAccX = 0f
private var moveAccY = 0f
/** Uncaptured mouse events on the TOUCH stream (position while a button is down). */
fun onTouchEvent(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_DOWN -> {
if (captureWanted && !captured && !userReleased) {
// The engaging click: grab the pointer and swallow the click (desktop
// parity — the click that captures never reaches the host). The paired
// BUTTON_RELEASE is dropped by the held-set guard in [button].
onRequestCapture?.invoke()
return true
}
sendAbs(ev)
}
MotionEvent.ACTION_MOVE -> sendAbs(ev)
// Button edges are documented on the generic stream, but be robust to either.
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
}
return true
}
/** Uncaptured mouse events on the GENERIC stream (hover motion, wheel, button edges). */
fun onGenericMotion(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_HOVER_MOVE -> sendAbs(ev)
MotionEvent.ACTION_SCROLL -> wheel(ev)
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_EXIT -> {}
else -> return false
}
return true
}
/**
* Captured-pointer events (the view holds [android.view.View.requestPointerCapture]): x/y ARE
* the relative deltas ([InputDevice.SOURCE_MOUSE_RELATIVE]), batched samples included. A
* captured touchpad reports absolute finger coordinates instead — not handled (the touch
* gesture layer is the touchpad story); returning false leaves those to the framework.
*/
fun onCapturedPointer(ev: MotionEvent): Boolean {
if (!ev.isFromSource(InputDevice.SOURCE_MOUSE_RELATIVE)) return false
when (ev.actionMasked) {
MotionEvent.ACTION_MOVE -> {
var dx = 0f
var dy = 0f
for (i in 0 until ev.historySize) {
dx += ev.getHistoricalX(i)
dy += ev.getHistoricalY(i)
}
dx += ev.x
dy += ev.y
moveAccX += dx
moveAccY += dy
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept w/ sign
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_SCROLL -> wheel(ev)
}
return true
}
/** Ctrl+Alt+Shift+Q: release the grab, or (re-)engage it — works even when auto-capture is off. */
fun toggleCapture() {
if (captured) {
userReleased = true
onReleaseCapture?.invoke()
} else {
userReleased = false
onRequestCapture?.invoke()
}
}
/** Auto-engage at stream start (setting on + a mouse actually present). */
fun engageFromStart() {
if (captureWanted && !captured && !userReleased && hasPhysicalMouse()) {
onRequestCapture?.invoke()
}
}
/** From [android.app.Activity.onPointerCaptureChanged] — the OS is the source of truth. */
fun onCaptureChanged(has: Boolean) {
captured = has
// Losing the grab (focus loss, chord) must not leave buttons held on the host.
if (!has) flushButtons()
}
/** Stream teardown: lift anything held and let the grab go. */
fun release() {
flushButtons()
if (captured) onReleaseCapture?.invoke()
}
private fun sendAbs(ev: MotionEvent) {
val (w, h) = surfaceSize()
if (w <= 0 || h <= 0) return
NativeBridge.nativeSendPointerAbs(
handle,
ev.x.roundToInt().coerceIn(0, w - 1),
ev.y.roundToInt().coerceIn(0, h - 1),
w,
h,
)
}
private fun wheel(ev: MotionEvent) {
val dir = if (invertScroll) -1f else 1f
// Android: AXIS_VSCROLL + = up/away, AXIS_HSCROLL + = right — the wire's convention too.
scrollAccV += ev.getAxisValue(MotionEvent.AXIS_VSCROLL) * 120f * dir
scrollAccH += ev.getAxisValue(MotionEvent.AXIS_HSCROLL) * 120f * dir
val v = scrollAccV.toInt()
if (v != 0) {
NativeBridge.nativeSendScroll(handle, 0, v)
scrollAccV -= v
}
val h = scrollAccH.toInt()
if (h != 0) {
NativeBridge.nativeSendScroll(handle, 1, h)
scrollAccH -= h
}
}
private fun button(actionButton: Int, down: Boolean) {
val b = when (actionButton) {
MotionEvent.BUTTON_PRIMARY -> 1
MotionEvent.BUTTON_TERTIARY -> 2
MotionEvent.BUTTON_SECONDARY -> 3
MotionEvent.BUTTON_BACK -> 4
MotionEvent.BUTTON_FORWARD -> 5
else -> return
}
if (down) {
heldButtons.add(b)
NativeBridge.nativeSendPointerButton(handle, b, true)
} else if (heldButtons.remove(b)) {
// Only release what we pressed — drops the release of a swallowed engaging click
// and anything that raced a capture transition.
NativeBridge.nativeSendPointerButton(handle, b, false)
}
}
private fun flushButtons() {
heldButtons.forEach { NativeBridge.nativeSendPointerButton(handle, it, false) }
heldButtons.clear()
}
}
@@ -0,0 +1,193 @@
package io.unom.punktfunk
import android.os.Handler
import android.os.Looper
import android.view.Choreographer
import android.view.KeyEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.hypot
// Hold this long on SELECT (pointer-mode toggle) / PLAY-PAUSE (keyboard toggle) for the long-press
// action instead of the tap action.
private const val LONG_PRESS_MS = 800L
// D-pad glide ballistics, in screen-widths per second: start slow enough to hit a close button,
// ramp over RAMP_S seconds of continuous hold so crossing the desktop doesn't take all day.
private const val SPEED_MIN = 0.14f
private const val SPEED_MAX = 0.70f
private const val RAMP_S = 1.2f
/**
* Android TV remote as a pointer — the Android analogue of the Apple client's Siri-remote pointer,
* adapted for D-pad-only remotes (most Android TV remotes have no touch surface). For the
* "TV as a desktop client" use case, where a plain remote is often the only thing in hand.
*
* While streaming on a TV, **hold SELECT ≈ 0.8 s** to toggle pointer mode. While active:
* * D-pad (held) glides the host cursor with ramping acceleration (relative `MouseMove`,
* Choreographer-paced, diagonal-normalized);
* * SELECT tap = left click; PLAY/PAUSE tap = right click (Siri-remote parity);
* * PLAY/PAUSE held = toggle the on-screen keyboard; BACK = leave pointer mode
* (a second BACK then leaves the stream as usual).
* While inactive, everything except the SELECT long-press passes through untouched (D-pad =
* arrow keys, SELECT tap = Enter — synthesized on release, since the down was held back to
* disambiguate the long-press).
*
* Only consulted for non-gamepad key events on TV devices (MainActivity gates the calls); all
* state lives on the main thread.
*/
class RemotePointer(
private val handle: Long,
private val surfaceWidth: () -> Int,
private val onActiveChanged: (Boolean) -> Unit,
private val onKeyboardToggle: () -> Unit,
) {
var active = false
private set
private val handler = Handler(Looper.getMainLooper())
private val held = mutableSetOf<Int>() // D-pad keycodes currently down
private var moveAccX = 0f
private var moveAccY = 0f
private var lastFrameNs = 0L
private var rampSec = 0f
private var tickerRunning = false
private var centerLongFired = false
private var playLongFired = false
private val centerLong = Runnable {
centerLongFired = true
toggle()
}
private val playLong = Runnable {
playLongFired = true
onKeyboardToggle()
}
private val frame = object : Choreographer.FrameCallback {
override fun doFrame(nowNs: Long) {
if (!tickerRunning) return
if (held.isEmpty() || !active) {
tickerRunning = false
return
}
val dt = if (lastFrameNs == 0L) {
1f / 60f
} else {
((nowNs - lastFrameNs) / 1e9f).coerceIn(0.001f, 0.1f)
}
lastFrameNs = nowNs
rampSec += dt
var vx = 0f
var vy = 0f
if (KeyEvent.KEYCODE_DPAD_LEFT in held) vx -= 1f
if (KeyEvent.KEYCODE_DPAD_RIGHT in held) vx += 1f
if (KeyEvent.KEYCODE_DPAD_UP in held) vy -= 1f
if (KeyEvent.KEYCODE_DPAD_DOWN in held) vy += 1f
val mag = hypot(vx, vy)
if (mag > 0f) {
val w = surfaceWidth().coerceAtLeast(640)
val speed = w * (SPEED_MIN + (SPEED_MAX - SPEED_MIN) * (rampSec / RAMP_S).coerceAtMost(1f))
moveAccX += vx / mag * speed * dt
moveAccY += vy / mag * speed * dt
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
Choreographer.getInstance().postFrameCallback(this)
}
}
/** One remote key event; true = consumed. Ignore key repeats — the ticker owns motion. */
fun onKey(event: KeyEvent): Boolean {
val down = event.action == KeyEvent.ACTION_DOWN
when (event.keyCode) {
KeyEvent.KEYCODE_DPAD_CENTER -> {
if (down) {
if (event.repeatCount == 0) {
centerLongFired = false
handler.postDelayed(centerLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(centerLong)
if (!centerLongFired) {
if (active) {
click(1)
} else {
// The down was held back to disambiguate the long-press, so the
// normal path never saw it — synthesize the Enter here instead.
NativeBridge.nativeSendKey(handle, 0x0D, true, 0)
NativeBridge.nativeSendKey(handle, 0x0D, false, 0)
}
}
}
return true
}
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN,
KeyEvent.KEYCODE_DPAD_LEFT, KeyEvent.KEYCODE_DPAD_RIGHT,
-> {
if (!active) return false
if (down) {
if (held.add(event.keyCode) && held.size == 1) startTicker()
} else {
held.remove(event.keyCode)
}
return true
}
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE -> {
if (!active) return false // inactive: the media-key VK path owns it
if (down) {
if (event.repeatCount == 0) {
playLongFired = false
handler.postDelayed(playLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(playLong)
if (!playLongFired) click(3)
}
return true
}
KeyEvent.KEYCODE_BACK -> {
if (!active) return false
if (!down) toggle() // leave pointer mode; the next BACK leaves the stream
return true
}
else -> return false
}
}
/** Stream teardown: stop timers/ticker; nothing wire-held to flush (clicks are edges). */
fun release() {
handler.removeCallbacks(centerLong)
handler.removeCallbacks(playLong)
active = false
held.clear()
tickerRunning = false
}
private fun toggle() {
active = !active
if (!active) {
held.clear()
tickerRunning = false
}
onActiveChanged(active)
}
private fun startTicker() {
rampSec = 0f
lastFrameNs = 0L
if (!tickerRunning) {
tickerRunning = true
Choreographer.getInstance().postFrameCallback(frame)
}
}
private fun click(button: Int) {
NativeBridge.nativeSendPointerButton(handle, button, true)
NativeBridge.nativeSendPointerButton(handle, button, false)
}
}
@@ -109,6 +109,27 @@ data class Settings(
* setup where the OS-level pad (lizard mode) is preferred. * setup where the OS-level pad (lizard mode) is preferred.
*/ */
val sc2Capture: Boolean = true, val sc2Capture: Boolean = true,
/**
* Lock a physical mouse to the stream ([android.view.View.requestPointerCapture]) and forward
* raw relative motion — FPS mouse-look, the iPad "Capture pointer for games" twin. Engages at
* stream start and on a click into the stream; Ctrl+Alt+Shift+Q toggles it live (the chord
* works even with this off). Off (default): a mouse points absolutely, desktop-style.
*/
val pointerCapture: Boolean = false,
/**
* Flip scroll direction — the mouse wheel and the two-finger touch scroll both. Parity with
* the Apple/GTK clients' "Invert scroll direction".
*/
val invertScroll: Boolean = false,
/**
* Sync text copied on this device to the host and vice versa while streaming (the desktop
* clients' shared clipboard, text-only here). Only effective when the host advertises the
* clipboard capability; the protocol is opt-in per session either way.
*/
val clipboardSync: Boolean = true,
) )
/** [Settings.touchMode] values; persisted by name. */ /** [Settings.touchMode] values; persisted by name. */
@@ -172,6 +193,9 @@ class SettingsStore(context: Context) {
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true), autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false), rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false),
sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true), sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true),
pointerCapture = prefs.getBoolean(K_POINTER_CAPTURE, false),
invertScroll = prefs.getBoolean(K_INVERT_SCROLL, false),
clipboardSync = prefs.getBoolean(K_CLIPBOARD_SYNC, true),
) )
fun save(s: Settings) { fun save(s: Settings) {
@@ -195,6 +219,9 @@ class SettingsStore(context: Context) {
.putBoolean(K_AUTO_WAKE, s.autoWakeEnabled) .putBoolean(K_AUTO_WAKE, s.autoWakeEnabled)
.putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone) .putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone)
.putBoolean(K_SC2_CAPTURE, s.sc2Capture) .putBoolean(K_SC2_CAPTURE, s.sc2Capture)
.putBoolean(K_POINTER_CAPTURE, s.pointerCapture)
.putBoolean(K_INVERT_SCROLL, s.invertScroll)
.putBoolean(K_CLIPBOARD_SYNC, s.clipboardSync)
.apply() .apply()
} }
@@ -233,6 +260,9 @@ class SettingsStore(context: Context) {
const val K_AUTO_WAKE = "auto_wake_enabled" const val K_AUTO_WAKE = "auto_wake_enabled"
const val K_RUMBLE_ON_PHONE = "rumble_on_phone" const val K_RUMBLE_ON_PHONE = "rumble_on_phone"
const val K_SC2_CAPTURE = "sc2_capture" const val K_SC2_CAPTURE = "sc2_capture"
const val K_POINTER_CAPTURE = "pointer_capture"
const val K_INVERT_SCROLL = "invert_scroll"
const val K_CLIPBOARD_SYNC = "clipboard_sync"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */ /** Legacy Boolean the enum replaced — read once as the migration default, never written. */
const val K_TRACKPAD = "trackpad_mode" const val K_TRACKPAD = "trackpad_mode"
@@ -412,6 +412,27 @@ private fun ControlsSettings(s: Settings, update: (Settings) -> Unit, onOpenCont
style = MaterialTheme.typography.bodySmall, style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant, color = MaterialTheme.colorScheme.onSurfaceVariant,
) )
ToggleRow(
title = "Capture pointer for games",
subtitle = "Lock a connected mouse to the stream and send raw relative motion " +
"(mouse-look). Ctrl+Alt+Shift+Q toggles it live; click the stream to re-capture. " +
"Off: the mouse points at the desktop directly",
checked = s.pointerCapture,
onCheckedChange = { on -> update(s.copy(pointerCapture = on)) },
)
ToggleRow(
title = "Invert scroll direction",
subtitle = "Flip the mouse wheel and two-finger touch scrolling",
checked = s.invertScroll,
onCheckedChange = { on -> update(s.copy(invertScroll = on)) },
)
ToggleRow(
title = "Shared clipboard",
subtitle = "Text copied here pastes on the host and vice versa (hosts with " +
"clipboard sharing enabled)",
checked = s.clipboardSync,
onCheckedChange = { on -> update(s.copy(clipboardSync = on)) },
)
} }
SettingsCard { SettingsCard {
SettingDropdown( SettingDropdown(
@@ -13,6 +13,7 @@ import android.net.wifi.WifiManager
import android.os.Build import android.os.Build
import android.text.InputType import android.text.InputType
import android.util.Log import android.util.Log
import android.view.KeyEvent
import android.view.SurfaceHolder import android.view.SurfaceHolder
import android.view.SurfaceView import android.view.SurfaceView
import android.view.View import android.view.View
@@ -49,6 +50,9 @@ import androidx.core.content.ContextCompat
import androidx.core.view.WindowCompat import androidx.core.view.WindowCompat
import androidx.core.view.WindowInsetsCompat import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat import androidx.core.view.WindowInsetsControllerCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.LifecycleOwner
import io.unom.punktfunk.kit.GamepadFeedback import io.unom.punktfunk.kit.GamepadFeedback
import io.unom.punktfunk.kit.GamepadRouter import io.unom.punktfunk.kit.GamepadRouter
import io.unom.punktfunk.kit.deviceBodyVibrator import io.unom.punktfunk.kit.deviceBodyVibrator
@@ -176,6 +180,14 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// "hold to quit" hint overlay. Set from the router's onExitArmed (main thread). // "hold to quit" hint overlay. Set from the router's onExitArmed (main thread).
var exitArming by remember { mutableStateOf(false) } var exitArming by remember { mutableStateOf(false) }
// True while the TV remote is acting as a pointer (hold SELECT toggles) — drives the mode hint.
var remotePointerOn by remember { mutableStateOf(false) }
// Focus anchor the soft keyboard is summoned onto AND the pointer-capture grab target (a grab
// needs a focusable view; captured-pointer events land on it). Declared before the effect
// below so the capture callbacks can reach the view once it exists.
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) }
DisposableEffect(handle) { DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON) window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
wifiLocks.forEach { lock -> wifiLocks.forEach { lock ->
@@ -221,6 +233,54 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Show a "hold to quit" hint the moment the chord completes (the router debounces the actual // Show a "hold to quit" hint the moment the chord completes (the router debounces the actual
// exit); it clears when the buttons release early or the hold elapses. Runs on the main thread. // exit); it clears when the buttons release early or the hold elapses. Runs on the main thread.
router.onExitArmed = { armed -> exitArming = armed } router.onExitArmed = { armed -> exitArming = armed }
// Physical mouse: uncaptured hover/click/wheel forwards as absolute pointing; captured
// (setting or the Ctrl+Alt+Shift+Q chord) raw deltas forward as relative mouse-look.
// The local cursor is hidden over the stream — the host's own cursor, composited into
// the video, is the one the user sees (twin of the desktop clients' hidden cursor).
val decor = window?.decorView
val priorPointerIcon = decor?.pointerIcon
decor?.pointerIcon = android.view.PointerIcon.getSystemIcon(
context,
android.view.PointerIcon.TYPE_NULL,
)
val mouse = MouseForwarder(
handle,
invertScroll = initialSettings.invertScroll,
captureWanted = initialSettings.pointerCapture,
surfaceSize = { (decor?.width ?: 0) to (decor?.height ?: 0) },
)
mouse.onRequestCapture = {
// The grab needs the (focusable) capture view: focus it, then ask. Posted so a
// request racing view attach/focus settles on the next frame.
keyCapture?.let { v ->
v.post {
v.requestFocus()
v.requestPointerCapture()
}
}
}
mouse.onReleaseCapture = { keyCapture?.releasePointerCapture() }
activity?.mouseForwarder = mouse
// TV remote-as-pointer: hold SELECT ≈ 0.8 s to toggle; the D-pad then glides the host
// cursor (see RemotePointer). TV only — a phone's remote-less keys stay on the VK path.
val remote = if (isTv) {
RemotePointer(
handle,
surfaceWidth = { decor?.width ?: 1920 },
onActiveChanged = { on -> remotePointerOn = on },
onKeyboardToggle = { keyCapture?.let { it.setImeVisible(!it.imeShown) } },
)
} else {
null
}
activity?.remotePointer = remote
// Shared clipboard (text v1): only when the user setting is on AND the host has a
// working clipboard service. Protocol-level opt-in + the poll thread live in the sync.
val clip = if (initialSettings.clipboardSync && NativeBridge.nativeClipSupported(handle)) {
ClipboardSync(context, handle).also { it.start() }
} else {
null
}
activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate
// Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad // Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad
// index via the router; poll threads stopped + joined before the router is released and the // index via the router; poll threads stopped + joined before the router is released and the
@@ -286,6 +346,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
} }
onDispose { onDispose {
closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it
clip?.stop() // stop + join the clipboard poll thread BEFORE the handle is freed
feedback.onHidRaw = null feedback.onHidRaw = null
feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed
sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } } sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
@@ -293,6 +354,12 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down
router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener
activity?.gamepadRouter = null activity?.gamepadRouter = null
// Mouse/remote-pointer teardown: lift held buttons, drop the grab, restore the cursor.
mouse.release()
activity?.mouseForwarder = null
remote?.release()
activity?.remotePointer = null
decor?.pointerIcon = priorPointerIcon
activity?.streamHandle = 0L activity?.streamHandle = 0L
activity?.requestStreamExit = null activity?.requestStreamExit = null
// Back in the menus: the SC2 (if present) resumes driving the console UI. // Back in the menus: the SC2 (if present) resumes driving the console UI.
@@ -320,8 +387,32 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger). // Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() } BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
// Focus anchor the three-finger keyboard swipe summons the IME onto (see KeyCaptureView). // Leaving the app (Home, task switch, screen off) MUST end the session. Android does not
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) } // suspend a process for going to background, so without this the native worker kept running and
// its QUIC connection kept answering the host's keep-alives — the user was long gone but the
// host still saw a live client and held the session (and its display + encoder) open until the
// OS eventually reclaimed the process, which on a TV box is effectively never.
//
// Route it through `onDisconnect()` so the composable's `onDispose` above runs the one real
// teardown path. Deliberately NOT a `nativeDisconnectQuit`: backgrounding isn't a user "quit",
// so the host should linger the display and make coming straight back a fast reconnect.
DisposableEffect(handle) {
val lifecycle = (context as? LifecycleOwner)?.lifecycle
val obs = LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_STOP) {
onDisconnect()
}
}
lifecycle?.addObserver(obs)
onDispose { lifecycle?.removeObserver(obs) }
}
// Auto-engage pointer capture at stream start (setting on + a mouse actually present).
// Delayed a beat: the grab needs window focus and the capture view attached.
LaunchedEffect(handle) {
delay(400)
activity?.mouseForwarder?.engageFromStart()
}
Box(modifier = Modifier.fillMaxSize()) { Box(modifier = Modifier.fillMaxSize()) {
AndroidView( AndroidView(
@@ -379,23 +470,49 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
if (exitArming) { if (exitArming) {
ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp)) ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
} }
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe // Remote-pointer mode hint — the remote's keys are remapped while it's on, so say so.
// up in the mouse modes) — it never draws or takes touches, it just owns IME focus. if (remotePointerOn) {
RemotePointerHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
}
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe up
// in the mouse modes) AND the pointer-capture grab target — it never draws or takes
// touches, it just owns IME focus and receives captured-pointer events.
AndroidView( AndroidView(
modifier = Modifier.size(1.dp), modifier = Modifier.size(1.dp),
factory = { ctx -> KeyCaptureView(ctx).also { keyCapture = it } }, factory = { ctx ->
KeyCaptureView(ctx).also { v ->
keyCapture = v
// Real IME text path when the host types committed text (see KeyCaptureView).
v.textHandle =
if (NativeBridge.nativeTextInputSupported(handle)) handle else 0L
v.setOnCapturedPointerListener { _, ev ->
(ctx as? MainActivity)?.mouseForwarder?.onCapturedPointer(ev) ?: false
}
}
},
) )
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture // Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no // vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no
// keyboard gesture: its fingers belong to the host verbatim (a swipe there may BE a // keyboard gesture: its fingers belong to the host verbatim (a swipe there may BE a
// host-OS gesture), so intercepting three fingers would corrupt real multi-touch. // host-OS gesture), so intercepting three fingers would corrupt real multi-touch.
// Stylus lane (design/pen-tablet-input.md §7): against a HOST_CAP_PEN host a stylus
// splits out of BOTH touch models onto the pen plane; its heartbeat coroutine keeps a
// stationary held stroke alive (and its cancellation lifts everything on teardown).
val stylus = remember(handle) {
if (NativeBridge.nativeHostSupportsPen(handle)) StylusStream(handle) else null
}
if (stylus != null) {
LaunchedEffect(stylus) { stylus.heartbeatLoop() }
}
Box( Box(
Modifier.fillMaxSize().pointerInput(handle, touchMode) { Modifier.fillMaxSize().pointerInput(handle, touchMode) {
when (touchMode) { when (touchMode) {
TouchMode.TOUCH -> streamTouchPassthrough(handle) TouchMode.TOUCH -> streamTouchPassthrough(handle, stylus)
else -> streamTouchInput( else -> streamTouchInput(
handle, handle,
stylus,
trackpad = touchMode == TouchMode.TRACKPAD, trackpad = touchMode == TouchMode.TRACKPAD,
invertScroll = initialSettings.invertScroll,
onCycleStats = { statsVerbosity = statsVerbosity.next() }, onCycleStats = { statsVerbosity = statsVerbosity.next() },
onKeyboard = { show -> keyCapture?.setImeVisible(show) }, onKeyboard = { show -> keyCapture?.setImeVisible(show) },
) )
@@ -423,14 +540,35 @@ private fun ExitChordHint(modifier: Modifier = Modifier) {
) )
} }
/**
* The remote-pointer mode cue: while active the remote's keys are remapped (D-pad glides the host
* cursor, SELECT clicks), so the overlay both confirms the toggle and teaches the vocabulary.
*/
@Composable
private fun RemotePointerHint(modifier: Modifier = Modifier) {
Text(
"Remote pointer — SELECT click · play/pause right-click · hold SELECT to exit",
modifier = modifier
.background(Color.Black.copy(alpha = 0.55f), RoundedCornerShape(8.dp))
.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White,
fontSize = 15.sp,
)
}
/** /**
* Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME * Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME
* onto this view. `TYPE_NULL` puts the IME in "dumb keyboard" mode — it delivers raw [KeyEvent]s * onto this view. Two IME models, picked by the host's capabilities:
* (no composing text, no autocorrect), which flow through `MainActivity.dispatchKeyEvent` → * * **Text path** ([textHandle] set — the host advertised `HOST_CAP_TEXT_INPUT`): a real
* `Keymap.toVk` → the host, the exact path a hardware keyboard takes. Text an IME insists on * editable [HostTextConnection], so the IME gives autocorrect, gesture typing, non-Latin
* committing instead still arrives: the non-editable [BaseInputConnection] synthesizes KeyEvents * composition and emoji, all mirrored to the host as committed text + diffs.
* for it via `KeyCharacterMap` (with Shift carried as meta state — see the IME-shift wrap in * * **Fallback** (older host): `TYPE_NULL` puts the IME in "dumb keyboard" mode — raw
* `MainActivity.dispatchKeyEvent`). * [KeyEvent]s flow through `MainActivity.dispatchKeyEvent` → `Keymap.toVk` → the host, the
* exact path a hardware keyboard takes (with the IME-shift wrap documented there).
*
* Doubles as the pointer-capture grab target: a grab needs a focusable view, and captured-pointer
* events are delivered to it (routed to [MouseForwarder.onCapturedPointer] via the listener the
* stream screen installs).
*/ */
private class KeyCaptureView(context: Context) : View(context) { private class KeyCaptureView(context: Context) : View(context) {
init { init {
@@ -438,22 +576,171 @@ private class KeyCaptureView(context: Context) : View(context) {
isFocusableInTouchMode = true isFocusableInTouchMode = true
} }
override fun onCheckIsTextEditor(): Boolean = true /** The session handle when the host types committed text; `0` = VK-only fallback. */
var textHandle: Long = 0L
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection { /** Whether [setImeVisible] last showed the IME — for toggle-style callers (remote pointer). */
outAttrs.inputType = InputType.TYPE_NULL var imeShown = false
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or EditorInfo.IME_FLAG_NO_FULLSCREEN private set
return BaseInputConnection(this, false)
override fun onCheckIsTextEditor(): Boolean = imeShown
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection? {
// Only an editor while the user has SUMMONED the keyboard (gesture / remote toggle).
// This view holds focus for the whole stream (it's the capture anchor), and with an
// always-live editable connection the IME counts input as active on it — TV IMEs then
// pop their UI the moment a PHYSICAL keyboard key arrives. With no connection, hardware
// typing stays on the raw dispatchKeyEvent → Keymap → wire path and no keyboard appears.
if (!imeShown) return null
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or
EditorInfo.IME_FLAG_NO_FULLSCREEN or EditorInfo.IME_FLAG_NO_ENTER_ACTION
return if (textHandle != 0L) {
outAttrs.inputType = InputType.TYPE_CLASS_TEXT or
InputType.TYPE_TEXT_FLAG_AUTO_CORRECT or InputType.TYPE_TEXT_FLAG_MULTI_LINE
HostTextConnection(this, textHandle)
} else {
outAttrs.inputType = InputType.TYPE_NULL
BaseInputConnection(this, false)
}
} }
fun setImeVisible(show: Boolean) { fun setImeVisible(show: Boolean) {
val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager
?: return ?: return
imeShown = show
if (show) { if (show) {
requestFocus() requestFocus()
// The view may already be focused from a null-connection state — restart so the
// framework re-queries onCreateInputConnection with the gate now open.
imm.restartInput(this)
imm.showSoftInput(this, 0) imm.showSoftInput(this, 0)
} else { } else {
imm.hideSoftInputFromWindow(windowToken, 0) imm.hideSoftInputFromWindow(windowToken, 0)
imm.restartInput(this) // gate closed — drop the editable connection
} }
} }
/**
* BACK while the summoned keyboard is up: the IME consumes it pre-IME to dismiss itself, so
* [setImeVisible] never hears about it — sync the gate here or a stale `imeShown` leaves the
* editable connection live and physical typing re-pops the keyboard.
*/
override fun onKeyPreIme(keyCode: Int, event: KeyEvent): Boolean {
if (keyCode == KeyEvent.KEYCODE_BACK && imeShown && event.action == KeyEvent.ACTION_UP) {
imeShown = false
(context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager)
?.restartInput(this)
}
return super.onKeyPreIme(keyCode, event)
}
}
/**
* IME → host text bridge (the `HOST_CAP_TEXT_INPUT` path): a real **editable** connection, so
* the IME runs its full machinery (autocorrect, gesture typing, non-Latin composition), mirrored
* to the host as it happens. The one piece of host-side state tracked is *what the host currently
* shows of the active composition* ([sentComposition]): composing updates send a common-prefix
* diff (backspaces + the new suffix) so corrections materialize live on the host; a commit
* settles it. [setComposingRegion] adopts already-committed text as the active composition
* (autocorrect-revert / backspace-into-word flows), so the next update diffs against it instead
* of retyping. Newlines become Enter taps; [deleteSurroundingText] becomes Backspace/Delete taps.
*
* Known approximation: diff lengths are counted in Unicode scalars, assuming one host Backspace
* deletes one scalar — true for the composition text IMEs actually produce (emoji and other
* multi-unit graphemes commit directly rather than composing).
*/
private class HostTextConnection(
view: KeyCaptureView,
private val handle: Long,
) : BaseInputConnection(view, true) {
/** What the host currently shows of the active composition ("" = none). */
private var sentComposition = ""
override fun commitText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
sentComposition = ""
val ok = super.commitText(text, newCursorPosition)
trimEditable()
return ok
}
override fun setComposingText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
return super.setComposingText(text, newCursorPosition)
}
override fun finishComposingText(): Boolean {
// The composition text stands as committed — the host already shows it verbatim.
sentComposition = ""
return super.finishComposingText()
}
override fun setComposingRegion(start: Int, end: Int): Boolean {
val e = editable
if (e != null) {
val a = start.coerceIn(0, e.length)
val b = end.coerceIn(0, e.length)
sentComposition = e.subSequence(minOf(a, b), maxOf(a, b)).toString()
}
return super.setComposingRegion(start, end)
}
override fun deleteSurroundingText(beforeLength: Int, afterLength: Int): Boolean {
repeat(beforeLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_BACK) }
repeat(afterLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_DELETE) }
return super.deleteSurroundingText(beforeLength, afterLength)
}
override fun performEditorAction(actionCode: Int): Boolean {
tapVk(VK_RETURN)
return true
}
/** Replace the host's view of the composition with [text] via a common-prefix diff. */
private fun retype(text: String) {
var common = sentComposition.commonPrefixWith(text)
// Never split a surrogate pair mid-diff — back off to the pair boundary.
if (common.isNotEmpty() && common.last().isHighSurrogate()) {
common = common.dropLast(1)
}
val stale = sentComposition.substring(common.length)
repeat(stale.codePointCount(0, stale.length).coerceAtMost(MAX_TAPS)) { tapVk(VK_BACK) }
sendText(text.substring(common.length))
sentComposition = text
}
/** Forward literal text, turning newlines into Enter taps (control chars never ride text). */
private fun sendText(s: String) {
var chunk = StringBuilder()
for (ch in s) {
if (ch == '\n') {
if (chunk.isNotEmpty()) {
NativeBridge.nativeSendText(handle, chunk.toString())
chunk = StringBuilder()
}
tapVk(VK_RETURN)
} else {
chunk.append(ch)
}
}
if (chunk.isNotEmpty()) NativeBridge.nativeSendText(handle, chunk.toString())
}
private fun tapVk(vk: Int) {
NativeBridge.nativeSendKey(handle, vk, true, 0)
NativeBridge.nativeSendKey(handle, vk, false, 0)
}
/** Bound the mirror buffer: once nothing is composing, old text serves no purpose. */
private fun trimEditable() {
val e = editable ?: return
if (getComposingSpanStart(e) == -1 && e.length > 4000) e.clear()
}
private companion object {
const val VK_BACK = 0x08
const val VK_RETURN = 0x0D
const val VK_DELETE = 0x2E
const val MAX_TAPS = 256
}
} }
@@ -0,0 +1,195 @@
package io.unom.punktfunk
import android.view.MotionEvent
import androidx.compose.ui.ExperimentalComposeUiApi
import androidx.compose.ui.input.pointer.PointerEvent
import androidx.compose.ui.input.pointer.PointerType
import androidx.compose.ui.unit.IntSize
import io.unom.punktfunk.kit.NativeBridge
import kotlinx.coroutines.delay
// Wire PEN_* state bits (punktfunk_core::quic::pen; mirrored, asserted by the Rust shim's docs).
private const val PEN_IN_RANGE = 1f
private const val PEN_TOUCHING = 2f
private const val PEN_BARREL1 = 4f
private const val PEN_BARREL2 = 8f
private const val STRIDE = 10
private const val MAX_SAMPLES = 8
/**
* Android stylus → the state-full pen plane (design/pen-tablet-input.md §7): pressure, tilt
* (`AXIS_TILT`, radians from the surface normal), azimuth (`AXIS_ORIENTATION` — Android's 0 =
* "pointed away from the user" IS the wire's north, no offset needed), hover with
* `AXIS_DISTANCE`, the eraser tool, both stylus barrel buttons, and historical (coalesced)
* samples batched oldest-first for full capture-rate fidelity. Android has no barrel-roll
* axis — roll stays unknown on this client.
*
* Both touch loops call [intercept] first; stylus/eraser pointers are consumed here (against a
* pen-capable host) and never reach the finger paths, independent of the touch-input mode.
* [heartbeatLoop] implements the ≤100 ms keepalive wire contract: a stationary held stylus is
* silent in Android's input pipeline, and the host force-releases a stroke after 200 ms
* without samples.
*/
internal class StylusStream(private val handle: Long) {
private var inRange = false
private var touching = false
private var sawHover = false
private val last = FloatArray(STRIDE)
private val batch = FloatArray(MAX_SAMPLES * STRIDE)
init {
idle(last)
}
/**
* Consume the event's stylus pointers into pen samples. Returns true when this event
* carried any (the caller's finger/gesture handling must then skip those changes).
*/
@OptIn(ExperimentalComposeUiApi::class)
fun intercept(ev: PointerEvent, size: IntSize): Boolean {
val stylusChanges = ev.changes.filter {
it.type == PointerType.Stylus || it.type == PointerType.Eraser
}
if (stylusChanges.isEmpty()) return false
stylusChanges.forEach { it.consume() }
val me = ev.motionEvent ?: return true
if (size.width <= 0 || size.height <= 0) return true
// At most one stylus exists — find its pointer index by tool type.
val idx = (0 until me.pointerCount).firstOrNull {
me.getToolType(it) == MotionEvent.TOOL_TYPE_STYLUS ||
me.getToolType(it) == MotionEvent.TOOL_TYPE_ERASER
} ?: return true
when (me.actionMasked) {
MotionEvent.ACTION_DOWN, MotionEvent.ACTION_POINTER_DOWN,
MotionEvent.ACTION_MOVE,
-> {
touching = true
inRange = true
emitSamples(me, idx, size)
}
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_MOVE -> {
sawHover = true
inRange = true
touching = false
emitSamples(me, idx, size)
}
MotionEvent.ACTION_UP, MotionEvent.ACTION_POINTER_UP -> {
touching = false
// Hover-capable hardware keeps proximity (HOVER_EXIT owns the leave);
// anything else leaves range on lift — the host never parks a phantom pen.
inRange = sawHover
emitSamples(me, idx, size)
}
MotionEvent.ACTION_HOVER_EXIT, MotionEvent.ACTION_CANCEL -> release()
else -> {}
}
return true
}
/** Session/composition teardown: leave range so the host lifts anything still inked. */
fun reset() {
if (inRange || touching) release()
sawHover = false
}
/** The ≤100 ms keepalive (80 ms leaves headroom for one lost datagram). Runs until
* cancelled; resends the last state-full sample while the pen is in range. */
suspend fun heartbeatLoop() {
try {
while (true) {
delay(80)
if (inRange || touching) {
last[9] = 0f // dt
NativeBridge.nativeSendPen(handle, last, 1)
}
}
} finally {
reset()
}
}
private fun release() {
touching = false
inRange = false
last[0] = 0f // state: out of range
last[4] = 0f // pressure
NativeBridge.nativeSendPen(handle, last, 1)
}
/** Historical (coalesced) samples oldest-first, then the current one — a single batch. */
private fun emitSamples(me: MotionEvent, idx: Int, size: IntSize) {
val history = minOf(me.historySize, MAX_SAMPLES - 1)
var count = 0
var prevT = if (history > 0) me.getHistoricalEventTime(0) else me.eventTime
for (h in (me.historySize - history) until me.historySize) {
val t = me.getHistoricalEventTime(h)
fill(
batch, count * STRIDE, size,
x = me.getHistoricalX(idx, h), y = me.getHistoricalY(idx, h),
pressure = me.getHistoricalPressure(idx, h),
tiltRad = me.getHistoricalAxisValue(MotionEvent.AXIS_TILT, idx, h),
orientRad = me.getHistoricalAxisValue(MotionEvent.AXIS_ORIENTATION, idx, h),
distance = me.getHistoricalAxisValue(MotionEvent.AXIS_DISTANCE, idx, h),
buttons = me.buttonState, tool = me.getToolType(idx),
dtUs = ((t - prevT) * 1000).coerceIn(0, 65535).toFloat(),
)
prevT = t
count++
}
fill(
batch, count * STRIDE, size,
x = me.getX(idx), y = me.getY(idx), pressure = me.getPressure(idx),
tiltRad = me.getAxisValue(MotionEvent.AXIS_TILT, idx),
orientRad = me.getAxisValue(MotionEvent.AXIS_ORIENTATION, idx),
distance = me.getAxisValue(MotionEvent.AXIS_DISTANCE, idx),
buttons = me.buttonState, tool = me.getToolType(idx),
dtUs = ((me.eventTime - prevT) * 1000).coerceIn(0, 65535).toFloat(),
)
count++
batch.copyInto(last, 0, (count - 1) * STRIDE, count * STRIDE)
NativeBridge.nativeSendPen(handle, batch, count)
}
private fun fill(
out: FloatArray,
off: Int,
size: IntSize,
x: Float,
y: Float,
pressure: Float,
tiltRad: Float,
orientRad: Float,
distance: Float,
buttons: Int,
tool: Int,
dtUs: Float,
) {
var state = 0f
if (inRange || touching) state += PEN_IN_RANGE
if (touching) state += PEN_TOUCHING
if (buttons and MotionEvent.BUTTON_STYLUS_PRIMARY != 0) state += PEN_BARREL1
if (buttons and MotionEvent.BUTTON_STYLUS_SECONDARY != 0) state += PEN_BARREL2
out[off + 0] = state
out[off + 1] = if (tool == MotionEvent.TOOL_TYPE_ERASER) 1f else 0f
out[off + 2] = (x / (size.width - 1).coerceAtLeast(1)).coerceIn(0f, 1f)
out[off + 3] = (y / (size.height - 1).coerceAtLeast(1)).coerceIn(0f, 1f)
out[off + 4] = if (touching) pressure.coerceIn(0f, 1f) else 0f
// AXIS_DISTANCE units are device-arbitrary; 0..1 covers real hardware, and 0 while
// hovering legitimately means "at the hover floor".
out[off + 5] = if (touching) 0f else distance.coerceIn(0f, 1f)
out[off + 6] = Math.toDegrees(tiltRad.toDouble()).toFloat().coerceIn(0f, 90f)
// AXIS_ORIENTATION: 0 = pointed away from the user (= wire north), clockwise, −π..π.
out[off + 7] = ((Math.toDegrees(orientRad.toDouble()) + 360.0) % 360.0).toFloat()
out[off + 8] = -1f // no barrel-roll axis on Android
out[off + 9] = dtUs
}
private fun idle(out: FloatArray) {
out.fill(0f)
out[5] = -1f // distance unknown
out[6] = -1f // tilt unknown
out[7] = -1f // azimuth unknown
out[8] = -1f // roll unknown
}
}
@@ -1,9 +1,11 @@
package io.unom.punktfunk package io.unom.punktfunk
import androidx.compose.foundation.gestures.awaitEachGesture import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown import androidx.compose.ui.input.pointer.AwaitPointerEventScope
import androidx.compose.ui.input.pointer.PointerId import androidx.compose.ui.input.pointer.PointerId
import androidx.compose.ui.input.pointer.PointerInputChange
import androidx.compose.ui.input.pointer.PointerInputScope import androidx.compose.ui.input.pointer.PointerInputScope
import androidx.compose.ui.input.pointer.PointerType
import androidx.compose.ui.input.pointer.changedToDownIgnoreConsumed import androidx.compose.ui.input.pointer.changedToDownIgnoreConsumed
import androidx.compose.ui.input.pointer.changedToUpIgnoreConsumed import androidx.compose.ui.input.pointer.changedToUpIgnoreConsumed
import androidx.compose.ui.input.pointer.positionChanged import androidx.compose.ui.input.pointer.positionChanged
@@ -56,7 +58,26 @@ private const val ACCEL_MAX = 3.0f
* normalizes and maps into the output). On teardown (stream leaves composition) every still-held * normalizes and maps into the output). On teardown (stream leaves composition) every still-held
* contact is lifted so nothing stays stuck on the host. * contact is lifted so nothing stays stuck on the host.
*/ */
internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) { /** Whether this change belongs to the stylus lane (only when a pen-capable host is live). */
private fun isStylus(c: PointerInputChange, stylus: StylusStream?): Boolean =
stylus != null && (c.type == PointerType.Stylus || c.type == PointerType.Eraser)
/** [awaitFirstDown] with the stylus lane split out: pen events feed [stylus] and never start a
* mouse/touch gesture. Toward a pen-less host ([stylus] == null) a stylus stays a finger. */
private suspend fun AwaitPointerEventScope.awaitFirstFingerDown(
stylus: StylusStream?,
): PointerInputChange {
while (true) {
val ev = awaitPointerEvent()
stylus?.intercept(ev, size)
val down = ev.changes.firstOrNull {
it.changedToDownIgnoreConsumed() && !isStylus(it, stylus)
}
if (down != null) return down
}
}
internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long, stylus: StylusStream?) {
val ids = mutableMapOf<PointerId, Int>() val ids = mutableMapOf<PointerId, Int>()
fun alloc(p: PointerId): Int { fun alloc(p: PointerId): Int {
var id = 0 var id = 0
@@ -68,10 +89,12 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
awaitPointerEventScope { awaitPointerEventScope {
while (true) { while (true) {
val ev = awaitPointerEvent() val ev = awaitPointerEvent()
stylus?.intercept(ev, size)
val sw = size.width val sw = size.width
val sh = size.height val sh = size.height
if (sw <= 0 || sh <= 0) continue if (sw <= 0 || sh <= 0) continue
for (c in ev.changes) { for (c in ev.changes) {
if (isStylus(c, stylus)) continue // the pen plane owns it
val x = c.position.x.roundToInt().coerceIn(0, sw - 1) val x = c.position.x.roundToInt().coerceIn(0, sw - 1)
val y = c.position.y.roundToInt().coerceIn(0, sh - 1) val y = c.position.y.roundToInt().coerceIn(0, sh - 1)
when { when {
@@ -98,10 +121,13 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
internal suspend fun PointerInputScope.streamTouchInput( internal suspend fun PointerInputScope.streamTouchInput(
handle: Long, handle: Long,
stylus: StylusStream?,
trackpad: Boolean, trackpad: Boolean,
invertScroll: Boolean,
onCycleStats: () -> Unit, onCycleStats: () -> Unit,
onKeyboard: (show: Boolean) -> Unit, onKeyboard: (show: Boolean) -> Unit,
) { ) {
val scrollDir = if (invertScroll) -1 else 1
var lastTapUp = 0L var lastTapUp = 0L
var lastTapX = 0f var lastTapX = 0f
var lastTapY = 0f var lastTapY = 0f
@@ -118,7 +144,7 @@ internal suspend fun PointerInputScope.streamTouchInput(
) )
} }
awaitEachGesture { awaitEachGesture {
val down = awaitFirstDown(requireUnconsumed = false) val down = awaitFirstFingerDown(stylus)
val startX = down.position.x val startX = down.position.x
val startY = down.position.y val startY = down.position.y
// A touch landing just after a quick tap nearby = tap-and-drag: hold the left // A touch landing just after a quick tap nearby = tap-and-drag: hold the left
@@ -155,7 +181,8 @@ internal suspend fun PointerInputScope.streamTouchInput(
while (true) { while (true) {
val ev = awaitPointerEvent() val ev = awaitPointerEvent()
val pressed = ev.changes.filter { it.pressed } stylus?.intercept(ev, size)
val pressed = ev.changes.filter { it.pressed && !isStylus(it, stylus) }
if (pressed.isEmpty()) { if (pressed.isEmpty()) {
upTime = ev.changes.firstOrNull()?.uptimeMillis ?: upTime upTime = ev.changes.firstOrNull()?.uptimeMillis ?: upTime
break break
@@ -184,12 +211,12 @@ internal suspend fun PointerInputScope.streamTouchInput(
val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up
val sx = ((cx - prevCx) / SCROLL_DIV).toInt() val sx = ((cx - prevCx) / SCROLL_DIV).toInt()
if (sy != 0) { if (sy != 0) {
NativeBridge.nativeSendScroll(handle, 0, sy * 120) NativeBridge.nativeSendScroll(handle, 0, sy * 120 * scrollDir)
prevCy = cy prevCy = cy
moved = true moved = true
} }
if (sx != 0) { if (sx != 0) {
NativeBridge.nativeSendScroll(handle, 1, sx * 120) NativeBridge.nativeSendScroll(handle, 1, sx * 120 * scrollDir)
prevCx = cx prevCx = cx
moved = true moved = true
} }
@@ -106,6 +106,17 @@ object Keymap {
KeyEvent.KEYCODE_DPAD_UP -> 0x26 KeyEvent.KEYCODE_DPAD_UP -> 0x26
KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27 KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27
KeyEvent.KEYCODE_DPAD_DOWN -> 0x28 KeyEvent.KEYCODE_DPAD_DOWN -> 0x28
// TV-remote SELECT = Enter (a gamepad's press routes via SOURCE_GAMEPAD before this).
KeyEvent.KEYCODE_DPAD_CENTER -> 0x0D
// Consumer/media keys — forwarded to the host while streaming (volume stays local:
// MainActivity's pass-through list wins before the map is consulted).
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE,
KeyEvent.KEYCODE_MEDIA_PLAY,
KeyEvent.KEYCODE_MEDIA_PAUSE -> 0xB3 // VK_MEDIA_PLAY_PAUSE
KeyEvent.KEYCODE_MEDIA_NEXT -> 0xB0 // VK_MEDIA_NEXT_TRACK
KeyEvent.KEYCODE_MEDIA_PREVIOUS -> 0xB1 // VK_MEDIA_PREV_TRACK
KeyEvent.KEYCODE_MEDIA_STOP -> 0xB2 // VK_MEDIA_STOP
// Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant) // Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant)
KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0 KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0
@@ -287,6 +287,66 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */ /** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int) external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
/**
* Whether the host advertised full-fidelity stylus injection (`HOST_CAP_PEN`) — the gate
* for splitting stylus pointers out of the touch path onto the pen plane. False on `0`.
*/
external fun nativeHostSupportsPen(handle: Long): Boolean
/**
* One stylus batch of STATE-FULL samples (the pen plane; design/pen-tablet-input.md §7):
* [count] × 10 floats, oldest first — `[state, tool, x, y, pressure, distance, tilt_deg,
* azimuth_deg, roll_deg, dt_us]`. `state` = the wire in-range/touching/barrel bits; `tool`
* 0=pen 1=eraser; x/y/pressure/distance normalized 0..1; distance/tilt/azimuth/roll < 0 =
* unknown. Send only when [nativeHostSupportsPen]; repeat the last sample ≤100 ms while the
* pen is in range (the host force-releases a silent stroke after 200 ms).
*/
external fun nativeSendPen(handle: Long, samples: FloatArray, count: Int)
/**
* Whether the host advertised committed-text injection (`HOST_CAP_TEXT_INPUT`) — its inject
* backend can type Unicode text directly. Picks the real IME `InputConnection` (autocorrect,
* gesture typing, non-Latin scripts) over the TYPE_NULL raw-key fallback. False on `0`.
*/
external fun nativeTextInputSupported(handle: Long): Boolean
/**
* Committed IME text → one `TextInput` wire event per Unicode scalar, in order. Control
* characters are skipped natively (Enter/Backspace ride [nativeSendKey]). Only meaningful
* when [nativeTextInputSupported] returned true — older hosts ignore the events.
*/
external fun nativeSendText(handle: Long, text: String)
// ---- Shared clipboard (text v1): Kotlin drives ClipboardManager, Rust the protocol ----
// Opt-in per session (nativeClipControl). Local copies are announced as lazy offers; bytes
// cross only when the host pastes (a "fetch:" event answered by nativeClipServeText). Host
// copies arrive as "offer:" events, fetched eagerly into the system clipboard.
/** Whether the host advertised a working shared-clipboard service (HOST_CAP_CLIPBOARD). */
external fun nativeClipSupported(handle: Long): Boolean
/** Session-level clipboard opt-in/out; nothing happens until enabled=true crosses. */
external fun nativeClipControl(handle: Long, enabled: Boolean)
/** Announce "this device's clipboard now holds text". [seq]: monotonic, newest wins. */
external fun nativeClipOfferText(handle: Long, seq: Int)
/** Pull the text of the host's offer [seq] → transfer id echoed on "data:"/"error:", or -1. */
external fun nativeClipFetchText(handle: Long, seq: Int): Int
/** Answer a "fetch:" event with the clipboard's current text (the host is pasting). */
external fun nativeClipServeText(handle: Long, reqId: Int, text: String)
/** Abort a clipboard transfer by id (either direction). */
external fun nativeClipCancel(handle: Long, id: Int)
/**
* Block ≤250 ms for the next clipboard event, as a compact string: `state:<0|1>` ·
* `offer:<seq>:<hasText>` · `fetch:<reqId>` · `data:<xferId>:<text>` · `cancel:<id>` ·
* `error:<id>:<code>` · `closed` (session gone) — null on timeout. Dedicated poll thread.
*/
external fun nativeNextClip(handle: Long): String?
// ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ---- // ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ----
// The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0, // The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0,
// so its wire is byte-identical to the old single-pad path. The core folds the per-transition // so its wire is byte-identical to the old single-pad path. The core folds the per-transition
@@ -0,0 +1,182 @@
//! Shared-clipboard plane (text-only v1): Kotlin drives the Android `ClipboardManager`, these
//! shims drive [`punktfunk_core::client::NativeClient`]'s clipboard surface.
//!
//! Model (mirrors the desktop clients): opt-in via `nativeClipControl(true)`; local copies are
//! announced lazily as format-list offers (`nativeClipOfferText`) and the bytes cross only when
//! the host pastes (a `fetch` event answered by `nativeClipServeText`); a host copy arrives as an
//! `offer` event, which the Kotlin side fetches eagerly (Android's clipboard has no lazy provider
//! path worth the complexity) and lands in the system clipboard on the `data` event.
//!
//! Events cross to Kotlin as compact strings from the blocking `nativeNextClip` poll (drained on
//! a dedicated thread, same pattern as `nativeNextRumble`):
//! `state:<0|1>` · `offer:<seq>:<has_text 0|1>` · `fetch:<req_id>` · `data:<xfer_id>:<text>` ·
//! `cancel:<id>` · `error:<id>:<code>` · `closed` — null on a poll timeout. Non-text fetch
//! requests are cancelled natively (only text is ever offered, so they shouldn't occur).
use std::time::Duration;
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jint, jlong, jstring};
use jni::JNIEnv;
use punktfunk_core::clipboard::ClipEventCore;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::quic::{ClipKind, CLIP_FILE_INDEX_NONE, HOST_CAP_CLIPBOARD};
use super::SessionHandle;
/// The portable wire MIME both ends map to their platform text type.
const TEXT_MIME: &str = "text/plain;charset=utf-8";
/// Deref the opaque handle (`0` → `None`).
///
/// SAFETY: live handle per the nativeConnect/nativeClose contract; every method used is `&self`
/// on the `Sync` connector.
fn client(handle: jlong) -> Option<&'static SessionHandle> {
if handle == 0 {
return None;
}
// SAFETY: see the function docs — the Kotlin side guarantees the handle outlives the call.
Some(unsafe { &*(handle as *const SessionHandle) })
}
/// `NativeBridge.nativeClipSupported(handle)` — the host advertised `HOST_CAP_CLIPBOARD`.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
client(handle).map_or(0, |h| {
u8::from(h.client.host_caps() & HOST_CAP_CLIPBOARD != 0)
})
}
/// `NativeBridge.nativeClipControl(handle, enabled)` — session-level opt-in/out. Nothing
/// clipboard-related happens on either side until an `enabled: true` crosses.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipControl(
_env: JNIEnv,
_this: JObject,
handle: jlong,
enabled: jboolean,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_control(enabled != 0, 0);
}
}
/// `NativeBridge.nativeClipOfferText(handle, seq)` — announce "the Android clipboard now holds
/// text" (format list only; bytes cross when the host fetches). `seq` is Kotlin's monotonic
/// counter, newest wins.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipOfferText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_offer(
seq as u32,
vec![ClipKind {
mime: TEXT_MIME.into(),
size_hint: 0,
}],
);
}
}
/// `NativeBridge.nativeClipFetchText(handle, seq)` — pull the text of the host's offer `seq`.
/// Returns the transfer id echoed on the matching `data:`/`error:` event, or 1.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipFetchText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) -> jint {
client(handle)
.and_then(|h| {
h.client
.clip_fetch(seq as u32, TEXT_MIME.into(), CLIP_FILE_INDEX_NONE)
.ok()
})
.map_or(-1, |xfer| xfer as jint)
}
/// `NativeBridge.nativeClipServeText(handle, reqId, text)` — answer a `fetch:` event with the
/// clipboard's current text (the host is pasting our offer).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipServeText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
req_id: jint,
text: JString,
) {
let Some(h) = client(handle) else { return };
let Ok(s) = env.get_string(&text) else {
let _ = h.client.clip_cancel(req_id as u32);
return;
};
let _ = h
.client
.clip_serve(req_id as u32, String::from(s).into_bytes(), true);
}
/// `NativeBridge.nativeClipCancel(handle, id)` — abort a transfer (either direction).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipCancel(
_env: JNIEnv,
_this: JObject,
handle: jlong,
id: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_cancel(id as u32);
}
}
/// `NativeBridge.nativeNextClip(handle)` — block ≤250 ms for the next clipboard event, encoded
/// as a compact string (module docs); null on timeout, `"closed"` once the session is gone.
/// Call from a dedicated poll thread.
///
/// Text payloads ride `data:<xfer_id>:<text>` decoded lossily — safe because the phase-0
/// clipboard task delivers a whole payload in ONE event (`last = true`), so a chunk boundary
/// can never split a UTF-8 sequence.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextClip(
env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jstring {
let Some(h) = client(handle) else {
return std::ptr::null_mut();
};
let msg = match h.client.next_clip(Duration::from_millis(250)) {
Ok(ClipEventCore::State { enabled, .. }) => format!("state:{}", u8::from(enabled)),
Ok(ClipEventCore::RemoteOffer { seq, kinds }) => {
let has_text = kinds.iter().any(|k| k.mime.starts_with("text/plain"));
format!("offer:{seq}:{}", u8::from(has_text))
}
Ok(ClipEventCore::FetchRequest { req_id, mime, .. }) => {
if mime.starts_with("text/plain") {
format!("fetch:{req_id}")
} else {
// We only ever offer text; cancel anything else rather than stall the host.
let _ = h.client.clip_cancel(req_id);
return std::ptr::null_mut();
}
}
Ok(ClipEventCore::Data { xfer_id, bytes, .. }) => {
format!("data:{xfer_id}:{}", String::from_utf8_lossy(&bytes))
}
Ok(ClipEventCore::Cancelled { id }) => format!("cancel:{id}"),
Ok(ClipEventCore::Error { id, code }) => format!("error:{id}:{code}"),
Err(PunktfunkError::NoFrame) => return std::ptr::null_mut(),
Err(_) => "closed".into(),
};
env.new_string(msg)
.map(|s| s.into_raw())
.unwrap_or(std::ptr::null_mut())
}
@@ -201,6 +201,9 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// No display-volume forwarding from Android yet (the panel tone-maps PQ itself via the // No display-volume forwarding from Android yet (the panel tone-maps PQ itself via the
// Surface dataspace + static metadata) — the host keeps its virtual-display EDID defaults. // Surface dataspace + static metadata) — the host keeps its virtual-display EDID defaults.
None, None,
// No non-video caps: this client does not render the host cursor locally (no shape/state
// planes in the jni surface), so advertising CLIENT_CAP_CURSOR would stream cursor-less.
0,
launch, // a store-qualified library id to boot into a game, or None for the desktop launch, // a store-qualified library id to boot into a game, or None for the desktop
pin, // Some → Crypto on host-fp mismatch pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous) identity, // owned (cert, key) PEM, or None (anonymous)
+131 -2
View File
@@ -6,11 +6,14 @@
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal, //! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side). //! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::{JByteBuffer, JObject}; use jni::objects::{JByteBuffer, JFloatArray, JObject, JString};
use jni::sys::{jboolean, jint, jlong}; use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv; use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX}; use punktfunk_core::quic::{
PenSample, PenTool, RichInput, HID_REPORT_MAX, HOST_CAP_PEN, HOST_CAP_TEXT_INPUT,
PEN_ANGLE_UNKNOWN, PEN_BATCH_MAX, PEN_DISTANCE_UNKNOWN, PEN_TILT_UNKNOWN,
};
use super::SessionHandle; use super::SessionHandle;
@@ -145,6 +148,132 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
send_event(handle, kind, vk as u32, 0, 0, mods as u32); send_event(handle, kind, vk as u32, 0, 0, mods as u32);
} }
/// `NativeBridge.nativeTextInputSupported(handle)` — whether the host advertised
/// `HOST_CAP_TEXT_INPUT` (its inject backend types committed text), so the Kotlin side can pick
/// the real IME `InputConnection` over the TYPE_NULL raw-key fallback. `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_TEXT_INPUT != 0)
}
/// `NativeBridge.nativeHostSupportsPen(handle)` — the host advertised `HOST_CAP_PEN`, so the
/// Kotlin side splits stylus pointers out of the touch path onto the pen plane
/// (design/pen-tablet-input.md §7). `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeHostSupportsPen(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_PEN != 0)
}
/// Floats per sample in the `nativeSendPen` flat array.
const PEN_JNI_STRIDE: usize = 10;
/// `NativeBridge.nativeSendPen(handle, samples, count)` — one stylus batch of STATE-FULL
/// samples, `count` × [`PEN_JNI_STRIDE`] floats, oldest first:
/// `[state, tool, x, y, pressure, distance, tilt_deg, azimuth_deg, roll_deg, dt_us]`.
/// `state` = the wire `PEN_*` bits; `tool` 0=pen 1=eraser; `x`/`y`/`pressure`/`distance`
/// normalized 0..1; `distance`/`tilt_deg`/`azimuth_deg`/`roll_deg` < 0 = unknown. Call only
/// against a [`nativeHostSupportsPen`] host; the client heartbeats the last sample ≤100 ms
/// while in range (Kotlin side — see `StylusStream`).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendPen(
env: JNIEnv,
_this: JObject,
handle: jlong,
samples: JFloatArray,
count: jint,
) {
if handle == 0 || count <= 0 {
return;
}
let count = (count as usize).min(PEN_BATCH_MAX);
let mut buf = [0f32; PEN_BATCH_MAX * PEN_JNI_STRIDE];
let flat = &mut buf[..count * PEN_JNI_STRIDE];
if env.get_float_array_region(&samples, 0, flat).is_err() {
return; // short array — a bridge bug, never worth a crash on the input path
}
let mut batch = [PenSample::default(); PEN_BATCH_MAX];
for (slot, s) in batch.iter_mut().zip(flat.chunks_exact(PEN_JNI_STRIDE)) {
if !s[2].is_finite() || !s[3].is_finite() {
return; // never forward a NaN coordinate
}
*slot = PenSample {
state: s[0] as u8,
tool: if s[1] as u8 == 1 {
PenTool::Eraser
} else {
PenTool::Pen
},
x: s[2].clamp(0.0, 1.0),
y: s[3].clamp(0.0, 1.0),
pressure: (s[4].clamp(0.0, 1.0) * 65535.0) as u16,
distance: if s[5] < 0.0 {
PEN_DISTANCE_UNKNOWN
} else {
(s[5].clamp(0.0, 1.0) * 65534.0) as u16
},
tilt_deg: if s[6] < 0.0 {
PEN_TILT_UNKNOWN
} else {
(s[6].clamp(0.0, 90.0)) as u8
},
azimuth_deg: if s[7] < 0.0 {
PEN_ANGLE_UNKNOWN
} else {
(s[7] as u16) % 360
},
roll_deg: if s[8] < 0.0 {
PEN_ANGLE_UNKNOWN
} else {
(s[8] as u16) % 360
},
dt_us: s[9].clamp(0.0, 65535.0) as u16,
};
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; send_pen is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
let _ = h.client.send_pen(&batch[..count]);
}
/// `NativeBridge.nativeSendText(handle, text)` — committed IME text, one `TextInput` event per
/// Unicode scalar (`code` = the scalar; multi-char commits are consecutive events in order).
/// Control characters are skipped — Enter/Backspace/Tab ride the VK key path. Call only when
/// [`Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported`] returned true.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
text: JString,
) {
if handle == 0 {
return;
}
let Ok(s) = env.get_string(&text) else {
return;
};
for ch in String::from(s).chars().filter(|c| !c.is_control()) {
send_event(handle, InputKind::TextInput, ch as u32, 0, 0, 0);
}
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input --------------- // ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried // Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried
// in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a // in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a
@@ -17,6 +17,7 @@
//! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode //! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode
//! renegotiation. Port the remaining orchestration from `clients/linux`. //! renegotiation. Port the remaining orchestration from `clients/linux`.
mod clipboard;
mod connect; mod connect;
mod input; mod input;
mod planes; mod planes;
@@ -144,14 +144,26 @@ struct ContentView: View {
// tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a // tap uses, so trust policy / WoL / the approval sheet all come along. Never starts a
// parallel session this drives the one `model` ContentView owns. // parallel session this drives the one `model` ContentView owns.
.onOpenURL { handleDeepLink($0) } .onOpenURL { handleDeepLink($0) }
#if os(iOS) #if os(iOS) || os(tvOS)
// Background keep-alive driver (opt-in). Only .background/.active matter; .inactive (a // Backgrounding driver. Only .background/.active matter; .inactive (a transient peek) is
// transient peek) is ignored so the disconnect timer never starts for a Control-Center pull. // ignored so neither branch fires for a Control-Center pull.
//
// Backgrounding MUST end the session one way or the other: the app keeps running while
// streaming (the `audio` background mode plus a live audio session), so its QUIC connection
// keeps answering the host's keep-alives with the user long gone the host has no way to
// tell that apart from someone watching, and the session survived indefinitely. Either hold
// it under the opt-in keep-alive (bounded by that path's own auto-disconnect timer) or end
// it here.
.onChange(of: scenePhase) { _, phase in .onChange(of: scenePhase) { _, phase in
switch phase { switch phase {
case .background: case .background:
if backgroundKeepAlive, model.phase == .streaming { guard model.phase == .streaming else { break }
if backgroundKeepAlive {
model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes) model.enterBackground(timeoutMinutes: backgroundTimeoutMinutes)
} else {
// Not deliberate: the user may come straight back, so let the host linger the
// display for a fast reconnect instead of tearing it down.
model.disconnect(deliberate: false)
} }
case .active: case .active:
model.exitBackground() model.exitBackground()
@@ -159,7 +171,11 @@ struct ContentView: View {
break break
} }
} }
// Live Activity lifecycle, driven from the model's published state. #endif
#if os(iOS)
// Live Activity lifecycle, driven from the model's published state. iPhone/iPad only
// ActivityKit (and so `liveActivity`) does not exist on tvOS, which is why this stays in its
// own os(iOS) block rather than riding the backgrounding driver's.
.onChange(of: model.phase) { _, phase in .onChange(of: model.phase) { _, phase in
switch phase { switch phase {
case .streaming: case .streaming:
@@ -0,0 +1,97 @@
// Keeps the local display awake for the duration of a streaming session.
//
// A stream is not "user activity" to the OS: the pixels arrive over the network and the input that
// drives them is often a game controller, which does NOT feed the HID idle timer on any Apple
// platform. So a controller-only session reliably idles the panel out from under the user the
// same reason the Android client holds FLAG_KEEP_SCREEN_ON while streaming (StreamScreen.kt).
//
// Held by SessionModel from `beginStreaming` to `disconnect`, so it is scoped to the session and
// never leaks past it (including a host-ended or timed-out background session, which both land in
// `disconnect`).
import Foundation
#if os(macOS)
import IOKit.pwr_mgt
#else
import UIKit
#endif
@MainActor
final class DisplaySleepGuard {
#if os(macOS)
/// The `beginActivity` token; non-nil exactly while held.
private var activity: NSObjectProtocol?
/// Re-used across heartbeats so the whole session shares one assertion instead of
/// accumulating one per tick.
private var userActivityAssertion: IOPMAssertionID = IOPMAssertionID(0)
private var heartbeat: Timer?
/// The power assertion defers DISPLAY SLEEP but not the screen saver that runs off the
/// HID idle timer, which a controller-only session never touches. Declaring user activity
/// on an interval well under the shortest selectable screen-saver delay (1 minute) keeps
/// that timer from ever reaching it. Side effect, and the intended one: an idle-lock
/// configured to follow the screen saver is deferred too, for the session only.
private static let heartbeatInterval: TimeInterval = 30
#endif
private(set) var isHeld = false
/// Idempotent a second acquire while held is a no-op.
func acquire() {
guard !isHeld else { return }
isHeld = true
#if os(macOS)
// The high-level Foundation API over IOKit power assertions: `.idleDisplaySleepDisabled`
// is the panel, `.userInitiated` also holds off idle SYSTEM sleep and sudden termination
// for a session the user is watching in real time.
activity = ProcessInfo.processInfo.beginActivity(
options: [.userInitiated, .idleDisplaySleepDisabled],
reason: "Punktfunk streaming session")
declareUserActivity()
let timer = Timer.scheduledTimer(withTimeInterval: Self.heartbeatInterval, repeats: true) {
[weak self] _ in
MainActor.assumeIsolated { self?.declareUserActivity() }
}
// The stream runs under a tracking run-loop mode while a menu or a window resize is up;
// .common keeps the heartbeat ticking through those.
RunLoop.main.add(timer, forMode: .common)
heartbeat = timer
#else
// iOS/iPadOS/tvOS: app-wide, and ignored while backgrounded the background keep-alive
// (audio-only, video dropped) correctly lets the device sleep without touching this.
UIApplication.shared.isIdleTimerDisabled = true
#endif
}
/// Idempotent safe to call when not held (`disconnect` runs on paths that never streamed).
func release() {
guard isHeld else { return }
isHeld = false
#if os(macOS)
heartbeat?.invalidate()
heartbeat = nil
if let activity {
ProcessInfo.processInfo.endActivity(activity)
self.activity = nil
}
if userActivityAssertion != IOPMAssertionID(0) {
IOPMAssertionRelease(userActivityAssertion)
userActivityAssertion = IOPMAssertionID(0)
}
#else
UIApplication.shared.isIdleTimerDisabled = false
#endif
}
#if os(macOS)
/// Resets the HID idle timer (see `heartbeatInterval`). `kIOPMUserActiveLocal` = activity at
/// this Mac's own display, which is what a stream being watched here is.
private func declareUserActivity() {
IOPMAssertionDeclareUserActivity(
"Punktfunk streaming session" as CFString,
kIOPMUserActiveLocal,
&userActivityAssertion)
}
#endif
}
@@ -196,6 +196,11 @@ final class SessionModel: ObservableObject {
/// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`. /// Bounded auto-disconnect for a backgrounded keep-alive session. Fires on `.main`.
private var backgroundTimer: DispatchSourceTimer? private var backgroundTimer: DispatchSourceTimer?
/// Holds off display sleep (and, on macOS, the screen saver) for the life of a session
/// nothing about watching a stream looks like user activity to the OS, least of all a
/// controller-only session. Acquired in `beginStreaming`, released in `disconnect`.
private let displaySleepGuard = DisplaySleepGuard()
/// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true /// `allowTofu` gates the trust-on-first-use prompt for an unpinned host: it is only true
/// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host /// when the host EXPLICITLY advertised `pair=optional` (rule 3a). For any other unpinned host
/// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair` /// `pair=required`, a manually-typed host, or a discovered host with no/unknown `pair`
@@ -302,13 +307,26 @@ final class SessionModel: ObservableObject {
if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported { if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported {
videoCodecs |= PunktfunkConnection.codecPyroWave videoCodecs |= PunktfunkConnection.codecPyroWave
} }
// Cursor channel (remote-desktop-sweep M2, macOS): sessions STARTING in the desktop
// mouse model advertise local cursor rendering the host then stops compositing
// the pointer and forwards shape/state, which StreamView draws as the real
// NSCursor. Capture-mode sessions keep today's composited pointer.
#if os(macOS)
let clientCaps: UInt8 =
(MouseInputMode(
rawValue: UserDefaults.standard.string(forKey: DefaultsKey.mouseMode) ?? "")
?? .capture) == .desktop ? 0x01 : 0
#else
let clientCaps: UInt8 = 0
#endif
let result = Result { try PunktfunkConnection( let result = Result { try PunktfunkConnection(
host: host.address, port: host.port, host: host.address, port: host.port,
width: width, height: height, refreshHz: hz, width: width, height: height, refreshHz: hz,
pinSHA256: pin, identity: identity, compositor: compositor, pinSHA256: pin, identity: identity, compositor: compositor,
gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps, gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps,
audioChannels: audioChannels, audioChannels: audioChannels,
videoCodecs: videoCodecs, preferredCodec: preferredCodec, launchID: launchID, videoCodecs: videoCodecs, preferredCodec: preferredCodec,
clientCaps: clientCaps, launchID: launchID,
// Delegated approval: the host holds this connect open until the operator approves // Delegated approval: the host holds this connect open until the operator approves
// it (~180 s) outwait that window so a slow approval still lands here. Normal // it (~180 s) outwait that window so a slow approval still lands here. Normal
// connects keep the snappy default. // connects keep the snappy default.
@@ -442,6 +460,8 @@ final class SessionModel: ObservableObject {
func disconnect(deliberate: Bool = true) { func disconnect(deliberate: Bool = true) {
statsTimer?.invalidate() statsTimer?.invalidate()
statsTimer = nil statsTimer = nil
// No-op when this session never reached `.streaming` (a refused/aborted connect).
displaySleepGuard.release()
// Drop any armed background keep-alive (incl. the timeout that just fired us). // Drop any armed background keep-alive (incl. the timeout that just fired us).
backgroundTimer?.cancel() backgroundTimer?.cancel()
backgroundTimer = nil backgroundTimer = nil
@@ -537,6 +557,7 @@ final class SessionModel: ObservableObject {
// Input capture itself is owned by StreamView (engaged by the captureEnabled // Input capture itself is owned by StreamView (engaged by the captureEnabled
// flip this phase change causes, released/re-engaged by the user from there). // flip this phase change causes, released/re-engaged by the user from there).
phase = .streaming phase = .streaming
displaySleepGuard.acquire()
// Audio starts with streaming, not during the trust prompt no host sound (or // Audio starts with streaming, not during the trust prompt no host sound (or
// mic uplink!) before the user trusted the host. Devices come from Settings; // mic uplink!) before the user trusted the host. Devices come from Settings;
// "" = system default. // "" = system default.
@@ -221,6 +221,9 @@ public final class PunktfunkConnection {
/// core). The clip *sends* (`clipControl`/`clipOffer`/`clipServe`) share this lock too: /// core). The clip *sends* (`clipControl`/`clipOffer`/`clipServe`) share this lock too:
/// they're quick non-blocking enqueues, and a single lock keeps close() ordering simple. /// they're quick non-blocking enqueues, and a single lock keeps close() ordering simple.
private let clipboardLock = NSLock() private let clipboardLock = NSLock()
/// Serializes the (single) cursor pull thread against close() both cursor planes are
/// drained by ONE thread, so one lock covers them.
private let cursorLock = NSLock()
/// Negotiated session mode (host-confirmed). /// Negotiated session mode (host-confirmed).
public private(set) var width: UInt32 = 0 public private(set) var width: UInt32 = 0
@@ -381,6 +384,106 @@ public final class PunktfunkConnection {
public var hostSupportsClipboard: Bool { public var hostSupportsClipboard: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_CLIPBOARD) != 0 hostCaps & UInt8(PUNKTFUNK_HOST_CAP_CLIPBOARD) != 0
} }
/// The host answered `HOST_CAP_CURSOR`: it stopped compositing the pointer and forwards
/// shape/state on the cursor planes the client MUST draw the cursor locally.
/// `0x08` the bit moved when `HOST_CAP_TEXT_INPUT` claimed `0x04` on main; testing the
/// old bit would mistake a text-input-capable host (e.g. Windows) for a cursor grant.
public var hostSupportsCursor: Bool {
hostCaps & 0x08 != 0
}
/// The host injects full-fidelity stylus input (`HOST_CAP_PEN`) the gate for splitting
/// Apple Pencil out of the touch path onto the pen plane (``sendPen(_:)``).
public var hostSupportsPen: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_PEN) != 0
}
/// One forwarded host-cursor shape (the cursor channel, ABI v11): straight-alpha RGBA,
/// `rgba.count == width * height * 4`, hotspot within the bitmap. Cache by `serial`
/// states reference shapes by it and a re-shown serial never resends pixels.
public struct CursorShapeEvent: Sendable {
public let serial: UInt32
public let width: Int
public let height: Int
public let hotX: Int
public let hotY: Int
public let rgba: Data
}
/// Per-host-tick cursor state: position (host video px, the pointer/hotspot point),
/// visibility, and the host-driven relative-mode hint (an app grabbed/hid the pointer
/// run captured relative; clear absolute, reappearing at `x`/`y`). Latest-wins.
public struct CursorStateEvent: Sendable {
public let serial: UInt32
public let visible: Bool
public let relativeHint: Bool
public let x: Int32
public let y: Int32
}
/// Pull the next forwarded cursor SHAPE (nil = timeout). Only a session connected with
/// `clientCaps` cursor bit against a `hostSupportsCursor` host receives any. Drain shape
/// AND state from ONE dedicated cursor thread (they share a lock).
public func nextCursorShape(timeoutMs: UInt32 = 0) throws -> CursorShapeEvent? {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkCursorShape()
let rc = punktfunk_connection_next_cursor_shape(h, &out, timeoutMs)
switch rc {
case statusOK:
// Copy out of the ABI borrow (valid until the next shape call) immediately.
let bytes = out.rgba.map { Data(bytes: $0, count: Int(out.len)) } ?? Data()
return CursorShapeEvent(
serial: out.serial, width: Int(out.w), height: Int(out.h),
hotX: Int(out.hot_x), hotY: Int(out.hot_y), rgba: bytes)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Pull the next cursor STATE (nil = timeout). Latest-wins drain the queue and apply
/// only the newest. Same thread + gate as [`nextCursorShape`].
public func nextCursorState(timeoutMs: UInt32 = 0) throws -> CursorStateEvent? {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkCursorState()
let rc = punktfunk_connection_next_cursor_state(h, &out, timeoutMs)
switch rc {
case statusOK:
return CursorStateEvent(
serial: out.serial,
visible: out.flags & 0x01 != 0,
relativeHint: out.flags & 0x02 != 0,
x: out.x, y: out.y)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Tell the host who renders the pointer (the §8 mid-stream mouse-model flip, ABI v12):
/// `clientDraws = true` this client draws it locally (the desktop mouse model; the host
/// excludes the pointer from the video and forwards shape/state); `false` the host
/// composites it into the video (the capture model, full fidelity). Idempotent,
/// latest-wins; harmless against hosts without the cursor cap. Fire-and-forget errors
/// are swallowed (a closed session is the only failure and it moots the flip).
public func setCursorRender(clientDraws: Bool) {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { return }
_ = punktfunk_connection_set_cursor_render(h, clientDraws)
}
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing /// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack). /// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) } public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
@@ -417,6 +520,7 @@ public final class PunktfunkConnection {
audioChannels: UInt8 = 2, audioChannels: UInt8 = 2,
videoCodecs: UInt8 = 0x02, // PUNKTFUNK_CODEC_HEVC the codecs this client can decode videoCodecs: UInt8 = 0x02, // PUNKTFUNK_CODEC_HEVC the codecs this client can decode
preferredCodec: UInt8 = 0, // 0 = auto; else PUNKTFUNK_CODEC_* soft preference preferredCodec: UInt8 = 0, // 0 = auto; else PUNKTFUNK_CODEC_* soft preference
clientCaps: UInt8 = 0, // ABI v11: PUNKTFUNK_CLIENT_CAP_CURSOR = render the host cursor locally
launchID: String? = nil, launchID: String? = nil,
timeoutMs: UInt32 = 10_000 timeoutMs: UInt32 = 10_000
) throws { ) throws {
@@ -436,18 +540,18 @@ public final class PunktfunkConnection {
withOptionalCString(launchID) { launch in withOptionalCString(launchID) { launch in
if let pin = pinSHA256 { if let pin = pinSHA256 {
return pin.withUnsafeBytes { p in return pin.withUnsafeBytes { p in
punktfunk_connect_ex8( punktfunk_connect_ex9(
cs, port, width, height, refreshHz, compositor.rawValue, cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, launch, videoCodecs, preferredCodec, clientCaps, launch,
p.bindMemory(to: UInt8.self).baseAddress, &observed, p.bindMemory(to: UInt8.self).baseAddress, &observed,
cert, key, timeoutMs, &connectStatus) cert, key, timeoutMs, &connectStatus)
} }
} }
return punktfunk_connect_ex8( return punktfunk_connect_ex9(
cs, port, width, height, refreshHz, compositor.rawValue, cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, launch, videoCodecs, preferredCodec, clientCaps, launch,
nil, &observed, cert, key, timeoutMs, &connectStatus) nil, &observed, cert, key, timeoutMs, &connectStatus)
} }
} }
@@ -1037,6 +1141,19 @@ public final class PunktfunkConnection {
_ = punktfunk_connection_send_input(h, &ev) _ = punktfunk_connection_send_input(h, &ev)
} }
/// Send one stylus sample batch ( `PUNKTFUNK_PEN_BATCH_MAX`, oldest first) on the pen
/// plane. Gate on ``hostSupportsPen`` the core refuses toward a host without the cap.
/// Thread-safe; silently dropped after close (input is lossy by design).
public func sendPen(_ samples: [PunktfunkPenSample]) {
guard !samples.isEmpty else { return }
abiLock.lock()
defer { abiLock.unlock() }
guard let h = handle, !closeRequested else { return }
samples.withUnsafeBufferPointer { buf in
_ = punktfunk_connection_send_pen(h, buf.baseAddress, UInt32(buf.count))
}
}
/// Signal a **deliberate** user-initiated quit before ``close()``: the connection closes with /// Signal a **deliberate** user-initiated quit before ``close()``: the connection closes with
/// `QUIT_CLOSE_CODE` (81) so the host tears the session down immediately instead of holding the /// `QUIT_CLOSE_CODE` (81) so the host tears the session down immediately instead of holding the
/// keep-alive linger for a reconnect. Call only from an explicit "Disconnect" action NOT from a /// keep-alive linger for a reconnect. Call only from an explicit "Disconnect" action NOT from a
@@ -1059,10 +1176,12 @@ public final class PunktfunkConnection {
feedbackLock.lock() feedbackLock.lock()
statsLock.lock() statsLock.lock()
clipboardLock.lock() clipboardLock.lock()
cursorLock.lock()
abiLock.lock() abiLock.lock()
let h = handle let h = handle
handle = nil handle = nil
abiLock.unlock() abiLock.unlock()
cursorLock.unlock()
clipboardLock.unlock() clipboardLock.unlock()
statsLock.unlock() statsLock.unlock()
feedbackLock.unlock() feedbackLock.unlock()
@@ -0,0 +1,268 @@
// Apple Pencil state-full wire pen samples (design/pen-tablet-input.md §7).
//
// Every sample carries the COMPLETE pen state (in-range/touching/buttons + all axes) the
// host diffs consecutive samples and synthesizes down/up/button transitions itself, so a lost
// datagram self-heals and this file never sends edge events. Three sources feed one stream:
// UITouch contacts (with coalesced samples for full 240 Hz fidelity), the hover gesture
// (zOffset > 0 distinguishes a hovering Pencil from a trackpad pointer), and
// UIPencilInteraction (squeeze held barrel 1, double-tap a momentary barrel 2
// Apple Pencil has no hardware eraser end or barrel buttons, so these mappings are how
// host-side apps get their stylus button/eraser affordances).
//
// HEARTBEAT (wire contract see `PunktfunkPenSample` in punktfunk_core.h): while the pen is
// in range or touching, the last sample repeats every 100 ms even when nothing changed.
// UIKit is silent for a stationary Pencil, and the host force-releases the stroke after
// 200 ms without samples (its dead-client failsafe) the timer keeps a held stroke alive.
#if os(iOS)
import PunktfunkCore
import UIKit
final class PencilStream: NSObject, UIPencilInteractionDelegate {
enum Phase { case down, move, up, cancel }
/// One assembled batch ( `PUNKTFUNK_PEN_BATCH_MAX` samples) ready for the connection.
var send: (([PunktfunkPenSample]) -> Void)?
/// View-space point normalized [0,1] video coordinates (the letterbox mapping the
/// touch path already uses). nil until a mode is negotiated samples are dropped then.
var videoNorm: ((CGPoint) -> (Float, Float)?)?
private var inRange = false
private var touching = false
/// Squeeze held (mapped to wire BARREL1).
private var squeezeHeld = false
/// Whether this device/Pencil pair has demonstrated hover decides what a lift means:
/// hover-capable hardware keeps proximity (the hover recognizer owns the exit), anything
/// else leaves range on lift so the host never parks a phantom hovering pen.
private var sawHover = false
/// A hover gesture is live right now (routes ended-state hover callbacks to us even when
/// the recognizer's final zOffset reads 0).
private(set) var hoverActive = false
private var last = PencilStream.idleSample()
private var heartbeat: Timer?
// MARK: - Contact path (UITouch, `.pencil` only)
func touches(_ touches: Set<UITouch>, event: UIEvent?, phase: Phase, in view: UIView) {
// At most one Pencil exists; a set with several is UIKit batching phases of the same
// stylus the last one carries the freshest state.
guard let touch = touches.max(by: { $0.timestamp < $1.timestamp }) else { return }
switch phase {
case .down, .move:
touching = true
inRange = true
// Coalesced samples restore the Pencil's full capture rate (UIKit delivers at
// display cadence); oldest first, `dt_us` preserving their spacing.
let raw = event?.coalescedTouches(for: touch) ?? [touch]
var batch: [PunktfunkPenSample] = []
var prevTs: TimeInterval?
for t in raw.suffix(Int(PUNKTFUNK_PEN_BATCH_MAX)) {
guard let s = contactSample(t, in: view, prevTs: prevTs) else { continue }
prevTs = t.timestamp
batch.append(s)
}
emit(batch)
case .up:
touching = false
// Hover-capable hardware: lift back to hover, the recognizer exits range later.
// Otherwise a lift IS the range exit (mirror of the host's GameStream heuristic).
inRange = sawHover
var s = last
s.pressure = 0
s.state = stateBits()
if let posSample = contactSample(touch, in: view, prevTs: nil) {
s.x = posSample.x
s.y = posSample.y
}
emit([s])
case .cancel:
release()
}
}
// MARK: - Hover path (forwarded from the view's hover recognizer)
/// Returns whether the event was consumed as Pencil hover; `false` hands it back to the
/// pointer path. A hovering Pencil reports `zOffset > 0`; trackpad/mouse hover is 0.
func maybeHover(_ r: UIHoverGestureRecognizer, in view: UIView) -> Bool {
switch r.state {
case .began, .changed:
guard r.zOffset > 0 || hoverActive else { return false }
hoverActive = true
sawHover = true
inRange = true
touching = false
guard let (x, y) = videoNorm?(r.location(in: view)) else { return true }
var s = PencilStream.idleSample()
s.state = stateBits()
s.x = x
s.y = y
s.distance = UInt16((r.zOffset.clamped(to: 0...1) * 65534).rounded())
s.tilt_deg = Self.tiltDeg(altitude: r.altitudeAngle)
s.azimuth_deg = Self.azimuthDeg(r.azimuthAngle(in: view))
if #available(iOS 17.5, *) { s.roll_deg = Self.rollDeg(r.rollAngle) }
emit([s])
return true
case .ended, .cancelled, .failed:
guard hoverActive else { return false }
hoverActive = false
if !touching { release() }
return true
default:
return hoverActive
}
}
// MARK: - UIPencilInteractionDelegate (squeeze barrel 1 held, tap barrel 2 click)
@available(iOS 17.5, *)
func pencilInteraction(
_ interaction: UIPencilInteraction, didReceiveSqueeze squeeze: UIPencilInteraction.Squeeze
) {
switch squeeze.phase {
case .began:
squeezeHeld = true
case .ended, .cancelled:
squeezeHeld = false
default:
return
}
guard inRange || touching else { return }
var s = last
s.state = stateBits()
emit([s])
}
func pencilInteractionDidTap(_ interaction: UIPencilInteraction) {
guard inRange || touching else { return }
// A momentary barrel-2 click: press + release as two state-full samples in ONE batch
// the host's tracker emits the button press and release in order.
var press = last
press.state = stateBits() | UInt8(PUNKTFUNK_PEN_BARREL2)
var releaseS = last
releaseS.state = stateBits()
emit([press, releaseS])
}
// MARK: - Lifecycle
/// Session stop / view teardown: leave range so the host lifts anything held.
func reset() {
if inRange || touching { release() }
sawHover = false
hoverActive = false
squeezeHeld = false
}
private func release() {
touching = false
inRange = false
var s = last
s.pressure = 0
s.state = 0
emit([s])
}
// MARK: - Sample assembly
private func contactSample(
_ t: UITouch, in view: UIView, prevTs: TimeInterval?
) -> PunktfunkPenSample? {
guard let (x, y) = videoNorm?(t.location(in: view)) else { return nil }
var s = PencilStream.idleSample()
s.state = stateBits()
s.x = x
s.y = y
// maximumPossibleForce is 0 until the system knows the stylus full force then
// (binary-stylus semantics, matching the host's unknown-pressure rule).
let maxForce = t.maximumPossibleForce
s.pressure =
maxForce > 0
? UInt16((Double(t.force / maxForce).clamped(to: 0...1) * 65535).rounded())
: UInt16.max
s.distance = 0
s.tilt_deg = Self.tiltDeg(altitude: t.altitudeAngle)
s.azimuth_deg = Self.azimuthDeg(t.azimuthAngle(in: view))
if #available(iOS 17.5, *) { s.roll_deg = Self.rollDeg(t.rollAngle) }
if let prevTs {
s.dt_us = UInt16(((t.timestamp - prevTs) * 1_000_000).clamped(to: 0...65535))
}
return s
}
private func stateBits() -> UInt8 {
var bits: UInt8 = 0
if inRange || touching { bits |= UInt8(PUNKTFUNK_PEN_IN_RANGE) }
if touching { bits |= UInt8(PUNKTFUNK_PEN_TOUCHING) }
if squeezeHeld { bits |= UInt8(PUNKTFUNK_PEN_BARREL1) }
return bits
}
private func emit(_ batch: [PunktfunkPenSample]) {
guard !batch.isEmpty else { return }
last = batch[batch.count - 1]
last.dt_us = 0
send?(batch)
armHeartbeat()
}
/// The 100 ms keepalive while in range (see the file header). 80 ms leaves headroom
/// under the host's 200 ms failsafe even with one lost datagram.
private func armHeartbeat() {
heartbeat?.invalidate()
guard inRange || touching else {
heartbeat = nil
return
}
heartbeat = Timer.scheduledTimer(withTimeInterval: 0.08, repeats: true) {
[weak self] _ in
guard let self, self.inRange || self.touching else {
self?.heartbeat?.invalidate()
self?.heartbeat = nil
return
}
self.send?([self.last])
}
}
// MARK: - Angle conversions
/// Altitude (π/2 = perpendicular) wire tilt-from-normal in degrees, 0...90.
private static func tiltDeg(altitude: CGFloat) -> UInt8 {
UInt8((90 - altitude * 180 / .pi).rounded().clamped(to: 0...90))
}
/// Apple azimuth (0 along the view's +x axis, clockwise, y-down) wire azimuth
/// (0 = north/up on screen, clockwise): +90° offset.
private static func azimuthDeg(_ apple: CGFloat) -> UInt16 {
let deg = (apple * 180 / .pi + 90).truncatingRemainder(dividingBy: 360)
return UInt16((deg + 360).truncatingRemainder(dividingBy: 360).rounded()) % 360
}
/// Pencil Pro roll (radians, π...π) wire barrel roll 0...359°.
private static func rollDeg(_ roll: CGFloat) -> UInt16 {
let deg = (roll * 180 / .pi).truncatingRemainder(dividingBy: 360)
return UInt16(((deg + 360).truncatingRemainder(dividingBy: 360)).rounded()) % 360
}
/// All-unknown baseline: sentinel angles/distance, tool = pen (the eraser is host-side
/// state driven by the squeeze/tap mappings, not a hardware end).
private static func idleSample() -> PunktfunkPenSample {
PunktfunkPenSample(
x: 0, y: 0, pressure: 0,
distance: UInt16(PUNKTFUNK_PEN_DISTANCE_UNKNOWN),
azimuth_deg: UInt16(PUNKTFUNK_PEN_ANGLE_UNKNOWN),
roll_deg: UInt16(PUNKTFUNK_PEN_ANGLE_UNKNOWN),
dt_us: 0, state: 0,
tool: UInt8(PUNKTFUNK_PEN_TOOL_PEN),
tilt_deg: UInt8(PUNKTFUNK_PEN_TILT_UNKNOWN),
_reserved: (0, 0, 0))
}
}
extension Comparable {
fileprivate func clamped(to range: ClosedRange<Self>) -> Self {
min(max(self, range.lowerBound), range.upperBound)
}
}
#endif
@@ -219,6 +219,32 @@ public final class StreamLayerView: NSView {
/// flipped live by M. A live flip re-engages capture in the new model so /// flipped live by M. A live flip re-engages capture in the new model so
/// disassociation + the abs/rel choice swap atomically. Main-thread only. /// disassociation + the abs/rel choice swap atomically. Main-thread only.
private var desktopMouse = false private var desktopMouse = false
/// Cursor channel (M2): the host forwards shape/state and WE draw the pointer. Active
/// when the Welcome carried `HOST_CAP_CURSOR` (only sessions that advertised the client
/// cap get it). Shapes cache by serial; state is latest-wins. Main-thread only.
private var cursorChannelActive = false
/// A forwarded host cursor shape, cached RAW (not as a finished `NSCursor`) so the pointer can be
/// (re)built at the CURRENT video-fit scale see `scaledCursor`. The host forwards the bitmap in
/// host FRAMEBUFFER pixels, whose size tracks the host's display scaling (32 px at 100%, 96 px at
/// 300% DPI); scaling by the video fit keeps the pointer sized to the streamed desktop at any host
/// scaling instead of ballooning on a high-DPI host.
private struct HostCursorShape {
let cg: CGImage
let width: Int
let height: Int
let hotX: Int
let hotY: Int
}
private var hostCursors: [UInt32: HostCursorShape] = [:]
private var cursorState: PunktfunkConnection.CursorStateEvent?
/// Last `CursorRenderMode.clientDraws` told to the host (the §8 mid-stream render flip);
/// nil = nothing sent yet. Edge-detected by [`reconcileCursorRender`] from the live mouse
/// model, so the chord, engage/release, and session start all reconcile through one path.
private var sentClientDraws: Bool?
/// M3 hint tracking: edge-triggered so a manual M isn't fought the override latch
/// holds until the HOST's intent next changes.
private var lastHint: Bool?
private var hintOverride = false
/// One-shot auto-engage request (stream start, trust confirmed) attempted as soon /// One-shot auto-engage request (stream start, trust confirmed) attempted as soon
/// as the view is in a window with real bounds, then dropped, so it can never fire /// as the view is in a window with real bounds, then dropped, so it can never fire
/// surprisingly later (e.g. on a resize). /// surprisingly later (e.g. on a resize).
@@ -456,6 +482,7 @@ public final class StreamLayerView: NSView {
window?.makeFirstResponder(self) window?.makeFirstResponder(self)
window?.invalidateCursorRects(for: self) // desktop model: hide-over-view engages window?.invalidateCursorRects(for: self) // desktop model: hide-over-view engages
notifyCaptureChange(true) notifyCaptureChange(true)
reconcileCursorRender()
} }
private func releaseCapture() { private func releaseCapture() {
@@ -466,6 +493,7 @@ public final class StreamLayerView: NSView {
captured = false captured = false
window?.invalidateCursorRects(for: self) window?.invalidateCursorRects(for: self)
notifyCaptureChange(false) notifyCaptureChange(false)
reconcileCursorRender() // released the host composites the pointer again
} }
/// A fully transparent cursor for the desktop mouse model's hide-over-view rect /// A fully transparent cursor for the desktop mouse model's hide-over-view rect
@@ -480,12 +508,179 @@ public final class StreamLayerView: NSView {
/// globally via `CursorCapture` (the pointer can't leave the view there). /// globally via `CursorCapture` (the pointer can't leave the view there).
override public func resetCursorRects() { override public func resetCursorRects() {
if captured && desktopMouse { if captured && desktopMouse {
addCursorRect(bounds, cursor: Self.invisibleCursor) // Cursor channel active: wear the HOST's pointer shape (it is no longer in the
// video); hidden host pointer (or no shape yet) = invisible. Without the channel,
// M1 behavior: invisible local cursor, the composited host cursor is the visible one.
if cursorChannelActive, let st = cursorState, st.visible,
let shape = hostCursors[st.serial] {
addCursorRect(bounds, cursor: scaledCursor(shape))
} else {
addCursorRect(bounds, cursor: Self.invisibleCursor)
}
} else { } else {
super.resetCursorRects() super.resetCursorRects()
} }
} }
/// Tell the host who renders the pointer (the §8 mid-stream render flip): we draw it only
/// while the DESKTOP model is engaged (the local OS cursor wears the host shape); under
/// the capture model and while released the host composites it into the video (full
/// fidelity, the pre-channel look). One edge-detected reconciler, called from every
/// transition (chord, engage/release, session start).
private func reconcileCursorRender() {
guard cursorChannelActive, let connection else { return }
let clientDraws = captured && desktopMouse
guard sentClientDraws != clientDraws else { return }
sentClientDraws = clientDraws
connection.setCursorRender(clientDraws: clientDraws)
}
/// Flip the mouse model with the atomic release/re-engage swap; `reappearAt` (host video
/// px the M3 hand-back position) warps the local pointer so leaving relative lands the
/// cursor exactly where the host last had it.
private func setDesktopMouse(_ on: Bool, reappearAt: (x: Int32, y: Int32)?) {
guard desktopMouse != on else { return }
let wasCaptured = captured
if wasCaptured { releaseCapture() }
desktopMouse = on
if wasCaptured { engageCapture(fromClick: false) }
window?.invalidateCursorRects(for: self)
if on, let p = reappearAt, let sp = cgScreenPoint(forHostX: p.x, p.y) {
CGWarpMouseCursorPosition(sp)
}
reconcileCursorRender()
}
/// The single cursor pull thread (both planes share the connection's cursor lock):
/// latest-wins state at a short timeout + a non-blocking shape poll per iteration.
/// Exits when the connection closes; events hop to main where all cursor state lives.
private func startCursorPump(_ connection: PunktfunkConnection) {
let thread = Thread { [weak self] in
while true {
do {
var newest: PunktfunkConnection.CursorStateEvent?
if let st = try connection.nextCursorState(timeoutMs: 100) {
newest = st
while let more = try connection.nextCursorState(timeoutMs: 0) {
newest = more // drain latest wins
}
}
while let shape = try connection.nextCursorShape(timeoutMs: 0) {
DispatchQueue.main.async { self?.applyCursorShape(shape) }
}
if let st = newest {
DispatchQueue.main.async { self?.applyCursorState(st) }
}
} catch {
return // connection closed the session is over
}
if self == nil { return }
}
}
thread.name = "pf-cursor-pump"
thread.start()
}
private func applyCursorShape(_ ev: PunktfunkConnection.CursorShapeEvent) {
guard let shape = Self.makeShape(ev) else {
streamInputLog.warning("cursor shape rejected (\(ev.width)x\(ev.height)) — keeping the previous cursor")
return
}
if hostCursors.count >= 64 { hostCursors.removeAll() } // degenerate host: reset
hostCursors[ev.serial] = shape
if cursorState?.serial == ev.serial {
window?.invalidateCursorRects(for: self)
}
}
private func applyCursorState(_ ev: PunktfunkConnection.CursorStateEvent) {
let prev = cursorState
cursorState = ev
if prev?.visible != ev.visible || prev?.serial != ev.serial {
window?.invalidateCursorRects(for: self)
}
// M3 host-driven auto-flip is DISABLED: `relative_hint` is derived from host cursor
// VISIBILITY, and Windows hides the pointer for ordinary desktop activity (clicking,
// typing) not just when a game grabs it. Acting on those transients flipped
// desktopcapturedesktop, which warped the cursor to view-centre and flushed held
// buttons (a spurious button-up ~200 ms into every press broke window drags). Until
// the host exposes a real pointer-LOCK signal (ClipCursor/raw-input, not visibility),
// the mouse model is user-driven only (M). The hint still rides the wire, unused.
_ = (lastHint, hintOverride)
}
/// Decode a forwarded straight-alpha RGBA shape into a CGImage + hotspot. The on-screen SIZE is
/// NOT baked in here it is applied per-use in `scaledCursor` from the live video-fit scale, so
/// the same shape re-fits across window resizes / retina moves without a re-forward.
private static func makeShape(_ ev: PunktfunkConnection.CursorShapeEvent) -> HostCursorShape? {
let (w, h) = (ev.width, ev.height)
guard w > 0, h > 0, ev.rgba.count >= w * h * 4,
let provider = CGDataProvider(data: ev.rgba as CFData),
let cg = CGImage(
width: w, height: h, bitsPerComponent: 8, bitsPerPixel: 32,
bytesPerRow: w * 4, space: CGColorSpaceCreateDeviceRGB(),
bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue),
provider: provider, decode: nil, shouldInterpolate: false,
intent: .defaultIntent)
else { return nil }
return HostCursorShape(
cg: cg, width: w, height: h,
hotX: min(ev.hotX, w - 1), hotY: min(ev.hotY, h - 1))
}
/// Points-per-host-pixel: the exact factor the video frame is aspect-fit into the view (the same
/// `AVMakeRect` fit `hostPoint`/`cgScreenPoint` use). The host forwards the pointer bitmap in host
/// framebuffer pixels the mode we drive is in the client's BACKING pixels, so on retina this is
/// ~1/backingScale and the pointer lands at its TRUE size relative to the streamed desktop
/// (crisp, 1:1 with the video) rather than the 2×-inflated pixel-as-points it used to be. Because
/// the bitmap grows with the host's display scaling (96 px at 300% DPI), scaling by this is what
/// keeps a high-DPI host from forwarding a giant pointer. Falls back to 1 before the first
/// mode/layout.
private func cursorFitScale() -> CGFloat {
guard let connection else { return 1 }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0, bounds.width > 0, bounds.height > 0 else { return 1 }
let fit = AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)), insideRect: bounds)
guard fit.width > 0 else { return 1 }
return fit.width / CGFloat(mode.width)
}
/// Build the `NSCursor` for a cached shape at the CURRENT video-fit scale (see `cursorFitScale`).
/// Both the image size and the hotspot scale together so the click point stays true.
private func scaledCursor(_ shape: HostCursorShape) -> NSCursor {
let scale = cursorFitScale()
let sw = max(1, (CGFloat(shape.width) * scale).rounded())
let sh = max(1, (CGFloat(shape.height) * scale).rounded())
let image = NSImage(cgImage: shape.cg, size: NSSize(width: sw, height: sh))
let hot = NSPoint(
x: min(CGFloat(shape.hotX) * scale, sw - 1),
y: min(CGFloat(shape.hotY) * scale, sh - 1))
return NSCursor(image: image, hotSpot: hot)
}
/// Host video px CG GLOBAL screen coordinates (top-left origin, the
/// `CGWarpMouseCursorPosition` convention `CursorCapture` established) through the
/// aspect-fit letterbox the inverse direction of `hostPoint(from:)`.
private func cgScreenPoint(forHostX hx: Int32, _ hy: Int32) -> CGPoint? {
guard let connection, let window else { return nil }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0 else { return nil }
let fit = AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
guard fit.width > 0, fit.height > 0 else { return nil }
let u = (CGFloat(hx) / CGFloat(mode.width)).clamped(to: 0...1)
let v = (CGFloat(hy) / CGFloat(mode.height)).clamped(to: 0...1)
let videoMinYTop = bounds.height - fit.maxY
let pTop = CGPoint(x: fit.minX + u * fit.width, y: videoMinYTop + v * fit.height)
let inView = CGPoint(x: pTop.x, y: bounds.height - pTop.y)
let inWindow = convert(inView, to: nil)
let onScreen = window.convertPoint(toScreen: inWindow)
let primaryHeight = NSScreen.screens.first?.frame.height ?? 0
return CGPoint(x: onScreen.x, y: primaryHeight - onScreen.y)
}
/// A single local monitor for motion + buttons, installed only while captured. A local /// A single local monitor for motion + buttons, installed only while captured. A local
/// monitor is more robust than view overrides for relative motion: it sidesteps the /// monitor is more robust than view overrides for relative motion: it sidesteps the
/// `window.acceptsMouseMovedEvents`/tracking-area/responder-chain requirements, and /// `window.acceptsMouseMovedEvents`/tracking-area/responder-chain requirements, and
@@ -647,12 +842,10 @@ public final class StreamLayerView: NSView {
streamInputLog.info("mouse-mode chord ignored: gamescope host is relative-only") streamInputLog.info("mouse-mode chord ignored: gamescope host is relative-only")
return return
} }
let wasCaptured = self.captured // A manual flip outranks the standing host hint until the hint next CHANGES.
if wasCaptured { self.releaseCapture() } self.hintOverride = true
self.desktopMouse.toggle() self.setDesktopMouse(!self.desktopMouse, reappearAt: nil)
if wasCaptured { self.engageCapture(fromClick: false) } streamInputLog.info("chord: mouse mode \(self.desktopMouse ? "desktop" : "capture", privacy: .public)")
self.window?.invalidateCursorRects(for: self)
streamInputLog.info("mouse mode: \(self.desktopMouse ? "desktop" : "capture", privacy: .public)")
} }
// The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by // The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by
// the monitor only while captured; the same key-window ownership rule as throughout. // the monitor only while captured; the same key-window ownership rule as throughout.
@@ -692,6 +885,14 @@ public final class StreamLayerView: NSView {
if mode == .desktop && !absOK { if mode == .desktop && !absOK {
streamInputLog.info("desktop mouse mode unavailable on a gamescope host (relative-only) — using capture") streamInputLog.info("desktop mouse mode unavailable on a gamescope host (relative-only) — using capture")
} }
// Cursor channel (M2): the host stopped compositing the pointer drain its shape/
// state planes and draw the pointer as the real NSCursor (plus the M3 auto-flip).
if connection.hostSupportsCursor {
cursorChannelActive = true
streamInputLog.info("cursor channel negotiated — host cursor renders locally")
startCursorPump(connection)
reconcileCursorRender() // initial render mode (a capture-model start composites)
}
// Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2 // Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2
// (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by // (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by
@@ -751,6 +952,11 @@ public final class StreamLayerView: NSView {
matchFollower?.noteSize( matchFollower?.noteSize(
widthPx: Int(px.width.rounded()), heightPx: Int(px.height.rounded())) widthPx: Int(px.width.rounded()), heightPx: Int(px.height.rounded()))
} }
// The video-fit scale just changed (resize / retina move); rebuild the worn host pointer at
// the new scale so it tracks the video instead of freezing at its build-time size.
if captured, desktopMouse, cursorChannelActive {
window?.invalidateCursorRects(for: self)
}
} }
public override func viewDidChangeBackingProperties() { public override func viewDidChangeBackingProperties() {
@@ -781,6 +987,14 @@ public final class StreamLayerView: NSView {
matchFollower = nil matchFollower = nil
lastDecodedContentSize = nil // the next session re-derives it from its first frame lastDecodedContentSize = nil // the next session re-derives it from its first frame
connection = nil connection = nil
// Cursor-channel state is per-session: without this reset a next session against a
// host WITHOUT the cap would wear this session's stale shapes (`cursorChannelActive`
// stayed latched true across sessions).
cursorChannelActive = false
cursorState = nil
hostCursors.removeAll()
sentClientDraws = nil
window?.invalidateCursorRects(for: self)
} }
deinit { deinit {
@@ -333,6 +333,13 @@ public final class StreamViewController: StreamViewControllerBase {
guard self?.captureEnabled == true else { return } guard self?.captureEnabled == true else { return }
connection?.send(event) connection?.send(event)
} }
// Apple Pencil the stylus plane, only against a pen-capable host (elsewhere the
// Pencil stays a finger, exactly as before). Same trust gate as touch.
streamView.penEnabled = connection.hostSupportsPen
streamView.onPenBatch = { [weak self, weak connection] batch in
guard self?.captureEnabled == true else { return }
connection?.sendPen(batch)
}
// Indirect pointer (mouse/trackpad) WITHOUT a lock absolute cursor + buttons + scroll. // Indirect pointer (mouse/trackpad) WITHOUT a lock absolute cursor + buttons + scroll.
// While the scene is pointer-LOCKED the GCMouse path owns motion AND buttons AND scroll, so // While the scene is pointer-LOCKED the GCMouse path owns motion AND buttons AND scroll, so
// the whole UIKit indirect path is gated off here (`gcMouseForwarding`). The trackpad and a // the whole UIKit indirect path is gated off here (`gcMouseForwarding`). The trackpad and a
@@ -499,6 +506,8 @@ public final class StreamViewController: StreamViewControllerBase {
// onTouchEvent can still deliver the button-up. // onTouchEvent can still deliver the button-up.
streamView.resetTouchInput() streamView.resetTouchInput()
streamView.onTouchEvent = nil streamView.onTouchEvent = nil
streamView.onPenBatch = nil // after reset the pen's leave-range sample rides it
streamView.penEnabled = false
streamView.onPointerMoveAbs = nil streamView.onPointerMoveAbs = nil
streamView.onPointerButton = nil streamView.onPointerButton = nil
streamView.onScroll = nil streamView.onScroll = nil
@@ -692,6 +701,12 @@ final class StreamLayerUIView: UIView {
/// Direct fingers / Pencil wire events: real touches in passthrough mode, or the /// Direct fingers / Pencil wire events: real touches in passthrough mode, or the
/// touch-driven mouse events (`TouchMouse`) in the trackpad/pointer modes. /// touch-driven mouse events (`TouchMouse`) in the trackpad/pointer modes.
var onTouchEvent: ((PunktfunkInputEvent) -> Void)? var onTouchEvent: ((PunktfunkInputEvent) -> Void)?
/// Apple Pencil state-full pen sample batches (the stylus plane). Active only while
/// `penEnabled`; without it the Pencil stays on the finger path exactly as before.
var onPenBatch: (([PunktfunkPenSample]) -> Void)?
/// The host advertised `HOST_CAP_PEN`, so Pencil input splits out of the finger path onto
/// the pen plane independent of the touch-input mode (drawing must not depend on it).
var penEnabled = false
/// Indirect pointer (mouse/trackpad with no lock) absolute cursor moves. /// Indirect pointer (mouse/trackpad with no lock) absolute cursor moves.
var onPointerMoveAbs: ((HostPoint) -> Void)? var onPointerMoveAbs: ((HostPoint) -> Void)?
/// Indirect-pointer buttons (GameStream ids: 1=left 3=right); `down` = press. /// Indirect-pointer buttons (GameStream ids: 1=left 3=right); `down` = press.
@@ -715,10 +730,21 @@ final class StreamLayerUIView: UIView {
/// The finger route latched at gesture start a Settings change mid-gesture applies to /// The finger route latched at gesture start a Settings change mid-gesture applies to
/// the NEXT touch, so one gesture never splits across input models. /// the NEXT touch, so one gesture never splits across input models.
private var fingerRoute: TouchInputMode? private var fingerRoute: TouchInputMode?
/// The Apple Pencil pipeline (contacts + hover + squeeze/tap pen samples).
private lazy var pencil: PencilStream = {
let stream = PencilStream()
stream.send = { [weak self] batch in self?.onPenBatch?(batch) }
stream.videoNorm = { [weak self] point in
guard let h = self?.hostPoint(from: point) else { return nil }
return (Float(h.x) / Float(max(h.w - 1, 1)), Float(h.y) / Float(max(h.h - 1, 1)))
}
return stream
}()
/// Release anything the touch-driven mouse holds and forget gesture state session stop. /// Release anything the touch-driven mouse holds and forget gesture state session stop.
func resetTouchInput() { func resetTouchInput() {
touchMouse.reset() touchMouse.reset()
pencil.reset() // leaves range the host lifts anything still inked
fingerRoute = nil fingerRoute = nil
setSoftKeyboardVisible(false) // a stream that's gone takes its keyboard with it setSoftKeyboardVisible(false) // a stream that's gone takes its keyboard with it
} }
@@ -755,6 +781,11 @@ final class StreamLayerUIView: UIView {
scrollPan.allowedScrollTypesMask = .all scrollPan.allowedScrollTypesMask = .all
scrollPan.allowedTouchTypes = [] scrollPan.allowedTouchTypes = []
addGestureRecognizer(scrollPan) addGestureRecognizer(scrollPan)
// Pencil squeeze / double-tap the pen plane's barrel buttons (no-op while
// `penEnabled` is false PencilStream ignores interactions out of range).
let pencilInteraction = UIPencilInteraction()
pencilInteraction.delegate = pencil
addInteraction(pencilInteraction)
#endif #endif
backgroundColor = .black backgroundColor = .black
} }
@@ -779,17 +810,32 @@ final class StreamLayerUIView: UIView {
private enum TouchKind { case down, move, up, cancel } private enum TouchKind { case down, move, up, cancel }
/// Split a touch batch by kind: an INDIRECT POINTER (mouse/trackpad with no lock) drives /// Split a touch batch by kind: an INDIRECT POINTER (mouse/trackpad with no lock) drives
/// the host cursor as an absolute mouse; everything else (direct finger, Pencil) is a host /// the host cursor as an absolute mouse; a Pencil goes to the pen plane when the host
/// touch. Mixed batches are possible, so partition rather than branch on the first touch. /// supports it; everything else (direct finger and the Pencil toward a pen-less host)
/// is a host touch. Mixed batches are possible, so partition rather than branch on the
/// first touch.
private func route(_ touches: Set<UITouch>, event: UIEvent?, kind: TouchKind) { private func route(_ touches: Set<UITouch>, event: UIEvent?, kind: TouchKind) {
var fingers: Set<UITouch> = [] var fingers: Set<UITouch> = []
var pencilTouches: Set<UITouch> = []
for touch in touches { for touch in touches {
if touch.type == .indirectPointer { if touch.type == .indirectPointer {
handleIndirectPointer(touch, event: event, kind: kind) handleIndirectPointer(touch, event: event, kind: kind)
} else if penEnabled, touch.type == .pencil {
pencilTouches.insert(touch)
} else { } else {
fingers.insert(touch) fingers.insert(touch)
} }
} }
if !pencilTouches.isEmpty {
let phase: PencilStream.Phase =
switch kind {
case .down: .down
case .move: .move
case .up: .up
case .cancel: .cancel
}
pencil.touches(pencilTouches, event: event, phase: phase, in: self)
}
if !fingers.isEmpty { forwardFingers(fingers, kind: kind) } if !fingers.isEmpty { forwardFingers(fingers, kind: kind) }
} }
@@ -861,8 +907,11 @@ final class StreamLayerUIView: UIView {
} }
} }
/// Button-less mouse/trackpad movement (no lock) absolute cursor move. /// Button-less mouse/trackpad movement (no lock) absolute cursor move unless it is a
/// hovering PENCIL (`zOffset > 0`) on a pen-capable host, which becomes in-range pen
/// samples (hover preview with distance/tilt/azimuth) instead of a cursor move.
@objc private func handleHover(_ recognizer: UIHoverGestureRecognizer) { @objc private func handleHover(_ recognizer: UIHoverGestureRecognizer) {
if penEnabled, pencil.maybeHover(recognizer, in: self) { return }
switch recognizer.state { switch recognizer.state {
case .began, .changed: case .began, .changed:
if let h = hostPoint(from: recognizer.location(in: self)) { onPointerMoveAbs?(h) } if let h = hostPoint(from: recognizer.location(in: self)) { onPointerMoveAbs?(h) }
+33 -3
View File
@@ -10,9 +10,17 @@ import {
showModal, showModal,
staticClasses, staticClasses,
} from "@decky/ui"; } from "@decky/ui";
import { definePlugin, routerHook } from "@decky/api"; import { definePlugin, routerHook, toaster } from "@decky/api";
import { FC } from "react"; import { FC } from "react";
import { FaDownload, FaLock, FaLockOpen, FaPlay, FaSyncAlt, FaTv } from "react-icons/fa"; import {
FaDownload,
FaLock,
FaLockOpen,
FaPlay,
FaPlus,
FaSyncAlt,
FaTv,
} from "react-icons/fa";
import { PluginErrorBoundary } from "./boundary"; import { PluginErrorBoundary } from "./boundary";
import { import {
applyUpdate, applyUpdate,
@@ -31,7 +39,19 @@ import {
import { streamPin } from "./library"; import { streamPin } from "./library";
import { PunktfunkRoute, ROUTE } from "./page"; import { PunktfunkRoute, ROUTE } from "./page";
import { PairModal } from "./pair"; import { PairModal } from "./pair";
import { ensureGamepadUiShortcut } from "./steam"; import { ensureGamepadUiShortcut, recreateShortcuts } from "./steam";
// Recovery action for "the Punktfunk library entry vanished" — recreates the visible shortcut.
// Deleting the shortcut (optionally + reinstalling the plugin) leaves a stale appId in Steam's
// CEF localStorage that self-heal fixes on the next mount, but this gives an in-session button
// that works even without a reload. Always ends in a toast so the tap has feedback.
async function recreatePunktfunkShortcut(): Promise<void> {
const appId = await recreateShortcuts();
toaster.toast({
title: "Punktfunk",
body: appId != null ? "Shortcut restored to your library" : "Couldn't create the shortcut",
});
}
// ---------------------------------------------------------------------------------------- // ----------------------------------------------------------------------------------------
// QAM panel — quick status + entry into the full page + one-tap stream for known hosts // QAM panel — quick status + entry into the full page + one-tap stream for known hosts
@@ -190,6 +210,16 @@ const QamPanel: FC = () => {
{checking ? "Checking…" : "Check for updates"} {checking ? "Checking…" : "Check for updates"}
</ButtonItem> </ButtonItem>
</PanelSectionRow> </PanelSectionRow>
<PanelSectionRow>
<ButtonItem
layout="below"
description="Missing the Punktfunk entry in your library? This puts it back."
onClick={() => void recreatePunktfunkShortcut()}
>
<FaPlus style={{ marginRight: "0.5em" }} />
Recreate library shortcut
</ButtonItem>
</PanelSectionRow>
</PanelSection> </PanelSection>
</> </>
); );
+54 -2
View File
@@ -55,6 +55,29 @@ declare const collectionStore:
| { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void } | { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void }
| undefined; | undefined;
// SteamUI's appStore indexes every registered app/shortcut by appId; a remembered appId whose
// overview is gone was deleted out from under us (the user removed the library entry). We must
// verify this because the remembered appId lives in Steam's CEF localStorage — which survives a
// plugin UNINSTALL/REINSTALL — so a manually-deleted shortcut otherwise leaves a dangling appId
// that the reuse path below silently repoints (SetShortcut* on a dead id is a no-op), and the
// entry never comes back.
declare const appStore:
| { GetAppOverviewByAppID?: (appId: number) => unknown | null }
| undefined;
/** True if a remembered appId still maps to a live Steam shortcut. When appStore is unavailable
* we can't tell, so assume it exists better to keep reusing than risk a duplicate library
* entry from a false "missing". A confident null means the shortcut was deleted recreate. */
function shortcutStillExists(appId: number): boolean {
try {
const get = appStore?.GetAppOverviewByAppID;
if (!get) return true; // no way to verify — preserve the reuse path
return get(appId) != null;
} catch {
return true;
}
}
/** Set a shortcut's library visibility (best-effort, deferred the overview registers a moment /** Set a shortcut's library visibility (best-effort, deferred the overview registers a moment
* after AddShortcut). Hides the stateful stream shortcut; keeps the gamepad-UI one visible. */ * after AddShortcut). Hides the stateful stream shortcut; keeps the gamepad-UI one visible. */
function setShortcutHidden(appId: number, hidden: boolean): void { function setShortcutHidden(appId: number, hidden: boolean): void {
@@ -199,8 +222,10 @@ async function ensureStreamShortcut(): Promise<{ appId: number; runner: string }
const startDir = info.runner.replace(/\/[^/]*$/, ""); // the plugin's bin/ dir const startDir = info.runner.replace(/\/[^/]*$/, ""); // the plugin's bin/ dir
void ensureControllerConfig(); // fire-and-forget — never blocks the launch void ensureControllerConfig(); // fire-and-forget — never blocks the launch
// Reuse the remembered shortcut only if it still exists — a stale appId (shortcut deleted, key
// outlived it across a reinstall) must fall through to AddShortcut, not be silently repointed.
const remembered = recall(STORAGE_KEY_STREAM); const remembered = recall(STORAGE_KEY_STREAM);
if (remembered != null) { if (remembered != null && shortcutStillExists(remembered)) {
SteamClient.Apps.SetShortcutExe(remembered, SHELL); SteamClient.Apps.SetShortcutExe(remembered, SHELL);
SteamClient.Apps.SetShortcutStartDir(remembered, startDir); SteamClient.Apps.SetShortcutStartDir(remembered, startDir);
SteamClient.Apps.SetShortcutName(remembered, SHORTCUT_NAME); SteamClient.Apps.SetShortcutName(remembered, SHORTCUT_NAME);
@@ -236,8 +261,11 @@ export async function ensureGamepadUiShortcut(): Promise<number | null> {
// home). %command% expands to the shortcut exe (/bin/sh); the wrapper rides behind as an arg. // home). %command% expands to the shortcut exe (/bin/sh); the wrapper rides behind as an arg.
const launchOpts = `PF_BROWSE=1 %command% "${info.runner}"`; const launchOpts = `PF_BROWSE=1 %command% "${info.runner}"`;
// Reuse the remembered entry only if it still exists; a stale appId (deleted shortcut whose
// localStorage key survived a plugin reinstall) falls through to AddShortcut so the visible
// library entry actually comes back instead of repointing a dead id.
let appId = recall(STORAGE_KEY_UI); let appId = recall(STORAGE_KEY_UI);
if (appId != null) { if (appId != null && shortcutStillExists(appId)) {
SteamClient.Apps.SetShortcutExe(appId, SHELL); SteamClient.Apps.SetShortcutExe(appId, SHELL);
SteamClient.Apps.SetShortcutStartDir(appId, startDir); SteamClient.Apps.SetShortcutStartDir(appId, startDir);
SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME); SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME);
@@ -256,6 +284,30 @@ export async function ensureGamepadUiShortcut(): Promise<number | null> {
} }
} }
/**
* Force the visible "Punktfunk" library entry back into existence the recovery button for
* "my shortcut disappeared". Drops any remembered appId that no longer maps to a live shortcut
* (so it can't shadow a fresh AddShortcut), then re-ensures. Safe to press anytime: a shortcut
* that still exists is left in place (no duplicate); a missing one is recreated. Covers the case
* self-heal-on-mount can't deleting the shortcut WITHOUT reinstalling (no mount no ensure).
* Returns the (new or existing) visible appId, or null on failure.
*/
export async function recreateShortcuts(): Promise<number | null> {
for (const key of [STORAGE_KEY_STREAM, STORAGE_KEY_UI]) {
const id = recall(key);
if (id != null && !shortcutStillExists(id)) {
try {
localStorage.removeItem(artKey(id)); // stale art marker for the dead appId
localStorage.removeItem(key);
} catch {
/* ignore */
}
}
}
// Recreate the visible entry now; the hidden stream shortcut re-registers lazily on next launch.
return ensureGamepadUiShortcut();
}
/** Launch the stateless gamepad-UI shortcut (console home) from the plugin, e.g. a QAM button. */ /** Launch the stateless gamepad-UI shortcut (console home) from the plugin, e.g. a QAM button. */
export async function launchGamepadUi(): Promise<void> { export async function launchGamepadUi(): Promise<void> {
const appId = await ensureGamepadUiShortcut(); const appId = await ensureGamepadUiShortcut();
+1
View File
@@ -547,6 +547,7 @@ impl AppModel {
crate::video::decodable_codecs(), // codecs (unused by the probe, but honest) crate::video::decodable_codecs(), // codecs (unused by the probe, but honest)
0, // preferred_codec: no preference 0, // preferred_codec: no preference
None, // display_hdr: probe connect, nothing presents None, // display_hdr: probe connect, nothing presents
0, // client_caps: probe connect, nothing renders a cursor
None, // launch: probe connect, no game None, // launch: probe connect, no game
pin, pin,
Some(identity), Some(identity),
+3
View File
@@ -523,6 +523,9 @@ async fn session(args: Args) -> Result<()> {
// writes it into the virtual display's EDID (CTA HDR block), so the EDID-forwarding // writes it into the virtual display's EDID (CTA HDR block), so the EDID-forwarding
// path can be validated headlessly (check the host's monitor caps / ADD log line). // path can be validated headlessly (check the host's monitor caps / ADD log line).
display_hdr: punktfunk_core::client::display_hdr_env_override(), display_hdr: punktfunk_core::client::display_hdr_env_override(),
// No CLIENT_CAP_CURSOR: this headless tool renders nothing — advertising it would
// just strip the pointer from the dumped bitstream.
client_caps: 0,
} }
.encode(), .encode(),
) )
+5
View File
@@ -172,6 +172,11 @@ mod session_main {
// defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session // defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session
// pump) pins one manually. // pump) pins one manually.
display_hdr: None, display_hdr: None,
// The presenter renders the host cursor locally in desktop mouse mode (M2 cursor
// channel); capture-mode sessions keep the composited cursor, so only advertise
// when the session STARTS in desktop mode. The host gates further (Linux portal
// compositors only).
cursor_forward: settings.mouse_mode() == trust::MouseMode::Desktop,
mic_enabled: settings.mic_enabled, mic_enabled: settings.mic_enabled,
clipboard, clipboard,
// The Settings preference (auto → VAAPI where it exists; the presenter // The Settings preference (auto → VAAPI where it exists; the presenter
+1
View File
@@ -56,6 +56,7 @@ pub fn run_speed_probe(
decodable_codecs(), decodable_codecs(),
0, // preferred_codec: no preference 0, // preferred_codec: no preference
None, // display_hdr: probe connect, nothing presents None, // display_hdr: probe connect, nothing presents
0, // client_caps: probe connect, nothing renders a cursor
None, // launch: no game None, // launch: no game
pin, pin,
Some(identity), Some(identity),
+8
View File
@@ -30,6 +30,12 @@ pipewire = "0.9"
libc = "0.2" libc = "0.2"
# ashpd 0.13 uses the tokio runtime for the one-time portal handshake (control plane). # ashpd 0.13 uses the tokio runtime for the one-time portal handshake (control plane).
tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] } tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] }
# XFixes cursor source for gamescope (remote-desktop-sweep Phase C): gamescope paints no
# `SPA_META_Cursor`, so the pointer never reaches the PipeWire node. We read the shape/hotspot/
# visibility from gamescope's nested Xwayland via XFixes instead and feed the existing cursor slot.
# `RustConnection` is the pure-Rust default (no libxcb link → no new C dependency on the host); the
# `xfixes` feature (auto-pulls `render` + `shape`) is what exposes GetCursorImage/SelectCursorInput.
x11rb = { version = "0.13", default-features = false, features = ["xfixes"] }
[target.'cfg(target_os = "windows")'.dependencies] [target.'cfg(target_os = "windows")'.dependencies]
# The host<->driver wire contract for the sealed frame channel (control IOCTL structs + frame header). # The host<->driver wire contract for the sealed frame channel (control IOCTL structs + frame header).
@@ -43,7 +49,9 @@ windows = { version = "0.62", features = [
"Win32_Graphics_Direct3D_Fxc", "Win32_Graphics_Direct3D_Fxc",
"Win32_Graphics_Dxgi", "Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common", "Win32_Graphics_Dxgi_Common",
"Win32_Graphics_Gdi",
"Win32_System_LibraryLoader", "Win32_System_LibraryLoader",
"Win32_System_StationsAndDesktops",
"Win32_System_Memory", "Win32_System_Memory",
"Win32_System_Threading", "Win32_System_Threading",
"Win32_UI_HiDpi", "Win32_UI_HiDpi",
+54
View File
@@ -69,6 +69,34 @@ pub trait Capturer: Send {
/// SDR / a backend that doesn't expose it (the default — Linux capture has no HDR path yet). /// SDR / a backend that doesn't expose it (the default — Linux capture has no HDR path yet).
/// The stream loop forwards this to the encoder (in-band SEI) and the client (`0xCE` datagram), /// The stream loop forwards this to the encoder (in-band SEI) and the client (`0xCE` datagram),
/// so the two stay a single source of truth. May change mid-session if the source is regraded. /// so the two stay a single source of truth. May change mid-session if the source is regraded.
/// The capture source's LIVE cursor state, when it arrives out-of-band from the frames
/// (the Windows IddCx hardware-cursor channel). Polled by the encode loop every tick and
/// preferred over `CapturedFrame::cursor` — with a hardware cursor, pointer-only moves
/// produce NO new frame, so the frame-attached overlay would go stale on a static desktop.
/// Default `None`: the Linux portal path attaches its cursor to frames instead.
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
None
}
/// LIVE cursor-render flip for a cursor-forward session (design/remote-desktop-sweep.md §8):
/// `on = true` — the client draws the pointer, keep it OUT of the video; `on = false` —
/// the capture mouse model, the pointer must be IN the video again. The Windows IDD
/// capturer implements the composite side ITSELF (slot-copy + alpha-blended quad from the
/// GDI poller) — a declared IddCx hardware cursor is irrevocable, so DWM can never be
/// handed the job back. Called every encode tick (implementations cache; steady state is
/// one compare). Default no-op: the Linux portal never bakes the pointer into frames —
/// the encode loop blends its overlay instead.
fn set_cursor_forward(&mut self, _on: bool) {}
/// Attach a gamescope cursor source (remote-desktop-sweep Phase C). gamescope paints no
/// `SPA_META_Cursor`, so [`cursor`](Self::cursor)'s slot stays empty — this hands the Linux
/// portal capturer gamescope's nested Xwayland `(DISPLAY, XAUTHORITY)` targets (it may run
/// several — one per `--xwayland-count`) so it reads the pointer shape/position over X11
/// (XFixes + QueryPointer), following whichever display is focused, and publishes it into that
/// same slot. Called once, after the capturer is built, only for gamescope sessions. Default
/// no-op: every non-gamescope capturer already has a cursor source.
fn attach_gamescope_cursor(&mut self, _targets: Vec<(String, Option<String>)>) {}
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> { fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
None None
} }
@@ -367,6 +395,26 @@ pub type FrameChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetFrameChannelRequest) -> Result<()> + Send + Sync, dyn Fn(&pf_driver_proto::control::SetFrameChannelRequest) -> Result<()> + Send + Sync,
>; >;
/// Delivery closure for the v5 hardware-cursor channel (`IOCTL_SET_CURSOR_CHANNEL`) — same
/// facade contract as [`FrameChannelSender`]. `Some` also OPTS THE SESSION IN: the capturer
/// creates + delivers the cursor section only when the host hands it a sender (the negotiated
/// cursor-forward sessions), and the driver only declares the hardware cursor once that
/// delivery lands — so a plain session keeps DWM's composited pointer untouched.
#[cfg(target_os = "windows")]
pub type CursorChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorChannelRequest) -> Result<()> + Send + Sync,
>;
/// The mid-stream cursor-render flip (`IOCTL_SET_CURSOR_FORWARD`, proto v6) as a host-facade
/// closure — same contract as [`CursorChannelSender`]. `bool` = declare the IddCx hardware
/// cursor (`true`) or stand it down (`false`; the host facade additionally forces the same-mode
/// re-commit that actualises the OS's software-cursor default). The capturer drives this from
/// its secure-desktop watch: UAC/Winlogon render only through the software-cursor path, so a
/// path pinned to the hardware cursor never presents them (the 0.18.0 secure-desktop
/// regression).
#[cfg(target_os = "windows")]
pub type CursorForwardSender = std::sync::Arc<dyn Fn(bool) -> Result<()> + Send + Sync>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads. // One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub mod pwinit; pub mod pwinit;
@@ -424,6 +472,7 @@ pub fn open_virtual_output(
allow_zerocopy: bool, allow_zerocopy: bool,
want_444: bool, want_444: bool,
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
expect_exact_dims: bool,
) -> Result<Box<dyn Capturer>> { ) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::from_virtual_output( linux::PortalCapturer::from_virtual_output(
remote_fd, remote_fd,
@@ -433,6 +482,7 @@ pub fn open_virtual_output(
allow_zerocopy, allow_zerocopy,
want_444, want_444,
policy, policy,
expect_exact_dims,
) )
.map(|c| Box::new(c) as Box<dyn Capturer>) .map(|c| Box::new(c) as Box<dyn Capturer>)
} }
@@ -450,6 +500,8 @@ pub fn open_idd_push(
pyrowave: bool, pyrowave: bool,
keepalive: Box<dyn Send>, keepalive: Box<dyn Send>,
sender: FrameChannelSender, sender: FrameChannelSender,
cursor_sender: Option<CursorChannelSender>,
cursor_forward: Option<CursorForwardSender>,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> { ) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open( idd_push::IddPushCapturer::open(
target, target,
@@ -459,6 +511,8 @@ pub fn open_idd_push(
pyrowave, pyrowave,
keepalive, keepalive,
sender, sender,
cursor_sender,
cursor_forward,
) )
.map(|c| Box::new(c) as Box<dyn Capturer>) .map(|c| Box::new(c) as Box<dyn Capturer>)
} }
+193 -12
View File
@@ -22,6 +22,10 @@
use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy}; use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy};
use anyhow::{anyhow, Context, Result}; use anyhow::{anyhow, Context, Result};
// gamescope cursor source (remote-desktop-sweep Phase C) — feeds `cursor_live` from XFixes when
// the PipeWire node carries no `SPA_META_Cursor` (gamescope's does not).
mod xfixes_cursor;
use std::os::fd::OwnedFd; use std::os::fd::OwnedFd;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, TryRecvError}; use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, TryRecvError};
@@ -57,6 +61,12 @@ pub struct PortalCapturer {
/// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream /// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream
/// is `Streaming`. /// is `Streaming`.
stall_since: Option<std::time::Instant>, stall_since: Option<std::time::Instant>,
/// The LIVE cursor overlay, published by the PipeWire thread from every buffer's
/// `SPA_META_Cursor` — including the cursor-only "corrupted" buffers that never become
/// frames. [`Capturer::cursor`] serves it so the encode loop's forwarder tracks pointer-only
/// motion on a static desktop; the frame-attached overlay alone goes stale between damage
/// frames (the same gap the Windows IddCx channel fills, and why the tick prefers LIVE).
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
/// True when this capture runs the VAAPI dmabuf passthrough (a LINEAR-dmabuf-only offer). If /// True when this capture runs the VAAPI dmabuf passthrough (a LINEAR-dmabuf-only offer). If
/// that offer never negotiates, [`next_frame`](Capturer::next_frame)'s timeout branch latches /// that offer never negotiates, [`next_frame`](Capturer::next_frame)'s timeout branch latches
/// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline /// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
@@ -82,6 +92,12 @@ pub struct PortalCapturer {
/// is, releasing the compositor-side output via the keepalive's own `Drop`. `None` for the /// is, releasing the compositor-side output via the keepalive's own `Drop`. `None` for the
/// portal source (its session ends with the portal thread's zbus connection). /// portal source (its session ends with the portal thread's zbus connection).
_keepalive: Option<Box<dyn Send>>, _keepalive: Option<Box<dyn Send>>,
/// The gamescope XFixes cursor reader (remote-desktop-sweep Phase C), when this capturer
/// serves a gamescope node. `Some` after
/// [`attach_gamescope_cursor`](Capturer::attach_gamescope_cursor); its `Drop` stops the reader
/// thread, so it lives exactly as long as the capturer. `None` on the portal path (its cursor
/// comes from `SPA_META_Cursor`).
_gs_cursor: Option<xfixes_cursor::XFixesCursorSource>,
} }
impl PortalCapturer { impl PortalCapturer {
@@ -114,10 +130,17 @@ impl PortalCapturer {
"ScreenCast portal session started; connecting PipeWire" "ScreenCast portal session started; connecting PipeWire"
); );
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before. // This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
Ok( Ok(spawn_pipewire(
spawn_pipewire(Some(fd), node_id, None, true, false, want_hdr, policy)? Some(fd),
.into_capturer(node_id, None), node_id,
) None,
true,
false,
want_hdr,
policy,
false,
)?
.into_capturer(node_id, None))
} }
/// Build a capturer from an already-created virtual output's PipeWire node. The host facade /// Build a capturer from an already-created virtual output's PipeWire node. The host facade
@@ -137,11 +160,13 @@ impl PortalCapturer {
allow_zerocopy: bool, allow_zerocopy: bool,
want_444: bool, want_444: bool,
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
expect_exact_dims: bool,
) -> Result<PortalCapturer> { ) -> Result<PortalCapturer> {
tracing::info!( tracing::info!(
node_id, node_id,
allow_zerocopy, allow_zerocopy,
want_444, want_444,
expect_exact_dims,
"connecting PipeWire to virtual output" "connecting PipeWire to virtual output"
); );
// Virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise 8-bit // Virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise 8-bit
@@ -154,6 +179,7 @@ impl PortalCapturer {
want_444, want_444,
false, false,
policy, policy,
expect_exact_dims,
)? )?
.into_capturer(node_id, Some(keepalive))) .into_capturer(node_id, Some(keepalive)))
} }
@@ -176,6 +202,8 @@ struct PwHandles {
hdr_offer: bool, hdr_offer: bool,
/// See [`PortalCapturer::hdr_negotiated`]. /// See [`PortalCapturer::hdr_negotiated`].
hdr_negotiated: Arc<AtomicBool>, hdr_negotiated: Arc<AtomicBool>,
/// See [`PortalCapturer::cursor_live`].
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
quit: ::pipewire::channel::Sender<()>, quit: ::pipewire::channel::Sender<()>,
join: thread::JoinHandle<()>, join: thread::JoinHandle<()>,
} }
@@ -195,10 +223,12 @@ impl PwHandles {
vaapi_dmabuf: self.vaapi_dmabuf, vaapi_dmabuf: self.vaapi_dmabuf,
hdr_offer: self.hdr_offer, hdr_offer: self.hdr_offer,
hdr_negotiated: self.hdr_negotiated, hdr_negotiated: self.hdr_negotiated,
cursor_live: self.cursor_live,
node_id, node_id,
quit: Some(self.quit), quit: Some(self.quit),
join: Some(self.join), join: Some(self.join),
_keepalive: keepalive, _keepalive: keepalive,
_gs_cursor: None,
} }
} }
} }
@@ -206,6 +236,7 @@ impl PwHandles {
/// Spawn the PipeWire consumer thread for `node_id` (fd `Some` = portal remote, `None` = /// Spawn the PipeWire consumer thread for `node_id` (fd `Some` = portal remote, `None` =
/// default daemon) and return its [`PwHandles`]. `preferred` seeds the format negotiation's /// default daemon) and return its [`PwHandles`]. `preferred` seeds the format negotiation's
/// default size/framerate — for Mutter virtual monitors this is what actually sizes the monitor. /// default size/framerate — for Mutter virtual monitors this is what actually sizes the monitor.
#[allow(clippy::too_many_arguments)]
fn spawn_pipewire( fn spawn_pipewire(
fd: Option<OwnedFd>, fd: Option<OwnedFd>,
node_id: u32, node_id: u32,
@@ -224,6 +255,12 @@ fn spawn_pipewire(
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way // Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6). // capture→encode edge (plan §W6).
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
// The producer's FIRST negotiation is for a sacrificial mode and a renegotiation to
// `preferred`'s dims is guaranteed to follow (KWin virtual outputs — see kwin.rs `create`):
// skip whole buffers until the negotiated size matches, so the pipeline never builds against
// the doomed birth mode. `false` everywhere else (Mutter SIZES the monitor from negotiation,
// gamescope fixates its own — gating those would starve legitimate first frames).
expect_exact_dims: bool,
) -> Result<PwHandles> { ) -> Result<PwHandles> {
// Frames flow from the pipewire thread over a small bounded channel. // Frames flow from the pipewire thread over a small bounded channel.
let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8); let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8);
@@ -237,6 +274,8 @@ fn spawn_pipewire(
let broken_cb = broken.clone(); let broken_cb = broken.clone();
let hdr_negotiated = Arc::new(AtomicBool::new(false)); let hdr_negotiated = Arc::new(AtomicBool::new(false));
let hdr_negotiated_cb = hdr_negotiated.clone(); let hdr_negotiated_cb = hdr_negotiated.clone();
let cursor_live = Arc::new(std::sync::Mutex::new(None::<pf_frame::CursorOverlay>));
let cursor_live_cb = cursor_live.clone();
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the // pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the // sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope. // inner `mod pipewire` shadows the crate name at this scope.
@@ -272,12 +311,14 @@ fn spawn_pipewire(
streaming_cb, streaming_cb,
broken_cb, broken_cb,
hdr_negotiated_cb, hdr_negotiated_cb,
cursor_live_cb,
zerocopy, zerocopy,
want_444, want_444,
want_hdr, want_hdr,
preferred, preferred,
quit_rx, quit_rx,
policy, policy,
expect_exact_dims,
) { ) {
tracing::error!(error = %format!("{e:#}"), "pipewire capture thread failed"); tracing::error!(error = %format!("{e:#}"), "pipewire capture thread failed");
} }
@@ -292,6 +333,7 @@ fn spawn_pipewire(
vaapi_dmabuf, vaapi_dmabuf,
hdr_offer: want_hdr, hdr_offer: want_hdr,
hdr_negotiated, hdr_negotiated,
cursor_live,
quit: quit_tx, quit: quit_tx,
join, join,
}) })
@@ -302,6 +344,24 @@ impl Capturer for PortalCapturer {
self.frame_within(Duration::from_secs(10)) self.frame_within(Duration::from_secs(10))
} }
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
// The PipeWire thread's live cursor slot (fed by every buffer's meta, frames or not) —
// lets the forwarder track pointer-only motion on a static desktop. See `cursor_live`.
// On a gamescope node the meta never arrives; the XFixes source (attached below) fills
// the same slot instead.
self.cursor_live.lock().ok().and_then(|slot| slot.clone())
}
fn attach_gamescope_cursor(&mut self, targets: Vec<(String, Option<String>)>) {
// gamescope paints no `SPA_META_Cursor`, so `cursor_live` would stay empty. Spawn the
// XFixes reader to publish gamescope's pointer into that SAME slot — `cursor()` above then
// serves it and the encode loop composites it, exactly like the portal path. It connects
// to every nested Xwayland and follows the focused one's pointer. A failure (no Xwayland /
// no XFixes) logs and leaves the slot empty = today's cursorless gamescope.
self._gs_cursor =
xfixes_cursor::XFixesCursorSource::spawn(targets, Arc::clone(&self.cursor_live));
}
fn next_frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> { fn next_frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
self.frame_within(budget) self.frame_within(budget)
} }
@@ -868,13 +928,21 @@ mod pipewire {
/// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves, /// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves,
/// so the GPU encoder re-uploads its cursor texture only on change. /// so the GPU encoder re-uploads its cursor texture only on change.
serial: u64, serial: u64,
/// The compositor-reported hotspot — carried on the overlay for the cursor-forward
/// channel (the blend path uses the pre-adjusted `x`/`y` and never reads it).
hot_x: i32,
hot_y: i32,
} }
impl CursorState { impl CursorState {
/// A shareable overlay for the GPU encode paths (blended at encode time), or `None` when /// A shareable overlay for the encode/forward paths, or `None` before the first bitmap
/// there is nothing to draw. Cheap: clones an `Arc` + a few scalars. /// arrived. A HIDDEN pointer still yields `Some` (with `visible: false`): the
/// cursor-forward channel needs "known but hidden" — an app grabbed the pointer, the
/// client's relative-mode hint (M3) — which is a different fact from "no cursor yet".
/// The encode loop strips invisible overlays before any blend path sees the frame.
/// Cheap: clones an `Arc` + a few scalars.
fn overlay(&self) -> Option<pf_frame::CursorOverlay> { fn overlay(&self) -> Option<pf_frame::CursorOverlay> {
if !self.visible || self.rgba.is_empty() { if self.rgba.is_empty() {
return None; return None;
} }
Some(pf_frame::CursorOverlay { Some(pf_frame::CursorOverlay {
@@ -884,6 +952,9 @@ mod pipewire {
h: self.bh, h: self.bh,
rgba: self.rgba.clone(), rgba: self.rgba.clone(),
serial: self.serial, serial: self.serial,
hot_x: self.hot_x.max(0) as u32,
hot_y: self.hot_y.max(0) as u32,
visible: self.visible,
}) })
} }
} }
@@ -934,6 +1005,21 @@ mod pipewire {
dbg_log_n: u64, dbg_log_n: u64,
/// Cursor-as-metadata state, composited into the CPU de-pad path (see `consume_frame`). /// Cursor-as-metadata state, composited into the CPU de-pad path (see `consume_frame`).
cursor: CursorState, cursor: CursorState,
/// LIVE overlay slot shared with [`super::PortalCapturer::cursor_live`] — refreshed after
/// every `update_cursor_meta`, including from cursor-only buffers that never become frames.
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
/// `Some((w, h))` while the producer's negotiated size is a sacrificial birth mode and a
/// renegotiation to these dims is guaranteed (KWin virtual outputs — kwin.rs `create`):
/// `.process` skips whole buffers until the negotiated size matches, then clears this
/// (self-disarming — later legitimate resizes are unaffected). `None` = no gating.
expect_dims: Option<(u32, u32)>,
/// Buffers skipped by the `expect_dims` gate (rate-limits its log).
gate_skips: u64,
/// When the gate first held a buffer — after [`GATE_DEADLINE`] with no renegotiation the
/// gate disarms and accepts what the producer serves (degraded dims beat a session wedged
/// into the first-frame-timeout retry loop; the promised renegotiation normally lands
/// within a frame or two).
gate_since: Option<std::time::Instant>,
} }
/// Consecutive tiled-import failures (worker alive, e.g. a per-buffer `EGL_BAD_MATCH`) before /// Consecutive tiled-import failures (worker alive, e.g. a per-buffer `EGL_BAD_MATCH`) before
@@ -1312,10 +1398,17 @@ mod pipewire {
value: pw::spa::pod::Value::Choice(pw::spa::pod::ChoiceValue::Int( value: pw::spa::pod::Value::Choice(pw::spa::pod::ChoiceValue::Int(
pw::spa::utils::Choice( pw::spa::utils::Choice(
pw::spa::utils::ChoiceFlags::empty(), pw::spa::utils::ChoiceFlags::empty(),
// The max must cover the producer's offer or the Meta param silently
// fails to negotiate and NO buffer ever carries the meta region:
// Mutter offers a FIXED `SPA_POD_Int(CURSOR_META_SIZE(384, 384))`
// (meta-screen-cast-stream-src.c, GNOME 50) — a 256² max made the
// intersection empty, which cost the whole Linux cursor channel
// on-glass. 1024² is headroom, not an allocation: the negotiated
// region follows the producer's value.
pw::spa::utils::ChoiceEnum::Range { pw::spa::utils::ChoiceEnum::Range {
default: meta_size(64, 64), default: meta_size(64, 64),
min: meta_size(1, 1), min: meta_size(1, 1),
max: meta_size(256, 256), max: meta_size(1024, 1024),
}, },
), ),
)), )),
@@ -1373,13 +1466,19 @@ mod pipewire {
) )
}; };
if id == 0 { if id == 0 {
// Compositor reports no visible pointer (e.g. a game grabbed/hid it). // SPA contract: id 0 = "no cursor information", NOT "cursor hidden". Mutter only
cursor.visible = false; // REWRITES a buffer's meta region when the cursor changed, so recycled buffers
// between damage frames carry a stale id-0 meta — treating that as hidden flickered
// the cursor off between hovers (on-glass round 5). Keep the last-known state; a
// pointer that really left/hid simply stops producing updates. (The M3 hidden hint
// loses its Mutter signal — Windows has its own CURSOR_SUPPRESSED source.)
return; return;
} }
cursor.visible = true; cursor.visible = true;
cursor.x = pos_x - hot_x; cursor.x = pos_x - hot_x;
cursor.y = pos_y - hot_y; cursor.y = pos_y - hot_y;
cursor.hot_x = hot_x;
cursor.hot_y = hot_y;
if bmp_off == 0 { if bmp_off == 0 {
// Position-only update — keep the cached bitmap. // Position-only update — keep the cached bitmap.
return; return;
@@ -1405,8 +1504,9 @@ mod pipewire {
(*bmp).offset as usize, (*bmp).offset as usize,
) )
}; };
// Ignore empty or implausibly large bitmaps (we requested <= 256×256). // Ignore empty or implausibly large bitmaps (the meta-size request covers <= 1024×1024;
if bw == 0 || bh == 0 || bw > 256 || bh > 256 { // real cursors are ≤96px — the cursor channel downscales >120px for the wire anyway).
if bw == 0 || bh == 0 || bw > 1024 || bh > 1024 {
return; return;
} }
let row = bw as usize * 4; let row = bw as usize * 4;
@@ -2006,6 +2106,9 @@ mod pipewire {
streaming: Arc<AtomicBool>, streaming: Arc<AtomicBool>,
broken: Arc<AtomicBool>, broken: Arc<AtomicBool>,
hdr_negotiated: Arc<AtomicBool>, hdr_negotiated: Arc<AtomicBool>,
// LIVE cursor publisher (see `PortalCapturer::cursor_live`): refreshed from every
// dequeued buffer's cursor meta, frames or not.
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
zerocopy: bool, zerocopy: bool,
// 4:4:4 session: tiled dmabufs take the worker's planar-YUV444 GPU convert. // 4:4:4 session: tiled dmabufs take the worker's planar-YUV444 GPU convert.
want_444: bool, want_444: bool,
@@ -2017,6 +2120,9 @@ mod pipewire {
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way // Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6). // capture→encode edge (plan §W6).
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
// See `spawn_pipewire`: the first negotiation is for a sacrificial mode; hold frames
// until the producer renegotiates to `preferred`'s dims.
expect_exact_dims: bool,
) -> Result<()> { ) -> Result<()> {
crate::pwinit::ensure_init(); crate::pwinit::ensure_init();
@@ -2201,6 +2307,14 @@ mod pipewire {
linear_nv12_failed: false, linear_nv12_failed: false,
dbg_log_n: 0, dbg_log_n: 0,
cursor: CursorState::default(), cursor: CursorState::default(),
cursor_live,
expect_dims: if expect_exact_dims {
preferred.map(|(w, h, _)| (w, h))
} else {
None
},
gate_skips: 0,
gate_since: None,
}; };
let stream = pw::stream::StreamBox::new( let stream = pw::stream::StreamBox::new(
@@ -2312,6 +2426,60 @@ mod pipewire {
newest = next; newest = next;
drained += 1; drained += 1;
} }
// Sacrificial-mode gate (kwin.rs `create`): until the producer renegotiates to the
// expected dims, every buffer — frame AND cursor meta, whose positions are in the
// doomed mode's space — belongs to the birth mode; consuming one would build the
// pipeline at the wrong size. Self-disarms on the first matching negotiation, or
// after `GATE_DEADLINE` without one — degraded dims beat wedging the session into
// the first-frame-timeout retry loop when the promised renegotiation never comes.
if let Some((ew, eh)) = ud.expect_dims {
/// The renegotiation normally lands within a frame or two of recording; well
/// past that, the producer is not going to deliver it (the on-glass case: the
/// real mode never actually applied) — stop starving the pipeline.
const GATE_DEADLINE: std::time::Duration = std::time::Duration::from_secs(3);
let sz = ud.info.size();
if sz.width == ew && sz.height == eh {
tracing::info!(
skipped = ud.gate_skips,
width = ew,
height = eh,
"producer renegotiated to the expected mode — frames flow"
);
ud.expect_dims = None;
} else if ud
.gate_since
.get_or_insert_with(std::time::Instant::now)
.elapsed()
> GATE_DEADLINE
{
tracing::warn!(
negotiated_w = sz.width,
negotiated_h = sz.height,
expected_w = ew,
expected_h = eh,
skipped = ud.gate_skips,
"producer never renegotiated to the expected mode — accepting its \
dims (session runs degraded rather than wedged)"
);
ud.expect_dims = None;
} else {
ud.gate_skips += 1;
if ud.gate_skips == 1 || ud.gate_skips.is_power_of_two() {
tracing::info!(
negotiated_w = sz.width,
negotiated_h = sz.height,
expected_w = ew,
expected_h = eh,
n = ud.gate_skips,
"holding frames until the producer renegotiates to the expected mode"
);
}
// SAFETY: `newest` was dequeued from this stream and not yet requeued;
// requeued exactly once here, then never touched (mirrors the null path).
unsafe { stream.queue_raw_buffer(newest) };
return;
}
}
// PipeWire dispatches from a C trampoline with no catch_unwind; a panic crossing that FFI // PipeWire dispatches from a C trampoline with no catch_unwind; a panic crossing that FFI
// boundary would abort the whole host. Contain the inspect/consume work — the only Rust // boundary would abort the whole host. Contain the inspect/consume work — the only Rust
// code here that can panic — and requeue `newest` unconditionally after it. // code here that can panic — and requeue `newest` unconditionally after it.
@@ -2325,6 +2493,19 @@ mod pipewire {
// pointer-only movements as metadata-only "corrupted" buffers we drop for their // pointer-only movements as metadata-only "corrupted" buffers we drop for their
// frame, but their cursor meta is fresh and must still move our overlay. // frame, but their cursor meta is fresh and must still move our overlay.
update_cursor_meta(&mut ud.cursor, spa_buf); update_cursor_meta(&mut ud.cursor, spa_buf);
// Publish the LIVE overlay (frames or not) so the encode loop's forwarder
// tracks pointer-only motion on a static desktop — the frame-attached overlay
// alone stales between damage frames. ONLY when we actually have one: a
// gamescope node carries no `SPA_META_Cursor`, so `overlay()` is always `None`
// here, and writing that would clobber — at frame rate — the `Some` the
// attached XFixes source publishes into this SAME slot, strobing the
// composited pointer on/off. Portal cursors are `None` only before the first
// bitmap (nothing to drop), and a HIDDEN pointer is still `Some(visible:false)`.
if let Some(overlay) = ud.cursor.overlay() {
if let Ok(mut slot) = ud.cursor_live.lock() {
*slot = Some(overlay);
}
}
// Inspect the newest buffer's header + first chunk for the diagnostic and the // Inspect the newest buffer's header + first chunk for the diagnostic and the
// CORRUPTED skip. SPA_META_Header is optional — `hdr` may be null. // CORRUPTED skip. SPA_META_Header is optional — `hdr` may be null.
@@ -0,0 +1,567 @@
//! XFixes cursor source for the gamescope capture path (remote-desktop-sweep Phase C).
//!
//! gamescope draws the pointer on a DRM hardware-cursor plane and its `paint_pipewire()`
//! deliberately excludes the cursor from the frame it feeds its built-in PipeWire node — so
//! `SPA_META_Cursor` never arrives and the ordinary [`cursor_live`](super::PortalCapturer) slot
//! stays empty (a KWin/GNOME session gets its cursor from that meta; gamescope can't embed one
//! either, its `set_hw_cursor` is inert). We instead read the pointer from gamescope's nested
//! Xwayland via X11 — the trick Sunshine uses — and publish a [`CursorOverlay`] into that same
//! slot, so the encoder blend composites the pointer into the video exactly like the portal path.
//!
//! **Multiple Xwaylands.** gamescope runs one Xwayland per `--xwayland-count` (Steam Gaming Mode
//! uses 2: one for Big Picture, one for the game). The pointer lives on whichever is FOCUSED — an
//! inactive display's pointer is frozen. So the source connects to ALL of them and publishes from
//! the one gamescope is actually drawing the pointer on; it reads that display's shape too, since
//! each Xwayland has its own current cursor. This is why a single-display read froze the pointer
//! the moment a game on the OTHER Xwayland took focus.
//!
//! Three X sources per display, split by cost (Sunshine's split, plus gamescope's own verdict):
//! * **Position** — core `QueryPointer` on the root, polled fast. Cheap (a few-byte reply, no
//! bitmap), so it can out-pace the stream fps and keep the composited pointer smooth.
//! * **Shape / hotspot** — `XFixesGetCursorImage`, refreshed only after an XFixes `CursorNotify`
//! (a real cursor change). A fully-transparent image reads as hidden.
//! * **Visibility + focus** — [`GAMESCOPE_CURSOR_VISIBLE_FEEDBACK`](GS_CURSOR_FEEDBACK) on the
//! root, read at connect and re-read on its `PropertyNotify`.
//!
//! **Why the feedback atom and not pointer motion.** gamescope hides its pointer by WARPING the X
//! pointer to the root's bottom-right corner pixel — it does NOT swap in a transparent X cursor, so
//! `XFixesGetCursorImage` keeps handing back the last opaque arrow. The original "follow whichever
//! display's pointer moved" heuristic therefore stuck to the parked display (a parked pointer never
//! moves again) and composited that arrow at `(w-1, h-1)`: a sliver of cursor welded to the corner
//! of the stream for the rest of the session, while the real pointer went undrawn. Reported
//! on-glass as "part of cursor shows up on bottom right … isn't where it really is", in every game
//! (a game grabs the pointer, so the hide is permanent). Measured on a live 1920x1080 Gaming Mode
//! session: real motion ⇒ pointer live + feedback `1`; 3 s idle (`--hide-cursor-delay 3000`) ⇒
//! pointer `(1919, 1079)` + feedback `0`. The atom answers BOTH questions correctly for a static
//! pointer, which motion cannot: which Xwayland owns it, and whether to draw it at all — so
//! honouring it also gives the stream gamescope's own idle auto-hide, which this source never had.
use std::sync::{
atomic::{AtomicBool, Ordering},
Arc, Mutex,
};
use std::time::Duration;
use pf_frame::CursorOverlay;
use x11rb::connection::Connection;
use x11rb::errors::ReplyError;
use x11rb::protocol::xfixes::{self, ConnectionExt as _, GetCursorImageReply};
use x11rb::protocol::xproto::{
Atom, AtomEnum, ChangeWindowAttributesAux, ConnectionExt as _, EventMask, QueryPointerReply,
Window,
};
use x11rb::protocol::Event;
use x11rb::rust_connection::RustConnection;
/// Serializes the brief `XAUTHORITY` env swap around a connect (the var is process-global). Only
/// ever contended if two gamescope sessions start at once — rare, and the swap is microseconds.
static XAUTH_LOCK: Mutex<()> = Mutex::new(());
/// Position out-paces the stream fps (`POLL`); shape rides `CursorNotify` events drained each tick.
/// 4 ms ≈ 250 Hz matches the Windows GDI poller — the polled position IS the composited position
/// and must out-run a 240 fps session or the pointer stutters.
const POLL: Duration = Duration::from_millis(4);
/// gamescope's own pointer verdict, published on EVERY nested Xwayland's root: `1` on the server
/// whose pointer gamescope is currently drawing, `0` on the others — and `0` on all of them once
/// the pointer is hidden (a game grabbed it, or `--hide-cursor-delay` fired). See the module docs
/// for why this, and not pointer motion, is the signal this source follows.
const GS_CURSOR_FEEDBACK: &str = "GAMESCOPE_CURSOR_VISIBLE_FEEDBACK";
/// Self-heal cadence for the feedback re-read: `PropertyNotify` drives it, this only covers a
/// missed/coalesced event (and a gamescope that publishes the atom after we connected). One
/// `GetProperty` per display per interval is nothing next to the 250 Hz pointer poll.
const FEEDBACK_RESYNC: Duration = Duration::from_millis(250);
/// A running XFixes cursor reader. Dropping it stops the worker thread and joins it, releasing the
/// X connections — so it lives exactly as long as the capturer that owns it.
pub(super) struct XFixesCursorSource {
stop: Arc<AtomicBool>,
join: Option<std::thread::JoinHandle<()>>,
}
impl XFixesCursorSource {
/// Connect to every gamescope nested Xwayland in `targets` (`(DISPLAY, XAUTHORITY)`) and start
/// publishing cursor overlays into `slot`, following the focused display's pointer. Returns
/// `None` — and logs — if NONE can be used (no X connection / no XFixes), so the caller
/// degrades to no gamescope cursor (today's behaviour) instead of failing the session.
pub(super) fn spawn(
targets: Vec<(String, Option<String>)>,
slot: Arc<Mutex<Option<CursorOverlay>>>,
) -> Option<Self> {
// Connect on the caller's thread so failures degrade cleanly and the displays are validated
// before we commit a thread.
let mut displays = Vec::new();
for (dpy, xauth) in targets {
match connect(&dpy, xauth.as_deref()) {
Ok((conn, root, feedback)) => {
displays.push(XDisplay::new(dpy, conn, root, feedback))
}
Err(e) => tracing::warn!(
dpy = %dpy,
error = %e,
"gamescope cursor: skipping a nested Xwayland we can't use"
),
}
}
if displays.is_empty() {
tracing::warn!(
"gamescope cursor: no usable nested Xwayland — no in-video pointer this session \
(falls back to today's cursorless gamescope stream)"
);
return None;
}
let names: Vec<&str> = displays.iter().map(|d| d.name.as_str()).collect();
let feedback = displays.iter().any(|d| d.gs_visible.is_some());
tracing::info!(
displays = ?names,
cursor_feedback = feedback,
"gamescope cursor: XFixes source live — following the Xwayland gamescope draws the \
pointer on (cursor_feedback=false this gamescope publishes no \
GAMESCOPE_CURSOR_VISIBLE_FEEDBACK, degrading to the pointer-motion heuristic)"
);
let stop = Arc::new(AtomicBool::new(false));
let stop_worker = Arc::clone(&stop);
let join = std::thread::Builder::new()
.name("pf-gs-cursor".into())
.spawn(move || run(displays, slot, stop_worker))
.ok()?;
Some(XFixesCursorSource {
stop,
join: Some(join),
})
}
}
impl Drop for XFixesCursorSource {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(j) = self.join.take() {
let _ = j.join();
}
}
}
/// Open the X connection, negotiate XFixes, and select cursor-change + root-property events —
/// returning the connection, root window and this display's initial
/// [`GAMESCOPE_CURSOR_FEEDBACK`](GS_CURSOR_FEEDBACK) reading (`(atom, value)`; the value is `None`
/// when gamescope publishes no such property here). `RustConnection` reads `XAUTHORITY` from the
/// env at connect time only, so set it under the lock (the host isn't a gamescope child), connect,
/// then restore.
type Connected = (RustConnection, Window, (Atom, Option<bool>));
fn connect(dpy: &str, xauthority: Option<&str>) -> Result<Connected, String> {
let (conn, screen_num) = {
let _g = XAUTH_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let prev = std::env::var_os("XAUTHORITY");
if let Some(x) = xauthority {
std::env::set_var("XAUTHORITY", x);
}
let out = RustConnection::connect(Some(dpy));
match (&prev, xauthority) {
(Some(p), _) => std::env::set_var("XAUTHORITY", p),
(None, Some(_)) => std::env::remove_var("XAUTHORITY"),
(None, None) => {}
}
out.map_err(|e| format!("connect: {e}"))?
};
// XFixes ≥ 1 gives GetCursorImage / SelectCursorInput; ask for a modern minor, take what we get.
conn.xfixes_query_version(5, 0)
.map_err(ReplyError::from)
.and_then(|c| c.reply())
.map_err(|e| format!("XFixes unavailable: {e}"))?;
let root = conn
.setup()
.roots
.get(screen_num)
.ok_or_else(|| format!("no X screen {screen_num}"))?
.root;
// Wake the worker's event drain whenever the cursor shape changes (incl. hide/show).
conn.xfixes_select_cursor_input(root, xfixes::CursorNotifyMask::DISPLAY_CURSOR)
.map_err(ReplyError::from)
.and_then(|c| c.check())
.map_err(|e| format!("SelectCursorInput: {e}"))?;
// …and whenever gamescope republishes its cursor verdict. Interned with `only_if_exists=false`
// so we hold a matchable atom id even on a gamescope that sets the property later; a failure to
// select PROPERTY_CHANGE is NOT fatal — the resync re-read still tracks it, just at 250 ms.
let feedback_atom = conn
.intern_atom(false, GS_CURSOR_FEEDBACK.as_bytes())
.map_err(ReplyError::from)
.and_then(|c| c.reply())
.map(|r| r.atom)
.unwrap_or(0);
if let Ok(c) = conn.change_window_attributes(
root,
&ChangeWindowAttributesAux::new().event_mask(EventMask::PROPERTY_CHANGE),
) {
let _ = c.check();
}
let _ = conn.flush();
let feedback = read_cursor_feedback(&conn, root, feedback_atom);
Ok((conn, root, (feedback_atom, feedback)))
}
/// Read `GAMESCOPE_CURSOR_VISIBLE_FEEDBACK` off `root`. `None` = the property is absent or
/// unreadable (not a gamescope that publishes it) — the caller then falls back to the pointer-motion
/// heuristic rather than blanking the cursor, so an older gamescope keeps today's behaviour.
fn read_cursor_feedback(conn: &RustConnection, root: Window, atom: Atom) -> Option<bool> {
if atom == 0 {
return None;
}
let reply = conn
.get_property(false, root, atom, AtomEnum::CARDINAL, 0, 1)
.ok()?
.reply()
.ok()?;
let value = reply.value32()?.next()?;
Some(value != 0)
}
/// One gamescope Xwayland the source tracks.
struct XDisplay {
name: String,
conn: RustConnection,
root: Window,
/// Last polled pointer position — a change since the previous tick marks this display FOCUSED.
last_pos: Option<(i32, i32)>,
/// Cached cursor shape, refreshed only after this display's XFixes `CursorNotify`.
shape: Shape,
/// A `CursorNotify` (or first read) is pending — fetch the shape when this display is active.
need_shape: bool,
/// Interned [`GS_CURSOR_FEEDBACK`] atom (`0` = intern failed — treated as absent).
feedback_atom: Atom,
/// gamescope's verdict for THIS display: `Some(true)` = it is drawing the pointer here,
/// `Some(false)` = it is not, `None` = this gamescope publishes no verdict at all.
gs_visible: Option<bool>,
/// The X connection died (game/Xwayland exited) — skip it.
dead: bool,
}
impl XDisplay {
fn new(
name: String,
conn: RustConnection,
root: Window,
(feedback_atom, gs_visible): (Atom, Option<bool>),
) -> Self {
XDisplay {
name,
conn,
root,
last_pos: None,
shape: Shape::default(),
need_shape: true,
feedback_atom,
gs_visible,
dead: false,
}
}
/// Re-read gamescope's verdict, keeping a previously-seen one if the read fails (a transient
/// failure must not look like "this gamescope has no feedback" and re-arm the motion heuristic).
fn resync_feedback(&mut self) {
let fresh = read_cursor_feedback(&self.conn, self.root, self.feedback_atom);
if fresh.is_some() || self.gs_visible.is_none() {
self.gs_visible = fresh;
}
}
}
/// Cached cursor shape for one display.
#[derive(Default)]
struct Shape {
/// Straight-alpha RGBA (`w*h*4`, bytes R,G,B,A); empty before the first image arrives.
rgba: Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
/// XFixes' own per-display cursor serial — bumps on every shape change.
serial: u64,
/// A hidden pointer arrives as an all-transparent image; kept so a position-only tick preserves
/// the last known visibility.
visible: bool,
}
fn run(
mut displays: Vec<XDisplay>,
slot: Arc<Mutex<Option<CursorOverlay>>>,
stop: Arc<AtomicBool>,
) {
let mut active = 0usize;
// The overlay serial must bump whenever the DRAWN cursor changes — either the active display's
// shape OR which display is active (per-display XFixes serials aren't comparable across
// displays, so switching could reuse a number and the encoder would keep the old texture).
let mut out_serial = 0u64;
let mut last_key = (usize::MAX, u64::MAX);
let mut warned_image = false;
let mut last_resync = std::time::Instant::now();
while !stop.load(Ordering::Relaxed) {
// A missed/coalesced PropertyNotify would otherwise strand the verdict — re-read on a slow
// cadence so the source always converges (also picks the atom up if gamescope adds it late).
let resync = last_resync.elapsed() >= FEEDBACK_RESYNC;
if resync {
last_resync = std::time::Instant::now();
}
// 1) Poll every display's pointer; note which moved since last tick (the fallback focus
// signal, used only when this gamescope publishes no cursor verdict).
let mut active_moved = false;
let mut other_moved: Option<usize> = None;
for (i, d) in displays.iter_mut().enumerate() {
if d.dead {
continue;
}
// Drain pending events. Two kinds are selected: XFixes CursorNotify (the shape
// changed) and root PropertyNotify (gamescope republished its cursor verdict, among
// the many other properties it keeps on the root — hence the atom match).
let mut need_feedback = resync;
loop {
match d.conn.poll_for_event() {
Ok(Some(Event::XfixesCursorNotify(_))) => d.need_shape = true,
Ok(Some(Event::PropertyNotify(ev))) => {
need_feedback |= d.feedback_atom != 0 && ev.atom == d.feedback_atom;
}
Ok(Some(_)) => {}
Ok(None) => break,
Err(_) => {
d.dead = true;
break;
}
}
}
if need_feedback && !d.dead {
d.resync_feedback();
}
match fetch_pointer(&d.conn, d.root) {
Ok(p) if p.same_screen => {
let pos = (i32::from(p.root_x), i32::from(p.root_y));
let moved = d.last_pos.is_some_and(|lp| lp != pos);
d.last_pos = Some(pos);
if moved {
if i == active {
active_moved = true;
} else if other_moved.is_none() {
other_moved = Some(i);
}
}
}
Ok(_) => {} // pointer on another screen — keep the last position.
Err(_) => d.dead = true,
}
}
// 2) Pick the display to publish from, and decide whether a pointer should be drawn at all.
let states: Vec<(bool, Option<bool>)> =
displays.iter().map(|d| (d.dead, d.gs_visible)).collect();
let hidden_by_gamescope;
(active, hidden_by_gamescope) = pick_active(&states, active, active_moved, other_moved);
if displays.get(active).is_none_or(|d| d.dead) {
match displays.iter().position(|d| !d.dead) {
Some(k) => active = k,
None => {
std::thread::sleep(POLL); // all connections dead — idle until Drop.
continue;
}
}
}
// 3) Fetch the active display's shape if a CursorNotify (or a focus switch) left it stale.
if displays[active].need_shape {
match fetch_cursor_image(&displays[active].conn) {
Ok(img) => {
update_shape(&mut displays[active].shape, &img);
displays[active].need_shape = false;
}
Err(e) => {
if !warned_image {
warned_image = true;
tracing::warn!(error = %e, "gamescope cursor: GetCursorImage failed — retrying");
}
}
}
}
// 4) Publish the ACTIVE display's pointer + shape (or clear the slot when it has no cursor
// of its own — so a focus switch never leaves the other display's stale pointer showing).
// A pointer gamescope is not drawing is published `visible: false`, NOT dropped: the
// encode loop overwrites the frame's overlay from this slot and strips invisible ones, so
// a `None` here would leave the last visible overlay standing on repeat frames.
let d = &displays[active];
let drawn = d.shape.visible && !hidden_by_gamescope;
let overlay = match (d.last_pos, d.shape.rgba.is_empty()) {
(Some((px, py)), false) => {
let key = (active, d.shape.serial);
if key != last_key {
out_serial += 1;
last_key = key;
}
Some(CursorOverlay {
// Top-left = pointer position hotspot (the overlay contract).
x: px - d.shape.hot_x as i32,
y: py - d.shape.hot_y as i32,
w: d.shape.w,
h: d.shape.h,
rgba: Arc::clone(&d.shape.rgba),
serial: out_serial,
hot_x: d.shape.hot_x,
hot_y: d.shape.hot_y,
visible: drawn,
})
}
_ => None,
};
if let Ok(mut s) = slot.lock() {
*s = overlay;
}
std::thread::sleep(POLL);
}
}
/// Which display to publish from, and whether gamescope is drawing NO pointer right now —
/// `(active, hidden)`. `states` is one `(dead, gs_visible)` per display, in `displays` order.
///
/// PREFERRED: gamescope's own verdict. It is authoritative for a STATIC pointer, which is exactly
/// the case motion cannot read — a game grabs the pointer, gamescope parks it in the bottom-right
/// corner, and "follow whichever moved" then never switches again (see the module docs).
///
/// FALLBACK, only when NO live display publishes the verdict: the original motion heuristic —
/// sticky to the active display while its pointer moves (no flapping), else follow another that
/// moved. `hidden` is never asserted on this path, so an older gamescope keeps today's behaviour.
fn pick_active(
states: &[(bool, Option<bool>)],
active: usize,
active_moved: bool,
other_moved: Option<usize>,
) -> (usize, bool) {
let live = |&(dead, _): &(bool, Option<bool>)| !dead;
if states.iter().filter(|s| live(s)).any(|(_, v)| v.is_some()) {
return match states.iter().position(|s| live(s) && s.1 == Some(true)) {
// gamescope is drawing the pointer here — publish from it.
Some(i) => (i, false),
// Drawing none of them (a game grabbed it, or the idle auto-hide fired). Keep `active`
// so the shape cache and last position stay warm for the re-show; the caller publishes
// the overlay `visible: false` instead of dropping it.
None => (active, true),
};
}
match other_moved {
Some(j) if !active_moved => (j, false),
_ => (active, false),
}
}
/// Update `shape` from a fresh `GetCursorImage` reply. A hidden pointer (all-transparent) keeps the
/// last bitmap (instant re-show) but flips visibility; the serial still bumps so the change shows.
fn update_shape(shape: &mut Shape, img: &GetCursorImageReply) {
let visible =
img.width > 0 && img.height > 0 && img.cursor_image.iter().any(|&p| (p >> 24) & 0xff != 0);
if visible {
shape.rgba = Arc::new(argb_premul_to_straight_rgba(&img.cursor_image));
shape.w = u32::from(img.width);
shape.h = u32::from(img.height);
shape.hot_x = u32::from(img.xhot);
shape.hot_y = u32::from(img.yhot);
}
shape.visible = visible;
shape.serial = u64::from(img.cursor_serial);
}
/// One request+reply — x11rb splits errors (the request is `ConnectionError`, `reply()` is
/// `ReplyError` which is `From<ConnectionError>`), so the request `?` converts into the reply error.
fn fetch_cursor_image(conn: &RustConnection) -> Result<GetCursorImageReply, ReplyError> {
conn.xfixes_get_cursor_image()?.reply()
}
fn fetch_pointer(conn: &RustConnection, root: Window) -> Result<QueryPointerReply, ReplyError> {
conn.query_pointer(root)?.reply()
}
/// XFixes cursor pixels are packed `0xAARRGGBB` with **premultiplied** alpha (the Xrender / Xcursor
/// convention). The overlay + both blend paths want **straight** alpha RGBA (R,G,B,A bytes), like
/// the `SPA_META_Cursor` path — so un-premultiply here. (If on-glass shows over-bright fringes the
/// source wasn't premultiplied after all; drop the divide.)
fn argb_premul_to_straight_rgba(argb: &[u32]) -> Vec<u8> {
let mut out = Vec::with_capacity(argb.len() * 4);
for &px in argb {
let a = (px >> 24) & 0xff;
let r = (px >> 16) & 0xff;
let g = (px >> 8) & 0xff;
let b = px & 0xff;
let (r, g, b) = match a {
0 => (0, 0, 0),
255 => (r, g, b),
a => (
((r * 255 + a / 2) / a).min(255),
((g * 255 + a / 2) / a).min(255),
((b * 255 + a / 2) / a).min(255),
),
};
out.extend_from_slice(&[r as u8, g as u8, b as u8, a as u8]);
}
out
}
#[cfg(test)]
mod tests {
use super::pick_active;
/// Steam Gaming Mode shape: display 0 = Big Picture's Xwayland, 1 = the game's.
const BPM: usize = 0;
const GAME: usize = 1;
#[test]
fn follows_the_display_gamescope_draws_on() {
// Verdict beats motion: gamescope says the game's Xwayland owns the pointer, so we publish
// from it even though only BPM's (parked) pointer looks like it moved.
let states = [(false, Some(false)), (false, Some(true))];
assert_eq!(pick_active(&states, BPM, false, Some(BPM)), (GAME, false));
}
/// THE REGRESSION: gamescope hides the pointer by warping it to the root's bottom-right corner
/// and leaves the opaque arrow as the X cursor. Nothing moves ever again, so the motion
/// heuristic stayed on the parked display and composited that arrow at (w-1, h-1) — a sliver of
/// cursor welded to the corner of the stream for the whole session, in every game.
#[test]
fn a_pointer_gamescope_draws_nowhere_is_hidden_not_parked() {
let states = [(false, Some(false)), (false, Some(false))];
// `active` is kept (shape cache + last position stay warm for the re-show) but hidden.
assert_eq!(pick_active(&states, GAME, false, None), (GAME, true));
}
#[test]
fn re_show_returns_to_the_drawing_display() {
let states = [(false, Some(true)), (false, Some(false))];
assert_eq!(pick_active(&states, GAME, false, None), (BPM, false));
}
#[test]
fn a_dead_displays_verdict_is_ignored() {
// The game's Xwayland exited mid-session with a stale `Some(true)`; BPM is the live one.
let states = [(false, Some(true)), (true, Some(true))];
assert_eq!(pick_active(&states, GAME, false, None), (BPM, false));
// …and a dead display's `Some` must not count as "this gamescope publishes a verdict",
// which would blank the cursor forever on a gamescope that publishes none.
let states = [(false, None), (true, Some(false))];
assert_eq!(pick_active(&states, BPM, false, Some(GAME)), (GAME, false));
}
#[test]
fn no_verdict_falls_back_to_the_motion_heuristic() {
let none = [(false, None), (false, None)];
// Sticky while the active display's own pointer moves…
assert_eq!(pick_active(&none, BPM, true, Some(GAME)), (BPM, false));
// …otherwise follow the one that moved…
assert_eq!(pick_active(&none, BPM, false, Some(GAME)), (GAME, false));
// …and never assert `hidden` on this path (that would regress an older gamescope to a
// cursorless stream).
assert_eq!(pick_active(&none, BPM, false, None), (BPM, false));
}
}
+2 -2
View File
@@ -161,7 +161,7 @@ pub fn install_gpu_pref_hook() {
}); });
} }
unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u8>> { pub(crate) unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u8>> {
let mut blob: Option<ID3DBlob> = None; let mut blob: Option<ID3DBlob> = None;
let mut errs: Option<ID3DBlob> = None; let mut errs: Option<ID3DBlob> = None;
let r = D3DCompile( let r = D3DCompile(
@@ -194,7 +194,7 @@ unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u
} }
/// Fullscreen-triangle vertex shader for the HDR conversion pass (3 verts, no input layout). /// Fullscreen-triangle vertex shader for the HDR conversion pass (3 verts, no input layout).
const HDR_VS: &str = r" pub(crate) const HDR_VS: &str = r"
struct VOut { float4 pos : SV_POSITION; float2 uv : TEXCOORD0; }; struct VOut { float4 pos : SV_POSITION; float2 uv : TEXCOORD0; };
VOut main(uint vid : SV_VertexID) { VOut main(uint vid : SV_VertexID) {
float2 uv = float2((vid << 1) & 2, vid & 2); float2 uv = float2((vid << 1) & 2, vid & 2);
+494 -22
View File
@@ -365,6 +365,12 @@ pub unsafe fn verify_is_wudfhost(process: HANDLE, wudf_pid: u32, what: &str) ->
#[path = "idd_push/channel.rs"] #[path = "idd_push/channel.rs"]
mod channel; mod channel;
#[path = "idd_push/cursor.rs"]
mod cursor;
#[path = "idd_push/cursor_blend.rs"]
mod cursor_blend;
#[path = "idd_push/cursor_poll.rs"]
mod cursor_poll;
#[path = "idd_push/descriptor.rs"] #[path = "idd_push/descriptor.rs"]
mod descriptor; mod descriptor;
#[path = "idd_push/stall.rs"] #[path = "idd_push/stall.rs"]
@@ -386,6 +392,64 @@ pub struct IddPushCapturer {
/// The sealed channel's handle-duplication broker (WUDFHost process + control device); used at open /// The sealed channel's handle-duplication broker (WUDFHost process + control device); used at open
/// and again on every ring recreate to deliver fresh duplicates. /// and again on every ring recreate to deliver fresh duplicates.
broker: ChannelBroker, broker: ChannelBroker,
/// The v5 hardware-cursor channel's host end (`Some` = delivered; the driver declared the
/// hardware cursor and seqlock-publishes into it). Survives ring recreates — the section is
/// independent of the frame ring's generation. With the channel delivered, the driver's
/// hardware cursor keeps DWM from compositing ANY cursor into the frame; the SHAPE now comes
/// from [`cursor_poll::CursorPoller`] (the IddCx query is alpha-only — see cursor_poll.rs),
/// and this shm read is the fallback if that poller dies.
cursor_shared: Option<cursor::CursorShared>,
/// The GDI cursor-shape poller (design §8): the overlay source while alive. `Some` when
/// `cursor_shared` is (both ride the negotiated cursor channel + successful delivery) — or
/// when `composite_forced` (no channel, but the target's sticky declare needs a blend source).
cursor_poll: Option<cursor_poll::CursorPoller>,
/// Retained delivery sender (`IOCTL_SET_CURSOR_CHANNEL`) for RE-delivery: a driver-side
/// monitor re-arrival (match-window re-arrival resize, a sibling session recreating the
/// shared slot) destroys the driver's cursor worker — the section here survives, so the
/// channel is re-delivered on ring recreates.
cursor_sender: Option<crate::CursorChannelSender>,
/// The cursor-render flip sender (`IOCTL_SET_CURSOR_FORWARD`) — the secure-desktop guard's
/// actuator. UAC/Winlogon render only through the OS's software-cursor path (its default on
/// every mode commit); with our hardware cursor declared (and re-declared on every
/// swap-chain assign) that path never comes back, and the secure desktop never presents —
/// the stream freezes on the last normal-desktop frame for the whole UAC/lock interaction.
/// [`Self::poll_secure_desktop`] flips the declare off/on at the secure-desktop edges.
cursor_forward: Option<crate::CursorForwardSender>,
/// The secure-desktop guard's edge state: `true` = the poller reports a secure input
/// desktop and the declare is currently stood down.
secure_active: bool,
/// The CAPTURE mouse model is active — the HOST composites the pointer into the frame
/// (see cursor_blend.rs for why DWM cannot: a declared IddCx hardware cursor is forever).
composite_cursor: bool,
/// This session never negotiated the cursor channel but its target carries an IRREVOCABLE
/// hardware-cursor declare from an earlier session (`WinCaptureTarget::cursor_excluded`,
/// §8.6): DWM delivers pointer-free frames and no client draws the cursor, so the ONLY path
/// to a visible pointer is compositing here. Pins `composite_cursor` on — nothing may turn
/// it off (there is no channel to hand the pointer to).
composite_forced: bool,
/// The cursor-quad blend pass (lazy; per capture device). `None` after a build failure —
/// composite mode then degrades to pointer-less frames (warned once).
cursor_blend: Option<cursor_blend::CursorBlendPass>,
cursor_blend_failed: bool,
/// The frame-sized blend scratch (slot copy + cursor quad): texture + SRV + (w, h, fmt)
/// it was built for — rebuilt when the ring geometry changes.
blend_scratch: Option<(
ID3D11Texture2D,
ID3D11ShaderResourceView,
u32,
u32,
DXGI_FORMAT,
)>,
/// The (serial, x, y, visible) of the LAST blended pointer — the composite-regen change
/// key: pointer-only motion produces no driver publish (the declared hardware cursor
/// doesn't dirty frames), so `try_consume` regenerates from the last slot when this moves.
last_blend_key: Option<(u64, i32, i32, bool)>,
/// The ring slot of the last FRESH publish — the regen source.
last_slot: Option<usize>,
/// The target's SDR-white scale (vs 80 nits) for HDR cursor compositing — refreshed on
/// each blend-scratch rebuild (first use + ring geometry changes). 2.5 ≈ the Windows
/// SDR-brightness default; without it the composited cursor renders visibly dark on HDR.
sdr_white_scale: f32,
width: u32, width: u32,
height: u32, height: u32,
slots: Vec<HostSlot>, slots: Vec<HostSlot>,
@@ -611,11 +675,22 @@ impl IddPushCapturer {
pyrowave: bool, pyrowave: bool,
keepalive: Box<dyn Send>, keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender, sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> { ) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so // The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime. // the stall log can correlate DWM holes with OS display events for the session's lifetime.
pf_win_display::display_events::spawn_once(); pf_win_display::display_events::spawn_once();
match Self::open_inner(target, preferred, client_10bit, want_444, pyrowave, sender) { match Self::open_inner(
target,
preferred,
client_10bit,
want_444,
pyrowave,
sender,
cursor_sender,
cursor_forward,
) {
Ok(mut me) => { Ok(mut me) => {
me._keepalive = keepalive; me._keepalive = keepalive;
Ok(me) Ok(me)
@@ -632,6 +707,8 @@ impl IddPushCapturer {
want_444: bool, want_444: bool,
pyrowave: bool, pyrowave: bool,
sender: crate::FrameChannelSender, sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> Result<Self> { ) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the // The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor // selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -654,6 +731,8 @@ impl IddPushCapturer {
pyrowave, pyrowave,
luid, luid,
sender.clone(), sender.clone(),
cursor_sender.clone(),
cursor_forward.clone(),
) { ) {
Ok(me) => Ok(me), Ok(me) => Ok(me),
Err(e) => { Err(e) => {
@@ -687,6 +766,8 @@ impl IddPushCapturer {
pyrowave, pyrowave,
drv, drv,
sender, sender,
cursor_sender,
cursor_forward,
) )
.context("IDD-push rebind to the driver's reported render adapter") .context("IDD-push rebind to the driver's reported render adapter")
} }
@@ -702,6 +783,8 @@ impl IddPushCapturer {
pyrowave: bool, pyrowave: bool,
luid: LUID, luid: LUID,
sender: crate::FrameChannelSender, sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
cursor_forward: Option<crate::CursorForwardSender>,
) -> Result<Self> { ) -> Result<Self> {
let (pw, ph, _hz) = preferred let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?; .context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -935,6 +1018,61 @@ impl IddPushCapturer {
) )
.context("deliver IDD-push frame channel to the driver")?; .context("deliver IDD-push frame channel to the driver")?;
// v5 hardware-cursor channel (M2c): create + deliver the CursorShm section. Failure
// is NON-fatal — the driver never declares the hardware cursor without this delivery,
// so the session degrades to today's composited pointer (and the forwarder simply
// never sees a live overlay).
let cursor_shared = cursor_sender.as_ref().and_then(|send_cursor| {
match cursor::CursorShared::create(target.target_id) {
Ok(cs) => {
// Deliver via the shared helper (also used for RE-delivery after a
// driver-side monitor re-arrival destroyed the worker).
deliver_cursor_channel(&broker, target.target_id, &cs, send_cursor)
.then_some(cs)
}
Err(e) => {
tracing::warn!(
"cursor section creation failed (composited cursor stays): {e:#}"
);
None
}
}
});
// No channel this session, but the target's sticky declare (an EARLIER session's —
// irrevocable, §8.6) keeps DWM's frames pointer-free with no client drawing either:
// the only visible pointer is the one composited here, so force composite mode on.
let composite_forced = target.cursor_excluded && cursor_sender.is_none();
if composite_forced {
tracing::info!(
target_id = target.target_id,
"target carries an irrevocable hardware-cursor declare from an earlier \
desktop-mode session and this session has no cursor channel the host \
composites the pointer into frames (forced, for the session's life)"
);
}
// The GDI shape poller rides the SAME gate as the delivered channel: with the driver's
// hardware cursor keeping the frame cursor-free, the poller supplies the full-fidelity
// shape (masked/monochrome included — the IddCx query can't; see cursor_poll.rs).
// Forced-composite sessions need it too — it is their only shape/position source.
let cursor_poll = (cursor_shared.is_some() || composite_forced).then(|| {
// Safety of the CCD call: read-only QueryDisplayConfig over owned locals (same
// call CursorShared::create makes) — already inside open_on's unsafe region.
let rect = pf_win_display::win_display::source_desktop_rect(target.target_id)
.unwrap_or((0, 0, i32::MAX, i32::MAX));
cursor_poll::CursorPoller::spawn(target.target_id, rect)
});
// Heal the driver's persisted cursor-forward state: a session that died on the
// secure desktop (client drops at the lock screen — the common case) leaves the
// per-target desired state `false`, and the NEXT session's channel delivery would
// adopt UNdeclared (the exact cross-session composite trap of §8.6). A fresh
// session always starts declared; the secure-desktop guard re-disables if the
// secure desktop is (still) up, via its first `poll_secure_desktop` edge.
if let (Some(_), Some(fwd)) = (cursor_shared.as_ref(), cursor_forward.as_ref()) {
if let Err(e) = fwd(true) {
tracing::debug!("cursor-forward reset at open failed (pre-v6 driver?): {e:#}");
}
}
tracing::info!( tracing::info!(
target_id = target.target_id, target_id = target.target_id,
wudf_pid = target.wudf_pid, wudf_pid = target.wudf_pid,
@@ -993,6 +1131,19 @@ impl IddPushCapturer {
last_seq: 0, last_seq: 0,
last_present: None, last_present: None,
status_logged: false, status_logged: false,
cursor_shared,
cursor_poll,
cursor_sender,
cursor_forward,
secure_active: false,
composite_cursor: composite_forced,
composite_forced,
cursor_blend: None,
cursor_blend_failed: false,
blend_scratch: None,
last_blend_key: None,
last_slot: None,
sdr_white_scale: 1.0,
// Held from BEFORE the first-frame gate (the display must not idle off while we // Held from BEFORE the first-frame gate (the display must not idle off while we
// wait for the first compose) until the capturer drops with the session. // wait for the first compose) until the capturer drops with the session.
_display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(), _display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(),
@@ -1337,6 +1488,14 @@ impl IddPushCapturer {
"IDD push: frame-channel re-delivery failed after ring recreate" "IDD push: frame-channel re-delivery failed after ring recreate"
); );
} }
// Ring recreates ride display churn that can also have re-arrived the MONITOR driver-side
// (destroying its cursor worker with it) — re-deliver the surviving cursor section so the
// hardware-cursor declaration follows the CURRENT monitor generation.
if let (Some(cs), Some(send)) = (self.cursor_shared.as_ref(), self.cursor_sender.as_ref()) {
let _ = deliver_cursor_channel(&self.broker, self.target_id, cs, send);
}
self.blend_scratch = None; // ring geometry/format changed — rebuild at next blend
self.last_slot = None; // old-ring slot indices are meaningless now
self.last_seq = 0; self.last_seq = 0;
self.out_ring.clear(); // the output format changed → rebuild lazily at the new format self.out_ring.clear(); // the output format changed → rebuild lazily at the new format
self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode
@@ -1622,8 +1781,201 @@ impl IddPushCapturer {
Ok(Some((self.pyro_fence_handle, value))) Ok(Some((self.pyro_fence_handle, value)))
} }
/// The (serial, x, y, visible) of the CURRENT polled cursor — the composite-regen change
/// key. `None` while the poller has no shape yet (or isn't running).
fn cursor_blend_key(&self) -> Option<(u64, i32, i32, bool)> {
self.cursor_poll
.as_ref()
.and_then(|p| p.read())
.map(|o| (o.serial, o.x, o.y, o.visible))
}
/// Composite the pointer for this convert: ensure the frame-sized blend scratch, copy the
/// slot into it, and alpha-blend the GDI poller's shape at its polled position. Returns the
/// scratch (texture + SRV) the conversion should read INSTEAD of the slot; `None` degrades
/// to the pointer-less slot (scratch/pass creation failed — warned once). A hidden pointer
/// blends nothing (the plain copy is the correct frame).
///
/// # Safety
/// D3D11 calls on the owning capture/encode thread's device + immediate context, called
/// while holding the slot's keyed mutex (the copy reads the slot).
unsafe fn prepare_blend_scratch(
&mut self,
slot_tex: &ID3D11Texture2D,
) -> Option<(ID3D11Texture2D, ID3D11ShaderResourceView)> {
let fmt = self.ring_format();
// (Re)build the scratch at the current ring geometry.
let stale = self
.blend_scratch
.as_ref()
.is_none_or(|(_, _, w, h, f)| (*w, *h, *f) != (self.width, self.height, fmt));
if stale {
self.blend_scratch = None;
let desc = D3D11_TEXTURE2D_DESC {
Width: self.width,
Height: self.height,
MipLevels: 1,
ArraySize: 1,
Format: fmt,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
..Default::default()
};
let mut tex: Option<ID3D11Texture2D> = None;
let built = self
.device
.CreateTexture2D(&desc, None, Some(&mut tex))
.ok()
.and(tex)
.and_then(|t| {
let mut srv: Option<ID3D11ShaderResourceView> = None;
self.device
.CreateShaderResourceView(&t, None, Some(&mut srv))
.ok()
.and(srv)
.map(|v| (t, v))
});
match built {
Some((t, v)) => {
self.blend_scratch = Some((t, v, self.width, self.height, fmt));
if self.display_hdr {
// Where DWM places SDR white on this HDR desktop — the composited
// cursor must match or it reads dark (~2.5x at the Windows default).
// Queried only here: scratch rebuilds are rare, and the CCD query
// contends on the display-config lock, which must stay OFF the
// per-frame path.
// Safety: read-only CCD query over owned locals (within unsafe fn).
let queried =
pf_win_display::win_display::sdr_white_level_scale(self.target_id);
self.sdr_white_scale = queried.unwrap_or(self.sdr_white_scale);
tracing::info!(
target_id = self.target_id,
queried = ?queried,
applied = self.sdr_white_scale,
"cursor composite: HDR SDR-white scale (1.0 = 80 nits; None = \
query failed keeping the prior value)"
);
}
}
None => {
if !self.cursor_blend_failed {
self.cursor_blend_failed = true;
tracing::warn!(
"cursor blend scratch creation failed — capture-model frames stay \
pointer-less this session"
);
}
return None;
}
}
}
let (tex, srv, ..) = self.blend_scratch.as_ref().expect("just ensured");
let (tex, srv) = (tex.clone(), srv.clone());
self.context.CopyResource(&tex, slot_tex);
// Blend the pointer (visible shapes only; hidden = the copy alone is the frame).
let overlay = self.cursor_poll.as_ref().and_then(|p| p.read());
self.last_blend_key = overlay.as_ref().map(|o| (o.serial, o.x, o.y, o.visible));
if let Some(ov) = overlay.filter(|o| o.visible) {
if self.cursor_blend.is_none() && !self.cursor_blend_failed {
match cursor_blend::CursorBlendPass::new(&self.device) {
Ok(p) => self.cursor_blend = Some(p),
Err(e) => {
self.cursor_blend_failed = true;
tracing::warn!(
"cursor blend pass build failed — capture-model frames stay \
pointer-less this session: {e:#}"
);
}
}
}
if let Some(pass) = self.cursor_blend.as_mut() {
// FP16 ring = scRGB linear composition (HDR): linearize the sRGB shape and
// scale it to the target's SDR white so it matches the desktop around it.
let scale = if self.display_hdr {
self.sdr_white_scale
} else {
0.0
};
if let Err(e) = pass.blend(&self.device, &self.context, &tex, &ov, scale) {
if !self.cursor_blend_failed {
self.cursor_blend_failed = true;
tracing::warn!("cursor blend draw failed — pointer-less frames: {e:#}");
}
}
}
}
Some((tex, srv))
}
/// The secure-desktop guard (the 0.18.0 UAC/Winlogon regression). UAC consent and Winlogon
/// live on the SECURE desktop, which the OS renders through the software-cursor path — its
/// per-mode-commit default. With this session's IddCx hardware cursor declared (and
/// re-declared by the driver on every swap-chain assign), that path never materialises, the
/// secure desktop never presents into our swap-chain, and the stream freezes on the last
/// normal-desktop frame for the entire UAC/lock interaction. On the poller's secure edge:
/// stand the declare down (`SET_CURSOR_FORWARD` off — the driver stops its per-assign
/// re-declare — plus the host facade's forced same-mode re-commit that actualises the
/// software cursor); on dismissal, re-declare. Runs on the capture/encode thread every tick
/// (it must keep running while frames are stalled — that is exactly the state it exits).
fn poll_secure_desktop(&mut self) {
let Some(fwd) = self.cursor_forward.as_ref() else {
return;
};
// Sessions with a declare possibly in play: the channel session that declared it, and
// the forced-composite session whose (reused) driver monitor may still run an earlier
// session's cursor worker. A plain session on a clean target has no poller — no guard.
if self.cursor_shared.is_none() && !self.composite_forced {
return;
}
let secure = self
.cursor_poll
.as_ref()
.is_some_and(|p| p.secure_desktop());
if secure == self.secure_active {
return;
}
self.secure_active = secure;
if secure {
tracing::info!(
target_id = self.target_id,
"secure desktop (UAC/Winlogon) active — standing the IddCx hardware-cursor \
declare down so the OS software-cursor path can render it"
);
if let Err(e) = fwd(false) {
tracing::warn!(
"secure-desktop cursor-forward stand-down failed (secure content may stay \
invisible this session): {e:#}"
);
}
} else {
tracing::info!(
target_id = self.target_id,
"secure desktop dismissed — restoring the cursor render model"
);
// Re-declare only for the session that RUNS the cursor channel; a forced-composite
// session never wanted the declare (leaving the driver's desired state off also
// stops a reused worker's per-assign re-declares for good — the next channel
// session's open-time reset re-arms it).
if self.cursor_shared.is_some() {
if let Err(e) = fwd(true) {
tracing::warn!(
"secure-desktop cursor-forward re-enable failed (client-drawn cursor \
may double with a composited one): {e:#}"
);
}
}
}
}
fn try_consume(&mut self) -> Result<Option<CapturedFrame>> { fn try_consume(&mut self) -> Result<Option<CapturedFrame>> {
self.log_driver_status_once(); self.log_driver_status_once();
// The secure-desktop guard first: while UAC/Winlogon is up there may be NO fresh frames
// at all — this edge is what brings them back.
self.poll_secure_desktop();
// Follow the display: a "Use HDR" flip recreates the ring at the matching format. // Follow the display: a "Use HDR" flip recreates the ring at the matching format.
self.poll_display_hdr(); self.poll_display_hdr();
// Recover-or-drop (GB1): if a descriptor change triggered a recreate but no fresh frame has resumed // Recover-or-drop (GB1): if a descriptor change triggered a recreate but no fresh frame has resumed
@@ -1682,9 +2034,25 @@ impl IddPushCapturer {
return Ok(None); return Ok(None);
} }
let seq = u64::from(tok.seq); let seq = u64::from(tok.seq);
let slot = tok.slot as usize; let mut slot = tok.slot as usize;
if seq == self.last_seq || slot >= self.slots.len() { let fresh = seq != self.last_seq && slot < self.slots.len();
return Ok(None); let mut regen = false;
if !fresh {
// Composite cursor model: pointer-only motion produces NO new publish (the declared
// hardware cursor never dirties the frame), so a static desktop would freeze the
// blended pointer. Regenerate from the LAST slot whenever the polled cursor state
// changed — the re-converted out-ring frame carries the pointer's new position.
let moved = self.composite_cursor
&& self.last_slot.is_some()
&& self.cursor_blend_key() != self.last_blend_key;
if !moved {
return Ok(None);
}
slot = self.last_slot.expect("checked above");
if slot >= self.slots.len() {
return Ok(None); // ring shrank across a recreate — wait for a fresh publish
}
regen = true;
} }
// Build the ring + converter BEFORE acquiring the slot so nothing between Acquire and Release // Build the ring + converter BEFORE acquiring the slot so nothing between Acquire and Release
// can `?`-return and leak the keyed-mutex lock (which would stall the driver on that slot). // can `?`-return and leak the keyed-mutex lock (which would stall the driver on that slot).
@@ -1718,18 +2086,31 @@ impl IddPushCapturer {
// Hold the slot's keyed mutex only across the convert/copy into the host out-ring (NOT across the // Hold the slot's keyed mutex only across the convert/copy into the host out-ring (NOT across the
// ~3 ms encode — NVENC reads the host out-ring slot, not the keyed-mutex slot), so the driver gets // ~3 ms encode — NVENC reads the host out-ring slot, not the keyed-mutex slot), so the driver gets
// the slot back immediately and the encode of the PREVIOUS frame overlaps this convert. // the slot back immediately and the encode of the PREVIOUS frame overlaps this convert.
let s = &self.slots[slot]; // Clone the slot's COM interfaces (an AddRef each) so the guard borrows LOCALS, leaving
// `self` free for the composite blend prep inside the lock.
let (slot_tex, slot_srv, slot_mutex) = {
let s = &self.slots[slot];
(s.tex.clone(), s.srv.clone(), s.mutex.clone())
};
// Acquire the slot's keyed mutex via a RAII guard, scoped to JUST the convert/copy below so it // Acquire the slot's keyed mutex via a RAII guard, scoped to JUST the convert/copy below so it
// releases at the same point as the old hand-written `ReleaseSync` (the driver gets the slot back // releases at the same point as the old hand-written `ReleaseSync` (the driver gets the slot back
// immediately, NOT held across the rest of `try_consume`) — but now leak-proof on any early return. // immediately, NOT held across the rest of `try_consume`) — but now leak-proof on any early return.
{ {
let Some(_lock) = KeyedMutexGuard::acquire(&s.mutex, 0, 8) else { let Some(_lock) = KeyedMutexGuard::acquire(&slot_mutex, 0, 8) else {
return Ok(None); return Ok(None);
}; };
// SAFETY: convert on the owning (encode) thread's immediate context, holding the slot lock. // SAFETY: convert on the owning (encode) thread's immediate context, holding the slot lock.
// A `?` here is leak-safe: `_lock` (the KeyedMutexGuard) drops on the early return, releasing // A `?` here is leak-safe: `_lock` (the KeyedMutexGuard) drops on the early return, releasing
// the slot back to the driver. // the slot back to the driver.
unsafe { unsafe {
// Composite cursor model: divert the convert input through the blend scratch —
// a slot copy with the pointer quad alpha-blended on top. `None` = compositing
// off or degraded (the conversion then reads the slot as always).
let blended = if self.composite_cursor {
self.prepare_blend_scratch(&slot_tex)
} else {
None
};
if self.pyrowave { if self.pyrowave {
// PyroWave: ring slot SRV (BGRA for SDR, scRGB FP16 for HDR) → the two separate // PyroWave: ring slot SRV (BGRA for SDR, scRGB FP16 for HDR) → the two separate
// plane textures via the mode-aware CSC; the shared fence signalled just after // plane textures via the mode-aware CSC; the shared fence signalled just after
@@ -1737,22 +2118,17 @@ impl IddPushCapturer {
// convert. The composition format is pinned to the negotiated depth. // convert. The composition format is pinned to the negotiated depth.
let (_, y_rtv, _, cbcr_rtv) = pyro_slot.as_ref().expect("pyro slot"); let (_, y_rtv, _, cbcr_rtv) = pyro_slot.as_ref().expect("pyro slot");
if let Some(conv) = self.pyro_conv.as_ref() { if let Some(conv) = self.pyro_conv.as_ref() {
conv.convert( let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
&self.context, conv.convert(&self.context, src, y_rtv, cbcr_rtv, self.width, self.height)?;
&s.srv,
y_rtv,
cbcr_rtv,
self.width,
self.height,
)?;
} }
} else if self.display_hdr { } else if self.display_hdr {
// HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010. // HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010.
if let Some(conv) = self.hdr_p010_conv.as_ref() { if let Some(conv) = self.hdr_p010_conv.as_ref() {
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
conv.convert( conv.convert(
&self.device, &self.device,
&self.context, &self.context,
&s.srv, src,
out.as_ref().expect("out ring"), out.as_ref().expect("out ring"),
self.width, self.width,
self.height, self.height,
@@ -1762,12 +2138,14 @@ impl IddPushCapturer {
// SDR 4:4:4: pass the BGRA slot through untouched — NVENC ingests full-chroma // SDR 4:4:4: pass the BGRA slot through untouched — NVENC ingests full-chroma
// RGB and CSCs to YUV 4:4:4 itself (per the always-written BT.709 VUI). Plain // RGB and CSCs to YUV 4:4:4 itself (per the always-written BT.709 VUI). Plain
// copy-engine move; the slot releases back to the driver immediately. // copy-engine move; the slot releases back to the driver immediately.
let src = blended.as_ref().map(|(t, _)| t).unwrap_or(&slot_tex);
self.context self.context
.CopyResource(out.as_ref().expect("out ring"), &s.tex); .CopyResource(out.as_ref().expect("out ring"), src);
} else { } else {
// SDR: BGRA slot → NV12 on the VIDEO engine; NVENC takes native NV12, no SM-side CSC. // SDR: BGRA slot → NV12 on the VIDEO engine; NVENC takes native NV12, no SM-side CSC.
if let Some(conv) = self.video_conv.as_ref() { if let Some(conv) = self.video_conv.as_ref() {
conv.convert(&s.tex, out.as_ref().expect("out ring"))?; let src = blended.as_ref().map(|(t, _)| t).unwrap_or(&slot_tex);
conv.convert(src, out.as_ref().expect("out ring"))?;
} }
} }
} }
@@ -1775,13 +2153,20 @@ impl IddPushCapturer {
} }
self.out_idx = (i + 1) % ring_len; self.out_idx = (i + 1) % ring_len;
self.last_seq = seq; self.last_seq = seq;
if fresh {
self.last_slot = Some(slot);
}
if let Some((y, _, cbcr, _)) = pyro_slot.as_ref() { if let Some((y, _, cbcr, _)) = pyro_slot.as_ref() {
self.pyro_last = Some((y.clone(), cbcr.clone())); self.pyro_last = Some((y.clone(), cbcr.clone()));
} else { } else {
self.last_present = Some((out.as_ref().expect("out ring").clone(), pf)); self.last_present = Some((out.as_ref().expect("out ring").clone(), pf));
} }
let now = Instant::now(); let now = Instant::now();
if self.recovering_since.take().is_some() { if regen {
// A regen re-encodes OLD desktop content at a new pointer position — it is not a
// fresh driver frame; feeding the freshness/stall bookkeeping would mask a dead
// driver and pollute stall attribution.
} else if self.recovering_since.take().is_some() {
// A fresh frame resumed → recovered. The recovery gap is self-inflicted (ring // A fresh frame resumed → recovered. The recovery gap is self-inflicted (ring
// recreate, already logged by the recreate path) — reset the stall watch so it // recreate, already logged by the recreate path) — reset the stall watch so it
// doesn't read as a DWM stall. // doesn't read as a DWM stall.
@@ -1853,10 +2238,12 @@ impl IddPushCapturer {
} }
} }
} }
self.last_fresh = now; // feeds the driver-death watch if !regen {
// Build the frame. For PyroWave the encode input is the Y plane self.last_fresh = now; // feeds the driver-death watch
// (`texture`) + the CbCr plane & fence in `pyro`; signal the shared fence }
// after the convert above. SAFETY: on the owning capture/encode thread. // Build the frame. For PyroWave the encode input is the Y plane
// (`texture`) + the CbCr plane & fence in `pyro`; signal the shared fence
// after the convert above. SAFETY: on the owning capture/encode thread.
let (texture, pyro) = if let Some((y, _, cbcr, _)) = pyro_slot { let (texture, pyro) = if let Some((y, _, cbcr, _)) = pyro_slot {
// SAFETY: on the owning capture/encode thread holding the immediate context. // SAFETY: on the owning capture/encode thread holding the immediate context.
let (fence_handle, fence_value) = let (fence_handle, fence_value) =
@@ -1981,7 +2368,82 @@ impl std::fmt::Display for AttachTexFail {
impl std::error::Error for AttachTexFail {} impl std::error::Error for AttachTexFail {}
/// Duplicate `cs`'s section into the driver's WUDFHost and send `IOCTL_SET_CURSOR_CHANNEL`.
/// `true` = the driver adopted it (worker declared per its `cursor_forward_on` state). Shared by
/// the open-time delivery and every RE-delivery (ring recreate / flip NOT_FOUND) — the request is
/// idempotent driver-side (a replaced worker is stopped + joined).
fn deliver_cursor_channel(
broker: &ChannelBroker,
target_id: u32,
cs: &cursor::CursorShared,
send_cursor: &crate::CursorChannelSender,
) -> bool {
// SAFETY: `cs.section_handle()` borrows the section mapping `cs` owns (live across this
// synchronous call); the broker's WUDFHost process handle is live for the broker's lifetime.
let value = match unsafe { broker.dup_into_public(cs.section_handle()) } {
Ok(v) => v,
Err(e) => {
tracing::warn!("cursor section duplication failed (composited cursor stays): {e:#}");
return false;
}
};
let req = pf_driver_proto::control::SetCursorChannelRequest {
target_id,
_pad: 0,
header_handle: value,
};
match send_cursor(&req) {
Ok(()) => {
tracing::info!(
target_id,
"IDD push(host): cursor channel delivered — driver declares the hardware cursor"
);
true
}
Err(e) => {
broker.close_remote_public(value);
tracing::warn!("cursor channel delivery failed (composited cursor stays): {e:#}");
false
}
}
}
impl Capturer for IddPushCapturer { impl Capturer for IddPushCapturer {
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
// A LIVE poller is the sole source — even while it still reports `None` (pre-first-shape):
// falling back to the shm mid-session would interleave two serial namespaces and poison
// the client's shape cache. The shm read only serves a poller that failed to start/died.
if let Some(p) = &self.cursor_poll {
if p.alive() {
return p.read();
}
}
self.cursor_shared.as_mut().and_then(|c| c.read())
}
fn set_cursor_forward(&mut self, on: bool) {
// The composite (capture) model is implemented HOST-side: the driver's hardware cursor
// stays declared for the session's whole life — the only dependable state (there is NO
// working un-declare; see cursor_blend.rs) — keeping every frame pointer-free, and the
// capturer blends the GDI poller's shape into the frame itself. No driver round-trip.
// `composite_forced` (a channel-less session on a sticky-declared target) is pinned ON:
// with no client drawing, un-compositing would erase the pointer entirely.
let composite = (!on && self.cursor_shared.is_some()) || self.composite_forced;
if self.composite_cursor != composite {
self.composite_cursor = composite;
self.last_blend_key = None; // regenerate immediately at the current pointer state
tracing::info!(
composite,
"cursor render model: host compositing {}",
if composite {
"ON (capture model — blending the pointer into frames)"
} else {
"OFF (client draws locally)"
}
);
}
}
fn next_frame(&mut self) -> Result<CapturedFrame> { fn next_frame(&mut self) -> Result<CapturedFrame> {
let deadline = Instant::now() + Duration::from_secs(20); let deadline = Instant::now() + Duration::from_secs(20);
loop { loop {
@@ -2115,6 +2577,16 @@ fn warn_444_hdr_downgrade_once() {
impl Drop for IddPushCapturer { impl Drop for IddPushCapturer {
fn drop(&mut self) { fn drop(&mut self) {
// A channel session ending while the secure-desktop guard is engaged must not leave the
// driver's per-target desired state off — the next session's channel delivery would
// adopt UNdeclared and silently run the composite model (§8.6's cross-session trap).
// The open-time reset also covers this (host-crash case); this is the orderly-teardown
// belt.
if self.secure_active && self.cursor_shared.is_some() {
if let Some(fwd) = self.cursor_forward.as_ref() {
let _ = fwd(true);
}
}
self.slots.clear(); self.slots.clear();
// The shared header section (`MappedSection`), the frame-ready `event` (`OwnedHandle`) and the // The shared header section (`MappedSection`), the frame-ready `event` (`OwnedHandle`) and the
// broker's WUDFHost process handle free themselves via RAII (unmap view, then close handle) — // broker's WUDFHost process handle free themselves via RAII (unmap view, then close handle) —
@@ -105,6 +105,22 @@ impl ChannelBroker {
Ok(out.0 as usize as u64) Ok(out.0 as usize as u64)
} }
/// Duplicate the cursor section into WUDFHost (v5 cursor channel) with the same
/// least-privilege section rights as the frame header. Thin `pub(super)` face over
/// [`dup_into`](Self::dup_into) for the cursor-delivery path in `open_on`.
///
/// # Safety
/// `h` must be a live handle of the current process.
pub(super) unsafe fn dup_into_public(&self, h: HANDLE) -> Result<u64> {
// SAFETY: forwarded contract — `h` is live per this fn's own contract.
unsafe { self.dup_into(h, Some(SECTION_MAP_RW)) }
}
/// [`close_remote`](Self::close_remote) for the cursor-delivery failure path.
pub(super) fn close_remote_public(&self, value: u64) {
self.close_remote(value);
}
/// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE` /// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE`
/// with no target closes the source handle regardless of the (ignored) result. /// with no target closes the source handle regardless of the (ignored) result.
fn close_remote(&self, value: u64) { fn close_remote(&self, value: u64) {
@@ -0,0 +1,194 @@
//! Host side of the v5 hardware-cursor channel (remote-desktop-sweep M2c): the capturer creates
//! an unnamed [`CursorShm`] section, delivers it to the pf-vdisplay driver (which declares an
//! IddCx hardware cursor — DWM then EXCLUDES the pointer from the frames we consume), and reads
//! the driver's seqlock publishes here at encode-tick pace, converting them into the same
//! [`pf_frame::CursorOverlay`] the Linux portal path produces — everything downstream (the
//! cursor forwarder, the wire, the client renderer) is shared.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use pf_driver_proto::cursor::{
CursorShm, CURSOR_MAGIC, CURSOR_SHAPE_BYTES, CURSOR_SHAPE_MAX, CURSOR_SHAPE_OFFSET,
CURSOR_SHM_SIZE, CURSOR_TYPE_MASKED_COLOR,
};
use std::sync::atomic::AtomicU32;
/// The host end of one monitor's cursor channel: the section (we created it — the mapping stays
/// valid for the capturer's life) plus the reader's conversion cache.
pub(super) struct CursorShared {
section: MappedSection,
/// The monitor's desktop origin — IddCx reports positions in DESKTOP coordinates; the
/// overlay wants frame-relative. Fetched at attach (the virtual monitor's placement is
/// stable for the session; a topology change recreates the pipeline anyway).
origin: (i32, i32),
/// Conversion cache: the last `shape_id` whose pixels were converted, and the result.
/// Position-only updates (the common case) reuse it — a refcount bump, no pixel work.
cached_id: u32,
cached: Option<ConvertedShape>,
}
struct ConvertedShape {
rgba: std::sync::Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
}
impl CursorShared {
/// Create + initialize the section (magic stamped, seq even/zero). The returned handle is
/// the section itself (owned by `self`); the caller duplicates it into the WUDFHost.
pub(super) fn create(target_id: u32) -> Result<CursorShared> {
// SAFETY: plain FFI. Unnamed pagefile-backed section, host-lifetime owned; the view is
// mapped once and unmapped never (the capturer's life = the session's life).
let section = unsafe {
let map = CreateFileMappingW(
INVALID_HANDLE_VALUE,
None,
PAGE_READWRITE,
0,
CURSOR_SHM_SIZE as u32,
PCWSTR::null(),
)
.context("CreateFileMapping(cursor)")?;
let map = OwnedHandle::from_raw_handle(map.0 as _);
let view = MapViewOfFile(
HANDLE(map.as_raw_handle()),
FILE_MAP_ALL_ACCESS,
0,
0,
CURSOR_SHM_SIZE,
);
if view.Value.is_null() {
bail!("MapViewOfFile failed for the cursor section");
}
let shm = view.Value.cast::<CursorShm>();
std::ptr::write_bytes(view.Value.cast::<u8>(), 0, CURSOR_SHM_SIZE);
// Magic LAST-ish (the driver validates it at adopt; seq 0 = even = consistent).
std::sync::atomic::fence(Ordering::Release);
(*shm).magic = CURSOR_MAGIC;
MappedSection { handle: map, view }
};
// Desktop origin of this monitor's source — for the desktop→frame coordinate shift.
// SAFETY: `source_desktop_rect` only runs the CCD QueryDisplayConfig FFI over owned
// locals (same call the compose-kick path makes).
let rect = unsafe { pf_win_display::win_display::source_desktop_rect(target_id) };
let origin = rect.map(|(x, y, _w, _h)| (x, y)).unwrap_or((0, 0));
Ok(CursorShared {
section,
origin,
cached_id: 0,
cached: None,
})
}
/// The section handle for the broker's duplication into the WUDFHost.
pub(super) fn section_handle(&self) -> HANDLE {
HANDLE(self.section.handle.as_raw_handle())
}
/// Seqlock-read the driver's latest publish → a frame-relative [`pf_frame::CursorOverlay`].
/// `None` until the first publish lands (or while the pointer has never been seen). A hidden
/// pointer returns `Some` with `visible: false` — the forwarder turns that into the client's
/// relative-mode hint, exactly like the Linux path.
pub(super) fn read(&mut self) -> Option<pf_frame::CursorOverlay> {
let shm = self.section.ptr::<CursorShm>();
// SAFETY: the view spans CURSOR_SHM_SIZE for self's lifetime; seq is 4-aligned in the
// fixed layout (offset 4).
let seq = unsafe { &*std::ptr::addr_of!((*shm).seq).cast::<AtomicU32>() };
for _ in 0..64 {
let s1 = seq.load(Ordering::Acquire);
if s1 == 0 {
return None; // no publish yet
}
if s1 & 1 != 0 {
std::hint::spin_loop();
continue; // writer mid-update
}
// SAFETY: header reads within the mapped view; consistency is validated by the
// seq re-check below (a torn read is discarded and retried).
let hdr = unsafe { std::ptr::read_volatile(shm) };
// Shape pixels: convert only when the OS minted a new shape id.
if hdr.visible != 0 && hdr.shape_id != self.cached_id {
let rows = hdr.height.min(CURSOR_SHAPE_MAX) as usize;
let width = hdr.width.min(CURSOR_SHAPE_MAX) as usize;
let pitch = (hdr.pitch as usize).min(CURSOR_SHAPE_BYTES / rows.max(1));
let mut raw = vec![0u8; rows * pitch];
// SAFETY: the shape region spans CURSOR_SHAPE_BYTES from CURSOR_SHAPE_OFFSET
// inside the mapped view; `rows * pitch` is clamped to it above.
unsafe {
std::ptr::copy_nonoverlapping(
self.section.ptr::<u8>().add(CURSOR_SHAPE_OFFSET),
raw.as_mut_ptr(),
rows * pitch,
);
}
// Discard the copy if the writer raced us mid-shape (seq moved) — retry.
if seq.load(Ordering::Acquire) != s1 {
continue;
}
self.cached = Some(convert_shape(&hdr, &raw, width, rows, pitch));
self.cached_id = hdr.shape_id;
} else if seq.load(Ordering::Acquire) != s1 {
continue;
}
let shape = self.cached.as_ref()?;
return Some(pf_frame::CursorOverlay {
x: hdr.x - self.origin.0,
y: hdr.y - self.origin.1,
w: shape.w,
h: shape.h,
rgba: shape.rgba.clone(),
serial: u64::from(hdr.shape_id),
hot_x: shape.hot_x,
hot_y: shape.hot_y,
visible: hdr.visible != 0,
});
}
None // persistent tearing (writer wedged mid-seq) — skip this tick
}
}
/// Convert the OS's 32-bpp pitch-strided shape rows into the overlay's packed straight RGBA.
/// ALPHA cursors are BGRA with straight per-pixel alpha (swap R↔B). MASKED_COLOR approximates:
/// alpha 0x00 = opaque color pixel; 0xFF = an XOR pixel we cannot honor client-side — rendered
/// as a translucent mid-gray so inversion cursors stay visible instead of vanishing.
fn convert_shape(
hdr: &CursorShm,
raw: &[u8],
width: usize,
rows: usize,
pitch: usize,
) -> ConvertedShape {
let masked = hdr.cursor_type == CURSOR_TYPE_MASKED_COLOR;
let mut rgba = Vec::with_capacity(width * rows * 4);
for y in 0..rows {
let row = &raw[y * pitch..];
for x in 0..width {
let o = x * 4;
if o + 4 > row.len() {
rgba.extend_from_slice(&[0, 0, 0, 0]);
continue;
}
let (b, g, r, a) = (row[o], row[o + 1], row[o + 2], row[o + 3]);
if masked {
if a == 0 {
rgba.extend_from_slice(&[r, g, b, 0xFF]);
} else {
rgba.extend_from_slice(&[0x80, 0x80, 0x80, 0xB4]);
}
} else {
rgba.extend_from_slice(&[r, g, b, a]);
}
}
}
ConvertedShape {
rgba: std::sync::Arc::new(rgba),
w: width as u32,
h: rows as u32,
hot_x: hdr.hot_x.min(width.saturating_sub(1) as u32),
hot_y: hdr.hot_y.min(rows.saturating_sub(1) as u32),
}
}
@@ -0,0 +1,216 @@
//! Host-side cursor compositing for the CAPTURE mouse model (design/remote-desktop-sweep.md §8).
//!
//! Why the host draws it: once a monitor has ever declared an IddCx hardware cursor, DWM will
//! not composite the software cursor back into its frames — there is no un-declare DDI (the
//! empty-caps re-setup is rejected `STATUS_INVALID_PARAMETER`), and a successful same-mode
//! re-commit with the driver's re-declare provably suppressed still leaves the pointer excluded
//! (all observed on-glass, 26100). So the driver keeps its hardware cursor declared for the
//! session's whole life — the state that works — and when the client flips to the capture model
//! the HOST composites the pointer into the frame itself: a slot→scratch copy plus one
//! alpha-blended quad (the GDI poller's full-fidelity shape at its polled position), entirely
//! GPU-side on the capture device, before the normal conversion runs from the scratch.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use windows::core::s;
use windows::Win32::Graphics::Direct3D::D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST;
use windows::Win32::Graphics::Direct3D11::{
ID3D11BlendState, ID3D11Buffer, ID3D11PixelShader, ID3D11SamplerState, ID3D11VertexShader,
D3D11_BIND_CONSTANT_BUFFER, D3D11_BLEND_DESC, D3D11_BLEND_INV_SRC_ALPHA, D3D11_BLEND_ONE,
D3D11_BLEND_OP_ADD, D3D11_BLEND_SRC_ALPHA, D3D11_BUFFER_DESC, D3D11_COMPARISON_NEVER,
D3D11_CPU_ACCESS_WRITE, D3D11_FILTER_MIN_MAG_MIP_LINEAR, D3D11_MAPPED_SUBRESOURCE,
D3D11_MAP_WRITE_DISCARD, D3D11_RENDER_TARGET_BLEND_DESC, D3D11_SAMPLER_DESC,
D3D11_SUBRESOURCE_DATA, D3D11_TEXTURE_ADDRESS_CLAMP, D3D11_USAGE_DYNAMIC, D3D11_VIEWPORT,
};
use windows::Win32::Graphics::Dxgi::Common::DXGI_FORMAT_R8G8B8A8_UNORM;
/// Straight-alpha sample of the cursor bitmap. `linear_scale` = 0 passes sRGB through (SDR
/// ring); non-zero linearizes sRGB→scRGB AND multiplies by the target's SDR-white scale
/// (`sdr_white_level_scale` — 1.0 would put cursor-white at 80 nits, visibly DARKER than the
/// surrounding SDR desktop content DWM composes at the user's SDR-brightness setting).
const CURSOR_PS: &str = r"
Texture2D<float4> tx : register(t0);
SamplerState sm : register(s0);
cbuffer C : register(b0) { float linear_scale; float3 pad; };
float4 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_Target {
float4 c = tx.Sample(sm, uv);
if (linear_scale != 0.0) {
c.rgb = pow(abs(c.rgb), 2.2) * linear_scale;
}
return c;
}
";
/// The cursor-quad blend pass + its shape-texture cache. One per capturer (device-scoped).
pub(super) struct CursorBlendPass {
vs: ID3D11VertexShader,
ps: ID3D11PixelShader,
sampler: ID3D11SamplerState,
blend: ID3D11BlendState,
cbuf: ID3D11Buffer,
cbuf_scale: Option<f32>,
/// The uploaded shape (serial-keyed): SRV + dims in host pixels.
shape: Option<(u64, ID3D11ShaderResourceView, u32, u32)>,
}
impl CursorBlendPass {
pub(super) unsafe fn new(device: &ID3D11Device) -> Result<Self> {
let vsb = crate::dxgi::compile_shader(crate::dxgi::HDR_VS, s!("main"), s!("vs_5_0"))?;
let psb = crate::dxgi::compile_shader(CURSOR_PS, s!("main"), s!("ps_5_0"))?;
let mut vs = None;
device.CreateVertexShader(&vsb, None, Some(&mut vs))?;
let mut ps = None;
device.CreatePixelShader(&psb, None, Some(&mut ps))?;
let sd = D3D11_SAMPLER_DESC {
// LINEAR: the quad is drawn 1:1 in frame pixels, so this only matters at the
// half-texel edges; linear keeps them soft instead of ringing.
Filter: D3D11_FILTER_MIN_MAG_MIP_LINEAR,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
ComparisonFunc: D3D11_COMPARISON_NEVER,
MaxLOD: f32::MAX,
..Default::default()
};
let mut sampler = None;
device.CreateSamplerState(&sd, Some(&mut sampler))?;
// Straight-alpha over: dst.rgb = src.rgb*a + dst.rgb*(1-a); keep dst alpha.
let mut bd = D3D11_BLEND_DESC::default();
bd.RenderTarget[0] = D3D11_RENDER_TARGET_BLEND_DESC {
BlendEnable: true.into(),
SrcBlend: D3D11_BLEND_SRC_ALPHA,
DestBlend: D3D11_BLEND_INV_SRC_ALPHA,
BlendOp: D3D11_BLEND_OP_ADD,
SrcBlendAlpha: D3D11_BLEND_ONE,
DestBlendAlpha: D3D11_BLEND_ONE,
BlendOpAlpha: D3D11_BLEND_OP_ADD,
RenderTargetWriteMask: 0x0F,
};
let mut blend = None;
device.CreateBlendState(&bd, Some(&mut blend))?;
let cbd = D3D11_BUFFER_DESC {
ByteWidth: 16, // float to_linear + float3 pad
Usage: D3D11_USAGE_DYNAMIC,
BindFlags: D3D11_BIND_CONSTANT_BUFFER.0 as u32,
CPUAccessFlags: D3D11_CPU_ACCESS_WRITE.0 as u32,
..Default::default()
};
let mut cbuf = None;
device.CreateBuffer(&cbd, None, Some(&mut cbuf))?;
Ok(Self {
vs: vs.context("cursor blend vs")?,
ps: ps.context("cursor blend ps")?,
sampler: sampler.context("cursor blend sampler")?,
blend: blend.context("cursor blend state")?,
cbuf: cbuf.context("cursor blend cbuf")?,
cbuf_scale: None,
shape: None,
})
}
/// Upload `ov`'s bitmap if its serial is new; reuse the cached SRV otherwise.
unsafe fn ensure_shape(
&mut self,
device: &ID3D11Device,
ov: &pf_frame::CursorOverlay,
) -> Result<()> {
if self.shape.as_ref().is_some_and(|(s, ..)| *s == ov.serial) {
return Ok(());
}
if ov.rgba.len() < (ov.w as usize) * (ov.h as usize) * 4 || ov.w == 0 || ov.h == 0 {
bail!("malformed cursor overlay ({}x{})", ov.w, ov.h);
}
let desc = D3D11_TEXTURE2D_DESC {
Width: ov.w,
Height: ov.h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R8G8B8A8_UNORM,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: ov.rgba.as_ptr().cast(),
SysMemPitch: ov.w * 4,
SysMemSlicePitch: 0,
};
let mut tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, Some(&init), Some(&mut tex))
.context("CreateTexture2D(cursor shape)")?;
let tex = tex.context("null cursor shape texture")?;
let mut srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&tex, None, Some(&mut srv))
.context("CreateShaderResourceView(cursor shape)")?;
self.shape = Some((ov.serial, srv.context("null cursor shape srv")?, ov.w, ov.h));
Ok(())
}
/// Alpha-blend `ov` onto `dst` (frame-sized, RENDER_TARGET-capable — the blend scratch).
/// `linear_scale`: 0 = SDR passthrough; non-zero = the frame is FP16 scRGB (HDR
/// composition) — linearize and scale to the target's SDR white. The quad is placed purely
/// via the viewport (the fullscreen-triangle VS fills whatever viewport is set), clipped by
/// the target automatically.
pub(super) unsafe fn blend(
&mut self,
device: &ID3D11Device,
ctx: &ID3D11DeviceContext,
dst: &ID3D11Texture2D,
ov: &pf_frame::CursorOverlay,
linear_scale: f32,
) -> Result<()> {
self.ensure_shape(device, ov)?;
let (_, srv, w, h) = self.shape.as_ref().expect("shape just ensured");
if self.cbuf_scale != Some(linear_scale) {
let cb: [f32; 4] = [linear_scale, 0.0, 0.0, 0.0];
let mut mapped = D3D11_MAPPED_SUBRESOURCE::default();
if ctx
.Map(&self.cbuf, 0, D3D11_MAP_WRITE_DISCARD, 0, Some(&mut mapped))
.is_ok()
{
std::ptr::copy_nonoverlapping(cb.as_ptr(), mapped.pData as *mut f32, cb.len());
ctx.Unmap(&self.cbuf, 0);
}
self.cbuf_scale = Some(linear_scale);
}
let mut rtv: Option<ID3D11RenderTargetView> = None;
device
.CreateRenderTargetView(dst, None, Some(&mut rtv))
.context("CreateRenderTargetView(cursor blend scratch)")?;
let rtv = rtv.context("null cursor blend rtv")?;
ctx.OMSetRenderTargets(Some(&[Some(rtv)]), None);
ctx.OMSetBlendState(&self.blend, None, 0xffff_ffff);
ctx.VSSetShader(&self.vs, None);
ctx.PSSetShader(&self.ps, None);
ctx.PSSetShaderResources(0, Some(&[Some(srv.clone())]));
ctx.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
ctx.PSSetConstantBuffers(0, Some(&[Some(self.cbuf.clone())]));
ctx.IASetInputLayout(None);
ctx.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
// Placement IS the viewport: the VS fills it, the OS clips it to the target.
let vp = D3D11_VIEWPORT {
TopLeftX: ov.x as f32,
TopLeftY: ov.y as f32,
Width: *w as f32,
Height: *h as f32,
MinDepth: 0.0,
MaxDepth: 1.0,
};
ctx.RSSetViewports(Some(&[vp]));
ctx.Draw(3, 0);
// Unbind so the scratch can be bound as a conversion INPUT without a hazard warning.
ctx.OMSetRenderTargets(None, None);
let none_srv: [Option<ID3D11ShaderResourceView>; 1] = [None];
ctx.PSSetShaderResources(0, Some(&none_srv));
Ok(())
}
}
@@ -0,0 +1,508 @@
//! GDI cursor poller — the Windows cursor-SHAPE source for the cursor-forward channel
//! (design/remote-desktop-sweep.md §8, the M2c redesign).
//!
//! Why not the IddCx hardware-cursor query (the v5 `CursorShm` path, now the fallback): it is
//! alpha-only BY DESIGN — `IDDCX_CURSOR_SHAPE_TYPE` has no monochrome value, the OS pre-converts
//! monochrome to masked-color (IDDCX_CURSOR_CAPS docs), and masked-color delivery is dead code on
//! modern builds (proven on-glass at every `ColorXorCursorSupport` level; no public evidence of a
//! MASKED_COLOR delivery anywhere). The driver keeps its hardware cursor declared at XOR FULL
//! purely so DWM EXCLUDES every cursor type from the IDD frame.
//!
//! Why not DXGI Desktop Duplication `GetFramePointerShape`: its `PointerPosition.Visible` goes
//! stale when the cursor moves only via injected input on current Win11 (Sunshine #5293 — exactly
//! a Punktfunk session's topology), it burns one of the session's four duplication slots, and its
//! per-output metadata on IDD monitors has conflicting field reports. The GDI path below is the
//! metadata-forwarding-remote-desktop pattern (RustDesk, WebRTC/Chrome Remote Desktop, OBS): the
//! cursor is per-session global state in win32k, readable cross-process, and `CURSOR_SHOWING` is
//! the logical visibility — immune to all of the above.
//!
//! Works because the capture host runs as SYSTEM *inside the interactive session* on
//! `winsta0\default` (the service supervisor retargets the token — `windows/service.rs`
//! `spawn_host`), so the poller thread sees the session's cursor directly; no helper process.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use windows::Win32::Graphics::Gdi::{
DeleteObject, GetDC, GetDIBits, GetObjectW, ReleaseDC, BITMAP, BITMAPINFO, BITMAPINFOHEADER,
BI_RGB, DIB_RGB_COLORS, HBITMAP, HDC,
};
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, GetUserObjectInformationW, OpenInputDesktop, SetThreadDesktop,
DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS, HDESK, UOI_NAME,
};
use windows::Win32::UI::HiDpi::{
SetThreadDpiAwarenessContext, DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2,
};
use windows::Win32::UI::WindowsAndMessaging::{
CopyIcon, DestroyIcon, GetCursorInfo, GetIconInfo, CURSORINFO, HICON, ICONINFO,
};
/// `CURSORINFO.flags` bits (WindowsAndMessaging): the pointer is logically shown /
/// touch-or-pen-suppressed. Named locally so the visibility rule below reads as the docs do.
const CURSOR_SHOWING: u32 = 0x1;
const CURSOR_SUPPRESSED: u32 = 0x2;
/// A converted shape: the cache the per-tick overlay is assembled from. `rgba` is `Arc` so the
/// slot publish (and every downstream frame attach) is a refcount bump.
struct Shape {
rgba: std::sync::Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
serial: u64,
}
/// Off-thread GDI cursor poller. Samples `GetCursorInfo` at ~60 Hz, rasterises the `HCURSOR` only
/// when its handle value changes, and publishes a ready [`pf_frame::CursorOverlay`] snapshot; the
/// capture thread's per-tick cost is one uncontended mutex read + an `Arc` clone
/// (same split as [`DescriptorPoller`], and for the same reason: user32/gdi32 calls have no place
/// on the capture/encode thread).
pub(super) struct CursorPoller {
slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>>,
stop: Arc<AtomicBool>,
/// The input desktop is a SECURE desktop (Winlogon — UAC consent / lock / logon). Classified
/// on every reattach; the capturer polls it to stand the IddCx hardware-cursor declare down
/// while the secure desktop needs the software-cursor path to render (see
/// `IddPushCapturer::poll_secure_desktop`).
secure: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl CursorPoller {
/// ~250 Hz: the polled position is ALSO the composite-blend position (capture model), so
/// it must out-pace the fastest session — at 16 ms a 240 fps stream re-used a stale
/// position for ~4 consecutive frames and the composited pointer visibly stuttered
/// against the video. A tick is one `GetCursorInfo` syscall (rasterisation only on shape
/// change), so 250 Hz is still negligible CPU.
const INTERVAL: Duration = Duration::from_millis(4);
/// Unconditional input-desktop reattach cadence — catches secure-desktop (UAC/lock) switches
/// without a failure signal (`GetCursorInfo` on a stale desktop *succeeds* with stale data).
/// 250 ms, not the original 2 s: the reattach now also feeds [`Self::secure_desktop`], which
/// gates when the secure desktop becomes VISIBLE in the stream (the hardware-cursor
/// stand-down) — a 2 s freeze at every UAC prompt is user-visible, ~4 `OpenInputDesktop`
/// syscalls/s are not.
const REATTACH: Duration = Duration::from_millis(250);
/// Spawn the poller for the virtual display `target_id`. `rect` = the target's desktop rect
/// (`source_desktop_rect` order: x, y, w, h) — cursor positions are desktop-global; the
/// overlay wants frame-relative, and a pointer outside the rect reports `visible: false`
/// (per-output semantics, matching the driver shm path and the Linux portal).
pub(super) fn spawn(target_id: u32, rect: (i32, i32, i32, i32)) -> Self {
let slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>> = Arc::new(Mutex::new(None));
let stop = Arc::new(AtomicBool::new(false));
let secure = Arc::new(AtomicBool::new(false));
let (slot_t, stop_t, secure_t) = (slot.clone(), stop.clone(), secure.clone());
let thread = std::thread::Builder::new()
.name("pf-cursor-poll".into())
.spawn(move || run(target_id, rect, &slot_t, &stop_t, &secure_t))
.ok();
if thread.is_none() {
tracing::warn!("cursor poller thread spawn failed — cursor falls back to driver shm");
}
Self {
slot,
stop,
secure,
thread,
}
}
/// The latest overlay snapshot (`None` until the first successful shape rasterisation).
pub(super) fn read(&self) -> Option<pf_frame::CursorOverlay> {
self.slot.lock().unwrap_or_else(|p| p.into_inner()).clone()
}
/// Whether the input desktop is currently a SECURE desktop (UAC consent / Winlogon lock or
/// logon). Latched by the poll thread on its reattach cadence (≤ [`Self::REATTACH`] stale).
pub(super) fn secure_desktop(&self) -> bool {
self.secure.load(Ordering::Relaxed)
}
/// Whether the worker thread is (still) alive — `false` degrades the capturer to the shm read.
pub(super) fn alive(&self) -> bool {
self.thread.as_ref().is_some_and(|t| !t.is_finished())
}
}
impl Drop for CursorPoller {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(t) = self.thread.take() {
let _ = t.join(); // worker sleeps ≤ INTERVAL — a bounded join
}
}
}
/// The poll loop. Owns the thread's input-desktop binding and the shape cache.
fn run(
target_id: u32,
rect: (i32, i32, i32, i32),
slot: &Mutex<Option<pf_frame::CursorOverlay>>,
stop: &AtomicBool,
secure: &AtomicBool,
) {
// Physical-pixel coordinates on this thread regardless of the process's DPI awareness:
// `rect` comes from CCD (always physical), and a DPI-virtualized `GetCursorInfo` position
// would land in the wrong frame pixel on any scaled display. Thread-scoped, so the rest of
// the host is untouched.
// SAFETY: takes and returns only a by-value context handle; affects this thread only.
let _ = unsafe { SetThreadDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) };
let mut desktop = DesktopBinding::default();
// best-effort: already on winsta0\default if this fails
publish_secure(secure, desktop.reattach());
let mut last_attach = Instant::now();
let mut shape: Option<Shape> = None;
let mut cached_handle: isize = 0;
let mut failed_handle: isize = 0; // don't re-rasterise a failing handle every tick
let mut serial: u64 = 0;
let mut logged_live = false;
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(CursorPoller::INTERVAL);
if last_attach.elapsed() >= CursorPoller::REATTACH {
last_attach = Instant::now();
publish_secure(secure, desktop.reattach());
}
let mut ci = CURSORINFO {
cbSize: std::mem::size_of::<CURSORINFO>() as u32,
..Default::default()
};
// SAFETY: `ci` is a live, correctly-sized out-param for this synchronous call; no pointer
// escapes it.
if unsafe { GetCursorInfo(&mut ci) }.is_err() {
// Desktop went away under us (secure-desktop switch mid-call) — rebind and retry
// next tick; the slot keeps its last snapshot meanwhile.
publish_secure(secure, desktop.reattach());
last_attach = Instant::now();
continue;
}
let flags = ci.flags.0;
let showing = flags & CURSOR_SHOWING != 0 && flags & CURSOR_SUPPRESSED == 0;
// Rasterise on handle change only (position-only ticks are a header update). Hidden
// cursors keep the cached shape — the forwarder's hidden-but-known contract needs the
// bitmap to have been seen. v1: animated cursors publish their first frame (the OBS
// behavior); frame cycling via DrawIconEx istep is a known follow-up.
let handle = ci.hCursor.0 as isize;
if showing && handle != 0 && handle != cached_handle && handle != failed_handle {
match rasterize(ci.hCursor) {
Some((rgba, w, h, hot_x, hot_y)) => {
serial += 1;
shape = Some(Shape {
rgba: std::sync::Arc::new(rgba),
w,
h,
hot_x,
hot_y,
serial,
});
cached_handle = handle;
failed_handle = 0;
if !logged_live {
logged_live = true;
tracing::info!(
target_id,
"cursor poller live — GDI shape source publishing (serial 1: {w}x{h})"
);
}
}
None => {
// The owning app may have destroyed the cursor mid-read; keep the previous
// shape and don't hammer this handle again until it changes.
failed_handle = handle;
}
}
}
let overlay = shape.as_ref().map(|s| {
let (px, py) = (ci.ptScreenPos.x - rect.0, ci.ptScreenPos.y - rect.1);
let in_rect = px >= 0 && py >= 0 && px < rect.2 && py < rect.3;
pf_frame::CursorOverlay {
// Overlay x/y = bitmap top-left (reported position hotspot), frame pixels.
x: px - s.hot_x as i32,
y: py - s.hot_y as i32,
w: s.w,
h: s.h,
rgba: s.rgba.clone(),
serial: s.serial,
hot_x: s.hot_x,
hot_y: s.hot_y,
visible: showing && in_rect,
}
});
*slot.lock().unwrap_or_else(|p| p.into_inner()) = overlay;
}
}
/// Store a reattach's secure-desktop verdict (`None` = classification unavailable — keep the
/// previous state rather than flapping the capturer's hardware-cursor stand-down).
fn publish_secure(secure: &AtomicBool, verdict: Option<bool>) {
if let Some(s) = verdict {
secure.store(s, Ordering::Relaxed);
}
}
/// The thread's owned input-desktop handle — the [`SendInputInjector`] reattach model
/// (`pf-inject` sendinput.rs): keep the current binding, swap on demand, close exactly once.
#[derive(Default)]
struct DesktopBinding(Option<HDESK>);
impl DesktopBinding {
/// Rebind to the CURRENT input desktop. Returns whether that desktop is a SECURE one
/// (`UOI_NAME` != "Default": "Winlogon" during UAC consent / lock / logon) — `None` when the
/// input desktop could not be opened, in which case the binding (and the caller's secure
/// state) stays put.
fn reattach(&mut self) -> Option<bool> {
const GENERIC_ALL: u32 = 0x1000_0000;
// SAFETY: `OpenInputDesktop`/`SetThreadDesktop`/`CloseDesktop` take only by-value args.
// `OpenInputDesktop` yields an owned `HDESK` only on `Ok`; it is either installed (and the
// previously-owned handle closed exactly once) or closed on failure — no handle is leaked
// or used after close. `SetThreadDesktop` rebinds only this calling thread (which owns
// no windows/hooks, so the rebind cannot fail on that account).
unsafe {
match OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
) {
Ok(h) => {
let secure = desktop_is_secure(h);
if SetThreadDesktop(h).is_ok() {
if let Some(old) = self.0.replace(h) {
let _ = CloseDesktop(old);
}
} else {
let _ = CloseDesktop(h);
}
Some(secure)
}
Err(_) => None, // not privileged for this desktop; stay put
}
}
}
}
/// `UOI_NAME` of `h` != "Default" — i.e. the input desktop is Winlogon (UAC consent / lock /
/// logon) or a screen-saver desktop, both of which need the OS's software-cursor render path.
/// Unnameable desktops read as NOT secure: the only in-contract failure is a too-small buffer,
/// and misreading secure-as-normal merely keeps today's behavior for a beat.
fn desktop_is_secure(h: HDESK) -> bool {
let mut name = [0u16; 64]; // "Default"/"Winlogon"/"Screen-saver" all fit with room to spare
let mut needed = 0u32;
// SAFETY: `h` is the live desktop handle the caller just opened; `name`/`needed` are live
// out-params sized exactly as passed; the call writes at most `nlength` bytes.
let ok = unsafe {
GetUserObjectInformationW(
windows::Win32::Foundation::HANDLE(h.0),
UOI_NAME,
Some(name.as_mut_ptr().cast()),
(name.len() * 2) as u32,
Some(&mut needed),
)
};
if ok.is_err() {
return false;
}
let len = name.iter().position(|&c| c == 0).unwrap_or(name.len());
let name = String::from_utf16_lossy(&name[..len]);
!name.eq_ignore_ascii_case("Default")
}
impl Drop for DesktopBinding {
fn drop(&mut self) {
if let Some(h) = self.0.take() {
// SAFETY: `h` is our owned desktop handle, closed exactly once here.
let _ = unsafe { CloseDesktop(h) };
}
}
}
/// Rasterise `hcursor` to straight-alpha RGBA: `(rgba, w, h, hot_x, hot_y)`. `None` on any
/// failure (caller keeps the previous shape).
fn rasterize(hcursor: windows::Win32::UI::WindowsAndMessaging::HCURSOR) -> RasterOut {
// CopyIcon first: the owning process can destroy its HCURSOR between GetCursorInfo and the
// reads below; the copy is ours (the OBS/WebRTC guard).
// SAFETY: `HICON(hcursor.0)` reinterprets the cursor handle as an icon handle (cursors ARE
// icons in user32); CopyIcon yields an owned HICON we destroy below.
let Ok(icon) = (unsafe { CopyIcon(HICON(hcursor.0)) }) else {
return None;
};
let mut ii = ICONINFO::default();
// SAFETY: `ii` is a live out-param. On Ok it hands us COPIES of the mask/color bitmaps —
// both deleted below (GDI-handle leak otherwise).
let got = unsafe { GetIconInfo(icon, &mut ii) };
let out = if got.is_ok() { convert(&ii) } else { None };
// SAFETY: deleting the two bitmap copies GetIconInfo returned (null-safe: DeleteObject on a
// null HGDIOBJ fails harmlessly) and the icon copy — each exactly once.
unsafe {
let _ = DeleteObject(ii.hbmColor.into());
let _ = DeleteObject(ii.hbmMask.into());
let _ = DestroyIcon(icon);
}
out.map(|(rgba, w, h)| {
let hot_x = ii.xHotspot.min(w.saturating_sub(1));
let hot_y = ii.yHotspot.min(h.saturating_sub(1));
(rgba, w, h, hot_x, hot_y)
})
}
type RasterOut = Option<(Vec<u8>, u32, u32, u32, u32)>;
/// Convert the ICONINFO bitmaps to straight RGBA. Two families:
/// - color (`hbmColor` set): 32bpp BGRA; if the alpha channel is entirely empty (old-style
/// cursors) the AND mask supplies it (mask bit 1 = transparent).
/// - monochrome (`hbmColor` null): `hbmMask` is DOUBLE height — AND plane over XOR plane, the
/// WebRTC truth table: (0,0) black, (0,1) white, (1,0) transparent, (1,1) invert. Invert
/// pixels — unrepresentable in straight alpha — become opaque black with a white outline
/// grown into adjacent transparency (the WebRTC approximation; keeps the I-beam legible on
/// any background, which the old translucent-gray stand-in did not).
fn convert(ii: &ICONINFO) -> Option<(Vec<u8>, u32, u32)> {
// SAFETY: GetDC(None) yields the screen DC, released below on every path; it is only used
// as the GetDIBits reference DC.
let dc = unsafe { GetDC(None) };
let result = (|| {
if !ii.hbmColor.is_invalid() {
let color = read_bitmap_32(dc, ii.hbmColor)?;
let (w, h) = (color.w as u32, color.h as u32);
let mut rgba = bgra_to_rgba(&color.bgra);
if rgba.chunks_exact(4).all(|p| p[3] == 0) {
// Alpha-less color cursor: transparency lives in the AND mask.
let mask = read_bitmap_32(dc, ii.hbmMask)?;
if mask.w != color.w || mask.h < color.h {
return None;
}
for (px, m) in rgba.chunks_exact_mut(4).zip(mask.bgra.chunks_exact(4)) {
px[3] = if m[0] != 0 { 0 } else { 0xFF }; // mask white (AND=1) = transparent
}
}
Some((rgba, w, h))
} else {
let mask = read_bitmap_32(dc, ii.hbmMask)?;
if mask.h < 2 || mask.h % 2 != 0 {
return None;
}
let (w, h) = (mask.w as usize, (mask.h / 2) as usize);
let row = w * 4;
let (and_plane, xor_plane) = mask.bgra.split_at(h * row);
let mut rgba = vec![0u8; w * h * 4];
let mut invert = vec![false; w * h];
for i in 0..w * h {
let (a, x) = (and_plane[i * 4] != 0, xor_plane[i * 4] != 0);
let px = &mut rgba[i * 4..i * 4 + 4];
match (a, x) {
(false, false) => px.copy_from_slice(&[0, 0, 0, 0xFF]),
(false, true) => px.copy_from_slice(&[0xFF, 0xFF, 0xFF, 0xFF]),
(true, false) => {} // transparent (already zeroed)
(true, true) => {
px.copy_from_slice(&[0, 0, 0, 0xFF]);
invert[i] = true;
}
}
}
// White outline around invert regions so the (now black) shape survives dark
// backgrounds: any transparent 8-neighbor of an invert pixel turns opaque white.
for y in 0..h as i32 {
for x in 0..w as i32 {
if !invert[(y * w as i32 + x) as usize] {
continue;
}
for (dx, dy) in NEIGHBORS {
let (nx, ny) = (x + dx, y + dy);
if nx < 0 || ny < 0 || nx >= w as i32 || ny >= h as i32 {
continue;
}
let o = (ny * w as i32 + nx) as usize * 4;
if rgba[o + 3] == 0 {
rgba[o..o + 4].copy_from_slice(&[0xFF, 0xFF, 0xFF, 0xFF]);
}
}
}
}
Some((rgba, w as u32, h as u32))
}
})();
// SAFETY: releasing the screen DC obtained above, exactly once.
unsafe {
ReleaseDC(None, dc);
}
result
}
const NEIGHBORS: [(i32, i32); 8] = [
(-1, -1),
(0, -1),
(1, -1),
(-1, 0),
(1, 0),
(-1, 1),
(0, 1),
(1, 1),
];
struct RawBitmap {
w: i32,
h: i32,
/// 32bpp top-down BGRA rows, `w*h*4` (monochrome sources arrive expanded: 0x00/0xFF channels).
bgra: Vec<u8>,
}
/// Read any GDI bitmap as 32bpp top-down via `GetDIBits` (which performs the 1bpp→32bpp
/// expansion for the mask planes).
fn read_bitmap_32(dc: HDC, hbm: HBITMAP) -> Option<RawBitmap> {
let mut bm = BITMAP::default();
// SAFETY: `bm` is a live out-param sized exactly as passed; GetObjectW only writes into it.
let n = unsafe {
GetObjectW(
hbm.into(),
std::mem::size_of::<BITMAP>() as i32,
Some((&mut bm as *mut BITMAP).cast()),
)
};
if n == 0 || bm.bmWidth <= 0 || bm.bmHeight <= 0 || bm.bmWidth > 512 || bm.bmHeight > 1024 {
return None; // 512/1024: sanity caps (256² is the wire max; XL accessibility ≤ that)
}
let (w, h) = (bm.bmWidth, bm.bmHeight);
let mut info = BITMAPINFO {
bmiHeader: BITMAPINFOHEADER {
biSize: std::mem::size_of::<BITMAPINFOHEADER>() as u32,
biWidth: w,
biHeight: -h, // top-down
biPlanes: 1,
biBitCount: 32,
biCompression: BI_RGB.0,
..Default::default()
},
..Default::default()
};
let mut buf = vec![0u8; (w as usize) * (h as usize) * 4];
// SAFETY: `buf` spans exactly `h` rows of `w` 32bpp pixels as described by `info`; both are
// live locals for this synchronous call, `hbm` is a live bitmap not selected into any DC
// (fresh GetIconInfo copies).
let rows = unsafe {
GetDIBits(
dc,
hbm,
0,
h as u32,
Some(buf.as_mut_ptr().cast()),
&mut info,
DIB_RGB_COLORS,
)
};
(rows != 0).then_some(RawBitmap { w, h, bgra: buf })
}
fn bgra_to_rgba(bgra: &[u8]) -> Vec<u8> {
let mut out = bgra.to_vec();
for px in out.chunks_exact_mut(4) {
px.swap(0, 2);
}
out
}
+10 -4
View File
@@ -876,10 +876,16 @@ impl Worker {
); );
return; return;
}; };
let pref = self let pref = match self.pad_info(id) {
.pad_info(id) // Steam Input's virtual pad standing in front of the Deck's built-in controls (the
.map(|p| p.pref) // only-pad-forwarded case, [`Self::forwarded_ids`]): declare the DECK kind, not the
.unwrap_or(GamepadPref::Xbox360); // wrapper's Xbox 360 identity. [`Self::auto_pref`] already resolves the SESSION
// default this way, but a current host honors the per-pad arrival over the session
// default — so without this the host builds an X-Box 360 pad on a real Deck.
Some(p) if p.steam_virtual && is_steam_deck() => GamepadPref::SteamDeck,
Some(p) => p.pref,
None => GamepadPref::Xbox360,
};
match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) { match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) {
Ok(pad) => { Ok(pad) => {
let mut slot = Slot::new(id, index, pref, pad); let mut slot = Slot::new(id, index, pref, pad);
+12
View File
@@ -44,6 +44,13 @@ pub struct SessionParams {
/// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The /// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The
/// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`. /// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`.
pub clipboard: bool, pub clipboard: bool,
/// Advertise `quic::CLIENT_CAP_CURSOR`: this embedder renders the host cursor locally
/// (the presenter's cursor channel, design/remote-desktop-sweep.md M2), so the host may
/// stop compositing the pointer into the video. Only set when the embedder actually
/// draws it (the SDL presenter in desktop mouse mode) — a session that advertises it
/// without rendering streams with NO visible cursor. The host answers `HOST_CAP_CURSOR`
/// when its capture can forward (Linux portal, not gamescope/Windows).
pub cursor_forward: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see /// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
/// `video::Decoder::new`). /// `video::Decoder::new`).
pub decoder: String, pub decoder: String,
@@ -255,6 +262,11 @@ fn pump(
// This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an // This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an
// A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600). // A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600).
punktfunk_core::client::display_hdr_env_override().or(params.display_hdr), punktfunk_core::client::display_hdr_env_override().or(params.display_hdr),
if params.cursor_forward {
punktfunk_core::quic::CLIENT_CAP_CURSOR
} else {
0
},
params.launch.clone(), params.launch.clone(),
params.pin, params.pin,
Some(params.identity), Some(params.identity),
+5
View File
@@ -325,6 +325,11 @@ pub fn connect_reject_message(reason: punktfunk_core::reject::RejectReason) -> S
"Client and host versions don't match — update both to the same release.".into() "Client and host versions don't match — update both to the same release.".into()
} }
R::Busy => "The host is busy with another session.".into(), R::Busy => "The host is busy with another session.".into(),
R::SetupFailed => {
"The host accepted the connection but couldn't start the stream — the host's log \
(web console Log) has the cause."
.into()
}
} }
} }
+213 -9
View File
@@ -66,7 +66,31 @@ pub const fn interface_guid_fields() -> (u32, u16, u16, [u8; 8]) {
/// too ([`MIN_DRIVER_PROTOCOL_VERSION`]) and simply falls back to the re-arrival resize against it; /// too ([`MIN_DRIVER_PROTOCOL_VERSION`]) and simply falls back to the re-arrival resize against it;
/// a v4 driver serving an older (v3-asserting) host fails that host's strict handshake — ship /// a v4 driver serving an older (v3-asserting) host fails that host's strict handshake — ship
/// driver+host together, as ever. /// driver+host together, as ever.
pub const PROTOCOL_VERSION: u32 = 4; /// v5: ADDITIVE — the IddCx HARDWARE CURSOR channel (remote-desktop-sweep M2c):
/// [`control::AddRequest::hw_cursor`] (the former tail `_reserved` — same size, same offsets)
/// asks the driver to declare a hardware cursor for this monitor (DWM then EXCLUDES the pointer
/// from the desktop frame and delivers shape/position out-of-band), and
/// [`control::IOCTL_SET_CURSOR_CHANNEL`] delivers the host-created [`cursor::CursorShm`] section
/// the driver's cursor thread seqlock-publishes into. Nothing existing changed; the host gates
/// the feature on the handshake-reported version (`>= 5`) and keeps composited-cursor behavior
/// against older drivers.
/// v6: ADDITIVE — [`control::IOCTL_SET_CURSOR_FORWARD`] (the mid-stream cursor-render flip,
/// remote-desktop-sweep §8): the client's mouse-model flip (un)declares the hardware cursor on a
/// LIVE monitor, so the capture mouse model gets DWM's composited pointer back (full fidelity)
/// and the desktop model gets exclusion + forwarding. Nothing existing changed; against a v5
/// driver the unknown IOCTL fails and the host logs + keeps the declared-at-ADD behavior.
/// v6 tail ext (no bump, the `AddRequest` luminance-tail discipline):
/// [`control::AddReply::cursor_excluded`] — the driver reports whether its ADAPTER already
/// carries a hardware-cursor declare from an earlier session. A declare is IRREVOCABLE
/// (remote-desktop-sweep §8.6, proven on-glass) and its exclusion reaches EVERY later monitor of
/// the adapter, not just the declaring target (on-glass 2026-07-23: a declare on one target left
/// a different client's fresh target cursor-less): DWM never composites the software cursor back
/// into any of the adapter's frames until the adapter resets. The host uses the flag to
/// self-composite the pointer (GDI poller + blend) in sessions that never negotiate the
/// cursor channel — without it those sessions are silently cursor-less. Both skews degrade
/// cleanly: an old driver writes only the 20-byte reply prefix (host reads `0` = unknown/clean),
/// an old host retrieves a 20-byte buffer (driver writes just the prefix).
pub const PROTOCOL_VERSION: u32 = 6;
/// The OLDEST driver protocol this host still drives (v4 is additive over v3 — see the v4 note on /// The OLDEST driver protocol this host still drives (v4 is additive over v3 — see the v4 note on
/// [`PROTOCOL_VERSION`]): a v3 driver lacks only `IOCTL_UPDATE_MODES`, which the host gates on the /// [`PROTOCOL_VERSION`]): a v3 driver lacks only `IOCTL_UPDATE_MODES`, which the host gates on the
@@ -110,6 +134,22 @@ pub mod control {
/// identity (saved per-monitor DPI) and the driver's swap-chain/stash machinery survive. A v3 /// identity (saved per-monitor DPI) and the driver's swap-chain/stash machinery survive. A v3
/// driver fails this unknown IOCTL → the host falls back to the re-arrival resize. /// driver fails this unknown IOCTL → the host falls back to the re-arrival resize.
pub const IOCTL_UPDATE_MODES: u32 = ctl_code(0x907); pub const IOCTL_UPDATE_MODES: u32 = ctl_code(0x907);
/// Deliver a monitor's hardware-cursor channel (v5): the handle VALUE of the unnamed
/// [`cursor::CursorShm`](crate::cursor) file mapping the host duplicated into the WUDFHost
/// (same delivery model as [`IOCTL_SET_FRAME_CHANNEL`], no event — the host polls the
/// seqlock at its encode-tick pace). Sent once after ADD, only for a monitor whose
/// [`AddRequest::hw_cursor`] was set. The driver maps it, calls
/// `IddCxMonitorSetupHardwareCursor`, and starts its cursor-query thread. Input
/// [`SetCursorChannelRequest`].
pub const IOCTL_SET_CURSOR_CHANNEL: u32 = ctl_code(0x908);
/// Flip a LIVE monitor's hardware-cursor declaration (v6, the mid-stream cursor-render
/// flip): `enable = 1` re-declares (`IddCxMonitorSetupHardwareCursor` — DWM excludes the
/// pointer, the query/shape machinery resumes), `enable = 0` un-declares (the driver stops
/// re-declaring on mode commits and asks the OS to revert to the software cursor — DWM
/// composites the pointer into the frame, the pre-channel behavior the capture mouse
/// model wants). Only meaningful for a monitor whose cursor channel was delivered. Input
/// [`SetCursorForwardRequest`].
pub const IOCTL_SET_CURSOR_FORWARD: u32 = ctl_code(0x909);
/// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns /// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns
/// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this /// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this
@@ -147,9 +187,13 @@ pub mod control {
/// The client display's min luminance in MILLI-nits (0.001 cd/m² — the CTA min-luminance /// The client display's min luminance in MILLI-nits (0.001 cd/m² — the CTA min-luminance
/// range lives well below 1 nit) → Desired Content Min Luminance. `0` = unknown. /// range lives well below 1 nit) → Desired Content Min Luminance. `0` = unknown.
pub min_luminance_millinits: u32, pub min_luminance_millinits: u32,
/// Pads the `u64`-aligned struct to a multiple of 8 (Pod forbids implicit tail padding); /// Non-zero = declare an IddCx HARDWARE CURSOR for this monitor (v5, remote-desktop-sweep
/// free expansion room for the next appended field. /// M2c): DWM stops compositing the pointer into the frame and the driver publishes
pub _reserved: u32, /// shape/position into the [`cursor::CursorShm`](crate::cursor) section delivered by
/// [`IOCTL_SET_CURSOR_CHANNEL`]. Byte-compatible with the old tail `_reserved` (offset 36):
/// an un-upgraded driver ignores it (cursor stays composited — the host already gates on
/// the handshake version, this is defense in depth), an un-upgraded host sends `0` (off).
pub hw_cursor: u32,
} }
/// [`AddRequest`]'s size before the client-HDR luminance tail — the prefix an un-upgraded /// [`AddRequest`]'s size before the client-HDR luminance tail — the prefix an un-upgraded
@@ -174,8 +218,22 @@ pub mod control {
/// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a /// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a
/// WUDFHost restart between sessions can never leave the host duplicating into a dead process. /// WUDFHost restart between sessions can never leave the host duplicating into a dead process.
pub wudf_pid: u32, pub wudf_pid: u32,
/// Non-zero = the ADAPTER already carries an IRREVOCABLE hardware-cursor declare from an
/// earlier session (remote-desktop-sweep §8.6; reach is adapter-wide, not per-target —
/// on-glass 2026-07-23): DWM excludes the pointer from every frame on every monitor until
/// the adapter resets, and a session without the cursor channel must
/// self-composite (GDI poller + blend) or stream a cursor-less desktop. Appended after
/// [`ADD_REPLY_LEGACY_SIZE`] under the same dual-size discipline as the `AddRequest`
/// luminance tail: an un-upgraded driver writes only the legacy prefix (the host's
/// zero-initialized buffer then reads `0` = unknown/clean), an un-upgraded host retrieves a
/// legacy-size buffer (the driver writes just the prefix).
pub cursor_excluded: u32,
} }
/// [`AddReply`]'s size before the `cursor_excluded` tail — the prefix an un-upgraded driver
/// writes and an un-upgraded host retrieves (see the field docs).
pub const ADD_REPLY_LEGACY_SIZE: usize = 20;
/// `IOCTL_REMOVE` input. /// `IOCTL_REMOVE` input.
#[repr(C)] #[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)] #[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
@@ -253,6 +311,29 @@ pub mod control {
/// at the compile-time maximum; `ring_len` says how many entries are live). /// at the compile-time maximum; `ring_len` says how many entries are live).
pub const RING_LEN_USIZE: usize = RING_LEN as usize; pub const RING_LEN_USIZE: usize = RING_LEN as usize;
/// `IOCTL_SET_CURSOR_CHANNEL` input (v5): the hardware-cursor section for one monitor.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct SetCursorChannelRequest {
/// The OS target id from [`AddReply`] — which monitor this channel belongs to.
pub target_id: u32,
pub _pad: u32,
/// The [`cursor::CursorShm`](crate::cursor) file-mapping handle VALUE, already duplicated
/// into the driver's WUDFHost process ([`AddReply::wudf_pid`]).
pub header_handle: u64,
}
/// `IOCTL_SET_CURSOR_FORWARD` input (v6): the mid-stream cursor-render flip for one monitor.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct SetCursorForwardRequest {
/// The OS target id from [`AddReply`] — which monitor to flip.
pub target_id: u32,
/// `1` = declare the hardware cursor (exclude + forward), `0` = un-declare (DWM
/// composites — the capture mouse model).
pub enable: u32,
}
// Layout is load-bearing across the process boundary — pin it. (bytemuck's Pod derive already // Layout is load-bearing across the process boundary — pin it. (bytemuck's Pod derive already
// rejects any internal padding; these assert the externally-visible sizes too.) The `offset_of!` // rejects any internal padding; these assert the externally-visible sizes too.) The `offset_of!`
// asserts additionally catch a SAME-SIZE field reorder, which the size+Pod checks alone miss. // asserts additionally catch a SAME-SIZE field reorder, which the size+Pod checks alone miss.
@@ -269,13 +350,18 @@ pub mod control {
assert!(offset_of!(AddRequest, max_luminance_nits) == ADD_REQUEST_LEGACY_SIZE); assert!(offset_of!(AddRequest, max_luminance_nits) == ADD_REQUEST_LEGACY_SIZE);
assert!(offset_of!(AddRequest, max_frame_avg_nits) == 28); assert!(offset_of!(AddRequest, max_frame_avg_nits) == 28);
assert!(offset_of!(AddRequest, min_luminance_millinits) == 32); assert!(offset_of!(AddRequest, min_luminance_millinits) == 32);
// v5: the former tail `_reserved` — same offset, same total size (rename-only).
assert!(offset_of!(AddRequest, hw_cursor) == 36);
assert!(size_of::<AddRequest>() == 40);
assert!(size_of::<AddReply>() == 20); assert!(size_of::<AddReply>() == 24);
assert!(offset_of!(AddReply, adapter_luid_low) == 0); assert!(offset_of!(AddReply, adapter_luid_low) == 0);
assert!(offset_of!(AddReply, adapter_luid_high) == 4); assert!(offset_of!(AddReply, adapter_luid_high) == 4);
assert!(offset_of!(AddReply, target_id) == 8); assert!(offset_of!(AddReply, target_id) == 8);
assert!(offset_of!(AddReply, resolved_monitor_id) == 12); assert!(offset_of!(AddReply, resolved_monitor_id) == 12);
assert!(offset_of!(AddReply, wudf_pid) == 16); assert!(offset_of!(AddReply, wudf_pid) == 16);
// The cursor-excluded tail starts exactly at the legacy boundary (prefix-compat).
assert!(offset_of!(AddReply, cursor_excluded) == ADD_REPLY_LEGACY_SIZE);
assert!(size_of::<SetFrameChannelRequest>() == 32 + 8 * RING_LEN_USIZE); assert!(size_of::<SetFrameChannelRequest>() == 32 + 8 * RING_LEN_USIZE);
assert!(offset_of!(SetFrameChannelRequest, target_id) == 0); assert!(offset_of!(SetFrameChannelRequest, target_id) == 0);
@@ -288,6 +374,13 @@ pub mod control {
assert!(size_of::<RemoveRequest>() == 8); assert!(size_of::<RemoveRequest>() == 8);
assert!(offset_of!(RemoveRequest, session_id) == 0); assert!(offset_of!(RemoveRequest, session_id) == 0);
assert!(size_of::<SetCursorChannelRequest>() == 16);
assert!(offset_of!(SetCursorChannelRequest, target_id) == 0);
assert!(offset_of!(SetCursorChannelRequest, header_handle) == 8);
assert!(size_of::<SetCursorForwardRequest>() == 8);
assert!(offset_of!(SetCursorForwardRequest, target_id) == 0);
assert!(offset_of!(SetCursorForwardRequest, enable) == 4);
assert!(size_of::<UpdateModesRequest>() == 24); assert!(size_of::<UpdateModesRequest>() == 24);
assert!(offset_of!(UpdateModesRequest, session_id) == 0); assert!(offset_of!(UpdateModesRequest, session_id) == 0);
assert!(offset_of!(UpdateModesRequest, width) == 8); assert!(offset_of!(UpdateModesRequest, width) == 8);
@@ -960,6 +1053,84 @@ pub mod mouse {
}; };
} }
/// The v5 hardware-cursor channel (remote-desktop-sweep M2c): one unnamed file mapping per
/// monitor, host-created, delivered by handle value ([`control::IOCTL_SET_CURSOR_CHANNEL`]).
/// The DRIVER's cursor thread (woken by its IddCx `hNewCursorDataAvailable` event) seqlock-writes
/// shape + position + visibility; the HOST reads at its encode-tick pace — no event crosses the
/// boundary. Writer: bump [`CursorShm::seq`] to ODD, write fields (+ shape bytes when the OS said
/// the shape changed), bump to EVEN. Reader: read seq (retry while odd), copy, re-read seq —
/// unchanged ⇒ consistent snapshot. Position-only updates never touch the shape bytes, so a
/// reader that skips unchanged `shape_id`s never copies torn pixels.
pub mod cursor {
use bytemuck::{Pod, Zeroable};
/// First field of [`CursorShm`] — `b"PFCU"` little-endian; anything else = not attached yet.
pub const CURSOR_MAGIC: u32 = u32::from_le_bytes(*b"PFCU");
/// Max cursor side (px) the driver declares to the OS (`IDDCX_CURSOR_CAPS::MaxX/MaxY`) and
/// the section's shape buffer is sized for. Windows XL accessibility cursors top out here;
/// the host's wire forwarder downscales to its own cap anyway.
pub const CURSOR_SHAPE_MAX: u32 = 256;
/// Shape-buffer bytes: 32-bpp at the declared max.
pub const CURSOR_SHAPE_BYTES: usize = (CURSOR_SHAPE_MAX * CURSOR_SHAPE_MAX * 4) as usize;
/// Byte offset of the shape pixels inside the section (one cache-line-ish header).
pub const CURSOR_SHAPE_OFFSET: usize = 64;
/// Total section size.
pub const CURSOR_SHM_SIZE: usize = CURSOR_SHAPE_OFFSET + CURSOR_SHAPE_BYTES;
/// `IDDCX_CURSOR_SHAPE_TYPE` values mirrored for the host (the driver writes the OS value
/// verbatim into [`CursorShm::cursor_type`]).
pub const CURSOR_TYPE_MASKED_COLOR: u32 = 1;
pub const CURSOR_TYPE_ALPHA: u32 = 2;
/// The section header (the shape pixels follow at [`CURSOR_SHAPE_OFFSET`]). `x`/`y` are the
/// shape's TOP-LEFT in desktop coordinates (the IddCx `IDARG_OUT_QUERY_HWCURSOR::X/Y`
/// convention — position hotspot, can be negative); `shape_id` is the OS's per-set counter
/// (bumps on every shape set, the overlay serial); pixels are the OS's 32-bpp rows at
/// `pitch` bytes (BGRA for ALPHA; color+mask for MASKED_COLOR — the host converts).
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct CursorShm {
pub magic: u32,
/// Seqlock: odd = writer mid-update.
pub seq: u32,
pub visible: u32,
pub cursor_type: u32,
pub x: i32,
pub y: i32,
pub shape_id: u32,
pub width: u32,
pub height: u32,
pub pitch: u32,
pub hot_x: u32,
pub hot_y: u32,
/// Reserved expansion room up to [`CURSOR_SHAPE_OFFSET`].
pub _reserved: [u32; 4],
}
// Layout is load-bearing across the process boundary — pin it.
const _: () = {
use core::mem::{offset_of, size_of};
assert!(size_of::<CursorShm>() == 64);
assert!(size_of::<CursorShm>() <= CURSOR_SHAPE_OFFSET);
assert!(offset_of!(CursorShm, magic) == 0);
assert!(offset_of!(CursorShm, seq) == 4);
assert!(offset_of!(CursorShm, visible) == 8);
assert!(offset_of!(CursorShm, cursor_type) == 12);
assert!(offset_of!(CursorShm, x) == 16);
assert!(offset_of!(CursorShm, y) == 20);
assert!(offset_of!(CursorShm, shape_id) == 24);
assert!(offset_of!(CursorShm, width) == 28);
assert!(offset_of!(CursorShm, height) == 32);
assert!(offset_of!(CursorShm, pitch) == 36);
assert!(offset_of!(CursorShm, hot_x) == 40);
assert!(offset_of!(CursorShm, hot_y) == 44);
};
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -1079,7 +1250,7 @@ mod tests {
max_luminance_nits: 800, max_luminance_nits: 800,
max_frame_avg_nits: 400, max_frame_avg_nits: 400,
min_luminance_millinits: 50, // 0.05 nits min_luminance_millinits: 50, // 0.05 nits
_reserved: 0, hw_cursor: 1,
}; };
let bytes = bytemuck::bytes_of(&req); let bytes = bytemuck::bytes_of(&req);
assert_eq!(bytes.len(), 40); assert_eq!(bytes.len(), 40);
@@ -1109,14 +1280,19 @@ mod tests {
target_id: 262, target_id: 262,
resolved_monitor_id: 7, resolved_monitor_id: 7,
wudf_pid: 4242, wudf_pid: 4242,
cursor_excluded: 1,
}; };
let rbytes = bytemuck::bytes_of(&reply); let rbytes = bytemuck::bytes_of(&reply);
assert_eq!(rbytes.len(), 20); assert_eq!(rbytes.len(), 24);
assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply); assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply);
// resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible. // resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible.
assert_eq!(rbytes[12..16], 7u32.to_le_bytes()); assert_eq!(rbytes[12..16], 7u32.to_le_bytes());
// The v2 duplication-target pid trails at offset 16. // The v2 duplication-target pid trails at offset 16.
assert_eq!(rbytes[16..20], 4242u32.to_le_bytes()); assert_eq!(rbytes[16..20], 4242u32.to_le_bytes());
// The cursor-excluded tail rides after the legacy boundary; an un-upgraded driver writes
// only the prefix, so a zero-filled tail reads as "unknown/clean" (see the field docs).
assert_eq!(rbytes[20..24], 1u32.to_le_bytes());
assert_eq!(control::ADD_REPLY_LEGACY_SIZE, 20);
} }
#[test] #[test]
@@ -1161,11 +1337,39 @@ mod tests {
req req
); );
assert_eq!(bytes[8..12], 2560u32.to_le_bytes()); assert_eq!(bytes[8..12], 2560u32.to_le_bytes());
// The compat window: v4 is additive over v3, so the host floor stays one below. // The compat window: v4v6 are additive over v3, so the host floor stays at 3.
assert_eq!(PROTOCOL_VERSION, 4); assert_eq!(PROTOCOL_VERSION, 6);
assert_eq!(MIN_DRIVER_PROTOCOL_VERSION, 3); assert_eq!(MIN_DRIVER_PROTOCOL_VERSION, 3);
} }
#[test]
fn cursor_shm_layout_is_pinned() {
use cursor::*;
// The header must leave the shape offset intact whatever grows inside `_reserved`.
assert_eq!(core::mem::size_of::<CursorShm>(), 64);
assert_eq!(CURSOR_SHM_SIZE, 64 + 256 * 256 * 4);
assert_eq!(CURSOR_MAGIC, u32::from_le_bytes(*b"PFCU"));
// Seqlock snapshot discipline survives a bytemuck roundtrip.
let hdr = CursorShm {
magic: CURSOR_MAGIC,
seq: 2,
visible: 1,
cursor_type: CURSOR_TYPE_ALPHA,
x: -3,
y: 7,
shape_id: 42,
width: 32,
height: 32,
pitch: 128,
hot_x: 4,
hot_y: 5,
_reserved: [0; 4],
};
let bytes = bytemuck::bytes_of(&hdr);
assert_eq!(*bytemuck::from_bytes::<CursorShm>(bytes), hdr);
assert_eq!(bytes[16..20], (-3i32).to_le_bytes());
}
#[test] #[test]
fn gamepad_names_and_magics_are_stable() { fn gamepad_names_and_magics_are_stable() {
assert_eq!(gamepad::xusb_boot_name(0), "Global\\pfxusb-boot-0"); assert_eq!(gamepad::xusb_boot_name(0), "Global\\pfxusb-boot-0");
@@ -1,105 +0,0 @@
// Cursor-overlay blend kernels for the CUDA/NVENC path (cursor-as-metadata). The cursor bitmap is
// straight-alpha RGBA, row-packed (stride = curW*4). Blended into the encoder-OWNED NVENC input
// surface — never the compositor's dmabuf. One thread per cursor pixel (ARGB / YUV444) or per 2x2
// chroma block (NV12). Coefficients are BT.709 limited, matching rgb2yuv.comp so the cursor colour
// matches the rest of the frame regardless of which zero-copy backend encodes it.
//
// Build (regenerate cursor_blend.ptx after editing):
// nvcc -ptx -arch=compute_75 cursor_blend.cu -o cursor_blend.ptx
// PTX is JIT'd by the driver forward to the actual GPU, so a compute_75 (Turing) baseline runs on
// every Turing-or-newer NVENC GPU. (CUDA 13's nvcc no longer targets pre-Turing archs.)
typedef unsigned char u8;
__device__ __forceinline__ u8 blend8(int dst, int src, int a) {
return (u8)((src * a + dst * (255 - a)) / 255);
}
// Packed 4-byte surface. NVENC's ARGB format stores bytes B,G,R,A in memory; the cursor is R,G,B,A.
extern "C" __global__ void blend_argb(
u8* surf, int pitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int cx = blockIdx.x * blockDim.x + threadIdx.x;
int cy = blockIdx.y * blockDim.y + threadIdx.y;
if (cx >= curW || cy >= curH) return;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) return;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) return;
u8* d = surf + (size_t)py * pitch + (size_t)px * 4;
d[0] = blend8(d[0], s[2], a); // B <- cursor B
d[1] = blend8(d[1], s[1], a); // G <- cursor G
d[2] = blend8(d[2], s[0], a); // R <- cursor R
}
// Planar YUV444: three full-res planes stacked at base, base+plane, base+2*plane (plane=pitch*surfH).
extern "C" __global__ void blend_yuv444(
u8* base, int pitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int cx = blockIdx.x * blockDim.x + threadIdx.x;
int cy = blockIdx.y * blockDim.y + threadIdx.y;
if (cx >= curW || cy >= curH) return;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) return;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) return;
float R = s[0], G = s[1], B = s[2];
int Y = (int)(16.0f + 0.1826f * R + 0.6142f * G + 0.0620f * B + 0.5f);
int U = (int)(128.0f - 0.1006f * R - 0.3386f * G + 0.4392f * B + 0.5f);
int V = (int)(128.0f + 0.4392f * R - 0.3989f * G - 0.0403f * B + 0.5f);
size_t plane = (size_t)pitch * surfH;
u8* yp = base + (size_t)py * pitch + px;
u8* up = base + plane + (size_t)py * pitch + px;
u8* vp = base + 2 * plane + (size_t)py * pitch + px;
*yp = blend8(*yp, Y, a);
*up = blend8(*up, U, a);
*vp = blend8(*vp, V, a);
}
// NV12: full-res Y plane + interleaved half-res UV plane. One thread per 2x2 luma block; each blends
// up to four Y samples and one (alpha-weighted) UV sample.
extern "C" __global__ void blend_nv12(
u8* yb, int yPitch, u8* uvb, int uvPitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int bx = blockIdx.x * blockDim.x + threadIdx.x;
int by = blockIdx.y * blockDim.y + threadIdx.y;
int base_cx = bx * 2, base_cy = by * 2;
if (base_cx >= curW || base_cy >= curH) return;
float ua = 0.0f, va = 0.0f, wa = 0.0f;
int cnt = 0;
for (int j = 0; j < 2; j++) {
for (int i = 0; i < 2; i++) {
int cx = base_cx + i, cy = base_cy + j;
if (cx >= curW || cy >= curH) continue;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) continue;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) continue;
float R = s[0], G = s[1], B = s[2];
int Y = (int)(16.0f + 0.1826f * R + 0.6142f * G + 0.0620f * B + 0.5f);
u8* yp = yb + (size_t)py * yPitch + px;
*yp = blend8(*yp, Y, a);
ua += (128.0f - 0.1006f * R - 0.3386f * G + 0.4392f * B) * a;
va += (128.0f + 0.4392f * R - 0.3989f * G - 0.0403f * B) * a;
wa += a;
cnt++;
}
}
if (wa <= 0.0f || cnt == 0) return;
// The chroma sample covering this block's top-left surface pixel.
int uvx = (ox + base_cx) / 2;
int uvy = (oy + base_cy) / 2;
if (uvx < 0 || uvy < 0 || uvx * 2 >= surfW || uvy * 2 >= surfH) return;
int U = (int)(ua / wa + 0.5f);
int V = (int)(va / wa + 0.5f);
int amean = (int)(wa / cnt + 0.5f);
u8* uv = uvb + (size_t)uvy * uvPitch + (size_t)uvx * 2;
uv[0] = blend8(uv[0], U, amean);
uv[1] = blend8(uv[1], V, amean);
}
@@ -1,576 +0,0 @@
//
// Generated by NVIDIA NVVM Compiler
//
// Compiler Build ID: CL-38244171
// Cuda compilation tools, release 13.3, V13.3.73
// Based on NVVM 7.0.1
//
.version 9.3
.target sm_75
.address_size 64
// .globl blend_argb
.visible .entry blend_argb(
.param .u64 blend_argb_param_0,
.param .u32 blend_argb_param_1,
.param .u32 blend_argb_param_2,
.param .u32 blend_argb_param_3,
.param .u64 blend_argb_param_4,
.param .u32 blend_argb_param_5,
.param .u32 blend_argb_param_6,
.param .u32 blend_argb_param_7,
.param .u32 blend_argb_param_8
)
{
.reg .pred %p<10>;
.reg .b16 %rs<2>;
.reg .b32 %r<34>;
.reg .b64 %rd<17>;
ld.param.u64 %rd2, [blend_argb_param_0];
ld.param.u32 %r5, [blend_argb_param_1];
ld.param.u32 %r6, [blend_argb_param_2];
ld.param.u32 %r7, [blend_argb_param_3];
ld.param.u64 %rd3, [blend_argb_param_4];
ld.param.u32 %r8, [blend_argb_param_5];
ld.param.u32 %r11, [blend_argb_param_6];
ld.param.u32 %r9, [blend_argb_param_7];
ld.param.u32 %r10, [blend_argb_param_8];
mov.u32 %r12, %ntid.x;
mov.u32 %r13, %ctaid.x;
mov.u32 %r14, %tid.x;
mad.lo.s32 %r1, %r13, %r12, %r14;
mov.u32 %r15, %ntid.y;
mov.u32 %r16, %ctaid.y;
mov.u32 %r17, %tid.y;
mad.lo.s32 %r2, %r16, %r15, %r17;
setp.ge.s32 %p1, %r1, %r8;
setp.ge.s32 %p2, %r2, %r11;
or.pred %p3, %p1, %p2;
@%p3 bra $L__BB0_4;
add.s32 %r3, %r1, %r9;
add.s32 %r4, %r2, %r10;
or.b32 %r18, %r4, %r3;
setp.lt.s32 %p4, %r18, 0;
setp.ge.s32 %p5, %r3, %r6;
or.pred %p6, %p5, %p4;
setp.ge.s32 %p7, %r4, %r7;
or.pred %p8, %p7, %p6;
@%p8 bra $L__BB0_4;
mad.lo.s32 %r19, %r2, %r8, %r1;
mul.wide.s32 %rd4, %r19, 4;
cvta.to.global.u64 %rd5, %rd3;
add.s64 %rd1, %rd5, %rd4;
ld.global.u8 %rs1, [%rd1+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB0_4;
cvt.u32.u16 %r20, %rs1;
mul.wide.s32 %rd6, %r4, %r5;
mul.wide.s32 %rd7, %r3, 4;
add.s64 %rd8, %rd6, %rd7;
cvta.to.global.u64 %rd9, %rd2;
add.s64 %rd10, %rd9, %rd8;
ld.global.u8 %r21, [%rd10];
ld.global.u8 %r22, [%rd1+2];
xor.b32 %r23, %r20, 255;
mul.lo.s32 %r24, %r23, %r21;
mad.lo.s32 %r25, %r22, %r20, %r24;
mul.wide.u32 %rd11, %r25, -2139062143;
shr.u64 %rd12, %rd11, 39;
st.global.u8 [%rd10], %rd12;
ld.global.u8 %r26, [%rd10+1];
ld.global.u8 %r27, [%rd1+1];
mul.lo.s32 %r28, %r23, %r26;
mad.lo.s32 %r29, %r27, %r20, %r28;
mul.wide.u32 %rd13, %r29, -2139062143;
shr.u64 %rd14, %rd13, 39;
st.global.u8 [%rd10+1], %rd14;
ld.global.u8 %r30, [%rd10+2];
ld.global.u8 %r31, [%rd1];
mul.lo.s32 %r32, %r23, %r30;
mad.lo.s32 %r33, %r31, %r20, %r32;
mul.wide.u32 %rd15, %r33, -2139062143;
shr.u64 %rd16, %rd15, 39;
st.global.u8 [%rd10+2], %rd16;
$L__BB0_4:
ret;
}
// .globl blend_yuv444
.visible .entry blend_yuv444(
.param .u64 blend_yuv444_param_0,
.param .u32 blend_yuv444_param_1,
.param .u32 blend_yuv444_param_2,
.param .u32 blend_yuv444_param_3,
.param .u64 blend_yuv444_param_4,
.param .u32 blend_yuv444_param_5,
.param .u32 blend_yuv444_param_6,
.param .u32 blend_yuv444_param_7,
.param .u32 blend_yuv444_param_8
)
{
.reg .pred %p<10>;
.reg .b16 %rs<5>;
.reg .f32 %f<16>;
.reg .b32 %r<49>;
.reg .b64 %rd<14>;
ld.param.u64 %rd2, [blend_yuv444_param_0];
ld.param.u32 %r5, [blend_yuv444_param_1];
ld.param.u32 %r6, [blend_yuv444_param_2];
ld.param.u32 %r7, [blend_yuv444_param_3];
ld.param.u64 %rd3, [blend_yuv444_param_4];
ld.param.u32 %r8, [blend_yuv444_param_5];
ld.param.u32 %r11, [blend_yuv444_param_6];
ld.param.u32 %r9, [blend_yuv444_param_7];
ld.param.u32 %r10, [blend_yuv444_param_8];
mov.u32 %r12, %ntid.x;
mov.u32 %r13, %ctaid.x;
mov.u32 %r14, %tid.x;
mad.lo.s32 %r1, %r13, %r12, %r14;
mov.u32 %r15, %ntid.y;
mov.u32 %r16, %ctaid.y;
mov.u32 %r17, %tid.y;
mad.lo.s32 %r2, %r16, %r15, %r17;
setp.ge.s32 %p1, %r1, %r8;
setp.ge.s32 %p2, %r2, %r11;
or.pred %p3, %p1, %p2;
@%p3 bra $L__BB1_4;
add.s32 %r3, %r1, %r9;
add.s32 %r4, %r2, %r10;
or.b32 %r18, %r4, %r3;
setp.lt.s32 %p4, %r18, 0;
setp.ge.s32 %p5, %r3, %r6;
or.pred %p6, %p5, %p4;
setp.ge.s32 %p7, %r4, %r7;
or.pred %p8, %p7, %p6;
@%p8 bra $L__BB1_4;
mad.lo.s32 %r19, %r2, %r8, %r1;
mul.wide.s32 %rd4, %r19, 4;
cvta.to.global.u64 %rd5, %rd3;
add.s64 %rd1, %rd5, %rd4;
ld.global.u8 %rs1, [%rd1+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB1_4;
cvt.u32.u16 %r20, %rs1;
ld.global.u8 %rs2, [%rd1];
cvt.rn.f32.u16 %f1, %rs2;
ld.global.u8 %rs3, [%rd1+1];
cvt.rn.f32.u16 %f2, %rs3;
ld.global.u8 %rs4, [%rd1+2];
cvt.rn.f32.u16 %f3, %rs4;
fma.rn.f32 %f4, %f1, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f5, %f2, 0f3F1D3C36, %f4;
fma.rn.f32 %f6, %f3, 0f3D7DF3B6, %f5;
add.f32 %f7, %f6, 0f3F000000;
cvt.rzi.s32.f32 %r21, %f7;
fma.rn.f32 %f8, %f1, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f9, %f2, 0fBEAD5CFB, %f8;
fma.rn.f32 %f10, %f3, 0f3EE0DED3, %f9;
add.f32 %f11, %f10, 0f3F000000;
cvt.rzi.s32.f32 %r22, %f11;
fma.rn.f32 %f12, %f1, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f13, %f2, 0fBECC3C9F, %f12;
fma.rn.f32 %f14, %f3, 0fBD25119D, %f13;
add.f32 %f15, %f14, 0f3F000000;
cvt.rzi.s32.f32 %r23, %f15;
mul.wide.s32 %rd6, %r4, %r5;
cvt.s64.s32 %rd7, %r3;
add.s64 %rd8, %rd6, %rd7;
cvta.to.global.u64 %rd9, %rd2;
add.s64 %rd10, %rd9, %rd8;
ld.global.u8 %r24, [%rd10];
mul.lo.s32 %r25, %r21, %r20;
xor.b32 %r26, %r20, 255;
mad.lo.s32 %r27, %r26, %r24, %r25;
mul.hi.s32 %r28, %r27, -2139062143;
add.s32 %r29, %r28, %r27;
shr.u32 %r30, %r29, 31;
shr.u32 %r31, %r29, 7;
add.s32 %r32, %r31, %r30;
st.global.u8 [%rd10], %r32;
mul.wide.s32 %rd11, %r7, %r5;
add.s64 %rd12, %rd10, %rd11;
ld.global.u8 %r33, [%rd12];
mul.lo.s32 %r34, %r22, %r20;
mad.lo.s32 %r35, %r26, %r33, %r34;
mul.hi.s32 %r36, %r35, -2139062143;
add.s32 %r37, %r36, %r35;
shr.u32 %r38, %r37, 31;
shr.u32 %r39, %r37, 7;
add.s32 %r40, %r39, %r38;
st.global.u8 [%rd12], %r40;
add.s64 %rd13, %rd12, %rd11;
ld.global.u8 %r41, [%rd13];
mul.lo.s32 %r42, %r23, %r20;
mad.lo.s32 %r43, %r26, %r41, %r42;
mul.hi.s32 %r44, %r43, -2139062143;
add.s32 %r45, %r44, %r43;
shr.u32 %r46, %r45, 31;
shr.u32 %r47, %r45, 7;
add.s32 %r48, %r47, %r46;
st.global.u8 [%rd13], %r48;
$L__BB1_4:
ret;
}
// .globl blend_nv12
.visible .entry blend_nv12(
.param .u64 blend_nv12_param_0,
.param .u32 blend_nv12_param_1,
.param .u64 blend_nv12_param_2,
.param .u32 blend_nv12_param_3,
.param .u32 blend_nv12_param_4,
.param .u32 blend_nv12_param_5,
.param .u64 blend_nv12_param_6,
.param .u32 blend_nv12_param_7,
.param .u32 blend_nv12_param_8,
.param .u32 blend_nv12_param_9,
.param .u32 blend_nv12_param_10
)
{
.reg .pred %p<43>;
.reg .b16 %rs<17>;
.reg .f32 %f<108>;
.reg .b32 %r<123>;
.reg .b64 %rd<35>;
ld.param.u64 %rd11, [blend_nv12_param_0];
ld.param.u32 %r21, [blend_nv12_param_1];
ld.param.u64 %rd10, [blend_nv12_param_2];
ld.param.u32 %r22, [blend_nv12_param_3];
ld.param.u32 %r23, [blend_nv12_param_4];
ld.param.u32 %r24, [blend_nv12_param_5];
ld.param.u64 %rd12, [blend_nv12_param_6];
ld.param.u32 %r25, [blend_nv12_param_7];
ld.param.u32 %r26, [blend_nv12_param_8];
ld.param.u32 %r27, [blend_nv12_param_9];
ld.param.u32 %r28, [blend_nv12_param_10];
cvta.to.global.u64 %rd1, %rd11;
cvta.to.global.u64 %rd2, %rd12;
mov.u32 %r29, %ntid.x;
mov.u32 %r30, %ctaid.x;
mov.u32 %r31, %tid.x;
mad.lo.s32 %r32, %r30, %r29, %r31;
mov.u32 %r33, %ntid.y;
mov.u32 %r34, %ctaid.y;
mov.u32 %r35, %tid.y;
mad.lo.s32 %r36, %r34, %r33, %r35;
shl.b32 %r1, %r32, 1;
shl.b32 %r2, %r36, 1;
setp.ge.s32 %p1, %r1, %r25;
setp.ge.s32 %p2, %r2, %r26;
or.pred %p3, %p1, %p2;
mov.f32 %f102, 0f00000000;
mov.f32 %f103, 0f00000000;
mov.f32 %f104, 0f00000000;
@%p3 bra $L__BB2_19;
cvt.s64.s32 %rd3, %r21;
add.s32 %r3, %r2, %r28;
setp.ge.s32 %p4, %r3, %r24;
mul.lo.s32 %r4, %r2, %r25;
mul.wide.s32 %rd4, %r3, %r21;
add.s32 %r5, %r1, %r27;
or.b32 %r38, %r5, %r3;
setp.lt.s32 %p5, %r38, 0;
mov.u32 %r121, 0;
setp.ge.s32 %p6, %r5, %r23;
or.pred %p7, %p6, %p5;
or.pred %p8, %p4, %p7;
@%p8 bra $L__BB2_4;
add.s32 %r40, %r1, %r4;
mul.wide.s32 %rd13, %r40, 4;
add.s64 %rd5, %rd2, %rd13;
ld.global.u8 %rs1, [%rd5+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB2_4;
cvt.u32.u16 %r42, %rs1;
ld.global.u8 %rs5, [%rd5];
cvt.rn.f32.u16 %f31, %rs5;
ld.global.u8 %rs6, [%rd5+1];
cvt.rn.f32.u16 %f32, %rs6;
ld.global.u8 %rs7, [%rd5+2];
cvt.rn.f32.u16 %f33, %rs7;
fma.rn.f32 %f34, %f31, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f35, %f32, 0f3F1D3C36, %f34;
fma.rn.f32 %f36, %f33, 0f3D7DF3B6, %f35;
add.f32 %f37, %f36, 0f3F000000;
cvt.rzi.s32.f32 %r43, %f37;
cvt.s64.s32 %rd14, %r5;
add.s64 %rd15, %rd4, %rd14;
add.s64 %rd16, %rd1, %rd15;
ld.global.u8 %r44, [%rd16];
mul.lo.s32 %r45, %r43, %r42;
xor.b32 %r46, %r42, 255;
mad.lo.s32 %r47, %r46, %r44, %r45;
mul.hi.s32 %r48, %r47, -2139062143;
add.s32 %r49, %r48, %r47;
shr.u32 %r50, %r49, 31;
shr.u32 %r51, %r49, 7;
add.s32 %r52, %r51, %r50;
st.global.u8 [%rd16], %r52;
fma.rn.f32 %f38, %f31, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f39, %f32, 0fBEAD5CFB, %f38;
fma.rn.f32 %f40, %f33, 0f3EE0DED3, %f39;
cvt.rn.f32.u16 %f104, %rs1;
fma.rn.f32 %f102, %f40, %f104, 0f00000000;
fma.rn.f32 %f41, %f31, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f42, %f32, 0fBECC3C9F, %f41;
fma.rn.f32 %f43, %f33, 0fBD25119D, %f42;
fma.rn.f32 %f103, %f43, %f104, 0f00000000;
mov.u32 %r121, 1;
$L__BB2_4:
add.s32 %r7, %r1, 1;
setp.ge.s32 %p10, %r7, %r25;
@%p10 bra $L__BB2_8;
add.s32 %r8, %r7, %r27;
or.b32 %r53, %r8, %r3;
setp.lt.s32 %p12, %r53, 0;
setp.ge.s32 %p13, %r8, %r23;
or.pred %p14, %p13, %p12;
or.pred %p15, %p4, %p14;
@%p15 bra $L__BB2_8;
add.s32 %r54, %r7, %r4;
mul.wide.s32 %rd17, %r54, 4;
add.s64 %rd6, %rd2, %rd17;
ld.global.u8 %rs2, [%rd6+3];
setp.eq.s16 %p16, %rs2, 0;
@%p16 bra $L__BB2_8;
cvt.u32.u16 %r55, %rs2;
ld.global.u8 %rs8, [%rd6];
cvt.rn.f32.u16 %f44, %rs8;
ld.global.u8 %rs9, [%rd6+1];
cvt.rn.f32.u16 %f45, %rs9;
ld.global.u8 %rs10, [%rd6+2];
cvt.rn.f32.u16 %f46, %rs10;
fma.rn.f32 %f47, %f44, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f48, %f45, 0f3F1D3C36, %f47;
fma.rn.f32 %f49, %f46, 0f3D7DF3B6, %f48;
add.f32 %f50, %f49, 0f3F000000;
cvt.rzi.s32.f32 %r56, %f50;
cvt.s64.s32 %rd18, %r8;
add.s64 %rd19, %rd4, %rd18;
add.s64 %rd20, %rd1, %rd19;
ld.global.u8 %r57, [%rd20];
mul.lo.s32 %r58, %r56, %r55;
xor.b32 %r59, %r55, 255;
mad.lo.s32 %r60, %r59, %r57, %r58;
mul.hi.s32 %r61, %r60, -2139062143;
add.s32 %r62, %r61, %r60;
shr.u32 %r63, %r62, 31;
shr.u32 %r64, %r62, 7;
add.s32 %r65, %r64, %r63;
st.global.u8 [%rd20], %r65;
fma.rn.f32 %f51, %f44, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f52, %f45, 0fBEAD5CFB, %f51;
fma.rn.f32 %f53, %f46, 0f3EE0DED3, %f52;
cvt.rn.f32.u16 %f54, %rs2;
fma.rn.f32 %f102, %f53, %f54, %f102;
fma.rn.f32 %f55, %f44, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f56, %f45, 0fBECC3C9F, %f55;
fma.rn.f32 %f57, %f46, 0fBD25119D, %f56;
fma.rn.f32 %f103, %f57, %f54, %f103;
add.f32 %f104, %f104, %f54;
add.s32 %r121, %r121, 1;
$L__BB2_8:
add.s32 %r11, %r2, 1;
setp.ge.s32 %p17, %r11, %r26;
add.s32 %r12, %r11, %r28;
add.s32 %r13, %r4, %r25;
cvt.s64.s32 %rd21, %r12;
mul.lo.s64 %rd7, %rd21, %rd3;
@%p17 bra $L__BB2_12;
setp.ge.s32 %p18, %r12, %r24;
or.b32 %r66, %r5, %r12;
setp.lt.s32 %p19, %r66, 0;
or.pred %p21, %p6, %p19;
or.pred %p22, %p18, %p21;
@%p22 bra $L__BB2_12;
add.s32 %r67, %r1, %r13;
mul.wide.s32 %rd22, %r67, 4;
add.s64 %rd8, %rd2, %rd22;
ld.global.u8 %rs3, [%rd8+3];
setp.eq.s16 %p23, %rs3, 0;
@%p23 bra $L__BB2_12;
cvt.u32.u16 %r68, %rs3;
ld.global.u8 %rs11, [%rd8];
cvt.rn.f32.u16 %f58, %rs11;
ld.global.u8 %rs12, [%rd8+1];
cvt.rn.f32.u16 %f59, %rs12;
ld.global.u8 %rs13, [%rd8+2];
cvt.rn.f32.u16 %f60, %rs13;
fma.rn.f32 %f61, %f58, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f62, %f59, 0f3F1D3C36, %f61;
fma.rn.f32 %f63, %f60, 0f3D7DF3B6, %f62;
add.f32 %f64, %f63, 0f3F000000;
cvt.rzi.s32.f32 %r69, %f64;
cvt.s64.s32 %rd23, %r5;
add.s64 %rd24, %rd7, %rd23;
add.s64 %rd25, %rd1, %rd24;
ld.global.u8 %r70, [%rd25];
mul.lo.s32 %r71, %r69, %r68;
xor.b32 %r72, %r68, 255;
mad.lo.s32 %r73, %r72, %r70, %r71;
mul.hi.s32 %r74, %r73, -2139062143;
add.s32 %r75, %r74, %r73;
shr.u32 %r76, %r75, 31;
shr.u32 %r77, %r75, 7;
add.s32 %r78, %r77, %r76;
st.global.u8 [%rd25], %r78;
fma.rn.f32 %f65, %f58, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f66, %f59, 0fBEAD5CFB, %f65;
fma.rn.f32 %f67, %f60, 0f3EE0DED3, %f66;
cvt.rn.f32.u16 %f68, %rs3;
fma.rn.f32 %f102, %f67, %f68, %f102;
fma.rn.f32 %f69, %f58, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f70, %f59, 0fBECC3C9F, %f69;
fma.rn.f32 %f71, %f60, 0fBD25119D, %f70;
fma.rn.f32 %f103, %f71, %f68, %f103;
add.f32 %f104, %f104, %f68;
add.s32 %r121, %r121, 1;
$L__BB2_12:
or.pred %p26, %p17, %p10;
@%p26 bra $L__BB2_16;
setp.ge.s32 %p27, %r12, %r24;
add.s32 %r16, %r7, %r27;
or.b32 %r79, %r16, %r12;
setp.lt.s32 %p28, %r79, 0;
setp.ge.s32 %p29, %r16, %r23;
or.pred %p30, %p29, %p28;
or.pred %p31, %p27, %p30;
@%p31 bra $L__BB2_16;
add.s32 %r80, %r7, %r13;
mul.wide.s32 %rd26, %r80, 4;
add.s64 %rd9, %rd2, %rd26;
ld.global.u8 %rs4, [%rd9+3];
setp.eq.s16 %p32, %rs4, 0;
@%p32 bra $L__BB2_16;
cvt.u32.u16 %r81, %rs4;
ld.global.u8 %rs14, [%rd9];
cvt.rn.f32.u16 %f72, %rs14;
ld.global.u8 %rs15, [%rd9+1];
cvt.rn.f32.u16 %f73, %rs15;
ld.global.u8 %rs16, [%rd9+2];
cvt.rn.f32.u16 %f74, %rs16;
fma.rn.f32 %f75, %f72, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f76, %f73, 0f3F1D3C36, %f75;
fma.rn.f32 %f77, %f74, 0f3D7DF3B6, %f76;
add.f32 %f78, %f77, 0f3F000000;
cvt.rzi.s32.f32 %r82, %f78;
cvt.s64.s32 %rd27, %r16;
add.s64 %rd28, %rd7, %rd27;
add.s64 %rd29, %rd1, %rd28;
ld.global.u8 %r83, [%rd29];
mul.lo.s32 %r84, %r82, %r81;
xor.b32 %r85, %r81, 255;
mad.lo.s32 %r86, %r85, %r83, %r84;
mul.hi.s32 %r87, %r86, -2139062143;
add.s32 %r88, %r87, %r86;
shr.u32 %r89, %r88, 31;
shr.u32 %r90, %r88, 7;
add.s32 %r91, %r90, %r89;
st.global.u8 [%rd29], %r91;
fma.rn.f32 %f79, %f72, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f80, %f73, 0fBEAD5CFB, %f79;
fma.rn.f32 %f81, %f74, 0f3EE0DED3, %f80;
cvt.rn.f32.u16 %f82, %rs4;
fma.rn.f32 %f102, %f81, %f82, %f102;
fma.rn.f32 %f83, %f72, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f84, %f73, 0fBECC3C9F, %f83;
fma.rn.f32 %f85, %f74, 0fBD25119D, %f84;
fma.rn.f32 %f103, %f85, %f82, %f103;
add.f32 %f104, %f104, %f82;
add.s32 %r121, %r121, 1;
$L__BB2_16:
setp.eq.s32 %p33, %r121, 0;
setp.le.f32 %p34, %f104, 0f00000000;
or.pred %p35, %p34, %p33;
@%p35 bra $L__BB2_19;
shr.u32 %r92, %r5, 31;
add.s32 %r93, %r5, %r92;
shr.s32 %r19, %r93, 1;
setp.lt.s32 %p36, %r3, -1;
setp.lt.s32 %p37, %r5, -1;
or.pred %p38, %p37, %p36;
and.b32 %r94, %r93, -2;
setp.ge.s32 %p39, %r94, %r23;
or.pred %p40, %p38, %p39;
shr.u32 %r95, %r3, 31;
add.s32 %r96, %r3, %r95;
shr.s32 %r20, %r96, 1;
and.b32 %r97, %r96, -2;
setp.ge.s32 %p41, %r97, %r24;
or.pred %p42, %p40, %p41;
@%p42 bra $L__BB2_19;
div.rn.f32 %f86, %f102, %f104;
add.f32 %f87, %f86, 0f3F000000;
cvt.rzi.s32.f32 %r98, %f87;
div.rn.f32 %f88, %f103, %f104;
add.f32 %f89, %f88, 0f3F000000;
cvt.rzi.s32.f32 %r99, %f89;
cvt.rn.f32.s32 %f90, %r121;
div.rn.f32 %f91, %f104, %f90;
add.f32 %f92, %f91, 0f3F000000;
cvt.rzi.s32.f32 %r100, %f92;
mul.wide.s32 %rd30, %r20, %r22;
mul.wide.s32 %rd31, %r19, 2;
add.s64 %rd32, %rd30, %rd31;
cvta.to.global.u64 %rd33, %rd10;
add.s64 %rd34, %rd33, %rd32;
ld.global.u8 %r101, [%rd34];
mul.lo.s32 %r102, %r100, %r98;
mov.u32 %r103, 255;
sub.s32 %r104, %r103, %r100;
mad.lo.s32 %r105, %r104, %r101, %r102;
mul.hi.s32 %r106, %r105, -2139062143;
add.s32 %r107, %r106, %r105;
shr.u32 %r108, %r107, 31;
shr.u32 %r109, %r107, 7;
add.s32 %r110, %r109, %r108;
st.global.u8 [%rd34], %r110;
ld.global.u8 %r111, [%rd34+1];
mul.lo.s32 %r112, %r100, %r99;
mad.lo.s32 %r113, %r104, %r111, %r112;
mul.hi.s32 %r114, %r113, -2139062143;
add.s32 %r115, %r114, %r113;
shr.u32 %r116, %r115, 31;
shr.u32 %r117, %r115, 7;
add.s32 %r118, %r117, %r116;
st.global.u8 [%rd34+1], %r118;
$L__BB2_19:
ret;
}
+225 -99
View File
@@ -69,6 +69,7 @@ use super::{AuChunk, ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
use anyhow::{anyhow, bail, Context, Result}; use anyhow::{anyhow, bail, Context, Result};
use pf_frame::{CapturedFrame, FramePayload}; use pf_frame::{CapturedFrame, FramePayload};
use pf_zerocopy::cuda::{self, InputSurface}; use pf_zerocopy::cuda::{self, InputSurface};
use pf_zerocopy::vkslot::{SlotFormat, VkSlotBlend, VkSlotRef};
use std::collections::VecDeque; use std::collections::VecDeque;
use std::ffi::c_void; use std::ffi::c_void;
use std::ptr; use std::ptr;
@@ -76,12 +77,6 @@ use std::sync::mpsc;
use nvidia_video_codec_sdk::sys::nvEncodeAPI as nv; use nvidia_video_codec_sdk::sys::nvEncodeAPI as nv;
/// Prebuilt PTX for the cursor-overlay blend kernels (cursor-as-metadata). Source is
/// `cursor_blend.cu` beside this file; regenerate with
/// `nvcc -ptx -arch=compute_75 cursor_blend.cu -o cursor_blend.ptx` after editing. JIT'd by the
/// driver, so it runs on any Turing-or-newer GPU.
const CURSOR_PTX: &[u8] = include_bytes!("cursor_blend.ptx");
// --------------------------------------------------------------------------------------------- // ---------------------------------------------------------------------------------------------
// Runtime-loaded NVENC entry table (Linux). Same shape as the Windows backend's `EncodeApi`, minus // Runtime-loaded NVENC entry table (Linux). Same shape as the Windows backend's `EncodeApi`, minus
// the async-event entry points (Windows-only). Resolved once from `libnvidia-encode.so.1` — the two // the async-event entry points (Windows-only). Resolved once from `libnvidia-encode.so.1` — the two
@@ -428,13 +423,54 @@ fn buffer_format(buf: &cuda::DeviceBuffer) -> nv::NV_ENC_BUFFER_FORMAT {
} }
} }
/// One encoder-owned CUDA input surface + its NVENC registration. The surface is copied into each /// One encoder-owned input surface + its NVENC registration. The surface is copied into each
/// use (device→device) and the registration is created once at session init, unregistered at teardown. /// use (device→device) and the registration is created once at session init, unregistered at teardown.
struct RingSlot { struct RingSlot {
surface: InputSurface, surface: SlotSurface,
reg: nv::NV_ENC_REGISTERED_PTR, reg: nv::NV_ENC_REGISTERED_PTR,
} }
/// The ring slot's backing allocation: Vulkan external memory CUDA-imported (the normal case —
/// blendable by the SPIR-V cursor pass, see `vkslot.rs`) or a plain pitched CUDA allocation (the
/// fallback when Vulkan bring-up fails: sessions still encode, composite mode just has no
/// cursor). Both present the same `(ptr, pitch, height)` NVENC-registration vocabulary.
enum SlotSurface {
Cuda(InputSurface),
/// Backing objects live in the encoder's [`VkSlotBlend`] (freed by its `free_slots`); the
/// ref itself is Copy and carries the registered geometry.
Vk(VkSlotRef),
}
/// The [`SlotFormat`] for an NVENC buffer format (the ring-build + blend vocabulary).
fn slot_fmt_of(fmt: nv::NV_ENC_BUFFER_FORMAT) -> SlotFormat {
match fmt {
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => SlotFormat::Yuv444,
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_NV12 => SlotFormat::Nv12,
_ => SlotFormat::Argb,
}
}
impl SlotSurface {
fn ptr(&self) -> pf_zerocopy::cuda::CUdeviceptr {
match self {
SlotSurface::Cuda(s) => s.ptr,
SlotSurface::Vk(r) => r.ptr,
}
}
fn pitch(&self) -> usize {
match self {
SlotSurface::Cuda(s) => s.pitch,
SlotSurface::Vk(r) => r.pitch,
}
}
fn height(&self) -> u32 {
match self {
SlotSurface::Cuda(s) => s.height,
SlotSurface::Vk(r) => r.height,
}
}
}
/// `doNotWait` sampling cadence inside [`Encoder::poll_chunk`] — the probe measured ~200 µs /// `doNotWait` sampling cadence inside [`Encoder::poll_chunk`] — the probe measured ~200 µs
/// between slice completions on the 5070 Ti, so 50 µs keeps the added per-chunk delivery delay /// between slice completions on the 5070 Ti, so 50 µs keeps the added per-chunk delivery delay
/// well under one slice time without hammering the driver. /// well under one slice time without hammering the driver.
@@ -530,16 +566,23 @@ pub struct NvencCudaEncoder {
split_mode: u32, split_mode: u32,
/// The last reference-frame range we invalidated — dedupes repeated RFI requests for one loss. /// The last reference-frame range we invalidated — dedupes repeated RFI requests for one loss.
last_rfi_range: Option<(i64, i64)>, last_rfi_range: Option<(i64, i64)>,
/// Cursor-as-metadata GPU blend (loaded lazily on the first frame that carries a cursor, once the /// Cursor-as-metadata GPU blend: the Vulkan device + SPIR-V compute pass the ring's
/// CUDA context is current). `None` until then or if the module load fails; `cursor_tried` stops /// external-memory slots are allocated through (`vkslot.rs`) — the driver-portable
/// re-attempting a failed load every frame. `cursor_serial` tracks the uploaded bitmap. /// replacement for the retired PTX kernels. Brought up once at session init (`cursor_tried`
cursor: Option<cuda::CursorBlend>, /// stops re-attempts); `None` = bring-up failed, the ring fell back to plain CUDA
/// allocations and composite mode degrades to no cursor. `cursor_serial` tracks the
/// uploaded bitmap.
vk_blend: Option<VkSlotBlend>,
/// The session may hand this encoder cursor overlays (`SessionPlan.cursor_blend` — only
/// cursor-channel sessions since Phase B). Off = skip the Vulkan bring-up entirely and ring
/// on plain CUDA surfaces: embedded-pointer sessions never carry an overlay, so they pay
/// zero blend cost, per-session or per-frame.
blend_wanted: bool,
cursor_tried: bool, cursor_tried: bool,
cursor_serial: u64, cursor_serial: u64,
/// Suppress-until-success latches for the per-frame cursor upload/blend warns: a persistent /// Suppress-until-success latch for the per-frame blend warn: a persistent failure sits in
/// failure sits in the submit() hot path, so warn once per failure streak (reset on success) /// the submit() hot path, so warn once per failure streak (reset on success) rather than on
/// rather than on every cursor-bearing frame, which would evict the log ring. /// every cursor-bearing frame, which would evict the log ring.
cursor_upload_warned: bool,
cursor_blend_warned: bool, cursor_blend_warned: bool,
/// One-shot latch for [`diagnose_failed_open`](Self::diagnose_failed_open) so a rebuild-retry /// One-shot latch for [`diagnose_failed_open`](Self::diagnose_failed_open) so a rebuild-retry
/// burst (the session loop's bounded encoder resets) logs the diagnosis once, not per attempt. /// burst (the session loop's bounded encoder resets) logs the diagnosis once, not per attempt.
@@ -607,6 +650,7 @@ impl NvencCudaEncoder {
_cuda: bool, _cuda: bool,
bit_depth: u8, bit_depth: u8,
chroma: ChromaFormat, chroma: ChromaFormat,
cursor_blend: bool,
) -> Result<Self> { ) -> Result<Self> {
// The runtime `.so` load is the real "is NVENC possible here" gate: fail the open with a // The runtime `.so` load is the real "is NVENC possible here" gate: fail the open with a
// clear reason instead of an opaque session error on the first frame. // clear reason instead of an opaque session error on the first frame.
@@ -645,10 +689,10 @@ impl NvencCudaEncoder {
frame_idx: 0, frame_idx: 0,
force_kf: false, force_kf: false,
pending_anchor: false, pending_anchor: false,
cursor: None, vk_blend: None,
blend_wanted: cursor_blend,
cursor_tried: false, cursor_tried: false,
cursor_serial: u64::MAX, cursor_serial: u64::MAX,
cursor_upload_warned: false,
cursor_blend_warned: false, cursor_blend_warned: false,
diagnosed: false, diagnosed: false,
inited: false, inited: false,
@@ -739,7 +783,12 @@ impl NvencCudaEncoder {
// (the forfeit contract), and the next session re-latches the arming at init. // (the forfeit contract), and the next session re-latches the arming at init.
self.subframe_chunks = false; self.subframe_chunks = false;
self.chunk = None; self.chunk = None;
self.ring.clear(); // drops the InputSurfaces, freeing their CUDA allocations self.ring.clear(); // drops the CUDA InputSurfaces; Vk slots are freed just below
if let Some(vk) = &mut self.vk_blend {
// The Vulkan-backed slots' memory (and its CUDA mapping) — the device itself stays
// up for the next session's ring (`cursor_tried` keeps bring-up one-shot).
vk.free_slots();
}
self.bitstreams.clear(); self.bitstreams.clear();
self.pending.clear(); self.pending.clear();
self.encoder = ptr::null_mut(); self.encoder = ptr::null_mut();
@@ -1128,36 +1177,102 @@ impl NvencCudaEncoder {
// Encoder-owned input-surface ring: allocate + register POOL surfaces in the negotiated // Encoder-owned input-surface ring: allocate + register POOL surfaces in the negotiated
// format. Registered once here, mapped per submit, unregistered at teardown. // format. Registered once here, mapped per submit, unregistered at teardown.
for _ in 0..POOL { // Preferred backing = Vulkan external memory CUDA-imported (`vkslot.rs`), so the
let surface = match self.buffer_fmt { // SPIR-V cursor blend can composite into the very bytes NVENC encodes; any bring-up
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => { // or per-slot failure falls back to plain pitched CUDA allocations (sessions always
InputSurface::alloc_yuv444(self.width, self.height) // encode — composite mode just loses the cursor, warned below).
} if !self.cursor_tried && self.blend_wanted {
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_NV12 => { self.cursor_tried = true;
InputSurface::alloc_nv12(self.width, self.height) match VkSlotBlend::new() {
} Ok(v) => self.vk_blend = Some(v),
_ => InputSurface::alloc_rgb(self.width, self.height), Err(e) => tracing::warn!(
error = %format!("{e:#}"),
"NVENC (Linux): Vulkan slot-blend bring-up failed — plain CUDA input \
surfaces, cursor compositing unavailable"
),
} }
.context("alloc NVENC input surface")?; }
let mut rr = nv::NV_ENC_REGISTER_RESOURCE { let slot_fmt = slot_fmt_of(self.buffer_fmt);
version: nv::NV_ENC_REGISTER_RESOURCE_VER, // Two attempts: the full ring on Vulkan slots, else (any failure) the full ring on
resourceType: // plain CUDA — never a mixed ring (it would blend on some slots only: a flickering
nv::NV_ENC_INPUT_RESOURCE_TYPE::NV_ENC_INPUT_RESOURCE_TYPE_CUDADEVICEPTR, // cursor) and never a short one.
width: self.width, 'ring: for use_vk in [self.vk_blend.is_some(), false] {
height: self.height, if !use_vk && self.vk_blend.is_some() {
pitch: surface.pitch as u32, // Second attempt: retire the Vulkan side wholesale first.
resourceToRegister: surface.ptr as *mut c_void, for s in self.ring.drain(..) {
bufferFormat: self.buffer_fmt, let _ = (api().unregister_resource)(self.encoder, s.reg);
bufferUsage: nv::NV_ENC_BUFFER_USAGE::NV_ENC_INPUT_IMAGE, }
..Default::default() if let Some(vk) = &mut self.vk_blend {
}; vk.free_slots();
(api().register_resource)(self.encoder, &mut rr) }
.nv_ok() self.vk_blend = None;
.map_err(|e| nvenc_status::call_err("register_resource (CUDADEVICEPTR)", e))?; }
self.ring.push(RingSlot { for _ in 0..POOL {
surface, let surface = if use_vk {
reg: rr.registeredResource, let vk = self.vk_blend.as_mut().expect("use_vk implies Some");
}); match vk.alloc_slot(slot_fmt, self.width, self.height) {
Ok(r) => SlotSurface::Vk(r),
Err(e) => {
tracing::warn!(
error = %format!("{e:#}"),
"NVENC (Linux): Vulkan slot alloc failed — rebuilding the \
ring on plain CUDA surfaces (cursor compositing \
unavailable)"
);
continue 'ring;
}
}
} else {
SlotSurface::Cuda(
match self.buffer_fmt {
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => {
InputSurface::alloc_yuv444(self.width, self.height)
}
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_NV12 => {
InputSurface::alloc_nv12(self.width, self.height)
}
_ => InputSurface::alloc_rgb(self.width, self.height),
}
.context("alloc NVENC input surface")?,
)
};
let mut rr = nv::NV_ENC_REGISTER_RESOURCE {
version: nv::NV_ENC_REGISTER_RESOURCE_VER,
resourceType:
nv::NV_ENC_INPUT_RESOURCE_TYPE::NV_ENC_INPUT_RESOURCE_TYPE_CUDADEVICEPTR,
width: self.width,
height: self.height,
pitch: surface.pitch() as u32,
resourceToRegister: surface.ptr() as *mut c_void,
bufferFormat: self.buffer_fmt,
bufferUsage: nv::NV_ENC_BUFFER_USAGE::NV_ENC_INPUT_IMAGE,
..Default::default()
};
match (api().register_resource)(self.encoder, &mut rr).nv_ok() {
Ok(()) => {}
Err(e) if use_vk => {
// NVENC refusing the imported pointer is a Vulkan-side condition
// too — same wholesale fallback.
tracing::warn!(
error = ?e,
"NVENC (Linux): registering a Vulkan-imported slot failed — \
rebuilding the ring on plain CUDA surfaces"
);
continue 'ring;
}
Err(e) => {
return Err(nvenc_status::call_err(
"register_resource (CUDADEVICEPTR)",
e,
))
}
}
self.ring.push(RingSlot {
surface,
reg: rr.registeredResource,
});
}
break 'ring; // full ring built
} }
self.inited = true; self.inited = true;
@@ -1242,9 +1357,9 @@ impl NvencCudaEncoder {
/// IO-stream binding (stream-ordered submit — see the gate in [`Encoder::submit`]). /// IO-stream binding (stream-ordered submit — see the gate in [`Encoder::submit`]).
fn copy_into_slot(&self, buf: &cuda::DeviceBuffer, slot: usize, sync: bool) -> Result<()> { fn copy_into_slot(&self, buf: &cuda::DeviceBuffer, slot: usize, sync: bool) -> Result<()> {
let s = &self.ring[slot].surface; let s = &self.ring[slot].surface;
let base = s.ptr; let base = s.ptr();
let pitch = s.pitch; let pitch = s.pitch();
let hh = s.height as u64; let hh = s.height() as u64;
match self.buffer_fmt { match self.buffer_fmt {
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => { nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => {
if !buf.yuv444 { if !buf.yuv444 {
@@ -1396,72 +1511,83 @@ impl Encoder for NvencCudaEncoder {
// `async_rt` must be absent too: in two-thread mode the frame may be recycled right after // `async_rt` must be absent too: in two-thread mode the frame may be recycled right after
// submit returns while the stream still holds its copy (belt-and-braces — an escalated // submit returns while the stream still holds its copy (belt-and-braces — an escalated
// session was rebuilt without the binding, so `stream_ordered` is false there anyway). // session was rebuilt without the binding, so `stream_ordered` is false there anyway).
let ordered = self.stream_ordered && self.async_rt.is_none() && self.pending.is_empty(); // Cursor-bearing frames additionally force the CPU-synced path: the Vulkan blend sits
// between the CUDA copy and the encode, and its cross-API ordering is fence/CPU-
// established, not stream-ordered. Frames without a cursor (games hide it; client-draws
// sessions strip it) keep the stream-ordered fast path untouched.
let ordered = self.stream_ordered
&& self.async_rt.is_none()
&& self.pending.is_empty()
&& captured.cursor.is_none();
let t0 = std::time::Instant::now(); let t0 = std::time::Instant::now();
// Copy the captured buffer into this slot's input surface before encoding it. // Copy the captured buffer into this slot's input surface before encoding it.
self.copy_into_slot(buf, slot, !ordered)?; self.copy_into_slot(buf, slot, !ordered)?;
let t_copy = t0.elapsed(); let t_copy = t0.elapsed();
// Cursor-as-metadata: blend the overlay into this slot's OWNED input surface (a tiny kernel // Cursor-as-metadata: blend the overlay into this slot's OWNED input surface via the
// over the cursor's rect — never the compositor's dmabuf). The PTX module loads lazily on the // SPIR-V compute pass (a dispatch over the cursor's rect — never the compositor's
// first cursor frame now that the CUDA context is current; any failure degrades to no cursor, // dmabuf). Cursor-bearing frames forced `ordered = false` above, so the CUDA copy has
// never a dropped frame. // completed before the Vulkan dispatch and the fence-waited dispatch completes before
// the encode below — the cross-API ordering is CPU-established. Any failure degrades to
// no cursor, never a dropped frame.
if let Some(ov) = &captured.cursor { if let Some(ov) = &captured.cursor {
if !self.cursor_tried { if let (Some(vk), SlotSurface::Vk(vref)) =
self.cursor_tried = true; (self.vk_blend.as_mut(), &self.ring[slot].surface)
match cuda::CursorBlend::new(CURSOR_PTX) { {
Ok(cb) => self.cursor = Some(cb),
Err(e) => tracing::warn!(
error = %format!("{e:#}"),
"NVENC (Linux): cursor blend module load failed — cursor not composited"
),
}
}
if let Some(cb) = &self.cursor {
if self.cursor_serial != ov.serial { if self.cursor_serial != ov.serial {
match cb.upload(ov.rgba.as_slice(), ov.w, ov.h) { vk.upload_cursor(ov.rgba.as_slice(), ov.w, ov.h);
Ok(()) => { self.cursor_serial = ov.serial;
self.cursor_serial = ov.serial;
self.cursor_upload_warned = false;
}
Err(e) => {
if !self.cursor_upload_warned {
self.cursor_upload_warned = true;
tracing::warn!(
error = %format!("{e:#}"),
serial = ov.serial,
"NVENC (Linux): cursor upload failed — cursor not composited"
);
}
}
}
} }
let s = &self.ring[slot].surface; // surfW = content width; the blend derives plane strides from the slot's luma
// surfW = content width; surfH = the surface's allocated height (matches // height. Cursor pixels past the content land in cropped padding rows — harmless.
// `copy_into_slot`'s plane math). Cursor pixels past the content are in cropped let r = vk.blend_ref(
// padding rows — harmless. vref,
let (w, h) = (self.width, s.height); slot_fmt_of(self.buffer_fmt),
let r = match self.buffer_fmt { self.width,
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => { ov.w,
cb.blend_yuv444(s.ptr, s.pitch, w, h, ov.w, ov.h, ov.x, ov.y, !ordered) ov.h,
} ov.x,
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_NV12 => { ov.y,
cb.blend_nv12(s.ptr, s.pitch, w, h, ov.w, ov.h, ov.x, ov.y, !ordered) );
}
_ => cb.blend_argb(s.ptr, s.pitch, w, h, ov.w, ov.h, ov.x, ov.y, !ordered),
};
if let Err(e) = r { if let Err(e) = r {
if !self.cursor_blend_warned { if !self.cursor_blend_warned {
self.cursor_blend_warned = true; self.cursor_blend_warned = true;
tracing::warn!( tracing::warn!(
error = %format!("{e:#}"), error = %format!("{e:#}"),
"NVENC (Linux): cursor blend launch failed — cursor not composited" "NVENC (Linux): cursor blend dispatch failed — cursor not composited"
); );
} }
} else { } else {
self.cursor_blend_warned = false; self.cursor_blend_warned = false;
// TEMP KWin composite probe (DROP BEFORE MERGE): prove the blend dispatches
// and with what geometry.
{
use std::sync::atomic::{AtomicU64, Ordering as ProbeOrd};
static PROBE_BLEND: AtomicU64 = AtomicU64::new(0);
let n = PROBE_BLEND.fetch_add(1, ProbeOrd::Relaxed) + 1;
if n == 1 || n % 512 == 0 {
tracing::info!(
n,
fmt = ?self.buffer_fmt,
ov_x = ov.x,
ov_y = ov.y,
ov_w = ov.w,
ov_h = ov.h,
visible = ov.visible,
"cursor blend probe: vulkan dispatch submitted"
);
}
}
} }
} else if !self.cursor_blend_warned {
self.cursor_blend_warned = true;
tracing::warn!(
blend_wanted = self.blend_wanted,
"NVENC (Linux): cursor overlay present but no Vulkan blend (bring-up failed, \
or a non-blend session unexpectedly carried an overlay) cursor not \
composited"
);
} }
} }
@@ -1509,7 +1635,7 @@ impl Encoder for NvencCudaEncoder {
version: nv::NV_ENC_PIC_PARAMS_VER, version: nv::NV_ENC_PIC_PARAMS_VER,
inputWidth: self.width, inputWidth: self.width,
inputHeight: self.height, inputHeight: self.height,
inputPitch: self.ring[slot].surface.pitch as u32, inputPitch: self.ring[slot].surface.pitch() as u32,
inputBuffer: mp.mappedResource, inputBuffer: mp.mappedResource,
bufferFmt: mp.mappedBufferFmt, bufferFmt: mp.mappedBufferFmt,
outputBitstream: self.bitstreams[slot], outputBitstream: self.bitstreams[slot],
+11 -6
View File
@@ -270,7 +270,7 @@ fn open_video_backend(
chroma: ChromaFormat, chroma: ChromaFormat,
cursor_blend: bool, cursor_blend: bool,
) -> Result<(Box<dyn Encoder>, &'static str)> { ) -> Result<(Box<dyn Encoder>, &'static str)> {
let _ = cursor_blend; // consumed only by the Linux vulkan-encode arm below let _ = cursor_blend; // consumed only by the Linux vulkan-encode + direct-NVENC arms below
validate_dimensions(codec, width, height)?; validate_dimensions(codec, width, height)?;
// Refresh/fps must be positive and sane: fps feeds the encoder time_base (`Rational(1, fps)`) // Refresh/fps must be positive and sane: fps feeds the encoder time_base (`Rational(1, fps)`)
// and the pts→ns conversion (`pts * 1e9 / fps`), so 0 builds a 1/0 rational / divides by zero. // and the pts→ns conversion (`pts * 1e9 / fps`), so 0 builds a 1/0 rational / divides by zero.
@@ -394,6 +394,7 @@ fn open_video_backend(
cuda, cuda,
bit_depth, bit_depth,
chroma, chroma,
cursor_blend,
) )
.map(|e| (e, "nvenc")) .map(|e| (e, "nvenc"))
}; };
@@ -729,12 +730,15 @@ fn open_nvenc_probed(
cuda: bool, cuda: bool,
bit_depth: u8, bit_depth: u8,
chroma: ChromaFormat, chroma: ChromaFormat,
cursor_blend: bool,
) -> Result<Box<dyn Encoder>> { ) -> Result<Box<dyn Encoder>> {
// Direct-SDK NVENC (design/linux-direct-nvenc.md): the DEFAULT on NVIDIA, and only for a CUDA #[cfg(not(feature = "nvenc"))]
// capture payload (it registers CUDADEVICEPTR inputs — a CPU/dmabuf frame can't feed it, so those let _ = cursor_blend; // consumed by the direct-SDK arm below
// keep the libav path; and `cuda` is false on AMD/Intel, so they stay on VAAPI). Set // Direct-SDK NVENC (design/linux-direct-nvenc.md): the DEFAULT on NVIDIA, and only for a CUDA
// PUNKTFUNK_NVENC_DIRECT=0 to fall back to libav. It self-clamps the bitrate internally (its own // capture payload (it registers CUDADEVICEPTR inputs — a CPU/dmabuf frame can't feed it, so those
// level-ceiling binary search at session open), so it skips the probe-loop stepping below. // keep the libav path; and `cuda` is false on AMD/Intel, so they stay on VAAPI). Set
// PUNKTFUNK_NVENC_DIRECT=0 to fall back to libav. It self-clamps the bitrate internally (its own
// level-ceiling binary search at session open), so it skips the probe-loop stepping below.
#[cfg(feature = "nvenc")] #[cfg(feature = "nvenc")]
if cuda && nvenc_direct_enabled() { if cuda && nvenc_direct_enabled() {
tracing::info!( tracing::info!(
@@ -751,6 +755,7 @@ fn open_nvenc_probed(
cuda, cuda,
bit_depth, bit_depth,
chroma, chroma,
cursor_blend,
)?) as Box<dyn Encoder>); )?) as Box<dyn Encoder>);
} }
// The silent-degrade trap: a build without `--features nvenc` compiles the direct-SDK // The silent-degrade trap: a build without `--features nvenc` compiles the direct-SDK
+7
View File
@@ -32,6 +32,13 @@ pub struct WinCaptureTarget {
/// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown /// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown
/// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse). /// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse).
pub wudf_pid: u32, pub wudf_pid: u32,
/// The ADD reply flagged the ADAPTER as carrying an IRREVOCABLE IddCx hardware-cursor declare
/// from an earlier session (remote-desktop-sweep §8.6; reach is adapter-wide, not per-target —
/// on-glass 2026-07-23, a GameStream session's fresh target streamed cursor-less): DWM
/// excludes the pointer from its frames until adapter reset, so a session WITHOUT the cursor
/// channel must self-composite (the IDD-push capturer's forced-composite gate) or the
/// streamed desktop has no cursor at all.
pub cursor_excluded: bool,
} }
/// The PyroWave (Windows) zero-copy sharing payload attached to a captured frame: the SECOND plane /// The PyroWave (Windows) zero-copy sharing payload attached to a captured frame: the SECOND plane
+18
View File
@@ -152,6 +152,12 @@ pub struct OutputFormat {
/// codec's backend, and the fallback family) would misread the two-plane buffer as packed /// codec's backend, and the fallback family) would misread the two-plane buffer as packed
/// RGB. Always `false` on Windows (the IDD-push capturer owns its own formats). /// RGB. Always `false` on Windows (the IDD-push capturer owns its own formats).
pub nv12_native: bool, pub nv12_native: bool,
/// The session negotiated the cursor-forward channel (remote-desktop-sweep M2c): on Windows
/// the IDD-push capturer creates + delivers the driver's hardware-cursor section, so DWM
/// stops compositing the pointer into the frames and the capturer surfaces it via
/// `Capturer::cursor()` instead. Ignored on Linux (the portal's `SPA_META_Cursor` already
/// separates the pointer; the session plan's `cursor_blend` gate handles the rest).
pub hw_cursor: bool,
} }
impl OutputFormat { impl OutputFormat {
@@ -169,6 +175,8 @@ impl OutputFormat {
chroma_444: false, chroma_444: false,
// GameStream never negotiates PyroWave (native punktfunk/1 only). // GameStream never negotiates PyroWave (native punktfunk/1 only).
pyrowave: false, pyrowave: false,
// GameStream/spike sessions never negotiate the cursor channel.
hw_cursor: false,
// Conservative: the GameStream + spike paths don't resolve the codec here, and a // Conservative: the GameStream + spike paths don't resolve the codec here, and a
// Moonlight client may negotiate H264 (whose VAAPI backend can't ingest NV12) — so // Moonlight client may negotiate H264 (whose VAAPI backend can't ingest NV12) — so
// they never prefer the producer-native NV12 pod. The punktfunk/1 plane opts in via // they never prefer the producer-native NV12 pod. The punktfunk/1 plane opts in via
@@ -196,6 +204,16 @@ pub struct CursorOverlay {
pub rgba: std::sync::Arc<Vec<u8>>, pub rgba: std::sync::Arc<Vec<u8>>,
/// Bumps whenever `rgba`/`w`/`h` change; stable across position-only moves. /// Bumps whenever `rgba`/`w`/`h` change; stable across position-only moves.
pub serial: u64, pub serial: u64,
/// Hotspot (the pixel that IS the pointer position) within `w`×`h`. The blend paths ignore
/// it (`x`/`y` are already hotspot-adjusted); the cursor-forward channel ships it to the
/// client so a locally-drawn OS cursor points with the right pixel.
pub hot_x: u32,
pub hot_y: u32,
/// Compositor-reported pointer visibility. `false` = an app on the host grabbed/hid the
/// pointer — the cursor-forward channel turns that into the client's relative-mode hint
/// (remote-desktop-sweep M3). The encode loop STRIPS invisible overlays before the frame
/// reaches any blend path, so encoders may keep treating `Some` as "draw it".
pub visible: bool,
} }
/// A captured frame. [`format`](Self::format)/dimensions describe the pixels regardless of /// A captured frame. [`format`](Self::format)/dimensions describe the pixels regardless of
+8
View File
@@ -48,6 +48,9 @@ xkbcommon = "0.8"
usbip-sim = { path = "../punktfunk-host/vendor/usbip-sim" } usbip-sim = { path = "../punktfunk-host/vendor/usbip-sim" }
[target.'cfg(target_os = "windows")'.dependencies] [target.'cfg(target_os = "windows")'.dependencies]
# The streamed-output desktop rect (CCD source rect) absolute input maps into — the same
# resolver the cursor-readback poller uses, so inject and readback always agree.
pf-win-display = { path = "../pf-win-display" }
windows = { version = "0.62", features = [ windows = { version = "0.62", features = [
"Win32_Foundation", "Win32_Foundation",
"Win32_Security", "Win32_Security",
@@ -61,5 +64,10 @@ windows = { version = "0.62", features = [
"Win32_System_StationsAndDesktops", "Win32_System_StationsAndDesktops",
"Win32_System_Threading", "Win32_System_Threading",
"Win32_UI_Input_KeyboardAndMouse", "Win32_UI_Input_KeyboardAndMouse",
# Synthetic pointer devices (PT_PEN / PT_TOUCH) — the pen/touch injectors. The device
# create/destroy + POINTER_TYPE_INFO live under Controls in this metadata layout;
# InjectSyntheticPointerInput + POINTER_*_INFO under Input_Pointer.
"Win32_UI_Controls",
"Win32_UI_Input_Pointer",
"Win32_UI_WindowsAndMessaging", "Win32_UI_WindowsAndMessaging",
] } ] }
+6
View File
@@ -36,6 +36,12 @@ pub fn vk_to_evdev(vk: u8) -> Option<u16> {
0x2D => Some(110), // VK_INSERT -> KEY_INSERT 0x2D => Some(110), // VK_INSERT -> KEY_INSERT
0x2E => Some(111), // VK_DELETE -> KEY_DELETE 0x2E => Some(111), // VK_DELETE -> KEY_DELETE
// --- Consumer/media keys (Android TV remotes, keyboard media rows) ---
0xB0 => Some(163), // VK_MEDIA_NEXT_TRACK -> KEY_NEXTSONG
0xB1 => Some(165), // VK_MEDIA_PREV_TRACK -> KEY_PREVIOUSSONG
0xB2 => Some(166), // VK_MEDIA_STOP -> KEY_STOPCD
0xB3 => Some(164), // VK_MEDIA_PLAY_PAUSE -> KEY_PLAYPAUSE
// --- Generic modifiers --- // --- Generic modifiers ---
0x10 => Some(42), // VK_SHIFT -> KEY_LEFTSHIFT 0x10 => Some(42), // VK_SHIFT -> KEY_LEFTSHIFT
0x11 => Some(29), // VK_CONTROL -> KEY_LEFTCTRL 0x11 => Some(29), // VK_CONTROL -> KEY_LEFTCTRL
@@ -424,6 +424,9 @@ impl InputInjector for KwinFakeInjector {
self.fake.touch_up(event.code); self.fake.touch_up(event.code);
self.fake.touch_frame(); self.fake.touch_frame();
} }
// fake_input can only press host-layout keycodes — no committed-text path (the
// HOST_CAP_TEXT_INPUT cap is not advertised on this backend).
InputKind::TextInput => {}
// Gamepads are injected through uinput, not the compositor. // Gamepads are injected through uinput, not the compositor.
InputKind::GamepadState InputKind::GamepadState
| InputKind::GamepadButton | InputKind::GamepadButton
+205 -30
View File
@@ -109,7 +109,7 @@ async fn session_main(mut rx: UnboundedReceiver<InputEvent>, source: EiSource) {
// Keep `_rd`/`_session` bound for the whole loop — dropping the portal session closes the // Keep `_rd`/`_session` bound for the whole loop — dropping the portal session closes the
// EIS connection. Bound the setup so a headless approval dialog (un-bypassed grant) can't // EIS connection. Bound the setup so a headless approval dialog (un-bypassed grant) can't
// hang the worker forever. // hang the worker forever.
let (_keepalive, context, mut events) = match tokio::time::timeout( let (_keepalive, context, mut events, output_hint) = match tokio::time::timeout(
Duration::from_secs(30), Duration::from_secs(30),
connect(source), connect(source),
) )
@@ -130,6 +130,7 @@ async fn session_main(mut rx: UnboundedReceiver<InputEvent>, source: EiSource) {
tracing::info!("libei: EIS connected — awaiting devices"); tracing::info!("libei: EIS connected — awaiting devices");
let mut state = EiState::new(); let mut state = EiState::new();
state.output_hint = output_hint;
// Watchdog: a healthy EIS server adds + resumes an input device within a beat of the handshake. // Watchdog: a healthy EIS server adds + resumes an input device within a beat of the handshake.
// If none has resumed by this deadline, the connection is dead-on-arrival (stale/half-ready // If none has resumed by this deadline, the connection is dead-on-arrival (stale/half-ready
// gamescope socket the handshake passed but no real server is behind) — exit so the next // gamescope socket the handshake passed but no real server is behind) — exit so the next
@@ -177,21 +178,29 @@ type Connected = (
Box<dyn Send>, Box<dyn Send>,
ei::Context, ei::Context,
reis::tokio::EiConvertEventStream, reis::tokio::EiConvertEventStream,
// The compositor's output size ("WxH" relay-file hint) — the scale target for absolute
// coordinates when the EIS advertises only a degenerate region (gamescope). `None` for
// the portal/Mutter paths, whose regions are real.
Option<(u32, u32)>,
); );
/// Reach an EIS server per `source` and run the EI sender handshake. /// Reach an EIS server per `source` and run the EI sender handshake.
async fn connect(source: EiSource) -> Result<Connected> { async fn connect(source: EiSource) -> Result<Connected> {
let (keepalive, stream): (Box<dyn Send>, UnixStream) = match source { let (keepalive, stream, output_hint): (Box<dyn Send>, UnixStream, Option<(u32, u32)>) =
EiSource::Portal => { match source {
let (rd, session, fd) = connect_portal().await?; EiSource::Portal => {
(Box::new((rd, session)), UnixStream::from(fd)) let (rd, session, fd) = connect_portal().await?;
} (Box::new((rd, session)), UnixStream::from(fd), None)
EiSource::MutterEis => { }
let (keepalive, fd) = connect_mutter().await?; EiSource::MutterEis => {
(keepalive, UnixStream::from(fd)) let (keepalive, fd) = connect_mutter().await?;
} (keepalive, UnixStream::from(fd), None)
EiSource::SocketPathFile(file) => (Box::new(()), connect_socket_file(&file).await?), }
}; EiSource::SocketPathFile(file) => {
let (stream, hint) = connect_socket_file(&file).await?;
(Box::new(()), stream, hint)
}
};
let context = ei::Context::new(stream).map_err(|e| anyhow!("reis EI context: {e}"))?; let context = ei::Context::new(stream).map_err(|e| anyhow!("reis EI context: {e}"))?;
// Bound the handshake. `UnixStream::connect` to a socket *file* succeeds the moment the path // Bound the handshake. `UnixStream::connect` to a socket *file* succeeds the moment the path
// exists, but a stale/half-ready gamescope (its socket created early in startup, or left behind // exists, but a stale/half-ready gamescope (its socket created early in startup, or left behind
@@ -206,7 +215,7 @@ async fn connect(source: EiSource) -> Result<Connected> {
anyhow!("EI handshake timed out (EIS server not responding — stale/half-ready socket?)") anyhow!("EI handshake timed out (EIS server not responding — stale/half-ready socket?)")
})? })?
.map_err(|e| anyhow!("EI handshake: {e}"))?; .map_err(|e| anyhow!("EI handshake: {e}"))?;
Ok((keepalive, context, events)) Ok((keepalive, context, events, output_hint))
} }
/// Open a RemoteDesktop portal session (pointer + keyboard) and obtain the EIS socket fd. /// Open a RemoteDesktop portal session (pointer + keyboard) and obtain the EIS socket fd.
@@ -294,8 +303,10 @@ async fn connect_mutter() -> Result<(Box<dyn Send>, std::os::fd::OwnedFd)> {
/// Poll `file` for the EIS socket path (the gamescope backend relays `LIBEI_SOCKET` there once /// Poll `file` for the EIS socket path (the gamescope backend relays `LIBEI_SOCKET` there once
/// the nested app launches), then connect. A bare name is resolved against `XDG_RUNTIME_DIR`, /// the nested app launches), then connect. A bare name is resolved against `XDG_RUNTIME_DIR`,
/// mirroring libei's own `LIBEI_SOCKET` semantics. /// mirroring libei's own `LIBEI_SOCKET` semantics. Line 2 of the relay file, when present,
async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> { /// carries the compositor's output size as `WxH` — returned as the absolute-coordinate scale
/// hint (gamescope's EIS region is degenerate, so the geometry can't come from the protocol).
async fn connect_socket_file(file: &std::path::Path) -> Result<(UnixStream, Option<(u32, u32)>)> {
// The relay file is rewritten each session with the CURRENT gamescope's `LIBEI_SOCKET`, and the // The relay file is rewritten each session with the CURRENT gamescope's `LIBEI_SOCKET`, and the
// socket may not be `listen()`ing the instant its name appears — or the file may briefly still // socket may not be `listen()`ing the instant its name appears — or the file may briefly still
// hold a prior, now-dead session's name (the host-lifetime injector reconnecting between // hold a prior, now-dead session's name (the host-lifetime injector reconnecting between
@@ -319,7 +330,12 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
)); ));
} }
if let Ok(s) = std::fs::read_to_string(file) { if let Ok(s) = std::fs::read_to_string(file) {
let name = s.trim(); let mut file_lines = s.lines();
let name = file_lines.next().unwrap_or("").trim();
let hint = file_lines.next().and_then(|l| {
let (w, h) = l.trim().split_once('x')?;
Some((w.parse::<u32>().ok()?, h.parse::<u32>().ok()?))
});
if !name.is_empty() { if !name.is_empty() {
let full = if name.starts_with('/') { let full = if name.starts_with('/') {
std::path::PathBuf::from(name) std::path::PathBuf::from(name)
@@ -334,7 +350,7 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
logged = name.to_string(); logged = name.to_string();
} }
match UnixStream::connect(&full) { match UnixStream::connect(&full) {
Ok(stream) => return Ok(stream), Ok(stream) => return Ok((stream, hint)),
// Refused = socket file exists but no listener yet (or a dead session); // Refused = socket file exists but no listener yet (or a dead session);
// NotFound = path not created yet. Both heal once the live gamescope's EIS is // NotFound = path not created yet. Both heal once the live gamescope's EIS is
// up — retry. Anything else (e.g. permission) is a real failure. // up — retry. Anything else (e.g. permission) is a real failure.
@@ -358,6 +374,25 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
} }
/// One EI device and its emulation state. /// One EI device and its emulation state.
/// Pick the region to map absolute coordinates into: the one whose logical size matches the
/// streamed mode (the session's virtual output). The device advertises one region per logical
/// monitor, and blindly taking `first()` next to a physical monitor put the pointer — and every
/// click — on whichever output the compositor happened to announce first (on-glass: GNOME with a
/// dummy HDMI beside the virtual primary; the seat cursor never entered the streamed monitor, so
/// neither embedded nor metadata cursor capture could see it). Size is the only key available
/// today: regions carry no output name, and matching the screencast stream's `mapping_id` needs
/// the stream id plumbed across crates (follow-up). Two same-sized monitors stay ambiguous.
fn region_for_mode(
regions: &[reis::event::Region],
w: f32,
h: f32,
) -> Option<&reis::event::Region> {
regions
.iter()
.find(|r| r.width as f32 == w && r.height as f32 == h)
.or_else(|| regions.first())
}
struct DeviceSlot { struct DeviceSlot {
device: reis::event::Device, device: reis::event::Device,
/// The device is resumed (allowed to emit). Devices arrive paused and may pause again. /// The device is resumed (allowed to emit). Devices arrive paused and may pause again.
@@ -386,6 +421,22 @@ struct EiState {
held_keys: Vec<u32>, held_keys: Vec<u32>,
held_buttons: Vec<u32>, held_buttons: Vec<u32>,
held_touches: Vec<u32>, held_touches: Vec<u32>,
/// The touch id currently degraded to the absolute pointer ([`EiState::degrade_touch`]) —
/// the "primary finger" while the EIS has no touchscreen device. `None` between touches.
degraded_touch: Option<u32>,
/// The compositor's output size (relay-file "WxH" hint) — scale target for absolute
/// coordinates when the device's region is degenerate/absent (gamescope). Without it the
/// fallback is raw client pixels, correct only when the stream runs at the output's size.
output_hint: Option<(u32, u32)>,
}
/// Is this EIS region a plausible OUTPUT geometry — something to map normalized coordinates
/// into? gamescope advertises a degenerate `(0,0,INT32_MAX,INT32_MAX)` "everything" region on
/// its virtual input device, meaning "absolute coordinates are raw"; normalizing into it
/// explodes a center tap to x≈1e9, which the compositor clamps to the far corner. 16384
/// comfortably covers real multi-monitor layouts while rejecting the sentinel.
fn sane_region(r: &reis::event::Region) -> bool {
r.width > 0 && r.height > 0 && r.width <= 16_384 && r.height <= 16_384
} }
/// Stable small index per [`InputKind`] for the `seen_kinds` bitmask. /// Stable small index per [`InputKind`] for the `seen_kinds` bitmask.
@@ -406,6 +457,7 @@ fn kind_bit(kind: InputKind) -> u32 {
InputKind::GamepadState => 12, InputKind::GamepadState => 12,
InputKind::GamepadRemove => 13, InputKind::GamepadRemove => 13,
InputKind::GamepadArrival => 14, InputKind::GamepadArrival => 14,
InputKind::TextInput => 15,
}; };
1 << i 1 << i
} }
@@ -422,6 +474,8 @@ impl EiState {
held_keys: Vec::new(), held_keys: Vec::new(),
held_buttons: Vec::new(), held_buttons: Vec::new(),
held_touches: Vec::new(), held_touches: Vec::new(),
degraded_touch: None,
output_hint: None,
} }
} }
@@ -430,6 +484,10 @@ impl EiState {
/// normal [`EiState::inject`] path so the compositor sees proper key-up / button-up / /// normal [`EiState::inject`] path so the compositor sees proper key-up / button-up /
/// touch-up frames before the devices disappear. /// touch-up frames before the devices disappear.
fn release_all(&mut self, ctx: &ei::Context) { fn release_all(&mut self, ctx: &ei::Context) {
// A degraded touch is held as a synthesized left button (in `held_buttons`), so the
// loop below releases it — but the primary-finger latch must clear too, or the NEXT
// session's first TouchDown reads as a second finger and is ignored.
self.degraded_touch = None;
let (keys, buttons, touches) = ( let (keys, buttons, touches) = (
std::mem::take(&mut self.held_keys), std::mem::take(&mut self.held_keys),
std::mem::take(&mut self.held_buttons), std::mem::take(&mut self.held_buttons),
@@ -506,6 +564,14 @@ impl EiState {
keyboard = dev.has_capability(DeviceCapability::Keyboard), keyboard = dev.has_capability(DeviceCapability::Keyboard),
button = dev.has_capability(DeviceCapability::Button), button = dev.has_capability(DeviceCapability::Button),
scroll = dev.has_capability(DeviceCapability::Scroll), scroll = dev.has_capability(DeviceCapability::Scroll),
// One region per logical monitor — which one absolute coords map into is
// resolved per event (`region_for_mode`); log them so a mis-mapped pointer
// (cursor/clicks on the wrong output) is diagnosable from the journal.
regions = ?dev
.regions()
.iter()
.map(|r| format!("{}x{}+{}+{}", r.width, r.height, r.x, r.y))
.collect::<Vec<_>>(),
"libei: device RESUMED (now emittable)" "libei: device RESUMED (now emittable)"
); );
} }
@@ -537,8 +603,85 @@ impl EiState {
} }
} }
/// Degrade touch to a single-finger ABSOLUTE POINTER on compositors whose EIS never
/// creates a touchscreen device (gamescope's "Gamescope Virtual Input" advertises
/// pointer/pointer_abs/button but no touch — observed live; headless KWin the same).
/// Down = abs-move + left press, move = abs-move, up = left release — synthesized through
/// the normal [`EiState::inject`] Mouse* paths so region mapping, held-state tracking and
/// [`EiState::release_all`] all apply. Only the FIRST finger drives the pointer; later
/// fingers are ignored (a pointer has no second contact — a pinch degrades to a drag).
fn degrade_touch(&mut self, ev: &InputEvent, ctx: &ei::Context) {
const GS_BUTTON_LEFT: u32 = 1;
match ev.kind {
InputKind::TouchDown => {
if self.degraded_touch.is_some() {
return; // secondary finger — single-pointer degradation
}
self.degraded_touch = Some(ev.code);
static NOTED: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(false);
if !NOTED.swap(true, std::sync::atomic::Ordering::Relaxed) {
tracing::info!(
"compositor's EIS has no touchscreen device — degrading touch to a \
single-finger absolute pointer (tap = left click; multi-touch \
gestures unavailable)"
);
}
self.inject(
&InputEvent {
kind: InputKind::MouseMoveAbs,
..*ev
},
ctx,
);
self.inject(
&InputEvent {
kind: InputKind::MouseButtonDown,
code: GS_BUTTON_LEFT,
..*ev
},
ctx,
);
}
InputKind::TouchMove if self.degraded_touch == Some(ev.code) => {
self.inject(
&InputEvent {
kind: InputKind::MouseMoveAbs,
..*ev
},
ctx,
);
}
InputKind::TouchUp if self.degraded_touch == Some(ev.code) => {
self.degraded_touch = None;
self.inject(
&InputEvent {
kind: InputKind::MouseButtonUp,
code: GS_BUTTON_LEFT,
..*ev
},
ctx,
);
}
_ => {}
}
}
/// Translate and emit one client input event, committing it as a single `frame`. /// Translate and emit one client input event, committing it as a single `frame`.
fn inject(&mut self, ev: &InputEvent, ctx: &ei::Context) { fn inject(&mut self, ev: &InputEvent, ctx: &ei::Context) {
// No ei_touchscreen device but an absolute pointer exists → degrade rather than drop
// (the game-mode "touch simply does not work" trap). Checked per event, not latched:
// a touchscreen device appearing later (compositor restart, capability change) takes
// over seamlessly on the next touch.
if matches!(
ev.kind,
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp
) && self.device_for(DeviceCapability::Touch).is_none()
&& self.device_for(DeviceCapability::PointerAbsolute).is_some()
{
self.degrade_touch(ev, ctx);
return;
}
let cap = match ev.kind { let cap = match ev.kind {
InputKind::MouseMove => DeviceCapability::Pointer, InputKind::MouseMove => DeviceCapability::Pointer,
InputKind::MouseMoveAbs => DeviceCapability::PointerAbsolute, InputKind::MouseMoveAbs => DeviceCapability::PointerAbsolute,
@@ -553,6 +696,9 @@ impl EiState {
| InputKind::GamepadAxis | InputKind::GamepadAxis
| InputKind::GamepadRemove | InputKind::GamepadRemove
| InputKind::GamepadArrival => return, // uinput path (later) | InputKind::GamepadArrival => return, // uinput path (later)
// libei presses keycodes against the server's negotiated keymap — no committed-text
// path (the HOST_CAP_TEXT_INPUT cap is not advertised on this backend).
InputKind::TextInput => return,
}; };
self.injected += 1; self.injected += 1;
let n = self.injected; let n = self.injected;
@@ -605,16 +751,33 @@ impl EiState {
InputKind::MouseMoveAbs => { InputKind::MouseMoveAbs => {
let w = ((ev.flags >> 16) & 0xffff) as f32; let w = ((ev.flags >> 16) & 0xffff) as f32;
let h = (ev.flags & 0xffff) as f32; let h = (ev.flags & 0xffff) as f32;
match ( match slot.interface::<ei::PointerAbsolute>() {
slot.interface::<ei::PointerAbsolute>(), Some(p) if w > 0.0 && h > 0.0 => {
slot.regions().first(), // Map the normalized client position into the region matching the streamed
) { // output's mode (`region_for_mode` picks the right one on a multi-monitor
(Some(p), Some(region)) if w > 0.0 && h > 0.0 => { // EIS). gamescope's "Gamescope Virtual Input" advertises a degenerate
// Map the normalized client position into the device's first region. // (0,0,INT32_MAX,INT32_MAX) region meaning "coordinates are raw" —
// `sane_region` rejects it (normalizing into it explodes a center tap to
// x≈1e9, clamped to the far corner), so a non-matching / insane region
// falls to the output hint (correct across a resolution mismatch), then
// raw client pixels as the last resort.
let nx = (ev.x as f32 / w).clamp(0.0, 1.0); let nx = (ev.x as f32 / w).clamp(0.0, 1.0);
let ny = (ev.y as f32 / h).clamp(0.0, 1.0); let ny = (ev.y as f32 / h).clamp(0.0, 1.0);
let x = region.x as f32 + nx * region.width as f32; let (x, y) = match region_for_mode(slot.regions(), w, h)
let y = region.y as f32 + ny * region.height as f32; .filter(|r| sane_region(r))
{
Some(region) => (
region.x as f32 + nx * region.width as f32,
region.y as f32 + ny * region.height as f32,
),
// Degenerate/absent region: scale into the relay-file output hint
// (correct even when the client streams at a different resolution
// than the session runs); raw client pixels as the last resort.
None => match self.output_hint {
Some((ow, oh)) => (nx * ow as f32, ny * oh as f32),
None => (ev.x as f32, ev.y as f32),
},
};
p.motion_absolute(x, y); p.motion_absolute(x, y);
} }
_ => emitted = false, _ => emitted = false,
@@ -680,12 +843,23 @@ impl EiState {
InputKind::TouchDown | InputKind::TouchMove => { InputKind::TouchDown | InputKind::TouchMove => {
let w = ((ev.flags >> 16) & 0xffff) as f32; let w = ((ev.flags >> 16) & 0xffff) as f32;
let h = (ev.flags & 0xffff) as f32; let h = (ev.flags & 0xffff) as f32;
match (slot.interface::<ei::Touchscreen>(), slot.regions().first()) { match slot.interface::<ei::Touchscreen>() {
(Some(t), Some(region)) if w > 0.0 && h > 0.0 => { Some(t) if w > 0.0 && h > 0.0 => {
let nx = (ev.x as f32 / w).clamp(0.0, 1.0); let nx = (ev.x as f32 / w).clamp(0.0, 1.0);
let ny = (ev.y as f32 / h).clamp(0.0, 1.0); let ny = (ev.y as f32 / h).clamp(0.0, 1.0);
let x = region.x as f32 + nx * region.width as f32; // Same region-selection + degenerate fallback ladder as MouseMoveAbs.
let y = region.y as f32 + ny * region.height as f32; let (x, y) = match region_for_mode(slot.regions(), w, h)
.filter(|r| sane_region(r))
{
Some(region) => (
region.x as f32 + nx * region.width as f32,
region.y as f32 + ny * region.height as f32,
),
None => match self.output_hint {
Some((ow, oh)) => (nx * ow as f32, ny * oh as f32),
None => (ev.x as f32, ev.y as f32),
},
};
if ev.kind == InputKind::TouchDown { if ev.kind == InputKind::TouchDown {
t.down(ev.code, x, y); t.down(ev.code, x, y);
} else { } else {
@@ -703,7 +877,8 @@ impl EiState {
| InputKind::GamepadButton | InputKind::GamepadButton
| InputKind::GamepadAxis | InputKind::GamepadAxis
| InputKind::GamepadRemove | InputKind::GamepadRemove
| InputKind::GamepadArrival => emitted = false, | InputKind::GamepadArrival
| InputKind::TextInput => emitted = false,
} }
if emitted { if emitted {
+396
View File
@@ -0,0 +1,396 @@
//! Virtual tablet ("Punktfunk Pen"): a uinput stylus device carrying the pen plane's full
//! fidelity — pressure, tilt, barrel roll, hover distance, eraser, barrel buttons
//! (design/pen-tablet-input.md §5).
//!
//! Deliberately a **uinput device, not a compositor-protocol citizen**: no virtual-tablet
//! protocol exists in EI, the RemoteDesktop portal, KWin `fake_input`, or wlroots — while
//! every compositor consumes evdev tablets via libinput and forwards them to apps over
//! `zwp_tablet_v2` with nothing to configure. udev's `input_id` builtin classifies the device
//! from its capabilities (`BTN_TOOL_PEN` + `ABS_X/Y` ⇒ `ID_INPUT_TABLET`), so Krita/GIMP/
//! Xournal++ see a real pen. Output mapping is the compositor's own tablet-mapping default
//! (single output ⇒ correct; multi-monitor pinning is the documented follow-up, which is why
//! the device carries a stable, distinctive identity to key rules on).
//!
//! The consumer feeds it [`PenTransition`]s straight from the core's
//! [`PenTracker`](punktfunk_core::quic::PenTracker); this file only translates transitions to
//! evdev events and groups them into SYN frames so proximity-enter carries its position in the
//! same frame (libinput would otherwise report an entry at a stale point).
//!
//! ioctl numbers/struct layouts mirror `gamepad.rs` (verified against the same kernel
//! generation); each backend file stays self-contained by convention.
use anyhow::{bail, Result};
use punktfunk_core::quic::{PenSample, PenTool, PenTransition, PEN_BARREL1, PEN_BARREL2};
use std::os::fd::{AsRawFd, OwnedFd};
// ioctls (x86_64).
const UI_DEV_CREATE: libc::c_ulong = 0x5501;
const UI_DEV_DESTROY: libc::c_ulong = 0x5502;
const UI_DEV_SETUP: libc::c_ulong = 0x405c_5503;
const UI_ABS_SETUP: libc::c_ulong = 0x401c_5504;
const UI_SET_EVBIT: libc::c_ulong = 0x4004_5564;
const UI_SET_KEYBIT: libc::c_ulong = 0x4004_5565;
const UI_SET_PROPBIT: libc::c_ulong = 0x4004_556e;
// input-event-codes.h subset.
const EV_SYN: u16 = 0x00;
const EV_KEY: u16 = 0x01;
const EV_ABS: u16 = 0x03;
const SYN_REPORT: u16 = 0;
const ABS_X: u16 = 0x00;
const ABS_Y: u16 = 0x01;
/// Barrel roll rides ABS_Z (the Wacom Art-Pen rotation convention); libinput normalizes the
/// declared min..max onto its 0..360° rotation axis.
const ABS_Z: u16 = 0x02;
const ABS_PRESSURE: u16 = 0x18;
const ABS_DISTANCE: u16 = 0x19;
const ABS_TILT_X: u16 = 0x1a;
const ABS_TILT_Y: u16 = 0x1b;
const BTN_TOOL_PEN: u16 = 0x140;
const BTN_TOOL_RUBBER: u16 = 0x141;
const BTN_TOUCH: u16 = 0x14a;
const BTN_STYLUS: u16 = 0x14b;
const BTN_STYLUS2: u16 = 0x14c;
/// The pen writes on the display it is mapped to (a "screen tablet"), not a desk pad —
/// libinput then maps the full ABS range onto the output rect, exactly the wire's normalized
/// coordinate contract.
const INPUT_PROP_DIRECT: libc::c_int = 0x01;
/// Full-scale wire pressure (u16) → the declared 0..4095 axis.
const PRESSURE_SHIFT: u32 = 4;
/// Wire hover distance (u16, 0xFFFF = unknown) → the declared 0..1023 axis.
const DISTANCE_SHIFT: u32 = 6;
const ABS_RANGE: f32 = 65535.0;
#[repr(C)]
struct InputId {
bustype: u16,
vendor: u16,
product: u16,
version: u16,
}
#[repr(C)]
struct UinputSetup {
id: InputId,
name: [u8; 80],
ff_effects_max: u32,
}
#[repr(C)]
#[derive(Default, Clone, Copy)]
struct AbsInfo {
value: i32,
minimum: i32,
maximum: i32,
fuzz: i32,
flat: i32,
resolution: i32,
}
#[repr(C)]
struct UinputAbsSetup {
code: u16,
_pad: u16,
absinfo: AbsInfo,
}
#[repr(C)]
#[derive(Clone, Copy)]
struct InputEventRaw {
time: libc::timeval,
type_: u16,
code: u16,
value: i32,
}
fn ioctl_int(fd: i32, req: libc::c_ulong, arg: libc::c_int, what: &str) -> Result<()> {
// SAFETY: every caller passes a UI_SET_*/UI_DEV_* request whose argument the kernel reads
// as a plain int; `fd` is a live uinput fd owned by the caller. No memory is handed over.
if unsafe { libc::ioctl(fd, req, arg) } < 0 {
bail!("{what}: {}", std::io::Error::last_os_error());
}
Ok(())
}
fn ioctl_ptr<T>(fd: i32, req: libc::c_ulong, arg: *mut T, what: &str) -> Result<()> {
// SAFETY: every caller passes a pointer to a live, initialized `#[repr(C)]` struct matching
// the request's expected layout (UI_DEV_SETUP/UI_ABS_SETUP); the kernel reads it during the
// call and retains nothing.
if unsafe { libc::ioctl(fd, req, arg) } < 0 {
bail!("{what}: {}", std::io::Error::last_os_error());
}
Ok(())
}
/// The active tool's evdev key, one in proximity at a time (Wacom semantics — the core's
/// [`PenTracker`](punktfunk_core::quic::PenTracker) already re-enters proximity on a tool
/// switch, so this only tracks which key to release on `ProximityOut`).
fn tool_key(tool: PenTool) -> u16 {
match tool {
PenTool::Eraser => BTN_TOOL_RUBBER,
// Unknown = a newer client's future tool — nearest ink-capable behavior is the pen.
PenTool::Pen | PenTool::Unknown => BTN_TOOL_PEN,
}
}
/// One per-session virtual tablet. Created lazily on the first pen batch (a session that never
/// draws never creates a device), destroyed with the session (Drop → `UI_DEV_DESTROY`).
pub struct VirtualPen {
fd: OwnedFd,
/// The tool key currently held in proximity (release target for `ProximityOut`).
tool: u16,
/// Whether the current SYN frame already carries a Motion — the frame-split trigger for
/// consecutive samples in one batch.
frame_has_motion: bool,
/// Whether the current SYN frame has any unflushed events.
frame_dirty: bool,
}
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
use std::os::fd::FromRawFd;
// SAFETY: `c"/dev/uinput"` is a 'static NUL-terminated C string literal; `open` reads it
// as a path, returns a fresh fd (or -1) and retains nothing.
let raw = unsafe {
libc::open(
c"/dev/uinput".as_ptr(),
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if raw < 0 {
bail!(
"open /dev/uinput: {} (install the udev rule granting the 'input' group access \
see scripts/60-punktfunk.rules and add the user to the 'input' group)",
std::io::Error::last_os_error()
);
}
// SAFETY: `raw >= 0` here, a freshly-opened fd owned nowhere else; `OwnedFd` becomes the
// unique owner and closes it exactly once on drop.
let fd = unsafe { OwnedFd::from_raw_fd(raw) };
ioctl_int(raw, UI_SET_EVBIT, EV_KEY as i32, "UI_SET_EVBIT(EV_KEY)")?;
ioctl_int(raw, UI_SET_EVBIT, EV_ABS as i32, "UI_SET_EVBIT(EV_ABS)")?;
for key in [
BTN_TOOL_PEN,
BTN_TOOL_RUBBER,
BTN_TOUCH,
BTN_STYLUS,
BTN_STYLUS2,
] {
ioctl_int(raw, UI_SET_KEYBIT, key as i32, "UI_SET_KEYBIT")?;
}
ioctl_int(
raw,
UI_SET_PROPBIT,
INPUT_PROP_DIRECT,
"UI_SET_PROPBIT(DIRECT)",
)?;
// Position spans the full u16 range; `resolution` (units/mm) only feeds libinput's mm
// math (nothing pen-relevant), but tablets without one trip its missing-resolution
// fixup — 100 declares a plausible ~655 mm drawing surface.
let pos = AbsInfo {
minimum: 0,
maximum: 65535,
resolution: 100,
..Default::default()
};
// Tilt in degrees from vertical, per evdev convention; resolution = units/radian (57
// ⇔ 1 unit = 1°, what the Wacom driver declares).
let tilt = AbsInfo {
minimum: -90,
maximum: 90,
resolution: 57,
..Default::default()
};
for (code, info) in [
(ABS_X, pos),
(ABS_Y, pos),
(
ABS_PRESSURE,
AbsInfo {
minimum: 0,
maximum: 4095,
..Default::default()
},
),
(
ABS_DISTANCE,
AbsInfo {
minimum: 0,
maximum: 1023,
..Default::default()
},
),
(ABS_TILT_X, tilt),
(ABS_TILT_Y, tilt),
(
// Barrel roll: libinput maps the declared range linearly onto 0..360°.
ABS_Z,
AbsInfo {
minimum: 0,
maximum: 359,
..Default::default()
},
),
] {
let mut a = UinputAbsSetup {
code,
_pad: 0,
absinfo: info,
};
ioctl_ptr(raw, UI_ABS_SETUP, &mut a, "UI_ABS_SETUP")?;
}
// A stable, distinctive identity (pid.codes open-source VID) so compositor
// tablet-mapping rules — GNOME's per-`vendor:product` gsettings path, sway
// `map_to_output` — can target exactly this device.
let mut setup = UinputSetup {
id: InputId {
bustype: 0x0006, // BUS_VIRTUAL
vendor: 0x1209,
product: 0x5046, // "PF"
version: 1,
},
name: [0; 80],
ff_effects_max: 0,
};
let name = b"Punktfunk Pen";
setup.name[..name.len()].copy_from_slice(name);
ioctl_ptr(raw, UI_DEV_SETUP, &mut setup, "UI_DEV_SETUP")?;
ioctl_int(raw, UI_DEV_CREATE, 0, "UI_DEV_CREATE")?;
tracing::info!("virtual tablet created (Punktfunk Pen, uinput)");
Ok(VirtualPen {
fd,
tool: BTN_TOOL_PEN,
frame_has_motion: false,
frame_dirty: false,
})
}
fn emit(&self, type_: u16, code: u16, value: i32) {
let ev = InputEventRaw {
time: libc::timeval {
tv_sec: 0,
tv_usec: 0,
},
type_,
code,
value,
};
// SAFETY: `ev` is a live local `#[repr(C)]` all-integer struct (no padding: timeval=16 +
// u16 + u16 + i32 = 24), so every byte is initialized; the slice spans exactly `ev`'s
// bytes and is used immediately below with no concurrent mutation.
let bytes = unsafe {
std::slice::from_raw_parts(
&ev as *const _ as *const u8,
std::mem::size_of::<InputEventRaw>(),
)
};
// Best-effort like the gamepad path: a full kernel queue drops the event; pen samples
// are state-full, so the next frame re-syncs axes (and the tracker re-syncs state).
// SAFETY: `self.fd` stays open for the synchronous call; `write` only reads
// `bytes.len()` bytes from the still-live local and retains nothing.
let _ = unsafe {
libc::write(
self.fd.as_raw_fd(),
bytes.as_ptr() as *const libc::c_void,
bytes.len(),
)
};
}
fn flush(&mut self) {
if self.frame_dirty {
self.emit(EV_SYN, SYN_REPORT, 0);
self.frame_dirty = false;
self.frame_has_motion = false;
}
}
fn motion(&mut self, s: &PenSample) {
self.emit(EV_ABS, ABS_X, (s.x * ABS_RANGE) as i32);
self.emit(EV_ABS, ABS_Y, (s.y * ABS_RANGE) as i32);
self.emit(EV_ABS, ABS_PRESSURE, (s.pressure >> PRESSURE_SHIFT) as i32);
if s.distance != punktfunk_core::quic::PEN_DISTANCE_UNKNOWN {
self.emit(EV_ABS, ABS_DISTANCE, (s.distance >> DISTANCE_SHIFT) as i32);
}
// Polar → tiltX/tiltY needs both angles; azimuth clockwise from north, so east (90°)
// tilts +X and south (180°, toward the user) tilts +Y — the evdev/W3C signs.
if s.tilt_deg != punktfunk_core::quic::PEN_TILT_UNKNOWN
&& s.azimuth_deg != punktfunk_core::quic::PEN_ANGLE_UNKNOWN
{
let az = (s.azimuth_deg as f32).to_radians();
let tilt = s.tilt_deg as f32;
self.emit(EV_ABS, ABS_TILT_X, (tilt * az.sin()).round() as i32);
self.emit(EV_ABS, ABS_TILT_Y, (-tilt * az.cos()).round() as i32);
}
if s.roll_deg != punktfunk_core::quic::PEN_ANGLE_UNKNOWN {
self.emit(EV_ABS, ABS_Z, (s.roll_deg % 360) as i32);
}
self.frame_dirty = true;
self.frame_has_motion = true;
}
/// Apply one decoded batch's transitions (the core tracker's output, in its documented
/// order), grouping them into SYN frames: a frame closes before a `ProximityIn` (an entry
/// is a new instant — and must carry its own position, not inherit the stale frame) and
/// before a second `Motion` (consecutive samples are consecutive instants), plus a final
/// close. So `[ProxIn, Motion, TipDown]` lands as ONE frame — libinput reports the entry
/// already at the right point with contact — while a drag batch's `[Motion, Motion]`
/// stays two.
pub fn apply_batch(&mut self, transitions: &[PenTransition]) {
for t in transitions {
match t {
PenTransition::ProximityIn { tool } => {
self.flush();
self.tool = tool_key(*tool);
self.emit(EV_KEY, self.tool, 1);
self.frame_dirty = true;
}
PenTransition::Motion { sample } => {
if self.frame_has_motion {
self.flush();
}
self.motion(sample);
}
PenTransition::TipDown => {
self.emit(EV_KEY, BTN_TOUCH, 1);
self.frame_dirty = true;
}
PenTransition::ButtonsChanged { pressed, released } => {
for (bit, key) in [(PEN_BARREL1, BTN_STYLUS), (PEN_BARREL2, BTN_STYLUS2)] {
if pressed & bit != 0 {
self.emit(EV_KEY, key, 1);
self.frame_dirty = true;
}
if released & bit != 0 {
self.emit(EV_KEY, key, 0);
self.frame_dirty = true;
}
}
}
PenTransition::TipUp => {
self.emit(EV_KEY, BTN_TOUCH, 0);
self.emit(EV_ABS, ABS_PRESSURE, 0);
self.frame_dirty = true;
}
PenTransition::ProximityOut => {
self.emit(EV_KEY, self.tool, 0);
self.frame_dirty = true;
}
}
}
self.flush();
}
}
impl Drop for VirtualPen {
fn drop(&mut self) {
// SAFETY: `self.fd` is still open (OwnedFd closes only after this body returns);
// UI_DEV_DESTROY takes no pointer argument. Errors are moot on teardown.
let _ = unsafe { libc::ioctl(self.fd.as_raw_fd(), UI_DEV_DESTROY, 0) };
}
}
+96
View File
@@ -98,9 +98,26 @@ pub struct WlrootsInjector {
keyboard: ZwpVirtualKeyboardV1, keyboard: ZwpVirtualKeyboardV1,
xkb_state: xkb::State, xkb_state: xkb::State,
_keymap_file: std::fs::File, // keep the memfd alive for the compositor's mmap _keymap_file: std::fs::File, // keep the memfd alive for the compositor's mmap
/// Dedicated committed-text device ([`InputKind::TextInput`]), created on first use.
text: Option<TextKeyboard>,
start: Instant, start: Instant,
} }
/// Cap on distinct characters the dynamic text keymap holds before it restarts from scratch
/// (keycodes grow upward from 9; xkb tops out at 255, so stay well under).
const TEXT_KEYMAP_MAX: usize = 200;
/// The dedicated **text** virtual keyboard: types committed IME text (`InputKind::TextInput`,
/// one Unicode scalar per event) by growing a keymap of Unicode keysyms on demand and pressing
/// the character's keycode — the `wtype` model. A separate `zwp_virtual_keyboard` so keymap
/// re-uploads never disturb the main device's layout/modifier state that VK key events ride on.
struct TextKeyboard {
keyboard: ZwpVirtualKeyboardV1,
/// Characters in keycode order: `chars[i]` types on wire keycode `i + 1` (xkb `i + 9`).
chars: Vec<char>,
_keymap_file: Option<std::fs::File>, // keep the memfd alive for the compositor's mmap
}
impl WlrootsInjector { impl WlrootsInjector {
pub fn open() -> Result<Self> { pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env() let conn = Connection::connect_to_env()
@@ -171,6 +188,7 @@ impl WlrootsInjector {
keyboard, keyboard,
xkb_state, xkb_state,
_keymap_file: file, _keymap_file: file,
text: None,
start: Instant::now(), start: Instant::now(),
}) })
} }
@@ -179,6 +197,54 @@ impl WlrootsInjector {
self.start.elapsed().as_millis() as u32 self.start.elapsed().as_millis() as u32
} }
/// Type one committed-text Unicode scalar on the dedicated text device (created lazily),
/// growing its keymap when the character is new. Control characters are dropped — Enter,
/// Backspace and Tab ride the VK key-event path.
fn type_text(&mut self, cp: u32) -> Result<()> {
let Some(ch) = char::from_u32(cp) else {
return Ok(()); // lone surrogate / out of range
};
if ch.is_control() {
return Ok(());
}
if self.text.is_none() {
let (Some(mgr), Some(seat)) =
(self.globals.keyboard_mgr.clone(), self.globals.seat.clone())
else {
return Ok(());
};
let kb = mgr.create_virtual_keyboard(&seat, &self.queue.handle(), ());
self.text = Some(TextKeyboard {
keyboard: kb,
chars: Vec::new(),
_keymap_file: None,
});
}
let t = self.now_ms();
let text = self.text.as_mut().expect("created above");
let code = match text.chars.iter().position(|&c| c == ch) {
Some(i) => (i + 1) as u32,
None => {
if text.chars.len() >= TEXT_KEYMAP_MAX {
text.chars.clear(); // restart the map; old codes are re-assigned lazily
}
text.chars.push(ch);
let keymap_str = text_keymap(&text.chars);
let file = memfd_with(&keymap_str)?;
text.keyboard.keymap(
1, /* XKB_V1 */
file.as_fd(),
keymap_str.len() as u32 + 1,
);
text._keymap_file = Some(file);
text.chars.len() as u32
}
};
text.keyboard.key(t, code, 1);
text.keyboard.key(t, code, 0);
Ok(())
}
/// Update xkb state for a key and tell the compositor the resulting modifier mask. /// Update xkb state for a key and tell the compositor the resulting modifier mask.
fn send_modifiers(&mut self, evdev: u16, down: bool) { fn send_modifiers(&mut self, evdev: u16, down: bool) {
let kc = xkb::Keycode::new(evdev as u32 + 8); // evdev -> xkb keycode let kc = xkb::Keycode::new(evdev as u32 + 8); // evdev -> xkb keycode
@@ -254,6 +320,9 @@ impl InputInjector for WlrootsInjector {
tracing::debug!(vk = event.code, "unmapped VK keycode — dropped"); tracing::debug!(vk = event.code, "unmapped VK keycode — dropped");
} }
} }
InputKind::TextInput => {
self.type_text(event.code)?;
}
InputKind::GamepadState InputKind::GamepadState
| InputKind::GamepadButton | InputKind::GamepadButton
| InputKind::GamepadAxis | InputKind::GamepadAxis
@@ -271,6 +340,33 @@ impl InputInjector for WlrootsInjector {
} }
} }
/// Build a minimal xkb keymap whose keycode `i + 9` (wire code `i + 1`) types `chars[i]`, using
/// Unicode keysym names (`U<hex>` — xkbcommon resolves them for any scalar, emoji included).
/// Types/compat `include "complete"` mirrors `wtype`'s generated keymap — proven on wlroots
/// compositors, and the system XKB data is present (the main keymap compiled from it in `open`).
fn text_keymap(chars: &[char]) -> String {
use std::fmt::Write as _;
let mut keycodes = String::new();
let mut symbols = String::new();
for (i, ch) in chars.iter().enumerate() {
let _ = writeln!(keycodes, " <T{i}> = {};", i + 9);
let _ = writeln!(symbols, " key <T{i}> {{ [ U{:04X} ] }};", *ch as u32);
}
format!(
"xkb_keymap {{\n\
xkb_keycodes \"punktfunk-text\" {{\n\
minimum = 8;\n\
maximum = {};\n\
{keycodes}\
}};\n\
xkb_types \"punktfunk-text\" {{ include \"complete\" }};\n\
xkb_compatibility \"punktfunk-text\" {{ include \"complete\" }};\n\
xkb_symbols \"punktfunk-text\" {{\n{symbols} }};\n\
}};\n",
chars.len() + 9,
)
}
/// Create an anonymous in-memory file holding `s` + a trailing NUL (for the keymap fd). /// Create an anonymous in-memory file holding `s` + a trailing NUL (for the keymap fd).
fn memfd_with(s: &str) -> Result<std::fs::File> { fn memfd_with(s: &str) -> Result<std::fs::File> {
let name = b"punktfunk-keymap\0"; let name = b"punktfunk-keymap\0";
@@ -0,0 +1,659 @@
//! Windows synthetic-pointer injection (design/pen-tablet-input.md §6): a per-session `PT_PEN`
//! device carrying the pen plane's full fidelity — pressure (rescaled to Windows' 0..1024),
//! polar tilt → tiltX/tiltY, barrel roll on `rotation` (0..359 — Windows Ink renders Pencil
//! Pro roll natively), barrel button, eraser (`PEN_FLAG_INVERTED`/`ERASER`), hover — plus a
//! `PT_TOUCH` device that closes the long-standing SendInput touch no-op for wire touches.
//!
//! Both follow Apollo's proven recipe (`design/apollo-comparison.md`): synthetic pointer state
//! goes STALE if not re-injected (~100 ms), so each device runs a small refresh thread that
//! re-asserts the last frame every [`REFRESH_MS`] while the pen is in range / contacts are
//! held — a stationary stylus must not hover-out (and a held finger must not auto-lift) just
//! because no new samples arrived. `CreateSyntheticPointerDevice` needs Win10 1809+;
//! [`crate::pen_supported`] probes it, so older hosts simply never advertise pen.
//!
//! Frame grouping mirrors the Linux uinput backend: a proximity-enter is injected together
//! with its position (never at a stale point), tip edges get their own DOWN/UP frames, and a
//! range-leave is a final frame without `INRANGE`.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{Context, Result};
use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{
PenSample, PenTool, PenTransition, PEN_ANGLE_UNKNOWN, PEN_BARREL1, PEN_TILT_UNKNOWN,
};
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use windows::Win32::Foundation::POINT;
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, GetThreadDesktop, OpenInputDesktop, SetThreadDesktop, DESKTOP_ACCESS_FLAGS,
DESKTOP_CONTROL_FLAGS, HDESK,
};
use windows::Win32::System::Threading::GetCurrentThreadId;
use windows::Win32::UI::Controls::{
CreateSyntheticPointerDevice, DestroySyntheticPointerDevice, HSYNTHETICPOINTERDEVICE,
POINTER_FEEDBACK_DEFAULT, POINTER_TYPE_INFO, POINTER_TYPE_INFO_0,
};
use windows::Win32::UI::Input::Pointer::{
InjectSyntheticPointerInput, POINTER_FLAGS, POINTER_FLAG_DOWN, POINTER_FLAG_FIRSTBUTTON,
POINTER_FLAG_INCONTACT, POINTER_FLAG_INRANGE, POINTER_FLAG_NEW, POINTER_FLAG_UP,
POINTER_FLAG_UPDATE, POINTER_INFO, POINTER_PEN_INFO, POINTER_TOUCH_INFO,
};
use windows::Win32::UI::WindowsAndMessaging::{
PEN_FLAG_BARREL, PEN_FLAG_ERASER, PEN_FLAG_INVERTED, PEN_FLAG_NONE, PEN_MASK_PRESSURE,
PEN_MASK_ROTATION, PEN_MASK_TILT_X, PEN_MASK_TILT_Y, PT_PEN, PT_TOUCH, TOUCH_FLAG_NONE,
TOUCH_MASK_NONE,
};
/// Re-inject cadence while state is held (in-range pen / touching contacts). Apollo uses
/// 50 ms against the ~100 ms staleness window; 40 ms keeps two refreshes inside it.
const REFRESH_MS: u64 = 40;
/// `GENERIC_ALL` for the desktop open — the `windows` crate models desktop rights as their own
/// flag type and doesn't re-export the generic ones (same constant `sendinput.rs` uses).
const DESKTOP_GENERIC_ALL: u32 = 0x1000_0000;
/// This thread's binding to the input desktop, restored on drop.
///
/// `SendInput` (`sendinput.rs`) solves the same problem by binding its DEDICATED injector thread
/// once and keeping it. That can't be borrowed here: `inject_pen`/`inject_touch_frame` are driven
/// from TWO threads — the caller's `apply_batch` and the `pf-pen-refresh`/`pf-touch-refresh`
/// staleness threads — and the batch caller is a shared task thread, which must not be left parked
/// on a `Winlogon` desktop that disappears when the prompt is dismissed. So the binding is scoped
/// to the retry: `GetThreadDesktop` hands back a BORROWED handle (never closed), and only the
/// `OpenInputDesktop` handle is closed, after the thread has moved back off it.
struct InputDesktopBinding {
previous: HDESK,
input: HDESK,
}
impl InputDesktopBinding {
fn bind() -> Option<Self> {
// SAFETY: FFI calls taking by-value args only. `OpenInputDesktop` yields an owned `HDESK`
// solely on `Ok`; it is either installed by `SetThreadDesktop` (then owned by this guard,
// closed exactly once in `Drop`) or closed here on failure — closed once on every path,
// never used after. `SetThreadDesktop` rebinds only the calling thread, which owns no
// windows or hooks, so it cannot fail on that account.
unsafe {
let previous = GetThreadDesktop(GetCurrentThreadId()).ok()?;
let input = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(DESKTOP_GENERIC_ALL),
)
.ok()?;
if SetThreadDesktop(input).is_err() {
let _ = CloseDesktop(input);
return None;
}
Some(Self { previous, input })
}
}
}
impl Drop for InputDesktopBinding {
fn drop(&mut self) {
// SAFETY: `previous` is the borrowed desktop this thread started on, `input` the handle
// this guard uniquely owns. The thread moves back FIRST, so the handle is not the thread's
// desktop when closed — closed exactly once, never used after.
unsafe {
let _ = SetThreadDesktop(self.previous);
let _ = CloseDesktop(self.input);
}
}
}
/// Inject one synthetic-pointer frame, following the input desktop if Windows refuses it.
///
/// While a UAC consent prompt (or the lock / logon screen) owns input, the secure desktop does, and
/// injection from the host's own `WinSta0\Default` thread comes back `ERROR_ACCESS_DENIED` — which
/// is why pen and touch went dead on a prompt a user could SEE in the stream (capture already
/// renders it, and `SendInput` mouse/keyboard already followed the switch, so only these two were
/// left behind). Field-reported 2026-07-23; the exact rc reproduced in a probe the same night.
///
/// Measured then, with a real consent prompt up and a SYSTEM host in the console session:
///
/// ```text
/// device created on Default, thread on Default -> 0x80070005 (the field failure)
/// device created on Default, thread on INPUT desktop -> OK
/// device created on INPUT, thread on INPUT desktop -> OK
/// ```
///
/// The middle row is the load-bearing one: the synthetic pointer device is NOT desktop-affine, so
/// rebinding the THREAD is sufficient and the device never has to be destroyed and recreated
/// across a desktop switch (which would drop in-flight contacts and the pen's in-range state).
///
/// # Safety
/// `dev` must be a live synthetic-pointer device and `frame` a live slice for the call.
unsafe fn inject_following_desktop(
dev: HSYNTHETICPOINTERDEVICE,
frame: &[POINTER_TYPE_INFO],
) -> windows::core::Result<()> {
// SAFETY: per this fn's contract — `dev` is live and `frame` outlives the call, which only
// reads it. Best-effort, exactly as the direct call was.
match InjectSyntheticPointerInput(dev, frame) {
Ok(()) => Ok(()),
Err(first) => {
// Only a desktop switch is worth a rebind; anything else would just fail identically.
let Some(_binding) = InputDesktopBinding::bind() else {
return Err(first);
};
// SAFETY: same live `dev`/`frame`, re-issued with this thread on the input desktop.
InjectSyntheticPointerInput(dev, frame)
}
}
}
/// Windows pen pressure is 0..1024 (vs the wire's full-scale u16).
const WIN_PEN_PRESSURE_MAX: u32 = 1024;
/// Map a normalized [0,1] coordinate pair onto desktop pixels over the STREAMED output's rect
/// ([`crate::stream_target`]) — the surface the SendInput absolute mouse targets too, so pen,
/// touch, and pointer all land identically. The wire normalizes over the streamed display's
/// frame, not the desktop: mapping over the whole virtual desktop is only right when they
/// coincide (Exclusive topology — the fallback when no stream target is live).
fn to_screen(x: f32, y: f32) -> POINT {
let (px, py) = crate::stream_target::map_normalized(x as f64, y as f64);
POINT { x: px, y: py }
}
/// An owned `HSYNTHETICPOINTERDEVICE` (destroyed exactly once on drop).
struct Device(HSYNTHETICPOINTERDEVICE);
// SAFETY: the handle is a plain kernel object identifier with no thread affinity —
// `InjectSyntheticPointerInput`/`DestroySyntheticPointerDevice` are documented callable from
// any thread; ownership transfer/sharing does not alias memory.
unsafe impl Send for Device {}
impl Drop for Device {
fn drop(&mut self) {
// SAFETY: `self.0` is the device this wrapper uniquely owns; destroyed once here.
unsafe { DestroySyntheticPointerDevice(self.0) };
}
}
/// Probe: can this Windows build create a synthetic pen device (Win10 1809+)?
pub fn synthetic_pen_available() -> bool {
// SAFETY: FFI create with by-value args; on success the returned handle is destroyed
// immediately by the `Device` wrapper, exactly once.
match unsafe { CreateSyntheticPointerDevice(PT_PEN, 1, POINTER_FEEDBACK_DEFAULT) } {
Ok(h) => {
drop(Device(h));
true
}
Err(_) => false,
}
}
/// The tracked pen state a refresh tick re-asserts.
#[derive(Default)]
struct PenState {
in_range: bool,
touching: bool,
barrel: bool,
eraser: bool,
x: f32,
y: f32,
pressure: u16,
tilt_deg: u8,
azimuth_deg: u16,
roll_deg: u16,
}
struct PenShared {
dev: Device,
state: PenState,
/// First injection failure logs at WARN (see `TouchShared::fail_warned`).
fail_warned: bool,
}
/// One per-session virtual pen (the Windows sibling of the Linux uinput tablet — same
/// [`PenTransition`] consumer API). Wire barrel button 2 has no Windows pen equivalent
/// (one barrel + eraser is the platform model) and is ignored here.
pub struct VirtualPen {
shared: Arc<Mutex<PenShared>>,
stop: Arc<AtomicBool>,
refresher: Option<std::thread::JoinHandle<()>>,
// Batch-local frame grouping (single-threaded within apply_batch).
edge_down: bool,
edge_up: bool,
is_new: bool,
frame_dirty: bool,
frame_has_motion: bool,
}
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
// SAFETY: FFI create with by-value args; the handle's sole owner becomes `Device`.
let dev = unsafe { CreateSyntheticPointerDevice(PT_PEN, 1, POINTER_FEEDBACK_DEFAULT) }
.context("CreateSyntheticPointerDevice(PT_PEN) — needs Windows 10 1809+")?;
let shared = Arc::new(Mutex::new(PenShared {
dev: Device(dev),
state: PenState::default(),
fail_warned: false,
}));
let stop = Arc::new(AtomicBool::new(false));
// The staleness guard: re-assert the last frame while in range so a stationary pen
// (native plane: between 100 ms heartbeats; GameStream: indefinitely) never hovers out.
let refresher = {
let shared = Arc::clone(&shared);
let stop = Arc::clone(&stop);
std::thread::Builder::new()
.name("pf-pen-refresh".into())
.spawn(move || {
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(std::time::Duration::from_millis(REFRESH_MS));
let s = &mut *shared.lock().unwrap();
if s.state.in_range {
inject_pen(s, POINTER_FLAG_UPDATE, false);
}
}
})
.context("spawn pen refresh thread")?
};
tracing::info!("virtual pen created (Windows synthetic pointer, PT_PEN)");
Ok(VirtualPen {
shared,
stop,
refresher: Some(refresher),
edge_down: false,
edge_up: false,
is_new: false,
frame_dirty: false,
frame_has_motion: false,
})
}
/// Apply one batch of tracker transitions — same grouping contract as the Linux backend:
/// `[ProxIn, Motion, TipDown]` is ONE frame (entry lands at its position, in contact),
/// consecutive `Motion`s split, tip edges own their frames, range-leave is a final
/// no-`INRANGE` frame.
pub fn apply_batch(&mut self, transitions: &[PenTransition]) {
for t in transitions {
match t {
PenTransition::ProximityIn { tool } => {
self.flush();
let mut s = self.shared.lock().unwrap();
s.state.in_range = true;
s.state.eraser = *tool == PenTool::Eraser;
drop(s);
self.is_new = true;
self.frame_dirty = true;
}
PenTransition::Motion { sample } => {
if self.frame_has_motion {
self.flush();
}
self.set_axes(sample);
self.frame_dirty = true;
self.frame_has_motion = true;
}
PenTransition::TipDown => {
self.shared.lock().unwrap().state.touching = true;
self.edge_down = true;
self.frame_dirty = true;
}
PenTransition::ButtonsChanged { pressed, released } => {
let mut s = self.shared.lock().unwrap();
if pressed & PEN_BARREL1 != 0 {
s.state.barrel = true;
}
if released & PEN_BARREL1 != 0 {
s.state.barrel = false;
}
drop(s);
self.frame_dirty = true;
}
PenTransition::TipUp => {
self.shared.lock().unwrap().state.touching = false;
self.edge_up = true;
self.frame_dirty = true;
self.flush(); // UP owns its frame (still INRANGE)
}
PenTransition::ProximityOut => {
self.shared.lock().unwrap().state.in_range = false;
self.frame_dirty = true;
self.flush(); // final frame without INRANGE
}
}
}
self.flush();
}
fn set_axes(&mut self, s: &PenSample) {
let mut sh = self.shared.lock().unwrap();
sh.state.x = s.x;
sh.state.y = s.y;
sh.state.pressure = s.pressure;
sh.state.tilt_deg = s.tilt_deg;
sh.state.azimuth_deg = s.azimuth_deg;
sh.state.roll_deg = s.roll_deg;
}
fn flush(&mut self) {
if !self.frame_dirty {
return;
}
let edge = if self.edge_down {
POINTER_FLAG_DOWN
} else if self.edge_up {
POINTER_FLAG_UP
} else {
POINTER_FLAG_UPDATE
};
inject_pen(&mut self.shared.lock().unwrap(), edge, self.is_new);
self.edge_down = false;
self.edge_up = false;
self.is_new = false;
self.frame_dirty = false;
self.frame_has_motion = false;
}
}
impl Drop for VirtualPen {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(h) = self.refresher.take() {
let _ = h.join();
}
// The device itself dies with `shared` (Device::drop) — Windows releases any held
// in-range/contact state when the synthetic device is destroyed.
}
}
/// Build + inject one pen frame from tracked state. `edge` is DOWN/UP/UPDATE;
/// `INRANGE`/`INCONTACT` derive from the state itself.
fn inject_pen(sh: &mut PenShared, edge: POINTER_FLAGS, is_new: bool) {
let st = &sh.state;
let mut flags = edge;
if st.in_range {
flags |= POINTER_FLAG_INRANGE;
}
if st.touching {
flags |= POINTER_FLAG_INCONTACT | POINTER_FLAG_FIRSTBUTTON;
}
if is_new {
flags |= POINTER_FLAG_NEW;
}
let mut pen_flags = PEN_FLAG_NONE;
if st.barrel {
pen_flags |= PEN_FLAG_BARREL;
}
if st.eraser {
pen_flags |= PEN_FLAG_INVERTED;
if st.touching {
pen_flags |= PEN_FLAG_ERASER;
}
}
let mut pen_mask = PEN_MASK_PRESSURE;
// Contact needs a nonzero pressure to ink; hover reports 0 (Windows convention).
let pressure = if st.touching {
((st.pressure as u32 * WIN_PEN_PRESSURE_MAX) / u16::MAX as u32).max(1)
} else {
0
};
let (mut tilt_x, mut tilt_y) = (0i32, 0i32);
if st.tilt_deg != PEN_TILT_UNKNOWN && st.azimuth_deg != PEN_ANGLE_UNKNOWN {
let az = (st.azimuth_deg as f32).to_radians();
let tilt = st.tilt_deg as f32;
tilt_x = (tilt * az.sin()).round() as i32;
tilt_y = (-tilt * az.cos()).round() as i32;
pen_mask |= PEN_MASK_TILT_X | PEN_MASK_TILT_Y;
}
let mut rotation = 0u32;
if st.roll_deg != PEN_ANGLE_UNKNOWN {
rotation = (st.roll_deg % 360) as u32;
pen_mask |= PEN_MASK_ROTATION;
}
let pt = to_screen(st.x, st.y);
let info = POINTER_TYPE_INFO {
r#type: PT_PEN,
Anonymous: POINTER_TYPE_INFO_0 {
penInfo: POINTER_PEN_INFO {
pointerInfo: POINTER_INFO {
pointerType: PT_PEN,
pointerId: 0,
pointerFlags: flags,
ptPixelLocation: pt,
ptPixelLocationRaw: pt,
..Default::default()
},
penFlags: pen_flags,
penMask: pen_mask,
pressure,
rotation,
tiltX: tilt_x,
tiltY: tilt_y,
},
},
};
// SAFETY: `sh.dev.0` is the live device this wrapper owns; the one-element array is a live
// stack value the call only reads. Best-effort like every injector write — a transient
// failure (desktop switch) is healed by the next refresh tick re-asserting state.
if let Err(e) = unsafe { inject_following_desktop(sh.dev.0, &[info]) } {
if !sh.fail_warned {
sh.fail_warned = true;
tracing::warn!(
error = %e,
flags = format!("{:#x}", flags.0),
pen_flags = format!("{pen_flags:#x}"),
pen_mask = format!("{pen_mask:#x}"),
pressure,
rotation,
tilt_x,
tilt_y,
x = pt.x,
y = pt.y,
"pen inject failed"
);
} else {
tracing::trace!(error = %e, "pen inject failed (transient)");
}
} else {
sh.fail_warned = false;
}
}
/// One live wire-touch contact. `slot` is the SMALL, DENSE pointer id handed to Windows —
/// synthetic-pointer injection rejects arbitrary large ids, and clients (Moonlight's
/// `pointerId` especially) send exactly those, so wire ids compact into the lowest free slot
/// for the contact's lifetime (Apollo's slot-contiguity rule; the on-glass symptom of passing
/// wire ids through was pen working while every touch silently failed to inject).
#[derive(Clone, Copy)]
struct Contact {
id: u32,
slot: u32,
x: f32,
y: f32,
}
/// Lowest slot not held by a live contact.
fn free_slot(contacts: &[Contact]) -> u32 {
let mut slot = 0u32;
while contacts.iter().any(|c| c.slot == slot) {
slot += 1;
}
slot
}
/// Windows can inject at most this many simultaneous synthetic touch contacts.
const MAX_CONTACTS: usize = 10;
struct TouchShared {
dev: Device,
contacts: Vec<Contact>,
/// First injection failure logs at WARN (an on-glass "touch does nothing" must be
/// visible in the host log); repeats stay at trace.
fail_warned: bool,
}
/// The `PT_TOUCH` device servicing wire `TouchDown/Move/Up` events — closes the SendInput
/// touch no-op. Contacts are keyed by the wire's finger id; every frame re-injects the FULL
/// active set (the synthetic-pointer contract), and a refresh thread re-asserts held contacts
/// against the ~100 ms staleness auto-lift.
pub struct SyntheticTouch {
shared: Arc<Mutex<TouchShared>>,
stop: Arc<AtomicBool>,
refresher: Option<std::thread::JoinHandle<()>>,
}
impl SyntheticTouch {
pub fn create() -> Result<SyntheticTouch> {
// SAFETY: FFI create with by-value args; the handle's sole owner becomes `Device`.
let dev = unsafe {
CreateSyntheticPointerDevice(PT_TOUCH, MAX_CONTACTS as u32, POINTER_FEEDBACK_DEFAULT)
}
.context("CreateSyntheticPointerDevice(PT_TOUCH) — needs Windows 10 1809+")?;
let shared = Arc::new(Mutex::new(TouchShared {
dev: Device(dev),
contacts: Vec::new(),
fail_warned: false,
}));
let stop = Arc::new(AtomicBool::new(false));
let refresher = {
let shared = Arc::clone(&shared);
let stop = Arc::clone(&stop);
std::thread::Builder::new()
.name("pf-touch-refresh".into())
.spawn(move || {
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(std::time::Duration::from_millis(REFRESH_MS));
let s = &mut *shared.lock().unwrap();
if !s.contacts.is_empty() {
inject_touch_frame(s, None);
}
}
})
.context("spawn touch refresh thread")?
};
tracing::info!("virtual touchscreen created (Windows synthetic pointer, PT_TOUCH)");
Ok(SyntheticTouch {
shared,
stop,
refresher: Some(refresher),
})
}
/// Apply one wire touch event (`code` = finger id, pixel x/y against the
/// `flags = (w << 16) | h` reference extent, exactly like `MouseMoveAbs`).
pub fn apply(&mut self, ev: &InputEvent) {
let (w, h) = ((ev.flags >> 16) as f32, (ev.flags & 0xFFFF) as f32);
if (w < 1.0 || h < 1.0) && ev.kind != InputKind::TouchUp {
return; // the documented zero-extent drop, as for MouseMoveAbs
}
let (x, y) = (ev.x as f32 / w.max(1.0), ev.y as f32 / h.max(1.0));
let s = &mut *self.shared.lock().unwrap();
match ev.kind {
InputKind::TouchDown => {
match s.contacts.iter().position(|c| c.id == ev.code) {
Some(i) => (s.contacts[i].x, s.contacts[i].y) = (x, y),
None if s.contacts.len() < MAX_CONTACTS => {
let slot = free_slot(&s.contacts);
s.contacts.push(Contact {
id: ev.code,
slot,
x,
y,
})
}
None => return, // beyond the platform max — drop, never evict a live finger
}
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_DOWN)));
}
InputKind::TouchMove => {
match s.contacts.iter().position(|c| c.id == ev.code) {
Some(i) => {
(s.contacts[i].x, s.contacts[i].y) = (x, y);
inject_touch_frame(s, None);
}
// A move for an unknown id (its DOWN was dropped/lost): synthesize the
// contact so the stroke self-heals, like the pen plane does.
None if s.contacts.len() < MAX_CONTACTS => {
let slot = free_slot(&s.contacts);
s.contacts.push(Contact {
id: ev.code,
slot,
x,
y,
});
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_DOWN)));
}
None => {}
}
}
InputKind::TouchUp => {
let Some(idx) = s.contacts.iter().position(|c| c.id == ev.code) else {
return;
};
// The UP frame still carries the lifting contact (with UP flags), then it
// leaves the active set.
inject_touch_frame(s, Some((ev.code, POINTER_FLAG_UP)));
s.contacts.remove(idx);
}
_ => {}
}
}
}
impl Drop for SyntheticTouch {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(h) = self.refresher.take() {
let _ = h.join();
}
}
}
/// Inject the full active-contact frame; `edge` marks one contact's DOWN/UP transition (by
/// WIRE id — everyone else is a held UPDATE). Windows sees the compacted `slot` ids only.
fn inject_touch_frame(sh: &mut TouchShared, edge: Option<(u32, POINTER_FLAGS)>) {
let contacts = &sh.contacts;
if contacts.is_empty() {
return;
}
let mut frame: Vec<POINTER_TYPE_INFO> = Vec::with_capacity(contacts.len());
for c in contacts {
let mut flags = POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT | POINTER_FLAG_UPDATE;
if let Some((id, e)) = edge {
if id == c.id {
if e == POINTER_FLAG_DOWN {
flags = POINTER_FLAG_INRANGE | POINTER_FLAG_INCONTACT | POINTER_FLAG_DOWN;
} else if e == POINTER_FLAG_UP {
flags = POINTER_FLAG_UP; // contact + range end together
}
}
}
let pt = to_screen(c.x, c.y);
frame.push(POINTER_TYPE_INFO {
r#type: PT_TOUCH,
Anonymous: POINTER_TYPE_INFO_0 {
touchInfo: POINTER_TOUCH_INFO {
pointerInfo: POINTER_INFO {
pointerType: PT_TOUCH,
pointerId: c.slot,
pointerFlags: flags,
ptPixelLocation: pt,
ptPixelLocationRaw: pt,
..Default::default()
},
touchFlags: TOUCH_FLAG_NONE,
touchMask: TOUCH_MASK_NONE,
..Default::default()
},
},
});
}
// SAFETY: `sh.dev.0` is the live owned device; `frame` is a live Vec the call only reads.
// Best-effort — a transient failure heals on the next event/refresh re-assertion.
if let Err(e) = unsafe { inject_following_desktop(sh.dev.0, &frame) } {
if !sh.fail_warned {
sh.fail_warned = true;
tracing::warn!(error = %e, contacts = frame.len(), "touch inject failed");
} else {
tracing::trace!(error = %e, "touch inject failed (transient)");
}
} else {
sh.fail_warned = false;
}
}
@@ -1,6 +1,6 @@
//! Windows input injection via `SendInput` (Win32 KeyboardAndMouse) — the Windows analogue of //! Windows input injection via `SendInput` (Win32 KeyboardAndMouse) — the Windows analogue of
//! [`super::wlr`]: absolute mouse normalized to the virtual desktop, relative mouse for games, //! [`super::wlr`]: absolute mouse mapped over the streamed output's rect
//! scancode keyboard, scroll, buttons. Survives UAC/lock desktop switches with Sunshine's //! ([`crate::stream_target`]), relative mouse for games, scancode keyboard, scroll, buttons. Survives UAC/lock desktop switches with Sunshine's
//! retry-on-failure model: the thread stays bound to its desktop and only reattaches //! retry-on-failure model: the thread stays bound to its desktop and only reattaches
//! (`OpenInputDesktop`/`SetThreadDesktop`) when `SendInput` reports a short write (the input //! (`OpenInputDesktop`/`SetThreadDesktop`) when `SendInput` reports a short write (the input
//! desktop switched) — no per-event reattach overhead. //! desktop switched) — no per-event reattach overhead.
@@ -27,25 +27,26 @@ use windows::Win32::System::StationsAndDesktops::{
use windows::Win32::UI::Input::KeyboardAndMouse::{ use windows::Win32::UI::Input::KeyboardAndMouse::{
GetKeyboardLayout, MapVirtualKeyExW, SendInput, HKL, INPUT, INPUT_0, INPUT_KEYBOARD, GetKeyboardLayout, MapVirtualKeyExW, SendInput, HKL, INPUT, INPUT_0, INPUT_KEYBOARD,
INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY, KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE, INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY, KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE,
MAPVK_VK_TO_VSC_EX, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL, MOUSEEVENTF_LEFTDOWN, KEYEVENTF_UNICODE, MAPVK_VK_TO_VSC_EX, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL,
MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP, MOUSEEVENTF_MOVE, MOUSEEVENTF_LEFTDOWN, MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP,
MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK, MOUSEEVENTF_WHEEL, MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK,
MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY, MOUSEEVENTF_WHEEL, MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY,
};
use windows::Win32::UI::WindowsAndMessaging::{
GetForegroundWindow, GetSystemMetrics, GetWindowThreadProcessId, SM_CXVIRTUALSCREEN,
SM_CYVIRTUALSCREEN, SM_XVIRTUALSCREEN, SM_YVIRTUALSCREEN,
}; };
use windows::Win32::UI::WindowsAndMessaging::{GetForegroundWindow, GetWindowThreadProcessId};
use super::InputInjector; use super::InputInjector;
const ABS_MAX: f64 = 65535.0; // SendInput absolute coords are 0..65535 over the chosen surface.
const GENERIC_ALL: u32 = 0x1000_0000; const GENERIC_ALL: u32 = 0x1000_0000;
const XBUTTON1: u32 = 0x0001; const XBUTTON1: u32 = 0x0001;
const XBUTTON2: u32 = 0x0002; const XBUTTON2: u32 = 0x0002;
pub struct SendInputInjector { pub struct SendInputInjector {
desktop: Option<HDESK>, desktop: Option<HDESK>,
/// PT_TOUCH synthetic device, created on the first wire-touch event (a session that never
/// touches never creates one). `None` after a failed create (pre-1809) — touch then stays
/// the historical no-op.
touch: Option<crate::pen::SyntheticTouch>,
touch_failed: bool,
} }
// SAFETY: `SendInputInjector` holds only an `Option<HDESK>` (a desktop handle). The host creates // SAFETY: `SendInputInjector` holds only an `Option<HDESK>` (a desktop handle). The host creates
@@ -58,7 +59,11 @@ unsafe impl Send for SendInputInjector {}
impl SendInputInjector { impl SendInputInjector {
pub fn open() -> Result<Self> { pub fn open() -> Result<Self> {
let mut me = Self { desktop: None }; let mut me = Self {
desktop: None,
touch: None,
touch_failed: false,
};
me.reattach_input_desktop(); // best-effort me.reattach_input_desktop(); // best-effort
tracing::info!("SendInput injector ready (Win32 KeyboardAndMouse)"); tracing::info!("SendInput injector ready (Win32 KeyboardAndMouse)");
Ok(me) Ok(me)
@@ -156,13 +161,16 @@ impl InputInjector for SendInputInjector {
if w == 0 || h == 0 { if w == 0 || h == 0 {
return Ok(()); // contract: drop zero extent return Ok(()); // contract: drop zero extent
} }
let (_vx, _vy, vw, vh) = virtual_desktop_rect(); // Client (0..w,0..h) → the STREAMED output's desktop rect
// One virtual output spanning the virtual desktop: map client (0..w,0..h) -> 0..65535. // ([`crate::stream_target`]; the whole virtual desktop only as fallback) →
// 0..65535 over the virtual desktop for MOUSEEVENTF_VIRTUALDESK. Mapping over
// the desktop alone is the Extend-topology offset bug the pen plane exposed
// (design/pen-tablet-input.md): correct only when the streamed display IS the
// whole desktop.
let cx = (event.x.clamp(0, w as i32)) as f64 / w as f64; let cx = (event.x.clamp(0, w as i32)) as f64 / w as f64;
let cy = (event.y.clamp(0, h as i32)) as f64 / h as f64; let cy = (event.y.clamp(0, h as i32)) as f64 / h as f64;
let ax = (cx * ABS_MAX).round() as i32; let px = crate::stream_target::map_normalized(cx, cy);
let ay = (cy * ABS_MAX).round() as i32; let (ax, ay) = crate::stream_target::desktop_px_to_virtualdesk(px);
let _ = (vw, vh); // virtual-desktop rect reserved for multi-output mapping
let mi = MOUSEINPUT { let mi = MOUSEINPUT {
dx: ax, dx: ax,
dy: ay, dy: ay,
@@ -297,15 +305,60 @@ impl InputInjector for SendInputInjector {
}; };
self.send(&[key(ki)]) self.send(&[key(ki)])
} }
// Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op. InputKind::TextInput => {
// Committed IME text: one Unicode scalar per event, injected as
// `KEYEVENTF_UNICODE` packets (wScan = UTF-16 unit, no scancode/layout involved
// — the receiving app gets the character verbatim via WM_CHAR). An astral-plane
// scalar (emoji) is its surrogate pair, each unit down+up in order — exactly how
// Windows expects supplementary characters from unicode injection.
let Some(ch) = char::from_u32(event.code) else {
return Ok(()); // lone surrogate / out of range — drop
};
if ch.is_control() {
return Ok(()); // control chars ride the VK path (Enter/Backspace/Tab)
}
let mut units = [0u16; 2];
let mut inputs: Vec<INPUT> = Vec::with_capacity(4);
for &unit in ch.encode_utf16(&mut units).iter() {
for flags in [KEYEVENTF_UNICODE, KEYEVENTF_UNICODE | KEYEVENTF_KEYUP] {
inputs.push(key(KEYBDINPUT {
wVk: VIRTUAL_KEY(0),
wScan: unit,
dwFlags: flags,
time: 0,
dwExtraInfo: 0,
}));
}
}
self.send(&inputs)
}
// Gamepad goes through the XUSB backend.
InputKind::GamepadButton InputKind::GamepadButton
| InputKind::GamepadAxis | InputKind::GamepadAxis
| InputKind::GamepadState | InputKind::GamepadState
| InputKind::GamepadRemove | InputKind::GamepadRemove
| InputKind::GamepadArrival | InputKind::GamepadArrival => Ok(()),
| InputKind::TouchDown // Wire touch → the PT_TOUCH synthetic pointer device (design/pen-tablet-input.md
| InputKind::TouchMove // §6; closes the historical SendInput no-op). Lazily created — a session that never
| InputKind::TouchUp => Ok(()), // touches never creates one; a pre-1809 create failure latches back to the no-op.
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp => {
if self.touch.is_none() && !self.touch_failed {
match crate::pen::SyntheticTouch::create() {
Ok(t) => self.touch = Some(t),
Err(e) => {
self.touch_failed = true;
tracing::warn!(
error = %format!("{e:#}"),
"touch: synthetic pointer unavailable — wire touch stays a no-op"
);
}
}
}
if let Some(t) = self.touch.as_mut() {
t.apply(event);
}
Ok(())
}
} }
} }
} }
@@ -324,19 +377,6 @@ fn key(ki: KEYBDINPUT) -> INPUT {
} }
} }
fn virtual_desktop_rect() -> (i32, i32, i32, i32) {
// SAFETY: each `GetSystemMetrics` takes a single by-value `SYSTEM_METRICS_INDEX` constant and
// returns an `i32`; it dereferences no pointer and has no side effects — FFI-`unsafe` only.
unsafe {
(
GetSystemMetrics(SM_XVIRTUALSCREEN),
GetSystemMetrics(SM_YVIRTUALSCREEN),
GetSystemMetrics(SM_CXVIRTUALSCREEN),
GetSystemMetrics(SM_CYVIRTUALSCREEN),
)
}
}
// VKs Windows wants flagged extended even when the scancode high bits aren't set: the editing // VKs Windows wants flagged extended even when the scancode high bits aren't set: the editing
// cluster (Ins/Del/Home/End/PgUp/PgDn = 0x21..0x28, 0x2D, 0x2E), the Win keys (0x5B/0x5C/0x5D), // cluster (Ins/Del/Home/End/PgUp/PgDn = 0x21..0x28, 0x2D, 0x2E), the Win keys (0x5B/0x5C/0x5D),
// RCtrl (0xA3), RAlt (0xA5), Pause (0x90). MAPVK_VK_TO_VSC_EX already encodes E0 for most; this is a // RCtrl (0xA3), RAlt (0xA5), Pause (0x90). MAPVK_VK_TO_VSC_EX already encodes E0 for most; this is a
@@ -0,0 +1,181 @@
//! The streamed display every absolute coordinate maps into (design/pen-tablet-input.md field
//! fix). Pen, touch, and absolute-mouse positions arrive normalized to the STREAMED output's
//! frame, but the injectors historically mapped them over the whole virtual desktop — correct
//! only when the virtual display is the sole active display (Exclusive topology, normalized to
//! origin). In Extend — a physical monitor kept on beside the virtual output, or an Exclusive
//! isolate degraded to the keep-physicals fallback — the streamed output sits at a non-zero
//! origin, so every sample landed shifted and mis-scaled (the pen exposed it first: a stylus is
//! strictly absolute, with no closed-loop correction onto the target like a cursor).
//!
//! The host publishes the streamed output's CCD target id at capture bring-up
//! ([`set_stream_target`]); the mapping sites resolve its CURRENT desktop rect through
//! [`pf_win_display::win_display::source_desktop_rect`] — the same resolver the cursor-readback
//! poller maps frames with, so the two directions always agree — TTL-cached because a
//! group-layout re-arrange moves a live output's origin mid-session. With no target set, or none
//! resolved yet, mapping falls back to the whole virtual desktop: the historical behavior, still
//! correct for Exclusive topology and the client-less devtest paths.
use std::sync::Mutex;
use std::time::{Duration, Instant};
use windows::Win32::UI::WindowsAndMessaging::{
GetSystemMetrics, SM_CXVIRTUALSCREEN, SM_CYVIRTUALSCREEN, SM_XVIRTUALSCREEN, SM_YVIRTUALSCREEN,
};
/// `(x, y, w, h)` in desktop coordinates, physical pixels (`source_desktop_rect` order).
type Rect = (i32, i32, i32, i32);
/// How long a resolved rect stays fresh: long enough that the CCD query cost vanishes at input
/// rates (pen samples + the 40 ms refresh threads), short enough that a mid-session layout move
/// (a parallel session joining the auto-row) is picked up within a blink.
const RECT_TTL: Duration = Duration::from_millis(250);
struct State {
target_id: Option<u32>,
rect: Option<Rect>,
queried: Option<Instant>,
}
static STATE: Mutex<State> = Mutex::new(State {
target_id: None,
rect: None,
queried: None,
});
/// Publish the streamed output (its CCD target id) that absolute input maps into. The host calls
/// this at capture bring-up; it is never cleared at teardown — a deactivated target simply stops
/// resolving (the last-known rect is kept, and nothing injects between sessions), and the next
/// session's bring-up re-targets. One slot per process: with parallel sessions the LAST bring-up
/// wins for every session's absolute input — per-session routing needs source-tagged input
/// events (parallel-displays plan) and the single slot is never worse than the historical
/// whole-desktop mapping.
pub fn set_stream_target(target_id: Option<u32>) {
let mut st = STATE.lock().unwrap();
if st.target_id != target_id {
tracing::info!(?target_id, "absolute-input stream target set");
st.target_id = target_id;
st.rect = None;
st.queried = None;
}
}
/// The streamed output's current desktop rect, TTL-cached. `None` = no target set / never
/// resolved (callers fall back to the whole virtual desktop).
fn stream_rect() -> Option<Rect> {
let mut st = STATE.lock().unwrap();
let target_id = st.target_id?;
let fresh = st.queried.is_some_and(|at| at.elapsed() < RECT_TTL);
if !fresh {
st.queried = Some(Instant::now());
// SAFETY: read-only QueryDisplayConfig over owned locals (`source_desktop_rect`'s
// contract — the same call the cursor-readback poller makes at spawn).
match unsafe { pf_win_display::win_display::source_desktop_rect(target_id) } {
Some(r) => {
if st.rect != Some(r) {
tracing::info!(target_id, rect = ?r, "stream-target desktop rect resolved");
}
st.rect = Some(r);
}
// Not an active path right now (teardown, or a topology commit in flight): keep the
// last-known rect — snapping mid-stroke to the whole-desktop mapping would visibly
// jump, and after teardown nothing injects until the next session re-targets.
None => {
if st.rect.is_some() {
tracing::debug!(
target_id,
"stream target not an active path — keeping last rect"
);
}
}
}
}
st.rect
}
/// Desktop-space pixel for a normalized `[0,1]²` coordinate over the streamed output's rect,
/// falling back to the whole virtual desktop when no stream target is live.
pub(crate) fn map_normalized(nx: f64, ny: f64) -> (i32, i32) {
map_into(stream_rect().unwrap_or_else(virtual_desktop_rect), nx, ny)
}
/// Pure mapping: `[0,1]²` over `(x, y, w, h)`, inclusive edges (1.0 lands on the last pixel).
fn map_into((x, y, w, h): Rect, nx: f64, ny: f64) -> (i32, i32) {
(
x + (nx.clamp(0.0, 1.0) * (w - 1).max(0) as f64).round() as i32,
y + (ny.clamp(0.0, 1.0) * (h - 1).max(0) as f64).round() as i32,
)
}
/// The virtual-desktop bounds `(x, y, w, h)` — the mapping fallback, and the surface
/// `MOUSEEVENTF_VIRTUALDESK` absolute coordinates normalize over.
pub(crate) fn virtual_desktop_rect() -> Rect {
// SAFETY: each `GetSystemMetrics` takes a single by-value `SYSTEM_METRICS_INDEX` constant and
// returns an `i32`; it dereferences no pointer and has no side effects — FFI-`unsafe` only.
unsafe {
(
GetSystemMetrics(SM_XVIRTUALSCREEN),
GetSystemMetrics(SM_YVIRTUALSCREEN),
GetSystemMetrics(SM_CXVIRTUALSCREEN).max(1),
GetSystemMetrics(SM_CYVIRTUALSCREEN).max(1),
)
}
}
/// A desktop-space pixel as the `MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_VIRTUALDESK` 0..65535
/// coordinate pair `SendInput` wants.
pub(crate) fn desktop_px_to_virtualdesk(px: (i32, i32)) -> (i32, i32) {
px_to_abs(virtual_desktop_rect(), px)
}
/// SendInput absolute coordinates span 0..65535 over the chosen surface.
const ABS_MAX: f64 = 65535.0;
/// Pure normalization: a desktop pixel inside `(x, y, w, h)` → 0..65535 over that surface.
fn px_to_abs((vx, vy, vw, vh): Rect, (px, py): (i32, i32)) -> (i32, i32) {
(
((px - vx) as f64 * ABS_MAX / (vw - 1).max(1) as f64).round() as i32,
((py - vy) as f64 * ABS_MAX / (vh - 1).max(1) as f64).round() as i32,
)
}
#[cfg(test)]
mod tests {
use super::*;
/// The Extend-topology field bug: physical 1920x1080 at (0,0), streamed virtual 2560x1440
/// beside it at (1920,0) — samples must land inside the VIRTUAL output, not at the desktop
/// origin.
#[test]
fn maps_over_the_streamed_rect_not_the_desktop() {
let r = (1920, 0, 2560, 1440);
assert_eq!(map_into(r, 0.0, 0.0), (1920, 0));
assert_eq!(map_into(r, 1.0, 1.0), (1920 + 2559, 1439));
assert_eq!(map_into(r, 0.5, 0.5), (1920 + 1280, 720));
}
#[test]
fn clamps_out_of_range_and_handles_negative_origins() {
// An output placed LEFT of / ABOVE the primary has a negative desktop origin.
let r = (-2560, -100, 2560, 1440);
assert_eq!(map_into(r, 0.0, 0.0), (-2560, -100));
assert_eq!(map_into(r, 2.0, -1.0), (-2560 + 2559, -100));
}
#[test]
fn degenerate_rect_pins_to_its_origin() {
assert_eq!(map_into((10, 20, 0, 0), 0.7, 0.7), (10, 20));
}
/// The VIRTUALDESK round trip: win32k maps an absolute coordinate back to a pixel roughly as
/// `px = ax * vw / 65536` (floor) — edge pixels and the streamed output's origin must survive.
#[test]
fn virtualdesk_normalization_round_trips() {
let v = (0, 0, 4480, 1080);
assert_eq!(px_to_abs(v, (0, 0)), (0, 0));
assert_eq!(px_to_abs(v, (4479, 1079)), (65535, 65535));
let (ax, _) = px_to_abs(v, (1920, 0));
assert_eq!((ax as i64 * 4480 / 65536) as i32, 1920);
// Negative-origin desktops (a monitor left of the primary) still normalize from 0.
let v = (-2560, 0, 4480, 1440);
assert_eq!(px_to_abs(v, (-2560, 0)), (0, 0));
}
}
+101
View File
@@ -174,6 +174,73 @@ pub fn default_backend() -> Backend {
Backend::Unsupported Backend::Unsupported
} }
/// Whether the session's inject backend can type **committed text**
/// ([`InputKind::TextInput`] — see `HOST_CAP_TEXT_INPUT`): Windows always (`KEYEVENTF_UNICODE`);
/// Linux only on the wlroots backend (a dedicated virtual keyboard with a dynamically-grown
/// Unicode keymap) — KWin fake-input/libei/gamescope can only press keycodes of the host layout.
/// Consulted at Welcome time to advertise the cap; a mid-session backend switch away from a
/// capable one just degrades to dropped text events (input is lossy by design).
#[cfg(target_os = "windows")]
pub fn text_input_supported() -> bool {
true
}
/// See the Windows variant: Linux types text only through the wlroots virtual-keyboard backend.
#[cfg(target_os = "linux")]
pub fn text_input_supported() -> bool {
matches!(default_backend(), Backend::WlrVirtual)
}
/// No injector ⇒ no text.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn text_input_supported() -> bool {
false
}
/// Whether this host can inject full-fidelity stylus input (`HOST_CAP_PEN` —
/// design/pen-tablet-input.md): Linux only, via the [`pen::VirtualPen`] uinput tablet, so the
/// probe is "can we open /dev/uinput" (the same permission the virtual gamepads need) plus the
/// `PUNKTFUNK_PEN=0` operator kill-switch. Consulted at Welcome time; clients without the bit
/// keep folding pen into touch/pointer. Windows PT_PEN synthetic pointers are the design's P3.
#[cfg(target_os = "linux")]
pub fn pen_supported() -> bool {
if std::env::var("PUNKTFUNK_PEN").as_deref() == Ok("0") {
return false;
}
// SAFETY: 'static NUL-terminated path literal; `open` returns a fresh fd (or -1) and
// retains nothing.
let fd = unsafe {
libc::open(
c"/dev/uinput".as_ptr(),
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if fd < 0 {
return false;
}
// SAFETY: `fd >= 0` is the fd opened above, owned by no one else; closed exactly once here.
unsafe { libc::close(fd) };
true
}
/// Windows: pen (and touch) inject via synthetic pointer devices — available on Win10 1809+,
/// probed by actually creating (and immediately destroying) a PT_PEN device. Same
/// `PUNKTFUNK_PEN=0` kill-switch as Linux. The probe result also stands in for PT_TOUCH
/// (both APIs arrived together in 1809).
#[cfg(target_os = "windows")]
pub fn pen_supported() -> bool {
if std::env::var("PUNKTFUNK_PEN").as_deref() == Ok("0") {
return false;
}
pen::synthetic_pen_available()
}
/// See the Linux/Windows variants — no pen injection elsewhere.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn pen_supported() -> bool {
false
}
#[path = "inject/service.rs"] #[path = "inject/service.rs"]
mod service; mod service;
pub use service::InjectorService; pub use service::InjectorService;
@@ -339,6 +406,40 @@ pub mod gamepad {
pub fn pump_rumble(&mut self, _send: impl FnMut(u16, u16, u16)) {} pub fn pump_rumble(&mut self, _send: impl FnMut(u16, u16, u16)) {}
} }
} }
/// Linux: the "Punktfunk Pen" uinput virtual tablet (design/pen-tablet-input.md §5) — the
/// per-session stylus device the native pen plane injects through.
#[cfg(target_os = "linux")]
#[path = "inject/linux/pen.rs"]
pub mod pen;
/// Windows: PT_PEN/PT_TOUCH synthetic pointer devices (design/pen-tablet-input.md §6).
/// `pen::VirtualPen` here is the PT_PEN device; `pen::SyntheticTouch` backs the SendInput
/// injector's wire-touch path.
#[cfg(target_os = "windows")]
#[path = "inject/windows/pointer_windows.rs"]
pub mod pen;
/// Windows: the streamed output's desktop rect that every absolute coordinate (pen, touch,
/// absolute mouse) maps into — published by the host at capture bring-up, resolved through the
/// CCD source rect (the cursor-readback poller's resolver, so both directions agree). Mapping
/// over the whole virtual desktop instead is the Extend-topology offset bug the pen exposed
/// (design/pen-tablet-input.md).
#[cfg(target_os = "windows")]
#[path = "inject/windows/stream_target.rs"]
pub mod stream_target;
#[cfg(target_os = "windows")]
pub use stream_target::set_stream_target;
/// Stub — pen injection needs the Linux uinput tablet or Windows synthetic pointers;
/// `pen_supported()` is false here, so no host advertises the cap and no batches arrive.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub mod pen {
use anyhow::{bail, Result};
pub struct VirtualPen;
impl VirtualPen {
pub fn create() -> Result<VirtualPen> {
bail!("no pen injection backend on this platform")
}
pub fn apply_batch(&mut self, _transitions: &[punktfunk_core::quic::PenTransition]) {}
}
}
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
#[path = "inject/linux/kwin_fake_input.rs"] #[path = "inject/linux/kwin_fake_input.rs"]
mod kwin_fake_input; mod kwin_fake_input;
+232
View File
@@ -0,0 +1,232 @@
//! Client-side cursor rendering (design/remote-desktop-sweep.md M2): the host forwards the
//! pointer's SHAPE (reliable control stream, cached by serial) and per-frame STATE (lossy
//! `0xD0` — position/visibility), and WE draw it as a real OS cursor — pointer feel stops
//! paying the video round-trip (the Parsec/RDP model). Active only when the session
//! negotiated it (`HOST_CAP_CURSOR` in the Welcome — the host stopped compositing then) and
//! only applied while the DESKTOP mouse model is engaged: under capture the pointer is
//! relative-locked (SDL hides it) and games draw their own cursor in-frame.
//!
//! The host sends the bitmap in host-FRAMEBUFFER pixels, whose size tracks the host virtual
//! display's DPI scaling (32 px at 100%, 96 px at 300%). Drawn 1:1 it balloons on a high-DPI
//! host; instead we scale it by the SAME aspect-fit factor the video is drawn at
//! (`min(window_px/mode)`), so the pointer stays sized to the streamed desktop at any host
//! scaling. SDL cursors are fixed-size from their surface (no draw-time scaling), so we cache
//! shapes RAW and resample per install — rebuilding when the serial OR the fit changes.
use punktfunk_core::client::NativeClient;
use punktfunk_core::quic::{CursorState, HOST_CAP_CURSOR};
use sdl3::mouse::{Cursor, MouseUtil, SystemCursor};
use sdl3::pixels::PixelFormat;
use sdl3::surface::Surface;
use std::collections::HashMap;
use std::time::Duration;
/// Shape serials cached at most — cursors cycle through a handful of shapes (arrow, I-beam,
/// resize…); a runaway host can't grow the map past this (the cache resets, shapes re-arrive
/// on the reliable stream via the serial-miss path).
const SHAPE_CACHE_MAX: usize = 64;
/// A forwarded cursor shape held RAW (host-framebuffer-pixel bytes + hotspot), so it can be
/// rebuilt into a scaled OS cursor whenever the video-fit changes (a window resize). Caching a
/// fixed-size `Cursor` instead would freeze the pointer at its build-time size.
struct RawShape {
rgba: Vec<u8>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
}
pub struct CursorChannel {
/// The Welcome carried `HOST_CAP_CURSOR` — the host forwards instead of compositing.
negotiated: bool,
/// Serial → raw forwarded shape (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, RawShape>,
/// The serial + fit scale the currently-installed OS cursor was built at (`None` =
/// default/system cursor). A change in EITHER forces a rebuild.
installed: Option<(u32, f32)>,
/// Keeps the installed `Cursor` alive — SDL requires it to outlive its `set()`.
installed_cursor: Option<Cursor>,
/// Latest `0xD0` state (latest-wins across a drained batch).
state: Option<CursorState>,
}
impl CursorChannel {
pub fn new(connector: &NativeClient) -> CursorChannel {
let negotiated = connector.host_caps() & HOST_CAP_CURSOR != 0;
if negotiated {
tracing::info!("cursor channel negotiated — host cursor renders locally");
}
CursorChannel {
negotiated,
shapes: HashMap::new(),
installed: None,
installed_cursor: None,
state: None,
}
}
/// Whether the host forwards the cursor this session (it no longer composites one).
pub fn negotiated(&self) -> bool {
self.negotiated
}
/// The latest drained `0xD0` state — the run loop reads `relative_hint` off it for the
/// M3 host-driven mode flip (and `x`/`y` as the reappear position when leaving relative).
pub fn state(&self) -> Option<CursorState> {
self.state
}
/// Drain the two planes and apply the newest state — once per run-loop iteration.
/// `desktop_active` = the desktop mouse model is engaged (captured + desktop): only then
/// do we own the local cursor's shape/visibility; under capture SDL's relative mode owns
/// it, and released the system cursor must look normal. `fit_scale` is host-framebuffer
/// pixels → window pixels (the aspect-fit factor the video is drawn at); the shape is
/// resampled by it so the pointer matches the streamed desktop at any host DPI.
pub fn pump(
&mut self,
connector: &NativeClient,
mouse: &MouseUtil,
desktop_active: bool,
fit_scale: f32,
) {
if !self.negotiated {
return;
}
while let Ok(shape) = connector.next_cursor_shape(Duration::ZERO) {
if self.shapes.len() >= SHAPE_CACHE_MAX {
// Degenerate host: reset — live shapes re-install via the serial-miss path.
self.shapes.clear();
self.installed = None;
}
let (w, h) = (shape.w as u32, shape.h as u32);
if w == 0 || h == 0 || shape.rgba.len() < (w * h * 4) as usize {
tracing::warn!(w, h, "cursor shape malformed — ignored");
continue;
}
// A re-sent serial replaces its entry; force re-install if it's current.
if matches!(self.installed, Some((s, _)) if s == shape.serial) {
self.installed = None;
}
self.shapes.insert(
shape.serial,
RawShape {
rgba: shape.rgba,
w,
h,
hot_x: shape.hot_x as u32,
hot_y: shape.hot_y as u32,
},
);
}
while let Ok(st) = connector.next_cursor_state(Duration::ZERO) {
self.state = Some(st); // latest wins
}
if !desktop_active {
// Capture mode / released: hand the cursor back to the system default so a
// released pointer over the window doesn't wear the host's shape.
if self.installed.take().is_some() {
if let Ok(c) = Cursor::from_system(SystemCursor::Arrow) {
c.set();
self.installed_cursor = Some(c); // keep it alive past set()
}
}
return;
}
let Some(st) = self.state else { return };
if st.visible() && self.installed != Some((st.serial, fit_scale)) {
if let Some(shape) = self.shapes.get(&st.serial) {
match build_scaled_cursor(shape, fit_scale) {
Ok(cursor) => {
cursor.set();
self.installed = Some((st.serial, fit_scale));
self.installed_cursor = Some(cursor); // outlive set()
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
}
}
// Serial miss: the (reliable) shape hasn't landed yet — keep the previous
// cursor for the RTT rather than flashing default.
}
// Visibility follows the host (a host app hid its pointer ⇒ ours hides too). Queried,
// not shadowed, so apply_capture's own show/hide calls can never desync us.
if mouse.is_cursor_showing() != st.visible() {
mouse.show_cursor(st.visible());
}
}
}
/// Resample a raw shape by `fit_scale` and build an SDL color cursor from it. The hotspot scales
/// with the bitmap so the click point stays true. `fit_scale <= 0` (or a degenerate result) is
/// clamped so we always hand SDL a ≥1×1 surface.
fn build_scaled_cursor(shape: &RawShape, fit_scale: f32) -> Result<Cursor, String> {
let scale = if fit_scale.is_finite() && fit_scale > 0.0 {
fit_scale
} else {
1.0
};
let dw = ((shape.w as f32 * scale).round() as u32).max(1);
let dh = ((shape.h as f32 * scale).round() as u32).max(1);
let hot_x = ((shape.hot_x as f32 * scale).round() as u32).min(dw - 1) as i32;
let hot_y = ((shape.hot_y as f32 * scale).round() as u32).min(dh - 1) as i32;
if dw == shape.w && dh == shape.h {
// 1:1 fit (mode == window) — no resample, feed the bytes straight through.
let mut data = shape.rgba.clone();
let surf = Surface::from_data(
&mut data,
shape.w,
shape.h,
shape.w * 4,
PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())?;
return Cursor::from_surface(&surf, hot_x, hot_y).map_err(|e| e.to_string());
}
let mut scaled = resample_rgba(&shape.rgba, shape.w, shape.h, dw, dh);
let surf = Surface::from_data(&mut scaled, dw, dh, dw * 4, PixelFormat::RGBA32)
.map_err(|e| e.to_string())?;
Cursor::from_surface(&surf, hot_x, hot_y).map_err(|e| e.to_string())
}
/// Area-average resample of a straight-alpha RGBA bitmap `(sw×sh) → (dw×dh)`. Averaging is done on
/// PREMULTIPLIED colour (weighting each source texel by its alpha) so transparent-pixel colour
/// can't bleed into the fringe, then un-premultiplied back to straight alpha. Handles both down-
/// and up-scale; for a cursor the common case is a downscale (high-DPI host → smaller pointer).
fn resample_rgba(src: &[u8], sw: u32, sh: u32, dw: u32, dh: u32) -> Vec<u8> {
let mut out = vec![0u8; (dw * dh * 4) as usize];
let fx = sw as f32 / dw as f32;
let fy = sh as f32 / dh as f32;
for dy in 0..dh {
let sy0 = (dy as f32 * fy).floor() as u32;
let sy1 = (((dy + 1) as f32 * fy).ceil() as u32).clamp(sy0 + 1, sh);
for dx in 0..dw {
let sx0 = (dx as f32 * fx).floor() as u32;
let sx1 = (((dx + 1) as f32 * fx).ceil() as u32).clamp(sx0 + 1, sw);
let (mut r, mut g, mut b, mut a_sum, mut n) = (0f32, 0f32, 0f32, 0f32, 0f32);
for sy in sy0..sy1 {
for sx in sx0..sx1 {
let i = ((sy * sw + sx) * 4) as usize;
let a = src[i + 3] as f32 / 255.0;
r += src[i] as f32 * a;
g += src[i + 1] as f32 * a;
b += src[i + 2] as f32 * a;
a_sum += a;
n += 1.0;
}
}
let di = ((dy * dw + dx) * 4) as usize;
if a_sum > 0.0 {
out[di] = (r / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 1] = (g / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 2] = (b / a_sum).round().clamp(0.0, 255.0) as u8;
out[di + 3] = (a_sum / n * 255.0).round().clamp(0.0, 255.0) as u8;
}
// else fully transparent — already zero-filled.
}
}
out
}
+14
View File
@@ -133,6 +133,20 @@ impl Capture {
Some(self.desktop) Some(self.desktop)
} }
/// Set the mouse model directly (the M3 host-driven flip — `relative_hint` says a host
/// app grabbed/hid the pointer, so run relative; hint clear = back to absolute). Same
/// gating and motion hygiene as [`toggle_desktop`](Self::toggle_desktop); returns whether
/// the model actually changed.
pub fn set_desktop(&mut self, on: bool) -> bool {
if !self.abs_ok || self.desktop == on {
return false;
}
self.desktop = on;
self.pending_rel = (0, 0);
self.pending_abs = None;
true
}
/// Whether a regained focus should re-engage: yes unless the user released /// Whether a regained focus should re-engage: yes unless the user released
/// deliberately (the chord keeps its meaning across an Alt-Tab). /// deliberately (the chord keeps its meaning across an Alt-Tab).
pub fn should_reengage(&self) -> bool { pub fn should_reengage(&self) -> bool {
+2
View File
@@ -16,6 +16,8 @@
#[cfg(any(target_os = "linux", windows))] #[cfg(any(target_os = "linux", windows))]
pub mod csc; pub mod csc;
#[cfg(any(target_os = "linux", windows))]
pub mod cursor;
#[cfg(windows)] #[cfg(windows)]
pub mod d3d11; pub mod d3d11;
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
+156 -10
View File
@@ -233,6 +233,19 @@ struct StreamState {
/// window-normalized position must be re-based onto the content rect). `None` until /// window-normalized position must be re-based onto the content rect). `None` until
/// the first frame; touches before then have nothing to map onto and are dropped. /// the first frame; touches before then have nothing to map onto and are dropped.
last_video: Option<(u32, u32)>, last_video: Option<(u32, u32)>,
/// Client-side cursor rendering (M2 cursor channel) — created with the connector; inert
/// when the host didn't negotiate the channel.
cursor_chan: Option<crate::cursor::CursorChannel>,
/// Last observed `relative_hint` (M3): the auto-flip fires on CHANGES only, so it never
/// fights a user who chorded away from the hinted model.
last_hint: Option<bool>,
/// The user flipped the model manually (⌃⌥⇧M) — the standing hint stops driving until
/// the HOST's intent next changes (a fresh hint edge clears this and applies).
hint_override: bool,
/// Last `CursorRenderMode.client_draws` told to the host (§8 mid-stream render flip);
/// `None` = nothing sent yet. Edge-detected each iteration from the live mouse model, so
/// the chord, the M3 auto-flip, and engage/release all reconcile through one path.
sent_client_draws: Option<bool>,
} }
impl StreamState { impl StreamState {
@@ -263,6 +276,10 @@ impl StreamState {
frames: wake_rx, frames: wake_rx,
connector: None, connector: None,
capture: None, capture: None,
cursor_chan: None,
last_hint: None,
hint_override: false,
sent_client_draws: None,
force_software, force_software,
canceled: false, canceled: false,
ready_announced: false, ready_announced: false,
@@ -554,18 +571,27 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// Mouse model flip (capture ⇄ desktop) — applies immediately when // Mouse model flip (capture ⇄ desktop) — applies immediately when
// engaged; a released stream just changes what the next engage does. // engaged; a released stream just changes what the next engage does.
if chord && sc == Scancode::M { if chord && sc == Scancode::M {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(st) = stream.as_mut() {
match cap.toggle_desktop() { let mut flipped = false;
Some(desktop) => { if let Some(cap) = st.capture.as_mut() {
if cap.captured() { match cap.toggle_desktop() {
apply_capture(&mut window, &mouse, true, desktop); Some(desktop) => {
if cap.captured() {
apply_capture(&mut window, &mouse, true, desktop);
}
flipped = true;
tracing::info!(desktop, "chord: mouse mode");
} }
tracing::info!(desktop, "chord: mouse mode"); None => tracing::info!(
"chord: mouse mode — host has no absolute pointer \
(gamescope), staying captured"
),
} }
None => tracing::info!( }
"chord: mouse mode — host has no absolute pointer \ // A manual flip outranks the standing hint until the host's
(gamescope), staying captured" // intent next CHANGES (M3 — the hint edge clears this).
), if flipped {
st.hint_override = true;
} }
} }
continue; continue;
@@ -744,6 +770,75 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
cap.flush_motion(); cap.flush_motion();
} }
// Cursor channel (M2): drain forwarded shape/state and drive the local OS cursor —
// only meaningful in the desktop mouse model (capture's relative lock hides it).
if let Some(st) = stream.as_mut() {
// Host-framebuffer px → window px: the aspect-fit factor the video is drawn at
// (same `min(surface/content)` as `finger_to_content`). The forwarded pointer is
// resampled by it so a high-DPI host's oversized bitmap lands sized to the streamed
// desktop rather than ballooning. 1:1 until the first frame gives `last_video`.
let fit_scale = st.last_video.map_or(1.0, |(vw, vh)| {
let (pw, ph) = window.size_in_pixels();
(pw as f32 / vw.max(1) as f32).min(ph as f32 / vh.max(1) as f32)
});
if let (Some(chan), Some(c)) = (st.cursor_chan.as_mut(), st.connector.as_ref()) {
let desktop_active = st
.capture
.as_ref()
.is_some_and(|cap| cap.captured() && cap.desktop());
chan.pump(c, &mouse, desktop_active, fit_scale);
// §8 mid-stream render flip: tell the host who renders the pointer whenever
// the local model changes. Desktop-active = we draw it (host excludes +
// forwards); anything else — the capture model OR a released pointer — the
// host composites it into the video (full fidelity, the pre-channel look).
// One edge-detected reconciler covers the chord, the M3 auto-flip, and
// engage/release alike.
if chan.negotiated() && st.sent_client_draws != Some(desktop_active) {
st.sent_client_draws = Some(desktop_active);
let _ = c.set_cursor_render(desktop_active);
}
}
// M3 — host-driven mode flip: `relative_hint` set = a host app grabbed/hid the
// pointer (run captured relative, like a game expects); clear = the desktop is
// back (return to absolute, local cursor reappearing at the host's position).
// Edge-triggered so a user's manual chord isn't fought: the override latch
// holds until the HOST's intent next changes.
let hint_state = st.cursor_chan.as_ref().and_then(|ch| ch.state());
if let Some(hs) = hint_state {
let hint = hs.relative_hint();
if st.last_hint != Some(hint) {
st.last_hint = Some(hint);
st.hint_override = false;
}
if !st.hint_override {
let video = st.last_video;
if let Some(cap) = st.capture.as_mut() {
// Desired model: hint ⇒ capture (desktop off); clear ⇒ desktop on.
if cap.captured() && cap.set_desktop(!hint) {
apply_capture(&mut window, &mouse, true, cap.desktop());
if cap.desktop() {
// Reappear where the host last had the pointer, so the
// hand-back is seamless (Parsec's positionX/Y idea).
if let Some(video) = video {
let (wx, wy) = content_to_window(
window.size(),
window.size_in_pixels(),
video,
hs.x,
hs.y,
);
mouse.warp_mouse_in_window(&window, wx, wy);
}
}
tracing::info!(
desktop = cap.desktop(),
"host cursor hint: mouse model flipped"
);
}
}
}
}
}
// Text input follows the overlay's editing state (edge-triggered). // Text input follows the overlay's editing state (edge-triggered).
let want_text = overlay.as_ref().is_some_and(|o| o.text_input_active()); let want_text = overlay.as_ref().is_some_and(|o| o.text_input_active());
@@ -888,6 +983,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
cap.engage(); // capture engages when the stream starts (ui_stream parity) cap.engage(); // capture engages when the stream starts (ui_stream parity)
apply_capture(&mut window, &mouse, true, cap.desktop()); apply_capture(&mut window, &mouse, true, cap.desktop());
st.capture = Some(cap); st.capture = Some(cap);
st.cursor_chan = Some(crate::cursor::CursorChannel::new(&c));
st.connector = Some(c); st.connector = Some(c);
if let Some(f) = opts.on_connected.as_mut() { if let Some(f) = opts.on_connected.as_mut() {
f(fingerprint); f(fingerprint);
@@ -1666,6 +1762,32 @@ fn finger_to_content(
(cx.round() as i32, cy.round() as i32, dw as u32, dh as u32) (cx.round() as i32, cy.round() as i32, dw as u32, dh as u32)
} }
/// The inverse direction of [`finger_to_content`] for the M3 reappear warp: a HOST-frame pixel
/// (`video` space — what `CursorState` carries) → LOGICAL window coordinates (what
/// `warp_mouse_in_window` takes). Maps through the aspect-fit letterbox (physical), then
/// physical → logical via the window's pixel density; out-of-range host coords clamp into the
/// content rect so the warp always lands on the video.
fn content_to_window(
logical: (u32, u32),
surface: (u32, u32),
video: (u32, u32),
x: i32,
y: i32,
) -> (f32, f32) {
let (pw, ph) = (f64::from(surface.0), f64::from(surface.1));
let (vw, vh) = (f64::from(video.0.max(1)), f64::from(video.1.max(1)));
let scale = (pw / vw).min(ph / vh);
let (dw, dh) = ((vw * scale).max(1.0), (vh * scale).max(1.0));
let ox = (pw - dw) / 2.0;
let oy = (ph - dh) / 2.0;
let px = ox + (f64::from(x).clamp(0.0, vw - 1.0)) * scale;
let py = oy + (f64::from(y).clamp(0.0, vh - 1.0)) * scale;
// Physical → logical (HiDPI): the ratio of the window's logical size to its pixel size.
let lx = px * f64::from(logical.0) / pw.max(1.0);
let ly = py * f64::from(logical.1) / ph.max(1.0);
(lx as f32, ly as f32)
}
/// The presenter's share of the unified stats window — folded into each printed line. /// The presenter's share of the unified stats window — folded into each printed line.
#[derive(Default)] #[derive(Default)]
struct PresentedWindow { struct PresentedWindow {
@@ -1763,6 +1885,30 @@ fn stats_text(
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn content_to_window_inverts_the_letterbox() {
// 1920×1080 video letterboxed in a 1600×1200 (4:3) window at 2× HiDPI: pillarless
// top/bottom bars — scale = 1600/1920, dh = 900, oy = 150 (physical).
let logical = (800u32, 600u32);
let surface = (1600u32, 1200u32);
let video = (1920u32, 1080u32);
// The host-frame center must land at the logical window center.
let (wx, wy) = content_to_window(logical, surface, video, 960, 540);
assert!((wx - 400.0).abs() < 1.0, "wx = {wx}");
assert!((wy - 300.0).abs() < 1.0, "wy = {wy}");
// Roundtrip through the forward mapping: normalized window pos → the same host
// content-rect pixel (finger_to_content returns content-RECT coords, i.e. the
// host pixel scaled by the letterbox factor).
let (nx, ny) = (wx / logical.0 as f32, wy / logical.1 as f32);
let (cx, cy, dw, dh) = finger_to_content(surface, video, nx, ny);
assert_eq!((dw, dh), (1600, 900));
assert!((cx - 800).abs() <= 1, "cx = {cx}"); // 960 * (1600/1920)
assert!((cy - 450).abs() <= 1, "cy = {cy}"); // 540 * ( 900/1080)
// Out-of-range host coords clamp into the video, never the bars.
let (_, wy_clamped) = content_to_window(logical, surface, video, 0, 10_000);
assert!(wy_clamped <= 300.0 + 225.0 + 1.0, "wy = {wy_clamped}"); // ≤ bottom of content
}
#[test] #[test]
fn resize_decision_follows_the_d2_discipline() { fn resize_decision_follows_the_d2_discipline() {
let t0 = Instant::now(); let t0 = Instant::now();
+3
View File
@@ -45,6 +45,9 @@ futures-util = "0.3"
wayland-client = "0.31" wayland-client = "0.31"
wayland-scanner = "0.31" wayland-scanner = "0.31"
wayland-backend = "0.3" wayland-backend = "0.3"
# wayland-scanner emits `bitflags::bitflags!` for the KDE output-device protocol's bitfield enums
# (kde-output-device-v2 `capability`/`flags`); needs the crate in scope (kwin_output_mgmt.rs).
bitflags = "2"
[target.'cfg(target_os = "windows")'.dependencies] [target.'cfg(target_os = "windows")'.dependencies]
# The host<->driver wire contract for the pf-vdisplay IddCx backend (control IOCTLs + Pod structs). # The host<->driver wire contract for the pf-vdisplay IddCx backend (control IOCTLs + Pod structs).
@@ -0,0 +1,653 @@
<?xml version="1.0" encoding="UTF-8"?>
<protocol name="kde_output_device_v2">
<copyright><![CDATA[
SPDX-FileCopyrightText: 2008-2011 Kristian Høgsberg
SPDX-FileCopyrightText: 2010-2011 Intel Corporation
SPDX-FileCopyrightText: 2012-2013 Collabora, Ltd.
SPDX-FileCopyrightText: 2015 Sebastian Kügler <sebas@kde.org>
SPDX-FileCopyrightText: 2021 Méven Car <meven.car@enioka.com>
SPDX-License-Identifier: MIT-CMU
]]></copyright>
<interface name="kde_output_device_registry_v2" version="24">
<description summary="output devices">
This interface can be used to list output devices.
If this global is bound with a version less than 21, the unsupported_version
protocol error will be posted.
</description>
<enum name="error">
<description summary="kde_output_device_registry_v2 error values">
These errors can be emitted in response to some requests.
</description>
<entry name="unsupported_version" value="0"
summary="the registry was bound with an unsupported version"/>
</enum>
<event name="finished" type="destructor" since="21">
<description summary="no new output announcements">
This event is sent in response to the stop request. The compositor will
immediately destroy the object after sending this event.
</description>
</event>
<request name="stop" since="21">
<description summary="stop receiving updates">
This request indicates that the client no longer wants to receive new
output announcements. The compositor will send the
kde_output_device_registry_v2.finished event in response to this request.
The compositor may still send new output announcements after calling this
request until the kde_output_device_registry_v2.finished event is sent.
</description>
</request>
<event name="output" since="21">
<description summary="new available output">
This event is sent when a new output is connected or after binding this
global to list all available outputs.
</description>
<arg name="output" type="new_id" interface="kde_output_device_v2"/>
</event>
</interface>
<interface name="kde_output_device_v2" version="24">
<description summary="output configuration representation">
An output device describes a display device available to the compositor.
output_device is similar to wl_output, but focuses on output
configuration management.
A client can query all global output_device objects to enlist all
available display devices, even those that may currently not be
represented by the compositor as a wl_output.
The client sends configuration changes to the server through the
outputconfiguration interface, and the server applies the configuration
changes to the hardware and signals changes to the output devices
accordingly.
This object is published as global during start up for every available
display devices, or when one later becomes available, for example by
being hotplugged via a physical connector.
Warning! The protocol described in this file is a desktop environment
implementation detail. Regular clients must not use this protocol.
Backward incompatible changes may be added without bumping the major
version of the extension.
</description>
<enum name="subpixel">
<description summary="subpixel geometry information">
This enumeration describes how the physical pixels on an output are
laid out.
</description>
<entry name="unknown" value="0"/>
<entry name="none" value="1"/>
<entry name="horizontal_rgb" value="2"/>
<entry name="horizontal_bgr" value="3"/>
<entry name="vertical_rgb" value="4"/>
<entry name="vertical_bgr" value="5"/>
</enum>
<enum name="transform">
<description summary="transform from framebuffer to output">
This describes the transform, that a compositor will apply to a
surface to compensate for the rotation or mirroring of an
output device.
The flipped values correspond to an initial flip around a
vertical axis followed by rotation.
The purpose is mainly to allow clients to render accordingly and
tell the compositor, so that for fullscreen surfaces, the
compositor is still able to scan out directly client surfaces.
</description>
<entry name="normal" value="0"/>
<entry name="90" value="1"/>
<entry name="180" value="2"/>
<entry name="270" value="3"/>
<entry name="flipped" value="4"/>
<entry name="flipped_90" value="5"/>
<entry name="flipped_180" value="6"/>
<entry name="flipped_270" value="7"/>
</enum>
<event name="geometry">
<description summary="geometric properties of the output">
The geometry event describes geometric properties of the output.
The event is sent when binding to the output object and whenever
any of the properties change.
</description>
<arg name="x" type="int"
summary="x position within the global compositor space"/>
<arg name="y" type="int"
summary="y position within the global compositor space"/>
<arg name="physical_width" type="int"
summary="width in millimeters of the output"/>
<arg name="physical_height" type="int"
summary="height in millimeters of the output"/>
<arg name="subpixel" type="int"
summary="subpixel orientation of the output"/>
<arg name="make" type="string"
summary="textual description of the manufacturer"/>
<arg name="model" type="string"
summary="textual description of the model"/>
<arg name="transform" type="int"
summary="transform that maps framebuffer to output"/>
</event>
<event name="current_mode">
<description summary="current mode">
This event describes the mode currently in use for this head. It is only
sent if the output is enabled.
</description>
<arg name="mode" type="object" interface="kde_output_device_mode_v2"/>
</event>
<event name="mode">
<description summary="advertise available output modes and current one">
The mode event describes an available mode for the output.
When the client binds to the output_device object, the server sends this
event once for every available mode the output_device can be operated by.
There will always be at least one event sent out on initial binding,
which represents the current mode.
Later if an output changes, its mode event is sent again for the
eventual added modes and lastly the current mode. In other words, the
current mode is always represented by the latest event sent with the current
flag set.
The size of a mode is given in physical hardware units of the output device.
This is not necessarily the same as the output size in the global compositor
space. For instance, the output may be scaled, as described in
kde_output_device_v2.scale, or transformed, as described in
kde_output_device_v2.transform.
</description>
<arg name="mode" type="new_id" interface="kde_output_device_mode_v2"/>
</event>
<event name="done">
<description summary="sent all information about output">
This event is sent after all other properties have been
sent on binding to the output object as well as after any
other output property change have been applied later on.
This allows to see changes to the output properties as atomic,
even if multiple events successively announce them.
</description>
</event>
<event name="scale">
<description summary="output scaling properties">
This event contains scaling geometry information
that is not in the geometry event. It may be sent after
binding the output object or if the output scale changes
later. If it is not sent, the client should assume a
scale of 1.
A scale larger than 1 means that the compositor will
automatically scale surface buffers by this amount
when rendering. This is used for high resolution
displays where applications rendering at the native
resolution would be too small to be legible.
It is intended that scaling aware clients track the
current output of a surface, and if it is on a scaled
output it should use wl_surface.set_buffer_scale with
the scale of the output. That way the compositor can
avoid scaling the surface, and the client can supply
a higher detail image.
</description>
<arg name="factor" type="fixed" summary="scaling factor of output"/>
</event>
<event name="edid">
<description summary="advertise EDID data for the output">
The edid event encapsulates the EDID data for the outputdevice.
The event is sent when binding to the output object. The EDID
data may be empty, in which case this event is sent anyway.
If the EDID information is empty, you can fall back to the name
et al. properties of the outputdevice.
</description>
<arg name="raw" type="string" summary="base64-encoded EDID string"/>
</event>
<event name="enabled">
<description summary="output is enabled or disabled">
The enabled event notifies whether this output is currently
enabled and used for displaying content by the server.
The event is sent when binding to the output object and
whenever later on an output changes its state by becoming
enabled or disabled.
</description>
<arg name="enabled" type="int" summary="output enabled state"/>
</event>
<event name="uuid">
<description summary="A unique id for this outputdevice">
The uuid can be used to identify the output. It's controlled by
the server entirely. The server should make sure the uuid is
persistent across restarts. An empty uuid is considered invalid.
</description>
<arg name="uuid" type="string" summary="output devices ID"/>
</event>
<event name="serial_number">
<description summary="Serial Number">
Serial ID of the monitor, sent on startup before the first done event.
</description>
<arg name="serialNumber" type="string"
summary="textual representation of serial number"/>
</event>
<event name="eisa_id">
<description summary="EISA ID">
EISA ID of the monitor, sent on startup before the first done event.
</description>
<arg name="eisaId" type="string"
summary="textual representation of EISA identifier"/>
</event>
<enum name="capability" bitfield="true">
<description summary="describes capabilities of the outputdevice">
Describes what capabilities this device has.
</description>
<entry name="overscan" value="0x1"
summary="if this output_device can use overscan"/>
<entry name="vrr" value="0x2"
summary="if this outputdevice supports variable refresh rate"/>
<entry name="rgb_range" value="0x4"
summary="if setting the rgb range is possible"/>
<entry name="high_dynamic_range" value="0x8" since="3"
summary="if this outputdevice supports high dynamic range"/>
<entry name="wide_color_gamut" value="0x10" since="3"
summary="if this outputdevice supports a wide color gamut"/>
<entry name="auto_rotate" value="0x20" since="4"
summary="if this outputdevice supports autorotation"/>
<entry name="icc_profile" value="0x40" since="5"
summary="if this outputdevice supports icc profiles"/>
<entry name="brightness" value="0x80" since="9"
summary="if this outputdevice supports the brightness setting"/>
<entry name="built_in_color" value="0x100" since="12"
summary="if this outputdevice supports the built-in color profile"/>
<entry name="ddc_ci" value="0x200" since="14"
summary="if this outputdevice supports DDC/CI"/>
<entry name="max_bits_per_color" value="0x400" since="15"
summary="if this outputdevice supports setting max bpc"/>
<entry name="edr" value="0x800" since="16"
summary="if this outputdevice supports EDR"/>
<entry name="sharpness" value="0x1000" since="17"
summary="if this outputdevice supports the sharpness setting"/>
<entry name="custom_modes" value="0x2000" since="18"
summary="if this outputdevice supports custom modes"/>
<entry name="auto_brightness" value = "0x4000" since="19"/>
<entry name="hdr_icc_profile" value="0x8000" since="22"
summary="if this outputdevice supports HDR ICC profiles"/>
<entry name="abm_level" value="0x10000" since="23"
summary="if this outputdevice supports the abm level setting"/>
</enum>
<event name="capabilities">
<description summary="capability flags">
What capabilities this device has, sent on startup before the first
done event.
</description>
<arg name="flags" type="uint" enum="capability"/>
</event>
<event name="overscan">
<description summary="overscan">
Overscan value of the monitor in percent, sent on startup before the
first done event.
</description>
<arg name="overscan" type="uint"
summary="amount of overscan of the monitor"/>
</event>
<enum name="vrr_policy">
<description summary="describes vrr policy">
Describes when the compositor may employ variable refresh rate
</description>
<entry name="never" value="0"/>
<entry name="always" value="1"/>
<entry name="automatic" value="2"/>
</enum>
<event name="vrr_policy">
<description summary="Variable Refresh Rate Policy">
What policy the compositor will employ regarding its use of variable
refresh rate.
</description>
<arg name="vrr_policy" type="uint" enum="vrr_policy"/>
</event>
<enum name="rgb_range">
<description summary="describes RGB range policy">
Whether full or limited color range should be used
</description>
<entry name="automatic" value="0"/>
<entry name="full" value="1"/>
<entry name="limited" value="2"/>
</enum>
<event name="rgb_range">
<description summary="RGB range">
What rgb range the compositor is using for this output
</description>
<arg name="rgb_range" type="uint" enum="rgb_range"/>
</event>
<event name="name" since="2">
<description summary="Output's name">
Name of the output, it's useful to cross-reference to an zxdg_output_v1 and ultimately QScreen
</description>
<arg name="name" type="string"/>
</event>
<event name="high_dynamic_range" since="3">
<description summary="if HDR is enabled">
Whether or not high dynamic range is enabled for this output
</description>
<arg name="hdr_enabled" type="uint" summary="1 if enabled, 0 if disabled"/>
</event>
<event name="sdr_brightness" since="3">
<description summary="the brightness of sdr if hdr is enabled">
If high dynamic range is used, this value defines the brightness in nits for content
that's in standard dynamic range format. Note that while the value is in nits, that
doesn't necessarily translate to the same brightness on the screen.
</description>
<arg name="sdr_brightness" type="uint"/>
</event>
<event name="wide_color_gamut" since="3">
<description summary="if WCG is enabled">
Whether or not the use of a wide color gamut is enabled for this output
</description>
<arg name="wcg_enabled" type="uint" summary="1 if enabled, 0 if disabled"/>
</event>
<enum name="auto_rotate_policy">
<description summary="describes when auto rotate should be used"/>
<entry name="never" value="0"/>
<entry name="in_tablet_mode" value="1"/>
<entry name="always" value="2"/>
</enum>
<event name="auto_rotate_policy" since="4">
<description summary="describes when auto rotate is used"/>
<arg name="policy" type="uint" enum="auto_rotate_policy"/>
</event>
<event name="icc_profile_path" since="5">
<description summary="describes the path to the ICC profile used in SDR mode"/>
<arg name="profile_path" type="string"/>
</event>
<event name="brightness_metadata" since="6">
<description summary="metadata about the screen's brightness limits"/>
<arg name="max_peak_brightness" type="uint" summary="in nits"/>
<arg name="max_frame_average_brightness" type="uint" summary="in nits"/>
<arg name="min_brightness" type="uint" summary="in 0.0001 nits"/>
</event>
<event name="brightness_overrides" since="6">
<description summary="overrides for the screen's brightness limits"/>
<arg name="max_peak_brightness" type="int" summary="-1 for no override, positive values are the brightness in nits"/>
<arg name="max_average_brightness" type="int" summary="-1 for no override, positive values are the brightness in nits"/>
<arg name="min_brightness" type="int" summary="-1 for no override, positive values are the brightness in 0.0001 nits"/>
</event>
<event name="sdr_gamut_wideness" since="6">
<description summary="describes which gamut is assumed for sRGB applications">
This can be used to provide the colors users assume sRGB applications should have based on the
default experience on many modern sRGB screens.
</description>
<arg name="gamut_wideness" type="uint" summary="0 means rec.709 primaries, 10000 means native primaries"/>
</event>
<enum name="color_profile_source" since="7">
<description summary="which source the compositor should use for the color profile on an output"/>
<entry name="sRGB" value="0"/>
<entry name="ICC" value="1"/>
<entry name="EDID" value="2"/>
</enum>
<event name="color_profile_source" since="7">
<description summary="describes which source the compositor uses for the color profile on an output in SDR mode"/>
<arg name="source" type="uint" enum="color_profile_source"/>
</event>
<event name="brightness" since="8">
<description summary="brightness multiplier">
This is the brightness modifier of the output. It doesn't specify
any absolute values, but is merely a multiplier on top of other
brightness values, like sdr_brightness and brightness_metadata.
0 is the minimum brightness (not completely dark) and 10000 is
the maximum brightness.
This is currently only supported / meaningful while HDR is active.
</description>
<arg name="brightness" type="uint" summary="brightness in 0-10000"/>
</event>
<enum name="color_power_tradeoff">
<description summary="tradeoff between power and accuracy">
The compositor can do a lot of things that trade between
performance, power and color accuracy. This setting describes
a high level preference from the user about in which direction
that tradeoff should be made.
</description>
<entry name="efficiency" value="0" summary="prefer efficiency and performance"/>
<entry name="accuracy" value="1" summary="prefer accuracy"/>
</enum>
<event name="color_power_tradeoff" since="10">
<description summary="the preferred color/power tradeoff"/>
<arg name="preference" type="uint" enum="color_power_tradeoff"/>
</event>
<event name="dimming" since="11">
<description summary="dimming multiplier">
This is the dimming multiplier of the output. This is similar to
the brightness setting, except it's meant to be a temporary setting
only, not persistent and may be implemented differently depending
on the display.
0 is the minimum dimming factor (not completely dark) and 10000
means the output is not dimmed.
</description>
<arg name="multiplier" type="uint" summary="multiplier in 0-10000"/>
</event>
<event name="replication_source" since="13">
<description summary="source output for mirroring"/>
<arg name="source" type="string" summary="uuid of the source output"/>
</event>
<event name="ddc_ci_allowed" since="14">
<description summary="if DDC/CI should be used to control brightness etc.">
If the ddc_ci capability is present, this determines if settings
such as brightness, contrast or others should be set using DDC/CI.
</description>
<arg name="allowed" type="uint" summary="1 if allowed, 0 if disabled"/>
</event>
<event name="max_bits_per_color" since="15">
<description summary="override max bpc">
This limits the amount of bits per color that are sent to the display.
</description>
<arg name="max_bpc" type="uint" summary="0 for the default / automatic"/>
</event>
<event name="max_bits_per_color_range" since="15">
<description summary="range of max bits per color value"/>
<arg name="min_value" type="uint" summary="the minimum supported by the driver"/>
<arg name="max_value" type="uint" summary="the maximum supported by the driver"/>
</event>
<event name="automatic_max_bits_per_color_limit" since="15">
<description summary="if and to what value automatic max bpc is limited"/>
<arg name="max_bpc_limit" type="uint"
summary="which value automatic bpc gets limited to. 0 if not limited"/>
</event>
<enum name="edr_policy" since="16">
<description summary="when the compositor may make use of EDR"/>
<entry name="never" value="0"/>
<entry name="always" value="1"/>
</enum>
<event name="edr_policy" since="16">
<description summary="when the compositor may apply EDR">
When EDR is enabled, the compositor may increase the backlight beyond
the user-specified setting, in order to present HDR content on displays
without native HDR support.
This will usually result in better visuals, but also increases battery
usage.
</description>
<arg name="policy" type="uint" enum="edr_policy"/>
</event>
<event name="sharpness" since="17">
<description summary="sharpness strength">
This is the sharpness modifier of the output.
0 is sharpness disabled and 10000 is the maximum sharpness
</description>
<arg name="sharpness" type="uint" summary="sharpness in 0-10000"/>
</event>
<event name="priority" since="18">
<description summary="output priority">
Describes the position of the output in the output order list,
with lower values being earlier in the list. There's no specific
value the list has to start at, this value is only used in sorting
outputs.
Note that the output order protocol is not sufficient for this,
as an output may not be in the output order if it's disabled or
mirroring another screen.
</description>
<arg name="priority" type="uint" summary="priority"/>
</event>
<event name="auto_brightness" since="20">
<description summary="whether or not automatic brightness is enabled"/>
<arg name="enabled" type="uint" summary="1 for enabled, 0 for disabled"/>
</event>
<request name="release" type="destructor" since="21">
<description summary="destroy the output device">
This notifies the compositor that the client no longer wishes to use
the kde_output_device_v2 object.
</description>
</request>
<event name="removed" since="21">
<description summary="the output has been removed">
This event is sent when the output device is disconnected and no new
updates will be sent. The client should call the kde_output_device_v2.release
request after receiving this event.
</description>
</event>
<event name="hdr_icc_profile_path" since="22">
<description summary="describes the path to the ICC profile used in HDR mode"/>
<arg name="profile_path" type="string"/>
</event>
<event name="hdr_color_profile_source" since="22">
<description summary="describes which source the compositor uses for the color profile on an output in HDR mode"/>
<arg name="source" type="uint" enum="color_profile_source"/>
</event>
<event name="abm_level" since="23">
<description summary="allowed level of adaptive backlight modulation">
Adaptive backlight modulation is a feature that reduces the backlight
and increases contrast of colors on the screen to improve power usage.
</description>
<arg name="level" type="uint" summary="0 is off, 4 is the maximum level"/>
</event>
</interface>
<interface name="kde_output_device_mode_v2" version="24">
<description summary="output mode">
This object describes an output mode.
Some heads don't support output modes, in which case modes won't be
advertised.
Properties sent via this interface are applied atomically via the
kde_output_device.done event. No guarantees are made regarding the order
in which properties are sent.
</description>
<event name="size">
<description summary="mode size">
This event describes the mode size. The size is given in physical
hardware units of the output device. This is not necessarily the same as
the output size in the global compositor space. For instance, the output
may be scaled or transformed.
</description>
<arg name="width" type="int" summary="width of the mode in hardware units"/>
<arg name="height" type="int" summary="height of the mode in hardware units"/>
</event>
<event name="refresh">
<description summary="mode refresh rate">
This event describes the mode's fixed vertical refresh rate. It is only
sent if the mode has a fixed refresh rate.
</description>
<arg name="refresh" type="int" summary="vertical refresh rate in mHz"/>
</event>
<event name="preferred">
<description summary="mode is preferred">
This event advertises this mode as preferred.
</description>
</event>
<event name="removed">
<description summary="the mode has been destroyed">
The compositor will destroy the object immediately after sending this
event, so it will become invalid and the client should release any
resources associated with it.
</description>
</event>
<enum name="flags">
<description summary="mode flags"/>
<entry name="custom" value="0x1"/>
<entry name="reduced_blanking" value="0x2" deprecated-since="24"/>
</enum>
<event name="flags" since="19">
<description summary="mode flags">
This event describes the mode's flags.
</description>
<arg name="flags" type="uint" enum="flags"/>
</event>
<event name="cvt" since="24">
<description summary="cvt timings">
This event describes the CVT timings associated with the mode.
If an output mode has no CVT timings, this event will not be sent.
</description>
<arg name="dot_clock" type="uint" summary="pixel clock in kHz"/>
<arg name="hdisplay" type="uint" summary="horizontal display size"/>
<arg name="hsync_start" type="uint" summary="horizontal sync start"/>
<arg name="hsync_end" type="uint" summary="horizontal sync end"/>
<arg name="htotal" type="uint" summary="horizontal total size"/>
<arg name="hskew" type="uint" summary="horizontal skew"/>
<arg name="vdisplay" type="uint" summary="vertical display size"/>
<arg name="vsync_start" type="uint" summary="vertical sync start"/>
<arg name="vsync_end" type="uint" summary="vertical sync end"/>
<arg name="vtotal" type="uint" summary="vertical total size"/>
<arg name="vscan" type="uint" summary="vertical scan"/>
<arg name="flags" type="uint" summary="flags, see DRM_MODE_FLAG_*"/>
</event>
</interface>
</protocol>
@@ -0,0 +1,539 @@
<?xml version="1.0" encoding="UTF-8"?>
<protocol name="kde_output_management_v2">
<copyright><![CDATA[
SPDX-FileCopyrightText: 2008-2011 Kristian Høgsberg
SPDX-FileCopyrightText: 2010-2011 Intel Corporation
SPDX-FileCopyrightText: 2012-2013 Collabora, Ltd.
SPDX-FileCopyrightText: 2015 Sebastian Kügler <sebas@kde.org>
SPDX-FileCopyrightText: 2021 Méven Car <meven.car@enioka.com>
SPDX-FileCopyrightText: 2023 Xaver Hugl <xaver.hugl@kde.org>
SPDX-License-Identifier: MIT-CMU
]]></copyright>
<interface name="kde_output_management_v2" version="22">
<description summary="configuration of server outputs through clients">
This interface enables clients to set properties of output devices for screen
configuration purposes via the server. To this end output devices are referenced
by global kde_output_device_v2 objects.
outputmanagement (wl_global)
--------------------------
request:
* create_configuration -> outputconfiguration (wl_resource)
outputconfiguration (wl_resource)
--------------------------
requests:
* enable(outputdevice, bool)
* mode(outputdevice, mode)
* transformation(outputdevice, flag)
* position(outputdevice, x, y)
* apply
events:
* applied
* failed
The server registers one outputmanagement object as a global object. In order
to configure outputs a client requests create_configuration, which provides a
resource referencing an outputconfiguration for one-time configuration. That
way the server knows which requests belong together and can group them by that.
On the outputconfiguration object the client calls for each output whether the
output should be enabled, which mode should be set (by referencing the mode from
the list of announced modes) and the output's global position. Once all outputs
are configured that way, the client calls apply.
At that point and not earlier the server should try to apply the configuration.
If this succeeds the server emits the applied signal, otherwise the failed
signal, such that the configuring client is noticed about the success of its
configuration request.
Through this design the interface enables atomic output configuration changes if
internally supported by the server.
Warning! The protocol described in this file is a desktop environment implementation
detail. Regular clients must not use this protocol. Backward incompatible
changes may be added without bumping the major version of the extension.
</description>
<request name="create_configuration">
<description summary="provide outputconfiguration object for configuring outputs">
Request an outputconfiguration object through which the client can configure
output devices.
</description>
<arg name="id" type="new_id" interface="kde_output_configuration_v2"/>
</request>
<request name="create_mode_list">
<description summary="create a list of custom modes">
For details, see the description of kde_mode_list_v2 and
kde_output_configuration_v2.set_custom_modes.
</description>
<arg name="id" type="new_id" interface="kde_mode_list_v2"/>
</request>
</interface>
<interface name="kde_output_configuration_v2" version="22">
<description summary="configure single output devices">
outputconfiguration is a client-specific resource that can be used to ask
the server to apply changes to available output devices.
The client receives a list of output devices from the registry. When it wants
to apply new settings, it creates a configuration object from the
outputmanagement global, writes changes through this object's enable, scale,
transform and mode calls. It then asks the server to apply these settings in
an atomic fashion, for example through Linux' DRM interface.
The server signals back whether the new settings have applied successfully
or failed to apply. outputdevice objects are updated after the changes have been
applied to the hardware and before the server side sends the applied event.
</description>
<enum name="error">
<description summary="kde_output_configuration_v2 error values">
These error can be emitted in response to kde_output_configuration_v2 requests.
</description>
<entry name="already_applied" value="0" summary="the config is already applied"/>
</enum>
<request name="enable">
<description summary="enable or disable an output">
Mark the output as enabled or disabled.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice to be en- or disabled"/>
<arg name="enable" type="int" summary="1 to enable or 0 to disable this output"/>
</request>
<request name="mode">
<description summary="switch output-device to mode">
Sets the mode for a given output.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this mode change applies to"/>
<arg name="mode" type="object" interface="kde_output_device_mode_v2" summary="the mode to apply"/>
</request>
<request name="transform">
<description summary="transform output-device">
Sets the transformation for a given output.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this transformation change applies to"/>
<arg name="transform" type="int" summary="transform enum"/>
</request>
<request name="position">
<description summary="position output in global space">
Sets the position for this output device. (x,y) describe the top-left corner
of the output in global space, whereby the origin (0,0) of the global space
has to be aligned with the top-left corner of the most left and in case this
does not define a single one the top output.
There may be no gaps or overlaps between outputs, i.e. the outputs are
stacked horizontally, vertically, or both on each other.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this position applies to"/>
<arg name="x" type="int" summary="position on the x-axis"/>
<arg name="y" type="int" summary="position on the y-axis"/>
</request>
<request name="scale">
<description summary="set scaling factor of this output">
Sets the scaling factor for this output device.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this scale change applies to"/>
<arg name="scale" type="fixed" summary="scaling factor"/>
</request>
<request name="apply">
<description summary="apply configuration changes to all output devices">
Asks the server to apply property changes requested through this outputconfiguration
object to all outputs on the server side.
The output configuration can be applied only once. The already_applied protocol error
will be posted if the apply request is called the second time.
</description>
</request>
<event name="applied">
<description summary="configuration changes have been applied">
Sent after the server has successfully applied the changes.
.
</description>
</event>
<event name="failed">
<description summary="configuration changes failed to apply">
Sent if the server rejects the changes or failed to apply them.
</description>
</event>
<request name="destroy" type="destructor">
<description summary="release the outputconfiguration object"/>
</request>
<request name="overscan">
<description summary="set overscan value">
Set the overscan value of this output device with a value in percent.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice overscan applies to"/>
<arg name="overscan" type="uint" summary="overscan value"/>
</request>
<enum name="vrr_policy">
<description summary="describes vrr policy">
Describes when the compositor may employ variable refresh rate
</description>
<entry name="never" value="0"/>
<entry name="always" value="1"/>
<entry name="automatic" value="2"/>
</enum>
<request name="set_vrr_policy">
<description summary="set the VRR policy">
Set what policy the compositor should employ regarding its use of
variable refresh rate.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this VRR policy applies to"/>
<arg name="policy" type="uint" enum="vrr_policy" summary="the vrr policy to apply"/>
</request>
<enum name="rgb_range">
<description summary="describes RGB range policy">
Whether this output should use full or limited rgb.
</description>
<entry name="automatic" value="0"/>
<entry name="full" value="1"/>
<entry name="limited" value="2"/>
</enum>
<request name="set_rgb_range">
<description summary="RGB range">
Whether full or limited color range should be used
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice the rgb range applies to"/>
<arg name="rgb_range" type="uint" enum="rgb_range"/>
</request>
<request name="set_primary_output" since="2">
<description summary="Select which primary output to use" />
<arg name="output" type="object" interface="kde_output_device_v2" allow-null="false"/>
</request>
<request name="set_priority" since="3">
<description summary="Set the order of outputs">
Set the position of the output in the output order list, with lower values
being earlier in the list. There's no specific value the list has to start
at, this value is only used in sorting outputs.
The order of outputs can be used to assign desktop environment components
to a specific screen, see kde_output_order_v1 and kde-output-device-v2 for
details. Note that for consistent behavior, the priority value needs to be
unique among all enabled outputs.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice the index applies to" />
<arg name="priority" type="uint" summary="the priority of the output" />
</request>
<request name="set_high_dynamic_range" since="4">
<description summary="change if HDR should be enabled">
Sets whether or not the output should be set to HDR mode.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="enable_hdr" type="uint" summary="1 to enable, 0 to disable hdr"/>
</request>
<request name="set_sdr_brightness" since="4">
<description summary="set the brightness for sdr content">
Sets the brightness of standard dynamic range content in nits. Only has an effect while the output is in HDR mode.
Note that while the value is in nits, that doesn't necessarily translate to the same brightness on the screen.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="sdr_brightness" type="uint"/>
</request>
<request name="set_wide_color_gamut" since="4">
<description summary="change if a wide color gamut should be used">
Whether or not the output should use a wide color gamut
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="enable_wcg" type="uint" summary="1 to enable, 0 to disable wcg"/>
</request>
<enum name="auto_rotate_policy">
<description summary="describes when auto rotate should be used"/>
<entry name="never" value="0"/>
<entry name="in_tablet_mode" value="1"/>
<entry name="always" value="2"/>
</enum>
<request name="set_auto_rotate_policy" since="5">
<description summary="change when auto rotate should be used"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="policy" type="uint" enum="auto_rotate_policy"/>
</request>
<request name="set_icc_profile_path" since="6">
<description summary="change the used icc profile for SDR mode"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="profile_path" type="string"/>
</request>
<request name="set_brightness_overrides" since="7">
<description summary="override metadata about the screen's brightness limits"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="max_peak_brightness" type="int" summary="-1 for not overriding, or positive values in nits"/>
<arg name="max_frame_average_brightness" type="int" summary="-1 for not overriding, or positive values in nits"/>
<arg name="min_brightness" type="int" summary="-1 for not overriding, or positive values in 0.0001 nits"/>
</request>
<request name="set_sdr_gamut_wideness" since="7">
<description summary="describes which gamut is assumed for sRGB applications">
This can be used to provide the colors users assume sRGB applications should have based on the
default experience on many modern sRGB screens.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="gamut_wideness" type="uint" summary="0 means rec.709 primaries, 10000 means native primaries"/>
</request>
<enum name="color_profile_source" since="7">
<description summary="which source the compositor should use for the color profile on an output"/>
<entry name="sRGB" value="0"/>
<entry name="ICC" value="1"/>
<entry name="EDID" value="2"/>
</enum>
<request name="set_color_profile_source" since="8">
<description summary="which source the compositor should use for the color profile on an output in SDR mode"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="color_profile_source" type="uint" enum="color_profile_source" summary="the color profile source"/>
</request>
<request name="set_brightness" since="9">
<description summary="brightness multiplier">
Set the brightness modifier of the output. It doesn't specify
any absolute values, but is merely a multiplier on top of other
brightness values, like sdr_brightness and brightness_metadata.
0 is the minimum brightness (not completely dark) and 10000 is
the maximum brightness.
This is supported while HDR is active in versions 8 and below,
or when the device supports the "brightness" capability in
versions 9 and above.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="brightness" type="uint" summary="brightness in 0-10000"/>
</request>
<enum name="color_power_tradeoff">
<description summary="tradeoff between power and accuracy">
The compositor can do a lot of things that trade between
performance, power and color accuracy. This setting describes
a high level preference from the user about in which direction
that tradeoff should be made.
</description>
<entry name="efficiency" value="0" summary="prefer efficiency and performance"/>
<entry name="accuracy" value="1" summary="prefer accuracy"/>
</enum>
<request name="set_color_power_tradeoff" since="10">
<description summary="set the preferred color/power tradeoff"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="preference" type="uint" enum="color_power_tradeoff"/>
</request>
<request name="set_dimming" since="11">
<description summary="dimming multiplier">
Set the dimming multiplier of the output. This is similar to the
brightness setting, except it's meant to be a temporary setting
only, not persistent and may be implemented differently depending
on the display.
0 is the minimum dimming factor (not completely dark) and 10000
means the output is not dimmed.
This is supported only when the "brightness" capability is
also supported.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="multiplier" type="uint" summary="multiplier in 0-10000"/>
</request>
<event name="failure_reason" since="12">
<description summary="reason for failure">
Describes why applying the output configuration failed. Is only
sent before the failure event.
</description>
<arg name="reason" type="string" summary="reason for failure"/>
</event>
<request name="set_replication_source" since="13">
<description summary="source output for mirroring">
Set the source output that the outputdevice should mirror its
viewport from.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="source" type="string" summary="uuid of the source output"/>
</request>
<request name="set_ddc_ci_allowed" since="14">
<description summary="if DDC/CI should be used to control brightness etc."/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="allowed" type="uint" summary="1 if allowed, 0 if disabled"/>
</request>
<request name="set_max_bits_per_color" since="15">
<description summary="override the max bpc">
This limits the amount of bits per color that are sent to the display.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="max_bpc" type="uint" summary="0 for the default / automatic"/>
</request>
<enum name="edr_policy" since="16">
<description summary="when the compositor may make use of EDR"/>
<entry name="never" value="0"/>
<entry name="always" value="1"/>
</enum>
<request name="set_edr_policy" since="16">
<description summary="set when the compositor may apply EDR">
When EDR is enabled, the compositor may increase the backlight beyond
the user-specified setting, in order to present HDR content on displays
without native HDR support.
This will usually result in better visuals, but also increases battery
usage.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="policy" type="uint" enum="edr_policy"/>
</request>
<request name="set_sharpness" since="17">
<description summary="sharpness strength">
This is the sharpness modifier of the output.
0 is sharpness disabled and 10000 is the maximum sharpness
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="sharpness" type="uint" summary="sharpness in 0-10000"/>
</request>
<request name="set_custom_modes" since="18">
<description summary="set the custom mode list">
Set the list of custom modes for this output. The compositor
will in response generate the requested modes and add them to
the output (or delete ones no longer in the list).
This can be useful for overclocking displays, or for working
around broken EDIDs.
Note that there is no guarantee for any custom mode to
actually work, or even to leave the display undamaged (in the
case of CRTs). It's entirely the responsibility of the user
to ensure each added mode is the right one for their display.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="modes" type="object" interface="kde_mode_list_v2"/>
</request>
<request name="set_auto_brightness" since="19">
<description summary="whether or not automatic brightness is enabled"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="enabled" type="uint" summary="1 for enabled, 0 for disabled"/>
</request>
<request name="set_hdr_icc_profile_path" since="20">
<description summary="change the used icc profile for HDR mode"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="profile_path" type="string"/>
</request>
<request name="set_hdr_color_profile_source" since="20">
<description summary="which source the compositor should use for the color profile on an output in HDR mode"/>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="color_profile_source" type="uint" enum="color_profile_source" summary="the color profile source"/>
</request>
<request name="set_abm_level" since="21">
<description summary="set the allowed level of adaptive backlight modulation">
Adaptive backlight modulation is a feature that reduces the backlight
and increases contrast of colors on the screen to improve power usage.
</description>
<arg name="outputdevice" type="object" interface="kde_output_device_v2" summary="outputdevice this setting applies to"/>
<arg name="level" type="uint" summary="0 is off, 4 is the maximum level"/>
</request>
</interface>
<interface name="kde_mode_list_v2" version="22">
<description summary="a list of custom modes">
This list is populated by first setting each relevant property,
and then calling add_mode to add a mode with these properties.
One would for example call
- set_resolution
- set_refresh_rate
- set_reduced_blanking
- add_mode
add_mode does not reset the properties that were previously set,
they are valid until the object is destroyed.
The compositor may additionally have sensible defaults for some
properties like reduced_blanking, but for consistent results,
it's best to always set each known property every time.
One can also specify custom modes with CVT timings, for example
- add_cvt
- add_cvt
The parameters resolution and refresh rate are required, if they
are not set, the missing_parameters error will be emitted.
</description>
<enum name="error">
<description summary="kde_mode_list_v2 error values">
These errors can be emitted in response to add_mode requests.
</description>
<entry name="missing_parameters" value="0" summary="a required parameter wasn't set"/>
</enum>
<request name="destroy" type="destructor">
<description summary="destroy the mode list object"/>
</request>
<request name="add_mode">
<description summary="Add the current mode configuration to the list"/>
</request>
<request name="set_resolution">
<arg name="width" type="uint"/>
<arg name="height" type="uint"/>
</request>
<request name="set_refresh_rate">
<arg name="rate" type="uint" summary="in milliHz"/>
</request>
<request name="set_reduced_blanking">
<description summary="whether or not the mode should have reduced blanking">
Reduced blanking is an optimization that can reduce bandwidth / timing
requirements for a display mode by reducing the time vblank takes.
As not all displays support it, it may be desired to still turn it off
though (like with CRTs, where full blanking is required).
</description>
<arg name="reduced" type="uint"
summary="1 for reduced blanking, 0 for normal vblank duration"/>
</request>
<request name="add_cvt" since="22">
<description summary="new mode with CVT timings">
Adds a new mode with the specified CVT timings.
</description>
<arg name="dot_clock" type="uint" summary="pixel clock in kHz"/>
<arg name="hdisplay" type="uint" summary="horizontal display size"/>
<arg name="hsync_start" type="uint" summary="horizontal sync start"/>
<arg name="hsync_end" type="uint" summary="horizontal sync end"/>
<arg name="htotal" type="uint" summary="horizontal total size"/>
<arg name="hskew" type="uint" summary="horizontal skew"/>
<arg name="vdisplay" type="uint" summary="vertical display size"/>
<arg name="vsync_start" type="uint" summary="vertical sync start"/>
<arg name="vsync_end" type="uint" summary="vertical sync end"/>
<arg name="vtotal" type="uint" summary="vertical total size"/>
<arg name="vscan" type="uint" summary="vertical scan"/>
<arg name="flags" type="uint" summary="flags, see DRM_MODE_FLAG_*"/>
</request>
</interface>
</protocol>
+17 -4
View File
@@ -58,6 +58,11 @@ pub(crate) fn emit_display_event(ev: DisplayEvent) {
pub(crate) mod backend; pub(crate) mod backend;
pub use backend::{DisplayOwnership, VirtualDisplay, VirtualOutput}; pub use backend::{DisplayOwnership, VirtualDisplay, VirtualOutput};
/// Time-bounded child-process helpers — every compositor query shells out, and an unbounded one
/// can wedge the calling (session) thread forever.
#[path = "vdisplay/proc.rs"]
pub(crate) mod proc;
/// Live-session detection + session-epoch + env retargeting (plan §W3). /// Live-session detection + session-epoch + env retargeting (plan §W3).
#[path = "vdisplay/session.rs"] #[path = "vdisplay/session.rs"]
pub(crate) mod session; pub(crate) mod session;
@@ -72,13 +77,14 @@ pub use session::{session_epoch, try_recover_session};
#[path = "vdisplay/routing.rs"] #[path = "vdisplay/routing.rs"]
pub(crate) mod routing; pub(crate) mod routing;
pub use routing::{ pub use routing::{
apply_input_env, restore_managed_session, restore_takeover_on_startup, start_restore_worker, apply_input_env, managed_session_available, restore_managed_session,
wants_dedicated_game_session, restore_takeover_on_startup, start_restore_worker, wants_dedicated_game_session,
}; };
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub use routing::{ pub use routing::{
cancel_pending_tv_restore, dedicated_game_exited, launch_into_gamescope_session, cancel_pending_tv_restore, dedicated_game_exited, gamescope_xwayland_cursor_targets,
launch_is_nested, steam_appid_from_launch, watch_steam_game_exit, launch_into_gamescope_session, launch_is_nested, steam_appid_from_launch,
watch_steam_game_exit,
}; };
/// Compositors punktfunk knows how to drive (plan §6). /// Compositors punktfunk knows how to drive (plan §6).
@@ -439,6 +445,13 @@ mod hyprland;
#[path = "vdisplay/linux/kwin.rs"] #[path = "vdisplay/linux/kwin.rs"]
mod kwin; mod kwin;
// In-process KDE output management (kde_output_management_v2) — the topology path that used to shell
// out to `kscreen-doctor`, driven over the compositor's own Wayland instead so it can't be wedged by
// a stuck libkscreen/kscreen-KDED backend. Consumed by `kwin` (best-effort, with kscreen fallback).
#[cfg(target_os = "linux")]
#[path = "vdisplay/linux/kwin_output_mgmt.rs"]
mod kwin_output_mgmt;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
#[path = "vdisplay/windows/manager.rs"] #[path = "vdisplay/windows/manager.rs"]
pub mod manager; pub mod manager;
@@ -70,6 +70,12 @@ pub struct VirtualOutput {
/// Linux-only (the keep-alive pool is Linux). /// Linux-only (the keep-alive pool is Linux).
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub pool_gen: Option<u64>, pub pool_gen: Option<u64>,
/// The backend created the output at a SACRIFICIAL mode and the producer will renegotiate the
/// live stream to `preferred_mode`'s dims (KWin's screencast only rebuilds its format offer —
/// and thus its refresh cap — on a size change while recording; see kwin.rs `create`). The
/// capturer must hold frames until that renegotiation lands. Linux-only.
#[cfg(target_os = "linux")]
pub expect_exact_dims: bool,
} }
impl VirtualOutput { impl VirtualOutput {
@@ -93,6 +99,8 @@ impl VirtualOutput {
reused_gen: None, reused_gen: None,
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pool_gen: None, pool_gen: None,
#[cfg(target_os = "linux")]
expect_exact_dims: false,
} }
} }
} }
@@ -134,6 +142,25 @@ pub trait VirtualDisplay: Send {
/// the backend's default EDID. Default: no-op — only the Windows pf-vdisplay backend can mint /// the backend's default EDID. Default: no-op — only the Windows pf-vdisplay backend can mint
/// per-monitor EDIDs today (the Linux compositors' virtual outputs take no EDID from us). /// per-monitor EDIDs today (the Linux compositors' virtual outputs take no EDID from us).
fn set_client_hdr(&mut self, _hdr: Option<punktfunk_core::quic::HdrMeta>) {} fn set_client_hdr(&mut self, _hdr: Option<punktfunk_core::quic::HdrMeta>) {}
/// Ask the backend for an OUT-OF-BAND cursor on the created output (the cursor channel):
/// the compositor/OS stops compositing the pointer into captured frames and the capture
/// layer surfaces shape/position separately. Carried on the backend instance; set once
/// before [`create`](Self::create) (both session paths pass `cursor_forward`). Off = the
/// compositor EMBEDS the pointer into frames — zero host-side cursor work, the pre-channel
/// path — which is what every session without the negotiated cursor cap gets (Moonlight /
/// GameStream / legacy clients / capture-mode starts), mirroring the Windows no-regression
/// gate. Implementations: Windows pf-vdisplay (IddCx hardware cursor, driver proto v5);
/// KWin (zkde `pointer` metadata vs embedded); Mutter (`cursor-mode` metadata vs embedded);
/// wlroots/hyprland (portal `CursorMode`). Default: no-op (gamescope has no cursor either
/// way — see the Phase C source).
fn set_hw_cursor(&mut self, _on: bool) {}
/// The out-of-band-cursor request currently set (see [`set_hw_cursor`](Self::set_hw_cursor)).
/// The registry includes it in the keep-alive REUSE key: a kept embedded-pointer display can
/// never serve a cursor-channel session (its stream has no cursor metadata to forward) nor
/// vice versa (the pointer would be missing from frames).
fn hw_cursor(&self) -> bool {
false
}
/// The stable identity slot the backend resolved for the most recent [`create`](Self::create) — /// The stable identity slot the backend resolved for the most recent [`create`](Self::create) —
/// the per-client id the identity policy assigned (`Some`), or `None` for shared/anonymous. The /// the per-client id the identity policy assigned (`Some`), or `None` for shared/anonymous. The
/// registry reads it right after `create` to key the display's group **arrangement** (manual /// registry reads it right after `create` to key the display's group **arrangement** (manual
@@ -61,6 +61,32 @@ static MANAGED_SESSION: std::sync::Mutex<Option<SessionState>> = std::sync::Mute
/// (single-instance), so [`schedule_restore_tv_session`] can restart them when the client disconnects. /// (single-instance), so [`schedule_restore_tv_session`] can restart them when the client disconnects.
static STOPPED_AUTOLOGIN: std::sync::Mutex<Vec<String>> = std::sync::Mutex::new(Vec::new()); static STOPPED_AUTOLOGIN: std::sync::Mutex<Vec<String>> = std::sync::Mutex::new(Vec::new());
/// The display-manager unit we stopped for the takeover on a mask-fragile DM flavor (Nobara's
/// `plasmalogin` — see [`dm_survives_masked_unit`]), so the restore brings the box back via
/// `reset-failed` + `restart` of the DM instead of a `--user start` of the gamescope unit (which
/// cannot work there: without a DM login session there is no seat, so gamescope never gets DRM
/// master — live-proven on the Nobara repro VM 2026-07-24).
static STOPPED_DM: std::sync::Mutex<Option<String>> = std::sync::Mutex::new(None);
/// mtime of the `steamos-session-select` sentinel at managed-session launch — the baseline the
/// in-stream "Switch to Desktop" detector compares against. Steam's session-select script writes
/// `~/.config/steamos-session-select` unconditionally in its USER pass, before any of its
/// display-manager checks — so it advances even under a DM-stop takeover, where the script's
/// config-rewrite tail is a silent no-op (every write branch is gated on the DM *running*;
/// diagnosed live on the Nobara repro VM 2026-07-24). An advanced mtime after a capture loss is
/// therefore the one durable trace of the user's switch request.
static SESSION_SELECT_BASELINE: std::sync::Mutex<Option<std::time::SystemTime>> =
std::sync::Mutex::new(None);
/// When [`honor_session_select_switch`] last ran. While recent, a managed (re)launch is refused —
/// the rebuild loop would otherwise race the booting desktop back into game mode (gamescope+Steam
/// come up faster than KWin, and a delivering managed pipeline ends the rebuild's re-detection).
static SWITCH_HONORED_AT: std::sync::Mutex<Option<Instant>> = std::sync::Mutex::new(None);
/// How long after honoring an in-stream desktop switch the managed path refuses to relaunch,
/// giving the DM's desktop session time to come up so re-detection follows it instead.
const SWITCH_HONOR_GRACE: Duration = Duration::from_secs(120);
/// A pending debounced TV-session restore: the instant [`do_restore_tv_session`] should fire after /// A pending debounced TV-session restore: the instant [`do_restore_tv_session`] should fire after
/// the last client disconnect. A reconnect inside the window clears it (and reuses the still-warm /// the last client disconnect. A reconnect inside the window clears it (and reuses the still-warm
/// managed session), so we never stop+relaunch gamescope per connect — that per-connect teardown is /// managed session), so we never stop+relaunch gamescope per connect — that per-connect teardown is
@@ -115,6 +141,10 @@ struct TakeoverState {
stopped_autologin: Vec<String>, stopped_autologin: Vec<String>,
/// Whether we took over SteamOS's `gamescope-session.target` (restore = remove drop-in + restart). /// Whether we took over SteamOS's `gamescope-session.target` (restore = remove drop-in + restart).
steamos: bool, steamos: bool,
/// The display-manager unit we stopped on a mask-fragile DM flavor (restore = `reset-failed` +
/// `restart` of the DM). `default` so takeover files from older hosts still parse.
#[serde(default)]
stopped_dm: Option<String>,
} }
/// Path of the persisted [`TakeoverState`], under `$XDG_RUNTIME_DIR` (per-user, 0700, tmpfs — cleared /// Path of the persisted [`TakeoverState`], under `$XDG_RUNTIME_DIR` (per-user, 0700, tmpfs — cleared
@@ -133,8 +163,9 @@ fn persist_takeover() {
.unwrap_or_else(|e| e.into_inner()) .unwrap_or_else(|e| e.into_inner())
.clone(), .clone(),
steamos: *STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()), steamos: *STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()),
stopped_dm: STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()).clone(),
}; };
if state.stopped_autologin.is_empty() && !state.steamos { if state.stopped_autologin.is_empty() && !state.steamos && state.stopped_dm.is_none() {
clear_takeover(); clear_takeover();
return; return;
} }
@@ -162,17 +193,22 @@ pub fn restore_takeover_on_startup() {
clear_takeover(); clear_takeover();
return; return;
}; };
if state.stopped_autologin.is_empty() && !state.steamos { if state.stopped_autologin.is_empty() && !state.steamos && state.stopped_dm.is_none() {
clear_takeover(); clear_takeover();
return; return;
} }
tracing::warn!( tracing::warn!(
units = ?state.stopped_autologin, units = ?state.stopped_autologin,
steamos = state.steamos, steamos = state.steamos,
stopped_dm = ?state.stopped_dm,
"gamescope: found a stranded takeover from a previous host instance — scheduling TV restore" "gamescope: found a stranded takeover from a previous host instance — scheduling TV restore"
); );
*STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()) = state.stopped_autologin; *STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()) = state.stopped_autologin;
*STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()) = state.steamos; *STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()) = state.steamos;
*STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()) = state.stopped_dm;
// Re-baseline the session-select sentinel: after a crash-restore the launch-time baseline is
// gone, and a long-existing sentinel file must not read as a fresh in-stream switch request.
record_session_select_baseline();
// A generous grace so a client reconnecting right after the restart cancels it (create_managed_session // A generous grace so a client reconnecting right after the restart cancels it (create_managed_session
// clears PENDING_RESTORE) and keeps the streamed session rather than bouncing to gaming mode. // clears PENDING_RESTORE) and keeps the streamed session rather than bouncing to gaming mode.
*PENDING_RESTORE.lock().unwrap_or_else(|e| e.into_inner()) = *PENDING_RESTORE.lock().unwrap_or_else(|e| e.into_inner()) =
@@ -253,6 +289,7 @@ impl VirtualDisplay for GamescopeDisplay {
ownership: DisplayOwnership::External, ownership: DisplayOwnership::External,
reused_gen: None, reused_gen: None,
pool_gen: None, pool_gen: None,
expect_exact_dims: false,
}); });
} }
check_gamescope_version(); // diagnostic only — warns on known-deadlock-prone versions check_gamescope_version(); // diagnostic only — warns on known-deadlock-prone versions
@@ -262,7 +299,10 @@ impl VirtualDisplay for GamescopeDisplay {
// A3 takeover machinery (recorded in STOPPED_AUTOLOGIN + persisted; restarted on session end via // A3 takeover machinery (recorded in STOPPED_AUTOLOGIN + persisted; restarted on session end via
// schedule_restore_tv_session). Non-Steam launches don't conflict, so they skip this. // schedule_restore_tv_session). Non-Steam launches don't conflict, so they skip this.
if self.cmd.as_deref().is_some_and(is_steam_launch) { if self.cmd.as_deref().is_some_and(is_steam_launch) {
stop_autologin_sessions(); // A dedicated launch NEEDS Steam's single instance — no attach degrade exists here, so
// a mask-fragile-DM box without takeover privilege fails with the actionable error.
stop_autologin_sessions()
.context("dedicated Steam launch needs the box's gaming session freed")?;
// B1b: a Steam running in a plain DESKTOP session (GNOME/KDE) holds the instance just // B1b: a Steam running in a plain DESKTOP session (GNOME/KDE) holds the instance just
// the same, and the autologin stop above can't see it — free it too, or fail loudly. // the same, and the autologin stop above can't see it — free it too, or fail loudly.
free_desktop_steam()?; free_desktop_steam()?;
@@ -325,6 +365,40 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
if steamos_session_present() { if steamos_session_present() {
return create_managed_session_steamos(mode); return create_managed_session_steamos(mode);
} }
// In-stream "Switch to Desktop" under a DM-stop takeover: the user's session-select inside
// the streamed game mode advanced the sentinel, but its config rewrite was a silent no-op
// (every write branch needs the DM running, and the takeover stopped it) — so without this,
// the capture loss it caused would just relaunch game mode ("thrown back in", field-tested
// 2026-07-24). Honor the request instead: restore the DM and replay the switch.
let dm_takeover = STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()).clone();
if let Some(dm) = dm_takeover {
if session_select_requested() {
*STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()) = None;
honor_session_select_switch(dm);
return Err(anyhow!(
"the user switched the box to the desktop session — display manager restored; \
re-detection follows the desktop compositor as it comes up"
));
}
}
// Post-honor grace: while the selected desktop boots, a managed relaunch would win the race
// (gamescope+Steam start faster than KWin) and a delivering pipeline ends the rebuild's
// re-detection — right back in game mode. A live box-owned game-mode unit supersedes the
// grace: the user already switched back, so managed may proceed.
let honor_pending = SWITCH_HONORED_AT
.lock()
.unwrap_or_else(|e| e.into_inner())
.is_some_and(|t| t.elapsed() < SWITCH_HONOR_GRACE);
if honor_pending {
if running_autologin_gamescope_unit().is_some() {
*SWITCH_HONORED_AT.lock().unwrap_or_else(|e| e.into_inner()) = None;
} else {
return Err(anyhow!(
"waiting for the desktop session the user selected — refusing to relaunch game \
mode (re-detection follows the desktop once it's up)"
));
}
}
// Attach-only rebuild probe: reuse a live same-mode session, but NEVER stop/relaunch box // Attach-only rebuild probe: reuse a live same-mode session, but NEVER stop/relaunch box
// sessions — right after a capture loss the caller's session detection can be stale, and a // sessions — right after a capture loss the caller's session detection can be stale, and a
// destructive rebuild here would fight the session the user just switched to. // destructive rebuild here would fight the session the user just switched to.
@@ -352,7 +426,28 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
// Bazzite default — `gamescope-session-plus@ogui-steam` on the TV), that session holds Steam and // Bazzite default — `gamescope-session-plus@ogui-steam` on the TV), that session holds Steam and
// renders to the TV's native mode, which we'd capture instead of the client's. Free Steam by // renders to the TV's native mode, which we'd capture instead of the client's. Free Steam by
// stopping it; [`schedule_restore_tv_session`] (on disconnect) brings it back after a debounce. // stopping it; [`schedule_restore_tv_session`] (on disconnect) brings it back after a debounce.
stop_autologin_sessions(); // On a mask-fragile-DM box without the privilege to stop the DM, the takeover would destabilize
// the seat — degrade to ATTACH instead: mirror the box's own live game-mode session (capture +
// inject, no lifecycle ownership), which needs no takeover at all.
if let Err(e) = stop_autologin_sessions() {
tracing::warn!(
error = %format!("{e:#}"),
"gamescope: managed takeover unavailable — degrading to ATTACH (mirroring the box's \
own game-mode session)"
);
let node_id = ensure_box_gamescope_mode(mode)?;
point_injector_at_eis();
return Ok(VirtualOutput {
node_id,
remote_fd: None,
preferred_mode: Some((mode.width, mode.height, mode.refresh_hz)),
keepalive: Box::new(()),
ownership: DisplayOwnership::External,
reused_gen: None,
pool_gen: None,
expect_exact_dims: false,
});
}
// B1b: a desktop-session Steam (outside any gamescope unit) also holds the single instance and // B1b: a desktop-session Steam (outside any gamescope unit) also holds the single instance and
// would make the managed session's own Steam exit at birth. The managed session's Steam itself // would make the managed session's own Steam exit at birth. The managed session's Steam itself
// is exempt (it lives in the SESSION_UNIT cgroup), so the same-mode reuse below is unaffected. // is exempt (it lives in the SESSION_UNIT cgroup), so the same-mode reuse below is unaffected.
@@ -378,7 +473,21 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
} }
// (Re)launch at the new mode. `launch_session` stops the old unit by name first, so there is // (Re)launch at the new mode. `launch_session` stops the old unit by name first, so there is
// exactly one gamescope `Video/Source` node for discovery. // exactly one gamescope `Video/Source` node for discovery.
let node_id = launch_session(client, SESSION_UNIT, mode)?; let node_id = match launch_session(client, SESSION_UNIT, mode) {
Ok(id) => id,
Err(e) => {
// The takeover already happened (autologin units stopped, possibly the DM down) — arm
// the restore now, or a failed launch strands the box sessionless until a host
// restart. Policy-timed; a quick client retry cancels it and relaunches warm.
// MANAGED_SESSION must be released first: the scheduler reads it (orphan detection).
drop(guard);
schedule_restore_tv_session();
return Err(e);
}
};
// Baseline the session-select sentinel NOW: only a write from INSIDE this session (the user's
// "Switch to Desktop") should read as a switch request, not the one that led here.
record_session_select_baseline();
point_injector_at_eis(); point_injector_at_eis();
*guard = Some(SessionState { *guard = Some(SessionState {
width: mode.width, width: mode.width,
@@ -409,6 +518,7 @@ fn managed_output(node_id: u32, mode: Mode) -> VirtualOutput {
ownership: DisplayOwnership::SessionManaged, ownership: DisplayOwnership::SessionManaged,
reused_gen: None, reused_gen: None,
pool_gen: None, pool_gen: None,
expect_exact_dims: false,
} }
} }
@@ -506,7 +616,7 @@ pub fn launch_into_session(cmd: &str) -> Result<std::process::Child> {
let mut c = Command::new("sh"); let mut c = Command::new("sh");
c.arg("-c").arg(cmd); c.arg("-c").arg(cmd);
match discover_session_display_env() { match discover_session_display_env() {
Some((x11, wayland)) => { Some((x11, wayland, _xauth)) => {
tracing::info!( tracing::info!(
command = %cmd, command = %cmd,
x11_display = x11.as_deref().unwrap_or("-"), x11_display = x11.as_deref().unwrap_or("-"),
@@ -531,11 +641,71 @@ pub fn launch_into_session(cmd: &str) -> Result<std::process::Child> {
.context("spawn launch command into gamescope session") .context("spawn launch command into gamescope session")
} }
/// Find the live gamescope session's `(DISPLAY, WAYLAND_DISPLAY)` by scanning same-uid processes /// EVERY nested Xwayland the running gamescope session exposes, as `(DISPLAY, XAUTHORITY)` pairs
/// for one whose environment carries `GAMESCOPE_WAYLAND_DISPLAY` (gamescope sets it for everything /// for the XFixes cursor source (remote-desktop-sweep Phase C). gamescope can run several
/// it runs — Steam, the game, our own nested `sh`). The Wayland value returned is that gamescope /// (`--xwayland-count N` — Steam Gaming Mode uses 2: one for Big Picture, one for the game), and
/// socket; `DISPLAY` is the nested Xwayland. Either can be individually absent. /// the pointer lives on whichever is FOCUSED — so the source connects to all and follows the one
fn discover_session_display_env() -> Option<(Option<String>, Option<String>)> { /// whose pointer moves. The host is not a gamescope child, so gamescope's auth cookie rides along
/// when a process exposes it. Empty when no gamescope session is running / none exposes a `DISPLAY`.
#[cfg(target_os = "linux")]
pub(crate) fn xwayland_cursor_targets() -> Vec<(String, Option<String>)> {
// SAFETY: `getuid()` is a parameterless POSIX call that always succeeds and touches no memory.
let uid = unsafe { libc::getuid() };
let mut out: Vec<(String, Option<String>)> = Vec::new();
let Ok(entries) = std::fs::read_dir("/proc") else {
return out;
};
for e in entries.flatten() {
let name = e.file_name();
let Some(pid_str) = name.to_str() else {
continue;
};
if !pid_str.bytes().all(|b| b.is_ascii_digit()) {
continue;
}
let Ok(md) = std::fs::metadata(e.path()) else {
continue;
};
use std::os::unix::fs::MetadataExt;
if md.uid() != uid {
continue;
}
let Ok(raw) = std::fs::read(e.path().join("environ")) else {
continue;
};
let (mut display, mut is_gamescope, mut xauth) = (None, false, None);
for kv in raw.split(|&b| b == 0) {
let kv = String::from_utf8_lossy(kv);
if kv.starts_with("GAMESCOPE_WAYLAND_DISPLAY=") {
is_gamescope = true;
} else if let Some(v) = kv.strip_prefix("DISPLAY=") {
if !v.is_empty() {
display = Some(v.to_string());
}
} else if let Some(v) = kv.strip_prefix("XAUTHORITY=") {
if !v.is_empty() {
xauth = Some(v.to_string());
}
}
}
if let (true, Some(d)) = (is_gamescope, display) {
// Distinct DISPLAY only; prefer the first non-empty XAUTHORITY seen for it.
match out.iter_mut().find(|(dd, _)| *dd == d) {
Some((_, xa)) if xa.is_none() => *xa = xauth,
Some(_) => {}
None => out.push((d, xauth)),
}
}
}
out
}
/// Find the live gamescope session's `(DISPLAY, WAYLAND_DISPLAY, XAUTHORITY)` by scanning same-uid
/// processes for one whose environment carries `GAMESCOPE_WAYLAND_DISPLAY` (gamescope sets it for
/// everything it runs — Steam, the game, our own nested `sh`). The Wayland value returned is that
/// gamescope socket; `DISPLAY` is the nested Xwayland; `XAUTHORITY` is its auth file (for X
/// clients that aren't gamescope children). Any one can be individually absent.
fn discover_session_display_env() -> Option<(Option<String>, Option<String>, Option<String>)> {
// SAFETY: `getuid()` is a parameterless POSIX call that always succeeds and touches no memory. // SAFETY: `getuid()` is a parameterless POSIX call that always succeeds and touches no memory.
let uid = unsafe { libc::getuid() }; let uid = unsafe { libc::getuid() };
for e in std::fs::read_dir("/proc").ok()?.flatten() { for e in std::fs::read_dir("/proc").ok()?.flatten() {
@@ -558,6 +728,7 @@ fn discover_session_display_env() -> Option<(Option<String>, Option<String>)> {
}; };
let mut display = None; let mut display = None;
let mut gs_wayland = None; let mut gs_wayland = None;
let mut xauth = None;
for kv in raw.split(|&b| b == 0) { for kv in raw.split(|&b| b == 0) {
let kv = String::from_utf8_lossy(kv); let kv = String::from_utf8_lossy(kv);
if let Some(v) = kv.strip_prefix("GAMESCOPE_WAYLAND_DISPLAY=") { if let Some(v) = kv.strip_prefix("GAMESCOPE_WAYLAND_DISPLAY=") {
@@ -568,11 +739,15 @@ fn discover_session_display_env() -> Option<(Option<String>, Option<String>)> {
if !v.is_empty() { if !v.is_empty() {
display = Some(v.to_string()); display = Some(v.to_string());
} }
} else if let Some(v) = kv.strip_prefix("XAUTHORITY=") {
if !v.is_empty() {
xauth = Some(v.to_string());
}
} }
} }
// Only a process INSIDE a gamescope session (it has the marker var) is a valid source. // Only a process INSIDE a gamescope session (it has the marker var) is a valid source.
if gs_wayland.is_some() { if gs_wayland.is_some() {
return Some((display, gs_wayland)); return Some((display, gs_wayland, xauth));
} }
} }
None None
@@ -740,6 +915,38 @@ fn ensure_box_gamescope_mode(mode: Mode) -> Result<u32> {
return Ok(node); return Ok(node);
} }
} }
// Attach-only rebuild probe (parity with both managed paths — this gap was the attach-path
// stale-detection hazard): right after a capture loss the caller's session detection can be
// stale, and a set-environment + unit restart here would fight the session the user just
// switched to. Mirror whatever live node exists at its own mode; refuse otherwise.
if crate::rebuild_probe_active() {
if let Some(node) = find_gamescope_node() {
tracing::info!(
node,
"gamescope: attach-only rebuild probe — mirroring the live node at its own mode"
);
return Ok(node);
}
return Err(anyhow!(
"no live gamescope node — attach-only rebuild probe refuses to restart the box's \
session (re-detection follows the live session)"
));
}
// A box driving a PHYSICAL display is mirrored at its own mode, never re-moded: the re-mode
// restart is the headless-box model (no panel ⇒ the game-mode resolution is ours to set);
// on-glass it would flip the user's own screen to the client's resolution — and on a
// DM-session-driven box (Nobara) the unit restart bounces the login session with it.
if physical_display_connected() {
if let Some(node) = find_gamescope_node() {
tracing::info!(
node,
client_w = mode.width,
client_h = mode.height,
"gamescope: box drives a physical display — attaching at its own mode (no re-mode)"
);
return Ok(node);
}
}
let Some(unit) = running_autologin_gamescope_unit() else { let Some(unit) = running_autologin_gamescope_unit() else {
// No box-owned autologin session to reconfigure (a bare/foreign gamescope): attach to // No box-owned autologin session to reconfigure (a bare/foreign gamescope): attach to
// whatever node exists, accepting its resolution. // whatever node exists, accepting its resolution.
@@ -905,17 +1112,193 @@ fn unmask_unit(unit: &str) {
.status(); .status();
} }
/// The unit name of the display manager driving this box's graphical logins, from the
/// `display-manager.service` alias symlink (the Fedora/Arch/openSUSE convention every
/// gamescope-session distro follows). `None` when no DM is installed (a box that boots straight
/// into a user session — getty autologin / an enabled user unit).
fn display_manager_unit() -> Option<String> {
display_manager_unit_under(std::path::Path::new("/etc/systemd/system"))
}
/// [`display_manager_unit`] against an arbitrary root (the unit-testable core).
fn display_manager_unit_under(base: &std::path::Path) -> Option<String> {
let target = std::fs::read_link(base.join("display-manager.service")).ok()?;
target.file_name().map(|n| n.to_string_lossy().into_owned())
}
/// Does this display manager's autologin loop SURVIVE the gamescope unit being masked? Only SDDM is
/// proven to (Bazzite/SteamOS `Relogin=true` — the mask is what stops its relogin from restarting
/// the unit mid-stream, diagnosed live on .181 2026-07-07; a failing autologin leaves sddm itself
/// running). Nobara's `plasmalogin` (KDE's SDDM successor) is proven FATAL: against a masked unit
/// its session Exec fails instantly, `Relogin=true` retries, and `plasmalogin.service` trips
/// systemd's start limit within ~1 s — the DM dies and the box is a permanent black screen that
/// only a root `reset-failed` + `restart` recovers (live-proven on the Nobara repro VM
/// 2026-07-24). Unknown DMs are treated as fragile: the fragile path degrades gracefully, a wrong
/// "safe" kills the seat.
fn dm_survives_masked_unit(dm: &str) -> bool {
dm == "sddm.service"
}
/// Stop the display manager for a takeover on a mask-fragile DM flavor. Plain `systemctl stop` on
/// the SYSTEM bus — succeeds as root or under an operator polkit rule scoped to the DM unit (see
/// docs); fails cleanly otherwise ("interactive authentication required"), in which case the
/// caller degrades to attach.
fn try_stop_display_manager(dm: &str) -> bool {
Command::new("systemctl")
.args(["stop", dm])
.status()
.map(|s| s.success())
.unwrap_or(false)
}
/// The distro's session-switch helper (ChimeraOS/Nobara layout). Its USER pass records the
/// sentinel + self-pkexecs (authorized `allow_any` by the distro's own polkit action policy, so
/// it works from our sessionless context); its ROOT pass rewrites the DM autologin config — but
/// only while the DM is RUNNING, which is why the takeover must restart the DM before calling it.
const OS_SESSION_SELECT: &str = "/usr/libexec/os-session-select";
/// The sentinel Steam's `steamos-session-select` writes in its user pass
/// (`~/.config/steamos-session-select`) — see [`SESSION_SELECT_BASELINE`].
fn session_select_sentinel() -> Option<std::path::PathBuf> {
let home = std::env::var("HOME").ok()?;
Some(
std::path::Path::new(&home)
.join(".config")
.join("steamos-session-select"),
)
}
/// Current mtime of the session-select sentinel (`None` when it doesn't exist yet).
fn session_select_mtime() -> Option<std::time::SystemTime> {
let path = session_select_sentinel()?;
std::fs::metadata(path).ok()?.modified().ok()
}
/// Record the sentinel baseline at managed-session launch, so a LATER write (the user's in-stream
/// "Switch to Desktop") is distinguishable from the switch that led into this session.
fn record_session_select_baseline() {
*SESSION_SELECT_BASELINE
.lock()
.unwrap_or_else(|e| e.into_inner()) = session_select_mtime();
}
/// Did a session-select run inside the managed session since its launch (sentinel newer than the
/// recorded baseline, or newly created)? Inside a managed game session the only switch Steam
/// offers is TO the desktop, so an advanced sentinel reads as that request.
fn session_select_requested() -> bool {
let baseline = *SESSION_SELECT_BASELINE
.lock()
.unwrap_or_else(|e| e.into_inner());
match (baseline, session_select_mtime()) {
(Some(base), Some(now)) => now > base,
(None, Some(_)) => true, // created during the session
_ => false,
}
}
/// Honor the user's in-stream "Switch to Desktop" under a DM-stop takeover. The OS flow was a
/// silent no-op (the switch script's config rewrite requires a running DM, which the takeover
/// stopped), so replay it with the DM up — every verb live-validated on the Nobara repro VM:
/// 1. consume the takeover and start the DM (its autologin heads back into game mode briefly —
/// the config still names it);
/// 2. run the distro's own `os-session-select desktop` as the user (its internal pkexec is
/// `allow_any`-authorized), which rewrites the DM autologin config to the desktop session;
/// 3. stop the autologin gamescope unit — the login session exits, and `Relogin=true` relogs
/// into the now-selected desktop.
///
/// The caller then refuses managed relaunches for [`SWITCH_HONOR_GRACE`] so the capture-loss
/// re-detection follows the desktop compositor once it's up instead of racing it.
fn honor_session_select_switch(dm: String) {
tracing::info!(
%dm,
"gamescope: in-stream session-select detected — restoring the display manager and \
switching the box to the desktop session"
);
// Consume the takeover state up front: from here on the box is the DM's again.
std::mem::take(&mut *STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()));
clear_takeover();
*MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None;
stop_session(SESSION_UNIT); // dead already (the switch shut its Steam down) — clear the unit
let _ = Command::new("systemctl")
.args(["reset-failed", &dm])
.status();
let _ = Command::new("systemctl").args(["start", &dm]).status();
let deadline = Instant::now() + Duration::from_secs(10);
while Instant::now() < deadline {
let active = Command::new("systemctl")
.args(["is-active", &dm])
.output()
.map(|o| String::from_utf8_lossy(&o.stdout).trim() == "active")
.unwrap_or(false);
if active {
break;
}
std::thread::sleep(Duration::from_millis(500));
}
// Rewrite the autologin session via the distro's own switch helper (needs the DM running).
// Absent/failing helper degrades to a plain DM restore — the box lands back in game mode on
// glass and the stream follows that instead (no black screen either way).
if std::path::Path::new(OS_SESSION_SELECT).exists() {
match Command::new(OS_SESSION_SELECT).arg("desktop").status() {
Ok(s) if s.success() => {
// The relogin only fires when the CURRENT (game-mode) login session exits: wait
// for its autologin unit to come up, then stop it. Never mask here — the mask is
// what start-limit-kills this DM flavor.
let deadline = Instant::now() + Duration::from_secs(15);
loop {
if let Some(unit) = running_autologin_gamescope_unit() {
systemctl_user(&["stop", &unit]);
tracing::info!(
%unit,
"gamescope: desktop selected — stopped the game-mode session so the \
DM relogs into the desktop"
);
break;
}
if Instant::now() >= deadline {
tracing::warn!(
"gamescope: game-mode session never appeared after the DM restart — \
the desktop switch may need a manual session exit"
);
break;
}
std::thread::sleep(Duration::from_millis(500));
}
}
other => tracing::warn!(
status = ?other,
"gamescope: os-session-select failed — leaving the box in its configured session"
),
}
} else {
tracing::warn!(
"gamescope: no {OS_SESSION_SELECT} on this box — restored the DM into its configured \
session instead of switching to the desktop"
);
}
record_session_select_baseline();
*SWITCH_HONORED_AT.lock().unwrap_or_else(|e| e.into_inner()) = Some(Instant::now());
}
/// Stop every autologin gaming-mode session (`gamescope-session-plus@*.service`) so its /// Stop every autologin gaming-mode session (`gamescope-session-plus@*.service`) so its
/// single-instance Steam is free for our own host-managed session. Records the units so /// single-instance Steam is free for our own host-managed session. Records the units so
/// [`schedule_restore_tv_session`] can restart them on disconnect. Our own session is the transient /// [`schedule_restore_tv_session`] can restart them on disconnect. Our own session is the transient
/// `punktfunk-gamescope` unit (not a `@`-instance), so it's never matched here. No-op when nothing /// `punktfunk-gamescope` unit (not a `@`-instance), so it's never matched here. No-op when nothing
/// is autologged in (e.g. a box that boots headless). Each unit is **masked first** ([`mask_unit`] — /// is autologged in (e.g. a box that boots headless).
/// SDDM's `Relogin=true` would otherwise restart it instantly), then torn down with **SIGKILL** ///
/// ([`kill_unit`]) to avoid the F44 GPU-context leak that the autologin's SIGTERM stop triggers. /// The teardown is DM-flavor-aware ([`dm_survives_masked_unit`]):
/// Matches every loaded instance, not just `running` ones — under the SDDM relogin churn the unit /// * **SDDM / no DM**: each unit is **masked first** ([`mask_unit`] — SDDM's `Relogin=true` would
/// flaps through `activating`/`failed` between cycles, and an unmasked flapping unit re-enters the /// otherwise restart it instantly), then torn down with **SIGKILL** ([`kill_unit`]) to avoid the
/// fight the moment the supervisor restarts it. /// F44 GPU-context leak that the autologin's SIGTERM stop triggers. Matches every loaded
fn stop_autologin_sessions() { /// instance, not just `running` ones — under the SDDM relogin churn the unit flaps through
/// `activating`/`failed` between cycles, and an unmasked flapping unit re-enters the fight the
/// moment the supervisor restarts it.
/// * **Mask-fragile DM** (Nobara's `plasmalogin`, unknown DMs): masking start-limit-kills the DM
/// itself (permanent black screen), so instead **stop the DM** — no supervisor left to relogin —
/// then SIGKILL the units unmasked. Needs privilege (root / an operator polkit rule); without it
/// nothing is touched and the error tells the caller to degrade to ATTACH (mirror the box's own
/// session) rather than destabilize the seat.
fn stop_autologin_sessions() -> Result<()> {
let Ok(out) = Command::new("systemctl") let Ok(out) = Command::new("systemctl")
.args([ .args([
"--user", "--user",
@@ -928,26 +1311,65 @@ fn stop_autologin_sessions() {
]) ])
.output() .output()
else { else {
return; return Ok(());
}; };
let mut stopped = Vec::new(); // `(unit, ACTIVE state)` — the `--plain` columns are UNIT LOAD ACTIVE SUB DESCRIPTION.
for line in String::from_utf8_lossy(&out.stdout).lines() { let listed: Vec<(String, String)> = String::from_utf8_lossy(&out.stdout)
if let Some(unit) = line.split_whitespace().next() { .lines()
if unit.starts_with("gamescope-session-plus@") && unit.ends_with(".service") { .filter_map(|l| {
mask_unit(unit); // block the SDDM relogin loop from restarting it mid-stream let mut cols = l.split_whitespace();
kill_unit(unit); // SIGKILL teardown — avoid the F44 GPU-context leak let unit = cols.next()?;
tracing::info!( let active = cols.nth(1).unwrap_or("");
unit, (unit.starts_with("gamescope-session-plus@") && unit.ends_with(".service"))
"freed Steam: masked + SIGKILL-stopped the autologin gaming session for this stream" .then(|| (unit.to_string(), active.to_string()))
); })
stopped.push(unit.to_string()); .collect();
} if listed.is_empty() {
return Ok(()); // nothing autologged in — Steam is already free
}
let dm = display_manager_unit();
let mask_safe = dm.as_deref().is_none_or(dm_survives_masked_unit);
if !mask_safe {
// Only a LIVE instance holds Steam / justifies touching the DM. A loaded-but-inactive
// leftover (the box switched back to the desktop earlier) must not stop the DM — that
// would kill the user's live desktop to free nothing.
if !listed
.iter()
.any(|(_, active)| matches!(active.as_str(), "active" | "activating"))
{
return Ok(());
} }
let dm = dm.expect("!is_none_or ⇒ Some");
if !try_stop_display_manager(&dm) {
bail!(
"the box's gaming session is driven by {dm}, which does not survive a masked \
session unit, and stopping it needs privilege install the punktfunk \
display-manager polkit rule (see docs) to enable the managed takeover"
);
}
tracing::info!(
%dm,
"freed Steam: stopped the display manager for this stream (mask-fragile DM flavor)"
);
*STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()) = Some(dm);
} }
if !stopped.is_empty() { let units: Vec<String> = listed.into_iter().map(|(u, _)| u).collect();
*STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()) = stopped; let mut stopped = Vec::new();
persist_takeover(); // A3: survive a host crash mid-stream for unit in units {
if mask_safe {
mask_unit(&unit); // block the SDDM relogin loop from restarting it mid-stream
}
kill_unit(&unit); // SIGKILL teardown — avoid the F44 GPU-context leak
tracing::info!(
%unit,
masked = mask_safe,
"freed Steam: stopped the autologin gaming session for this stream"
);
stopped.push(unit);
} }
*STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()) = stopped;
persist_takeover(); // A3: survive a host crash mid-stream
Ok(())
} }
/// How long a desktop Steam gets to honor `steam -shutdown` before the spawn fails. Steam tears /// How long a desktop Steam gets to honor `steam -shutdown` before the spawn fails. Steam tears
@@ -1092,7 +1514,16 @@ pub fn schedule_restore_tv_session() {
.lock() .lock()
.unwrap_or_else(|e| e.into_inner()) .unwrap_or_else(|e| e.into_inner())
.is_empty() .is_empty()
&& !*STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()); && !*STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner())
&& STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()).is_none()
// A managed session that took nothing over (started beside a live desktop — e.g. a client
// gamescope pin on a KDE box) still owns the transient SESSION_UNIT: without this arm it
// was ORPHANED forever after disconnect ("closing the app does not end the session",
// field report 2026-07-24) — the restore stops it even with no autologin to bring back.
&& MANAGED_SESSION
.lock()
.unwrap_or_else(|e| e.into_inner())
.is_none();
if nothing_to_restore { if nothing_to_restore {
return; // nothing was taken over → nothing to restore (also the non-managed path) return; // nothing was taken over → nothing to restore (also the non-managed path)
} }
@@ -1116,6 +1547,26 @@ pub fn schedule_restore_tv_session() {
} }
} }
/// Does any DRM connector report a physically `connected` display? Scans
/// `/sys/class/drm/*/status` — only connector nodes (`card0-eDP-1`, `card0-HDMI-A-1`, …) have a
/// `status` file, so the bare `cardN` device dirs and `renderD*` nodes filter themselves out. A
/// headless box (VM, panel-less mini PC) has none — in which case a "restore to the physical
/// panel" can only fail, gamescope having no output to drive. Errors (no DRM at all, sysfs
/// unreadable) read as headless: the safe direction is keeping the working session.
fn physical_display_connected() -> bool {
connected_connector_under(std::path::Path::new("/sys/class/drm"))
}
/// [`physical_display_connected`] against an arbitrary sysfs root (the unit-testable core).
fn connected_connector_under(base: &std::path::Path) -> bool {
let Ok(entries) = std::fs::read_dir(base) else {
return false;
};
entries.flatten().any(|e| {
std::fs::read_to_string(e.path().join("status")).is_ok_and(|s| s.trim() == "connected")
})
}
/// Tear down our host-managed session (freeing Steam) and restart the autologin gaming session(s) /// Tear down our host-managed session (freeing Steam) and restart the autologin gaming session(s)
/// we stopped on connect — so the TV returns to gaming mode when no one is streaming. Invoked by /// we stopped on connect — so the TV returns to gaming mode when no one is streaming. Invoked by
/// [`start_restore_worker`] once the debounce deadline passes; takes the stopped-unit list so a /// [`start_restore_worker`] once the debounce deadline passes; takes the stopped-unit list so a
@@ -1127,6 +1578,19 @@ fn do_restore_tv_session() {
{ {
let mut took = STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()); let mut took = STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner());
if *took { if *took {
// A box with no physically connected display (a VM, a panel-less mini PC) has no
// "physical gaming session" to restore TO: removing the drop-in and restarting the
// target just crash-loops gamescope (no output to drive) and strands every later
// connect on "no usable compositor". Keep the headless session — and the takeover
// state, so a same-mode reconnect reuses it warm — instead. Checked at restore time
// (not connect time) so plugging a panel in later restores normally.
if !physical_display_connected() {
tracing::info!(
"gamescope (SteamOS): no physical display connected — keeping the headless \
session (nothing to restore to)"
);
return;
}
*took = false; *took = false;
clear_takeover(); // A3: takeover undone — drop the persisted crash-restore marker clear_takeover(); // A3: takeover undone — drop the persisted crash-restore marker
*MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None; *MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None;
@@ -1154,8 +1618,24 @@ fn do_restore_tv_session() {
} }
} }
let units = std::mem::take(&mut *STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner())); let units = std::mem::take(&mut *STOPPED_AUTOLOGIN.lock().unwrap_or_else(|e| e.into_inner()));
if units.is_empty() { let dm = std::mem::take(&mut *STOPPED_DM.lock().unwrap_or_else(|e| e.into_inner()));
return; // nothing was stolen → nothing to restore (also the non-Bazzite path) if units.is_empty() && dm.is_none() {
// Nothing was stolen — but a managed session that started BESIDE a live desktop (client
// gamescope pin on a KDE box) still owns the transient unit; stop it so it doesn't run
// orphaned forever after the disconnect. No-op when the unit isn't running.
if MANAGED_SESSION
.lock()
.unwrap_or_else(|e| e.into_inner())
.take()
.is_some()
{
stop_session(SESSION_UNIT);
tracing::info!(
"gamescope: stopped the idle managed session (nothing was taken over — no box \
session to restore)"
);
}
return;
} }
clear_takeover(); // A3: takeover consumed — drop the persisted crash-restore marker clear_takeover(); // A3: takeover consumed — drop the persisted crash-restore marker
stop_session(SESSION_UNIT); // our gamescope/Steam session, so Steam is free for the autologin stop_session(SESSION_UNIT); // our gamescope/Steam session, so Steam is free for the autologin
@@ -1166,18 +1646,54 @@ fn do_restore_tv_session() {
} }
*MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None; *MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None;
// Only bring the gaming autologin BACK if the box is still meant to be in gaming mode. If the // Only bring the gaming autologin BACK if the box is still meant to be in gaming mode. If the
// user switched to a desktop session (KDE/GNOME/wlroots) in the meantime, don't yank them back // user switched to a desktop session (KDE/GNOME/wlroots/Hyprland) in the meantime, don't yank
// to gaming — leave the desktop alone. (We still stopped our idle managed session above.) // them back to gaming — leave the desktop alone. (We still stopped our idle managed session
// above.)
use super::ActiveKind; use super::ActiveKind;
if matches!( if matches!(
super::detect_active_session().kind, super::detect_active_session().kind,
ActiveKind::DesktopKde | ActiveKind::DesktopGnome | ActiveKind::DesktopWlroots ActiveKind::DesktopKde
| ActiveKind::DesktopGnome
| ActiveKind::DesktopWlroots
| ActiveKind::DesktopHyprland
) { ) {
tracing::info!( tracing::info!(
"gamescope: a desktop session is active — not restoring the TV gaming session" "gamescope: a desktop session is active — not restoring the TV gaming session"
); );
return; return;
} }
// Mask-fragile-DM takeover: the gamescope unit CANNOT be `--user start`ed back — without a DM
// login session there is no seat, so gamescope never gets DRM master (unit goes `failed`,
// screen stays black — live-proven on the Nobara repro VM). Restore the DM instead
// (`reset-failed` clears the relogin start-limit accounting, then `restart`); its autologin
// session Exec starts the gamescope unit itself.
if let Some(dm) = dm {
let _ = Command::new("systemctl")
.args(["reset-failed", &dm])
.status();
let restart = Command::new("systemctl")
.args(["restart", &dm])
.status()
.map(|s| s.success())
.unwrap_or(false);
if restart {
tracing::info!(%dm, "restored the display manager (its autologin brings gaming mode back)");
} else if crate::try_recover_session() {
tracing::warn!(
%dm,
"display-manager restart lost its privilege — fired PUNKTFUNK_RECOVER_SESSION_CMD \
to bring the session back"
);
} else {
tracing::error!(
%dm,
"could not restart the display manager and no PUNKTFUNK_RECOVER_SESSION_CMD is \
configured the box has no graphical session until someone runs \
`systemctl reset-failed {dm} && systemctl restart {dm}` as root"
);
}
return;
}
for unit in units { for unit in units {
let _ = Command::new("systemctl") let _ = Command::new("systemctl")
.args(["--user", "start", &unit]) .args(["--user", "start", &unit])
@@ -1226,15 +1742,31 @@ pub fn start_restore_worker() -> std::sync::Arc<()> {
/// session). Shared by the attach and host-managed-session paths. /// session). Shared by the attach and host-managed-session paths.
fn point_injector_at_eis() { fn point_injector_at_eis() {
match find_gamescope_eis_socket() { match find_gamescope_eis_socket() {
Some(sock) => match std::fs::write(ei_socket_file(), &sock) { Some(sock) => {
Ok(()) => { // Relay format: line 1 = socket, optional line 2 = the session's CURRENT output
tracing::info!(socket = %sock, "gamescope: pointed injector at the session's EIS socket") // size as "WxH". gamescope's EIS advertises only a degenerate INT32_MAX region, so
// the injector can't learn the output geometry from the protocol — the hint lets
// it scale normalized client positions correctly even when the client streams at
// a different resolution than the session runs (foreign attach, supersample).
let size = current_gamescope_output_size();
let body = match size {
Some((w, h)) => format!("{sock}\n{w}x{h}"),
None => sock.clone(),
};
match std::fs::write(ei_socket_file(), body) {
Ok(()) => {
tracing::info!(
socket = %sock,
output = ?size,
"gamescope: pointed injector at the session's EIS socket"
)
}
Err(e) => tracing::warn!(
error = %e,
"gamescope: could not write the EIS relay file — input may not reach the session"
),
} }
Err(e) => tracing::warn!( }
error = %e,
"gamescope: could not write the EIS relay file — input may not reach the session"
),
},
None => tracing::warn!( None => tracing::warn!(
"gamescope: no connectable gamescope EIS socket found — input won't reach the session" "gamescope: no connectable gamescope EIS socket found — input won't reach the session"
), ),
@@ -1524,7 +2056,60 @@ impl Drop for GamescopeProc {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{cgroup_is_punktfunk_owned, is_steam_launch, shape_dedicated_command}; use super::{
cgroup_is_punktfunk_owned, connected_connector_under, display_manager_unit_under,
dm_survives_masked_unit, is_steam_launch, shape_dedicated_command,
};
#[test]
fn display_manager_flavor_detection() {
let base = std::env::temp_dir().join(format!("pf-dm-scan-{}", std::process::id()));
std::fs::create_dir_all(&base).unwrap();
// No alias symlink (no DM installed — getty autologin boxes) → None.
assert_eq!(display_manager_unit_under(&base), None);
// The Fedora-style alias symlink resolves to its target's basename (read_link, not
// canonicalize — the target needn't exist on the build box).
std::os::unix::fs::symlink(
"/usr/lib/systemd/system/plasmalogin.service",
base.join("display-manager.service"),
)
.unwrap();
assert_eq!(
display_manager_unit_under(&base).as_deref(),
Some("plasmalogin.service")
);
// Only SDDM is proven to survive a masked session unit; plasmalogin start-limit-kills
// itself (live-proven), and unknown DMs default to fragile.
assert!(dm_survives_masked_unit("sddm.service"));
assert!(!dm_survives_masked_unit("plasmalogin.service"));
assert!(!dm_survives_masked_unit("gdm.service"));
std::fs::remove_dir_all(&base).unwrap();
}
#[test]
fn connector_status_scan() {
let base = std::env::temp_dir().join(format!("pf-drm-scan-{}", std::process::id()));
let mk = |name: &str, status: Option<&str>| {
let dir = base.join(name);
std::fs::create_dir_all(&dir).unwrap();
if let Some(s) = status {
std::fs::write(dir.join("status"), s).unwrap();
}
};
// Headless layout: device + render nodes only (no status files) → not connected.
mk("card0", None);
mk("renderD128", None);
assert!(!connected_connector_under(&base));
// Connectors present but nothing plugged in → still not connected.
mk("card0-HDMI-A-1", Some("disconnected\n"));
assert!(!connected_connector_under(&base));
// A live panel → connected.
mk("card0-eDP-1", Some("connected\n"));
assert!(connected_connector_under(&base));
// A missing base dir (no DRM at all) reads as headless.
assert!(!connected_connector_under(&base.join("nope")));
std::fs::remove_dir_all(&base).unwrap();
}
#[test] #[test]
fn steam_launch_detection() { fn steam_launch_detection() {
@@ -92,11 +92,19 @@ fn next_output_name() -> String {
/// The Hyprland virtual-display driver. Stateless — each [`create`](VirtualDisplay::create) adds one /// The Hyprland virtual-display driver. Stateless — each [`create`](VirtualDisplay::create) adds one
/// named headless output and spins up a portal thread owning the cast on it. /// named headless output and spins up a portal thread owning the cast on it.
pub struct HyprlandDisplay; pub struct HyprlandDisplay {
/// Out-of-band cursor request (`set_hw_cursor`, the negotiated cursor channel): portal
/// `CursorMode::Metadata` — shapes/positions ride `SPA_META_Cursor` for the channel + the
/// composite blend. Off (every non-channel session): `Embedded` — the compositor paints the
/// pointer into frames, zero host-side cursor work (the pre-channel default this backend
/// always had). ⚠️ Metadata is UNTESTED on-glass for this backend (Phase B wired it so the
/// channel isn't silently dead here; KWin/Mutter are the validated legs).
hw_cursor: bool,
}
impl HyprlandDisplay { impl HyprlandDisplay {
pub fn new() -> Result<Self> { pub fn new() -> Result<Self> {
Ok(HyprlandDisplay) Ok(HyprlandDisplay { hw_cursor: false })
} }
} }
@@ -141,6 +149,14 @@ impl VirtualDisplay for HyprlandDisplay {
"hyprland" "hyprland"
} }
fn set_hw_cursor(&mut self, on: bool) {
self.hw_cursor = on;
}
fn hw_cursor(&self) -> bool {
self.hw_cursor
}
fn create(&mut self, mode: Mode) -> Result<VirtualOutput> { fn create(&mut self, mode: Mode) -> Result<VirtualOutput> {
// Log the permission-system caveat once per process (silent black frames otherwise). // Log the permission-system caveat once per process (silent black frames otherwise).
preflight_once(); preflight_once();
@@ -167,9 +183,10 @@ impl VirtualDisplay for HyprlandDisplay {
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>(); let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>();
let stop = Arc::new(AtomicBool::new(false)); let stop = Arc::new(AtomicBool::new(false));
let stop_thread = stop.clone(); let stop_thread = stop.clone();
let hw_cursor = self.hw_cursor;
thread::Builder::new() thread::Builder::new()
.name("punktfunk-hypr-vout".into()) .name("punktfunk-hypr-vout".into())
.spawn(move || portal_thread(setup_tx, stop_thread)) .spawn(move || portal_thread(setup_tx, stop_thread, hw_cursor))
.context("spawn hyprland portal thread")?; .context("spawn hyprland portal thread")?;
let (fd, node_id) = match setup_rx.recv_timeout(Duration::from_secs(20)) { let (fd, node_id) = match setup_rx.recv_timeout(Duration::from_secs(20)) {
@@ -199,6 +216,7 @@ impl VirtualDisplay for HyprlandDisplay {
ownership: DisplayOwnership::Owned, ownership: DisplayOwnership::Owned,
reused_gen: None, reused_gen: None,
pool_gen: None, pool_gen: None,
expect_exact_dims: false,
}) })
} }
} }
@@ -490,7 +508,17 @@ fn ensure_xdph_config() -> Result<()> {
/// stopped (the zbus connection is the cast's lifetime). xdph answers source selection via our /// stopped (the zbus connection is the cast's lifetime). xdph answers source selection via our
/// custom picker, no dialog. (Kept separate from wlroots' copy so each wlr-family backend stays /// custom picker, no dialog. (Kept separate from wlroots' copy so each wlr-family backend stays
/// self-owned per D1; unify if they ever diverge no further.) /// self-owned per D1; unify if they ever diverge no further.)
fn portal_thread(setup_tx: Sender<Result<(OwnedFd, u32), String>>, stop: Arc<AtomicBool>) { fn portal_thread(
setup_tx: Sender<Result<(OwnedFd, u32), String>>,
stop: Arc<AtomicBool>,
hw_cursor: bool,
) {
// Portal cursor mode per the session's channel negotiation (see the struct doc).
let cursor_mode = if hw_cursor {
CursorMode::Metadata
} else {
CursorMode::Embedded
};
use ashpd::desktop::screencast::{CursorMode, Screencast, SelectSourcesOptions, SourceType}; use ashpd::desktop::screencast::{CursorMode, Screencast, SelectSourcesOptions, SourceType};
use ashpd::desktop::PersistMode; use ashpd::desktop::PersistMode;
use ashpd::enumflags2::BitFlags; use ashpd::enumflags2::BitFlags;
@@ -523,7 +551,7 @@ fn portal_thread(setup_tx: Sender<Result<(OwnedFd, u32), String>>, stop: Arc<Ato
.select_sources( .select_sources(
&session, &session,
SelectSourcesOptions::default() SelectSourcesOptions::default()
.set_cursor_mode(CursorMode::Embedded) .set_cursor_mode(cursor_mode)
// xdph offers MONITOR; the custom picker selects our output. // xdph offers MONITOR; the custom picker selects our output.
.set_sources(BitFlags::from_flag(SourceType::Monitor)) .set_sources(BitFlags::from_flag(SourceType::Monitor))
.set_multiple(false) .set_multiple(false)
+634 -150
View File
@@ -56,8 +56,13 @@ use zkde::zkde_screencast_stream_unstable_v1::{
}; };
use zkde::zkde_screencast_unstable_v1::ZkdeScreencastUnstableV1 as Screencast; use zkde::zkde_screencast_unstable_v1::ZkdeScreencastUnstableV1 as Screencast;
/// `pointer` attachment mode (the protocol enum): render the cursor into the stream so the /// `pointer` attachment modes (the protocol enum), chosen per session by `set_hw_cursor`
/// remote sees it move with injected input. /// (Phase B — the Windows no-regression gate mirrored): a CURSOR-CHANNEL session gets METADATA
/// (`SPA_META_Cursor` on the stream — shapes forwarded to the client, the composite flip blends
/// host-side; embedded would leave both with nothing, the round-1 mutter trap), every other
/// session gets EMBEDDED — KWin composites the pointer into frames itself, zero host-side
/// cursor work, the pre-channel path Moonlight/legacy clients always had.
const POINTER_METADATA: u32 = 4;
const POINTER_EMBEDDED: u32 = 2; const POINTER_EMBEDDED: u32 = 2;
/// The name we give the created output; KWin exposes it to output-management as `Virtual-<name>`. /// The name we give the created output; KWin exposes it to output-management as `Virtual-<name>`.
@@ -79,14 +84,24 @@ pub struct KwinDisplay {
/// `None` for shared/anonymous) — reported to the registry via [`last_identity_slot`] so it can key /// `None` for shared/anonymous) — reported to the registry via [`last_identity_slot`] so it can key
/// the group arrangement + `/display/state` slot to the same id this backend named the output with. /// the group arrangement + `/display/state` slot to the same id this backend named the output with.
last_slot: Option<u32>, last_slot: Option<u32>,
/// The base output name the last `create` used (`punktfunk` / `punktfunk-<id>`) — so /// The RESOLVED kscreen address of the last `create`'s output — the numeric kscreen output id
/// [`apply_position`](VirtualDisplay::apply_position) can address the KWin output `Virtual-<name>`. /// when [`resolve_kscreen_addr`] found it, else the `Virtual-<name>` fallback — so
/// [`apply_position`](VirtualDisplay::apply_position) addresses OUR output even while a
/// superseded same-name sibling is still alive.
last_name: Option<String>, last_name: Option<String>,
/// The RESOLVED `kde_output_device_v2` UUID of the last `create`'s output, when the in-process
/// output-management path handled the topology. A stable per-output id (unlike the shared name),
/// so [`apply_position`](VirtualDisplay::apply_position) and restore address exactly OUR output
/// across a supersede — preferred over `last_name`, which is only the kscreen-doctor fallback.
our_uuid: Option<String>,
/// The topology-restore action the last `create` prepared (re-enable the outputs an `exclusive` /// The topology-restore action the last `create` prepared (re-enable the outputs an `exclusive`
/// topology disabled), pending pickup by the registry via [`take_topology_restore`] — so the /// topology disabled), pending pickup by the registry via [`take_topology_restore`] — so the
/// physical is re-enabled only when the display GROUP's last member drops (§6.1), not this session's. /// physical is re-enabled only when the display GROUP's last member drops (§6.1), not this session's.
/// A backstop [`Drop`] runs it if the registry never took it (so a physical is never left dark). /// A backstop [`Drop`] runs it if the registry never took it (so a physical is never left dark).
pending_restore: Option<Box<dyn FnOnce() + Send>>, pending_restore: Option<Box<dyn FnOnce() + Send>>,
/// Out-of-band cursor request (`set_hw_cursor`, i.e. the session negotiated the cursor
/// channel): METADATA pointer mode at creation; off = EMBEDDED (see the consts above).
hw_cursor: bool,
} }
impl Drop for KwinDisplay { impl Drop for KwinDisplay {
@@ -104,6 +119,48 @@ impl KwinDisplay {
pub fn new() -> Result<Self> { pub fn new() -> Result<Self> {
Ok(KwinDisplay::default()) Ok(KwinDisplay::default())
} }
/// Apply the effective display topology for the just-created output `our_prefix` (current size
/// `dims`), preferring the in-process `kde_output_management_v2` path and falling back to
/// `kscreen-doctor` if the compositor doesn't answer in budget or the management global is
/// absent. Records the output's UUID (in-process) or kscreen address (fallback) for
/// [`apply_position`](VirtualDisplay::apply_position), and returns the disabled outputs (each
/// `(name, "WxH@Hz")`) for the group teardown restore. `Extend`/`Auto` disable nothing.
fn apply_topology(
&mut self,
name: &str,
our_prefix: &str,
dims: (u32, u32),
) -> Vec<(String, String)> {
use crate::kwin_output_mgmt::TopologyKind;
use crate::policy::Topology;
let topology = crate::effective_topology();
let kind = match topology {
Topology::Exclusive => TopologyKind::Exclusive,
Topology::Primary => TopologyKind::Primary,
Topology::Extend | Topology::Auto => return Vec::new(),
};
// In-process over Wayland — immune to whatever wedges the standalone kscreen-doctor.
let outcome = crate::kwin_output_mgmt::apply_topology(our_prefix, dims.0, dims.1, kind);
if outcome.handled {
self.our_uuid = outcome.our_uuid;
return outcome.disabled;
}
// Fallback: kscreen-doctor — resolve our address the old way, then shell out the topology.
tracing::info!(
"KWin topology: kde_output_management unavailable — kscreen-doctor fallback"
);
let addr = resolve_kscreen_addr(name, dims.0, dims.1);
self.last_name = Some(addr.clone());
match topology {
Topology::Exclusive => apply_virtual_primary(&addr),
Topology::Primary => {
apply_virtual_primary_only(&addr);
Vec::new()
}
Topology::Extend | Topology::Auto => Vec::new(),
}
}
} }
impl VirtualDisplay for KwinDisplay { impl VirtualDisplay for KwinDisplay {
@@ -123,17 +180,30 @@ impl VirtualDisplay for KwinDisplay {
self.pending_restore.take() self.pending_restore.take()
} }
fn set_hw_cursor(&mut self, on: bool) {
self.hw_cursor = on;
}
fn hw_cursor(&self) -> bool {
self.hw_cursor
}
fn apply_position(&mut self, x: i32, y: i32) { fn apply_position(&mut self, x: i32, y: i32) {
let Some(name) = self.last_name.clone() else { // Prefer the in-process path: address OUR output by its stable UUID (supersede-robust) over
// kde_output_management_v2 — immune to a wedged kscreen-doctor backend (see kwin_output_mgmt).
if let Some(uuid) = self.our_uuid.clone() {
if crate::kwin_output_mgmt::set_position(&uuid, x, y) {
return;
}
}
// Fallback: kscreen-doctor. `last_name` holds the RESOLVED kscreen address (numeric output id,
// or the `Virtual-<name>` fallback) — never re-derive from the name: during a supersede two
// outputs share it and the command would hit the old one (see `create`).
let Some(output) = self.last_name.clone() else {
return; return;
}; };
let output = format!("Virtual-{name}"); // kscreen-doctor position syntax: `output.<name-or-id>.position.<x>,<y>`.
// kscreen-doctor position syntax: `output.<name>.position.<x>,<y>`. let ok = kscreen_ok(&[format!("output.{output}.position.{x},{y}")]);
let ok = std::process::Command::new("kscreen-doctor")
.arg(format!("output.{output}.position.{x},{y}"))
.status()
.map(|s| s.success())
.unwrap_or(false);
if ok { if ok {
tracing::info!(output, x, y, "KWin: placed output in the desktop layout"); tracing::info!(output, x, y, "KWin: placed output in the desktop layout");
} else { } else {
@@ -158,32 +228,129 @@ impl VirtualDisplay for KwinDisplay {
None => VOUT_NAME.to_string(), None => VOUT_NAME.to_string(),
}; };
self.last_name = Some(name.clone()); // for apply_position (registry-driven §6.2 layout) self.last_name = Some(name.clone()); // for apply_position (registry-driven §6.2 layout)
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<u32, String>>();
let stop = Arc::new(AtomicBool::new(false));
let stop_thread = stop.clone();
let (width, height) = (mode.width, mode.height); let (width, height) = (mode.width, mode.height);
let name_thread = name.clone(); let pointer_mode = if self.hw_cursor {
thread::Builder::new() POINTER_METADATA
.name("punktfunk-kwin-vout".into()) } else {
.spawn(move || virtual_output_thread(width, height, name_thread, setup_tx, stop_thread)) POINTER_EMBEDDED
.context("spawn KWin virtual-output thread")?;
let node_id = match setup_rx.recv_timeout(Duration::from_secs(20)) {
Ok(Ok(v)) => v,
Ok(Err(e)) => bail!("KWin virtual output failed: {e}"),
Err(_) => bail!("timed out creating the KWin virtual output"),
}; };
tracing::info!(node_id, width, height, "KWin virtual output ready"); let spawn_vout = |w: u32, h: u32| -> Result<(u32, Arc<AtomicBool>)> {
// KWin creates virtual outputs at a hardcoded 60 Hz and `stream_virtual_output` has no let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<u32, String>>();
// refresh argument, so above 60 Hz we install + select a custom mode (supported on virtual let stop = Arc::new(AtomicBool::new(false));
// outputs since KWin 6.6) before capture connects PipeWire, so the stream negotiates at the let stop_thread = stop.clone();
// higher rate. First cut shells out to kscreen-doctor; the in-process let name_thread = name.clone();
// kde_output_management_v2 client is a follow-up. `set_custom_refresh` reads back and thread::Builder::new()
// returns what KWin *actually* achieved so the encoder paces to the real source rate (a .name("punktfunk-kwin-vout".into())
// rejected custom mode leaves the output at 60 Hz). At ≤60 Hz there's nothing to install — .spawn(move || {
// the source runs 60 Hz and the encoder downsamples — so carry the requested rate through. virtual_output_thread(w, h, name_thread, pointer_mode, setup_tx, stop_thread)
let achieved_hz = if mode.refresh_hz > 60 { })
set_custom_refresh(width, height, mode.refresh_hz, &name) .context("spawn KWin virtual-output thread")?;
match setup_rx.recv_timeout(Duration::from_secs(20)) {
Ok(Ok(v)) => Ok((v, stop)),
Ok(Err(e)) => bail!("KWin virtual output failed: {e}"),
Err(_) => bail!("timed out creating the KWin virtual output"),
}
};
// KWin creates virtual outputs at a hardcoded 60 Hz, `stream_virtual_output` has no
// refresh argument — and its screencast stream builds its PipeWire format offer, INCLUDING
// the `maxFramerate` cap it actively throttles delivery to, ONCE at stream creation
// (screencaststream.cpp `buildFormats`, verified against the live offer with `pw-dump`:
// the offer stays `max=60` after a kscreen mode change, and consumers connecting later
// still negotiate against it). The ONLY path that rebuilds the offer is the stream's own
// resize handling: when the source's texture size changes while recording, KWin re-runs
// `buildFormats` — picking up the output's CURRENT refresh — and renegotiates the live
// stream via `pw_stream_update_params`. So above 60 Hz the output is born at a
// SACRIFICIAL height: installing + selecting the real high-refresh custom mode (supported
// on virtual outputs since KWin 6.6) then changes the SIZE, and the first buffers recorded
// after the consumer connects trigger KWin's resize → a renegotiation to that mode. The
// capturer holds frames until that lands (`expect_exact_dims`), so the pipeline never
// builds against the birth mode. The install/select runs in-process over
// kde_output_management_v2 (`kwin_output_mgmt::set_custom_mode`), with kscreen-doctor
// (`set_custom_refresh`) as the fallback; either reads back what KWin *actually* gave — both
// the rate (so the encoder paces to the real source) and the size, which KWin's CVT
// generator may have aligned down (see `CVT_H_GRANULARITY`). At ≤60 Hz there's nothing to
// install — the output is born at the real size and 60 Hz is the offer anyway.
let want_high = mode.refresh_hz > 60;
let birth_h = if want_high { height + 16 } else { height };
let (mut node_id, mut stop) = spawn_vout(width, birth_h)?;
tracing::info!(
node_id,
width,
height,
birth_h,
embedded_pointer = !self.hw_cursor,
"KWin virtual output ready"
);
// Topology + positioning address OUR output by its kde_output_management UUID (resolved
// in-process in `apply_topology`, supersede-robust) — no early kscreen-doctor resolve, so
// the path never shells out. `Virtual-<name>` is the name KWin exposes our output as.
let our_prefix = format!("Virtual-{name}");
let mut expect_exact_dims = false;
// The size the output actually ENDS UP at — the request, unless KWin's CVT generator had to
// shrink the width to the cell grain (see `CVT_H_GRANULARITY`). Reported as the output's
// `preferred_mode`, which is what the capturer's renegotiation gate waits for and what the
// encoder opens against, so a CVT-aligned mode flows end-to-end instead of starving.
let mut final_dims = (width, height);
let achieved_hz = if want_high {
// >60 Hz needs the real high-refresh custom mode installed + selected (sacrificial-birth,
// see above). In-process over kde_output_management_v2 first (no kscreen-doctor); fall
// back to the kscreen-doctor shell-out on pre-6.6 KWin (no `set_custom_modes`) or if the
// compositor doesn't answer in budget.
let active = crate::kwin_output_mgmt::set_custom_mode(
&our_prefix,
width,
birth_h,
width,
height,
mode.refresh_hz,
)
.or_else(|| {
// ⚠️ ADDRESS BY NUMERIC KSCREEN ID, NEVER BY NAME: a supersede reuses the per-slot
// name while the superseded sibling is still alive, so a name-addressed kscreen
// command hits the OLD output. Resolve our kscreen id for the shell-out install +
// keep it as apply_position's kscreen fallback.
let addr = resolve_kscreen_addr(&name, width, birth_h);
self.last_name = Some(addr.clone());
set_custom_refresh(width, height, mode.refresh_hz, &addr)
});
// Accept only an active mode that IS our custom one: the exact requested height, and a
// width at or just below the request (a CVT alignment). That also proves the output
// left the sacrificial birth size, so the recording stream will renegotiate to it.
match active {
Some((aw, ah, ahz))
if ah == height && aw <= width && width - aw < CVT_H_GRANULARITY =>
{
expect_exact_dims = true;
final_dims = (aw, ah);
ahz
}
other => {
// Custom-mode install/select rejected (pre-6.6 KWin / stale kscreen-doctor): the
// output is STUCK at the sacrificial birth size — unusable. Recreate plain at the
// real size (the pre-sacrifice behavior: correct size, KWin's native 60 Hz).
tracing::warn!(
active = ?other,
requested_w = width,
requested_h = height,
requested_hz = mode.refresh_hz,
"KWin rejected the custom mode — recreating the virtual output at the real \
size (60 Hz ceiling on this KWin)"
);
stop.store(true, Ordering::Relaxed);
// Let KWin retire the doomed output before re-using its name.
std::thread::sleep(Duration::from_millis(300));
let (nid, st) = spawn_vout(width, height)?;
node_id = nid;
stop = st;
tracing::info!(
node_id,
width,
height,
"KWin virtual output ready (fallback)"
);
60
}
}
} else { } else {
mode.refresh_hz mode.refresh_hz
}; };
@@ -191,16 +358,16 @@ impl VirtualDisplay for KwinDisplay {
// `Primary` makes it the primary output but keeps the bootstrap/physical outputs enabled; // `Primary` makes it the primary output but keeps the bootstrap/physical outputs enabled;
// `Exclusive` makes it the SOLE desktop (others disabled, restored on teardown) — so // `Exclusive` makes it the SOLE desktop (others disabled, restored on teardown) — so
// plasmashell + windows land on the streamed surface, not the headless `kwin --virtual` // plasmashell + windows land on the streamed surface, not the headless `kwin --virtual`
// bootstrap output. Read from the policy (replacing the PUNKTFUNK_KWIN_VIRTUAL_PRIMARY boolean). // bootstrap output. Applied over kde_output_management_v2 in-process (immune to a wedged
use crate::policy::Topology; // kscreen-doctor backend; see `apply_topology`), with a kscreen-doctor fallback. `disabled`
let disabled = match crate::effective_topology() { // is the physical/bootstrap outputs, each `(name, "WxH@Hz")`, to restore on teardown.
Topology::Exclusive => apply_virtual_primary(&name), let disabled = self.apply_topology(&name, &our_prefix, final_dims);
Topology::Primary => { // A plain managed name is enough for apply_position's kscreen-doctor fallback when the
apply_virtual_primary_only(&name); // in-process UUID path isn't set (single-output sessions are unambiguous; a supersede uses
Vec::new() // nothing disabled → nothing to restore // the UUID path instead). `want_high` already set `last_name` to the resolved kscreen id.
} if self.last_name.is_none() {
Topology::Extend | Topology::Auto => Vec::new(), self.last_name = Some(our_prefix);
}; }
// Per-group restore (§6.1): DON'T bind the re-enable to this session's keepalive (a per-session // Per-group restore (§6.1): DON'T bind the re-enable to this session's keepalive (a per-session
// `StopGuard` restore would re-enable the physical the moment the FIRST of several exclusive // `StopGuard` restore would re-enable the physical the moment the FIRST of several exclusive
// sessions drops — under a still-live sibling). Instead stash it as a closure the registry lifts // sessions drops — under a still-live sibling). Instead stash it as a closure the registry lifts
@@ -208,22 +375,31 @@ impl VirtualDisplay for KwinDisplay {
// that display's output is reclaimed, so KWin never sees zero outputs). Empty ⇒ nothing to restore. // that display's output is reclaimed, so KWin never sees zero outputs). Empty ⇒ nothing to restore.
self.pending_restore = (!disabled.is_empty()).then(|| { self.pending_restore = (!disabled.is_empty()).then(|| {
let disabled = disabled.clone(); let disabled = disabled.clone();
Box::new(move || reenable_outputs(&disabled)) as Box<dyn FnOnce() + Send> // In-process first; fall back to kscreen-doctor if the compositor doesn't answer in budget.
Box::new(move || {
if !crate::kwin_output_mgmt::reenable_outputs(&disabled) {
reenable_outputs_kscreen(&disabled);
}
}) as Box<dyn FnOnce() + Send>
}); });
// Layout position (§6.2) is applied by the registry via `apply_position` right after create // Layout position (§6.2) is applied by the registry via `apply_position` right after create
// (it owns the display group, so it computes auto-row / manual placement over the whole group). // (it owns the display group, so it computes auto-row / manual placement over the whole group).
Ok(VirtualOutput::owned( let mut out = VirtualOutput::owned(
node_id, node_id,
Some((mode.width, mode.height, achieved_hz)), Some((final_dims.0, final_dims.1, achieved_hz)),
Box::new(StopGuard { stop }), Box::new(StopGuard { stop }),
)) );
out.expect_exact_dims = expect_exact_dims;
Ok(out)
} }
} }
/// Re-enable the outputs an `exclusive` topology disabled (bootstrap / physical), so KWin re-homes onto /// Re-enable the outputs an `exclusive` topology disabled (bootstrap / physical) via `kscreen-doctor`
/// them. Called by the registry when the display group's last member is torn down (design §6.1), BEFORE /// the fallback for the in-process [`crate::kwin_output_mgmt::reenable_outputs`], run by the restore
/// that member's output is reclaimed — so KWin is never momentarily left with zero enabled outputs. /// closure only when the in-process path reports the compositor didn't answer. Called by the registry
fn reenable_outputs(outputs: &[(String, String)]) { /// when the display group's last member is torn down (design §6.1), BEFORE that member's output is
/// reclaimed — so KWin is never momentarily left with zero enabled outputs.
fn reenable_outputs_kscreen(outputs: &[(String, String)]) {
if outputs.is_empty() { if outputs.is_empty() {
return; return;
} }
@@ -234,9 +410,7 @@ fn reenable_outputs(outputs: &[(String, String)]) {
.iter() .iter()
.map(|(name, _)| format!("output.{name}.enable")) .map(|(name, _)| format!("output.{name}.enable"))
.collect(); .collect();
let _ = std::process::Command::new("kscreen-doctor") let _ = kscreen_ok(&enable_args);
.args(&enable_args)
.status();
// THEN re-assert each captured mode, best-effort — a bare re-enable lets KWin fall back to the // THEN re-assert each captured mode, best-effort — a bare re-enable lets KWin fall back to the
// EDID-preferred mode (a 120 Hz panel returns at ~60 Hz); this restores the exact refresh. The // EDID-preferred mode (a 120 Hz panel returns at ~60 Hz); this restores the exact refresh. The
// output is enabled now, so the mode set is valid; a rejected mode just leaves KWin's default. // output is enabled now, so the mode set is valid; a rejected mode just leaves KWin's default.
@@ -246,55 +420,298 @@ fn reenable_outputs(outputs: &[(String, String)]) {
.map(|(name, mode)| format!("output.{name}.mode.{mode}")) .map(|(name, mode)| format!("output.{name}.mode.{mode}"))
.collect(); .collect();
if !mode_args.is_empty() { if !mode_args.is_empty() {
let _ = std::process::Command::new("kscreen-doctor") let _ = kscreen_ok(&mode_args);
.args(&mode_args)
.status();
} }
std::thread::sleep(Duration::from_millis(200)); std::thread::sleep(Duration::from_millis(200));
tracing::info!(reenabled = ?outputs, "KWin: restored the physical/bootstrap outputs at their captured modes (group empty)"); tracing::info!(reenabled = ?outputs, "KWin: restored the physical/bootstrap outputs at their captured modes (group empty)");
} }
/// Best-effort: raise the just-created virtual output's refresh above KWin's default 60 Hz by /// Resolve the kscreen address of the virtual output the host JUST created: the managed-prefix
/// installing + selecting a custom mode via `kscreen-doctor` (the output is `Virtual-<VOUT_NAME>`, /// name alone is ambiguous during a supersede (the replacement deliberately reuses the per-slot
/// refresh given in mHz), then **read back the active mode** and return the refresh KWin actually /// name while the superseded sibling is still alive), so match on the birth mode's size too —
/// gave us. The apply command can report success yet leave the output at 60 Hz (mode rejected), /// only the just-created output sits at the sacrificial `(w, h)` — and prefer the HIGHEST output
/// and a silent rate mismatch surfaces downstream as judder / duplicated frames — so the caller /// id (the newest) if several match. Returns the numeric id as a string (kscreen-doctor accepts
/// paces the encoder to the *achieved* rate, not the requested one. /// `output.<id>.…`), falling back to the ambiguous `Virtual-<name>` if the output hasn't reached
fn set_custom_refresh(width: u32, height: u32, hz: u32, name: &str) -> u32 { /// kscreen's model yet after a few tries (single-output sessions are unambiguous anyway).
let output = format!("Virtual-{name}"); fn resolve_kscreen_addr(name: &str, w: u32, h: u32) -> String {
let mhz = hz.saturating_mul(1000); let fallback = format!("Virtual-{name}");
let run = |arg: String| { for attempt in 0..3 {
std::process::Command::new("kscreen-doctor") if attempt > 0 {
.arg(arg) std::thread::sleep(Duration::from_millis(150));
.status()
.map(|s| s.success())
.unwrap_or(false)
};
// Add the custom mode (a fresh output has none), then select it.
let _ = run(format!(
"output.{output}.addCustomMode.{width}.{height}.{mhz}.full"
));
let applied = run(format!("output.{output}.mode.{width}x{height}@{hz}"));
match read_active_refresh(&output) {
Some(achieved) if achieved >= hz => {
tracing::info!(
output,
requested = hz,
achieved,
"KWin virtual output: custom refresh applied"
);
achieved
} }
Some(achieved) => { let Some(doc) = kscreen_json() else { continue };
let Some(outputs) = doc.get("outputs").and_then(|o| o.as_array()) else {
continue;
};
let best = outputs
.iter()
.filter(|o| {
o.get("name")
.and_then(|n| n.as_str())
.is_some_and(|n| n.starts_with(&fallback))
&& output_active_size(o) == Some((w, h))
})
.filter_map(|o| o.get("id").and_then(|i| i.as_u64()))
.max();
if let Some(id) = best {
tracing::info!(id, name, w, h, "KWin: resolved the new output's kscreen id");
return id.to_string();
}
}
tracing::warn!(
name,
w,
h,
"KWin: could not resolve the new output's kscreen id — falling back to name addressing \
(ambiguous during a mode-switch supersede)"
);
fallback
}
/// Budget for one `kscreen-doctor` call.
///
/// It is a Wayland client of the very compositor it configures, so against a wedged KWin it blocks
/// in its own connect and never returns — and these calls run on the session's stream thread, whose
/// only way to end a session is to return. Generous next to a healthy call (tens of ms).
const KSCREEN_BUDGET: Duration = Duration::from_secs(5);
/// `kscreen-doctor <args>` run for its exit status, bounded by [`KSCREEN_BUDGET`]. A timeout reads
/// as a failed apply — the same best-effort path a rejected argument already takes.
fn kscreen_ok(args: &[String]) -> bool {
crate::proc::status_within(
std::process::Command::new("kscreen-doctor").args(args),
KSCREEN_BUDGET,
)
.map(|s| s.success())
.unwrap_or(false)
}
/// `kscreen-doctor -j` stdout, bounded by [`KSCREEN_BUDGET`]; `None` on any failure.
fn kscreen_json_bytes() -> Option<Vec<u8>> {
crate::proc::output_within(
std::process::Command::new("kscreen-doctor").arg("-j"),
KSCREEN_BUDGET,
)
.ok()
.map(|o| o.stdout)
}
/// `kscreen-doctor -j` parsed, `None` on any failure.
fn kscreen_json() -> Option<serde_json::Value> {
serde_json::from_slice(&kscreen_json_bytes()?).ok()
}
/// The `(width, height)` of an output's CURRENT mode from its `kscreen-doctor -j` entry.
fn output_active_size(o: &serde_json::Value) -> Option<(u32, u32)> {
let as_id = |v: &serde_json::Value| -> Option<String> {
v.as_str()
.map(|s| s.to_string())
.or_else(|| v.as_u64().map(|n| n.to_string()))
};
let current = o.get("currentModeId").and_then(as_id)?;
let mode = o
.get("modes")?
.as_array()?
.iter()
.find(|m| m.get("id").and_then(as_id).as_deref() == Some(current.as_str()))?;
let size = mode.get("size")?;
Some((
size.get("width").and_then(|v| v.as_u64())? as u32,
size.get("height").and_then(|v| v.as_u64())? as u32,
))
}
/// CVT's horizontal cell granularity. KWin generates every custom mode's timing with **libxcvt**,
/// whose first step is `hdisplay_rnd = hdisplay - (hdisplay % 8)` — so a width that isn't a multiple
/// of 8 comes back NARROWER than asked, and the clock-step rounding that follows lands a fractional
/// refresh. A 2868x1320@120 request (an iPhone 16 Pro Max panel) becomes **2864x1320@119.92**.
///
/// That is why a custom mode must never be selected by the `WxH@Hz` string we *requested*:
/// kscreen-doctor's `findMode` matches a mode's id or its own `WxH@qRound(Hz)` name, so
/// `2868x1320@120` matches nothing, the select silently no-ops, the output stays on its sacrificial
/// birth mode, and the caller falls back to 60 Hz — while KDE's display list shows the perfectly
/// good 2864x1320@119.92 mode sitting there unselected. Widths like 1920/2560/3840 are all
/// multiples of 8, which is why only phone-shaped clients ever hit it.
const CVT_H_GRANULARITY: u32 = 8;
/// One row of an output's mode list, as parsed from `kscreen-doctor -j`.
#[derive(Clone, Debug, PartialEq)]
struct KModeRow {
/// kscreen's mode id — what we address the mode by (never the requested `WxH@Hz` string).
id: String,
w: u32,
h: u32,
hz: f64,
}
/// A kscreen JSON id, which is a string on some KWin versions and a number on others.
fn json_id(v: &serde_json::Value) -> Option<String> {
v.as_str()
.map(|s| s.to_string())
.or_else(|| v.as_u64().map(|n| n.to_string()))
}
/// The full mode list of `output` (a RESOLVED kscreen address — numeric id or name) from a parsed
/// `kscreen-doctor -j` document. Split from the process call so the picker can be tested on
/// captured JSON.
fn modes_from_json(doc: &serde_json::Value, output: &str) -> Vec<KModeRow> {
let Some(o) = doc
.get("outputs")
.and_then(|v| v.as_array())
.and_then(|outs| {
outs.iter().find(|o| {
o.get("name").and_then(|n| n.as_str()) == Some(output)
|| o.get("id").and_then(json_id).as_deref() == Some(output)
})
})
else {
return Vec::new();
};
o.get("modes")
.and_then(|m| m.as_array())
.map(|ms| {
ms.iter()
.filter_map(|m| {
let size = m.get("size")?;
Some(KModeRow {
id: m.get("id").and_then(json_id)?,
w: size.get("width").and_then(|v| v.as_u64())? as u32,
h: size.get("height").and_then(|v| v.as_u64())? as u32,
hz: m.get("refreshRate").and_then(|r| r.as_f64())?,
})
})
.collect()
})
.unwrap_or_default()
}
/// [`modes_from_json`] against a live `kscreen-doctor -j`.
fn output_modes(output: &str) -> Vec<KModeRow> {
kscreen_json()
.map(|doc| modes_from_json(&doc, output))
.unwrap_or_default()
}
/// The mode in `modes` that actually fulfils a `width`x`height`@`hz` request, tolerating the CVT
/// alignment KWin applies when it generates the timing (see [`CVT_H_GRANULARITY`]): the height must
/// match exactly (CVT never touches the vertical active), the width may be up to one cell narrower
/// than asked (never wider — that would be a different mode), and the refresh must land within 1 Hz
/// of the request (which excludes the output's native 60 Hz entry for every rate we install a custom
/// mode for). Widest wins, then fastest — so an exact-width mode always beats an aligned one, and a
/// list carrying duplicate custom modes from earlier sessions still resolves.
fn pick_custom_mode<'a>(
modes: &'a [KModeRow],
width: u32,
height: u32,
hz: u32,
) -> Option<&'a KModeRow> {
modes
.iter()
.filter(|m| {
m.h == height
&& m.w <= width
&& width - m.w < CVT_H_GRANULARITY
&& (m.hz - f64::from(hz)).abs() < 1.0
})
.max_by(|a, b| {
a.w.cmp(&b.w)
.then(a.hz.partial_cmp(&b.hz).unwrap_or(std::cmp::Ordering::Equal))
})
}
/// Best-effort: install + select the `width`x`height`@`hz` custom mode on the just-created virtual
/// output via `kscreen-doctor` (`output` is the RESOLVED kscreen address — numeric id or name, see
/// [`resolve_kscreen_addr`] — refresh given in mHz), then **read back the active mode** and return
/// it as `(width, height, refresh_hz)`. `None` if the read-back failed entirely.
///
/// The apply command can report success yet leave the output on its old mode (rejected), and a
/// silent size/rate mismatch surfaces downstream as a starved capture gate or judder — so the
/// caller drives the pipeline off the *achieved* mode, not the requested one. The mode is selected
/// by kscreen **mode id** resolved from the output's own list, never by the requested `WxH@Hz`
/// string, because KWin's CVT generator may hand back a slightly different one
/// ([`CVT_H_GRANULARITY`]).
fn set_custom_refresh(width: u32, height: u32, hz: u32, output: &str) -> Option<(u32, u32, u32)> {
let output = output.to_string();
let mhz = hz.saturating_mul(1000);
let run = |arg: String| kscreen_ok(&[arg]);
// Install the mode only if the output doesn't already carry a usable one: kscreen-doctor
// APPENDS to the output's custom-mode list and KWin PERSISTS that list per output name
// (`kwinoutputconfig.json`, which is why the same per-slot name is reused across sessions) — so
// re-adding on every connect would grow the user's display list without bound.
let mut modes = output_modes(&output);
if pick_custom_mode(&modes, width, height, hz).is_none() {
let _ = run(format!(
"output.{output}.addCustomMode.{width}.{height}.{mhz}.full"
));
modes = output_modes(&output);
}
let applied = match pick_custom_mode(&modes, width, height, hz) {
Some(target) => {
if (target.w, target.h) != (width, height) {
tracing::info!(
output,
requested_w = width,
requested_h = height,
mode_w = target.w,
mode_h = target.h,
mode_hz = target.hz,
"KWin aligned the custom mode to the CVT cell grain — streaming at its size"
);
}
// By id first; the human `WxH@Hz` form (built from the mode's OWN size/refresh, not the
// request) is the fallback for builds whose ids don't round-trip through the CLI.
run(format!("output.{output}.mode.{}", target.id))
|| run(format!(
"output.{output}.mode.{}x{}@{}",
target.w,
target.h,
target.hz.round() as u32
))
}
None => {
tracing::warn!( tracing::warn!(
output, output,
requested = hz, requested_w = width,
achieved, requested_h = height,
applied, requested_hz = hz,
"KWin virtual output refresh below requested — pacing the encoder to the achieved \ offered = ?modes,
rate (custom-mode install rejected? is kscreen-doctor up to date?)" "KWin offers no mode matching the request after addCustomMode — is kscreen-doctor \
up to date, and KWin 6.6 (custom modes on virtual outputs)?"
); );
achieved.max(1) false
}
};
match read_active_mode(&output) {
Some((w, h, achieved)) => {
if achieved >= hz && (w, h) == (width, height) {
tracing::info!(
output,
requested = hz,
achieved,
"KWin virtual output: custom refresh applied"
);
} else if achieved >= hz {
tracing::info!(
output,
requested = hz,
achieved,
active_w = w,
active_h = h,
"KWin virtual output: custom refresh applied at a CVT-aligned size"
);
} else {
tracing::warn!(
output,
requested = hz,
achieved,
active_w = w,
active_h = h,
applied,
"KWin virtual output mode below requested — pacing the encoder to the \
achieved rate (custom-mode install rejected? is kscreen-doctor up to date?)"
);
}
Some((w, h, achieved.max(1)))
} }
None => { None => {
tracing::warn!( tracing::warn!(
@@ -304,38 +721,37 @@ fn set_custom_refresh(width: u32, height: u32, hz: u32, name: &str) -> u32 {
"could not read back KWin virtual output refresh — assuming 60 Hz (is \ "could not read back KWin virtual output refresh — assuming 60 Hz (is \
kscreen-doctor installed?)" kscreen-doctor installed?)"
); );
60 None
} }
} }
} }
/// Read the active refresh (Hz, rounded) of `output` from `kscreen-doctor -j`. `None` if the /// Read the active mode (`(width, height, refresh_hz)`, Hz rounded) of `output` — a RESOLVED
/// tool, the output, or its current mode can't be found. Mode/output ids come through as either /// kscreen address (numeric id or name, see [`resolve_kscreen_addr`]) — from `kscreen-doctor -j`.
/// JSON strings or numbers depending on the KWin version, so both are accepted. /// `None` if the tool, the output, or its current mode can't be found. Mode/output ids come
fn read_active_refresh(output: &str) -> Option<u32> { /// through as either JSON strings or numbers depending on the KWin version, so both are accepted.
let out = std::process::Command::new("kscreen-doctor") fn read_active_mode(output: &str) -> Option<(u32, u32, u32)> {
.arg("-j") let doc = kscreen_json()?;
.output()
.ok()?;
let doc: serde_json::Value = serde_json::from_slice(&out.stdout).ok()?;
let as_id = |v: &serde_json::Value| -> Option<String> { let as_id = |v: &serde_json::Value| -> Option<String> {
v.as_str() v.as_str()
.map(|s| s.to_string()) .map(|s| s.to_string())
.or_else(|| v.as_u64().map(|n| n.to_string())) .or_else(|| v.as_u64().map(|n| n.to_string()))
}; };
let o = doc let o = doc.get("outputs")?.as_array()?.iter().find(|o| {
.get("outputs")? o.get("name").and_then(|n| n.as_str()) == Some(output)
.as_array()? || o.get("id").and_then(as_id).as_deref() == Some(output)
.iter() })?;
.find(|o| o.get("name").and_then(|n| n.as_str()) == Some(output))?;
let current = o.get("currentModeId").and_then(as_id)?; let current = o.get("currentModeId").and_then(as_id)?;
let mode = o let mode = o
.get("modes")? .get("modes")?
.as_array()? .as_array()?
.iter() .iter()
.find(|m| m.get("id").and_then(as_id).as_deref() == Some(current.as_str()))?; .find(|m| m.get("id").and_then(as_id).as_deref() == Some(current.as_str()))?;
let size = mode.get("size")?;
let w = size.get("width").and_then(|v| v.as_u64())? as u32;
let h = size.get("height").and_then(|v| v.as_u64())? as u32;
let hz = mode.get("refreshRate").and_then(|r| r.as_f64())?; let hz = mode.get("refreshRate").and_then(|r| r.as_f64())?;
Some(hz.round() as u32) Some((w, h, hz.round() as u32))
} }
/// The prefix EVERY managed KWin output shares — Stage 3 names them `punktfunk` / `punktfunk-<id>`, /// The prefix EVERY managed KWin output shares — Stage 3 names them `punktfunk` / `punktfunk-<id>`,
@@ -374,16 +790,13 @@ fn output_current_mode_spec(o: &serde_json::Value) -> Option<String> {
/// bare re-enable drops a 120 Hz panel to KWin's default ~60 Hz). /// bare re-enable drops a 120 Hz panel to KWin's default ~60 Hz).
/// **Group-aware (§6.1):** excludes the WHOLE managed family (the [`MANAGED_PREFIX`]), not just this /// **Group-aware (§6.1):** excludes the WHOLE managed family (the [`MANAGED_PREFIX`]), not just this
/// session's own output — so a 2nd `exclusive` session (with a distinct per-slot name) never disables /// session's own output — so a 2nd `exclusive` session (with a distinct per-slot name) never disables
/// the 1st session's live output. Parsed from `kscreen-doctor -j` (same source as [`read_active_refresh`]). /// the 1st session's live output. Parsed from `kscreen-doctor -j` (same source as [`read_active_mode`]).
fn other_enabled_outputs() -> Vec<(String, String)> { fn other_enabled_outputs() -> Vec<(String, String)> {
let out = match std::process::Command::new("kscreen-doctor") let out = match kscreen_json_bytes() {
.arg("-j") Some(o) => o,
.output() None => return Vec::new(),
{
Ok(o) => o,
Err(_) => return Vec::new(),
}; };
let doc: serde_json::Value = match serde_json::from_slice(&out.stdout) { let doc: serde_json::Value = match serde_json::from_slice(&out) {
Ok(d) => d, Ok(d) => d,
Err(_) => return Vec::new(), Err(_) => return Vec::new(),
}; };
@@ -412,13 +825,10 @@ fn other_enabled_outputs() -> Vec<(String, String)> {
/// then sets itself primary — the pre-group behavior). Recent kscreen marks the primary with /// then sets itself primary — the pre-group behavior). Recent kscreen marks the primary with
/// `"priority": 1`; older builds used a `"primary": true` bool — accept either. /// `"priority": 1`; older builds used a `"primary": true` bool — accept either.
fn a_managed_output_is_primary() -> bool { fn a_managed_output_is_primary() -> bool {
let Ok(out) = std::process::Command::new("kscreen-doctor") let Some(out) = kscreen_json_bytes() else {
.arg("-j")
.output()
else {
return false; return false;
}; };
let Ok(doc) = serde_json::from_slice::<serde_json::Value>(&out.stdout) else { let Ok(doc) = serde_json::from_slice::<serde_json::Value>(&out) else {
return false; return false;
}; };
doc.get("outputs") doc.get("outputs")
@@ -437,19 +847,13 @@ fn a_managed_output_is_primary() -> bool {
.unwrap_or(false) .unwrap_or(false)
} }
/// Set `Virtual-punktfunk` primary and disable the bootstrap output(s) so the managed group becomes /// Set our output primary and disable the bootstrap output(s) so the managed group becomes
/// the sole desktop (KWin re-homes plasmashell + windows onto it). Returns the disabled outputs for /// the sole desktop (KWin re-homes plasmashell + windows onto it). `ours` is the RESOLVED kscreen
/// address (numeric id or name, see [`resolve_kscreen_addr`]). Returns the disabled outputs for
/// the keepalive to re-enable on teardown. Best-effort: on failure, streaming continues (just possibly /// the keepalive to re-enable on teardown. Best-effort: on failure, streaming continues (just possibly
/// showing only the wallpaper) rather than failing the session. /// showing only the wallpaper) rather than failing the session.
fn apply_virtual_primary(name: &str) -> Vec<(String, String)> { fn apply_virtual_primary(ours: &str) -> Vec<(String, String)> {
let ours = format!("Virtual-{name}"); let kscreen = |args: &[String]| kscreen_ok(args);
let kscreen = |args: &[String]| {
std::process::Command::new("kscreen-doctor")
.args(args)
.status()
.map(|s| s.success())
.unwrap_or(false)
};
// First-slot-wins (§6.1): only grab primary if no managed group member is primary yet — so a 2nd // First-slot-wins (§6.1): only grab primary if no managed group member is primary yet — so a 2nd
// exclusive session joins as a secondary monitor of the shared desktop instead of stealing the // exclusive session joins as a secondary monitor of the shared desktop instead of stealing the
// shell off the 1st session's output. KWin usually then re-homes the desktop + disables the // shell off the 1st session's output. KWin usually then re-homes the desktop + disables the
@@ -480,13 +884,8 @@ fn apply_virtual_primary(name: &str) -> Vec<(String, String)> {
/// **Primary** (Stage 2): make the streamed output the primary but KEEP the other outputs enabled /// **Primary** (Stage 2): make the streamed output the primary but KEEP the other outputs enabled
/// (don't disable the bootstrap/physical) — so the shell re-homes onto the streamed surface while a /// (don't disable the bootstrap/physical) — so the shell re-homes onto the streamed surface while a
/// physical screen stays usable. Nothing to restore on teardown (we disabled nothing). /// physical screen stays usable. Nothing to restore on teardown (we disabled nothing).
fn apply_virtual_primary_only(name: &str) { fn apply_virtual_primary_only(ours: &str) {
let ours = format!("Virtual-{name}"); let ok = kscreen_ok(&[format!("output.{ours}.primary")]);
let ok = std::process::Command::new("kscreen-doctor")
.arg(format!("output.{ours}.primary"))
.status()
.map(|s| s.success())
.unwrap_or(false);
if ok { if ok {
tracing::info!("KWin: streamed output set primary (physical outputs kept)"); tracing::info!("KWin: streamed output set primary (physical outputs kept)");
} else { } else {
@@ -496,8 +895,9 @@ fn apply_virtual_primary_only(name: &str) {
/// Dropping this releases the KWin virtual output: it flips the keepalive thread's `stop`, which /// Dropping this releases the KWin virtual output: it flips the keepalive thread's `stop`, which
/// drops the Wayland connection and makes KWin reclaim the output. The topology **restore** is no /// drops the Wayland connection and makes KWin reclaim the output. The topology **restore** is no
/// longer bound here — it moved to the registry's display group (§6.1, [`reenable_outputs`]), which /// longer bound here — it moved to the registry's display group (§6.1, restored in-process via
/// runs it once when the group's last member drops, BEFORE this keepalive is dropped. /// [`crate::kwin_output_mgmt::reenable_outputs`], `kscreen-doctor` fallback), which runs it once when
/// the group's last member drops, BEFORE this keepalive is dropped.
struct StopGuard { struct StopGuard {
stop: Arc<AtomicBool>, stop: Arc<AtomicBool>,
} }
@@ -578,10 +978,11 @@ fn virtual_output_thread(
width: u32, width: u32,
height: u32, height: u32,
name: String, name: String,
pointer_mode: u32,
setup_tx: Sender<Result<u32, String>>, setup_tx: Sender<Result<u32, String>>,
stop: Arc<AtomicBool>, stop: Arc<AtomicBool>,
) { ) {
if let Err(e) = run(width, height, &name, &setup_tx, &stop) { if let Err(e) = run(width, height, &name, pointer_mode, &setup_tx, &stop) {
// If we never delivered a node id, report the failure to the waiting opener. // If we never delivered a node id, report the failure to the waiting opener.
let _ = setup_tx.send(Err(format!("{e:#}"))); let _ = setup_tx.send(Err(format!("{e:#}")));
} }
@@ -622,6 +1023,7 @@ fn run(
width: u32, width: u32,
height: u32, height: u32,
name: &str, name: &str,
pointer_mode: u32,
setup_tx: &Sender<Result<u32, String>>, setup_tx: &Sender<Result<u32, String>>,
stop: &AtomicBool, stop: &AtomicBool,
) -> Result<()> { ) -> Result<()> {
@@ -642,13 +1044,14 @@ fn run(
) )
})?; })?;
// Create the virtual output sized to the client, cursor composited into the stream. // Create the virtual output sized to the client; the pointer rides as stream metadata
// (cursor-channel session) or KWin embeds it into frames (everyone else — see the consts).
let stream = screencast.stream_virtual_output( let stream = screencast.stream_virtual_output(
name.to_string(), name.to_string(),
width as i32, width as i32,
height as i32, height as i32,
1.0, // scale (logical == physical) 1.0, // scale (logical == physical)
POINTER_EMBEDDED, pointer_mode,
&qh, &qh,
(), (),
); );
@@ -716,7 +1119,88 @@ fn run(
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::MANAGED_PREFIX; use super::{modes_from_json, pick_custom_mode, KModeRow, MANAGED_PREFIX};
fn row(id: &str, w: u32, h: u32, hz: f64) -> KModeRow {
KModeRow {
id: id.to_string(),
w,
h,
hz,
}
}
/// The reported regression: an iPhone 16 Pro Max asks for 2868x1320@120; libxcvt rounds the
/// width down to the 8-pixel cell grain and the clock step lands 119.92, so KWin's list holds
/// 2864x1320@119.92. Selecting by the REQUESTED `2868x1320@120` string matched nothing — the
/// output stayed on its birth mode and the session fell back to 60 Hz. The picker must find it.
#[test]
fn picks_the_cvt_aligned_mode() {
let modes = [
row("1", 2868, 1320, 60.0), // the virtual output's native/birth mode
row("2", 2864, 1320, 119.92), // the custom mode KWin actually generated
];
let got = pick_custom_mode(&modes, 2868, 1320, 120).expect("CVT-aligned mode");
assert_eq!((got.id.as_str(), got.w, got.h), ("2", 2864, 1320));
}
/// A width already on the cell grain (every PC resolution: 1920/2560/3840) round-trips exactly,
/// and an exact-width mode outranks an aligned one when both are offered.
#[test]
fn exact_width_outranks_an_aligned_one() {
let modes = [
row("1", 2560, 1440, 60.0),
row("2", 2552, 1440, 119.93), // a stale narrower custom mode from an earlier session
row("3", 2560, 1440, 119.98),
];
let got = pick_custom_mode(&modes, 2560, 1440, 120).expect("exact mode");
assert_eq!(got.id, "3");
}
/// The picker must never wander onto an unrelated mode: not the 60 Hz native entry (the old
/// fallback the reporter got stuck on), not a different height, not a wider width, and not a
/// mode more than one cell narrower than asked.
#[test]
fn rejects_modes_that_are_not_the_request() {
let modes = [
row("1", 2868, 1320, 60.0), // native — refresh too far off
row("2", 2868, 1080, 119.92), // wrong height
row("3", 2880, 1320, 119.92), // wider than requested
row("4", 2856, 1320, 119.92), // two cells narrower — not a CVT alignment of 2868
row("5", 1920, 1080, 120.0), // unrelated
];
assert!(pick_custom_mode(&modes, 2868, 1320, 120).is_none());
}
/// Mode + output ids come through as JSON strings on some KWin versions and numbers on others;
/// both must parse, and a mode row missing its size/refresh is skipped rather than poisoning
/// the list.
#[test]
fn parses_both_id_encodings() {
let doc: serde_json::Value = serde_json::from_str(
r#"{"outputs":[
{"id":7,"name":"Virtual-punktfunk","modes":[
{"id":"m1","size":{"width":2868,"height":1320},"refreshRate":60.0},
{"id":42,"size":{"width":2864,"height":1320},"refreshRate":119.92},
{"id":"broken","size":{"width":800}}
]},
{"id":1,"name":"eDP-1","modes":[
{"id":"x","size":{"width":2864,"height":1320},"refreshRate":119.92}
]}
]}"#,
)
.expect("fixture parses");
// Addressable by numeric id (how `resolve_kscreen_addr` returns it) and by name.
for addr in ["7", "Virtual-punktfunk"] {
let modes = modes_from_json(&doc, addr);
assert_eq!(modes.len(), 2, "the malformed row is dropped ({addr})");
assert_eq!(modes[1].id, "42", "numeric mode ids stringify ({addr})");
let got = pick_custom_mode(&modes, 2868, 1320, 120).expect("aligned mode");
assert_eq!(got.id, "42");
}
// Never reads another output's list (the eDP-1 entry carries a matching mode).
assert!(modes_from_json(&doc, "Virtual-nope").is_empty());
}
/// Group-aware exclusive (§6.1): with two managed group members + a physical panel enabled, /// Group-aware exclusive (§6.1): with two managed group members + a physical panel enabled,
/// exclusive disables ONLY the non-managed panel — never a sibling session's per-slot output /// exclusive disables ONLY the non-managed panel — never a sibling session's per-slot output
@@ -0,0 +1,820 @@
//! In-process KDE output management (`kde_output_management_v2` + `kde_output_device_v2`).
//!
//! Topology — make the streamed output primary, disable the physical/bootstrap outputs, capture
//! their modes for restore, re-enable them on teardown, position the output — used to shell out to
//! `kscreen-doctor` (see [`super::kwin`]). But `kscreen-doctor` drives a *separate* stack:
//! libkscreen picks a backend and, depending on the setup, waits on the kscreen KDED module over
//! D-Bus. On a machine where THAT layer is wedged it blocks in its own connect and never returns —
//! a field report (Nobara, KWin 6.6.4) showed every topology `kscreen-doctor` timing out at its 5 s
//! budget, so the streamed output never became the desktop (`also_disabled=[]`) and bring-up took
//! ~26 s.
//!
//! The compositor's OWN Wayland is provably responsive on that same session — the host just created
//! a virtual output over it via `zkde_screencast` — so we drive `kde_output_management_v2` directly
//! over Wayland here, sidestepping whatever wedges the standalone tool. Every wait is time-bounded,
//! so a genuinely wedged compositor degrades to `handled = false` and the caller falls back to the
//! `kscreen-doctor` path rather than hanging.
//!
//! KWin advertises one `kde_output_device_v2` global per output (the classic model; verified live:
//! `kde_output_management_v2` v19, `kde_output_device_v2` v20 on KWin 6.6.4). We bind them all, read
//! each output's name / enabled / priority / current-mode size, then build a
//! `kde_output_configuration_v2` and `apply()` it, waiting for `applied` / `failed`.
#![allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
#![deny(clippy::undocumented_unsafe_blocks)]
use std::collections::HashMap;
use std::os::fd::{AsFd, AsRawFd};
use std::time::{Duration, Instant};
use wayland_client::backend::ObjectId;
use wayland_client::protocol::wl_callback::{self, WlCallback};
use wayland_client::protocol::wl_registry::{self, WlRegistry};
use wayland_client::{event_created_child, Connection, Dispatch, Proxy, QueueHandle};
// Generate the client bindings for the two vendored KDE protocols inline (no build.rs). The
// management protocol references the device interfaces, so they can't share one `__interfaces`
// module (each `generate_interfaces!` emits its own helper items, which collide). Instead — the
// interdependent-protocol pattern from the `wayland-protocols` crate — `device` is a self-contained
// module and `management` pulls in `device`'s interface statics + generated proxy types before its
// own codegen, so its cross-protocol object args (`kde_output_device_v2`, `…_mode_v2`) resolve.
#[allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
pub mod device {
use wayland_client;
use wayland_client::protocol::*;
pub mod __interfaces {
use wayland_client::protocol::__interfaces::*;
wayland_scanner::generate_interfaces!("protocols/kde-output-device-v2.xml");
}
use self::__interfaces::*;
wayland_scanner::generate_client_code!("protocols/kde-output-device-v2.xml");
}
#[allow(clippy::all, dead_code, non_camel_case_types, non_snake_case, unused)]
pub mod management {
use wayland_client;
use wayland_client::protocol::*;
pub mod __interfaces {
use super::super::device::__interfaces::*;
use wayland_client::protocol::__interfaces::*;
wayland_scanner::generate_interfaces!("protocols/kde-output-management-v2.xml");
}
use self::__interfaces::*;
// The device protocol's generated modules/types, so the foreign object args resolve.
use super::device::*;
wayland_scanner::generate_client_code!("protocols/kde-output-management-v2.xml");
}
use device::kde_output_device_mode_v2::{Event as ModeEvent, KdeOutputDeviceModeV2 as DeviceMode};
use device::kde_output_device_v2::{Event as DeviceEvent, KdeOutputDeviceV2 as OutputDevice};
use management::kde_mode_list_v2::KdeModeListV2 as ModeList;
use management::kde_output_configuration_v2::{
Event as ConfigEvent, KdeOutputConfigurationV2 as OutputConfig,
};
use management::kde_output_management_v2::KdeOutputManagementV2 as OutputManagement;
/// Highest interface versions we drive; we bind `min(advertised, MAX)`. Every request we issue is
/// `since ≤ 2` (`create_configuration`/`enable`/`mode`/`position`/`apply` are v1, `set_primary_output`
/// is v2) and every event we read is `since ≤ 18` (`priority`), so binding high and calling low is
/// always in range on any KWin that advertises the globals.
const MGMT_MAX: u32 = 22;
const DEVICE_MAX: u32 = 24;
/// The opcode of `kde_output_device_v2.mode` (0-based event index) — the event that creates a child
/// `kde_output_device_mode_v2`. Kept in sync with the vendored `kde-output-device-v2.xml`.
const DEVICE_MODE_EVENT_OPCODE: u16 = 2;
/// Overall budget for one enumerate-then-apply operation. Generous next to a healthy roundtrip (a
/// few ms); it exists only so a wedged compositor can't pin the session's stream thread.
const OP_BUDGET: Duration = Duration::from_secs(3);
/// Poll slice while waiting on the Wayland fd (matches the keepalive loop's cadence in `kwin.rs`).
const POLL_MS: i32 = 100;
/// KWin's CVT generator aligns a custom mode's width DOWN to a multiple of this (libxcvt's cell
/// grain), so the mode it builds for a `set_custom_modes` request may be a few px narrower than
/// asked — matches `kwin::CVT_H_GRANULARITY`. Used when matching the generated mode back.
const CVT_H_GRANULARITY: u32 = 8;
/// Which topology to apply once our output is resolved.
#[derive(Clone, Copy, PartialEq, Eq)]
pub(crate) enum TopologyKind {
/// Make ours the sole desktop: primary + disable every other enabled output.
Exclusive,
/// Make ours primary but leave the other outputs enabled.
Primary,
}
/// Outcome of [`apply_topology`].
pub(crate) struct TopologyOutcome {
/// UUID of our resolved virtual output — a stable per-output id that survives a mode-switch
/// supersede (unlike the shared name) — for later [`set_position`] / restore addressing.
pub our_uuid: Option<String>,
/// The outputs we disabled, each `(name, "WxH@Hz")`, so teardown can restore them at their exact
/// mode. Empty for `Primary`, or when nothing else was enabled.
pub disabled: Vec<(String, String)>,
/// `true` if the in-process path bound management, resolved our output, and applied (or tried to)
/// a configuration. `false` ⇒ the compositor didn't answer in budget or our output never
/// appeared, so the caller should fall back to `kscreen-doctor`.
pub handled: bool,
}
/// One output as read from `kde_output_device_v2`.
#[derive(Default, Clone)]
struct DeviceState {
/// The global `name` number (higher = more recently advertised) — used to pick the newest of two
/// same-named outputs during a supersede.
global: u32,
name: Option<String>,
uuid: Option<String>,
enabled: bool,
/// KWin's output priority; 1 is the primary. `None` until the `priority` event (device ≥ v18).
priority: Option<u32>,
/// The `current_mode` object id; its size is looked up in [`State::mode_dims`].
current_mode: Option<ObjectId>,
/// Every mode this output advertised, in announce order — `(mode object id, proxy)` — so restore
/// can pick the one matching a captured `WxH@Hz`.
modes: Vec<(ObjectId, DeviceMode)>,
/// Set once this output's `done` burst has been seen (its state is coherent to read).
seen_done: bool,
proxy: Option<OutputDevice>,
}
/// Everything the enumerate/apply queue accumulates on one connection.
#[derive(Default)]
struct State {
management: Option<OutputManagement>,
mgmt_name_version: Option<(u32, u32)>,
devices: HashMap<ObjectId, DeviceState>,
/// mode object id → `(width, height, refresh_mHz)`.
mode_dims: HashMap<ObjectId, (u32, u32, u32)>,
/// Highest `wl_callback` serial whose `done` has arrived — the barrier the pump waits on.
sync_done: u32,
/// Configuration apply verdict: `Some(true)` = applied, `Some(false)` = failed.
applied: Option<bool>,
failure_reason: Option<String>,
}
impl Dispatch<WlRegistry, ()> for State {
fn event(
state: &mut Self,
registry: &WlRegistry,
event: wl_registry::Event,
_: &(),
_: &Connection,
qh: &QueueHandle<Self>,
) {
match event {
wl_registry::Event::Global {
name,
interface,
version,
} => {
if interface == OutputManagement::interface().name {
let v = version.min(MGMT_MAX);
state.management =
Some(registry.bind::<OutputManagement, _, _>(name, v, qh, ()));
state.mgmt_name_version = Some((name, v));
} else if interface == OutputDevice::interface().name {
let v = version.min(DEVICE_MAX);
// The device's `name` global carries into the device's UserData so the event
// handler can record it (newest-wins tie-break during a supersede).
let dev = registry.bind::<OutputDevice, _, _>(name, v, qh, name);
let id = dev.id();
state.devices.entry(id).or_default().proxy = Some(dev);
}
}
wl_registry::Event::GlobalRemove { .. } => {}
_ => {}
}
}
}
// Management has no events.
impl Dispatch<OutputManagement, ()> for State {
fn event(
_: &mut Self,
_: &OutputManagement,
_: management::kde_output_management_v2::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
// A client-built custom-mode list has no events; it just needs a Dispatch impl to be created.
impl Dispatch<ModeList, ()> for State {
fn event(
_: &mut Self,
_: &ModeList,
_: management::kde_mode_list_v2::Event,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
}
}
/// The device's UserData is its global `name` number.
impl Dispatch<OutputDevice, u32> for State {
fn event(
state: &mut Self,
device: &OutputDevice,
event: DeviceEvent,
global: &u32,
_: &Connection,
_: &QueueHandle<Self>,
) {
let entry = state.devices.entry(device.id()).or_default();
entry.global = *global;
if entry.proxy.is_none() {
entry.proxy = Some(device.clone());
}
match event {
DeviceEvent::Name { name } => entry.name = Some(name),
DeviceEvent::Uuid { uuid } => entry.uuid = Some(uuid),
DeviceEvent::Enabled { enabled } => entry.enabled = enabled != 0,
DeviceEvent::Priority { priority } => entry.priority = Some(priority),
DeviceEvent::CurrentMode { mode } => entry.current_mode = Some(mode.id()),
DeviceEvent::Mode { mode } => entry.modes.push((mode.id(), mode)),
DeviceEvent::Done => entry.seen_done = true,
_ => {}
}
}
// The `mode` event hands us a server-created `kde_output_device_mode_v2`. The opcode is a bare
// literal (the macro's fragment matcher rejects a `const` in some wayland-client versions); it is
// pinned to `DEVICE_MODE_EVENT_OPCODE` by `mode_event_opcode_is_two` below.
event_created_child!(State, OutputDevice, [
2 => (DeviceMode, ()),
]);
}
impl Dispatch<DeviceMode, ()> for State {
fn event(
state: &mut Self,
mode: &DeviceMode,
event: ModeEvent,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
let entry = state.mode_dims.entry(mode.id()).or_insert((0, 0, 0));
match event {
ModeEvent::Size { width, height } => {
entry.0 = width.max(0) as u32;
entry.1 = height.max(0) as u32;
}
ModeEvent::Refresh { refresh } => entry.2 = refresh.max(0) as u32,
_ => {}
}
}
}
impl Dispatch<OutputConfig, ()> for State {
fn event(
state: &mut Self,
_: &OutputConfig,
event: ConfigEvent,
_: &(),
_: &Connection,
_: &QueueHandle<Self>,
) {
match event {
ConfigEvent::Applied => state.applied = Some(true),
ConfigEvent::Failed => state.applied = Some(false),
ConfigEvent::FailureReason { reason } => state.failure_reason = Some(reason),
_ => {}
}
}
}
impl Dispatch<WlCallback, u32> for State {
fn event(
state: &mut Self,
_: &WlCallback,
event: wl_callback::Event,
serial: &u32,
_: &Connection,
_: &QueueHandle<Self>,
) {
if let wl_callback::Event::Done { .. } = event {
state.sync_done = state.sync_done.max(*serial);
}
}
}
/// A connected, bound output-management session on its own Wayland connection.
struct Session {
conn: Connection,
queue: wayland_client::EventQueue<State>,
state: State,
next_sync: u32,
}
impl Session {
/// Connect to the KWin Wayland socket, bind `kde_output_management_v2` + every
/// `kde_output_device_v2`, and read each output's state — all bounded by `OP_BUDGET`. `None` if
/// we can't connect, the management global isn't advertised, or the compositor doesn't answer in
/// budget (the wedge case — the caller then falls back to `kscreen-doctor`).
fn open() -> Option<Session> {
let conn = Connection::connect_to_env().ok()?;
let queue = conn.new_event_queue();
let qh = queue.handle();
let _registry = conn.display().get_registry(&qh, ());
let mut s = Session {
conn,
queue,
state: State::default(),
next_sync: 0,
};
let deadline = Instant::now() + OP_BUDGET;
// Phase 1: process the registry globals (binds management + every device in the handler).
if !s.sync_barrier(deadline) {
return None;
}
if s.state.management.is_none() {
tracing::debug!(
"KWin does not advertise kde_output_management_v2 to this client — kscreen-doctor \
fallback"
);
return None;
}
// Phase 2: flush the device binds issued in phase 1 and drain each output's state burst
// (name / enabled / priority / current_mode / mode sizes / done).
if !s.sync_barrier(deadline) {
return None;
}
Some(s)
}
/// Send a `wl_display.sync` and pump the queue until its `done` arrives or `deadline` passes.
/// Returns `true` on the barrier, `false` on timeout.
fn sync_barrier(&mut self, deadline: Instant) -> bool {
self.next_sync += 1;
let serial = self.next_sync;
let qh = self.queue.handle();
let _cb = self.conn.display().sync(&qh, serial);
self.pump_until(deadline, |st| st.sync_done >= serial)
}
/// Bounded manual event loop: flush, dispatch what's queued, then poll the connection fd for up
/// to `POLL_MS` and read. Mirrors the keepalive loop in `kwin.rs::run` (blocking_dispatch can't
/// be interrupted, so we poll the fd instead). Returns `true` once `done(&state)` holds.
fn pump_until(&mut self, deadline: Instant, done: impl Fn(&State) -> bool) -> bool {
loop {
if done(&self.state) {
return true;
}
if self.queue.dispatch_pending(&mut self.state).is_err() {
return false;
}
if done(&self.state) {
return true;
}
if Instant::now() >= deadline {
return false;
}
if self.conn.flush().is_err() {
return false;
}
let Some(guard) = self.conn.prepare_read() else {
continue; // events already queued — loop dispatches them
};
let mut pfd = libc::pollfd {
fd: self.conn.as_fd().as_raw_fd(),
events: libc::POLLIN,
revents: 0,
};
let remaining = deadline.saturating_duration_since(Instant::now());
let timeout = (remaining.as_millis() as i32).clamp(0, POLL_MS);
// SAFETY: `&mut pfd` points at one live, fully-initialized `libc::pollfd` on the stack and
// the count `1` matches that single element, so `poll` reads `fd`/`events` and writes
// `revents` strictly within `pfd`. `pfd.fd` is the Wayland connection's fd, valid because
// `self.conn` (and the `prepare_read` guard) outlive the call. `poll` blocks up to
// `timeout` ms and writes only `revents`; `pfd` is a fresh local that aliases nothing.
let r = unsafe { libc::poll(&mut pfd, 1, timeout) };
if r > 0 && (pfd.revents & libc::POLLIN) != 0 {
let _ = guard.read();
} // else: timeout/signal — drop the guard, re-check the deadline
}
}
/// A fresh `kde_output_configuration_v2` on this connection.
fn new_config(&self) -> OutputConfig {
let qh = self.queue.handle();
self.state
.management
.as_ref()
.unwrap()
.create_configuration(&qh, ())
}
/// A fresh (empty) `kde_mode_list_v2` on this connection, for a `set_custom_modes` request.
fn new_mode_list(&self) -> ModeList {
let qh = self.queue.handle();
self.state
.management
.as_ref()
.unwrap()
.create_mode_list(&qh, ())
}
/// `apply()` the config and pump until `applied`/`failed` or the deadline. Returns the verdict
/// (`true` applied, `false` failed/timeout).
fn apply(&mut self, config: &OutputConfig, deadline: Instant) -> bool {
self.state.applied = None;
config.apply();
let ok = self.pump_until(deadline, |st| st.applied.is_some());
if !ok {
return false;
}
matches!(self.state.applied, Some(true))
}
/// The current-mode size of a device as `(w, h, refresh_mHz)`, if known.
fn current_dims(&self, dev: &DeviceState) -> Option<(u32, u32, u32)> {
let id = dev.current_mode.as_ref()?;
self.state.mode_dims.get(id).copied()
}
}
/// `(width, height, "WxH@Hz")` capture of a device's current mode, Hz rounded — the same shape the
/// `kscreen-doctor` restore path used, so teardown can put a panel back at its real refresh.
fn mode_spec(dims: (u32, u32, u32)) -> String {
let hz = ((dims.2 as f64) / 1000.0).round() as u32;
format!("{}x{}@{}", dims.0, dims.1, hz)
}
/// Prefix EVERY managed KWin output shares (mirrors `kwin::MANAGED_PREFIX`) — the streamed outputs
/// are `Virtual-punktfunk` / `Virtual-punktfunk-<id>`, so a same-family sibling session is never
/// treated as a physical to disable, and its primary is never stolen (first-slot-wins).
const MANAGED_PREFIX: &str = "Virtual-punktfunk";
/// Make the streamed output (name starts with `our_prefix`, current size `our_w`×`our_h`) the
/// primary — and, for `Exclusive`, disable every other enabled output — over `kde_output_management_v2`.
/// See the module docs for why this is done in-process instead of via `kscreen-doctor`.
pub(crate) fn apply_topology(
our_prefix: &str,
our_w: u32,
our_h: u32,
kind: TopologyKind,
) -> TopologyOutcome {
let miss = || TopologyOutcome {
our_uuid: None,
disabled: Vec::new(),
handled: false,
};
let Some(mut sess) = Session::open() else {
return miss();
};
let deadline = Instant::now() + OP_BUDGET;
// Resolve OUR output: managed-prefix name AND current size == the birth size (only the
// just-created output sits there during a supersede); newest global wins the tie.
let ours = sess
.state
.devices
.values()
.filter(|d| {
d.name.as_deref().is_some_and(|n| n.starts_with(our_prefix))
&& sess.current_dims(d).map(|(w, h, _)| (w, h)) == Some((our_w, our_h))
})
.max_by_key(|d| d.global)
.cloned();
let Some(ours) = ours else {
tracing::warn!(
our_prefix,
our_w,
our_h,
"KWin output management: our virtual output hasn't appeared yet — kscreen-doctor fallback"
);
return miss();
};
let our_uuid = ours.uuid.clone();
let our_id = ours.proxy.as_ref().map(|p| p.id());
// First-slot-wins (§6.1): don't steal primary if another managed sibling already holds it
// (priority 1) — a 2nd exclusive session joins as a secondary of the shared desktop.
let sibling_is_primary = sess.state.devices.values().any(|d| {
d.enabled
&& d.priority == Some(1)
&& d.proxy.as_ref().map(|p| p.id()) != our_id
&& d.name
.as_deref()
.is_some_and(|n| n.starts_with(MANAGED_PREFIX))
});
// The physical/bootstrap outputs to disable for `Exclusive`: enabled, not any managed sibling,
// not ours. Captured WITH their current mode so teardown restores the exact refresh.
let mut to_disable: Vec<(OutputDevice, String, String)> = Vec::new();
if kind == TopologyKind::Exclusive {
for d in sess.state.devices.values() {
let is_ours = d.proxy.as_ref().map(|p| p.id()) == our_id;
let managed = d
.name
.as_deref()
.is_some_and(|n| n.starts_with(MANAGED_PREFIX));
if d.enabled && !is_ours && !managed {
if let (Some(name), Some(proxy)) = (d.name.clone(), d.proxy.clone()) {
let spec = sess.current_dims(d).map(mode_spec).unwrap_or_default();
to_disable.push((proxy, name, spec));
}
}
}
}
// Build one configuration: ensure ours is enabled, take primary (unless a sibling holds it),
// disable the others. `apply()` is atomic — KWin re-homes the shell onto the remaining desktop.
let config = sess.new_config();
if let Some(proxy) = ours.proxy.as_ref() {
config.enable(proxy, 1);
if !sibling_is_primary {
config.set_primary_output(proxy);
}
}
for (proxy, _, _) in &to_disable {
config.enable(proxy, 0);
}
let applied = sess.apply(&config, deadline);
config.destroy();
// Always report the outputs we ASKED to disable so teardown restores them: re-enabling an output
// that never actually got disabled is a harmless no-op, whereas dropping them here would strand a
// physical dark if KWin processed the disable but the `applied` ack didn't land in budget.
let disabled: Vec<(String, String)> = to_disable
.into_iter()
.map(|(_, name, spec)| (name, spec))
.collect();
if applied {
tracing::info!(
also_disabled = ?disabled,
primary_taken = !sibling_is_primary,
"KWin output management: streamed output set as the desktop (in-process)"
);
} else {
tracing::warn!(
reason = ?sess.state.failure_reason,
also_disabled = ?disabled,
"KWin output management: apply() not confirmed in budget — proceeding (restore will re-enable)"
);
}
// We resolved our output and drove the config over Wayland; don't ALSO run kscreen-doctor — that
// would double-apply (and on a wedged box it would just add 26 s of timeouts). `handled` is true
// even on an unconfirmed apply; a genuinely absent management global / unresolved output took the
// `handled = false` early returns above and falls back to kscreen-doctor.
TopologyOutcome {
our_uuid,
disabled,
handled: true,
}
}
/// Install + select a `want_w`×`want_h`@`want_hz` custom mode on the just-created virtual output
/// (name starts with `our_prefix`, currently at its sacrificial birth size `birth_w`×`birth_h`) —
/// entirely over `kde_output_management_v2`, the in-process replacement for the `kscreen-doctor`
/// `addCustomMode` + `mode` shell-out (`set_custom_refresh`).
///
/// `set_custom_modes` hands KWin a one-entry mode list; KWin generates the CVT timing (so the width
/// may align DOWN — see [`CVT_H_GRANULARITY`]) and adds the mode. We then SELECT it, which changes
/// the output's size and triggers the screencast stream's renegotiation to the real refresh (the
/// sacrificial-birth mechanism in `kwin::create`). Returns the ACTIVE mode read back after selection
/// (Hz rounded), or `None` if management is absent, the output/generated mode never appeared, or an
/// apply didn't confirm — the caller then falls back to `kscreen-doctor`. `set_custom_modes` REPLACES
/// the custom list (idempotent across reconnects — no per-connect list growth), and it is `since 18`,
/// so pre-6.6 KWin without it simply takes the `None` → kscreen-doctor path.
pub(crate) fn set_custom_mode(
our_prefix: &str,
birth_w: u32,
birth_h: u32,
want_w: u32,
want_h: u32,
want_hz: u32,
) -> Option<(u32, u32, u32)> {
let mut sess = Session::open()?;
let deadline = Instant::now() + OP_BUDGET;
// `set_custom_modes` is `since 18`; calling it on an older bound management object is a protocol
// error, so bail to the kscreen-doctor fallback there (pre-6.6 KWin). Our bound version is
// `min(advertised, MGMT_MAX)`.
if sess.state.mgmt_name_version.map(|(_, v)| v).unwrap_or(0) < 18 {
return None;
}
// Resolve our output at its birth size (newest global wins a supersede).
let our_proxy = sess
.state
.devices
.values()
.filter(|d| {
d.name.as_deref().is_some_and(|n| n.starts_with(our_prefix))
&& sess.current_dims(d).map(|(w, h, _)| (w, h)) == Some((birth_w, birth_h))
})
.max_by_key(|d| d.global)
.and_then(|d| d.proxy.clone())?;
let our_key = our_proxy.id();
let want_mhz = want_hz.saturating_mul(1000);
// A generated mode IS our custom one iff: exact height, width at/just-below the request (a CVT
// alignment), and refresh within 1 Hz — which excludes the sacrificial 60 Hz birth mode.
let mode_matches = move |st: &State, mid: &ObjectId| -> bool {
st.mode_dims.get(mid).is_some_and(|&(w, h, mhz)| {
h == want_h
&& w <= want_w
&& want_w - w < CVT_H_GRANULARITY
&& (mhz as i64 - want_mhz as i64).abs() <= 1000
})
};
// Build a one-entry custom-mode list (full blanking, like kscreen-doctor's `.full`) and install it.
let mode_list = sess.new_mode_list();
mode_list.set_resolution(want_w, want_h);
mode_list.set_refresh_rate(want_mhz);
mode_list.set_reduced_blanking(0);
mode_list.add_mode();
let config = sess.new_config();
config.set_custom_modes(&our_proxy, &mode_list);
let installed = sess.apply(&config, deadline);
config.destroy();
mode_list.destroy();
if !installed {
return None;
}
// Wait for KWin to generate the mode and advertise it on the output.
let found = |st: &State| -> bool {
st.devices
.get(&our_key)
.is_some_and(|d| d.modes.iter().any(|(mid, _)| mode_matches(st, mid)))
};
if !sess.pump_until(deadline, found) {
tracing::warn!(
want_w,
want_h,
want_hz,
"KWin output management: generated custom mode never appeared — kscreen-doctor fallback"
);
return None;
}
// Grab the generated mode's proxy, then select it (this is what changes the size).
let mode_proxy = {
let dev = sess.state.devices.get(&our_key)?;
dev.modes
.iter()
.find(|(mid, _)| mode_matches(&sess.state, mid))
.map(|(_, p)| p.clone())?
};
let config = sess.new_config();
config.mode(&our_proxy, &mode_proxy);
let selected = sess.apply(&config, deadline);
config.destroy();
if !selected {
return None;
}
// Read back the active mode after selection — the size that really landed paces the encoder.
let want_dims = sess
.state
.mode_dims
.get(&mode_proxy.id())
.map(|&(w, h, _)| (w, h));
let landed = |st: &State| -> bool {
st.devices
.get(&our_key)
.and_then(|d| d.current_mode.as_ref())
.and_then(|mid| st.mode_dims.get(mid))
.map(|&(w, h, _)| (w, h))
== want_dims
};
sess.pump_until(deadline, landed);
let dev = sess.state.devices.get(&our_key)?;
let (cw, ch, cmhz) = sess.current_dims(dev)?;
let hz = ((cmhz as f64) / 1000.0).round() as u32;
tracing::info!(
want_w,
want_h,
want_hz,
active_w = cw,
active_h = ch,
active_hz = hz,
"KWin output management: custom mode installed + selected (in-process)"
);
Some((cw, ch, hz.max(1)))
}
/// Re-enable outputs by name at their captured `WxH@Hz` modes (teardown), in-process. Returns
/// `true` if the config applied; `false` (compositor unresponsive / management absent) tells the
/// caller to fall back to `kscreen-doctor`.
pub(crate) fn reenable_outputs(outputs: &[(String, String)]) -> bool {
if outputs.is_empty() {
return true;
}
let Some(mut sess) = Session::open() else {
return false;
};
let deadline = Instant::now() + OP_BUDGET;
let config = sess.new_config();
for (name, spec) in outputs {
// Find the device by name (physical names are stable across a session).
let Some(dev) = sess
.state
.devices
.values()
.find(|d| d.name.as_deref() == Some(name.as_str()))
.cloned()
else {
continue;
};
let Some(proxy) = dev.proxy.as_ref() else {
continue;
};
// Enable first — a bare enable always succeeds, so a physical is never left dark.
config.enable(proxy, 1);
// Then re-assert the captured mode so a 120 Hz panel doesn't return at KWin's ~60 Hz default.
if let Some(mode) = find_mode(&sess, &dev, spec) {
config.mode(proxy, &mode);
}
}
let ok = sess.apply(&config, deadline);
config.destroy();
if ok {
tracing::info!(reenabled = ?outputs, "KWin output management: restored outputs (in-process)");
}
ok
}
/// Position the output identified by `uuid` at `(x, y)` in the desktop layout, in-process. Returns
/// `true` if applied; `false` tells the caller to fall back to `kscreen-doctor`.
pub(crate) fn set_position(uuid: &str, x: i32, y: i32) -> bool {
let Some(mut sess) = Session::open() else {
return false;
};
let deadline = Instant::now() + OP_BUDGET;
let Some(dev) = sess
.state
.devices
.values()
.find(|d| d.uuid.as_deref() == Some(uuid))
.cloned()
else {
return false;
};
let Some(proxy) = dev.proxy.as_ref() else {
return false;
};
let config = sess.new_config();
config.position(proxy, x, y);
let ok = sess.apply(&config, deadline);
config.destroy();
if ok {
tracing::info!(
uuid,
x,
y,
"KWin output management: placed output (in-process)"
);
}
ok
}
/// Find a device's advertised mode proxy matching a captured `"WxH@Hz"` spec (Hz rounded), for
/// restore. `None` if the spec is empty or no mode matches (the caller then enables without a mode
/// and lets KWin pick its preferred one).
fn find_mode(sess: &Session, dev: &DeviceState, spec: &str) -> Option<DeviceMode> {
if spec.is_empty() {
return None;
}
let (wh, hz) = spec.split_once('@')?;
let (w, h) = wh.split_once('x')?;
let (w, h, hz): (u32, u32, u32) = (w.parse().ok()?, h.parse().ok()?, hz.parse().ok()?);
dev.modes.iter().find_map(|(id, proxy)| {
let (mw, mh, mmhz) = sess.state.mode_dims.get(id).copied()?;
let mhz = ((mmhz as f64) / 1000.0).round() as u32;
((mw, mh, mhz) == (w, h, hz)).then(|| proxy.clone())
})
}
#[cfg(test)]
mod tests {
use super::*;
/// The `WxH@Hz` capture rounds mHz to whole Hz — the shape teardown parses back.
#[test]
fn mode_spec_rounds_millihertz() {
assert_eq!(mode_spec((2560, 1440, 59940)), "2560x1440@60");
assert_eq!(mode_spec((1920, 1080, 60000)), "1920x1080@60");
assert_eq!(mode_spec((3840, 2160, 119880)), "3840x2160@120");
}
/// The vendored device XML must keep `mode` at the opcode the `event_created_child!` macro
/// hardcodes — a reorder there would bind the child to the wrong event and desync mode sizes.
#[test]
fn mode_event_opcode_is_two() {
assert_eq!(DEVICE_MODE_EVENT_OPCODE, 2);
}
}
@@ -46,7 +46,17 @@ const BUS_DC: &str = "org.gnome.Mutter.DisplayConfig";
/// e.g. when our virtual output is torn down — so we never persist a layout to monitors.xml). /// e.g. when our virtual output is torn down — so we never persist a layout to monitors.xml).
const APPLY_TEMPORARY: u32 = 1; const APPLY_TEMPORARY: u32 = 1;
/// Mutter cursor mode: render the cursor into the stream (matches the KWin/gamescope backends). /// Mutter cursor mode: ship the pointer as `SPA_META_Cursor` metadata instead of burning it into
/// the frames. The capturer always negotiates the meta (pf-capture `meta_param`) and the encoder
/// blend composites it for sessions where the client does not draw the cursor itself — while a
/// cursor-forwarding session strips the overlay and sends shape/state over the cursor channel.
/// Embedded mode would leave BOTH paths blind: no metadata means nothing to forward AND nothing
/// to blend, and Mutter's own embedded painting is what the pre-channel path relied on.
const CURSOR_METADATA: u32 = 2;
/// `cursor-mode` embedded (mutter enum 1): Mutter composites the pointer into frames itself —
/// zero host-side cursor work, the pre-channel path. Chosen for every session WITHOUT the
/// negotiated cursor channel (`set_hw_cursor` off — Phase B, the Windows no-regression gate
/// mirrored); metadata stays the cursor-channel sessions' mode (shapes forwarded / host blend).
const CURSOR_EMBEDDED: u32 = 1; const CURSOR_EMBEDDED: u32 = 1;
/// Serializes, process-wide, every Mutter operation that adds/removes a virtual monitor or applies /// Serializes, process-wide, every Mutter operation that adds/removes a virtual monitor or applies
@@ -69,6 +79,9 @@ pub struct MutterDisplay {
/// sole-monitor config (which would disable the first session's virtual). Defaults true (a lone /// sole-monitor config (which would disable the first session's virtual). Defaults true (a lone
/// session establishes topology as before). /// session establishes topology as before).
first_in_group: bool, first_in_group: bool,
/// Out-of-band cursor request (`set_hw_cursor`): metadata cursor-mode at creation; off =
/// embedded (see [`CURSOR_EMBEDDED`]).
hw_cursor: bool,
/// The connecting client's cert fingerprint (set before [`create`](VirtualDisplay::create)) — /// The connecting client's cert fingerprint (set before [`create`](VirtualDisplay::create)) —
/// keys the per-client persisted **scale** (GNOME can't persist it itself: Mutter mints a fresh /// keys the per-client persisted **scale** (GNOME can't persist it itself: Mutter mints a fresh
/// EDID serial per `RecordVirtual` monitor, so `monitors.xml` never rematches; see /// EDID serial per `RecordVirtual` monitor, so `monitors.xml` never rematches; see
@@ -84,6 +97,7 @@ impl MutterDisplay {
pub fn new() -> Result<Self> { pub fn new() -> Result<Self> {
Ok(MutterDisplay { Ok(MutterDisplay {
first_in_group: true, first_in_group: true,
hw_cursor: false,
client_fp: None, client_fp: None,
last_slot: None, last_slot: None,
}) })
@@ -112,6 +126,14 @@ impl VirtualDisplay for MutterDisplay {
self.client_fp = fingerprint; self.client_fp = fingerprint;
} }
fn set_hw_cursor(&mut self, on: bool) {
self.hw_cursor = on;
}
fn hw_cursor(&self) -> bool {
self.hw_cursor
}
fn last_identity_slot(&self) -> Option<u32> { fn last_identity_slot(&self) -> Option<u32> {
self.last_slot self.last_slot
} }
@@ -141,6 +163,7 @@ impl VirtualDisplay for MutterDisplay {
let stop = Arc::new(AtomicBool::new(false)); let stop = Arc::new(AtomicBool::new(false));
let stop_thread = stop.clone(); let stop_thread = stop.clone();
let first_in_group = self.first_in_group; let first_in_group = self.first_in_group;
let hw_cursor = self.hw_cursor;
thread::Builder::new() thread::Builder::new()
.name("punktfunk-mutter-vout".into()) .name("punktfunk-mutter-vout".into())
.spawn(move || { .spawn(move || {
@@ -149,6 +172,7 @@ impl VirtualDisplay for MutterDisplay {
stop_thread, stop_thread,
mode, mode,
first_in_group, first_in_group,
hw_cursor,
scale_key, scale_key,
remembered_scale, remembered_scale,
) )
@@ -203,6 +227,7 @@ fn session_thread(
stop: Arc<AtomicBool>, stop: Arc<AtomicBool>,
mode: Mode, mode: Mode,
first_in_group: bool, first_in_group: bool,
hw_cursor: bool,
scale_key: String, scale_key: String,
remembered_scale: Option<f64>, remembered_scale: Option<f64>,
) { ) {
@@ -262,7 +287,7 @@ fn session_thread(
} }
}; };
let session = match connect(mode, remembered_scale).await { let session = match connect(mode, hw_cursor, remembered_scale).await {
Ok(s) => s, Ok(s) => s,
Err(e) => { Err(e) => {
let _ = setup_tx.send(Err(format!("{e:#}"))); let _ = setup_tx.send(Err(format!("{e:#}")));
@@ -363,7 +388,11 @@ struct MutterSession {
/// desktop scale, passed as the virtual mode's `preferred-scale` so Mutter creates the monitor /// desktop scale, passed as the virtual mode's `preferred-scale` so Mutter creates the monitor
/// already scaled (Mutter ≥ 48; older Mutter ignores unknown mode keys) — this covers the /// already scaled (Mutter ≥ 48; older Mutter ignores unknown mode keys) — this covers the
/// `extend` topology, where we never issue our own ApplyMonitorsConfig. /// `extend` topology, where we never issue our own ApplyMonitorsConfig.
async fn connect(mode: Mode, preferred_scale: Option<f64>) -> Result<MutterSession> { async fn connect(
mode: Mode,
hw_cursor: bool,
preferred_scale: Option<f64>,
) -> Result<MutterSession> {
let conn = zbus::Connection::session() let conn = zbus::Connection::session()
.await .await
.context("connect session D-Bus")?; .context("connect session D-Bus")?;
@@ -424,7 +453,14 @@ async fn connect(mode: Mode, preferred_scale: Option<f64>) -> Result<MutterSessi
// once gated behind PUNKTFUNK_MUTTER_VIRTUAL_REFRESH; the stop-screencast-before-any-monitor- // once gated behind PUNKTFUNK_MUTTER_VIRTUAL_REFRESH; the stop-screencast-before-any-monitor-
// reconfig teardown below fixed the crash, so pinning the client's refresh is now the default.) // reconfig teardown below fixed the crash, so pinning the client's refresh is now the default.)
let mut rec: HashMap<&str, Value> = HashMap::new(); let mut rec: HashMap<&str, Value> = HashMap::new();
rec.insert("cursor-mode", Value::from(CURSOR_EMBEDDED)); rec.insert(
"cursor-mode",
Value::from(if hw_cursor {
CURSOR_METADATA
} else {
CURSOR_EMBEDDED
}),
);
if mode.refresh_hz > 60 || preferred_scale.is_some() { if mode.refresh_hz > 60 || preferred_scale.is_some() {
let mut vmode: HashMap<&str, Value> = HashMap::new(); let mut vmode: HashMap<&str, Value> = HashMap::new();
vmode.insert("size", Value::from((mode.width, mode.height))); vmode.insert("size", Value::from((mode.width, mode.height)));
@@ -54,11 +54,19 @@ chooser_cmd=cat {} 2>/dev/null || echo 'Monitor: HEADLESS-1'\n",
/// The wlroots/Sway virtual-display driver. Stateless — each [`create`](VirtualDisplay::create) /// The wlroots/Sway virtual-display driver. Stateless — each [`create`](VirtualDisplay::create)
/// adds one headless output and spins up a portal thread owning the cast on it. /// adds one headless output and spins up a portal thread owning the cast on it.
pub struct WlrootsDisplay; pub struct WlrootsDisplay {
/// Out-of-band cursor request (`set_hw_cursor`, the negotiated cursor channel): portal
/// `CursorMode::Metadata` — shapes/positions ride `SPA_META_Cursor` for the channel + the
/// composite blend. Off (every non-channel session): `Embedded` — the compositor paints the
/// pointer into frames, zero host-side cursor work (the pre-channel default this backend
/// always had). ⚠️ Metadata is UNTESTED on-glass for this backend (Phase B wired it so the
/// channel isn't silently dead here; KWin/Mutter are the validated legs).
hw_cursor: bool,
}
impl WlrootsDisplay { impl WlrootsDisplay {
pub fn new() -> Result<Self> { pub fn new() -> Result<Self> {
Ok(WlrootsDisplay) Ok(WlrootsDisplay { hw_cursor: false })
} }
} }
@@ -73,6 +81,14 @@ impl VirtualDisplay for WlrootsDisplay {
"wlroots" "wlroots"
} }
fn set_hw_cursor(&mut self, on: bool) {
self.hw_cursor = on;
}
fn hw_cursor(&self) -> bool {
self.hw_cursor
}
fn create(&mut self, mode: Mode) -> Result<VirtualOutput> { fn create(&mut self, mode: Mode) -> Result<VirtualOutput> {
let before = output_names() let before = output_names()
.context("swaymsg get_outputs (is the host inside the sway session env — SWAYSOCK?)")?; .context("swaymsg get_outputs (is the host inside the sway session env — SWAYSOCK?)")?;
@@ -104,9 +120,10 @@ impl VirtualDisplay for WlrootsDisplay {
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>(); let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>();
let stop = Arc::new(AtomicBool::new(false)); let stop = Arc::new(AtomicBool::new(false));
let stop_thread = stop.clone(); let stop_thread = stop.clone();
let hw_cursor = self.hw_cursor;
thread::Builder::new() thread::Builder::new()
.name("punktfunk-wlr-vout".into()) .name("punktfunk-wlr-vout".into())
.spawn(move || portal_thread(setup_tx, stop_thread)) .spawn(move || portal_thread(setup_tx, stop_thread, hw_cursor))
.context("spawn wlroots portal thread")?; .context("spawn wlroots portal thread")?;
let (fd, node_id) = match setup_rx.recv_timeout(Duration::from_secs(20)) { let (fd, node_id) = match setup_rx.recv_timeout(Duration::from_secs(20)) {
@@ -136,6 +153,7 @@ impl VirtualDisplay for WlrootsDisplay {
ownership: DisplayOwnership::Owned, ownership: DisplayOwnership::Owned,
reused_gen: None, reused_gen: None,
pool_gen: None, pool_gen: None,
expect_exact_dims: false,
}) })
} }
} }
@@ -254,7 +272,17 @@ fn ensure_xdpw_config() -> Result<()> {
/// The ScreenCast portal handshake (same shape as the capture module's portal thread, but it /// The ScreenCast portal handshake (same shape as the capture module's portal thread, but it
/// reports the fd + node id and parks until stopped — the zbus connection is the cast's /// reports the fd + node id and parks until stopped — the zbus connection is the cast's
/// lifetime). xdpw answers the source selection via the chooser, no dialog. /// lifetime). xdpw answers the source selection via the chooser, no dialog.
fn portal_thread(setup_tx: Sender<Result<(OwnedFd, u32), String>>, stop: Arc<AtomicBool>) { fn portal_thread(
setup_tx: Sender<Result<(OwnedFd, u32), String>>,
stop: Arc<AtomicBool>,
hw_cursor: bool,
) {
// Portal cursor mode per the session's channel negotiation (see the struct doc).
let cursor_mode = if hw_cursor {
CursorMode::Metadata
} else {
CursorMode::Embedded
};
use ashpd::desktop::screencast::{CursorMode, Screencast, SelectSourcesOptions, SourceType}; use ashpd::desktop::screencast::{CursorMode, Screencast, SelectSourcesOptions, SourceType};
use ashpd::desktop::PersistMode; use ashpd::desktop::PersistMode;
use ashpd::enumflags2::BitFlags; use ashpd::enumflags2::BitFlags;
@@ -287,7 +315,7 @@ fn portal_thread(setup_tx: Sender<Result<(OwnedFd, u32), String>>, stop: Arc<Ato
.select_sources( .select_sources(
&session, &session,
SelectSourcesOptions::default() SelectSourcesOptions::default()
.set_cursor_mode(CursorMode::Embedded) .set_cursor_mode(cursor_mode)
// xdpw offers MONITOR only; the chooser picks our output. // xdpw offers MONITOR only; the chooser picks our output.
.set_sources(BitFlags::from_flag(SourceType::Monitor)) .set_sources(BitFlags::from_flag(SourceType::Monitor))
.set_multiple(false) .set_multiple(false)
+114
View File
@@ -0,0 +1,114 @@
//! Time-bounded child-process helpers.
//!
//! Every compositor query this crate makes shells out to a helper (`kscreen-doctor`, `systemctl`,
//! `pw-dump`, …), and most of them are *clients of the very thing being diagnosed*: `kscreen-doctor`
//! is a Wayland client, so against a wedged KWin it blocks in its own connect and **never returns**.
//! `Command::status()` / `Command::output()` have no timeout, so one hung helper pinned the calling
//! thread forever — and on the host that thread is the session's stream thread, whose only way to
//! end a session is to return. A stuck query therefore became a permanently stuck session.
//!
//! These wrappers bound the wait: poll for exit until the budget runs out, then kill the child and
//! report [`std::io::ErrorKind::TimedOut`], so callers see a plain "the helper failed" error and
//! take their existing failure path instead of hanging.
use std::io::{Error, ErrorKind, Result};
use std::process::{Command, ExitStatus, Output};
use std::time::{Duration, Instant};
/// Poll interval while waiting for a child to exit. Short enough that a fast helper (the normal
/// case — `kscreen-doctor` answers in tens of ms) isn't measurably delayed.
const POLL: Duration = Duration::from_millis(20);
/// Run `cmd` to completion, killing it if it outlives `budget`.
///
/// Stdout/stderr are left as the caller configured them (inherited by default), so this is for
/// commands run for their exit status alone — see [`output_within`] when the output is read.
pub(crate) fn status_within(cmd: &mut Command, budget: Duration) -> Result<ExitStatus> {
let mut child = cmd.spawn()?;
let deadline = Instant::now() + budget;
loop {
match child.try_wait()? {
Some(status) => return Ok(status),
None if Instant::now() >= deadline => {
let _ = child.kill();
let _ = child.wait(); // reap it — never leave a zombie behind
return Err(timed_out(cmd, budget));
}
None => std::thread::sleep(POLL),
}
}
}
/// Run `cmd` to completion and capture its stdout/stderr, killing it if it outlives `budget`.
///
/// The output is read only after the child has exited, so a helper that fills the pipe buffer and
/// stalls is caught by the budget rather than deadlocking the reader (these helpers emit at most a
/// few hundred KiB, well under any real pipe pressure).
pub(crate) fn output_within(cmd: &mut Command, budget: Duration) -> Result<Output> {
let mut child = cmd
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()?;
let deadline = Instant::now() + budget;
loop {
match child.try_wait()? {
// Exited: `wait_with_output` now only drains already-buffered pipes.
Some(_) => return child.wait_with_output(),
None if Instant::now() >= deadline => {
let _ = child.kill();
let _ = child.wait();
return Err(timed_out(cmd, budget));
}
None => std::thread::sleep(POLL),
}
}
}
fn timed_out(cmd: &Command, budget: Duration) -> Error {
let program = cmd.get_program().to_string_lossy().to_string();
tracing::warn!(
program,
budget_ms = budget.as_millis() as u64,
"helper did not exit within its budget — killed it (a wedged compositor/session bus is the \
usual cause); treating it as a failed query"
);
Error::new(
ErrorKind::TimedOut,
format!("`{program}` did not exit within {budget:?}"),
)
}
#[cfg(test)]
mod tests {
use super::*;
/// A helper that never exits must be killed at the budget and reported as `TimedOut` — the
/// whole point of the module (an unbounded `status()` here is what wedged a whole session).
#[test]
fn a_hung_child_is_killed_at_the_budget() {
let started = Instant::now();
let err = status_within(Command::new("sleep").arg("30"), Duration::from_millis(150))
.expect_err("must time out");
assert_eq!(err.kind(), ErrorKind::TimedOut);
assert!(
started.elapsed() < Duration::from_secs(5),
"must return at its budget, not the child's lifetime (took {:?})",
started.elapsed()
);
}
/// The normal path is unaffected: a quick command still yields its status and its output.
#[test]
fn a_quick_child_returns_normally() {
let st = status_within(&mut Command::new("true"), Duration::from_secs(5)).expect("ran");
assert!(st.success());
let out = output_within(
Command::new("echo").arg("punktfunk"),
Duration::from_secs(5),
)
.expect("ran");
assert!(out.status.success());
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "punktfunk");
}
}
+39 -5
View File
@@ -84,20 +84,28 @@ fn topology_str() -> String {
/// `quit` is the session's deliberate-quit flag: when the session ends with it set (the client closed /// `quit` is the session's deliberate-quit flag: when the session ends with it set (the client closed
/// with the quit application code — a user "stop", not a network drop), the display is torn down /// with the quit application code — a user "stop", not a network drop), the display is torn down
/// **immediately**, skipping the keep-alive linger. A bare disconnect leaves it `false` → normal linger. /// **immediately**, skipping the keep-alive linger. A bare disconnect leaves it `false` → normal linger.
///
/// `supersedes`: the pool gen of a display this acquire REPLACES (a mid-stream mode switch creates
/// the new display before retiring the old — create-before-drop). The replacement inherits group
/// topology ownership: without this, the dying predecessor counts as a live sibling and the new
/// display "extends" behind it, losing a Primary/Exclusive topology on every resize. `None`
/// everywhere else.
pub fn acquire( pub fn acquire(
vd: &mut Box<dyn super::VirtualDisplay>, vd: &mut Box<dyn super::VirtualDisplay>,
mode: super::Mode, mode: super::Mode,
quit: std::sync::Arc<std::sync::atomic::AtomicBool>, quit: std::sync::Arc<std::sync::atomic::AtomicBool>,
supersedes: Option<u64>,
) -> Result<super::VirtualOutput> { ) -> Result<super::VirtualOutput> {
let backend = vd.name(); let backend = vd.name();
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
let out = linux::acquire(vd, mode, quit); let out = linux::acquire(vd, mode, quit, supersedes);
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
let out = { let out = {
// Windows leases in the manager (its own linger); its deliberate-quit skip is wired through // Windows leases in the manager (its own linger); its deliberate-quit skip is wired through
// `VirtualDisplay::set_quit_flag` on the backend instance (set by the session before any // `VirtualDisplay::set_quit_flag` on the backend instance (set by the session before any
// `create`, so the retry-hold lease gets it too) — not through this parameter. // `create`, so the retry-hold lease gets it too) — not through this parameter. The
let _ = quit; // supersede handoff is Linux-pool-only too (the manager resizes in place).
let _ = (quit, supersedes);
vd.create(mode) vd.create(mode)
}; };
if out.is_ok() { if out.is_ok() {
@@ -260,6 +268,11 @@ mod linux {
/// Generation stamp: a [`DisplayLease`] only releases if its gen still matches (a stale lease /// Generation stamp: a [`DisplayLease`] only releases if its gen still matches (a stale lease
/// — its entry was reused + re-stamped — is a no-op). /// — its entry was reused + re-stamped — is a no-op).
gen: u64, gen: u64,
/// The out-of-band-cursor mode this display was CREATED with (Phase B): metadata-pointer
/// (cursor-channel session) vs compositor-embedded. Reuse requires an exact match — a kept
/// embedded display has no cursor metadata for a channel session to forward, and a kept
/// metadata display would leave a channel-less session with no pointer in its frames.
hw_cursor: bool,
} }
/// A per-group topology-restore action (see [`Entry::topology_restore`]). /// A per-group topology-restore action (see [`Entry::topology_restore`]).
@@ -421,6 +434,7 @@ mod linux {
vd: &mut Box<dyn VirtualDisplay>, vd: &mut Box<dyn VirtualDisplay>,
mode: Mode, mode: Mode,
quit: Arc<AtomicBool>, quit: Arc<AtomicBool>,
supersedes: Option<u64>,
) -> Result<VirtualOutput> { ) -> Result<VirtualOutput> {
ensure_timer(); ensure_timer();
let backend = vd.name(); let backend = vd.name();
@@ -464,6 +478,7 @@ mod linux {
) && e.backend == backend ) && e.backend == backend
&& e.mode == mode && e.mode == mode
&& e.launch == launch && e.launch == launch
&& e.hw_cursor == vd.hw_cursor()
&& epoch_matches(e.backend, e.epoch, cur_epoch) && epoch_matches(e.backend, e.epoch, cur_epoch)
}) })
.map(|e| (e.gen, e.node_id)) .map(|e| (e.gen, e.node_id))
@@ -535,9 +550,19 @@ mod linux {
// §6.1) — so a topology-establishing backend (Mutter exclusive) extends into an already-exclusive // §6.1) — so a topology-establishing backend (Mutter exclusive) extends into an already-exclusive
// desktop rather than re-clobbering the first session's virtual. Best-effort (a concurrent create // desktop rather than re-clobbering the first session's virtual. Best-effort (a concurrent create
// is a narrow race); single-session is always `first == true` → today's behavior. // is a narrow race); single-session is always `first == true` → today's behavior.
// Siblings that don't demote the newcomer:
// * the display this acquire SUPERSEDES (mode switch, create-before-drop) — still Active
// here because the old lease drops only after the new pipeline is up, but it's leaving,
// and deferring to it loses the group's Primary/Exclusive topology on every resize;
// * kept (Lingering/Pinned) entries — no session owns them, so there is no live desktop to
// clobber; a new session next to an unclaimed leftover should still establish topology.
let first_in_group = { let first_in_group = {
let es = r.entries.lock().unwrap(); let es = r.entries.lock().unwrap();
!es.iter().any(|e| e.backend == backend) !es.iter().any(|e| {
e.backend == backend
&& Some(e.gen) != supersedes
&& matches!(e.life, lifecycle::State::Active { .. })
})
}; };
vd.set_first_in_group(first_in_group); vd.set_first_in_group(first_in_group);
@@ -565,6 +590,11 @@ mod linux {
let node_id = real.node_id; let node_id = real.node_id;
let preferred_mode = real.preferred_mode; let preferred_mode = real.preferred_mode;
// Fresh creates only: the backend may have birthed the output at a sacrificial mode whose
// stream must renegotiate before frames count (KWin >60 Hz — see backend.rs). A REUSED kept
// display already renegotiated in its prior session (the producer's rebuilt offer persists
// across consumer reconnects), so the reuse path above correctly leaves the flag off.
let expect_exact_dims = real.expect_exact_dims;
// The backend's topology-restore action (KWin `exclusive` → re-enable the disabled physicals), // The backend's topology-restore action (KWin `exclusive` → re-enable the disabled physicals),
// lifted into the group so it runs once when the group's last member drops (§6.1), not at this // lifted into the group so it runs once when the group's last member drops (§6.1), not at this
// session's teardown. `None` for non-exclusive / non-first / backends whose topology auto-reverts. // session's teardown. `None` for non-exclusive / non-first / backends whose topology auto-reverts.
@@ -584,6 +614,7 @@ mod linux {
launch: launch.clone(), launch: launch.clone(),
epoch: cur_epoch, epoch: cur_epoch,
gen, gen,
hw_cursor: vd.hw_cursor(),
}; };
// Compute this new display's position in its group (design §6.2) BEFORE pushing, then push // Compute this new display's position in its group (design §6.2) BEFORE pushing, then push
@@ -628,7 +659,9 @@ mod linux {
if (position.x, position.y) != (0, 0) { if (position.x, position.y) != (0, 0) {
vd.apply_position(position.x, position.y); vd.apply_position(position.x, position.y);
} }
Ok(output_for(node_id, preferred_mode, gen, quit, false)) let mut out = output_for(node_id, preferred_mode, gen, quit, false);
out.expect_exact_dims = expect_exact_dims;
Ok(out)
} }
/// The linger a releasing session actually gets. A deliberate quit (`force_immediate` — the /// The linger a releasing session actually gets. A deliberate quit (`force_immediate` — the
@@ -1003,6 +1036,7 @@ mod linux {
launch: None, launch: None,
epoch: 0, epoch: 0,
gen, gen,
hw_cursor: false,
} }
} }
@@ -162,6 +162,15 @@ pub fn launch_into_gamescope_session(cmd: &str) -> Result<std::process::Child> {
gamescope::launch_into_session(cmd) gamescope::launch_into_session(cmd)
} }
/// Every nested Xwayland `(DISPLAY, XAUTHORITY)` of the running gamescope session for the XFixes
/// cursor source (remote-desktop-sweep Phase C) — gamescope can run several, and the pointer is on
/// whichever is focused. Empty when no gamescope session is running / it exposes no Xwayland (the
/// host then leaves gamescope cursorless, today's behaviour).
#[cfg(target_os = "linux")]
pub fn gamescope_xwayland_cursor_targets() -> Vec<(String, Option<String>)> {
gamescope::xwayland_cursor_targets()
}
/// B2: has a **dedicated** gamescope game session's game exited (its `node_id` doesn't reappear within a /// B2: has a **dedicated** gamescope game session's game exited (its `node_id` doesn't reappear within a
/// short window after capture loss)? The dedicated-spawn session ends cleanly on `true` instead of the /// short window after capture loss)? The dedicated-spawn session ends cleanly on `true` instead of the
/// capture-loss rebuild. Scoped to the session's OWN node so a coexisting gamescope doesn't mask the /// capture-loss rebuild. Scoped to the session's OWN node so a coexisting gamescope doesn't mask the
@@ -207,6 +216,20 @@ pub fn cancel_pending_tv_restore() {
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
pub fn cancel_pending_tv_restore() {} pub fn cancel_pending_tv_restore() {}
/// Can the MANAGED gamescope path stand a session up from nothing on this box (SteamOS's
/// `gamescope-session` launcher or Bazzite's `gamescope-session-plus` present)? Lets the connect
/// path route a "no live graphical session" box to the gamescope takeover — which rebuilds the
/// session at the client's mode — instead of failing the connect. Always `false` off Linux.
#[cfg(target_os = "linux")]
pub fn managed_session_available() -> bool {
gamescope::managed_session_available()
}
#[cfg(not(target_os = "linux"))]
pub fn managed_session_available() -> bool {
false
}
/// Call when a client session ends: if the host-managed gamescope path took over a box's autologin /// Call when a client session ends: if the host-managed gamescope path took over a box's autologin
/// gaming session (stopped its single-instance Steam to stream at the client's mode), **schedule** a /// gaming session (stopped its single-instance Steam to stream at the client's mode), **schedule** a
/// debounced restore so the TV returns to gaming mode — unless a client reconnects within the window /// debounced restore so the TV returns to gaming mode — unless a client reconnects within the window
+40 -19
View File
@@ -6,6 +6,15 @@
use super::*; use super::*;
/// Budget for one `systemctl --user` / `dbus-update-activation-environment` call.
///
/// These talk to the session bus, and a bus that is itself restarting or wedged answers nothing —
/// unbounded, that pinned the caller (on the host, the session's stream thread) forever. A restart
/// of the portal units is the slowest legitimate case, hence the generous window; missing it just
/// means the portal env settles late, which the callers already treat as best-effort.
#[cfg(target_os = "linux")]
const SYSTEMD_BUDGET: std::time::Duration = std::time::Duration::from_secs(10);
/// The **session epoch** — bumped whenever session detection observes a different compositor /// The **session epoch** — bumped whenever session detection observes a different compositor
/// *instance*: an [`ActiveKind`] change, **or** a new compositor PID for the same kind (the /// *instance*: an [`ActiveKind`] change, **or** a new compositor PID for the same kind (the
/// Desktop→Game→Desktop bounce that brings up a fresh KWin/gamescope with an unrelated node-id space). /// Desktop→Game→Desktop bounce that brings up a fresh KWin/gamescope with an unrelated node-id space).
@@ -86,9 +95,15 @@ pub fn observe_session_instance(active: &ActiveSession) {
/// via the next [`settle_desktop_portal`], so scrubbing on a bounce is harmless.) /// via the next [`settle_desktop_portal`], so scrubbing on a bounce is harmless.)
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
fn scrub_desktop_manager_env() { fn scrub_desktop_manager_env() {
let _ = std::process::Command::new("systemctl") let _ = crate::proc::status_within(
.args(["--user", "unset-environment", "WAYLAND_DISPLAY", "DISPLAY"]) std::process::Command::new("systemctl").args([
.status(); "--user",
"unset-environment",
"WAYLAND_DISPLAY",
"DISPLAY",
]),
SYSTEMD_BUDGET,
);
} }
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
@@ -499,40 +514,46 @@ pub fn settle_desktop_portal(chosen: Compositor) {
]; ];
// Push our (correct) env into the systemd --user manager + the D-Bus activation environment so a // Push our (correct) env into the systemd --user manager + the D-Bus activation environment so a
// re-activated portal/backend inherits the live session. // re-activated portal/backend inherits the live session.
let _ = std::process::Command::new("systemctl") let _ = crate::proc::status_within(
.args(["--user", "import-environment"]) std::process::Command::new("systemctl")
.args(VARS) .args(["--user", "import-environment"])
.status(); .args(VARS),
let _ = std::process::Command::new("dbus-update-activation-environment") SYSTEMD_BUDGET,
.arg("--systemd") );
.args(VARS) let _ = crate::proc::status_within(
.status(); std::process::Command::new("dbus-update-activation-environment")
.arg("--systemd")
.args(VARS),
SYSTEMD_BUDGET,
);
// KWin input goes through the xdg RemoteDesktop portal; the frontend routes RemoteDesktop to a // KWin input goes through the xdg RemoteDesktop portal; the frontend routes RemoteDesktop to a
// backend by its OWN startup XDG_CURRENT_DESKTOP, so restart it (+ the KDE backend) to re-read // backend by its OWN startup XDG_CURRENT_DESKTOP, so restart it (+ the KDE backend) to re-read
// the now-live session, then let it settle before the injector reopens against it. // the now-live session, then let it settle before the injector reopens against it.
if chosen == Compositor::Kwin { if chosen == Compositor::Kwin {
let _ = std::process::Command::new("systemctl") let _ = crate::proc::status_within(
.args([ std::process::Command::new("systemctl").args([
"--user", "--user",
"try-restart", "try-restart",
"xdg-desktop-portal-kde.service", "xdg-desktop-portal-kde.service",
"xdg-desktop-portal.service", "xdg-desktop-portal.service",
]) ]),
.status(); SYSTEMD_BUDGET,
);
std::thread::sleep(std::time::Duration::from_millis(600)); std::thread::sleep(std::time::Duration::from_millis(600));
} }
// Hyprland capture rides the xdg ScreenCast portal serviced by xdph (G5): on a mid-stream switch // Hyprland capture rides the xdg ScreenCast portal serviced by xdph (G5): on a mid-stream switch
// xdph may still hold the old session's Wayland/instance env, so restart it (+ the frontend) to // xdph may still hold the old session's Wayland/instance env, so restart it (+ the frontend) to
// re-read the now-live session, mirroring the KWin settling above. // re-read the now-live session, mirroring the KWin settling above.
if chosen == Compositor::Hyprland { if chosen == Compositor::Hyprland {
let _ = std::process::Command::new("systemctl") let _ = crate::proc::status_within(
.args([ std::process::Command::new("systemctl").args([
"--user", "--user",
"try-restart", "try-restart",
"xdg-desktop-portal-hyprland.service", "xdg-desktop-portal-hyprland.service",
"xdg-desktop-portal.service", "xdg-desktop-portal.service",
]) ]),
.status(); SYSTEMD_BUDGET,
);
std::thread::sleep(std::time::Duration::from_millis(600)); std::thread::sleep(std::time::Duration::from_millis(600));
} }
tracing::info!( tracing::info!(
@@ -71,6 +71,15 @@ struct Monitor {
/// selectable). The pin is never re-issued on reuse, so this is what the driver still renders /// selectable). The pin is never re-issued on reuse, so this is what the driver still renders
/// on — [`warn_if_pick_moved`] compares the CURRENT pick against it. /// on — [`warn_if_pick_moved`] compares the CURRENT pick against it.
render_pin: Option<LUID>, render_pin: Option<LUID>,
/// This monitor was ADDed with the v5 hardware-cursor flag (already driver-proto-gated) —
/// preserved across a re-arrival resize so the recreated monitor keeps the cursor channel.
hw_cursor: bool,
/// The ADD reply flagged this monitor's OS target as carrying an IRREVOCABLE hardware-cursor
/// declare from an earlier session (§8.6): DWM excludes the pointer from its frames forever
/// (the sticky state survives monitor REMOVE→ADD via the stable per-client target id), so a
/// session WITHOUT the cursor channel must self-composite — carried into
/// [`WinCaptureTarget`] for the IDD-push capturer's forced-composite gate.
cursor_excluded: bool,
/// The driver's WUDFHost pid (from the ADD reply) — carried into [`WinCaptureTarget`] so the /// The driver's WUDFHost pid (from the ADD reply) — carried into [`WinCaptureTarget`] so the
/// IDD-push capturer knows where to duplicate the sealed frame channel's handles. The SAME /// IDD-push capturer knows where to duplicate the sealed frame channel's handles. The SAME
/// process for every parallel monitor (one devnode → one WUDFHost hosts all publishers), which /// process for every parallel monitor (one devnode → one WUDFHost hosts all publishers), which
@@ -98,6 +107,7 @@ impl Monitor {
gdi_name: n, gdi_name: n,
target_id: self.target_id, target_id: self.target_id,
wudf_pid: self.wudf_pid, wudf_pid: self.wudf_pid,
cursor_excluded: self.cursor_excluded,
}) })
} }
} }
@@ -267,10 +277,44 @@ pub fn vdm() -> &'static VirtualDisplayManager {
/// for the process lifetime — a dead one is RETIRED (kept alive, see [`DeviceSlot`]), so a stale copy /// for the process lifetime — a dead one is RETIRED (kept alive, see [`DeviceSlot`]), so a stale copy
/// can only fail IOCTLs, never dangle. `None` before the first backend open — impossible for a /// can only fail IOCTLs, never dangle. `None` before the first backend open — impossible for a
/// capturer, which only exists on a monitor the manager created. /// capturer, which only exists on a monitor the manager created.
/// Can this host's pf-vdisplay driver run the v5 hardware-cursor channel? Reads the
/// handshake-latched protocol version, opening the control device once if no session has
/// opened it yet this service run (the same open every session performs anyway) — so the
/// Welcome-time capability decision never guesses. `false` when the driver is missing/stale.
///
/// The FIRST session's Welcome precedes any backend construction (`vdisplay::open` runs at
/// display prep, after the Welcome), so this must not assume an initialised manager —
/// `init` is idempotent and constructing the driver facade is free (on-glass finding: the
/// `vdm()` expect panicked the very first handshake of a fresh service).
pub fn hw_cursor_capable() -> bool {
let m = init(Box::new(crate::driver::PfVdisplayDriver));
let v = m.driver_proto.load(Ordering::Relaxed);
if v != 0 {
return v >= 5;
}
let _ = m.ensure_device();
m.driver_proto.load(Ordering::Relaxed) >= 5
}
pub fn control_device_handle() -> Option<HANDLE> { pub fn control_device_handle() -> Option<HANDLE> {
VDM.get().and_then(VirtualDisplayManager::device_handle) VDM.get().and_then(VirtualDisplayManager::device_handle)
} }
/// Re-commit the CURRENT display config under the manager `state` lock (the sole-topology-mutator
/// contract of [`force_mode_reenumeration`]). The secure-desktop guard's actuator: the OS only
/// reverts a path to its software-cursor default ON a mode commit, so standing the hardware-cursor
/// declare down (`IOCTL_SET_CURSOR_FORWARD` off) needs this nudge for UAC/Winlogon to actually
/// render. `false` (no-op) before the first backend open — no monitors exist to re-commit for.
pub fn force_recommit() -> bool {
let Some(m) = VDM.get() else {
return false;
};
let _guard = m.state.lock().unwrap();
// SAFETY: `force_mode_reenumeration`'s contract is "call under the manager `state` lock";
// held above. The call reads + re-applies the current CCD config over owned locals.
unsafe { pf_win_display::win_display::force_mode_reenumeration() }
}
/// Best-effort "is this WUDFHost pid still alive?" — the monitor-liveness probe for the JOIN path. /// Best-effort "is this WUDFHost pid still alive?" — the monitor-liveness probe for the JOIN path.
/// `OpenProcess` failing (pid reaped) or the process being signaled ⇒ dead. Pid reuse could /// `OpenProcess` failing (pid reaped) or the process being signaled ⇒ dead. Pid reuse could
/// theoretically alias a fresh process and read "alive"; the joining session then just retries into /// theoretically alias a fresh process and read "alive"; the joining session then just retries into
@@ -390,6 +434,7 @@ impl VirtualDisplayManager {
mode: Mode, mode: Mode,
client_fp: Option<[u8; 32]>, client_fp: Option<[u8; 32]>,
client_hdr: Option<punktfunk_core::quic::HdrMeta>, client_hdr: Option<punktfunk_core::quic::HdrMeta>,
hw_cursor: bool,
quit: Option<Arc<AtomicBool>>, quit: Option<Arc<AtomicBool>>,
) -> Result<VirtualOutput> { ) -> Result<VirtualOutput> {
// Console-session guard: a host outside the ACTIVE console session cannot drive the display // Console-session guard: a host outside the ACTIVE console session cannot drive the display
@@ -622,7 +667,9 @@ impl VirtualDisplayManager {
// SAFETY: `create_monitor` requires `dev` to be a valid control handle; `dev` is the handle // SAFETY: `create_monitor` requires `dev` to be a valid control handle; `dev` is the handle
// `ensure_device()` returned above (cached handles are never closed — a dead one is retired, // `ensure_device()` returned above (cached handles are never closed — a dead one is retired,
// kept alive; see `DeviceSlot`), and we hold the `state` lock. // kept alive; see `DeviceSlot`), and we hold the `state` lock.
let mon = match unsafe { self.create_monitor(dev, mode, slot, client_hdr, &mut inner) } { let mon = match unsafe {
self.create_monitor(dev, mode, slot, client_hdr, hw_cursor, &mut inner)
} {
// The cached device died under us (driver upgrade / WUDFHost restart, detected only // The cached device died under us (driver upgrade / WUDFHost restart, detected only
// now — e.g. the host sat idle past the pinger-less window). Retire it, reopen, and // now — e.g. the host sat idle past the pinger-less window). Retire it, reopen, and
// retry ONCE so the reconnect-after-driver-restart succeeds first try instead of // retry ONCE so the reconnect-after-driver-restart succeeds first try instead of
@@ -635,7 +682,7 @@ impl VirtualDisplayManager {
); );
// SAFETY: as above — `dev` is the handle the reopening `ensure_device` just // SAFETY: as above — `dev` is the handle the reopening `ensure_device` just
// returned, and the `state` lock is still held. // returned, and the `state` lock is still held.
unsafe { self.create_monitor(dev, mode, slot, client_hdr, &mut inner)? } unsafe { self.create_monitor(dev, mode, slot, client_hdr, hw_cursor, &mut inner)? }
} }
r => r?, r => r?,
}; };
@@ -861,6 +908,7 @@ impl VirtualDisplayManager {
mode: Mode, mode: Mode,
slot: u32, slot: u32,
client_hdr: Option<punktfunk_core::quic::HdrMeta>, client_hdr: Option<punktfunk_core::quic::HdrMeta>,
hw_cursor: bool,
inner: &mut MgrInner, inner: &mut MgrInner,
) -> Result<Monitor> { ) -> Result<Monitor> {
// The slot id doubles as the driver-preferred monitor id (EDID serial / ConnectorIndex), so // The slot id doubles as the driver-preferred monitor id (EDID serial / ConnectorIndex), so
@@ -868,13 +916,17 @@ impl VirtualDisplayManager {
// `0` (anonymous) = the driver auto-allocates the lowest-free id. // `0` (anonymous) = the driver auto-allocates the lowest-free id.
let preferred_id = slot; let preferred_id = slot;
let render_pin = resolve_render_pin(); let render_pin = resolve_render_pin();
// Hardware cursor only against a driver that implements the v5 channel: an older driver
// ignores the AddRequest field anyway (composited cursor), but gating here keeps the
// capture layer from creating + delivering a section nobody will ever publish into.
let hw_cursor = hw_cursor && self.driver_proto.load(Ordering::Relaxed) >= 5;
// SAFETY: `create_monitor`'s own `# Safety` contract guarantees `dev` is the live control // SAFETY: `create_monitor`'s own `# Safety` contract guarantees `dev` is the live control
// handle; we forward it unchanged to `add_monitor`, whose precondition is exactly that. // handle; we forward it unchanged to `add_monitor`, whose precondition is exactly that.
// `render_pin` is an `Option<LUID>` by value (plain `Copy`), so no borrowed memory // `render_pin` is an `Option<LUID>` by value (plain `Copy`), so no borrowed memory
// crosses the call. // crosses the call.
let added = unsafe { let added = unsafe {
self.driver self.driver
.add_monitor(dev, mode, render_pin, preferred_id, client_hdr)? .add_monitor(dev, mode, render_pin, preferred_id, client_hdr, hw_cursor)?
}; };
// Mandatory keepalive: ping inside the watchdog window or the driver tears all displays down. // Mandatory keepalive: ping inside the watchdog window or the driver tears all displays down.
@@ -1061,6 +1113,8 @@ impl VirtualDisplayManager {
resolved_monitor_id: added.resolved_monitor_id, resolved_monitor_id: added.resolved_monitor_id,
position: (0, 0), position: (0, 0),
gen: self.gen.fetch_add(1, Ordering::Relaxed), gen: self.gen.fetch_add(1, Ordering::Relaxed),
hw_cursor,
cursor_excluded: added.cursor_excluded,
}) })
} }
@@ -1231,7 +1285,7 @@ impl VirtualDisplayManager {
// values passed by value — no borrow crosses the call. // values passed by value — no borrow crosses the call.
let added = unsafe { let added = unsafe {
self.driver self.driver
.add_monitor(dev, mode, render_pin, slot, client_hdr) .add_monitor(dev, mode, render_pin, slot, client_hdr, old.hw_cursor)
.context("re-arrival ADD at the new mode")? .context("re-arrival ADD at the new mode")?
}; };
self.ensure_pinger(); self.ensure_pinger();
@@ -1283,6 +1337,10 @@ impl VirtualDisplayManager {
resolved_monitor_id: added.resolved_monitor_id, resolved_monitor_id: added.resolved_monitor_id,
position: old.position, position: old.position,
gen: old.gen, gen: old.gen,
hw_cursor: old.hw_cursor,
// Fresh from THIS reply, not `old`: the driver's per-target declare registry is the
// ground truth (this session may itself have declared since the original ADD).
cursor_excluded: added.cursor_excluded,
}) })
} }
@@ -22,6 +22,11 @@ pub(crate) struct AddedMonitor {
pub luid: LUID, pub luid: LUID,
pub wudf_pid: u32, pub wudf_pid: u32,
pub resolved_monitor_id: u32, pub resolved_monitor_id: u32,
/// The driver reports the OS target already carries an IRREVOCABLE hardware-cursor declare
/// from an earlier session (`AddReply::cursor_excluded`, remote-desktop-sweep §8.6): DWM
/// excludes the pointer from this target's frames forever, so a session without the cursor
/// channel must self-composite (GDI poller + blend) or stream a cursor-less desktop.
pub cursor_excluded: bool,
} }
/// The backend-specific IOCTL surface — the *only* thing that differs between SudoVDA and pf-vdisplay. /// The backend-specific IOCTL surface — the *only* thing that differs between SudoVDA and pf-vdisplay.
@@ -56,6 +61,7 @@ pub(crate) trait VdisplayDriver: Send + Sync {
render_luid: Option<LUID>, render_luid: Option<LUID>,
preferred_monitor_id: u32, preferred_monitor_id: u32,
client_hdr: Option<punktfunk_core::quic::HdrMeta>, client_hdr: Option<punktfunk_core::quic::HdrMeta>,
hw_cursor: bool,
) -> Result<AddedMonitor>; ) -> Result<AddedMonitor>;
/// Refresh the LIVE monitor `key`'s advertised mode list to lead with `mode` (the in-place /// Refresh the LIVE monitor `key`'s advertised mode list to lead with `mode` (the in-place
/// mid-stream resize, latency plan P2 — pf-vdisplay `IOCTL_UPDATE_MODES`, driver protocol v4). /// mid-stream resize, latency plan P2 — pf-vdisplay `IOCTL_UPDATE_MODES`, driver protocol v4).
@@ -234,6 +234,55 @@ pub unsafe fn send_frame_channel(dev: HANDLE, req: &control::SetFrameChannelRequ
.context("pf-vdisplay SET_FRAME_CHANNEL") .context("pf-vdisplay SET_FRAME_CHANNEL")
} }
/// Deliver a monitor's hardware-cursor section (`IOCTL_SET_CURSOR_CHANNEL`, proto v5) — the
/// cursor sibling of [`send_frame_channel`], same delivery/ownership contract.
///
/// # Safety
/// `dev` must be a live pf-vdisplay control handle (see [`super::manager::control_device_handle`]).
pub unsafe fn send_cursor_channel(
dev: HANDLE,
req: &control::SetCursorChannelRequest,
) -> Result<()> {
let mut none: [u8; 0] = [];
// SAFETY: per this fn's contract `dev` is the live control handle; `bytes_of(req)` borrows the
// caller's request across this synchronous call; no output buffer.
unsafe {
ioctl(
dev,
control::IOCTL_SET_CURSOR_CHANNEL,
bytemuck::bytes_of(req),
&mut none,
)
}
.map(|_| ())
.context("pf-vdisplay SET_CURSOR_CHANNEL")
}
/// Flip a LIVE monitor's hardware-cursor declaration (`IOCTL_SET_CURSOR_FORWARD`, proto v6) —
/// the mid-stream cursor-render flip. Fails against a pre-v6 driver (unknown IOCTL); callers
/// log and keep the declared-at-ADD behavior.
///
/// # Safety
/// `dev` must be a live pf-vdisplay control handle (see [`super::manager::control_device_handle`]).
pub unsafe fn send_cursor_forward(
dev: HANDLE,
req: &control::SetCursorForwardRequest,
) -> Result<()> {
let mut none: [u8; 0] = [];
// SAFETY: per this fn's contract `dev` is the live control handle; `bytes_of(req)` borrows the
// caller's request across this synchronous call; no output buffer.
unsafe {
ioctl(
dev,
control::IOCTL_SET_CURSOR_FORWARD,
bytemuck::bytes_of(req),
&mut none,
)
}
.map(|_| ())
.context("pf-vdisplay SET_CURSOR_FORWARD")
}
/// RAII over a SetupAPI device-info list: every exit path of [`open_device`] destroys it (the error /// RAII over a SetupAPI device-info list: every exit path of [`open_device`] destroys it (the error
/// paths used to leak one `HDEVINFO` per failed open — and a driverless / mid-upgrade box probes /// paths used to leak one `HDEVINFO` per failed open — and a driverless / mid-upgrade box probes
/// repeatedly). /// repeatedly).
@@ -460,6 +509,7 @@ impl VdisplayDriver for PfVdisplayDriver {
render_luid: Option<LUID>, render_luid: Option<LUID>,
preferred_monitor_id: u32, preferred_monitor_id: u32,
client_hdr: Option<punktfunk_core::quic::HdrMeta>, client_hdr: Option<punktfunk_core::quic::HdrMeta>,
hw_cursor: bool,
) -> Result<AddedMonitor> { ) -> Result<AddedMonitor> {
let session_id = next_session_id(); let session_id = next_session_id();
// The client display's volume rides into the monitor's EDID CTA HDR block; all-zero = // The client display's volume rides into the monitor's EDID CTA HDR block; all-zero =
@@ -485,7 +535,11 @@ impl VdisplayDriver for PfVdisplayDriver {
max_luminance_nits, max_luminance_nits,
max_frame_avg_nits, max_frame_avg_nits,
min_luminance_millinits, min_luminance_millinits,
_reserved: 0, // v5 cursor channel: the driver declares an IddCx hardware cursor for this monitor
// (DWM stops compositing the pointer into the frame); the capture layer delivers the
// CursorShm section right after its ring. Zero toward older drivers is harmless —
// the host only sets this when the handshake-reported proto is >= 5.
hw_cursor: hw_cursor as u32,
}; };
// SET_RENDER_ADAPTER (opt-in; pf-vdisplay IMPLEMENTS it). Non-fatal on failure: the driver reports // SET_RENDER_ADAPTER (opt-in; pf-vdisplay IMPLEMENTS it). Non-fatal on failure: the driver reports
// its real render LUID in the shared header, so the host binds correctly even if this is ignored. // its real render LUID in the shared header, so the host binds correctly even if this is ignored.
@@ -551,10 +605,13 @@ impl VdisplayDriver for PfVdisplayDriver {
})?; })?;
// Fail closed on a short reply — `target_id`/`wudf_pid`/`luid` below feed OpenProcess + the // Fail closed on a short reply — `target_id`/`wudf_pid`/`luid` below feed OpenProcess + the
// WUDFHost verification, so don't decode a partially-written (zeroed) reply as authoritative. // WUDFHost verification, so don't decode a partially-written (zeroed) reply as authoritative.
if (n as usize) < size_of::<control::AddReply>() { // The LEGACY size, not the full struct: an un-upgraded driver writes only the prefix before
// the `cursor_excluded` tail; `out` is zero-initialized, so the missing tail reads `0`
// (= unknown/clean — exactly what a driver that can't track declares should report).
if (n as usize) < control::ADD_REPLY_LEGACY_SIZE {
anyhow::bail!( anyhow::bail!(
"pf-vdisplay ADD returned {n} bytes, expected {}", "pf-vdisplay ADD returned {n} bytes, expected at least {}",
size_of::<control::AddReply>() control::ADD_REPLY_LEGACY_SIZE
); );
} }
// `pod_read_unaligned` (NOT `from_bytes`): `out` is a stack `[u8; N]` with no guaranteed 4-byte // `pod_read_unaligned` (NOT `from_bytes`): `out` is a stack `[u8; N]` with no guaranteed 4-byte
@@ -569,6 +626,7 @@ impl VdisplayDriver for PfVdisplayDriver {
target_id = reply.target_id, target_id = reply.target_id,
adapter_luid = %format_args!("{:#x}", luid.LowPart), adapter_luid = %format_args!("{:#x}", luid.LowPart),
wudf_pid = reply.wudf_pid, wudf_pid = reply.wudf_pid,
cursor_excluded = reply.cursor_excluded != 0,
"pf-vdisplay monitor created {}x{}@{}", "pf-vdisplay monitor created {}x{}@{}",
mode.width, mode.width,
mode.height, mode.height,
@@ -605,6 +663,7 @@ impl VdisplayDriver for PfVdisplayDriver {
luid, luid,
wudf_pid: reply.wudf_pid, wudf_pid: reply.wudf_pid,
resolved_monitor_id: reply.resolved_monitor_id, resolved_monitor_id: reply.resolved_monitor_id,
cursor_excluded: reply.cursor_excluded != 0,
}) })
} }
@@ -682,6 +741,10 @@ pub struct PfVdisplayDisplay {
/// freshly created monitor's EDID advertises this volume so host apps tone-map to the client's /// freshly created monitor's EDID advertises this volume so host apps tone-map to the client's
/// real panel. /// real panel.
client_hdr: Option<punktfunk_core::quic::HdrMeta>, client_hdr: Option<punktfunk_core::quic::HdrMeta>,
/// Declare an IddCx hardware cursor on the created monitor (the M2c cursor channel). Set by
/// [`set_hw_cursor`](VirtualDisplay::set_hw_cursor) before `create`; only honored when the
/// driver handshake reported proto >= 5.
hw_cursor: bool,
/// The session's deliberate-quit flag (`None` = no signal → the linger policy applies). Set by /// The session's deliberate-quit flag (`None` = no signal → the linger policy applies). Set by
/// [`set_quit_flag`](VirtualDisplay::set_quit_flag) before `create`; rides into every lease this /// [`set_quit_flag`](VirtualDisplay::set_quit_flag) before `create`; rides into every lease this
/// backend mints so a user "stop" tears the monitor down immediately instead of lingering. /// backend mints so a user "stop" tears the monitor down immediately instead of lingering.
@@ -694,6 +757,7 @@ impl PfVdisplayDisplay {
Ok(Self { Ok(Self {
client_fp: None, client_fp: None,
client_hdr: None, client_hdr: None,
hw_cursor: false,
quit: None, quit: None,
}) })
} }
@@ -712,12 +776,26 @@ impl VirtualDisplay for PfVdisplayDisplay {
self.client_hdr = hdr; self.client_hdr = hdr;
} }
fn set_hw_cursor(&mut self, on: bool) {
self.hw_cursor = on;
}
fn hw_cursor(&self) -> bool {
self.hw_cursor
}
fn set_quit_flag(&mut self, quit: std::sync::Arc<std::sync::atomic::AtomicBool>) { fn set_quit_flag(&mut self, quit: std::sync::Arc<std::sync::atomic::AtomicBool>) {
self.quit = Some(quit); self.quit = Some(quit);
} }
fn create(&mut self, mode: Mode) -> Result<VirtualOutput> { fn create(&mut self, mode: Mode) -> Result<VirtualOutput> {
super::manager::vdm().acquire(mode, self.client_fp, self.client_hdr, self.quit.clone()) super::manager::vdm().acquire(
mode,
self.client_fp,
self.client_hdr,
self.hw_cursor,
self.quit.clone(),
)
} }
} }
@@ -804,17 +882,12 @@ mod tests {
// Live-run diagnostics: surface the manager/backend tracing (activation ladder, settle // Live-run diagnostics: surface the manager/backend tracing (activation ladder, settle
// waits, UPDATE_MODES) on stdout — a bare test harness has no subscriber, which made the // waits, UPDATE_MODES) on stdout — a bare test harness has no subscriber, which made the
// first on-glass run blind. // first on-glass run blind.
let _ = tracing_subscriber::fmt() // (tracing-subscriber is not a dep of this crate — run the host binary for traced runs.)
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| "debug".into()),
)
.try_init();
// Context probe: can this process see the CCD active-path set at all? (`None` = the query // Context probe: can this process see the CCD active-path set at all? (`None` = the query
// itself fails in this session/window-station — the whole ladder would be blind, and a // itself fails in this session/window-station — the whole ladder would be blind, and a
// "monitor never activated" verdict would be an artifact of the test context.) // "monitor never activated" verdict would be an artifact of the test context.)
// SAFETY: CCD query over an owned empty slice (test-only diagnostics). // SAFETY: CCD query over an owned empty slice (test-only diagnostics).
let active0 = unsafe { crate::win_display::count_other_active(&[]) }; let active0 = unsafe { pf_win_display::win_display::count_other_active(&[]) };
println!("spike: CCD active paths visible before create: {active0:?}"); println!("spike: CCD active paths visible before create: {active0:?}");
let mut vd = PfVdisplayDisplay::new().expect("open pf-vdisplay"); let mut vd = PfVdisplayDisplay::new().expect("open pf-vdisplay");
let first = vd let first = vd
@@ -847,7 +920,7 @@ mod tests {
.target_id; .target_id;
let in_place = t1 == t2; let in_place = t1 == t2;
// SAFETY: CCD query over a Copy target id (test-only diagnostics). // SAFETY: CCD query over a Copy target id (test-only diagnostics).
let active = unsafe { crate::win_display::active_resolution(t2) }; let active = unsafe { pf_win_display::win_display::active_resolution(t2) };
println!( println!(
"in-place resize spike: in_place={in_place} (target {t1} -> {t2}) took {resize_ms} ms, \ "in-place resize spike: in_place={in_place} (target {t1} -> {t2}) took {resize_ms} ms, \
active resolution now {active:?}" active resolution now {active:?}"
+3
View File
@@ -31,5 +31,8 @@ windows = { version = "0.62", features = [
"Win32_System_LibraryLoader", "Win32_System_LibraryLoader",
# console_session_mismatch: WTSGetActiveConsoleSessionId + ProcessIdToSessionId + GetCurrentProcessId. # console_session_mismatch: WTSGetActiveConsoleSessionId + ProcessIdToSessionId + GetCurrentProcessId.
"Win32_System_RemoteDesktop", "Win32_System_RemoteDesktop",
# input_desktop: OpenInputDesktop/SetThreadDesktop/GetUserObjectInformationW — display writes
# follow the input desktop so a UAC/lock screen can't refuse them.
"Win32_System_StationsAndDesktops",
"Win32_System_Threading", "Win32_System_Threading",
] } ] }
+200
View File
@@ -0,0 +1,200 @@
//! Make display-config writes follow the INPUT desktop.
//!
//! Windows refuses `ChangeDisplaySettingsEx` / `SetDisplayConfig` issued from a thread that is not
//! on the desktop currently receiving input. While a UAC consent prompt (or the lock / logon
//! screen) is up, the input desktop is `Winlogon`; a host thread sitting on `WinSta0\Default` — the
//! desktop the service explicitly launches it onto — then gets `DISP_CHANGE_FAILED` /
//! `ERROR_ACCESS_DENIED` for EVERY write. A session starting in that window can never set its
//! virtual display's mode, so the capturer sizes its ring to a stale mode, every composed frame is
//! dropped for a size mismatch, and the client sits on a black screen until bring-up runs out of
//! retries. Field-reported 2026-07-23: a consent prompt left up on an unattended host made the box
//! unreachable until someone walked over to it.
//!
//! Measured on-glass that same day (RTX 4090 box, SYSTEM host in console session 2, a real
//! interactive consent prompt up, virtual display `\\.\DISPLAY42` active):
//!
//! ```text
//! INPUT desktop = Winlogon
//! UNBOUND CDS_TEST -> -1 (DISP_CHANGE_FAILED) <- the field failure, reproduced
//! UNBOUND SDC_VALIDATE -> 0x5 (ERROR_ACCESS_DENIED) <- the field failure, reproduced
//! bound thread desktop -> Winlogon
//! BOUND CDS_TEST -> 0 (DISP_CHANGE_SUCCESSFUL)
//! BOUND SDC_VALIDATE -> 0x0 (ERROR_SUCCESS)
//! ```
//!
//! The retry model mirrors `pf-inject`'s `sendinput.rs`: do NOT pay
//! `OpenInputDesktop`/`SetThreadDesktop` on every write — issue the write, and only when it fails
//! the way a wrong-desktop write fails do we rebind and retry once. That keeps the normal path
//! byte-for-byte as it was (a working write is never touched) and makes this strictly additive.
//!
//! Unlike sendinput's injector — a dedicated thread that KEEPS its binding — these helpers run on
//! shared/task threads, so the binding here is SCOPED: [`InputDesktopBinding`] restores the
//! thread's original desktop on drop. A thread left bound to a `Winlogon` desktop that is later
//! destroyed (prompt dismissed) would fail every subsequent display write for the life of the
//! process — the exact wedge this module exists to remove, so it must not be introduced here.
use windows::Win32::Foundation::HANDLE;
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, GetThreadDesktop, GetUserObjectInformationW, OpenInputDesktop, SetThreadDesktop,
DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS, HDESK, UOI_NAME,
};
use windows::Win32::System::Threading::GetCurrentThreadId;
/// `GENERIC_ALL` for the desktop open. The `windows` crate models desktop access as its own flag
/// type and doesn't export the generic rights, so spell it out (same constant `sendinput.rs` and
/// the cursor poller use).
const GENERIC_ALL: u32 = 0x1000_0000;
/// This thread's binding to the input desktop, restored on drop.
///
/// `GetThreadDesktop` returns a BORROWED handle (documented: it creates no new handle and must not
/// be closed), so only the handle `OpenInputDesktop` produced is closed here — and only after the
/// thread has been moved back off it.
pub(crate) struct InputDesktopBinding {
previous: HDESK,
input: HDESK,
/// `UOI_NAME` of the desktop bound to, for the "this write only landed because we rebound"
/// log. Read once at bind time (the failure path only), never in steady state.
name: String,
}
impl InputDesktopBinding {
/// Bind the calling thread to the desktop currently receiving input. `None` when there is no
/// reachable input desktop (not privileged for `Winlogon` — a host that is not SYSTEM) or the
/// rebind is refused, in which case the caller keeps whatever result it already had rather than
/// changing behaviour.
pub(crate) fn bind() -> Option<Self> {
// SAFETY: all four are FFI calls taking by-value args only. `GetThreadDesktop` yields a
// borrowed handle for THIS thread (never closed here). `OpenInputDesktop` yields an owned
// `HDESK` only on `Ok`; it is either installed by `SetThreadDesktop` (and then owned by the
// returned guard, which closes it exactly once in `Drop`) or closed right here on failure —
// so it is closed exactly once on every path and never used after close. `SetThreadDesktop`
// rebinds only the calling thread, which owns no windows or hooks (these are display-config
// worker threads), so it cannot fail on that account.
unsafe {
let previous = GetThreadDesktop(GetCurrentThreadId()).ok()?;
let input = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
)
.ok()?;
let name = desktop_name(input).unwrap_or_else(|| "<unnamed>".into());
if SetThreadDesktop(input).is_err() {
let _ = CloseDesktop(input);
return None;
}
Some(Self {
previous,
input,
name,
})
}
}
}
impl Drop for InputDesktopBinding {
fn drop(&mut self) {
// SAFETY: `self.previous` is the borrowed desktop this thread started on and `self.input`
// is the handle this guard uniquely owns. The thread is moved back FIRST, so the handle is
// no longer the thread's desktop when it is closed — closed exactly once, never used after.
unsafe {
let _ = SetThreadDesktop(self.previous);
let _ = CloseDesktop(self.input);
}
}
}
/// `UOI_NAME` of the current input desktop — `Some("Winlogon")` while a UAC consent prompt, the
/// lock screen or the logon screen owns input, `Some("Default")` in normal operation, `None` when
/// it cannot be opened at all.
pub(crate) fn input_desktop_name() -> Option<String> {
// SAFETY: `OpenInputDesktop` yields an owned handle only on `Ok`, which is closed exactly once
// below and not used after.
unsafe {
let h = OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
)
.ok()?;
let name = desktop_name(h);
let _ = CloseDesktop(h);
name
}
}
/// `UOI_NAME` of an already-open desktop handle. Borrows `h` — closing it stays the caller's job.
///
/// # Safety
/// `h` must be a live desktop handle for the duration of the call.
unsafe fn desktop_name(h: HDESK) -> Option<String> {
let mut name = [0u16; 64]; // "Default"/"Winlogon"/"Screen-saver" all fit with room to spare
let mut needed = 0u32;
// SAFETY: `h` is live per this fn's contract; `name`/`needed` are live out-params and the call
// writes at most `nlength` bytes, exactly the size passed.
GetUserObjectInformationW(
HANDLE(h.0),
UOI_NAME,
Some(name.as_mut_ptr().cast()),
(name.len() * 2) as u32,
Some(&mut needed),
)
.ok()?;
let len = name.iter().position(|&c| c == 0).unwrap_or(name.len());
Some(String::from_utf16_lossy(&name[..len]))
}
/// `true` when the input desktop is a SECURE one (`UOI_NAME` != `Default`: `Winlogon` for UAC
/// consent / lock / logon, or a screen-saver desktop) — i.e. when display writes from the host's
/// own desktop are being refused for that reason. `false` when it is normal OR unreadable: this
/// only ever phrases a diagnostic, so an unknown desktop must not claim the secure one is up.
pub(crate) fn input_desktop_is_secure() -> bool {
input_desktop_name().is_some_and(|n| !n.eq_ignore_ascii_case("Default"))
}
/// `ERROR_ACCESS_DENIED` — exactly how Windows refuses a `SetDisplayConfig` issued from a thread
/// that is not on the input desktop (measured, see the module header). Narrower than "any error" on
/// purpose: the CCD path also returns `ERROR_INVALID_PARAMETER` (0x57) for the unrelated
/// exclusive-mode topology bug, and re-issuing THAT on another desktop would only confuse its
/// diagnosis.
const SDC_ACCESS_DENIED: i32 = 5;
/// [`retry_on_input_desktop`] specialised for the CCD writes, which all return a Win32 error code.
pub(crate) fn retry_set_display_config(write: impl FnMut() -> i32) -> i32 {
retry_on_input_desktop(|rc| *rc == SDC_ACCESS_DENIED, write)
}
/// Run a display-config write; if it comes back the way a write from the wrong desktop comes back,
/// bind this thread to the CURRENT input desktop and run it exactly once more.
///
/// `denied` decides which results are worth a retry — keep it NARROW (the specific
/// wrong-desktop verdict), so a genuinely bad mode or a driver-level refusal is not re-issued and
/// mis-attributed. The binding is dropped before returning, so the calling thread always leaves on
/// the desktop it arrived on.
pub(crate) fn retry_on_input_desktop<T>(
denied: impl Fn(&T) -> bool,
mut write: impl FnMut() -> T,
) -> T {
let first = write();
if !denied(&first) {
return first;
}
// Only worth the rebind when the input desktop is genuinely elsewhere; on a normal desktop the
// refusal means something else entirely and a retry would just repeat it.
let Some(binding) = InputDesktopBinding::bind() else {
return first;
};
let second = write();
if !denied(&second) {
// Say so. A silent save is indistinguishable in a log from a write that never needed
// saving, which makes "did the fix fire?" unanswerable in the field — and made the first
// on-glass verification of this very change inconclusive.
tracing::info!(
desktop = %binding.name,
"display write was refused off the input desktop — retried bound to it and it applied \
(a UAC prompt / lock screen owns input; the session continues normally)"
);
}
second
}
+3
View File
@@ -11,6 +11,9 @@
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub mod display_events; pub mod display_events;
/// Bind display-config writes to the input desktop so a UAC / lock screen can't refuse them.
#[cfg(target_os = "windows")]
mod input_desktop;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub mod monitor_devnode; pub mod monitor_devnode;
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
+354 -78
View File
@@ -23,10 +23,10 @@ use windows::core::PCWSTR;
use windows::Win32::Devices::Display::{ use windows::Win32::Devices::Display::{
DisplayConfigGetDeviceInfo, DisplayConfigSetDeviceInfo, GetDisplayConfigBufferSizes, DisplayConfigGetDeviceInfo, DisplayConfigSetDeviceInfo, GetDisplayConfigBufferSizes,
QueryDisplayConfig, SetDisplayConfig, DISPLAYCONFIG_DEVICE_INFO_GET_ADVANCED_COLOR_INFO, QueryDisplayConfig, SetDisplayConfig, DISPLAYCONFIG_DEVICE_INFO_GET_ADVANCED_COLOR_INFO,
DISPLAYCONFIG_DEVICE_INFO_GET_SOURCE_NAME, DISPLAYCONFIG_DEVICE_INFO_GET_TARGET_NAME, DISPLAYCONFIG_DEVICE_INFO_GET_SDR_WHITE_LEVEL, DISPLAYCONFIG_DEVICE_INFO_GET_SOURCE_NAME,
DISPLAYCONFIG_DEVICE_INFO_SET_ADVANCED_COLOR_STATE, DISPLAYCONFIG_GET_ADVANCED_COLOR_INFO, DISPLAYCONFIG_DEVICE_INFO_GET_TARGET_NAME, DISPLAYCONFIG_DEVICE_INFO_SET_ADVANCED_COLOR_STATE,
DISPLAYCONFIG_MODE_INFO, DISPLAYCONFIG_MODE_INFO_TYPE_SOURCE, DISPLAYCONFIG_GET_ADVANCED_COLOR_INFO, DISPLAYCONFIG_MODE_INFO,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_COMPONENT_VIDEO, DISPLAYCONFIG_MODE_INFO_TYPE_SOURCE, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_COMPONENT_VIDEO,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_COMPOSITE_VIDEO, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_COMPOSITE_VIDEO,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DISPLAYPORT_EMBEDDED, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DISPLAYPORT_EMBEDDED,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DISPLAYPORT_EXTERNAL, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DVI, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DISPLAYPORT_EXTERNAL, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DVI,
@@ -35,16 +35,17 @@ use windows::Win32::Devices::Display::{
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SDI, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SDTVDONGLE, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SDI, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SDTVDONGLE,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SVIDEO, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_UDI_EMBEDDED, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SVIDEO, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_UDI_EMBEDDED,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_UDI_EXTERNAL, DISPLAYCONFIG_PATH_INFO, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_UDI_EXTERNAL, DISPLAYCONFIG_PATH_INFO,
DISPLAYCONFIG_SET_ADVANCED_COLOR_STATE, DISPLAYCONFIG_SOURCE_DEVICE_NAME, DISPLAYCONFIG_SDR_WHITE_LEVEL, DISPLAYCONFIG_SET_ADVANCED_COLOR_STATE,
DISPLAYCONFIG_TARGET_DEVICE_NAME, DISPLAYCONFIG_VIDEO_OUTPUT_TECHNOLOGY, QDC_ALL_PATHS, DISPLAYCONFIG_SOURCE_DEVICE_NAME, DISPLAYCONFIG_TARGET_DEVICE_NAME,
QDC_ONLY_ACTIVE_PATHS, SDC_ALLOW_CHANGES, SDC_APPLY, SDC_FORCE_MODE_ENUMERATION, DISPLAYCONFIG_VIDEO_OUTPUT_TECHNOLOGY, QDC_ALL_PATHS, QDC_ONLY_ACTIVE_PATHS, SDC_ALLOW_CHANGES,
SDC_SAVE_TO_DATABASE, SDC_TOPOLOGY_EXTEND, SDC_USE_SUPPLIED_DISPLAY_CONFIG, SDC_APPLY, SDC_FORCE_MODE_ENUMERATION, SDC_SAVE_TO_DATABASE, SDC_TOPOLOGY_EXTEND,
SDC_USE_SUPPLIED_DISPLAY_CONFIG,
}; };
use windows::Win32::Foundation::POINTL; use windows::Win32::Foundation::POINTL;
use windows::Win32::Graphics::Gdi::{ use windows::Win32::Graphics::Gdi::{
ChangeDisplaySettingsExW, EnumDisplaySettingsW, CDS_TEST, CDS_UPDATEREGISTRY, DEVMODEW, ChangeDisplaySettingsExW, EnumDisplaySettingsW, CDS_TEST, CDS_UPDATEREGISTRY, DEVMODEW,
DISP_CHANGE_SUCCESSFUL, DM_BITSPERPEL, DM_DISPLAYFREQUENCY, DM_PELSHEIGHT, DM_PELSWIDTH, DISP_CHANGE_FAILED, DISP_CHANGE_SUCCESSFUL, DM_BITSPERPEL, DM_DISPLAYFREQUENCY, DM_PELSHEIGHT,
ENUM_CURRENT_SETTINGS, ENUM_DISPLAY_SETTINGS_MODE, DM_PELSWIDTH, ENUM_CURRENT_SETTINGS, ENUM_DISPLAY_SETTINGS_MODE,
}; };
use punktfunk_core::Mode; use punktfunk_core::Mode;
@@ -61,7 +62,9 @@ use punktfunk_core::Mode;
pub unsafe fn force_extend_topology() { pub unsafe fn force_extend_topology() {
// A topology flag with no supplied path/mode arrays tells the OS to recompute + apply that preset // A topology flag with no supplied path/mode arrays tells the OS to recompute + apply that preset
// for the currently-connected displays (the same code path DisplaySwitch.exe drives). // for the currently-connected displays (the same code path DisplaySwitch.exe drives).
let rc = SetDisplayConfig(None, None, SDC_APPLY | SDC_TOPOLOGY_EXTEND); let rc = crate::input_desktop::retry_set_display_config(|| {
SetDisplayConfig(None, None, SDC_APPLY | SDC_TOPOLOGY_EXTEND)
});
if rc == 0 { if rc == 0 {
tracing::info!( tracing::info!(
"display topology forced to EXTEND (a new IddCx monitor would otherwise be CLONED onto the \ "display topology forced to EXTEND (a new IddCx monitor would otherwise be CLONED onto the \
@@ -151,11 +154,13 @@ pub unsafe fn activate_target_path(target_id: u32) -> bool {
// SAVE_TO_DATABASE so Windows remembers the arrangement — the next same-identity ADD (the driver // SAVE_TO_DATABASE so Windows remembers the arrangement — the next same-identity ADD (the driver
// reuses the slot's EDID serial/ConnectorIndex) then auto-activates from the persistence DB and // reuses the slot's EDID serial/ConnectorIndex) then auto-activates from the persistence DB and
// skips this whole fallback ladder. // skips this whole fallback ladder.
let rc = SetDisplayConfig( let rc = crate::input_desktop::retry_set_display_config(|| {
Some(supplied.as_slice()), SetDisplayConfig(
Some(modes.as_slice()), Some(supplied.as_slice()),
SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES | SDC_SAVE_TO_DATABASE, Some(modes.as_slice()),
); SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES | SDC_SAVE_TO_DATABASE,
)
});
if rc == 0 { if rc == 0 {
tracing::info!( tracing::info!(
target_id, target_id,
@@ -272,14 +277,16 @@ pub unsafe fn force_mode_reenumeration() -> bool {
let Some((paths, modes)) = query_active_config() else { let Some((paths, modes)) = query_active_config() else {
return false; return false;
}; };
let rc = SetDisplayConfig( let rc = crate::input_desktop::retry_set_display_config(|| {
Some(paths.as_slice()), SetDisplayConfig(
Some(modes.as_slice()), Some(paths.as_slice()),
SDC_APPLY Some(modes.as_slice()),
| SDC_USE_SUPPLIED_DISPLAY_CONFIG SDC_APPLY
| SDC_ALLOW_CHANGES | SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_FORCE_MODE_ENUMERATION, | SDC_ALLOW_CHANGES
); | SDC_FORCE_MODE_ENUMERATION,
)
});
if rc != 0 { if rc != 0 {
tracing::debug!("force mode re-enumeration: SetDisplayConfig rc={rc:#x}"); tracing::debug!("force mode re-enumeration: SetDisplayConfig rc={rc:#x}");
} }
@@ -481,6 +488,52 @@ pub unsafe fn advanced_color_enabled(target_id: u32) -> Option<bool> {
None None
} }
/// The target's SDR white level as a SCALE relative to 80 nits (`1.0` = 80 nits): where DWM
/// places SDR-white when composing SDR content onto this HDR desktop. An SDR-authored overlay
/// (the composited cursor) must be multiplied by this in scRGB space or it renders visibly
/// darker than the surrounding SDR desktop content (the Windows "SDR content brightness"
/// slider default alone is ~2.5x). `None` = query failed / target not active (callers keep
/// their last value or 1.0).
///
/// # Safety
/// Runs the read-only CCD query FFI over owned locals (same shape as [`advanced_color_enabled`]).
pub unsafe fn sdr_white_level_scale(target_id: u32) -> Option<f32> {
let mut np = 0u32;
let mut nm = 0u32;
if GetDisplayConfigBufferSizes(QDC_ONLY_ACTIVE_PATHS, &mut np, &mut nm).is_err() {
return None;
}
let mut paths = vec![DISPLAYCONFIG_PATH_INFO::default(); np as usize];
let mut modes = vec![DISPLAYCONFIG_MODE_INFO::default(); nm as usize];
if QueryDisplayConfig(
QDC_ONLY_ACTIVE_PATHS,
&mut np,
paths.as_mut_ptr(),
&mut nm,
modes.as_mut_ptr(),
None,
)
.is_err()
{
return None;
}
for p in paths.iter().take(np as usize) {
if p.targetInfo.id == target_id {
let mut info = DISPLAYCONFIG_SDR_WHITE_LEVEL::default();
info.header.r#type = DISPLAYCONFIG_DEVICE_INFO_GET_SDR_WHITE_LEVEL;
info.header.size = size_of::<DISPLAYCONFIG_SDR_WHITE_LEVEL>() as u32;
info.header.adapterId = p.targetInfo.adapterId;
info.header.id = p.targetInfo.id;
if DisplayConfigGetDeviceInfo(&mut info.header) == 0 && info.SDRWhiteLevel > 0 {
// Contract: SDRWhiteLevel/1000 * 80 = nits, i.e. the /1000 IS the 80-nit scale.
return Some(info.SDRWhiteLevel as f32 / 1000.0);
}
return None;
}
}
None
}
/// Force the freshly-added virtual monitor to the client's exact `WxH@Hz`. The ADD IOCTL only /// Force the freshly-added virtual monitor to the client's exact `WxH@Hz`. The ADD IOCTL only
/// ADVERTISES the mode; Windows otherwise activates an IDD target at a 1280x720 default, so the /// ADVERTISES the mode; Windows otherwise activates an IDD target at a 1280x720 default, so the
/// ACTIVE mode (what DXGI Desktop Duplication captures) must be set explicitly. CDS_TEST first so a /// ACTIVE mode (what DXGI Desktop Duplication captures) must be set explicitly. CDS_TEST first so a
@@ -578,9 +631,15 @@ pub fn set_active_mode(gdi_name: &str, mode: Mode) {
// SAFETY: `wname` is a live NUL-terminated UTF-16 device name and `&dm` is a live DEVMODEW describing // SAFETY: `wname` is a live NUL-terminated UTF-16 device name and `&dm` is a live DEVMODEW describing
// the requested mode; both outlive the call. CDS_TEST only validates the mode (no apply), the two // the requested mode; both outlive the call. CDS_TEST only validates the mode (no apply), the two
// trailing args are null, and the API only reads its inputs. // trailing args are null, and the API only reads its inputs.
let test = unsafe { // A CDS write from a thread that is not on the input desktop is refused with DISP_CHANGE_FAILED
ChangeDisplaySettingsExW(PCWSTR(wname.as_ptr()), Some(&dm), None, CDS_TEST, None) // (UAC consent / lock screen up) — retry it bound to that desktop rather than declaring the mode
}; // unsupported, which is what stranded sessions on a black screen for a whole bring-up.
let test = crate::input_desktop::retry_on_input_desktop(
|rc| *rc == DISP_CHANGE_FAILED,
|| unsafe {
ChangeDisplaySettingsExW(PCWSTR(wname.as_ptr()), Some(&dm), None, CDS_TEST, None)
},
);
if test != DISP_CHANGE_SUCCESSFUL { if test != DISP_CHANGE_SUCCESSFUL {
tracing::warn!( tracing::warn!(
result = test.0, result = test.0,
@@ -595,15 +654,20 @@ pub fn set_active_mode(gdi_name: &str, mode: Mode) {
// SAFETY: same inputs as the CDS_TEST call above — `wname` (live NUL-terminated device name) and // SAFETY: same inputs as the CDS_TEST call above — `wname` (live NUL-terminated device name) and
// `&dm` (live DEVMODEW) both outlive the call; CDS_UPDATEREGISTRY applies the already-validated mode, // `&dm` (live DEVMODEW) both outlive the call; CDS_UPDATEREGISTRY applies the already-validated mode,
// and the API only reads its inputs. // and the API only reads its inputs.
let apply = unsafe { // Same wrong-desktop retry as the validate above: the two calls bind independently, so an apply
ChangeDisplaySettingsExW( // still lands even when the secure desktop came up between them.
PCWSTR(wname.as_ptr()), let apply = crate::input_desktop::retry_on_input_desktop(
Some(&dm), |rc| *rc == DISP_CHANGE_FAILED,
None, || unsafe {
CDS_UPDATEREGISTRY, ChangeDisplaySettingsExW(
None, PCWSTR(wname.as_ptr()),
) Some(&dm),
}; None,
CDS_UPDATEREGISTRY,
None,
)
},
);
if apply == DISP_CHANGE_SUCCESSFUL { if apply == DISP_CHANGE_SUCCESSFUL {
tracing::info!( tracing::info!(
"{gdi_name}: active mode set to {}x{}@{}", "{gdi_name}: active mode set to {}x{}@{}",
@@ -625,12 +689,20 @@ pub fn set_active_mode(gdi_name: &str, mode: Mode) {
/// Human decode for a failed `ChangeDisplaySettingsExW` result. The two codes worth telling apart /// Human decode for a failed `ChangeDisplaySettingsExW` result. The two codes worth telling apart
/// in a field log: `BADMODE` (the display's mode list genuinely lacks the mode) vs `FAILED` (the /// in a field log: `BADMODE` (the display's mode list genuinely lacks the mode) vs `FAILED` (the
/// write itself was rejected — on a healthy driver that is the signature of a host process without /// write itself was rejected). An earlier revision printed "mode not advertised?" for BOTH, which
/// console-session access, e.g. one trapped in a disconnected RDP session). An earlier revision /// sent a lid-closed field report chasing the wrong cause.
/// printed "mode not advertised?" for BOTH, which sent a lid-closed field report chasing the wrong ///
/// cause. /// `FAILED` itself has two causes needing opposite fixes — no console-session access (disconnected
/// RDP) versus the right session but the wrong desktop (UAC / lock / logon screen owns input) — so
/// it asks which before naming one. See [`sdc_access_denied_hint`] for the same split on the CCD
/// side.
fn disp_change_reason(rc: i32) -> &'static str { fn disp_change_reason(rc: i32) -> &'static str {
match rc { match rc {
-1 if crate::input_desktop::input_desktop_is_secure() => {
"DISP_CHANGE_FAILED: the SECURE desktop owns input — a UAC consent prompt, the lock \
screen or the logon screen is up, and display writes are refused off it. Dismiss the \
prompt on the host"
}
-1 => { -1 => {
"DISP_CHANGE_FAILED: the display write was rejected — a host without console-session \ "DISP_CHANGE_FAILED: the display write was rejected — a host without console-session \
access (disconnected RDP session / non-console session) fails ALL display writes \ access (disconnected RDP session / non-console session) fails ALL display writes \
@@ -644,15 +716,28 @@ fn disp_change_reason(rc: i32) -> &'static str {
} }
} }
/// Appended to `SetDisplayConfig` failure logs when rc is `ERROR_ACCESS_DENIED` (0x5) — per MS /// Appended to `SetDisplayConfig` failure logs when rc is `ERROR_ACCESS_DENIED` (0x5). Every other
/// docs "the caller does not have access to the console session", the field signature of a host /// rc gets no hint.
/// running in a disconnected RDP / non-console session. Every other rc gets no hint. ///
/// `ERROR_ACCESS_DENIED` has TWO field causes and they need opposite fixes, so ask which one before
/// naming it. MS docs say only "the caller does not have access to the console session", which is
/// the disconnected-RDP / non-console case — but the SAME rc comes back when the host IS in the
/// console session and merely off the input desktop (UAC consent, lock or logon screen up). Naming
/// only the first sent a 2026-07-23 investigation chasing a phantom RDP session while a consent
/// prompt was the actual cause, on a host the message told to "run via the installed service" —
/// which it already was. [`crate::input_desktop`] now retries these bound to the input desktop, so
/// seeing this at all means even that did not help.
fn sdc_access_denied_hint(rc: i32) -> &'static str { fn sdc_access_denied_hint(rc: i32) -> &'static str {
if rc == 5 { if rc != 5 {
return "";
}
if crate::input_desktop::input_desktop_is_secure() {
" (ERROR_ACCESS_DENIED: the SECURE desktop owns input — a UAC consent prompt, the lock \
screen or the logon screen is up, and display writes are refused off it. Dismiss the \
prompt on the host)"
} else {
" (ERROR_ACCESS_DENIED: the host has no console-session access — disconnected RDP \ " (ERROR_ACCESS_DENIED: the host has no console-session access — disconnected RDP \
session? run via the installed service so it tracks the console session)" session? run via the installed service so it tracks the console session)"
} else {
""
} }
} }
@@ -854,30 +939,75 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
// live topology each attempt and re-apply until ONLY the keep set is active. Secure-desktop // live topology each attempt and re-apply until ONLY the keep set is active. Secure-desktop
// correctness depends on this — the lock screen must not land on a stray panel while we stream. // correctness depends on this — the lock screen must not land on a stray panel while we stream.
for attempt in 1..=4u32 { for attempt in 1..=4u32 {
let (mut paths, modes) = query_active_config()?; let (mut paths, mut modes) = query_active_config()?;
let mut others = 0u32; let mut others = 0u32;
for p in paths.iter_mut() { for p in paths.iter_mut() {
if keep_target_ids.contains(&p.targetInfo.id) { if keep_target_ids.contains(&p.targetInfo.id) {
continue; continue;
} }
if p.flags & DISPLAYCONFIG_PATH_ACTIVE != 0 { if p.flags & DISPLAYCONFIG_PATH_ACTIVE != 0 {
p.flags &= !DISPLAYCONFIG_PATH_ACTIVE; // mark this path inactive // Mark the path inactive AND unpin its modes: per the SetDisplayConfig
// contract a path being turned OFF needs BOTH mode indexes marked invalid,
// and leaving them referencing the queried mode entries gets the whole
// supplied config rejected with 0x57 ERROR_INVALID_PARAMETER on some
// driver/topology combinations (field-reported: exclusive mode left the
// physical panel lit, every retry failing 0x57). Writing the all-ones
// sentinel to the whole union is also correct under the virtual-mode-aware
// interpretation (cloneGroupId/sourceModeInfoIdx both become their 0xffff
// INVALID values).
p.flags &= !DISPLAYCONFIG_PATH_ACTIVE;
p.sourceInfo.Anonymous.modeInfoIdx = DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
p.targetInfo.Anonymous.modeInfoIdx = DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
others += 1; others += 1;
} }
} }
// The doomed display may have HELD the desktop origin (the physical is primary in the
// single-slot exclusive case): re-anchor the kept sources so the supplied config still
// contains a primary — an origin-less desktop is rejected 0x57 no matter which array
// shape carries it (see `anchor_kept_sources_at_origin`).
if others > 0 {
anchor_kept_sources_at_origin(&paths, &mut modes);
}
// Commit the config. Even when nothing needed deactivating we re-commit: a legacy mode-set does // Commit the config. Even when nothing needed deactivating we re-commit: a legacy mode-set does
// NOT drive the IddCx adapter's EVT_IDD_CX_ADAPTER_COMMIT_MODES, and without COMMIT_MODES the OS // NOT drive the IddCx adapter's EVT_IDD_CX_ADAPTER_COMMIT_MODES, and without COMMIT_MODES the OS
// never calls ASSIGN_SWAPCHAIN, so the driver receives no frames. SDC_FORCE_MODE_ENUMERATION // never calls ASSIGN_SWAPCHAIN, so the driver receives no frames. SDC_FORCE_MODE_ENUMERATION
// forces the re-commit; SAVE_TO_DATABASE only in the sole-path case (matches prior behavior — // forces the re-commit; SAVE_TO_DATABASE only in the sole-path case (matches prior behavior —
// don't permanently rewrite the user's multi-display layout; the teardown restore handles it). // don't permanently rewrite the user's multi-display layout; the teardown restore handles it).
let mut flags = SDC_APPLY // The supplied shape is decided (and its escalation announced) ONCE, outside the write, so a
| SDC_USE_SUPPLIED_DISPLAY_CONFIG // wrong-desktop retry re-issues the identical config instead of logging the escalation twice.
| SDC_ALLOW_CHANGES let keep_only = (others > 0 && attempt >= 2).then(|| {
| SDC_FORCE_MODE_ENUMERATION; // ESCALATION (attempt 2+): supply ONLY the keep paths. Kept as belt-and-braces —
if others == 0 { // the field 0x57 this was built for turned out to be the missing desktop origin
flags |= SDC_SAVE_TO_DATABASE; // (see `anchor_kept_sources_at_origin`), which rejected BOTH shapes identically;
} // but should some other validator still choke on the full array, the minimal
let rc = SetDisplayConfig(Some(paths.as_slice()), Some(modes.as_slice()), flags); // shape is the best last word. The final attempt also drops
// SDC_FORCE_MODE_ENUMERATION in case the driver rejects it combined with a real
// topology change — an actual path removal drives COMMIT_MODES on its own, so the
// re-commit rationale doesn't need the flag here.
let (kp, km) = keep_only_supplied(&paths, &modes);
let mut esc = SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES;
if attempt < 4 {
esc |= SDC_FORCE_MODE_ENUMERATION;
}
tracing::info!(
"display isolate (CCD): escalating to a keep-only supplied config (attempt {attempt}/4, paths {}→{}, modes {}→{})",
paths.len(), kp.len(), modes.len(), km.len()
);
(kp, km, esc)
});
let rc = crate::input_desktop::retry_set_display_config(|| match &keep_only {
Some((kp, km, esc)) => SetDisplayConfig(Some(kp.as_slice()), Some(km.as_slice()), *esc),
None => {
let mut flags = SDC_APPLY
| SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_ALLOW_CHANGES
| SDC_FORCE_MODE_ENUMERATION;
if others == 0 {
flags |= SDC_SAVE_TO_DATABASE;
}
SetDisplayConfig(Some(paths.as_slice()), Some(modes.as_slice()), flags)
}
});
// A failed apply must be VISIBLE even when the verification below passes vacuously (nothing // A failed apply must be VISIBLE even when the verification below passes vacuously (nothing
// else was active to deactivate — the lid-closed laptop case, where the success INFO used to // else was active to deactivate — the lid-closed laptop case, where the success INFO used to
// swallow rc=0x5): the re-commit above is load-bearing (COMMIT_MODES → ASSIGN_SWAPCHAIN), // swallow rc=0x5): the re-commit above is load-bearing (COMMIT_MODES → ASSIGN_SWAPCHAIN),
@@ -900,10 +1030,132 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
tracing::warn!("display isolate (CCD): {survivors} display(s) STILL active after attempt {attempt}/4 (deactivated {others}, rc={rc:#x}) — re-querying + retrying"); tracing::warn!("display isolate (CCD): {survivors} display(s) STILL active after attempt {attempt}/4 (deactivated {others}, rc={rc:#x}) — re-querying + retrying");
std::thread::sleep(std::time::Duration::from_millis(250)); std::thread::sleep(std::time::Duration::from_millis(250));
} }
tracing::error!("display isolate (CCD): failed to isolate target set {keep_target_ids:?} after 4 attempts — a non-virtual display stayed active (field-reported exclusive-mode bug)"); // Name the survivors instead of assuming their kind — the field logs showed this path fire
// with a sibling VIRTUAL display as the survivor (linger-expiry shrink), where the old
// "a non-virtual display stayed active" wording sent the triage the wrong way.
let survivors: Vec<String> = target_inventory()
.iter()
.filter(|t| t.active && !keep_target_ids.contains(&t.target_id))
.map(|t| format!("{} {} \"{}\"", t.target_id, t.tech, t.friendly))
.collect();
tracing::error!(
"display isolate (CCD): failed to isolate target set {keep_target_ids:?} after 4 attempts — still active: [{}] (field-reported exclusive-mode bug)",
survivors.join(", ")
);
Some(saved) Some(saved)
} }
/// Build the ESCALATED supplied config for [`isolate_displays_ccd`]: ONLY the paths still flagged
/// ACTIVE (the keep set — the caller already cleared ACTIVE on every doomed path), with the mode
/// table rebuilt to just the entries those paths reference (indexes remapped). Docs-wise the
/// dropped inactive entries were declared ignored anyway ("Only the paths within this array that
/// have the DISPLAYCONFIG_PATH_ACTIVE flag set are set"), so this shape asks for the identical
/// topology — minus the array contents some driver/OS validation combos reject with 0x57.
unsafe fn keep_only_supplied(
paths: &[DISPLAYCONFIG_PATH_INFO],
modes: &[DISPLAYCONFIG_MODE_INFO],
) -> (Vec<DISPLAYCONFIG_PATH_INFO>, Vec<DISPLAYCONFIG_MODE_INFO>) {
let mut out_paths = Vec::new();
let mut out_modes = Vec::new();
// old mode index → new. Shared entries dedup through here: a clone-style pair references ONE
// source mode, and the docs require each source/target mode to appear in the table only once.
let mut remap = std::collections::HashMap::new();
for p in paths {
if p.flags & DISPLAYCONFIG_PATH_ACTIVE == 0 {
continue;
}
let mut q = *p;
q.sourceInfo.Anonymous.modeInfoIdx = remap_mode_idx(
q.sourceInfo.Anonymous.modeInfoIdx,
modes,
&mut out_modes,
&mut remap,
);
q.targetInfo.Anonymous.modeInfoIdx = remap_mode_idx(
q.targetInfo.Anonymous.modeInfoIdx,
modes,
&mut out_modes,
&mut remap,
);
out_paths.push(q);
}
(out_paths, out_modes)
}
/// Move `modes[old]` into `out` (once — `remap` dedups) and return its new index. INVALID and
/// out-of-range indexes stay INVALID — `SDC_ALLOW_CHANGES` lets best-mode logic fill the gap.
fn remap_mode_idx(
old: u32,
modes: &[DISPLAYCONFIG_MODE_INFO],
out: &mut Vec<DISPLAYCONFIG_MODE_INFO>,
remap: &mut std::collections::HashMap<u32, u32>,
) -> u32 {
if old == DISPLAYCONFIG_PATH_MODE_IDX_INVALID {
return old;
}
let Some(m) = modes.get(old as usize) else {
return DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
};
*remap.entry(old).or_insert_with(|| {
out.push(*m);
(out.len() - 1) as u32
})
}
/// A committable desktop must still contain a PRIMARY — a source pinned exactly at the origin
/// `(0,0)`. Deactivating the display that held the origin (the physical, in the exclusive
/// topology) while the kept virtual stays pinned at its EXTEND offset supplies an origin-less
/// desktop, and Windows rejects that wholesale with 0x57 ERROR_INVALID_PARAMETER no matter the
/// array shape — the field box failed identically with the doomed path carried inactive AND with
/// the keep-only escalation, yet the very same call converged rc=0 whenever a kept member already
/// sat at `(0,0)`; the origin was the real variable all along. Translate the kept sources RIGIDLY
/// (relative arrangement preserved) so the top-left-most lands exactly on the origin. A set that
/// already covers `(0,0)` is left untouched, so a plain re-commit stays byte-identical and a
/// user's negative-coordinate multi-monitor layout is never rearranged.
unsafe fn anchor_kept_sources_at_origin(
paths: &[DISPLAYCONFIG_PATH_INFO],
modes: &mut [DISPLAYCONFIG_MODE_INFO],
) {
// Unique source-mode entries of the still-ACTIVE (kept) paths — clone-style pairs share one,
// and a shared entry must be translated once.
let mut idxs: Vec<usize> = Vec::new();
for p in paths {
if p.flags & DISPLAYCONFIG_PATH_ACTIVE == 0 {
continue;
}
let idx = p.sourceInfo.Anonymous.modeInfoIdx as usize;
let Some(m) = modes.get(idx) else { continue };
if m.infoType == DISPLAYCONFIG_MODE_INFO_TYPE_SOURCE && !idxs.contains(&idx) {
idxs.push(idx);
}
}
let positions: Vec<(i32, i32)> = idxs
.iter()
.map(|&i| {
let pos = modes[i].Anonymous.sourceMode.position;
(pos.x, pos.y)
})
.collect();
if positions.contains(&(0, 0)) {
return; // the kept set already holds the primary — don't touch a valid layout
}
// Lexicographic min over the actual positions — the anchor IS one of the kept sources, so
// after translation one source sits exactly at (0,0), which is what the validator wants.
let Some((ax, ay)) = positions.iter().copied().min() else {
return; // no pinned kept sources — placement is the OS's (SDC_ALLOW_CHANGES), nothing to anchor
};
for &i in &idxs {
let sm = &mut modes[i].Anonymous.sourceMode;
sm.position.x -= ax;
sm.position.y -= ay;
}
tracing::info!(
"display isolate (CCD): kept source(s) re-anchored onto the desktop origin (primary) — the doomed display held (0,0) delta=({},{})",
-ax,
-ay
);
}
/// The desktop-space rectangle `(x, y, w, h)` of `target_id`'s SOURCE — where this display's /// The desktop-space rectangle `(x, y, w, h)` of `target_id`'s SOURCE — where this display's
/// region lives in the desktop coordinate space. `None` while the target isn't an active path. /// region lives in the desktop coordinate space. `None` while the target isn't an active path.
/// Used by the IDD-push compose kick to dirty THE TARGET display: with parallel displays the /// Used by the IDD-push compose kick to dirty THE TARGET display: with parallel displays the
@@ -998,14 +1250,16 @@ pub unsafe fn apply_source_positions(positions: &[(u32, i32, i32)]) {
if moved == 0 { if moved == 0 {
return; return;
} }
let rc = SetDisplayConfig( let rc = crate::input_desktop::retry_set_display_config(|| {
Some(paths.as_slice()), SetDisplayConfig(
Some(modes.as_slice()), Some(paths.as_slice()),
SDC_APPLY Some(modes.as_slice()),
| SDC_USE_SUPPLIED_DISPLAY_CONFIG SDC_APPLY
| SDC_ALLOW_CHANGES | SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_FORCE_MODE_ENUMERATION, | SDC_ALLOW_CHANGES
); | SDC_FORCE_MODE_ENUMERATION,
)
});
if rc == 0 { if rc == 0 {
tracing::info!( tracing::info!(
?positions, ?positions,
@@ -1090,14 +1344,16 @@ pub unsafe fn set_virtual_primary_ccd(keep_target_id: u32) -> Option<SavedConfig
} }
} }
let rc = SetDisplayConfig( let rc = crate::input_desktop::retry_set_display_config(|| {
Some(paths.as_slice()), SetDisplayConfig(
Some(modes.as_slice()), Some(paths.as_slice()),
SDC_APPLY Some(modes.as_slice()),
| SDC_USE_SUPPLIED_DISPLAY_CONFIG SDC_APPLY
| SDC_ALLOW_CHANGES | SDC_USE_SUPPLIED_DISPLAY_CONFIG
| SDC_FORCE_MODE_ENUMERATION, | SDC_ALLOW_CHANGES
); | SDC_FORCE_MODE_ENUMERATION,
)
});
if rc == 0 { if rc == 0 {
tracing::info!("display primary (CCD): virtual target {keep_target_id} set PRIMARY at (0,0); {others} other display(s) kept ACTIVE + packed to its right"); tracing::info!("display primary (CCD): virtual target {keep_target_id} set PRIMARY at (0,0); {others} other display(s) kept ACTIVE + packed to its right");
} else { } else {
@@ -1117,11 +1373,13 @@ pub unsafe fn restore_displays_ccd(saved: &SavedConfig) {
if paths.is_empty() { if paths.is_empty() {
return; return;
} }
let rc = SetDisplayConfig( let rc = crate::input_desktop::retry_set_display_config(|| {
Some(paths.as_slice()), SetDisplayConfig(
Some(modes.as_slice()), Some(paths.as_slice()),
SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES, Some(modes.as_slice()),
); SDC_APPLY | SDC_USE_SUPPLIED_DISPLAY_CONFIG | SDC_ALLOW_CHANGES,
)
});
if rc == 0 { if rc == 0 {
tracing::info!("display isolate (CCD): restored original topology"); tracing::info!("display isolate (CCD): restored original topology");
} else { } else {
@@ -1130,4 +1388,22 @@ pub unsafe fn restore_displays_ccd(saved: &SavedConfig) {
sdc_access_denied_hint(rc) sdc_access_denied_hint(rc)
); );
} }
// GUARANTEE the desk is never left all-dark. The saved config can be unappliable (field
// rc=0x64a ERROR_BAD_CONFIGURATION: it pinned a virtual target incarnation that was since
// removed) or even apply rc=0 yet re-light nothing (snapshotted while an earlier failed
// teardown had the physicals off — the poisoned-snapshot chain from the field logs). Either
// way, if no external physical panel is active after the apply while at least one is
// connected, fall back to the OS database EXTEND preset, which re-activates every connected
// display. Internal panels deliberately don't count as lit-able here — a closed clamshell
// lid must not be forced back on.
let (connected, lit) = target_inventory()
.iter()
.filter(|t| t.external_physical)
.fold((0u32, 0u32), |(c, a), t| (c + 1, a + u32::from(t.active)));
if connected > 0 && lit == 0 {
tracing::warn!(
"display isolate (CCD): no external physical display active after the restore (rc={rc:#x}, connected={connected}) — forcing the EXTEND preset so the desk is not left dark"
);
force_extend_topology();
}
} }
+6 -255
View File
@@ -7,8 +7,12 @@
//! and ffmpeg's `hevc_nvenc` (encode thread) — each thread makes it current before use; //! and ffmpeg's `hevc_nvenc` (encode thread) — each thread makes it current before use;
//! * device memory: pitched allocations, the reusable `BufferPool`/`DeviceBuffer`, IPC //! * device memory: pitched allocations, the reusable `BufferPool`/`DeviceBuffer`, IPC
//! export/import, host readback, and the plane copies; //! export/import, host readback, and the plane copies;
//! * GL / external-memory interop (`RegisteredTexture`, `ExternalDmabuf`); and //! * GL / external-memory interop (`RegisteredTexture`, `ExternalDmabuf`).
//! * the CUDA cursor-blend kernel (`CursorBlend`). //!
//! (The CUDA cursor-blend PTX kernel that used to live here is retired: vendored PTX is JIT'd
//! against the driver's ISA ceiling and silently dies on older drivers. The NVENC cursor blend
//! is now the SPIR-V compute pass in [`super::vkslot`], dispatched over Vulkan-allocated,
//! CUDA-imported input slots.)
//! //!
//! (We use GL interop, not EGL interop: `cuGraphicsEGLRegisterImage` is Tegra-only on the desktop //! (We use GL interop, not EGL interop: `cuGraphicsEGLRegisterImage` is Tegra-only on the desktop
//! driver — see [`super::egl`].) //! driver — see [`super::egl`].)
@@ -18,7 +22,6 @@
#![deny(clippy::undocumented_unsafe_blocks)] #![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{bail, Result}; use anyhow::{bail, Result};
use std::ffi::CStr;
use std::os::raw::{c_uint, c_void}; use std::os::raw::{c_uint, c_void};
use std::sync::{Arc, Mutex, OnceLock}; use std::sync::{Arc, Mutex, OnceLock};
@@ -280,258 +283,6 @@ pub fn copy_stream_handle() -> *mut c_void {
/// Max cursor-overlay bitmap edge (px) uploaded to the device blend buffer — matches the Vulkan path. /// Max cursor-overlay bitmap edge (px) uploaded to the device blend buffer — matches the Vulkan path.
pub const CURSOR_MAX: u32 = 256; pub const CURSOR_MAX: u32 = 256;
/// GPU cursor-overlay compositor for the NVENC path (cursor-as-metadata): loads the `cursor_blend`
/// PTX module once and blends a straight-alpha RGBA cursor into an encoder-OWNED NVENC input surface
/// (ARGB / NV12 / YUV444) with a small kernel launched over the cursor's rectangle — no full-frame
/// pass, and the compositor's dmabuf is never touched. The cursor bitmap lives in a device buffer
/// re-uploaded only when it changes. Requires `context()` to have succeeded (driver present).
pub struct CursorBlend {
module: CUmodule,
f_argb: CUfunction,
f_nv12: CUfunction,
f_yuv444: CUfunction,
cur_buf: CUdeviceptr, // device RGBA staging (CURSOR_MAX²·4, tight rows)
}
// SAFETY: process-lifetime driver handles used only from the encode thread with the shared context
// current — like [`DeviceBuffer`], moving the struct between threads cannot dangle or race.
unsafe impl Send for CursorBlend {}
impl CursorBlend {
/// Load the embedded PTX image and resolve the three blend kernels + a device cursor buffer.
pub fn new(ptx: &[u8]) -> Result<CursorBlend> {
// cuModuleLoadData reads a PTX image as a NUL-terminated string; the embedded .ptx is not,
// so append a terminator.
let mut image = ptx.to_vec();
image.push(0);
let mut module: CUmodule = std::ptr::null_mut();
// SAFETY: `&mut module` is a live out-param the driver fills; `image` is a NUL-terminated PTX
// byte image that outlives the synchronous load. `ck` bails on error before `module` is used.
unsafe {
ck(
cuModuleLoadData(&mut module, image.as_ptr() as *const c_void),
"cuModuleLoadData(cursor_blend)",
)?;
}
let getf = |name: &CStr| -> Result<CUfunction> {
let mut f: CUfunction = std::ptr::null_mut();
// SAFETY: `module` loaded above; each name is a valid NUL-terminated symbol present in
// the module (verified in the .ptx `.entry` list); `&mut f` is a live out-param.
unsafe {
ck(
cuModuleGetFunction(&mut f, module, name.as_ptr()),
"cuModuleGetFunction",
)?;
}
Ok(f)
};
let f_argb = getf(c"blend_argb")?;
let f_nv12 = getf(c"blend_nv12")?;
let f_yuv444 = getf(c"blend_yuv444")?;
let mut cur_buf: CUdeviceptr = 0;
// SAFETY: `&mut cur_buf` is a live out-param; the size fits the CURSOR_MAX² RGBA buffer.
unsafe {
ck(
cuMemAlloc_v2(&mut cur_buf, (CURSOR_MAX * CURSOR_MAX * 4) as usize),
"cuMemAlloc(cursor)",
)?;
}
Ok(CursorBlend {
module,
f_argb,
f_nv12,
f_yuv444,
cur_buf,
})
}
/// Upload the cursor RGBA (`cw*ch*4`, tight rows) into the device blend buffer. Call only when
/// the bitmap changes; position moves are just kernel args.
pub fn upload(&self, rgba: &[u8], cw: u32, ch: u32) -> Result<()> {
let cw = cw.min(CURSOR_MAX);
let ch = ch.min(CURSOR_MAX);
let row = cw as usize * 4;
let copy = CUDA_MEMCPY2D {
srcMemoryType: 1, // HOST
srcHost: rgba.as_ptr() as *const c_void,
srcPitch: row,
dstMemoryType: CU_MEMORYTYPE_DEVICE,
dstDevice: self.cur_buf,
dstPitch: row,
WidthInBytes: row,
Height: ch as usize,
..Default::default()
};
// SAFETY: HOST→DEVICE 2D copy of `row*ch` bytes; `rgba` covers at least that (caller passes
// `cw*ch*4`), `cur_buf` is the CURSOR_MAX²·4 device alloc (row ≤ CURSOR_MAX·4, ch ≤ CURSOR_MAX).
// Synchronous via `copy_blocking`. Requires the context current (caller's contract).
unsafe { copy_blocking(&copy, "cursor HtoD") }
}
/// Blend into a packed 4-byte (NVENC ARGB) owned surface at `(ox,oy)`.
#[allow(clippy::too_many_arguments)] // surface geometry + cursor size + offset — a struct would just be unpacked at the call
pub fn blend_argb(
&self,
surf: CUdeviceptr,
pitch: usize,
w: u32,
h: u32,
cw: u32,
ch: u32,
ox: i32,
oy: i32,
sync: bool,
) -> Result<()> {
let (mut a_surf, mut a_cur) = (surf, self.cur_buf);
let (mut a_pitch, mut a_w, mut a_h) = (pitch as i32, w as i32, h as i32);
let (mut a_cw, mut a_ch) = (cw.min(CURSOR_MAX) as i32, ch.min(CURSOR_MAX) as i32);
let (mut a_ox, mut a_oy) = (ox, oy);
let mut args: [*mut c_void; 9] = [
&mut a_surf as *mut _ as *mut c_void,
&mut a_pitch as *mut _ as *mut c_void,
&mut a_w as *mut _ as *mut c_void,
&mut a_h as *mut _ as *mut c_void,
&mut a_cur as *mut _ as *mut c_void,
&mut a_cw as *mut _ as *mut c_void,
&mut a_ch as *mut _ as *mut c_void,
&mut a_ox as *mut _ as *mut c_void,
&mut a_oy as *mut _ as *mut c_void,
];
self.launch(self.f_argb, a_cw as u32, a_ch as u32, &mut args, sync)
}
/// Blend into an owned planar YUV444 surface (3 stacked full-res planes) at `(ox,oy)`.
#[allow(clippy::too_many_arguments)] // surface geometry + cursor size + offset — a struct would just be unpacked at the call
pub fn blend_yuv444(
&self,
base: CUdeviceptr,
pitch: usize,
w: u32,
h: u32,
cw: u32,
ch: u32,
ox: i32,
oy: i32,
sync: bool,
) -> Result<()> {
let (mut a_base, mut a_cur) = (base, self.cur_buf);
let (mut a_pitch, mut a_w, mut a_h) = (pitch as i32, w as i32, h as i32);
let (mut a_cw, mut a_ch) = (cw.min(CURSOR_MAX) as i32, ch.min(CURSOR_MAX) as i32);
let (mut a_ox, mut a_oy) = (ox, oy);
let mut args: [*mut c_void; 9] = [
&mut a_base as *mut _ as *mut c_void,
&mut a_pitch as *mut _ as *mut c_void,
&mut a_w as *mut _ as *mut c_void,
&mut a_h as *mut _ as *mut c_void,
&mut a_cur as *mut _ as *mut c_void,
&mut a_cw as *mut _ as *mut c_void,
&mut a_ch as *mut _ as *mut c_void,
&mut a_ox as *mut _ as *mut c_void,
&mut a_oy as *mut _ as *mut c_void,
];
self.launch(self.f_yuv444, a_cw as u32, a_ch as u32, &mut args, sync)
}
/// Blend into an owned NV12 surface (Y plane at `base`, interleaved UV at `base + pitch*h`).
#[allow(clippy::too_many_arguments)] // surface geometry + cursor size + offset — a struct would just be unpacked at the call
pub fn blend_nv12(
&self,
base: CUdeviceptr,
pitch: usize,
w: u32,
h: u32,
cw: u32,
ch: u32,
ox: i32,
oy: i32,
sync: bool,
) -> Result<()> {
let (mut a_yb, mut a_uvb, mut a_cur) = (base, base + pitch as u64 * h as u64, self.cur_buf);
let (mut a_yp, mut a_uvp) = (pitch as i32, pitch as i32);
let (mut a_w, mut a_h) = (w as i32, h as i32);
let (mut a_cw, mut a_ch) = (cw.min(CURSOR_MAX) as i32, ch.min(CURSOR_MAX) as i32);
let (mut a_ox, mut a_oy) = (ox, oy);
let mut args: [*mut c_void; 11] = [
&mut a_yb as *mut _ as *mut c_void,
&mut a_yp as *mut _ as *mut c_void,
&mut a_uvb as *mut _ as *mut c_void,
&mut a_uvp as *mut _ as *mut c_void,
&mut a_w as *mut _ as *mut c_void,
&mut a_h as *mut _ as *mut c_void,
&mut a_cur as *mut _ as *mut c_void,
&mut a_cw as *mut _ as *mut c_void,
&mut a_ch as *mut _ as *mut c_void,
&mut a_ox as *mut _ as *mut c_void,
&mut a_oy as *mut _ as *mut c_void,
];
// One thread per 2x2 luma block → grid over ceil(cw/2) × ceil(ch/2).
self.launch(
self.f_nv12,
(a_cw as u32).div_ceil(2),
(a_ch as u32).div_ceil(2),
&mut args,
sync,
)
}
/// Launch `f` over a `work_w × work_h` grid (16×16 blocks) on the copy stream; `sync` waits
/// for it, `!sync` leaves completion to the stream (stream-ordered consumers only — the
/// kernel PARAMETERS are copied at launch time, so the arg locals need not outlive the call).
fn launch(
&self,
f: CUfunction,
work_w: u32,
work_h: u32,
args: &mut [*mut c_void],
sync: bool,
) -> Result<()> {
if work_w == 0 || work_h == 0 {
return Ok(());
}
const B: u32 = 16;
let stream = copy_stream();
// SAFETY: `f` is a resolved kernel from our loaded module; `args` holds pointers to live
// locals whose types match the kernel's C parameters (per the call site above) — CUDA
// copies the parameter values during `cuLaunchKernel` itself, so they need not outlive
// the call. Grid/block dims are non-zero. Launched on the copy stream (ordered after the
// input-surface copy issued on the same stream); `sync` waits, `!sync` leaves ordering to
// the stream (the NVENC IO-stream binding). Requires the context current.
unsafe {
ck(
cuLaunchKernel(
f,
work_w.div_ceil(B),
work_h.div_ceil(B),
1,
B,
B,
1,
0,
stream,
args.as_mut_ptr(),
std::ptr::null_mut(),
),
"cuLaunchKernel(cursor)",
)?;
if sync {
ck(cuStreamSynchronize(stream), "cuStreamSynchronize(cursor)")?;
}
Ok(())
}
}
}
impl Drop for CursorBlend {
fn drop(&mut self) {
// SAFETY: `cur_buf`/`module` are our own handles, freed exactly once here; the context is
// current on the encode thread that drops the encoder. Errors are ignored on teardown.
unsafe {
let _ = cuMemFree_v2(self.cur_buf);
let _ = cuModuleUnload(self.module);
}
}
}
/// Allocate one pitched device buffer for `width`x`height` 4-byte pixels; returns `(ptr, pitch)`. /// Allocate one pitched device buffer for `width`x`height` 4-byte pixels; returns `(ptr, pitch)`.
fn alloc_pitched(width: u32, height: u32) -> Result<(CUdeviceptr, usize)> { fn alloc_pitched(width: u32, height: u32) -> Result<(CUdeviceptr, usize)> {
let mut ptr: CUdeviceptr = 0; let mut ptr: CUdeviceptr = 0;
+1 -75
View File
@@ -9,7 +9,7 @@
#![deny(clippy::undocumented_unsafe_blocks)] #![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{bail, Result}; use anyhow::{bail, Result};
use std::os::raw::{c_char, c_int, c_uint, c_void}; use std::os::raw::{c_int, c_uint, c_void};
use std::sync::OnceLock; use std::sync::OnceLock;
pub type CUresult = c_uint; // CUDA_SUCCESS == 0 pub type CUresult = c_uint; // CUDA_SUCCESS == 0
@@ -20,8 +20,6 @@ pub type CUdeviceptr = u64;
pub type CUgraphicsResource = *mut c_void; pub type CUgraphicsResource = *mut c_void;
pub type CUarray = *mut c_void; pub type CUarray = *mut c_void;
pub type CUexternalMemory = *mut c_void; // opaque CUextMemory_st* pub type CUexternalMemory = *mut c_void; // opaque CUextMemory_st*
pub type CUmodule = *mut c_void; // opaque CUmod_st*
pub type CUfunction = *mut c_void; // opaque CUfunc_st*
/// `CUmemorytype` (cuda.h): HOST=1, DEVICE=2, ARRAY=3, UNIFIED=4. /// `CUmemorytype` (cuda.h): HOST=1, DEVICE=2, ARRAY=3, UNIFIED=4.
pub const CU_MEMORYTYPE_DEVICE: c_uint = 2; pub const CU_MEMORYTYPE_DEVICE: c_uint = 2;
@@ -144,26 +142,6 @@ pub(crate) struct CudaApi {
cuIpcGetMemHandle: unsafe extern "C" fn(*mut CUipcMemHandle, CUdeviceptr) -> CUresult, cuIpcGetMemHandle: unsafe extern "C" fn(*mut CUipcMemHandle, CUdeviceptr) -> CUresult,
cuIpcOpenMemHandle: unsafe extern "C" fn(*mut CUdeviceptr, CUipcMemHandle, c_uint) -> CUresult, cuIpcOpenMemHandle: unsafe extern "C" fn(*mut CUdeviceptr, CUipcMemHandle, c_uint) -> CUresult,
cuIpcCloseMemHandle: unsafe extern "C" fn(CUdeviceptr) -> CUresult, cuIpcCloseMemHandle: unsafe extern "C" fn(CUdeviceptr) -> CUresult,
// Cursor-overlay blend: a linear device alloc + a PTX module with the blend kernels launched
// over the cursor's small rectangle (see [`CursorBlend`]).
cuMemAlloc_v2: unsafe extern "C" fn(*mut CUdeviceptr, usize) -> CUresult,
cuModuleLoadData: unsafe extern "C" fn(*mut CUmodule, *const c_void) -> CUresult,
cuModuleUnload: unsafe extern "C" fn(CUmodule) -> CUresult,
cuModuleGetFunction: unsafe extern "C" fn(*mut CUfunction, CUmodule, *const c_char) -> CUresult,
#[allow(clippy::type_complexity)]
cuLaunchKernel: unsafe extern "C" fn(
CUfunction,
c_uint,
c_uint,
c_uint,
c_uint,
c_uint,
c_uint,
c_uint,
CUstream,
*mut *mut c_void,
*mut *mut c_void,
) -> CUresult,
} }
// SAFETY: every field is a bare `extern "C" fn` address into the leaked, process-lifetime // SAFETY: every field is a bare `extern "C" fn` address into the leaked, process-lifetime
// `libcuda` mapping (`cuda_api` `forget`s the `Library`, so it is never unloaded) — an immutable // `libcuda` mapping (`cuda_api` `forget`s the `Library`, so it is never unloaded) — an immutable
@@ -236,11 +214,6 @@ pub(crate) fn cuda_api() -> Option<&'static CudaApi> {
.or_else(|_| lib.get(b"cuIpcOpenMemHandle\0")) .or_else(|_| lib.get(b"cuIpcOpenMemHandle\0"))
.ok()?, .ok()?,
cuIpcCloseMemHandle: *lib.get(b"cuIpcCloseMemHandle\0").ok()?, cuIpcCloseMemHandle: *lib.get(b"cuIpcCloseMemHandle\0").ok()?,
cuMemAlloc_v2: *lib.get(b"cuMemAlloc_v2\0").ok()?,
cuModuleLoadData: *lib.get(b"cuModuleLoadData\0").ok()?,
cuModuleUnload: *lib.get(b"cuModuleUnload\0").ok()?,
cuModuleGetFunction: *lib.get(b"cuModuleGetFunction\0").ok()?,
cuLaunchKernel: *lib.get(b"cuLaunchKernel\0").ok()?,
}; };
std::mem::forget(lib); // keep libcuda mapped for the fn pointers' lifetime (process) std::mem::forget(lib); // keep libcuda mapped for the fn pointers' lifetime (process)
Some(api) Some(api)
@@ -304,53 +277,6 @@ pub(crate) unsafe fn cuMemFree_v2(dptr: CUdeviceptr) -> CUresult {
None => CU_ERROR_NOT_LOADED, None => CU_ERROR_NOT_LOADED,
} }
} }
pub(crate) unsafe fn cuMemAlloc_v2(dptr: *mut CUdeviceptr, size: usize) -> CUresult {
match cuda_api() {
Some(a) => (a.cuMemAlloc_v2)(dptr, size),
None => CU_ERROR_NOT_LOADED,
}
}
pub(crate) unsafe fn cuModuleLoadData(m: *mut CUmodule, image: *const c_void) -> CUresult {
match cuda_api() {
Some(a) => (a.cuModuleLoadData)(m, image),
None => CU_ERROR_NOT_LOADED,
}
}
pub(crate) unsafe fn cuModuleUnload(m: CUmodule) -> CUresult {
match cuda_api() {
Some(a) => (a.cuModuleUnload)(m),
None => CU_ERROR_NOT_LOADED,
}
}
pub(crate) unsafe fn cuModuleGetFunction(
f: *mut CUfunction,
m: CUmodule,
name: *const c_char,
) -> CUresult {
match cuda_api() {
Some(a) => (a.cuModuleGetFunction)(f, m, name),
None => CU_ERROR_NOT_LOADED,
}
}
#[allow(clippy::too_many_arguments)]
pub(crate) unsafe fn cuLaunchKernel(
f: CUfunction,
gx: c_uint,
gy: c_uint,
gz: c_uint,
bx: c_uint,
by: c_uint,
bz: c_uint,
shmem: c_uint,
stream: CUstream,
params: *mut *mut c_void,
extra: *mut *mut c_void,
) -> CUresult {
match cuda_api() {
Some(a) => (a.cuLaunchKernel)(f, gx, gy, gz, bx, by, bz, shmem, stream, params, extra),
None => CU_ERROR_NOT_LOADED,
}
}
pub(crate) unsafe fn cuMemcpy2DAsync_v2(copy: *const CUDA_MEMCPY2D, stream: CUstream) -> CUresult { pub(crate) unsafe fn cuMemcpy2DAsync_v2(copy: *const CUDA_MEMCPY2D, stream: CUstream) -> CUresult {
match cuda_api() { match cuda_api() {
Some(a) => (a.cuMemcpy2DAsync_v2)(copy, stream), Some(a) => (a.cuMemcpy2DAsync_v2)(copy, stream),
@@ -0,0 +1,170 @@
#version 450
// Cursor-overlay blend for the direct-SDK NVENC path (cursor-as-metadata), dispatched over the
// cursor's rectangle only — the Vulkan replacement for the retired cursor_blend.cu PTX kernels
// (PTX is JIT'd against the driver's ISA ceiling, so a vendored blob silently dies on older
// drivers: CUDA errors 222/218 on-glass; SPIR-V has no such coupling). The NVENC input surface is
// Vulkan-allocated, CUDA-imported external memory (see vkslot.rs), so this shader writes the very
// bytes NVENC encodes.
//
// MODE (spec constant): 0 = packed 4-byte ARGB (NVENC byte order B,G,R,A), 1 = NV12 (Y plane +
// interleaved half-res UV at row surfH), 2 = planar YUV444 (3 full-res planes stacked at
// pitch*surfH). BT.709 limited-range coefficients — identical to rgb2nv12_buf.comp and the
// retired .cu, so the cursor colour matches the frame regardless of backend.
//
// The surface SSBO is uint[] (no 8-bit storage dependency — maximum driver reach): every
// invocation exclusively owns the 32-bit words it read-modify-writes. ARGB: one invocation per
// cursor pixel = one word. NV12/YUV444: one invocation per WORD-ALIGNED 4-px luma span (per two
// rows for NV12, whose 2 chroma bytes-pairs land in one exclusive word). Spans are aligned to the
// SURFACE, not the cursor, so neighbouring invocations never share a word even at odd `ox`.
//
// Rebuild: glslc cursor_blend.comp -o cursor_blend.spv (vendored beside this file)
layout(local_size_x = 8, local_size_y = 8, local_size_z = 1) in;
layout(constant_id = 0) const uint MODE = 0;
layout(std430, binding = 0) buffer Surf { uint surf[]; };
layout(std430, binding = 1) readonly buffer Cur { uint cur[]; };
layout(push_constant) uniform Push {
uint pitch; // surface row stride, bytes (4-aligned by construction)
uint surfW; // content width, px
uint surfH; // luma rows (plane stride multiplier)
uint curW; // cursor bitmap width, px
uint curH; // cursor bitmap height, px
int ox; // cursor top-left on the surface, px (may be negative)
int oy;
} pc;
// Cursor texel (straight-alpha RGBA, tight rows) or (0,0,0,0) outside the bitmap.
uvec4 cursor_px(int cx, int cy) {
if (cx < 0 || cy < 0 || cx >= int(pc.curW) || cy >= int(pc.curH)) return uvec4(0);
uint w = cur[uint(cy) * pc.curW + uint(cx)];
return uvec4(w & 0xFFu, (w >> 8) & 0xFFu, (w >> 16) & 0xFFu, (w >> 24) & 0xFFu); // R,G,B,A
}
uint blend8(uint dst, uint src, uint a) {
return (src * a + dst * (255u - a)) / 255u;
}
// BT.709 limited RGB→Y/U/V (matches the retired .cu / rgb2nv12_buf.comp).
uint y_of(uvec4 s) {
return uint(clamp(16.0 + 0.1826 * float(s.r) + 0.6142 * float(s.g) + 0.0620 * float(s.b) + 0.5, 0.0, 255.0));
}
float u_of(uvec4 s) { return 128.0 - 0.1006 * float(s.r) - 0.3386 * float(s.g) + 0.4392 * float(s.b); }
float v_of(uvec4 s) { return 128.0 + 0.4392 * float(s.r) - 0.3989 * float(s.g) - 0.0403 * float(s.b); }
// Read-modify-write one byte lane of a word index.
void rmw_byte(uint word_idx, uint lane, uint val8, uint a) {
uint w = surf[word_idx];
uint shift = lane * 8u;
uint d = (w >> shift) & 0xFFu;
uint b = blend8(d, val8, a);
surf[word_idx] = (w & ~(0xFFu << shift)) | (b << shift);
}
void main() {
if (MODE == 0u) {
// ARGB: one invocation per cursor pixel; each surface pixel is one exclusive word.
int cx = int(gl_GlobalInvocationID.x);
int cy = int(gl_GlobalInvocationID.y);
if (cx >= int(pc.curW) || cy >= int(pc.curH)) return;
int px = pc.ox + cx, py = pc.oy + cy;
if (px < 0 || py < 0 || px >= int(pc.surfW) || py >= int(pc.surfH)) return;
uvec4 s = cursor_px(cx, cy);
if (s.a == 0u) return;
uint idx = (uint(py) * pc.pitch + uint(px) * 4u) / 4u;
uint w = surf[idx];
uint b = blend8(w & 0xFFu, s.b, s.a); // B lane
uint g = blend8((w >> 8) & 0xFFu, s.g, s.a); // G lane
uint r = blend8((w >> 16) & 0xFFu, s.r, s.a); // R lane
surf[idx] = (w & 0xFF000000u) | (r << 16) | (g << 8) | b;
return;
}
// NV12 / YUV444: one invocation per SURFACE-word-aligned 4-px luma span. Span origin:
// x0 = floor(ox/4)*4 + span*4 (surface px), rows walk the cursor rect.
int span = int(gl_GlobalInvocationID.x);
int row = int(gl_GlobalInvocationID.y);
int x0 = (pc.ox >> 2) << 2; // word-aligned start at/left-of ox (ox may be negative)
int px0 = x0 + span * 4;
if (MODE == 2u) {
// YUV444: rows walk cursor rows one at a time.
if (row >= int(pc.curH)) return;
int py = pc.oy + row;
if (py < 0 || py >= int(pc.surfH)) return;
uint plane = pc.pitch * pc.surfH;
for (int i = 0; i < 4; i++) {
int px = px0 + i;
int cx = px - pc.ox;
if (px < 0 || px >= int(pc.surfW)) continue;
uvec4 s = cursor_px(cx, row);
if (s.a == 0u) continue;
uint off = uint(py) * pc.pitch + uint(px);
uint U = uint(clamp(u_of(s) + 0.5, 0.0, 255.0));
uint V = uint(clamp(v_of(s) + 0.5, 0.0, 255.0));
rmw_byte(off / 4u, off % 4u, y_of(s), s.a);
rmw_byte((plane + off) / 4u, (plane + off) % 4u, U, s.a);
rmw_byte((2u * plane + off) / 4u, (2u * plane + off) % 4u, V, s.a);
}
return;
}
// NV12: rows walk 2-row luma blocks (row = block row). The span's 4 luma px × 2 rows are
// exclusive words; its 2 chroma samples (4 bytes) are one exclusive word.
int base_cy = row * 2;
if (base_cy >= int(pc.curH)) return;
// Luma: 4 px × 2 rows.
for (int j = 0; j < 2; j++) {
int cy = base_cy + j;
int py = pc.oy + cy;
if (cy >= int(pc.curH) || py < 0 || py >= int(pc.surfH)) continue;
for (int i = 0; i < 4; i++) {
int px = px0 + i;
int cx = px - pc.ox;
if (px < 0 || px >= int(pc.surfW)) continue;
uvec4 s = cursor_px(cx, cy);
if (s.a == 0u) continue;
uint off = uint(py) * pc.pitch + uint(px);
rmw_byte(off / 4u, off % 4u, y_of(s), s.a);
}
}
// Chroma: two UV samples covering the span's 2x2 blocks, alpha-weighted like the .cu kernel.
// The UV plane starts at row surfH; sample (uvx, uvy) lives at uv_base + uvy*pitch + uvx*2.
// Guard: only spans whose px0 is 4-aligned own their chroma word (px0 is by construction).
int py_top = pc.oy + base_cy;
int uvy = py_top >> 1;
if (py_top < 0 || uvy < 0 || uvy * 2 >= int(pc.surfH)) return;
uint uv_base = pc.pitch * pc.surfH;
for (int hf = 0; hf < 2; hf++) {
// Each hf = one 2x2 luma block = one UV sample (2 bytes).
int bx = px0 + hf * 2;
if (bx < 0 || bx >= int(pc.surfW)) continue;
int uvx = bx >> 1;
float ua = 0.0, va = 0.0, wa = 0.0;
int cnt = 0;
for (int j = 0; j < 2; j++) {
for (int i = 0; i < 2; i++) {
int px = bx + i;
int py = py_top + j;
int cx = px - pc.ox;
int cy = base_cy + j;
if (px < 0 || py < 0 || px >= int(pc.surfW) || py >= int(pc.surfH)) continue;
uvec4 s = cursor_px(cx, cy);
if (s.a == 0u) continue;
ua += u_of(s) * float(s.a);
va += v_of(s) * float(s.a);
wa += float(s.a);
cnt++;
}
}
if (wa <= 0.0 || cnt == 0) continue;
uint U = uint(clamp(ua / wa + 0.5, 0.0, 255.0));
uint V = uint(clamp(va / wa + 0.5, 0.0, 255.0));
uint amean = uint(clamp(wa / float(cnt) + 0.5, 0.0, 255.0));
uint off = uv_base + uint(uvy) * pc.pitch + uint(uvx) * 2u;
rmw_byte(off / 4u, off % 4u, U, amean);
rmw_byte((off + 1u) / 4u, (off + 1u) % 4u, V, amean);
}
}
Binary file not shown.
+1
View File
@@ -15,6 +15,7 @@ pub mod client;
pub mod cuda; pub mod cuda;
pub mod egl; pub mod egl;
pub mod proto; pub mod proto;
pub mod vkslot;
pub mod vulkan; pub mod vulkan;
pub mod worker; pub mod worker;
+710
View File
@@ -0,0 +1,710 @@
//! Vulkan-allocated NVENC input slots + the Vulkan compute cursor blend — the driver-portable
//! replacement for the retired `cursor_blend.cu` PTX kernels (design: remote-desktop-sweep §8,
//! Phase A). A vendored PTX blob is JIT'd against the driver's ISA ceiling, so it silently dies
//! on drivers older than the generating toolkit (CUDA errors 222/218 on-glass — the KWin leg's
//! invisible composite cursor); SPIR-V has no such coupling.
//!
//! ```text
//! exportable VkBuffer ──vkGetMemoryFdKHR(OPAQUE_FD)──▶ cuImportExternalMemory ──▶ CUdeviceptr
//! ▲ │ NVENC registers + encodes
//! └── cursor_blend.comp dispatch (cursor rect only) ◀───┘ CUDA copies frames in
//! ```
//!
//! The direct-SDK NVENC encoder allocates its input ring through [`VkSlotBlend::alloc_slot`]
//! instead of `cuMemAllocPitch`: same contiguous layouts (`InputSurface` docs), but the memory is
//! Vulkan external memory both APIs address. Per cursor-bearing frame the encoder CPU-syncs its
//! CUDA copy, then [`VkSlotBlend::blend`] dispatches the compute blend over the cursor's
//! rectangle and fence-waits — the same coherence ceremony [`super::vulkan::VkBridge`] ships for
//! its CSC (fence-ordered cross-API access on NVIDIA, no queue-family transfer needed). Frames
//! without a cursor never touch Vulkan, keeping the stream-ordered fast path intact.
//!
//! Falls back cleanly: if bring-up fails the encoder allocates plain CUDA surfaces and composite
//! mode degrades to no cursor (warned once) — never a failed session.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::cuda::{self, CUdeviceptr};
use anyhow::{anyhow, Context as _, Result};
use ash::vk;
/// Max cursor-overlay bitmap edge (px) — matches [`cuda::CURSOR_MAX`] and the capture-side clamp.
pub const CURSOR_MAX: u32 = cuda::CURSOR_MAX;
/// The vendored SPIR-V for `cursor_blend.comp` (beside this file; rebuild with
/// `glslc cursor_blend.comp -o cursor_blend.spv`).
const CURSOR_SPV: &[u8] = include_bytes!("cursor_blend.spv");
/// NVENC input-surface layout — selects the spec-constant `MODE` pipeline and the allocation
/// arithmetic (mirroring `InputSurface`'s contiguous layouts).
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum SlotFormat {
/// Packed 4-byte ARGB (NVENC byte order B,G,R,A): `pitch × height`.
Argb,
/// NV12: Y rows `[0, H)` + interleaved UV rows `[H, 3H/2)` under one pitch.
Nv12,
/// Planar YUV444: three full-res planes stacked at `pitch × height` intervals.
Yuv444,
}
impl SlotFormat {
fn mode(self) -> u32 {
match self {
SlotFormat::Argb => 0,
SlotFormat::Nv12 => 1,
SlotFormat::Yuv444 => 2,
}
}
fn row_bytes(self, width: u32) -> u64 {
match self {
SlotFormat::Argb => width as u64 * 4,
SlotFormat::Nv12 | SlotFormat::Yuv444 => width as u64,
}
}
fn rows(self, height: u32) -> u64 {
match self {
SlotFormat::Argb => height as u64,
SlotFormat::Nv12 => height as u64 + (height as u64 / 2).max(1),
SlotFormat::Yuv444 => height as u64 * 3,
}
}
}
/// What the encoder holds per ring slot: the CUDA view it registers with NVENC plus the id it
/// hands back to [`VkSlotBlend::blend`]. The backing Vulkan objects + CUDA mapping live in the
/// [`VkSlotBlend`] (freed by [`free_slots`](VkSlotBlend::free_slots) / drop), so this is Copy —
/// the encoder's ring keeps its existing shape.
#[derive(Clone, Copy)]
pub struct VkSlotRef {
/// Device pointer NVENC registers (CUDA's mapping of the Vulkan memory).
pub ptr: CUdeviceptr,
/// Row stride in bytes (ours: row bytes rounded up to 256).
pub pitch: usize,
/// Luma rows (the plane-stride multiplier, as in `InputSurface`).
pub height: u32,
/// Index into the blend's slot table.
pub id: usize,
}
/// One allocated slot's backing objects, freed together in reverse order (CUDA mapping first).
struct SlotAlloc {
buffer: vk::Buffer,
memory: vk::DeviceMemory,
/// CUDA's import of the exported OPAQUE_FD — must drop BEFORE the Vulkan memory is freed.
cuda: cuda::ExternalDmabuf,
size: u64,
}
/// 28-byte push-constant block matching `cursor_blend.comp`'s `Push`.
#[repr(C)]
struct Push {
pitch: u32,
surf_w: u32,
surf_h: u32,
cur_w: u32,
cur_h: u32,
ox: i32,
oy: i32,
}
pub struct VkSlotBlend {
_entry: ash::Entry,
instance: ash::Instance,
device: ash::Device,
ext_fd: ash::khr::external_memory_fd::Device,
queue: vk::Queue,
cmd_pool: vk::CommandPool,
cmd: vk::CommandBuffer,
fence: vk::Fence,
mem_props: vk::PhysicalDeviceMemoryProperties,
shader: vk::ShaderModule,
desc_layout: vk::DescriptorSetLayout,
pipe_layout: vk::PipelineLayout,
desc_pool: vk::DescriptorPool,
desc_set: vk::DescriptorSet,
/// One pipeline per [`SlotFormat`], indexed by `mode()` (spec constant).
pipelines: [vk::Pipeline; 3],
/// Host-visible cursor bitmap staging (CURSOR_MAX²·4, tight rows), persistently mapped.
cur_buf: vk::Buffer,
cur_mem: vk::DeviceMemory,
cur_map: *mut u8,
slots: Vec<SlotAlloc>,
}
// SAFETY: raw Vulkan handles + a persistently-mapped pointer, all uniquely owned by this struct
// and destroyed exactly once in `Drop`; used from the encoder thread but moved with it. `Send`
// only (not `Sync`), matching the single-thread use — transferring opaque handles cannot dangle.
unsafe impl Send for VkSlotBlend {}
impl VkSlotBlend {
/// Bring up the device + blend pipelines. Requires the CUDA shared context (the encoder's) to
/// be established; picks the NVIDIA physical device (the NVENC path is NVIDIA by definition).
pub fn new() -> Result<VkSlotBlend> {
// SAFETY: standard ash bring-up, same shape as `VkBridge::new` — every call is `unsafe`
// only because ash cannot statically verify handle/CreateInfo validity. Every
// `*CreateInfo`/`AllocateInfo` is built from locals that live for the duration of the
// synchronous call reading them; every handle passed was created and `?`-checked in this
// same function. Shares nothing across threads.
unsafe {
let entry = ash::Entry::load().context("load libvulkan")?;
let app = vk::ApplicationInfo::default().api_version(vk::API_VERSION_1_1);
let instance = entry
.create_instance(
&vk::InstanceCreateInfo::default().application_info(&app),
None,
)
.context("vkCreateInstance")?;
let phys = match instance
.enumerate_physical_devices()
.context("enumerate GPUs")?
.into_iter()
.find(|&p| instance.get_physical_device_properties(p).vendor_id == 0x10DE)
{
Some(p) => p,
None => {
instance.destroy_instance(None);
return Err(anyhow!("no NVIDIA Vulkan device"));
}
};
let mem_props = instance.get_physical_device_memory_properties(phys);
let qf = match instance
.get_physical_device_queue_family_properties(phys)
.iter()
.position(|q| q.queue_flags.contains(vk::QueueFlags::COMPUTE))
{
Some(i) => i as u32,
None => {
instance.destroy_instance(None);
return Err(anyhow!("no compute-capable queue family"));
}
};
let prio = [1.0f32];
let qci = [vk::DeviceQueueCreateInfo::default()
.queue_family_index(qf)
.queue_priorities(&prio)];
let exts = [ash::khr::external_memory_fd::NAME.as_ptr()];
let device = match instance.create_device(
phys,
&vk::DeviceCreateInfo::default()
.queue_create_infos(&qci)
.enabled_extension_names(&exts),
None,
) {
Ok(d) => d,
Err(e) => {
instance.destroy_instance(None);
return Err(e).context("vkCreateDevice (external_memory_fd supported?)");
}
};
// From here teardown-on-error goes through `destroy_partial`, which tolerates null
// handles — build everything into an incrementally-filled struct.
let ext_fd = ash::khr::external_memory_fd::Device::new(&instance, &device);
let queue = device.get_device_queue(qf, 0);
let mut me = VkSlotBlend {
_entry: entry,
instance,
device,
ext_fd,
queue,
cmd_pool: vk::CommandPool::null(),
cmd: vk::CommandBuffer::null(),
fence: vk::Fence::null(),
mem_props,
shader: vk::ShaderModule::null(),
desc_layout: vk::DescriptorSetLayout::null(),
pipe_layout: vk::PipelineLayout::null(),
desc_pool: vk::DescriptorPool::null(),
desc_set: vk::DescriptorSet::null(),
pipelines: [vk::Pipeline::null(); 3],
cur_buf: vk::Buffer::null(),
cur_mem: vk::DeviceMemory::null(),
cur_map: std::ptr::null_mut(),
slots: Vec::new(),
};
me.init_objects(qf).inspect_err(|_| {
// `Drop` runs the same teardown and tolerates the nulls left by a partial init.
})?;
tracing::info!(
"Vulkan slot blend ready (exportable NVENC inputs + SPIR-V cursor blend)"
);
Ok(me)
}
}
/// The non-device objects: command machinery, cursor staging, descriptor + pipelines.
fn init_objects(&mut self, qf: u32) -> Result<()> {
// SAFETY: same contract as `new` — ash calls on the live `self.device` with builder infos
// from locals outliving each synchronous call; created handles are stored into `self`
// immediately so the caller's `Drop` frees them on any later failure.
unsafe {
let d = &self.device;
self.cmd_pool = d
.create_command_pool(
&vk::CommandPoolCreateInfo::default()
.queue_family_index(qf)
.flags(vk::CommandPoolCreateFlags::RESET_COMMAND_BUFFER),
None,
)
.context("create command pool")?;
self.cmd = d
.allocate_command_buffers(
&vk::CommandBufferAllocateInfo::default()
.command_pool(self.cmd_pool)
.level(vk::CommandBufferLevel::PRIMARY)
.command_buffer_count(1),
)
.context("allocate command buffer")?[0];
self.fence = d
.create_fence(&vk::FenceCreateInfo::default(), None)
.context("create fence")?;
// Cursor staging: host-visible+coherent SSBO, persistently mapped.
let cur_size = (CURSOR_MAX * CURSOR_MAX * 4) as u64;
self.cur_buf = d
.create_buffer(
&vk::BufferCreateInfo::default()
.size(cur_size)
.usage(vk::BufferUsageFlags::STORAGE_BUFFER),
None,
)
.context("create cursor buffer")?;
let reqs = d.get_buffer_memory_requirements(self.cur_buf);
let mem_type = self
.memory_type(
reqs.memory_type_bits,
vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT,
)
.context("cursor buffer memory type")?;
self.cur_mem = d
.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(reqs.size)
.memory_type_index(mem_type),
None,
)
.context("allocate cursor memory")?;
d.bind_buffer_memory(self.cur_buf, self.cur_mem, 0)
.context("bind cursor memory")?;
self.cur_map = d
.map_memory(self.cur_mem, 0, cur_size, vk::MemoryMapFlags::empty())
.context("map cursor memory")? as *mut u8;
// Descriptor set: binding 0 = surface SSBO (rebound per blend), 1 = cursor SSBO.
let bindings = [
vk::DescriptorSetLayoutBinding::default()
.binding(0)
.descriptor_type(vk::DescriptorType::STORAGE_BUFFER)
.descriptor_count(1)
.stage_flags(vk::ShaderStageFlags::COMPUTE),
vk::DescriptorSetLayoutBinding::default()
.binding(1)
.descriptor_type(vk::DescriptorType::STORAGE_BUFFER)
.descriptor_count(1)
.stage_flags(vk::ShaderStageFlags::COMPUTE),
];
self.desc_layout = d
.create_descriptor_set_layout(
&vk::DescriptorSetLayoutCreateInfo::default().bindings(&bindings),
None,
)
.context("create descriptor layout")?;
let pc = [vk::PushConstantRange::default()
.stage_flags(vk::ShaderStageFlags::COMPUTE)
.size(std::mem::size_of::<Push>() as u32)];
let dl = [self.desc_layout];
self.pipe_layout = d
.create_pipeline_layout(
&vk::PipelineLayoutCreateInfo::default()
.set_layouts(&dl)
.push_constant_ranges(&pc),
None,
)
.context("create pipeline layout")?;
let pool_sizes = [vk::DescriptorPoolSize::default()
.ty(vk::DescriptorType::STORAGE_BUFFER)
.descriptor_count(2)];
self.desc_pool = d
.create_descriptor_pool(
&vk::DescriptorPoolCreateInfo::default()
.max_sets(1)
.pool_sizes(&pool_sizes),
None,
)
.context("create descriptor pool")?;
let dls = [self.desc_layout];
self.desc_set = d
.allocate_descriptor_sets(
&vk::DescriptorSetAllocateInfo::default()
.descriptor_pool(self.desc_pool)
.set_layouts(&dls),
)
.context("allocate descriptor set")?[0];
// The shader + one pipeline per MODE (spec constant 0).
if CURSOR_SPV.len() % 4 != 0 {
anyhow::bail!("cursor_blend.spv is not word-aligned");
}
let words: Vec<u32> = CURSOR_SPV
.chunks_exact(4)
.map(|c| u32::from_le_bytes([c[0], c[1], c[2], c[3]]))
.collect();
self.shader = d
.create_shader_module(&vk::ShaderModuleCreateInfo::default().code(&words), None)
.context("create blend shader module")?;
for mode in 0u32..3 {
let entries = [vk::SpecializationMapEntry::default()
.constant_id(0)
.offset(0)
.size(4)];
let data = mode.to_le_bytes();
let spec = vk::SpecializationInfo::default()
.map_entries(&entries)
.data(&data);
let stage = vk::PipelineShaderStageCreateInfo::default()
.stage(vk::ShaderStageFlags::COMPUTE)
.module(self.shader)
.name(c"main")
.specialization_info(&spec);
let info = [vk::ComputePipelineCreateInfo::default()
.stage(stage)
.layout(self.pipe_layout)];
let p = d
.create_compute_pipelines(vk::PipelineCache::null(), &info, None)
.map_err(|(_, e)| e)
.context("create blend pipeline")?[0];
self.pipelines[mode as usize] = p;
}
}
Ok(())
}
fn memory_type(&self, type_bits: u32, flags: vk::MemoryPropertyFlags) -> Result<u32> {
(0..self.mem_props.memory_type_count)
.find(|&i| {
type_bits & (1 << i) != 0
&& self.mem_props.memory_types[i as usize]
.property_flags
.contains(flags)
})
.ok_or_else(|| anyhow!("no memory type for flags {flags:?}"))
}
/// Allocate one NVENC input slot as exportable Vulkan memory mapped into CUDA. Layout matches
/// `InputSurface` (contiguous planes under one pitch); pitch = row bytes rounded to 256.
pub fn alloc_slot(&mut self, fmt: SlotFormat, width: u32, height: u32) -> Result<VkSlotRef> {
let pitch = (fmt.row_bytes(width) + 255) & !255;
let size = pitch * fmt.rows(height);
// SAFETY: exportable-buffer allocation, the exact `VkBridge::ensure_dst` incantation:
// `ExternalMemoryBufferCreateInfo`/`ExportMemoryAllocateInfo` declare OPAQUE_FD,
// `MemoryDedicatedAllocateInfo` ties the memory to the buffer; every info is a local
// outliving its synchronous call and every failure path destroys the objects created so
// far exactly once. `get_memory_fd` hands us an fd that `import_owned_fd` either adopts
// (driver owns it) or closes on failure.
unsafe {
let d = &self.device;
let mut ext_info = vk::ExternalMemoryBufferCreateInfo::default()
.handle_types(vk::ExternalMemoryHandleTypeFlags::OPAQUE_FD);
let buffer = d
.create_buffer(
&vk::BufferCreateInfo::default()
.size(size)
.usage(vk::BufferUsageFlags::STORAGE_BUFFER)
.push_next(&mut ext_info),
None,
)
.context("create slot buffer")?;
let reqs = d.get_buffer_memory_requirements(buffer);
let mem_type = match self
.memory_type(reqs.memory_type_bits, vk::MemoryPropertyFlags::DEVICE_LOCAL)
{
Ok(t) => t,
Err(e) => {
d.destroy_buffer(buffer, None);
return Err(e);
}
};
let mut export = vk::ExportMemoryAllocateInfo::default()
.handle_types(vk::ExternalMemoryHandleTypeFlags::OPAQUE_FD);
let mut dedicated = vk::MemoryDedicatedAllocateInfo::default().buffer(buffer);
let memory = match d.allocate_memory(
&vk::MemoryAllocateInfo::default()
.allocation_size(reqs.size)
.memory_type_index(mem_type)
.push_next(&mut export)
.push_next(&mut dedicated),
None,
) {
Ok(m) => m,
Err(e) => {
d.destroy_buffer(buffer, None);
return Err(e).context("allocate exportable slot memory");
}
};
if let Err(e) = d.bind_buffer_memory(buffer, memory, 0) {
d.free_memory(memory, None);
d.destroy_buffer(buffer, None);
return Err(e).context("bind slot memory");
}
let fd = match self.ext_fd.get_memory_fd(
&vk::MemoryGetFdInfoKHR::default()
.memory(memory)
.handle_type(vk::ExternalMemoryHandleTypeFlags::OPAQUE_FD),
) {
Ok(f) => f,
Err(e) => {
d.free_memory(memory, None);
d.destroy_buffer(buffer, None);
return Err(e).context("vkGetMemoryFdKHR(slot)");
}
};
let ext = match cuda::ExternalDmabuf::import_owned_fd(fd, reqs.size) {
Ok(c) => c,
Err(e) => {
d.free_memory(memory, None);
d.destroy_buffer(buffer, None);
return Err(e).context("cuImportExternalMemory(slot OPAQUE_FD)");
}
};
let r = VkSlotRef {
ptr: ext.ptr,
pitch: pitch as usize,
height,
id: self.slots.len(),
};
self.slots.push(SlotAlloc {
buffer,
memory,
cuda: ext,
size,
});
Ok(r)
}
}
/// Free every allocated slot (encoder teardown, alongside its ring clear). CUDA mappings drop
/// first (field order in [`SlotAlloc`] frees `cuda` via its own `Drop` before we free the VK
/// objects explicitly here).
pub fn free_slots(&mut self) {
for s in self.slots.drain(..) {
drop(s.cuda); // CUDA's view of the memory goes first
// SAFETY: `buffer`/`memory` were created in `alloc_slot`, are uniquely owned by the
// drained `SlotAlloc`, and are destroyed exactly once here. No queue work can be
// in flight: every `blend` fence-waits before returning.
unsafe {
self.device.destroy_buffer(s.buffer, None);
self.device.free_memory(s.memory, None);
}
}
}
/// Upload the cursor RGBA (`cw*ch*4`, tight rows) into the mapped staging buffer. Call only
/// when the bitmap changes; position moves are push constants.
pub fn upload_cursor(&mut self, rgba: &[u8], cw: u32, ch: u32) {
let cw = cw.min(CURSOR_MAX);
let ch = ch.min(CURSOR_MAX);
let len = (cw * ch * 4) as usize;
let len = len.min(rgba.len());
// SAFETY: `cur_map` is the live persistent mapping of the CURSOR_MAX²·4 host-coherent
// allocation (created in `init_objects`, unmapped only in `Drop`); `len` is clamped to
// both the source slice and the buffer capacity. No blend is in flight (every `blend`
// fence-waits before returning), so no GPU read races this host write.
unsafe {
std::ptr::copy_nonoverlapping(rgba.as_ptr(), self.cur_map, len);
}
}
/// Blend the uploaded cursor into `slot` at `(ox, oy)`: record, submit, fence-wait. The
/// caller has CPU-synced its CUDA frame copy first; the fence wait makes the shader's writes
/// visible to the subsequent NVENC encode (the `VkBridge` precedent: fence-ordered cross-API
/// access, no queue-family transfer — NVIDIA-only path).
#[allow(clippy::too_many_arguments)] // surface geometry + cursor rect — unpacked kernel args
pub fn blend_ref(
&mut self,
slot: &VkSlotRef,
fmt: SlotFormat,
surf_w: u32,
cw: u32,
ch: u32,
ox: i32,
oy: i32,
) -> Result<()> {
let alloc = self
.slots
.get(slot.id)
.ok_or_else(|| anyhow!("bad slot id {}", slot.id))?;
let cw = cw.min(CURSOR_MAX);
let ch = ch.min(CURSOR_MAX);
if cw == 0 || ch == 0 {
return Ok(());
}
let push = Push {
pitch: slot.pitch as u32,
surf_w,
surf_h: slot.height,
cur_w: cw,
cur_h: ch,
ox,
oy,
};
// Dispatch geometry (must match cursor_blend.comp): ARGB = per cursor px; NV12/YUV444 =
// per word-aligned 4-px span × (2-row blocks | rows).
let (gx, gy) = match fmt {
SlotFormat::Argb => (cw.div_ceil(8), ch.div_ceil(8)),
_ => {
let x0 = (ox >> 2) << 2;
let spans = ((ox + cw as i32) - x0 + 3).div_euclid(4).max(1) as u32;
let rows = match fmt {
SlotFormat::Nv12 => ch.div_ceil(2),
_ => ch,
};
(spans.div_ceil(8), rows.div_ceil(8))
}
};
// SAFETY: single-threaded record/submit/wait on handles this struct owns. The descriptor
// update is safe because no prior submission is in flight (every blend fence-waits and
// the fence is reset before reuse). Buffer infos and barrier structs are locals outliving
// their synchronous calls. The dispatch's shader accesses stay in-bounds by the shader's
// own guards (surfW/surfH/curW/curH from `push`) against the slot allocation sized in
// `alloc_slot` for exactly that geometry.
unsafe {
let d = &self.device;
let surf_info = [vk::DescriptorBufferInfo::default()
.buffer(alloc.buffer)
.offset(0)
.range(alloc.size)];
let cur_info = [vk::DescriptorBufferInfo::default()
.buffer(self.cur_buf)
.offset(0)
.range((CURSOR_MAX * CURSOR_MAX * 4) as u64)];
let writes = [
vk::WriteDescriptorSet::default()
.dst_set(self.desc_set)
.dst_binding(0)
.descriptor_type(vk::DescriptorType::STORAGE_BUFFER)
.buffer_info(&surf_info),
vk::WriteDescriptorSet::default()
.dst_set(self.desc_set)
.dst_binding(1)
.descriptor_type(vk::DescriptorType::STORAGE_BUFFER)
.buffer_info(&cur_info),
];
d.update_descriptor_sets(&writes, &[]);
d.begin_command_buffer(
self.cmd,
&vk::CommandBufferBeginInfo::default()
.flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT),
)
.context("begin blend cmd")?;
// CUDA wrote the frame into this memory outside Vulkan's view — make it visible to
// the shader (external-memory coherence ceremony; NVIDIA honors this with the fence
// ordering alone, the barrier is the spec-shaped belt-and-braces).
let acquire = [vk::MemoryBarrier::default()
.src_access_mask(vk::AccessFlags::MEMORY_WRITE)
.dst_access_mask(vk::AccessFlags::SHADER_READ | vk::AccessFlags::SHADER_WRITE)];
d.cmd_pipeline_barrier(
self.cmd,
vk::PipelineStageFlags::TOP_OF_PIPE,
vk::PipelineStageFlags::COMPUTE_SHADER,
vk::DependencyFlags::empty(),
&acquire,
&[],
&[],
);
d.cmd_bind_pipeline(
self.cmd,
vk::PipelineBindPoint::COMPUTE,
self.pipelines[fmt.mode() as usize],
);
d.cmd_bind_descriptor_sets(
self.cmd,
vk::PipelineBindPoint::COMPUTE,
self.pipe_layout,
0,
&[self.desc_set],
&[],
);
let bytes = std::slice::from_raw_parts(
(&push as *const Push) as *const u8,
std::mem::size_of::<Push>(),
);
d.cmd_push_constants(
self.cmd,
self.pipe_layout,
vk::ShaderStageFlags::COMPUTE,
0,
bytes,
);
d.cmd_dispatch(self.cmd, gx.max(1), gy.max(1), 1);
// Release the shader's writes so the post-fence CUDA/NVENC reads see them.
let release = [vk::MemoryBarrier::default()
.src_access_mask(vk::AccessFlags::SHADER_WRITE)
.dst_access_mask(vk::AccessFlags::MEMORY_READ)];
d.cmd_pipeline_barrier(
self.cmd,
vk::PipelineStageFlags::COMPUTE_SHADER,
vk::PipelineStageFlags::BOTTOM_OF_PIPE,
vk::DependencyFlags::empty(),
&release,
&[],
&[],
);
d.end_command_buffer(self.cmd).context("end blend cmd")?;
let cmds = [self.cmd];
let submit = [vk::SubmitInfo::default().command_buffers(&cmds)];
d.queue_submit(self.queue, &submit, self.fence)
.context("submit blend")?;
let r = d.wait_for_fences(&[self.fence], true, 1_000_000_000);
d.reset_fences(&[self.fence]).ok();
r.context("blend fence wait")?;
}
Ok(())
}
}
impl Drop for VkSlotBlend {
fn drop(&mut self) {
self.free_slots();
// SAFETY: every handle below was created in `new`/`init_objects` (or is null from a
// partial init — Vulkan destroy/free calls are defined no-ops on null handles) and is
// uniquely owned; each is destroyed exactly once here, pipelines/layouts/pools before the
// device, the device before the instance. No work is in flight (`blend` fence-waits).
unsafe {
let d = &self.device;
for p in self.pipelines {
if p != vk::Pipeline::null() {
d.destroy_pipeline(p, None);
}
}
if self.shader != vk::ShaderModule::null() {
d.destroy_shader_module(self.shader, None);
}
if self.desc_pool != vk::DescriptorPool::null() {
d.destroy_descriptor_pool(self.desc_pool, None);
}
if self.pipe_layout != vk::PipelineLayout::null() {
d.destroy_pipeline_layout(self.pipe_layout, None);
}
if self.desc_layout != vk::DescriptorSetLayout::null() {
d.destroy_descriptor_set_layout(self.desc_layout, None);
}
if !self.cur_map.is_null() {
d.unmap_memory(self.cur_mem);
}
if self.cur_buf != vk::Buffer::null() {
d.destroy_buffer(self.cur_buf, None);
}
if self.cur_mem != vk::DeviceMemory::null() {
d.free_memory(self.cur_mem, None);
}
if self.fence != vk::Fence::null() {
d.destroy_fence(self.fence, None);
}
if self.cmd_pool != vk::CommandPool::null() {
d.destroy_command_pool(self.cmd_pool, None);
}
d.destroy_device(None);
self.instance.destroy_instance(None);
}
}
}
+372 -3
View File
@@ -555,6 +555,9 @@ pub struct PunktfunkConnection {
/// (a fetched payload, an offer's format list, or a fetch-request's MIME) — /// (a fetched payload, an offer's format list, or a fetch-request's MIME) —
/// borrow-until-next-call, same contract as `last`. /// borrow-until-next-call, same contract as `last`.
last_clip: std::sync::Mutex<Option<Vec<u8>>>, last_clip: std::sync::Mutex<Option<Vec<u8>>>,
/// The last cursor shape handed out — `next_cursor_shape`'s `rgba` pointer borrows it
/// until the next cursor-shape call (the `last_audio` contract).
last_cursor_shape: std::sync::Mutex<Option<crate::quic::CursorShape>>,
} }
/// Lazily-initialized in-core Opus decode state. A coupled-1-stream multistream decoder is /// Lazily-initialized in-core Opus decode state. A coupled-1-stream multistream decoder is
@@ -867,6 +870,94 @@ impl PunktfunkRichInputEx {
} }
} }
/// [`PunktfunkPenSample::state`] bit: the pen hovers in range (implied by `TOUCHING`).
pub const PUNKTFUNK_PEN_IN_RANGE: u8 = 0x01;
/// [`PunktfunkPenSample::state`] bit: the tip is in contact.
pub const PUNKTFUNK_PEN_TOUCHING: u8 = 0x02;
/// [`PunktfunkPenSample::state`] bit: primary barrel button (or squeeze mapping) held.
pub const PUNKTFUNK_PEN_BARREL1: u8 = 0x04;
/// [`PunktfunkPenSample::state`] bit: secondary barrel button (or double-tap mapping) held.
pub const PUNKTFUNK_PEN_BARREL2: u8 = 0x08;
/// [`PunktfunkPenSample::tool`]: the pen tip.
pub const PUNKTFUNK_PEN_TOOL_PEN: u8 = 0;
/// [`PunktfunkPenSample::tool`]: the eraser (a client-side mode — Apple Pencil has no
/// hardware eraser end; the squeeze/double-tap mapping usually drives this).
pub const PUNKTFUNK_PEN_TOOL_ERASER: u8 = 1;
/// Most samples one [`punktfunk_connection_send_pen`] call accepts (one wire batch).
pub const PUNKTFUNK_PEN_BATCH_MAX: u32 = 8;
/// [`PunktfunkPenSample::tilt_deg`] sentinel: no tilt reading.
pub const PUNKTFUNK_PEN_TILT_UNKNOWN: u8 = 0xFF;
/// [`PunktfunkPenSample::azimuth_deg`] / `roll_deg` sentinel: no reading.
pub const PUNKTFUNK_PEN_ANGLE_UNKNOWN: u16 = 0xFFFF;
/// [`PunktfunkPenSample::distance`] sentinel: no hover-distance reading.
pub const PUNKTFUNK_PEN_DISTANCE_UNKNOWN: u16 = 0xFFFF;
/// One complete stylus state at one instant ([`punktfunk_connection_send_pen`];
/// design/pen-tablet-input.md). STATE-FULL, never an edge event: fill every field on every
/// sample (unknown axes take their `*_UNKNOWN` sentinel) — the host diffs consecutive samples
/// and synthesizes down/up/button transitions itself, which is what makes a lost datagram
/// self-heal. `x`/`y` are normalized `0.0..=1.0` in VIDEO-FRAME space (map your letterbox
/// before filling, exactly like wire touches).
#[cfg(feature = "quic")]
#[repr(C)]
#[derive(Clone, Copy)]
pub struct PunktfunkPenSample {
/// Normalized `0.0..=1.0` across the video frame. Must be finite.
pub x: f32,
/// Normalized `0.0..=1.0` across the video frame. Must be finite.
pub y: f32,
/// Tip force, `0..=65535` full scale (`0` while hovering).
pub pressure: u16,
/// Hover distance `0..=65534` (0 = at the hover floor), or `PUNKTFUNK_PEN_DISTANCE_UNKNOWN`.
pub distance: u16,
/// Tilt azimuth, degrees `0..=359` clockwise from north, or `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
pub azimuth_deg: u16,
/// Barrel roll (Apple Pencil Pro `rollAngle`), degrees `0..=359`, or
/// `PUNKTFUNK_PEN_ANGLE_UNKNOWN`.
pub roll_deg: u16,
/// µs since the previous sample in the same call (`0` for the first) — the coalesced
/// capture spacing.
pub dt_us: u16,
/// Bitfield of `PUNKTFUNK_PEN_*` state bits. Unknown bits are rejected (`InvalidArg`).
pub state: u8,
/// `PUNKTFUNK_PEN_TOOL_PEN` or `PUNKTFUNK_PEN_TOOL_ERASER`.
pub tool: u8,
/// Tilt from the surface normal, degrees `0..=90`, or `PUNKTFUNK_PEN_TILT_UNKNOWN`.
pub tilt_deg: u8,
/// Set to 0.
pub _reserved: [u8; 3],
}
#[cfg(feature = "quic")]
impl PunktfunkPenSample {
/// `None` = invalid field (non-finite coordinate, unknown state bit, unknown tool) —
/// embedder input is validated strictly, unlike the loss-tolerant wire decode.
fn to_sample(self) -> Option<crate::quic::PenSample> {
use crate::quic as q;
let known = q::PEN_IN_RANGE | q::PEN_TOUCHING | q::PEN_BARREL1 | q::PEN_BARREL2;
if !self.x.is_finite() || !self.y.is_finite() || self.state & !known != 0 {
return None;
}
let tool = match self.tool {
PUNKTFUNK_PEN_TOOL_PEN => q::PenTool::Pen,
PUNKTFUNK_PEN_TOOL_ERASER => q::PenTool::Eraser,
_ => return None,
};
Some(q::PenSample {
state: self.state,
tool,
x: self.x,
y: self.y,
pressure: self.pressure,
distance: self.distance,
tilt_deg: self.tilt_deg,
azimuth_deg: self.azimuth_deg,
roll_deg: self.roll_deg,
dt_us: self.dt_us,
})
}
}
/// Read an optional NUL-terminated UTF-8 string parameter; `Err` = invalid pointer/UTF-8. /// Read an optional NUL-terminated UTF-8 string parameter; `Err` = invalid pointer/UTF-8.
#[cfg(feature = "quic")] #[cfg(feature = "quic")]
unsafe fn opt_cstr<'a>(p: *const std::os::raw::c_char) -> std::result::Result<Option<&'a str>, ()> { unsafe fn opt_cstr<'a>(p: *const std::os::raw::c_char) -> std::result::Result<Option<&'a str>, ()> {
@@ -985,6 +1076,12 @@ pub const PUNKTFUNK_HOST_CAP_GAMEPAD_STATE: u8 = 0x01;
/// Host-capability bit in [`punktfunk_connection_host_caps`]: the host supports the shared /// Host-capability bit in [`punktfunk_connection_host_caps`]: the host supports the shared
/// clipboard, so a client may offer the toggle. (Mirrors `quic::HOST_CAP_CLIPBOARD`.) /// clipboard, so a client may offer the toggle. (Mirrors `quic::HOST_CAP_CLIPBOARD`.)
pub const PUNKTFUNK_HOST_CAP_CLIPBOARD: u8 = 0x02; pub const PUNKTFUNK_HOST_CAP_CLIPBOARD: u8 = 0x02;
/// Host-capability bit in [`punktfunk_connection_host_caps`]: the host injects full-fidelity
/// stylus input, so a capable client splits pen contacts out of its touch path and sends them
/// via [`punktfunk_connection_send_pen`]; without the bit that call returns `Unsupported` and
/// the client keeps its pen-as-touch fallback. (Mirrors `quic::HOST_CAP_PEN`;
/// design/pen-tablet-input.md.)
pub const PUNKTFUNK_HOST_CAP_PEN: u8 = 0x10;
// Keep the ABI cap bits in lockstep with the wire constants (compile-time guard against drift). // Keep the ABI cap bits in lockstep with the wire constants (compile-time guard against drift).
#[cfg(feature = "quic")] #[cfg(feature = "quic")]
@@ -998,6 +1095,15 @@ const _: () = {
assert!(PUNKTFUNK_CODEC_PYROWAVE == crate::quic::CODEC_PYROWAVE); assert!(PUNKTFUNK_CODEC_PYROWAVE == crate::quic::CODEC_PYROWAVE);
assert!(PUNKTFUNK_HOST_CAP_GAMEPAD_STATE == crate::quic::HOST_CAP_GAMEPAD_STATE); assert!(PUNKTFUNK_HOST_CAP_GAMEPAD_STATE == crate::quic::HOST_CAP_GAMEPAD_STATE);
assert!(PUNKTFUNK_HOST_CAP_CLIPBOARD == crate::quic::HOST_CAP_CLIPBOARD); assert!(PUNKTFUNK_HOST_CAP_CLIPBOARD == crate::quic::HOST_CAP_CLIPBOARD);
assert!(PUNKTFUNK_HOST_CAP_PEN == crate::quic::HOST_CAP_PEN);
assert!(PUNKTFUNK_PEN_IN_RANGE == crate::quic::PEN_IN_RANGE);
assert!(PUNKTFUNK_PEN_TOUCHING == crate::quic::PEN_TOUCHING);
assert!(PUNKTFUNK_PEN_BARREL1 == crate::quic::PEN_BARREL1);
assert!(PUNKTFUNK_PEN_BARREL2 == crate::quic::PEN_BARREL2);
assert!(PUNKTFUNK_PEN_BATCH_MAX as usize == crate::quic::PEN_BATCH_MAX);
assert!(PUNKTFUNK_PEN_TILT_UNKNOWN == crate::quic::PEN_TILT_UNKNOWN);
assert!(PUNKTFUNK_PEN_ANGLE_UNKNOWN == crate::quic::PEN_ANGLE_UNKNOWN);
assert!(PUNKTFUNK_PEN_DISTANCE_UNKNOWN == crate::quic::PEN_DISTANCE_UNKNOWN);
}; };
// Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift). // Keep the ABI gamepad constants in lockstep with the wire enum (compile-time guard against drift).
@@ -1400,6 +1506,7 @@ pub unsafe extern "C" fn punktfunk_connect_ex7(
connect_ex_impl( connect_ex_impl(
host, host,
port, port,
0, // pre-v11 variant: no client caps
width, width,
height, height,
refresh_hz, refresh_hz,
@@ -1459,6 +1566,7 @@ pub unsafe extern "C" fn punktfunk_connect_ex8(
connect_ex_impl( connect_ex_impl(
host, host,
port, port,
0, // pre-v11 variant: no client caps
width, width,
height, height,
refresh_hz, refresh_hz,
@@ -1480,6 +1588,70 @@ pub unsafe extern "C" fn punktfunk_connect_ex8(
} }
} }
/// Like [`punktfunk_connect_ex8`], plus `client_caps` (ABI v11): a bitfield of
/// `PUNKTFUNK_CLIENT_CAP_CURSOR` (0x01). Setting the cursor bit asks the host to STOP
/// compositing the pointer into the video and forward it out-of-band instead — the embedder
/// MUST then drain [`punktfunk_connection_next_cursor_shape`] /
/// [`punktfunk_connection_next_cursor_state`] and draw the pointer itself, or the session has
/// no visible cursor at all. Pass 0 for the composited behavior of every earlier variant.
///
/// # Safety
/// Same as [`punktfunk_connect_ex8`].
#[cfg(feature = "quic")]
#[no_mangle]
#[allow(clippy::too_many_arguments)]
pub unsafe extern "C" fn punktfunk_connect_ex9(
host: *const std::os::raw::c_char,
port: u16,
width: u32,
height: u32,
refresh_hz: u32,
compositor: u32,
gamepad: u32,
bitrate_kbps: u32,
video_caps: u8,
audio_channels: u8,
video_codecs: u8,
preferred_codec: u8,
client_caps: u8,
launch_id: *const std::os::raw::c_char,
pin_sha256: *const u8,
observed_sha256_out: *mut u8,
client_cert_pem: *const std::os::raw::c_char,
client_key_pem: *const std::os::raw::c_char,
timeout_ms: u32,
status_out: *mut i32,
) -> *mut PunktfunkConnection {
unsafe {
connect_ex_impl(
host,
port,
client_caps,
width,
height,
refresh_hz,
compositor,
gamepad,
bitrate_kbps,
video_caps,
audio_channels,
video_codecs,
preferred_codec,
launch_id,
pin_sha256,
observed_sha256_out,
client_cert_pem,
client_key_pem,
timeout_ms,
status_out,
)
}
}
/// [`punktfunk_connect_ex9`] `client_caps` bit: render the host cursor locally (the cursor
/// channel, `design/remote-desktop-sweep.md` M2).
pub const PUNKTFUNK_CLIENT_CAP_CURSOR: u8 = 0x01;
/// Shared body of [`punktfunk_connect_ex7`] / [`punktfunk_connect_ex8`]: `status_out` /// Shared body of [`punktfunk_connect_ex7`] / [`punktfunk_connect_ex8`]: `status_out`
/// (nullable) is written on EVERY path — `Ok`, the mapped [`PunktfunkError`], /// (nullable) is written on EVERY path — `Ok`, the mapped [`PunktfunkError`],
/// `InvalidArg` for bad arguments, `Panic` if the connect panicked. /// `InvalidArg` for bad arguments, `Panic` if the connect panicked.
@@ -1488,6 +1660,7 @@ pub unsafe extern "C" fn punktfunk_connect_ex8(
unsafe fn connect_ex_impl( unsafe fn connect_ex_impl(
host: *const std::os::raw::c_char, host: *const std::os::raw::c_char,
port: u16, port: u16,
client_caps: u8,
width: u32, width: u32,
height: u32, height: u32,
refresh_hz: u32, refresh_hz: u32,
@@ -1574,6 +1747,10 @@ unsafe fn connect_ex_impl(
// themselves (EDR / MediaCodec), so the host's EDID defaults are fine there. An `ex8` // themselves (EDR / MediaCodec), so the host's EDID defaults are fine there. An `ex8`
// variant can carry it if a passthrough embedder ever needs it. // variant can carry it if a passthrough embedder ever needs it.
None, None,
// ABI v11 ([`punktfunk_connect_ex9`]): CLIENT_CAP_CURSOR here asks the host to STOP
// compositing the pointer — only an embedder that renders the cursor planes
// ([`punktfunk_connection_next_cursor_shape`]/`_state`) may set it. ex7/ex8 pass 0.
client_caps,
launch, launch,
pin, pin,
identity, identity,
@@ -1593,6 +1770,7 @@ unsafe fn connect_ex_impl(
last_audio: std::sync::Mutex::new(None), last_audio: std::sync::Mutex::new(None),
audio_pcm: std::sync::Mutex::new(AudioPcmState::default()), audio_pcm: std::sync::Mutex::new(AudioPcmState::default()),
last_clip: std::sync::Mutex::new(None), last_clip: std::sync::Mutex::new(None),
last_cursor_shape: std::sync::Mutex::new(None),
})) }))
} }
Err(e) => { Err(e) => {
@@ -2242,6 +2420,151 @@ pub unsafe extern "C" fn punktfunk_connection_next_hdr_meta(
}) })
} }
/// One forwarded host-cursor shape (ABI v11, the cursor channel): straight-alpha RGBA8, no
/// padding, `len == w * h * 4`, hotspot within `w`×`h`. `serial` is the identity
/// [`PunktfunkCursorState`] refers to — cache the built OS cursor by it.
#[repr(C)]
pub struct PunktfunkCursorShape {
pub serial: u32,
pub w: u16,
pub h: u16,
pub hot_x: u16,
pub hot_y: u16,
/// Borrows connection memory until the NEXT cursor-shape call (the audio contract).
pub rgba: *const u8,
pub len: usize,
}
/// Per-frame host-cursor state (ABI v11): position (the pointer/hotspot point in the host
/// video's pixel space), visibility, and the host-driven relative-mode hint. `flags` bit 0 =
/// visible, bit 1 = relative hint (a host app grabbed/hid the pointer — run captured
/// relative; clear = return to absolute, reappearing at `x`/`y`).
#[repr(C)]
pub struct PunktfunkCursorState {
pub serial: u32,
pub flags: u8,
pub x: i32,
pub y: i32,
}
/// Pull the next forwarded cursor SHAPE (sent on pointer-bitmap change over the reliable
/// control stream; only a session connected with `PUNKTFUNK_CLIENT_CAP_CURSOR` against a
/// capable host receives any). On `Ok`, `out->rgba` borrows connection memory until the next
/// cursor-shape call on this handle. Drain from a dedicated thread (one thread per plane).
///
/// # Safety
/// `c` is a valid connection handle; `out` is writable. At most one thread pulls cursor
/// shapes; it may run concurrently with every other plane's puller.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_next_cursor_shape(
c: *mut PunktfunkConnection,
out: *mut PunktfunkCursorShape,
timeout_ms: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if out.is_null() {
return PunktfunkStatus::NullPointer;
}
match c
.inner
.next_cursor_shape(std::time::Duration::from_millis(timeout_ms as u64))
{
Ok(shape) => {
let mut slot = c.last_cursor_shape.lock().unwrap();
*slot = Some(shape);
let sh = slot.as_ref().unwrap();
unsafe {
*out = PunktfunkCursorShape {
serial: sh.serial,
w: sh.w,
h: sh.h,
hot_x: sh.hot_x,
hot_y: sh.hot_y,
rgba: sh.rgba.as_ptr(),
len: sh.rgba.len(),
};
}
PunktfunkStatus::Ok
}
Err(e) => e.status(),
}
})
}
/// Pull the next cursor STATE (a `0xD0` datagram per host encode tick — latest-wins; drain
/// the queue and apply only the newest). Same negotiation gate as
/// [`punktfunk_connection_next_cursor_shape`].
///
/// # Safety
/// `c` is a valid connection handle; `out` is writable. At most one thread pulls cursor
/// state; it may run concurrently with every other plane's puller.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_next_cursor_state(
c: *mut PunktfunkConnection,
out: *mut PunktfunkCursorState,
timeout_ms: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if out.is_null() {
return PunktfunkStatus::NullPointer;
}
match c
.inner
.next_cursor_state(std::time::Duration::from_millis(timeout_ms as u64))
{
Ok(st) => {
unsafe {
*out = PunktfunkCursorState {
serial: st.serial,
flags: st.flags,
x: st.x,
y: st.y,
};
}
PunktfunkStatus::Ok
}
Err(e) => e.status(),
}
})
}
/// Tell the host who renders the pointer (design/remote-desktop-sweep.md §8 — the mid-stream
/// mouse-model flip): `client_draws = true` = this client draws it locally (the desktop mouse
/// model; the host excludes the pointer from the video and forwards shape/state), `false` =
/// the host composites it into the video (the capture model — full fidelity, the pre-channel
/// look). Idempotent, latest-wins; harmless against hosts without the cursor cap (an unknown
/// control message type, ignored). ABI v12.
///
/// # Safety
/// `c` is a valid connection handle.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_set_cursor_render(
c: *mut PunktfunkConnection,
client_draws: bool,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
match c.inner.set_cursor_render(client_draws) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// Pull the next per-AU host timing (0xCF) into `*out`: the host's capture→sent duration for one /// Pull the next per-AU host timing (0xCF) into `*out`: the host's capture→sent duration for one
/// access unit, correlated to the AU by `pts_ns` (see [`PunktfunkHostTiming`]). /// access unit, correlated to the AU by `pts_ns` (see [`PunktfunkHostTiming`]).
/// [`PunktfunkStatus::NoFrame`] on timeout, [`PunktfunkStatus::Closed`] once the session ended. /// [`PunktfunkStatus::NoFrame`] on timeout, [`PunktfunkStatus::Closed`] once the session ended.
@@ -2543,6 +2866,51 @@ pub unsafe extern "C" fn punktfunk_connection_send_rich_input2(
}) })
} }
/// Send one stylus sample batch — `count` (`1..=PUNKTFUNK_PEN_BATCH_MAX`) state-full
/// [`PunktfunkPenSample`]s, oldest first (a capture callback's coalesced samples) — as one
/// `0xCC/0x05` pen datagram (non-blocking enqueue; design/pen-tablet-input.md). Split longer
/// runs into consecutive calls. Gate on `punktfunk_connection_host_caps() &
/// PUNKTFUNK_HOST_CAP_PEN`: toward a host without the bit this returns
/// [`PunktfunkStatus::Unsupported`] — keep the pen-as-touch fallback there.
/// [`PunktfunkStatus::InvalidArg`] on a bad count or a bad sample (non-finite coordinate,
/// unknown state bit / tool).
///
/// # Safety
/// `c` is a valid connection handle; `samples` is null or points to `count` valid
/// [`PunktfunkPenSample`]s.
#[cfg(feature = "quic")]
#[no_mangle]
pub unsafe extern "C" fn punktfunk_connection_send_pen(
c: *mut PunktfunkConnection,
samples: *const PunktfunkPenSample,
count: u32,
) -> PunktfunkStatus {
guard(|| {
let c = match unsafe { c.as_ref() } {
Some(c) => c,
None => return PunktfunkStatus::NullPointer,
};
if samples.is_null() {
return PunktfunkStatus::NullPointer;
}
if count == 0 || count > PUNKTFUNK_PEN_BATCH_MAX {
return PunktfunkStatus::InvalidArg;
}
let raw = unsafe { std::slice::from_raw_parts(samples, count as usize) };
let mut batch = [crate::quic::PenSample::default(); crate::quic::PEN_BATCH_MAX];
for (slot, s) in batch.iter_mut().zip(raw) {
match s.to_sample() {
Some(v) => *slot = v,
None => return PunktfunkStatus::InvalidArg,
}
}
match c.inner.send_pen(&batch[..count as usize]) {
Ok(()) => PunktfunkStatus::Ok,
Err(e) => e.status(),
}
})
}
/// The currently active session mode — the Welcome's, until an accepted /// The currently active session mode — the Welcome's, until an accepted
/// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect. /// [`punktfunk_connection_request_mode`] switches it. Safe any time after connect.
/// ///
@@ -2757,9 +3125,10 @@ fn build_clip_event(
} }
/// The host capability bitfield the session's `Welcome` carried — a bitfield of /// The host capability bitfield the session's `Welcome` carried — a bitfield of
/// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD`. A client tests /// `PUNKTFUNK_HOST_CAP_GAMEPAD_STATE` / `PUNKTFUNK_HOST_CAP_CLIPBOARD` /
/// `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide whether to offer the shared-clipboard toggle. /// `PUNKTFUNK_HOST_CAP_PEN`. A client tests `caps & PUNKTFUNK_HOST_CAP_CLIPBOARD` to decide
/// Safe any time after connect. /// whether to offer the shared-clipboard toggle, `caps & PUNKTFUNK_HOST_CAP_PEN` before
/// sending stylus batches. Safe any time after connect.
/// ///
/// # Safety /// # Safety
/// `c` is a valid connection handle; `caps` is writable (NULL is skipped). /// `c` is a valid connection handle; `caps` is writable (NULL is skipped).
@@ -28,6 +28,10 @@ pub(crate) enum CtrlRequest {
/// Announce that the local clipboard changed — the lazy format-list offer (bytes cross later on /// Announce that the local clipboard changed — the lazy format-list offer (bytes cross later on
/// a fetch stream). Symmetric message; the host may send one too. /// a fetch stream). Symmetric message; the host may send one too.
ClipOffer(ClipOffer), ClipOffer(ClipOffer),
/// Who renders the pointer (cursor-forward sessions): `true` = client draws locally (the
/// desktop mouse model — host excludes + forwards), `false` = host composites into the
/// video (the capture model). Sent on every mouse-model flip; idempotent, latest-wins.
CursorRender(crate::quic::CursorRenderMode),
} }
/// What the worker reports to [`NativeClient::connect`] once the handshake lands: the /// What the worker reports to [`NativeClient::connect`] once the handshake lands: the
+107 -7
View File
@@ -20,7 +20,7 @@ use crate::quic::{
RfiRequest, RichInput, RfiRequest, RichInput,
}; };
use crate::session::Frame; use crate::session::Frame;
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU32, AtomicU64, Ordering}; use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU16, AtomicU32, AtomicU64, Ordering};
use std::sync::mpsc::{Receiver, RecvTimeoutError}; use std::sync::mpsc::{Receiver, RecvTimeoutError};
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
@@ -42,8 +42,8 @@ pub use self::rumble::{ActuatorQuirks, RumbleCommand};
use self::control::{CtrlRequest, Negotiated}; use self::control::{CtrlRequest, Negotiated};
use self::frame_channel::{DecodeLatAcc, FrameChannel, FramePop}; use self::frame_channel::{DecodeLatAcc, FrameChannel, FramePop};
use self::planes::{ use self::planes::{
RumbleUpdate, AUDIO_QUEUE, CLIP_EVENT_QUEUE, HDR_META_QUEUE, HIDOUT_QUEUE, HOST_TIMING_QUEUE, RumbleUpdate, AUDIO_QUEUE, CLIP_EVENT_QUEUE, CURSOR_SHAPE_QUEUE, CURSOR_STATE_QUEUE,
RUMBLE_QUEUE, HDR_META_QUEUE, HIDOUT_QUEUE, HOST_TIMING_QUEUE, RUMBLE_QUEUE,
}; };
use self::probe::ProbeState; use self::probe::ProbeState;
use self::pump::run_pump; use self::pump::run_pump;
@@ -93,13 +93,22 @@ pub struct NativeClient {
/// Inbound per-AU host capture→send timings — 0xCF datagrams (the client always advertises /// Inbound per-AU host capture→send timings — 0xCF datagrams (the client always advertises
/// [`quic::VIDEO_CAP_HOST_TIMING`]; an older host simply never sends any). /// [`quic::VIDEO_CAP_HOST_TIMING`]; an older host simply never sends any).
host_timing: Mutex<Receiver<crate::quic::HostTiming>>, host_timing: Mutex<Receiver<crate::quic::HostTiming>>,
/// Inbound cursor shapes (control-stream [`crate::quic::CursorShape`]) — only a session
/// that advertised [`quic::CLIENT_CAP_CURSOR`] against a [`quic::HOST_CAP_CURSOR`] host
/// ever receives any.
cursor_shape: Mutex<Receiver<crate::quic::CursorShape>>,
/// Inbound per-frame cursor state — `0xD0` datagrams (same negotiation gate as shapes).
cursor_state: Mutex<Receiver<crate::quic::CursorState>>,
input_tx: tokio::sync::mpsc::UnboundedSender<InputEvent>, input_tx: tokio::sync::mpsc::UnboundedSender<InputEvent>,
/// Outbound mic frames `(seq, pts_ns, opus)` → encoded as 0xCB datagrams by the worker. /// Outbound mic frames `(seq, pts_ns, opus)` → encoded as 0xCB datagrams by the worker.
/// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing /// Bounded ([`MIC_QUEUE`]): a wedged worker drops fresh frames (logged) instead of queueing
/// audio-latency (and memory) without limit — mic is best-effort end to end. /// audio-latency (and memory) without limit — mic is best-effort end to end.
mic_tx: tokio::sync::mpsc::Sender<(u32, u64, Vec<u8>)>, mic_tx: tokio::sync::mpsc::Sender<(u32, u64, Vec<u8>)>,
/// Outbound rich input (DualSense touchpad / motion) → 0xCC datagrams by the worker. /// Outbound 0xCC rich-input plane, PRE-ENCODED datagrams: [`RichInput`] touchpad/motion
rich_input_tx: tokio::sync::mpsc::UnboundedSender<RichInput>, /// (encoded in [`NativeClient::send_rich_input`]) and stylus [`crate::quic::PenBatch`]es
/// (encoded in [`NativeClient::send_pen`]) share the channel — the worker's task just
/// forwards bytes, so a new 0xCC kind never touches the pump.
rich_input_tx: tokio::sync::mpsc::UnboundedSender<Vec<u8>>,
/// Outbound control-stream requests (mode switch, speed test) → the worker's control task. /// Outbound control-stream requests (mode switch, speed test) → the worker's control task.
/// Bounded ([`CTRL_QUEUE`]) — the requests are sparse; a full queue means the control task /// Bounded ([`CTRL_QUEUE`]) — the requests are sparse; a full queue means the control task
/// is wedged/dead, and callers treat it like a closed session. /// is wedged/dead, and callers treat it like a closed session.
@@ -115,6 +124,9 @@ pub struct NativeClient {
/// Monotonic id for outbound fetches ([`NativeClient::clip_fetch`]); stays below /// Monotonic id for outbound fetches ([`NativeClient::clip_fetch`]); stays below
/// [`crate::clipboard::INBOUND_REQ_FLAG`] so it never collides with an inbound serve `req_id`. /// [`crate::clipboard::INBOUND_REQ_FLAG`] so it never collides with an inbound serve `req_id`.
next_xfer_id: AtomicU32, next_xfer_id: AtomicU32,
/// Wrapping per-connection [`crate::quic::PenBatch::seq`] counter, stamped by
/// [`NativeClient::send_pen`] (the host's reorder gate compares it).
pen_seq: AtomicU16,
/// The host capability bitfield ([`crate::quic::Welcome::host_caps`]) — see /// The host capability bitfield ([`crate::quic::Welcome::host_caps`]) — see
/// [`NativeClient::host_caps`]. /// [`NativeClient::host_caps`].
pub host_caps: u8, pub host_caps: u8,
@@ -316,6 +328,12 @@ impl NativeClient {
// display's EDID so host apps tone-map to the client's real panel; `None` = unknown/SDR // display's EDID so host apps tone-map to the client's real panel; `None` = unknown/SDR
// (the host keeps its built-in EDID defaults). See [`crate::quic::Hello::display_hdr`]. // (the host keeps its built-in EDID defaults). See [`crate::quic::Hello::display_hdr`].
display_hdr: Option<HdrMeta>, display_hdr: Option<HdrMeta>,
// Non-video client capabilities ([`crate::quic::Hello::client_caps`]) — set
// [`crate::quic::CLIENT_CAP_CURSOR`] ONLY if this embedder actually renders the host
// cursor locally (shape + state planes): the host stops compositing the pointer into
// the video for a session that advertises it, so a non-rendering embedder that sets it
// streams with NO visible cursor at all. `0` = today's composited behavior.
client_caps: u8,
launch: Option<String>, launch: Option<String>,
pin: Option<[u8; 32]>, pin: Option<[u8; 32]>,
identity: Option<(String, String)>, identity: Option<(String, String)>,
@@ -332,11 +350,15 @@ impl NativeClient {
std::sync::mpsc::sync_channel::<crate::quic::HostTiming>(HOST_TIMING_QUEUE); std::sync::mpsc::sync_channel::<crate::quic::HostTiming>(HOST_TIMING_QUEUE);
let (input_tx, input_rx) = tokio::sync::mpsc::unbounded_channel::<InputEvent>(); let (input_tx, input_rx) = tokio::sync::mpsc::unbounded_channel::<InputEvent>();
let (mic_tx, mic_rx) = tokio::sync::mpsc::channel::<(u32, u64, Vec<u8>)>(MIC_QUEUE); let (mic_tx, mic_rx) = tokio::sync::mpsc::channel::<(u32, u64, Vec<u8>)>(MIC_QUEUE);
let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<RichInput>(); let (rich_input_tx, rich_input_rx) = tokio::sync::mpsc::unbounded_channel::<Vec<u8>>();
let (ctrl_tx, ctrl_rx) = tokio::sync::mpsc::channel::<CtrlRequest>(CTRL_QUEUE); let (ctrl_tx, ctrl_rx) = tokio::sync::mpsc::channel::<CtrlRequest>(CTRL_QUEUE);
let (clip_event_tx, clip_event_rx) = let (clip_event_tx, clip_event_rx) =
std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE); std::sync::mpsc::sync_channel::<ClipEventCore>(CLIP_EVENT_QUEUE);
let (clip_cmd_tx, clip_cmd_rx) = tokio::sync::mpsc::unbounded_channel::<ClipCommand>(); let (clip_cmd_tx, clip_cmd_rx) = tokio::sync::mpsc::unbounded_channel::<ClipCommand>();
let (cursor_shape_tx, cursor_shape_rx) =
std::sync::mpsc::sync_channel::<crate::quic::CursorShape>(CURSOR_SHAPE_QUEUE);
let (cursor_state_tx, cursor_state_rx) =
std::sync::mpsc::sync_channel::<crate::quic::CursorState>(CURSOR_STATE_QUEUE);
let (ready_tx, ready_rx) = std::sync::mpsc::channel::<Result<Negotiated>>(); let (ready_tx, ready_rx) = std::sync::mpsc::channel::<Result<Negotiated>>();
let shutdown = Arc::new(AtomicBool::new(false)); let shutdown = Arc::new(AtomicBool::new(false));
let quit = Arc::new(AtomicBool::new(false)); let quit = Arc::new(AtomicBool::new(false));
@@ -390,6 +412,7 @@ impl NativeClient {
video_codecs, video_codecs,
preferred_codec, preferred_codec,
display_hdr, display_hdr,
client_caps,
launch, launch,
pin, pin,
identity, identity,
@@ -401,6 +424,8 @@ impl NativeClient {
hidout_tx, hidout_tx,
hdr_meta_tx, hdr_meta_tx,
host_timing_tx, host_timing_tx,
cursor_shape_tx,
cursor_state_tx,
input_rx, input_rx,
mic_rx, mic_rx,
rich_input_rx, rich_input_rx,
@@ -445,6 +470,8 @@ impl NativeClient {
hidout: Mutex::new(hidout_rx), hidout: Mutex::new(hidout_rx),
hdr_meta: Mutex::new(hdr_meta_rx), hdr_meta: Mutex::new(hdr_meta_rx),
host_timing: Mutex::new(host_timing_rx), host_timing: Mutex::new(host_timing_rx),
cursor_shape: Mutex::new(cursor_shape_rx),
cursor_state: Mutex::new(cursor_state_rx),
input_tx, input_tx,
mic_tx, mic_tx,
rich_input_tx, rich_input_tx,
@@ -452,6 +479,7 @@ impl NativeClient {
clip: Mutex::new(clip_event_rx), clip: Mutex::new(clip_event_rx),
clip_cmd_tx, clip_cmd_tx,
next_xfer_id: AtomicU32::new(1), next_xfer_id: AtomicU32::new(1),
pen_seq: AtomicU16::new(0),
host_caps: negotiated.host_caps, host_caps: negotiated.host_caps,
probe, probe,
shutdown, shutdown,
@@ -552,6 +580,20 @@ impl NativeClient {
.map_err(|_| PunktfunkError::Closed) .map_err(|_| PunktfunkError::Closed)
} }
/// Tell the host who renders the pointer (cursor-forward sessions —
/// design/remote-desktop-sweep.md §8): `true` = this client draws it locally (the desktop
/// mouse model; the host excludes the pointer from the video and forwards shape/state),
/// `false` = the host composites it into the video (the capture model — full fidelity,
/// the pre-channel behavior). Call on every mouse-model flip; idempotent, latest-wins,
/// no-op on hosts without [`HOST_CAP_CURSOR`](crate::quic::HOST_CAP_CURSOR).
pub fn set_cursor_render(&self, client_draws: bool) -> Result<()> {
self.ctrl_tx
.try_send(CtrlRequest::CursorRender(crate::quic::CursorRenderMode {
client_draws,
}))
.map_err(|_| PunktfunkError::Closed)
}
/// Ask the host's encoder to emit a fresh IDR keyframe now (client recovery on a stalled /// Ask the host's encoder to emit a fresh IDR keyframe now (client recovery on a stalled
/// decode). Non-blocking, fire-and-forget — the recovered keyframe is the only ack. The /// decode). Non-blocking, fire-and-forget — the recovered keyframe is the only ack. The
/// caller should throttle (the decode stays wedged across several frames until the IDR /// caller should throttle (the decode stays wedged across several frames until the IDR
@@ -892,6 +934,32 @@ impl NativeClient {
} }
} }
/// Pull the next host cursor shape (design/remote-desktop-sweep.md M2): RGBA bitmap +
/// hotspot, sent on pointer-bitmap change over the reliable control stream. The embedder
/// caches by `serial` and builds an OS cursor from it; [`NativeClient::next_cursor_state`]
/// references shapes by serial. Only a session that advertised
/// [`crate::quic::CLIENT_CAP_CURSOR`] against a capable host receives any. Same
/// timeout/closed semantics as [`NativeClient::next_hidout`].
pub fn next_cursor_shape(&self, timeout: Duration) -> Result<crate::quic::CursorShape> {
match self.cursor_shape.lock().unwrap().recv_timeout(timeout) {
Ok(s) => Ok(s),
Err(RecvTimeoutError::Timeout) => Err(PunktfunkError::NoFrame),
Err(RecvTimeoutError::Disconnected) => Err(PunktfunkError::Closed),
}
}
/// Pull the next per-frame cursor state (`0xD0`): position, visibility and the M3
/// relative-mode hint, referencing a shape by serial. Latest-wins — an embedder should
/// drain the queue and apply only the newest. Same negotiation gate and timeout/closed
/// semantics as [`NativeClient::next_cursor_shape`].
pub fn next_cursor_state(&self, timeout: Duration) -> Result<crate::quic::CursorState> {
match self.cursor_state.lock().unwrap().recv_timeout(timeout) {
Ok(s) => Ok(s),
Err(RecvTimeoutError::Timeout) => Err(PunktfunkError::NoFrame),
Err(RecvTimeoutError::Disconnected) => Err(PunktfunkError::Closed),
}
}
/// Pull the next per-AU host timing (0xCF): the host's capture→sent duration for one access /// Pull the next per-AU host timing (0xCF): the host's capture→sent duration for one access
/// unit, correlated to the AU by `pts_ns`. Feeds the unified stats HUD's `host` / `network` /// unit, correlated to the AU by `pts_ns`. Feeds the unified stats HUD's `host` / `network`
/// split (`network = (received + clock_offset pts) host_us`); a stats consumer should /// split (`network = (received + clock_offset pts) host_us`); a stats consumer should
@@ -1012,7 +1080,39 @@ impl NativeClient {
/// loss like every datagram. No-op unless the host runs the DualSense gamepad backend. /// loss like every datagram. No-op unless the host runs the DualSense gamepad backend.
pub fn send_rich_input(&self, rich: RichInput) -> Result<()> { pub fn send_rich_input(&self, rich: RichInput) -> Result<()> {
self.rich_input_tx self.rich_input_tx
.send(rich) .send(rich.encode())
.map_err(|_| PunktfunkError::Closed)
}
/// Queue one stylus sample batch for delivery as a `0xCC/0x05` pen datagram
/// (design/pen-tablet-input.md). `samples` are state-full and oldest-first (a capture
/// callback's coalesced samples), at most [`crate::quic::PEN_BATCH_MAX`] per call — split
/// longer runs into consecutive calls so the stamped wrapping `seq` keeps them ordered.
/// Best-effort like every datagram: a lost batch self-heals on the next one (the samples
/// carry full state, the host diffs — see [`crate::quic::PenTracker`]).
///
/// **Heartbeat contract**: while the pen is in range or touching, repeat the last sample
/// at least every ~100 ms even when nothing changed (capture APIs are silent for a
/// stationary pen) — the host force-releases the stroke after
/// [`crate::quic::PEN_TOUCH_TIMEOUT_MS`] of silence as its dead-client failsafe.
///
/// Requires the host to have advertised [`crate::quic::HOST_CAP_PEN`]; toward an older
/// host this returns `Unsupported` (embedders keep their pen-as-touch fallback instead of
/// spraying 240 Hz datagrams the host drops unread).
pub fn send_pen(&self, samples: &[crate::quic::PenSample]) -> Result<()> {
if self.host_caps & crate::quic::HOST_CAP_PEN == 0 {
return Err(PunktfunkError::Unsupported(
"host did not advertise HOST_CAP_PEN",
));
}
if samples.is_empty() || samples.len() > crate::quic::PEN_BATCH_MAX {
return Err(PunktfunkError::InvalidArg(
"pen batch must hold 1..=PEN_BATCH_MAX samples",
));
}
let seq = self.pen_seq.fetch_add(1, Ordering::Relaxed);
self.rich_input_tx
.send(crate::quic::PenBatch::new(seq, samples).encode())
.map_err(|_| PunktfunkError::Closed) .map_err(|_| PunktfunkError::Closed)
} }

Some files were not shown because too many files have changed in this diff Show More