Compare commits

...
90 Commits
Author SHA1 Message Date
enricobuehlerandClaude Opus 4.8 1a6deeb781 feat(host/library): scan the user's non-Steam shortcuts into the game library
apple / swift (push) Successful in 1m21s
ci / web (push) Successful in 51s
ci / docs-site (push) Successful in 57s
apple / screenshots (push) Successful in 6m41s
ci / bench (push) Successful in 6m22s
ci / rust (push) Successful in 24m42s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
decky / build-publish (push) Successful in 28s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
android-screenshots / screenshots (push) Successful in 3m10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 4m41s
windows-host / package (push) Successful in 9m53s
linux-client-screenshots / screenshots (push) Successful in 7m11s
deb / build-publish (push) Successful in 11m1s
docker / deploy-docs (push) Successful in 24s
deb / build-publish-host (push) Successful in 11m55s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m31s
arch / build-publish (push) Successful in 12m57s
web-screenshots / screenshots (push) Successful in 3m12s
android / android (push) Successful in 13m46s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 2m34s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m0s
release / apple (push) Successful in 10m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m45s
flatpak / build-publish (push) Successful in 5m51s
Steam's "Add a Non-Steam Game to My Library" entries carry no `appmanifest`, so
`scan_manifests` can't see them and they never appear in the streamed library.
Read each Steam account's binary `userdata/<id>/config/shortcuts.vdf` directly
(a binary KeyValues parse) to surface the user's own custom entries, plus their
grid artwork from `userdata/<id>/config/grid`. Best-effort: an unreadable or
absent shortcuts.vdf contributes nothing, and hidden shortcuts / duplicate
appids are dropped. Verified: clippy -D warnings + compile green on Linux.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 00:45:25 +02:00
enricobuehlerandClaude Opus 4.8 8c9baf3214 fix(decky): recreate the library shortcut when its cached appId is stale
The visible "Punktfunk" shortcut's Steam appId is cached in SteamUI
localStorage, which lives in Steam's CEF profile — so it survives a
plugin uninstall/reinstall. Deleting the shortcut left the key dangling;
on the next mount ensure*Shortcut recalled the dead appId, took the reuse
branch, and ran SetShortcut* against a non-existent id (silent no-ops).
AddShortcut was never reached, so the entry never came back. The same
stale-reuse also made the hidden stream shortcut RunGame a dead gameid.

Verify the remembered appId still maps to a live shortcut via
appStore.GetAppOverviewByAppID before reusing it; a confident miss falls
through to AddShortcut. When appStore is unavailable, assume it exists so
a false negative can't spawn a duplicate library entry.

Also add a "Recreate library shortcut" recovery button to the QAM About
section (recreateShortcuts): drops any dead cached keys and re-ensures.
Covers deleting the shortcut without reinstalling, where no mount fires
the self-heal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 00:41:32 +02:00
enricobuehlerandClaude Fable 5 d83eeedde4 ci(flatpak): drop the use-vc TCP-DNS hack — the runner-side cache made it the failure
audit / bun-audit (push) Successful in 11s
ci / web (push) Successful in 46s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m21s
audit / cargo-audit (push) Successful in 2m20s
decky / build-publish (push) Successful in 19s
ci / bench (push) Successful in 6m53s
apple / screenshots (push) Successful in 6m28s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 1m0s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m27s
windows-drivers / probe-and-proto (push) Successful in 16s
windows-drivers / driver-build (push) Successful in 1m27s
deb / build-publish-host (push) Successful in 9m37s
android / android (push) Successful in 11m56s
docker / deploy-docs (push) Successful in 24s
deb / build-publish (push) Successful in 11m31s
arch / build-publish (push) Successful in 15m34s
windows-host / package (push) Successful in 11m29s
ci / rust (push) Successful in 24m30s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m37s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m28s
flatpak / build-publish (push) Failing after 8m18s
Every flatpak Tooling failure on 2026-07-22 (4 runs, incl. two AFTER the
runner fleet got its local dnsmasq cache) died in flatpak remote-add with an
instant 'Could not resolve hostname' ×10 while dnf in the SAME container
resolved fine. The one thing only this leg does is force glibc onto TCP DNS
(use-vc) — a 2026-07-11 mitigation for the embedded resolver dropping UDP
under fleet load. With job containers now resolving against the on-box
dnsmasq cache (daemon.json dns → docker0), UDP is reliable again and the TCP
path through the same chain is the flaky one under concurrency. Remove the
hack; keep the nss-resolve drop and retry.sh as the backstop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 00:27:52 +02:00
enricobuehlerandClaude Fable 5 786f6ad28f fix(pf-driver-proto): catch the layout tests up with the cursor-channel proto bump
The cursor-channel sweep grew AddReply by the cursor_excluded u32 tail (24
bytes, legacy prefix 20) and bumped PROTOCOL_VERSION to 6, but left two lib
tests behind: the AddReply byte-layout roundtrip still built the legacy
initializer (a compile error that turned the rust CI leg red on main), and
the version-coherence assertion still pinned v5 (hidden behind that compile
error). Cover the new tail bytes + the legacy-size constant, and widen the
compat-window note to v4–v6 (floor stays 3).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 00:22:34 +02:00
enricobuehlerandClaude Fable 5 1e9957d97b chore(web): clear the 2026-07-22 JS advisory wave — overrides + lock re-resolve
bun-audit went red mid-day on a fresh advisory wave (13 findings across two
snapshots of a moving DB): node-tar DoS trio+1 (critical), sharp/libvips CVEs,
fast-uri host confusion, undici ×7, brace-expansion/linkify-it/js-yaml/postcss
DoS-class, immutable, dompurify. All transitive pins in the management-console
tree; none ship in the streaming stack.

Every fix version is in-range for its consumers, so: pin overrides for the
single-major packages (tar/dompurify/linkify-it/sharp/fast-uri/immutable/
undici/postcss/js-yaml — bun ignores range-scoped override keys, so plain keys)
and a full lockfile re-resolve, which also lifts brace-expansion 2.1.1→2.1.2
in place while its 5.x line (minimatch@10) stays untouched. : No
vulnerabilities found.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 00:20:44 +02:00
enricobuehlerandClaude Opus 4.8 92f38ec3dd chore(cleanup): drop TEMP cursor probes + clear KWin-leg clippy debt
audit / bun-audit (push) Failing after 14s
ci / web (push) Successful in 52s
windows-drivers / probe-and-proto (push) Successful in 54s
ci / docs-site (push) Successful in 1m6s
apple / swift (push) Successful in 1m21s
audit / cargo-audit (push) Successful in 2m49s
windows-drivers / driver-build (push) Successful in 2m0s
decky / build-publish (push) Successful in 22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m47s
ci / rust (push) Failing after 10m23s
release / apple (push) Successful in 9m24s
deb / build-publish-host (push) Successful in 9m51s
apple / swift (pull_request) Successful in 1m21s
apple / screenshots (pull_request) Skipped
android / android (push) Successful in 12m25s
flatpak / build-publish (push) Failing after 8m8s
deb / build-publish (push) Successful in 12m16s
ci / web (pull_request) Successful in 50s
ci / docs-site (pull_request) Successful in 51s
docker / deploy-docs (push) Successful in 26s
arch / build-publish (push) Successful in 14m34s
windows-host / package (push) Successful in 15m5s
apple / screenshots (push) Successful in 6m33s
ci / rust (pull_request) Failing after 6m56s
ci / bench (pull_request) Successful in 6m5s
android / android (pull_request) Successful in 9m56s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m14s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m42s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m13s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m34s
windows / build (x86_64-pc-windows-msvc) (push) Canceled after 4m47s
windows-drivers / probe-and-proto (pull_request) Successful in 15s
windows-drivers / driver-build (pull_request) Successful in 1m30s
windows / build (aarch64-pc-windows-msvc) (pull_request) Successful in 1m3s
windows / build (x86_64-pc-windows-msvc) (pull_request) Successful in 1m54s
The KWin/Phase-A/B commits were built but never clippy-checked (that distrobox
had no clippy component), so they left TEMP on-glass probes and lint debt in the
tree. With clippy now runnable (fedora rust 1.96.1 = CI parity):

- drop the `fec424ee`/`8cff30d5` TEMP probes: the `update_cursor_meta` SPA_META
  diagnostic logs (also un-detaches the `// SAFETY:` comment from its `unsafe`
  block → fixes `undocumented_unsafe_blocks`) and the KWin composite-arm probe in
  the encode loop.
- `#[allow(clippy::too_many_arguments)]` on `spawn_pipewire` (8 params since the
  KWin leg added `expect_exact_dims`; mirrors `from_virtual_output`).

clippy `-p pf-capture -p pf-vdisplay -p punktfunk-host --locked --features
nvenc,vulkan-encode -- -D warnings` is now green. (--all-targets additionally
trips a pre-existing env mismatch: the fedora libspa binding lacks
`SPA_VIDEO_TRANSFER_SMPTE2084`, referenced only by a `#[cfg(test)]` guard-test —
not a code issue; CI's pinned pipewire has it.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:54:56 +02:00
enricobuehlerandClaude Fable 5 7058647264 fix(vdisplay/capture): channel-less sessions composite the pointer on a sticky-declared target
A declared IddCx hardware cursor is IRREVOCABLE for its OS target's life
(§8.6), and the sticky exclusion survives monitor REMOVE→ADD because each
client gets a STABLE target id — so once any desktop-mode session declared,
every later pure-capture session on that target streamed a cursor-less
desktop: DWM excluded the pointer, no channel forwarded it, no blend drew it
(the exact no-regression gap §8.6's per-session cap gate cannot see).

Driver: track every successful SetupHardwareCursor per target
(DECLARED_TARGETS — scoped to the WUDFHost's life, exactly the sticky
state's scope) and report it in a new AddReply::cursor_excluded tail field
(dual-size discipline, both skews degrade cleanly; no proto bump).

Host: the flag rides AddedMonitor → Monitor → WinCaptureTarget; a session
WITHOUT the cursor channel on a flagged target forces composite mode in the
IDD-push capturer — GDI poller + blend for the session's life, pinned on
(set_cursor_forward cannot clear it: with no client drawing, un-compositing
would erase the pointer entirely).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Opus 4.8 d5ae8dcc3e feat(capture): gamescope cursor via XFixes shape + QueryPointer position (Phase C)
gamescope excludes its pointer from the PipeWire node it feeds us and can't embed
one either (`set_hw_cursor` is inert), so every gamescope stream was cursorless.
Read the pointer from gamescope's nested Xwayland instead — XFixesGetCursorImage
for shape/hotspot/visibility, core QueryPointer for position — and publish a
CursorOverlay into the capturer's existing `cursor_live` slot, so the encoder
blend composites it into the video exactly like the SPA_META_Cursor path.

- pf-capture/src/linux/xfixes_cursor.rs (new): the XFixes reader. Connects to
  EVERY nested Xwayland (Gaming Mode runs one per --xwayland-count) and follows
  the focused one each tick (the display whose pointer moves), reading that
  display's own cursor shape. Un-premultiplies ARGB -> straight RGBA. Drop stops
  the thread.
- Capturer::attach_gamescope_cursor + the PortalCapturer override spawn it into
  the same `cursor_live` slot; pf_vdisplay::gamescope_xwayland_cursor_targets
  discovers the (DISPLAY, XAUTHORITY) pairs via the GAMESCOPE_WAYLAND_DISPLAY scan.
- host: SessionPlan.gamescope_cursor (set from the compositor at both resolve
  sites AND on a mid-stream Desktop->Gaming switch); the blend gate now builds the
  encoder blend for gamescope; a sibling composite arm attaches capturer.cursor()
  per tick for capture-mode clients (no channel needed). native NV12 is disabled
  for these sessions — that encode path can't blend the cursor (it assumes
  gamescope embeds the pointer), so we capture RGB and route to the proven CUDA
  VkSlotBlend / compute-CSC blend.
- the pipewire thread no longer clobbers `cursor_live` with None on a buffer that
  carries no SPA_META_Cursor (gamescope) — that raced the XFixes writer and
  strobed the composited pointer on/off.

On-glass (home-bazzite-2, RTX 5070 Ti, Gaming Mode): cursor visible + steady in
Big Picture and CS2 menus, follows focus between the two Xwaylands, hidden in-game.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 925130d1f9 feat(vdisplay): compositor-embedded pointer for sessions without the cursor channel (Phase B)
Since the cursor-channel work, every Linux virtual output was created in
metadata pointer mode — making ALL sessions depend on host-side cursor
compositing, including the ones that can never use the channel (Moonlight/
GameStream, legacy clients, capture-mode starts). Those sessions paid the
blend bring-up per session and, whenever a visible cursor was composited,
the loss of NVENC's stream-ordered submit — for a strictly worse cursor
than the compositor's own.

Mirror the Windows no-regression gate: the already-wired per-session
set_hw_cursor(cursor_forward) now drives the Linux backends too. A
cursor-channel session gets metadata (shapes forwarded, composite flip
blends host-side — today's validated path, unchanged); every other session
gets the pointer compositor-EMBEDDED at creation (KWin zkde pointer=2,
Mutter cursor-mode=1, wlroots/hyprland portal CursorMode::Embedded — their
pre-channel default; the portal pair also gains the Metadata arm for
channel sessions, wired but untested on-glass). SessionPlan.cursor_blend
narrows to cursor-forward sessions and rides into the direct-SDK NVENC
open, which skips the Vulkan slot-blend bring-up entirely when off —
embedded sessions ring on plain CUDA surfaces and pay zero cursor cost,
per-session or per-frame.

The keep-alive registry's reuse key grows the created pointer mode (new
VirtualDisplay::hw_cursor getter): a kept embedded display has no cursor
metadata for a channel session to forward, and a kept metadata display
would leave a channel-less session with no pointer in its frames.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 d2c46eaf3c feat(encode/nvenc): SPIR-V cursor blend over Vulkan-allocated input slots — retire the PTX kernels
A vendored PTX blob is JIT'd against the driver's ISA ceiling, so the
cursor-blend module silently dies on drivers older than the generating
toolkit (CUDA_ERROR_UNSUPPORTED_PTX_VERSION/INVALID_PTX, 222/218 — the
KWin leg's invisible composite cursor on driver 595/CUDA 13.2 vs a
CUDA 13.3 blob). SPIR-V has no such coupling, and the in-tree precedent
already exists twice (vulkan_video's CSC blend, VkBridge's exportable
OPAQUE_FD → cuImportExternalMemory bridge).

New pf_zerocopy::vkslot::VkSlotBlend: the direct-SDK NVENC encoder now
allocates its input ring as exportable Vulkan buffers CUDA-imports (same
contiguous InputSurface layouts, pitch = row bytes rounded to 256), and
the cursor composite is a compute dispatch over the cursor's rectangle
(cursor_blend.comp, vendored .spv; spec-constant selects ARGB/NV12/YUV444;
BT.709 limited, matching the retired .cu). The surface SSBO is uint[] with
every invocation owning whole words — no 8-bit-storage device dependency.
Cursor-bearing frames force the existing CPU-synced submit path so the
CUDA copy → Vulkan dispatch (fence-waited) → NVENC encode ordering is
CPU-established; cursorless frames keep the stream-ordered fast path
untouched. Any bring-up/alloc/registration failure falls back wholesale
to plain pitched CUDA surfaces (never a mixed or short ring): sessions
always encode, composite mode just loses the cursor, warned once.

cursor_blend.cu / cursor_blend.ptx and the CursorBlend PTX loader are
deleted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 33121ece4d chore(kwin-composite): TEMP blend-path probes — DROP BEFORE MERGE
Rate-limited journal logging bracketing the silent segment of the Linux
composite path: what overlay (if any) the encode loop's composite arm
hands the encoder, and whether the NVENC cursor-blend kernel launches and
with what geometry. On-glass (KWin leg) the blend module loads yet the
composite cursor stays invisible — these two probes attribute it to
no-overlay / stripped-en-route / kernel-draws-nothing. The module-loaded
INFO line is permanent (success must be as attributable as failure);
everything else in this commit is temporary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 af87549052 fix(linux/vdisplay): KWin virtual outputs stream above 60 Hz via sacrificial-birth renegotiation
KWin's ScreenCastStream builds its PipeWire format offer — including the
maxFramerate cap it actively throttles delivery to — ONCE at stream
creation, when the virtual output sits at its hardcoded birth 60 Hz. A
kscreen custom-mode change afterwards updates the OUTPUT (readback says
240) but never the offer, so every consumer negotiates max=60 and the
stream delivers 60 (pw-dump-verified on KWin 6.6.4). The only path that
rebuilds the offer is the stream's own resize handling: a source SIZE
change while recording re-runs buildFormats — picking up the output's
current refresh — and renegotiates the live stream.

So above 60 Hz, birth the output at a sacrificial height (+16), then
install + select the real WxH@hz custom mode: the first frame recorded
after the consumer connects triggers KWin's resize path, which renegotiates
the live stream to WxH@hz. The capturer holds (requeues) buffers until the
negotiated size matches — new expect_exact_dims plumbing VirtualOutput →
registry (fresh creates only) → open_virtual_output → a self-disarming
gate in the process callback — bounded by a 3 s deadline that accepts the
producer's dims rather than wedging the session into the first-frame
retry loop. set_custom_refresh reads back the full active mode; a size
reject (pre-6.6 KWin) recreates plain at the real size @60.

Every kscreen operation now addresses the output by its NUMERIC id,
resolved by matching the managed name-prefix AND the just-created birth
size (newest id wins): a mode-switch supersede reuses the per-slot output
NAME (deliberately, for KWin's per-name config persistence) while the
superseded sibling is still alive, and name addressing hit the FIRST
match = the OLD output — on-glass that resized the live session's display
out from under it (wrong-res/black), read the old output back as 'mode
applied', and set the old output primary while the replacement starved at
its birth mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 09113c9899 fix(encode/nvenc): cursor-blend PTX must carry a driver-portable ISA version
The vendored cursor_blend.ptx was regenerated with the CUDA 13.3 toolkit,
which stamps '.version 9.3' — and a driver whose JIT predates that ISA
refuses the whole module with CUDA_ERROR_UNSUPPORTED_PTX_VERSION (222),
silently killing host-side cursor compositing (on-glass: composite-mode
cursor invisible on driver 595.58, KWin leg). The kernels use only baseline
arithmetic, so hand-lower the version directive to 8.0 (CUDA 12.0), which
every supported Turing+ driver JITs. Both files now warn that an nvcc
regeneration re-stamps the version and must be re-lowered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 4436c7fb61 fix(host/stream): composite mode re-blends the LIVE cursor on repeat ticks
The frame-attached overlay is the pointer position at the last damage
frame; repeats re-encoding a static desktop froze the blended cursor
between redraws — on-glass the composite-mode cursor stuttered while
window drags (constant damage) were smooth. Refresh the repeat's overlay
from Capturer::cursor each tick so pointer-only motion re-blends at tick
rate — the bandwidth the pre-channel embedded mode already paid. Linux
only: the Windows capturer composites internally and its frames must
never carry an overlay a blend path would double-draw.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 088004cac4 feat(linux/capture): serve the LIVE cursor overlay through Capturer::cursor
The forwarder read only frame.cursor — the overlay attached to the LAST
EMITTED frame. Mutter's pointer-only buffers update the capture-side cursor
state but are skipped as frames, so between damage frames the forwarder
re-sent a stale snapshot: on-glass the client cursor appeared exactly when
clicking or crossing hover targets (damage) and froze/vanished otherwise —
the same pointer-only-motion gap the Windows IddCx channel fills, which is
why the encode tick already prefers the capturer's LIVE cursor. Publish the
overlay from every dequeued buffer (frames or not) into a shared slot and
implement the Capturer::cursor hook on PortalCapturer.

Host-composite mode on Linux still updates the blended pointer only on
damage frames (no re-encode of a static desktop on cursor motion) — the
regen-on-cursor-change the Windows path has is a follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 63d83217ef fix(linux/capture): a stale id-0 cursor meta is 'no information', not 'hidden'
Mutter only rewrites a buffer's SPA_META_Cursor region when the cursor
changed; recycled buffers between damage frames carry a stale id-0 meta.
Treating id 0 as a hidden pointer flickered the client cursor off between
hovers on-glass — per the SPA contract id 0 is 'invalid/no cursor info',
so keep the last-known state (OBS's consumer does the same). A genuinely
hidden pointer stops producing updates; the M3 relative-mode hint keeps
its Windows CURSOR_SUPPRESSED source.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 9db4841ce4 fix(linux/capture): cursor meta size range must cover Mutter's fixed 384x384 offer
Mutter offers SPA_META_Cursor with a FIXED size pod —
SPA_POD_Int(CURSOR_META_SIZE(384, 384)) in meta-screen-cast-stream-src.c —
while our request capped the range at meta_size(256, 256). The intersection
is empty, so the Meta param silently failed to negotiate and no buffer ever
carried the meta region: the entire Linux cursor pipeline (forward AND
blend) was blind on GNOME, proven by an on-glass probe counting 32k+
buffers with zero metas. KWin's offer fits inside 256², which is why the
same consumer code passed there. Raise the max to 1024² headroom (the
negotiated allocation follows the producer's value, not our max) and align
the bitmap parse guard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 151adc4bcc chore(m2b): TEMP cursor-meta probe — DROP BEFORE MERGE
Rate-limited journal logging of SPA_META_Cursor arrival/absence, reported
id/position/bitmap offset, and bitmap acceptance — the Linux cursor
pipeline is otherwise blind end-to-end during Mutter RecordVirtual
metadata bring-up on .21.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 0619af391e fix(linux/inject): map absolute pointer/touch into the streamed output's region
The libei absolute device advertises one region per logical monitor, and
both MouseMoveAbs and touch mapped into regions().first() — whichever
output the compositor announced first. Next to a physical monitor (on-glass:
GNOME with a dummy HDMI beside the virtual primary) that put the pointer and
every click on the WRONG output: the seat cursor never entered the streamed
monitor, so neither embedded nor metadata cursor capture could ever see it,
and the cursor channel had nothing to forward. Pick the region whose logical
size matches the streamed mode (the wire flags already carry it); fall back
to first() for the single-monitor case. Region geometry now rides the
device-RESUMED log line for diagnosability; matching the screencast
mapping_id instead of the size is the follow-up for same-sized-monitor
ambiguity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:47:53 +02:00
enricobuehlerandClaude Fable 5 afe120bf34 fix(linux/vdisplay): kwin virtual output ships the cursor as metadata too
Same trap as the Mutter backend: zkde_screencast pointer mode 2 (embedded)
never delivers SPA_META_Cursor, starving both the cursor channel and the
encoder blend. Mode 4 (metadata) feeds both.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 c29186bd1a fix(host+vdisplay): a mode-switch replacement display inherits the group topology
The resize rebuild creates the new virtual display BEFORE retiring the old
one (create-before-drop), so the registry saw a live same-backend sibling —
its own dying predecessor — and told the backend it was not first in group.
Mutter then skipped the Primary/Exclusive apply ("joining an existing
display group — extending") and the retiring owner took the topology with
it: every resize silently demoted the virtual output to an extended,
shell-less desktop. Thread the superseded pool gen through acquire so the
replacement establishes topology, and stop counting kept (Lingering/Pinned)
entries as demoting siblings — no session owns them, so there is no live
desktop to clobber.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 315ffaf144 fix(linux/vdisplay): mutter virtual monitor ships the cursor as SPA_META_Cursor
RecordVirtual was created with cursor-mode EMBEDDED, which never delivers
cursor metadata — the cursor channel had nothing to forward (client-drawn
cursor invisible) and the always-built encoder blend had nothing to
composite (host-drawn cursor invisible too). Metadata mode feeds both:
non-forwarding sessions get the blended pointer (same pixels Mutter would
have embedded), forwarding sessions strip the overlay and send shape/state.
The capturer already negotiates the meta on every stream (meta_param).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 c63ac48c9c fix(windows/capture): actually QUERY the HDR SDR-white scale — the refresh block was silently dropped
faad4b57 shipped the field, shader scale, and plumbing but the
scripted edit inserting the sdr_white_level_scale() refresh in the
scratch-rebuild arm matched nothing and silently no-op'd (no assert) —
sdr_white_scale stayed at its 1.0 init, i.e. 80 nits, i.e. exactly the
old dark cursor. Now queried on every scratch rebuild, with an info
log of queried/applied so an HDR session shows what it uses.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 cecc059830 fix(windows/capture): 250 Hz cursor poll + HDR white-scale observability
- The polled position is also the composite-blend position: at 16 ms a
  240 fps session reused a stale position for ~4 consecutive frames and
  the composited pointer visibly stuttered against the video. 4 ms
  out-paces every session rate; a tick is one GetCursorInfo syscall.
- Log the queried DISPLAYCONFIG_SDR_WHITE_LEVEL scale on each blend-
  scratch rebuild so an HDR session shows what is actually applied
  (the virtual display's own SDR-brightness setting is the suspect
  when the cursor still reads dark).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 d31d41b6b8 fix(windows/capture): match the HDR-composited cursor to the target's SDR white level
sRGB→linear alone lands cursor-white at 80 nits (scRGB 1.0) while DWM
composes the surrounding SDR desktop at the user's SDR-brightness
setting (~200+ nits by default) — the cursor read visibly dark on HDR.
Scale by DISPLAYCONFIG_SDR_WHITE_LEVEL (new
win_display::sdr_white_level_scale, queried on blend-scratch rebuilds —
the CCD query stays off the per-frame path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 6e37da8b4e fix(windows/capture): clippy unnecessary_lazy_evaluations in scratch build
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 0af280a793 feat(windows/capture): host-side cursor compositing for the capture model — the deterministic fix
Root cause, proven on-glass across five driver builds: a declared IddCx
hardware cursor is IRREVOCABLE. There is no un-declare DDI, the
empty-caps re-setup is rejected INVALID_PARAMETER, and after a
successful same-mode re-commit with the driver's re-declare provably
suppressed (sticky per-target flag, zero re-setups logged) DWM still
never composites the software cursor back into the monitor's frames.
Every DWM-based composite flip is therefore unfixable.

The composite (capture) model is now implemented in OUR pipeline:
- the driver keeps its hardware cursor declared for the session's whole
  life (the state that works) — frames stay pointer-free always;
- in composite mode try_consume routes the conversion through a blend
  scratch: slot copy + one alpha-blended quad of the GDI poller's shape
  at its polled position (CursorBlendPass — fullscreen-triangle VS with
  viewport placement, straight-alpha PS, sRGB→linear for FP16/HDR
  rings), covering all four convert paths (NV12, 4:4:4 copy, P010,
  PyroWave) GPU-side under the slot's keyed mutex;
- pointer-only motion produces no driver publish (declared hw cursor),
  so try_consume REGENERATES from the last slot whenever the polled
  cursor state changes — the cursor moves on a static desktop at tick
  rate, without faking freshness for the stall/death watchdogs;
- set_cursor_forward becomes purely host-internal state; the
  SET_CURSOR_FORWARD IOCTL machinery is no longer used by the host
  (driver keeps it, dormant) and its sender plumbing is removed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 89cfef429e fix(windows/vdisplay): cursor-render state sticky per TARGET across monitor generations
On-glass: 're-setup on swap-chain assign -> 0x0' fired AFTER 'enable=0
stored' — duplicate/successor monitor entries (re-arrival churn) kept
their default-true flag and re-declared the hardware cursor right after
the composite flip's re-commit, undoing it.

The desired state now lives in a per-target static
(CURSOR_FORWARD_DESIRED): every arrival inherits it (no generation can
resurrect a declare the session turned off), and the flip stamps EVERY
live entry matching the target, declaring against whichever has the
live worker.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 b50e4767de fix(windows/cursor-flip): flag + forced same-mode re-commit — the working un-declare
The empty-caps un-declare is REJECTED by IddCx (STATUS_INVALID_PARAMETER,
on-glass driver 9.9.0722.1407) — there is no un-declare DDI. The
composite flip now works with the only lever the OS gives us: every
mode COMMIT reverts the path to the software cursor, and the driver
skips its per-commit re-declare while cursor_forward_on is off. So:

- driver: disable stores the flag only (no DDI call); enable still
  declares immediately against the live worker's event.
- host capturer: after a successful disable flip, force a same-mode
  re-commit (win_display::force_mode_reenumeration) — DWM composites
  the pointer from the very next commit; the swap-chain flap is the
  class the driver's preserved-publisher machinery already rides out.
- state now survives re-arrivals end-to-end: the capturer caches the
  APPLIED state (cleared on every channel (re)delivery, which
  re-created driver entries need — their flag defaults to declared),
  and the stream loop re-applies every tick instead of edge-gating
  (steady-state cost: one Option compare).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 79d30fde64 fix(windows/capture): unsafe block for the extracted dup_into_public call
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 6a8df2ba97 fix(windows/cursor-flip): survive driver-side monitor re-arrival — flag-based flip + channel re-delivery
On-glass (.173, match-window session): every window-size convergence
re-creates the driver-side monitor (re-arrival resize / sibling-session
slot churn), destroying the cursor worker the channel was delivered to —
the flip then died NOT_FOUND (no entry with target_id ∧ worker) and the
declared state was lost entirely.

Driver: set_cursor_forward becomes STATE, not an edge on one monitor
generation — find by target_id ∧ hw_cursor, store cursor_forward_on
even without a live worker (it steers the next delivery/re-setup);
declare/un-declare only when a worker exists. Channel delivery honors
the flag: setup_and_spawn(declare=false) adopts + spawns WITHOUT
declaring (composite mode), so a later enable-flip declares against the
worker's event.

Host: retain the SET_CURSOR_CHANNEL sender; extract
deliver_cursor_channel and RE-deliver the surviving section on every
ring recreate and — with one retry — when a flip IOCTL fails
(the re-arrived entry gets a fresh worker, declared per its flag).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 030a779391 feat(windows/vdisplay): the driver leg of the mid-stream cursor-render flip
Belongs to 493f4fae (which shipped only the crates/ side — this
packaging/ half was left unstaged, so the deployed driver silently
lacked the IOCTL and every flip died NOT_FOUND at dispatch):
IOCTL_SET_CURSOR_FORWARD dispatch + monitor::set_cursor_forward
(cursor_forward_on flag, resetup gate) + cursor_worker::
unsetup_hardware_cursor (empty-caps un-declare experiment).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 d86073dc6a feat(apple/cursor): send the mid-stream cursor-render flip — ABI v12
PunktfunkConnection.setCursorRender(clientDraws:) over the new
punktfunk_connection_set_cursor_render export, driven by one
edge-detected reconciler in StreamView (clientDraws = captured &&
desktopMouse) called from every transition — the ⌃⌥⇧M chord,
engage/release, and session start. Capture model and released now get
the host-composited pointer back in the video (full fidelity); the
desktop model keeps the local NSCursor path. xcframework rebuilt v12.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 92761813a9 feat(core+host+driver+client): mid-stream cursor-render flip — host composites in the capture model
The cursor channel excluded the host pointer from the video for the
whole session, but the client only draws it under the DESKTOP mouse
model — flipping ⌃⌥⇧M to capture mid-stream left the session
pointer-blind (user-found). The fix makes render ownership LIVE state:

- wire: CursorRenderMode 0x51 (client→host, control stream) —
  client_draws: true = desktop model (host excludes + forwards),
  false = capture model / released (host composites, full fidelity —
  DWM on Windows incl. real XOR inversion, encoder blend on Linux).
  Sessions start client_draws=true (the pre-message behavior).
- host: control task stores the flag; the encode loop edge-detects it —
  forwarding + frame.cursor strip while the client draws, quiet
  forwarder + overlay-into-blend while composited. SessionPlan now
  grants blend CAPABILITY wherever the capture has a pointer (dropping
  the !cursor_forward gate) so the composite side needs no encoder
  rebuild; per-tick frame.cursor decides what is drawn.
- Windows: Capturer::set_cursor_forward → retained
  IOCTL_SET_CURSOR_FORWARD sender (proto v6) → driver (un)declares the
  IddCx hardware cursor on the LIVE monitor. Un-declare re-issues the
  setup with EMPTY caps (candidate mechanism — the DDI has no
  documented un-setup); resetup-on-commit is skipped while off, so a
  mode commit's software-cursor default is the fallback path. Failure
  against a pre-v6 driver logs and keeps declared-at-ADD behavior.
- SDL client: one edge-detected reconciler at the cursor pump site —
  desktop-active = client draws; capture model or released = host
  composites. Covers the chord, the M3 auto-flip, and engage/release.
- C ABI v12: punktfunk_connection_set_cursor_render (additive; wire
  version unchanged — pre-§8 hosts ignore the unknown message type).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 0420be9fa5 fix(apple/cursor): reset per-session cursor-channel state in stop()
cursorChannelActive/hostCursors/cursorState stayed latched across
sessions — a next session against a host WITHOUT the cursor cap would
wear the previous session's stale shapes via resetCursorRects.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 f2082f8b57 fix(windows/capture): drop redundant unsafe at the poller-spawn CCD call (inside open_on's unsafe region)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 9162e7b451 fix(windows/vdisplay): drop now-unused INFINITE import (poll-loop wait)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 e59634a578 feat(windows/capture): GDI cursor-shape poller — masked/monochrome cursors for the cursor channel
The M2c redesign (design/remote-desktop-sweep.md §8): the IddCx hardware-
cursor query is alpha-only by design — IDDCX_CURSOR_SHAPE_TYPE has no
monochrome value, the OS pre-converts monochrome to masked-color, and
masked-color delivery is dead code on modern builds (proven on-glass at
every ColorXorCursorSupport level). The driver keeps its hardware cursor
declared (XOR FULL) purely so DWM excludes every cursor type from the
IDD frame; the SHAPE now comes from a GDI poller in the capture host —
which runs as SYSTEM inside the interactive session, so no helper
process is needed.

CursorPoller (idd_push/cursor_poll.rs, the RustDesk/WebRTC/OBS pattern):
~60 Hz GetCursorInfo on a dedicated thread; rasterise via
CopyIcon→GetIconInfo→GetDIBits only when the HCURSOR value changes;
monochrome via the WebRTC truth table with invert pixels as opaque black
plus a white outline; AND-mask alpha for alpha-less color cursors;
CURSOR_SUPPRESSED = hidden; per-output visibility via the target's
desktop rect; per-monitor-V2 DPI awareness thread-scoped; input-desktop
reattach on failure + 2 s cadence (GetCursorInfo on a stale desktop
succeeds with stale data). Spawned iff the cursor channel was delivered;
a live poller is the sole overlay source (no serial-namespace mixing) —
the driver shm read remains as fallback if the poller dies.

Not chosen: DXGI Desktop Duplication GetFramePointerShape —
PointerPosition.Visible goes stale under injected-only input on current
Win11 (Sunshine #5293, exactly our topology), it burns one of the
session's four duplication slots, and IDD-output metadata has
conflicting field reports.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Opus 4.8 d2cc770938 fix(windows/vdisplay): QueryHardwareCursor3 + per-mode-commit cursor re-setup — M2c findings
Durable results of the M2c on-glass bring-up (.173, 2026-07-22):

- The base IddCxMonitorQueryHardwareCursor DDI slot is stubbed to
  STATUS_NOT_SUPPORTED on WDK 26100 — add the ...QueryHardwareCursor3
  wrapper (wdk-iddcx) and drain it instead; v3 X/Y are only meaningful
  when PositionValid, so a position-invalid tick keeps the prior position.
- Hardware-cursor setup is per-mode-commit: the OS silently reverts to a
  software cursor on every mode commit, so re-issue the setup on each
  swap-chain assignment (monitor::resetup_cursor, called from
  assign_swap_chain) or the query fails NOT_SUPPORTED forever.
- One caps definition for initial setup and re-setup, resting at XOR
  FULL: the query delivers ONLY alpha shapes in every configuration
  (all three ColorXorCursorSupport levels, event-driven and 30 Hz
  polled) — masked/monochrome cursors never arrive. FULL keeps the
  frame cursor-free for ALL cursor types; the full-fidelity shape comes
  from the session-side cursor source in the host
  (design/remote-desktop-sweep.md §8).
- Poll the query at ~30 Hz instead of pure event-wait: masked cursors
  never fire the data event, and the seqlock's position/visibility
  should stay fresh regardless.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 2bbcbc81c6 fix(apple/cursor): disable M3 visibility-based auto-flip — it broke desktop drags
On-glass (Mac ↔ Windows M2c host): the host cursor's VISIBILITY is not a
usable 'a game grabbed the mouse' signal — Windows hides the pointer for
ordinary desktop activity (clicking, typing). The per-frame forwarder
turned those transients into relative_hint=true, so the client flipped
desktop→capture→desktop, which (a) warped the cursor to view-centre and
(b) flushed held buttons via releaseAll — a spurious mouse-up ~200 ms
into every press that broke window dragging/resizing.

Disable the auto-flip; the mouse model stays user-driven (⌃⌥⇧M). The
hint still rides the wire, unused, for a future M3 that keys off a REAL
pointer-lock signal (host-side ClipCursor/raw-input detection) instead
of visibility.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 a99ef7f940 fix(windows): hw_cursor_capable must not assume an initialised display manager
The FIRST session's Welcome runs before any backend construction
(vdisplay::open happens at display prep, after the Welcome), so the
capability probe's vdm() expect panicked the very first handshake of a
fresh service — the client saw a dead connect and auto-wake kicked in.
init() is idempotent and the driver facade is a unit struct, so the
probe now initialises the manager itself. Found on-glass (.173, first
Mac-client connect), fix deployed there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 ec8ca9a535 feat(apple): cursor channel on the Mac client — ABI v11 + NSCursor rendering
The Mac client now exercises the full cursor channel against capable
hosts: desktop-mode sessions advertise CLIENT_CAP_CURSOR, the host
stops compositing the pointer, and StreamView draws the forwarded
shapes as the real NSCursor — plus the M3 host-driven model flip.

- ABI v11: punktfunk_connect_ex9 (adds client_caps; ex7/ex8 pass 0 —
  Android untouched), PunktfunkCursorShape/State + the two
  next_cursor_* poll fns (audio-style borrow contract), and
  PUNKTFUNK_CLIENT_CAP_CURSOR. Fixed in passing: the first insertion
  split next_host_timing's cfg/no_mangle attributes off the fn, which
  silently dropped the symbol from the header AND dylib — caught by
  the Swift build, reattached with its docs.
- PunktfunkConnection: clientCaps connect param, hostSupportsCursor,
  CursorShapeEvent/CursorStateEvent + nextCursorShape/nextCursorState
  (one cursor thread drains both planes; cursorLock joins the close()
  ladder).
- SessionModel: desktop-mode sessions (DefaultsKey.mouseMode) connect
  with the cursor cap on macOS.
- StreamView: shape cache (serial → NSCursor, straight-alpha RGBA via
  CGImage), resetCursorRects wears the HOST shape while the desktop
  model is engaged (hidden host pointer ⇒ invisible), latest-wins
  state pump, and the M3 auto-flip — edge-triggered on relative_hint,
  ⌃⌥⇧M sets an override latch cleared by the next host edge, leaving
  relative warps the pointer to the host position via the inverse
  letterbox mapping (CGWarpMouseCursorPosition, CursorCapture's
  coordinate convention).

Verified: swift build + full test suites green (xcframework rebuilt at
ABI v11, signed); core 218 tests + clippy -D warnings on Linux (.21).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 3a34440a6b feat(windows): IddCx hardware-cursor channel — remote-desktop sweep M2c
Brings the cursor channel to Windows hosts. The pf-vdisplay driver
declares an IddCx hardware cursor for sessions that negotiated
cursor-forward — DWM then EXCLUDES the pointer from the IDD frame and
delivers shape/position out-of-band, into the same CursorOverlay →
forwarder → wire → client pipeline the Linux portal path uses.

- pf-driver-proto v5 (additive, host floor stays 3): AddRequest's spare
  tail becomes hw_cursor (same size/offsets); IOCTL_SET_CURSOR_CHANNEL
  delivers a host-created CursorShm section (64-byte seqlock header +
  256² shape buffer, layout pinned + tested). No event crosses the
  boundary — the host polls at encode-tick pace.
- driver: wdk-iddcx grows the two cursor DDI wrappers; a per-monitor
  cursor worker (event wait → QueryHardwareCursor → seqlock publish)
  starts only when BOTH the ADD asked and the channel arrived, so a
  failed delivery leaves DWM compositing as today. Shape bytes ship raw
  (BGRA/masked + pitch); the host converts.
- host: the section rides the existing sealed-channel broker (least-
  privilege dup, remote reap on failure); IddPushCapturer::cursor()
  seqlock-reads → CursorOverlay (BGRA→RGBA, masked-color approximation,
  desktop→frame origin shift, per-shape conversion cache). New
  Capturer::cursor() trait hook — the encode loop prefers it over the
  frame-attached overlay because hardware-cursor moves produce NO new
  frame on a static desktop. hw_cursor survives the re-arrival resize
  (carried on the manager's Monitor).
- negotiation: cursor_forward grows the Windows arm (client cap ∧
  driver proto ≥ 5, probed once via the control device); SessionPlan
  carries cursor_forward → OutputFormat.hw_cursor.
- drive-by: wdk-probe's two pre-existing same-type casts (clippy) and
  pf-vdisplay's stale spike-test refs (crate::win_display moved to
  pf-win-display; tracing-subscriber was never a dep) repaired.

Verified: proto tests on Mac AND MSVC (15/15 incl. the CursorShm
layout pin); clippy -D warnings for proto/frame/capture/vdisplay/host
on Linux (.21) and native Windows (.173); the DRIVER workspace clippy
-D warnings green against the real WDK 10.0.26100 bindgen (DDI names,
enum variants and IDARG layouts all bind). On-box driver deploy +
on-glass validation follow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 c3cbffe662 feat(host+client): host-driven mouse-model flip — remote-desktop sweep M3
The full Parsec model: launching a game from a desktop session flips the
client into captured relative automatically, and back — no chords.

- Capture overlay now distinguishes 'hidden' from 'no cursor yet':
  CursorOverlay grows a visible flag, overlay() returns Some whenever a
  bitmap is known (the encode loop strips invisible overlays after
  forwarding so no blend path ever draws one; the CPU composite guarded
  on visibility already).
- Host forwarder maps it onto the reserved wire bit: visible ⇒ VISIBLE,
  hidden-but-known ⇒ RELATIVE_HINT (an app grabbed/hid the pointer),
  never any hint before the first bitmap — a cold start can't flip a
  desktop session into capture.
- Presenter: edge-triggered auto-flip on hint changes via the new
  Capture::set_desktop (chord semantics untouched); a manual ⌃⌥⇧M sets
  an override latch that holds until the HOST's intent next changes, so
  the hint never fights the user. Leaving relative warps the local
  cursor to the host's last pointer position through the new
  content_to_window inverse-letterbox mapping (unit-tested against
  finger_to_content) — the hand-back is seamless.

Verified on .21: fmt + clippy -D warnings + tests (presenter 20 incl.
the inverse-mapping roundtrip, host 245).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 4a01bc4463 feat(core+host+client): cursor channel — remote-desktop sweep M2a+M2b
The host cursor stops riding the video and becomes a real OS cursor on
the client (the Parsec/RDP model): pointer feel no longer pays the
capture→encode→network→decode→present round trip.

Wire (M2a):
- Hello grows a client_caps trailing byte (CLIENT_CAP_CURSOR) after the
  fixed display_hdr block — presence disambiguated by remaining length,
  which caps the post-HDR tail at 27 bytes (documented); Welcome answers
  HOST_CAP_CURSOR (capable-and-asked, the 444/clipboard precedent).
- CursorShape (0x50, control stream): serial + dims + hotspot + straight
  RGBA, ≤120px/side so the u16 frame always fits (128² would overshoot);
  client caches by serial — re-showing a known shape costs 14 bytes, not
  a bitmap (RDP pointer-cache for free).
- CursorState (0xD0 datagram): serial + visible/relative_hint flags +
  position, sent once per encode-loop tick — latest-wins, self-healing
  under loss, no refresh timer. relative_hint is reserved for M3.
- Client core: two new planes (control-task + datagram-task arms) →
  next_cursor_shape/next_cursor_state; connect() grows client_caps
  (C ABI passes 0 until the v11 cursor poll fns exist).

Host (M2b, Linux portal only):
- handshake::cursor_forward is THE predicate (client asked ∧ Linux ∧
  compositor ≠ gamescope) — Welcome bit and session wiring both read it.
- SessionPlan.cursor_blend goes false for a forwarding session; the
  encode loop ticks a CursorForwarder every iteration: shape-serial diff
  → control-task bridge (mirrors probe_result), state datagram → conn.
- CursorOverlay/capture CursorState carry the hotspot through
  (nearest-neighbor downscale backstop for XL cursors, unit-tested).

Presenter:
- CursorChannel drains both planes per loop iteration; shapes become
  SDL color cursors (from_surface + hotspot), applied while the desktop
  mouse model is engaged; visibility follows the host; capture/released
  hands back the system cursor. Sessions advertise the cap when they
  START in desktop mode.

Verified on .21: fmt + clippy -D warnings (7 crates) + tests green
(core 218 incl. new wire roundtrips, host 245 incl. e2e + forwarder
downscale tests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:43:23 +02:00
enricobuehlerandClaude Fable 5 97d20e1f0a feat(apple/input): desktop (absolute) mouse mode on macOS — remote-desktop sweep M1
Folds the parked client-side-cursor machinery (cursorMode auto/always/
never, hidden while disabled) into the cross-client mouse model:
MouseInputMode capture|desktop under DefaultsKey.mouseMode, picked in
Settings ▸ Keyboard & mouse (macOS), resolved at session start and
gated off on gamescope hosts (relative-only EIS).

Desktop model = the un-neutered absolute path with the SDL cursor
policy: pointer never disassociated (enters/leaves the stream freely),
monitor forwards letterboxed absolute positions, and the local cursor
hides only while over the view via an invisible-cursor rect (the
host's composited cursor is the one you see; AppKit manages the rect,
so no hide/unhide balancing). ⌘⇧C becomes ⌃⌥⇧M — the same chord as
the SDL clients — flipping the model live with an atomic
release/re-engage.

Verified: swift build + full test suites green (macOS arm64).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:37:32 +02:00
enricobuehlerandClaude Fable 5 df04880273 feat(client/input): desktop (absolute) mouse mode — remote-desktop sweep M1
New physical-mouse model beside capture: Desktop leaves the pointer
uncaptured and sends absolute positions through the letterbox
(MouseMoveAbs — every host injector already consumes it). Capture
stays the default and the game model.

- pf-client-core: MouseMode (capture|desktop) persisted in Settings,
  default capture so existing stores are unchanged.
- pf-presenter: Capture grows a desktop model — latest-wins pending
  abs position coalesced per loop iteration (same 1000 Hz discipline
  as relative), flushed before clicks/keys/wheel so they land where
  the cursor is; Ctrl+Alt+Shift+M flips the model live; local cursor
  stays hidden over the window (the host's composited cursor is the
  one you see until the M2 cursor channel); Windows keyboard grab
  only engages for capture (a desktop stream is something you
  Alt-Tab away from).
- gamescope gating: its EIS is relative-only, so desktop mode is
  pinned off there (resolved_compositor), with a log note.
- Settings surfaces: GTK row (dynamic caption), WinUI combo,
  console-UI row + step test, capture-hint line.

Verified: fmt + clippy -D warnings + tests (33/40/19) on Linux .21;
clippy -D warnings for all five crates incl. punktfunk-client-windows
native on .173.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 23:37:32 +02:00
enricobuehlerandClaude Fable 5 73fee86337 chore(release): bump workspace version to 0.18.0
audit / bun-audit (push) Failing after 13s
ci / docs-site (push) Failing after 22s
ci / web (push) Successful in 58s
apple / swift (push) Successful in 1m19s
audit / cargo-audit (push) Successful in 3m15s
decky / build-publish (push) Successful in 17s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 55s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 19s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / bench (push) Successful in 8m25s
deb / build-publish (push) Successful in 9m47s
docker / deploy-docs (push) Successful in 25s
release / apple (push) Successful in 10m10s
android / android (push) Successful in 12m56s
deb / build-publish-host (push) Successful in 13m10s
windows-host / package (push) Successful in 16m18s
arch / build-publish (push) Successful in 17m16s
apple / screenshots (push) Successful in 6m39s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m5s
ci / rust (push) Successful in 19m43s
flatpak / build-publish (push) Successful in 6m25s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m10s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m33s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m12s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m32s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m24s
The 0.17.2 placeholder underestimated this cycle: Android mouse & keyboard
(physical mouse + capture, TV remote-as-pointer, IME text, clipboard), the new
committed-text input plane (InputKind::TextInput / HOST_CAP_TEXT_INPUT), and
the ChaCha20-Poly1305 session cipher are feature work — a minor, not a patch.
Wire/driver/C-ABI markers all unchanged (2 / 4 / 10): every addition is
negotiated or ignored by older peers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 22:13:20 +02:00
enricobuehlerandClaude Fable 5 b5ed486243 ci(android): survive fleet-load download truncation in the SDK setup
The android leg failed twice on 2026-07-22 with sdkmanager's "Error on
ZipFile unknown archive" — once in our NDK/build-tools step, once inside
setup-android's default package set. Root cause is the shared runner fleet
dropping packets under parallel-job load (the same class retry.sh documents
for the flatpak leg): sdkmanager unzips while streaming, so a truncated
stream reads as a corrupt archive. A solo re-download on the same runner
verified clean.

- setup-android: install only platform-tools — the action's default legacy
  `tools` package drags in the ~250 MB emulator this job never uses, which
  was the largest and flakiest download in the whole leg.
- NDK/build-tools/CMake: wrap sdkmanager in retry.sh (4 attempts, linear
  backoff) — a failed attempt leaves no partial package, so re-invoking is
  safe.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 22:12:02 +02:00
enricobuehlerandClaude Opus 4.8 0e7f6d85d6 feat(gamestream): signal HDR mode to the client via the 0x010e control message
ci / web (push) Successful in 1m14s
apple / swift (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m20s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
ci / bench (push) Successful in 6m41s
apple / screenshots (push) Successful in 6m12s
deb / build-publish (push) Successful in 9m48s
docker / deploy-docs (push) Successful in 25s
deb / build-publish-host (push) Successful in 10m26s
windows-host / package (push) Successful in 16m13s
android / android (push) Successful in 16m27s
arch / build-publish (push) Successful in 17m20s
ci / rust (push) Successful in 20m1s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m43s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 19m41s
punktfunk negotiates HDR correctly (the stream is BT.2020 PQ 10-bit) but never
told the client to switch its display into HDR picture mode, so HDR streams
stayed in SDR on the TV. Sunshine sends an async control-stream message
(IDX_HDR_MODE, type 0x010e, carrying SS_HDR_METADATA) shortly after the stream
starts; Moonlight only flips the TV into HDR on that cue. We sent exactly one
host→client control message (rumble 0x010B) and never this one. (aurora-tv PR
#53 worked around it client-side by inferring HDR from the negotiated format;
this fixes it host-side so stock Moonlight clients switch too.)

Once the control stream is live and the session negotiated HDR, send a one-shot
0x010e message (enable + 26-byte SS_HDR_METADATA from generic HDR10 defaults),
latched per connection and re-armed on Disconnect. The rumble sequence counter
is unified into a single monotonic `host_seq` shared by both message types —
the GCM nonce derives from `seq`, so a separate per-type counter would reuse
(key, nonce) pairs in the host direction.

The BT.2020 PQ VUI already rides the bitstream (the encoder derives it from the
P010 capture format), so this is purely the missing "switch now" signal; no
encoder change needed. Wire layout locked by unit tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 21:35:16 +02:00
enricobuehlerandClaude Opus 4.8 2dddf0fefc fix(gamestream): accept legacy X.509 v2 Moonlight client certs in mTLS pairing
rustls-webpki 0.103 accepts only X.509 v3 certs and returns
`UnsupportedCertVersion` for anything older, aborting the mTLS handshake at
the `CertificateVerify` step before it ever checks the signature. The
moonlight-embedded client family (moonlight-embedded, aurora-tv, …) still mint
self-signed **v2** certs with no keyUsage extension, so pairing fails against
punktfunk while it works against Sunshine — whose OpenSSL verify callback never
inspects the cert version. (aurora-tv PR #53 worked around this client-side by
switching to v3 + keyUsage; this fixes it host-side for stock clients too.)

The cert's X.509 version and extensions carry no security weight here: the
client cert is self-signed and pinned by SHA-256 (`peer_is_paired`), and the PIN
pairing ceremony is the real proof of identity. The property that DOES matter —
that the peer holds the private key for the cert it presented — is still
enforced: when webpki rejects the cert we re-run exactly that `CertificateVerify`
check with a version-agnostic parser (x509-parser SPKI + the same `rsa` verify
`pairing::verify256` uses). This is a full cryptographic check, never a bypass:
a bad signature still fails, and non-RSA / unsupported schemes fall through to
webpki's original error. Matches Sunshine's leniency without loosening the
pinned trust model.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 21:35:03 +02:00
enricobuehlerandClaude Fable 5 c90343c22f feat(android): shared clipboard (text) — device↔host sync while streaming
ci / web (push) Successful in 1m4s
ci / docs-site (push) Successful in 1m13s
apple / swift (push) Successful in 1m21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
decky / build-publish (push) Successful in 33s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 8m18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m49s
deb / build-publish (push) Successful in 10m26s
release / apple (push) Successful in 9m22s
docker / deploy-docs (push) Successful in 28s
deb / build-publish-host (push) Successful in 12m44s
arch / build-publish (push) Successful in 15m51s
windows-host / package (push) Successful in 16m14s
apple / screenshots (push) Successful in 6m37s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m6s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m18s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
ci / rust (push) Successful in 24m48s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m31s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m21s
android / android (push) Successful in 11m30s
flatpak / build-publish (push) Successful in 6m16s
The desktop clients' clipboard protocol, adopted on Android (text-only v1):
opt-in ClipControl at stream start when the host advertises HOST_CAP_CLIPBOARD
and the new "Shared clipboard" setting (default on) allows.

Device → host: local copies (primary-clip listener + a probe at start) are
announced as lazy format-list offers; the text crosses only when the host
actually pastes (FetchRequest → served from the live clipboard). Host →
device: a host copy's offer is fetched eagerly and lands in the system
clipboard (Android has no practical lazy-paste provider), with an echo guard
so the resulting clip-changed callback doesn't bounce it back as a new offer.

Native side: session/clipboard.rs JNI shims over NativeClient's clip_*
surface; events cross to Kotlin as compact strings from a blocking
nativeNextClip poll on a dedicated thread (the nativeNextRumble pattern),
joined before the session handle is freed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 19:27:19 +02:00
enricobuehlerandClaude Fable 5 abe6228b42 feat(core+host+android): committed-text input plane (IME path) — InputKind::TextInput
The VK key-event vocabulary cannot express text an input method COMMITS
(autocorrect, gesture typing, non-Latin scripts, emoji). Add a first-class
text event and negotiate it:

- punktfunk-core: InputKind::TextInput (= 15) carries one Unicode scalar per
  event in `code`; HOST_CAP_TEXT_INPUT (0x04) in Welcome::host_caps.
- Host advertises the cap only where the session's inject backend can type
  text: Windows SendInput (KEYEVENTF_UNICODE, surrogate-pair aware) and the
  Linux wlroots backend — a dedicated second zwp_virtual_keyboard whose xkb
  keymap grows Unicode keysyms on demand (the wtype model), so keymap
  re-uploads never disturb the main device's layout/modifier state. The
  KWin-fake-input/libei/gamescope backends can only press layout keycodes, so
  those sessions don't set the bit and clients keep the VK fallback.
- GameStream plane: Moonlight's UTF-8 text packet (MAGIC_UTF8, previously
  recognized-and-dropped) now decodes to the same TextInput events.
- Android: KeyCaptureView picks a real editable InputConnection when the host
  has the cap — the IME runs its full machinery, mirrored to the host live via
  common-prefix diffs of the composition (backspaces + new suffix), with
  setComposingRegion adopting committed text so autocorrect-revert flows diff
  instead of retyping; newline→Enter, deleteSurroundingText→Backspace/Delete.
  Older hosts keep the TYPE_NULL raw-key path unchanged.
- keymap: media VKs (0xB0-0xB3) → evdev so the Android media keys land on
  Linux hosts too.

Verified: punktfunk-core + host gamestream + pf-inject tests green on Linux
(Ubuntu box), clippy clean; Android app+native builds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 19:19:11 +02:00
enricobuehlerandClaude Fable 5 ddb72edcba feat(android): physical mouse + TV remote-as-pointer + keyboard hardening
Phase A — hardware mouse: new MouseForwarder routes every SOURCE_MOUSE event
while streaming. Uncaptured (default): hover/drag positions forward as absolute
cursor moves, wheel with fractional accumulation (high-res wheels), all five
buttons incl. back/forward as X1/X2; the local cursor is hidden over the stream
(TYPE_NULL pointer icon) so the host's composited cursor is the visible one.
Captured ("Capture pointer for games" setting, or Ctrl+Alt+Shift+Q live): the
OS pointer is grabbed via requestPointerCapture on the (repurposed) key-capture
view and raw relative deltas forward as mouse-look; held buttons are tracked
and flushed on capture loss/teardown, and the engaging click is swallowed
(desktop parity). Mouse clicks are intercepted before Compose so they can never
be misread as trackpad taps; a mouse back-button's FALLBACK BACK is dropped so
it can't yank the user out of the stream.

Phase B — keyboard: media/consumer keys (play/pause/next/prev/stop) now map to
their VKs instead of falling to the system; TV-remote SELECT maps to Enter.

Phase D — Android TV: RemotePointer turns a D-pad-only remote into a pointer
(Siri-remote analogue): hold SELECT ~0.8s toggles, D-pad glides the host cursor
with ramped acceleration (Choreographer-paced, diagonal-normalized), SELECT
taps click, play/pause right-clicks, holding it toggles the on-screen keyboard,
BACK leaves pointer mode; an overlay hint teaches the vocabulary.

Settings: "Capture pointer for games" + "Invert scroll direction" (applies
to the wheel and two-finger touch scrolling).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 18:56:38 +02:00
enricobuehlerandClaude Fable 5 5e088af7ed fix(pf-win-display): unpin mode indexes on paths the CCD isolate deactivates
ci / web (push) Successful in 49s
apple / swift (push) Successful in 1m24s
ci / docs-site (push) Successful in 55s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
apple / screenshots (push) Successful in 6m33s
ci / bench (push) Successful in 6m41s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m23s
android / android (push) Successful in 12m50s
docker / deploy-docs (push) Successful in 25s
deb / build-publish-host (push) Successful in 10m10s
arch / build-publish (push) Successful in 17m18s
deb / build-publish (push) Successful in 12m51s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m3s
ci / rust (push) Successful in 27m3s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 18m32s
Field report: with Exclusive Topology, the physical monitor sometimes stayed
lit — every isolate attempt re-committed and failed SetDisplayConfig with
0x57 ERROR_INVALID_PARAMETER. Clearing DISPLAYCONFIG_PATH_ACTIVE alone leaves
the deactivated path's source/target modeInfoIdx pinned to the queried mode
entries, and per the SetDisplayConfig contract a path being turned off must
have BOTH mode indexes set to DISPLAYCONFIG_PATH_MODE_IDX_INVALID; some
driver/topology combinations (notably when the doomed path shares a source
with the kept virtual path) reject the supplied config outright, so the
retry loop could never converge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 12:26:44 +02:00
enricobuehlerandClaude Fable 5 b7a00137eb feat(host): hold a logind sleep/idle inhibitor while clients stream
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 1m0s
android / android (push) Failing after 1m6s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 48s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 52s
ci / bench (push) Successful in 8m13s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m40s
deb / build-publish (push) Successful in 11m0s
release / apple (push) Successful in 9m50s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10m19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m57s
docker / deploy-docs (push) Successful in 35s
deb / build-publish-host (push) Successful in 14m3s
apple / screenshots (push) Successful in 6m27s
arch / build-publish (push) Successful in 17m49s
windows-host / package (push) Successful in 18m15s
flatpak / build-publish (push) Failing after 8m25s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m24s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m37s
ci / rust (push) Successful in 25m51s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m48s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m52s
Remote INPUT resets the compositor's idle timers, but a passive
(video-only) viewer sends none — observed live: a SteamOS Game-Mode
host s2idled mid-day and dropped off the network (in a VM with GPU
passthrough it never woke again). Refcounted across planes: the native
LiveSessionGuard and the GameStream video worker each hold a share;
first hold takes the logind sleep:idle block inhibitor on a dedicated
plain thread (zbus blocking must not run on a tokio worker), last drop
releases. Best-effort: no logind → warn once and stream on. No-op off
Linux (macOS/Windows manage their own assertions).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 12:09:47 +02:00
enricobuehlerandClaude Fable 5 acce43ebbf feat(steamdeck): self-healing reliability — post-OS-update rebuild check + script/docs polish
- rebuild-check.sh + punktfunk-rebuild-check.service (enabled, ordered
  Before=punktfunk-host): ldd-probes the host binary at session start —
  milliseconds when healthy, a full update.sh rebuild only when a
  SteamOS update actually broke its library links. update.sh restarts
  go --no-block so the check → update.sh → restart chain can't deadlock
  against the unit ordering. update.sh retrofits the unit.
- installer summary: web console is https (the unit serves TLS).
- docs: steamos-host.md (runner in the build step, keep-list + auto
  rebuild = updates survive hands-free), gamescope.md (Gaming Mode
  touch = single-finger pointer, exact taps, no multi-touch),
  plugins.mdx (SteamOS runner note), scripts/steamdeck/README.md
  (rebuild-check, runner payload, keep list, honest packaging
  trade-off + the CI-prebuilt future direction).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 11:56:09 +02:00
enricobuehlerandClaude Fable 5 35923080fb fix(host/linux): scale degenerate-region absolute coords by the session's real output size
The raw-client-pixels fallback is exact only while the managed session
runs at the client's mode. When they diverge — foreign-gamescope
attach at its own resolution, supersample/under-render, transitions —
raw pixels drift or land out of range. The EIS relay file now carries
the session's current output size as a second "WxH" line (from
current_gamescope_output_size()); the injector scales normalized
client positions into it, keeping raw pixels only as the last resort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:53:14 +02:00
enricobuehlerandClaude Fable 5 0c9461242c fix(host/linux): don't normalize absolute coords into gamescope's degenerate INT32_MAX EIS region
The previous commit's no-region fallback never fired: gamescope DOES
advertise a region — (0,0,INT32_MAX,INT32_MAX), meaning "coordinates
are raw". Mapping normalized positions into it explodes a center tap
to x≈1e9, clamped by gamescope to the far corner — the cursor pinned
at (1279,799) and every degraded-touch tap landing there. Treat a
region as an output geometry only when plausibly sized (≤16384 px);
otherwise emit raw client pixels.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:40:02 +02:00
enricobuehlerandClaude Fable 5 cc6b37fef0 fix(host/linux): emit absolute pointer/touch on region-less EIS devices + abs input-test probe
gamescope's "Gamescope Virtual Input" advertises pointer_abs but no
region, so every MouseMoveAbs (and the degraded-touch moves built on
it) was silently dropped (emitted=false) — clicks then landed at a
stale cursor position. With no region, the managed session runs at the
client's mode, so client pixels are output pixels: emit them raw.
Also: log region count/dims at device-resume, and add
PUNKTFUNK_INPUT_TEST_ABS=WxH to `input-test` (corners + center, 1s
apart) so the degraded-touch path is verifiable with xdotool alone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:36:05 +02:00
enricobuehlerandClaude Fable 5 1927077cbe fix(host/linux): degrade touch to absolute pointer when the EIS has no touchscreen device
gamescope's EIS ("Gamescope Virtual Input") advertises pointer/
pointer_abs/button but never a touchscreen — so in Game Mode every
remote TouchDown/Move/Up was dropped on the floor (Desktop Mode works
via KwinFakeInput, which does touch). Headless KWin's libei has the
same gap. Instead of dropping: degrade to a single-finger absolute
pointer — down = abs-move + left press, move = abs-move, up = release —
synthesized through the normal Mouse* inject paths so region mapping,
held-state tracking, and release_all apply. First finger drives the
pointer; later fingers are ignored (a pinch degrades to a drag). A
touchscreen device appearing later takes over on the next touch —
checked per event, never latched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:24:28 +02:00
enricobuehlerandClaude Fable 5 068da456f5 fix(steamdeck): survive SteamOS A/B updates — register system tuning on the atomic-update keep list
Verified live on a SteamOS 3.8.14→3.8.16 update: the A/B partset switch
rebuilds /etc and drops everything not on Valve's keep list — the udev
rule (uhid access: virtual pads silently degrade to Xbox 360), the
vhci-hcd autoload (native Deck pad transport), and the UDP buffer sysctl
(back to 208 KB → stutter). Valve's sanctioned extension point is a
drop-in under /etc/atomic-update.conf.d/ — itself on the stock keep
list, so it self-preserves. install.sh §4 and update.sh now register
scripts/punktfunk-atomic-keep.conf there.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:07:19 +02:00
enricobuehlerandClaude Fable 5 256aa5f7f2 fix(steamdeck): install the plugin runner + harden Deck pad typing + repair update.sh
- scripts/steamdeck/install.sh (§2b) + update.sh: build and install the
  scripting runner user-scoped (~/.local wrapper + pinned bun + bundle) —
  SteamOS's read-only /usr can't take the .deb layout, so the console's
  plugin store reported "the plugin runner isn't installed" on every
  SteamOS host. update.sh retrofits it onto existing installs.
- plugins.rs runner discovery: check the ~/.local layout after the /usr
  ones; mention the SteamOS path in the not-installed error.
- update.sh: define warn() — it was used but never defined, so under
  `set -e` the first warn aborted the update before services restarted.
- pf-client-core gamepad: a Steam Input virtual pad forwarded on a real
  Deck (the only-pad case) now declares the DECK kind in its per-pad
  arrival instead of the wrapper's Xbox 360 identity — the session-level
  auto_pref already resolved SteamDeck, but the arrival overrides it on
  current hosts.
- reject tests: cover SetupFailed; move the foreign-code probe off 0x68.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 10:01:54 +02:00
enricobuehlerandClaude Fable 5 2562663fc6 fix(host/linux): survive and self-heal dead gamescope sessions on headless SteamOS boxes
Three fixes for the "most connects simply fail" trap on a SteamOS host
with no physical display (VM testbox, panel-less mini PC):

- restore path: skip the restore-to-physical-panel when no DRM connector
  reports a connected display — removing the headless drop-in and
  restarting gamescope-session.target on such a box just crash-loops
  gamescope and strands every later connect on "no usable compositor".
  Keep the headless session (and the takeover state) instead.
- connect path: when no live graphical session exists but the MANAGED
  gamescope infra is present (SteamOS gamescope-session / Bazzite
  session-plus), route to the gamescope takeover — which rebuilds the
  session at the client's mode — instead of failing the connect.
- error delivery: a session-setup failure used to just drop the QUIC
  connection, reaching the client as "control stream finished mid-frame"
  (indistinguishable from transport trouble). Close with a typed
  setup-failed code (0x68, RejectedSetupFailed = -29) carrying the error
  text, and give the client handshake/pairing error paths a short grace
  for the CONNECTION_CLOSE to beat the stream error it caused.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 09:50:30 +02:00
enricobuehler 87e7c82cbc chore(release): bump workspace version to 0.17.2
audit / bun-audit (push) Failing after 18s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
apple / swift (push) Successful in 1m23s
audit / cargo-audit (push) Successful in 2m44s
ci / bench (push) Successful in 6m26s
apple / screenshots (push) Successful in 6m38s
ci / rust (push) Successful in 24m53s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m10s
decky / build-publish (push) Successful in 22s
android-screenshots / screenshots (push) Successful in 3m7s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 44s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 14s
deb / build-publish (push) Successful in 10m38s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9m3s
release / apple (push) Successful in 10m50s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m41s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m4s
deb / build-publish-host (push) Successful in 12m44s
android / android (push) Successful in 14m35s
docker / deploy-docs (push) Successful in 23s
arch / build-publish (push) Successful in 15m35s
web-screenshots / screenshots (push) Successful in 3m1s
linux-client-screenshots / screenshots (push) Successful in 7m31s
windows-host / package (push) Successful in 13m4s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m24s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m32s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m39s
flatpak / build-publish (push) Successful in 6m40s
2026-07-21 23:33:53 +02:00
enricobuehlerandClaude Opus 4.8 b2e3d1b540 fix(steamdeck/install): add cmake dep + acquire sudo in preflight
ci / web (push) Successful in 1m20s
ci / docs-site (push) Successful in 1m52s
apple / swift (push) Successful in 1m22s
android / android (push) Successful in 13m4s
ci / bench (push) Successful in 8m2s
decky / build-publish (push) Successful in 23s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 12s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 10s
deb / build-publish (push) Successful in 9m14s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m14s
apple / screenshots (push) Successful in 6m51s
arch / build-publish (push) Successful in 18m54s
docker / deploy-docs (push) Successful in 27s
ci / rust (push) Successful in 19m17s
deb / build-publish-host (push) Successful in 9m17s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 14m33s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m29s
pyrowave-sys's build.rs runs cmake to build the vendored PyroWave C library,
but cmake wasn't in the distrobox apt list — so `cargo build -p punktfunk-host`
dies on every SteamOS host with "is cmake not installed?". Add it.

Also move sudo acquisition from step 4 (after the ~15-min build) into preflight
and keep the timestamp warm across the build. Before, a non-interactive run
(`ssh host 'bash install.sh'`) silently skipped the UDP-buffer/udev/vhci-hcd/
input-group tuning AND enable-linger — a "successful" host with no gamepad
passthrough, lossy streaming, and no headless persistence (dies on logout).
Now sudo is prompted once up front (authorize and walk away), and a no-TTY /
no-password run fails loudly before the build instead of silently at the end.

Validated end-to-end on SteamOS 3.8 in a VM with Radeon 780M passthrough:
full host + web build, all planes listening, VAAPI render node, headless
auto-start via linger.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:50:00 +02:00
enricobuehlerandClaude Fable 5 d36bec6e9d feat(core+host): negotiate ChaCha20-Poly1305 as the session cipher for soft-AES clients
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
apple / swift (push) Has been cancelled
arch / build-publish (push) Has been cancelled
audit / bun-audit (push) Failing after 13s
audit / cargo-audit (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / web (push) Has been cancelled
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Successful in 10m1s
windows-host / package (push) Successful in 11m20s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m29s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 4m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 22m16s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m27s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 5m28s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m39s
Lifts the ~100 Mbps decrypt ceiling on clients without hardware AES — the
armv7 soft-AES targets (webOS TVs), where AES-128-GCM resolves to fixsliced
software AES + software GHASH (~50-100 cpb) while ChaCha20-Poly1305's ARX
construction runs ~10-17 cpb portable, a 4-7x lift that PyroWave-on-TV needs
(design/chacha20-session-cipher.md).

Phase 1 (core crypto, no wire change): SessionKey merges cipher choice and
key material (invalid combinations unrepresentable, zeroize + redacted-Debug
discipline kept); SessionCrypto dispatches both aead-0.5 ciphers per call —
the salt||seq nonce scheme, per-direction salts, seq-as-AAD and replay
window carry over verbatim (same 96-bit nonce / 16-byte tag, const-asserted).
Config.key becomes SessionKey; validate's zero-key rejection follows the
active variant. The C ABI keeps its fixed 16-byte key mapped to AES — no
ABI_VERSION bump.

Phase 2 (negotiation): VIDEO_CAP_CHACHA20 (0x40) — support-plus-request in
one bit, the VIDEO_CAP_444 precedent. Welcome grows cipher@68 +
key_chacha@69..101, emitted only when non-zero so an AES session's Welcome
stays byte-identical to the pre-cipher form; decode is fail-closed (short
key or unknown id -> Err, never a silent AES fallback). No WIRE_VERSION
bump; downgrade resistance inherited from the pinned-TLS control channel.

Phase 3 (host): grant only when the client advertised the bit and the
PUNKTFUNK_CHACHA20 kill-switch (default on, documented) allows; fresh
32-byte per-session key from the same RNG discipline, legacy key field
stays independently random; resolved cipher logged at session start.

Verification: seal/open suites parameterized over both ciphers + a
cross-cipher tamper case; Welcome roundtrip/truncation/fail-closed tests;
ChaCha lossy-loopback soak (loss/replay is cipher-independent); bench
gains _chacha20 series (AES ids unchanged for CI history) — host-side
sealing line-rate-trivial on both x86 (~640 MiB/s) and Apple Silicon
(~535 MiB/s). punktfunk-probe drives the interop matrix via
PUNKTFUNK_CLIENT_CHACHA20=1 and logs the negotiated cipher.

Phase 4 (pf-webos pin bump + unconditional cap bit) follows the next
core release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:50:00 +02:00
enricobuehlerandClaude Fable 5 abc54a7d13 fix(decky): actually ship shortcut artwork + Steam Input layout in the published plugin
apple / swift (push) Successful in 1m23s
ci / web (push) Successful in 52s
ci / docs-site (push) Successful in 56s
android / android (push) Has been cancelled
apple / screenshots (push) Has been cancelled
arch / build-publish (push) Has been cancelled
ci / rust (push) Has been cancelled
ci / bench (push) Has been cancelled
deb / build-publish (push) Has been cancelled
deb / build-publish-host (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8m58s
The CI workflow assembled the store zip by hand and silently drifted from
scripts/package.sh when 60d46530 added assets/ and controller_config/ — every
published build (canary and stable through 0.17.1) was missing both, so the
Steam shortcut a fresh install creates had no images and no controller layout
(shortcut_art() found nothing to send; apply_controller_config failed with
template-missing). CI now stages through scripts/package.sh — the plugin file
list lives in one place — and a backstop verifies the runtime-loaded pieces are
in the zip before publishing.

Frontend hardening for installs poisoned by those builds: applyArtwork stamped
its per-appId ART_VERSION marker even when zero assets were applied, so
shipping the files alone would never heal existing shortcuts — only mark done
when something actually landed, and bump ART_VERSION 2→3 to force one re-apply.
Same guard for the controller-config marker (ok with zero account configset
dirs is not done), plus one deferred artwork retry for the fresh-AddShortcut
registration race setShortcutHidden already defers around.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:36:55 +02:00
enricobuehlerandClaude Fable 5 309e37f1e1 fix(host/linux): scope the GPU clock pin to live client sessions
The PUNKTFUNK_PIN_CLOCKS clock pin (AMD power_dpm_force_performance_level=high / NVIDIA nvmlDeviceSetGpuLockedClocks) was armed once at host start and released only at host exit, so the box's clocks stayed pinned the whole time the host ran — even with no client connected.

Move the pin behind a box-wide refcount (gpuclocks::session_pin()) shared across both streaming planes (native + GameStream): it arms on the first live client and releases on the last disconnect. on_host_start() now only installs the process-scoped NVIDIA P2-cap application profile. No behavior change when PUNKTFUNK_PIN_CLOCKS is unset.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:36:55 +02:00
enricobuehlerandClaude Fable 5 56d21f9445 fix(client/wol): retry the QUIC dial across the connect budget so wake-from-sleep launches survive
ci / web (push) Successful in 57s
ci / docs-site (push) Successful in 1m3s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 40s
ci / bench (push) Successful in 6m51s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 18s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 16s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m6s
deb / build-publish (push) Successful in 9m54s
release / apple (push) Successful in 9m5s
deb / build-publish-host (push) Successful in 9m35s
flatpak / build-publish (push) Failing after 8m19s
android / android (push) Successful in 16m57s
apple / screenshots (push) Successful in 6m29s
arch / build-publish (push) Successful in 17m35s
windows-host / package (push) Successful in 18m46s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
docker / deploy-docs (push) Successful in 26s
ci / rust (push) Successful in 25m24s
Launching from the Decky plugin against a sleeping host fired the
Wake-on-LAN packet and dialed immediately — but the dial was a single
quinn attempt that gave up after the transport's 8 s idle window, far
shorter than a suspend-to-RAM resume. The host woke to find the client
already gone (its retransmitted Initials show up host-side as
"QUIC accept failed error=timed out"), the stream shortcut exited, and
the launch looked cancelled.

- punktfunk-core: dial in a loop until the embedder's connect budget is
  spent — short 3 s attempts keep the Initial cadence dense, so the
  connect lands within ~a second of the host's network returning. Only
  silence is retried: pin mismatches, typed rejections, and any real
  answer surface immediately, and the shutdown flag stops the loop.
- decky: stretch the budget to 75 s when a magic packet actually went
  out (PF_CONNECT_TIMEOUT → --connect-timeout via the wrapper), so the
  retry window covers the resume; an awake host still connects in <1 s.
- host: a clean close before the control handshake is a reachability
  probe (hosts-page online pips), not a failed session — log it at
  debug instead of WARN "session ended with error", which buried the
  real failures in the reporter's log.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:26:10 +02:00
enricobuehlerandClaude Opus 4.8 f36d13e371 fix(steamdeck/install): prompt for sudo instead of requiring passwordless
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m19s
decky / build-publish (push) Successful in 31s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 16s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 14s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
ci / bench (push) Successful in 6m5s
apple / screenshots (push) Successful in 6m34s
deb / build-publish (push) Successful in 9m43s
docker / deploy-docs (push) Successful in 48s
deb / build-publish-host (push) Successful in 10m9s
android / android (push) Successful in 18m18s
arch / build-publish (push) Successful in 19m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m2s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 22m16s
ci / rust (push) Successful in 28m27s
The system-tuning block (UDP buffers, gamepad udev rule, vhci-hcd autoload,
input group) was gated on `sudo -n true` — passwordless sudo — so on a stock
SteamOS box (the deck account needs a password) the whole block was silently
skipped: /dev/uhid stayed root-only so the gamepad degraded to an Xbox 360 pad,
and the UDP buffers stayed at the 416 KB default. Video was unaffected because
Game Mode/gamescope needs none of it, which masked the skip on-glass.

- install.sh + update.sh: acquire sudo interactively (`sudo -v` prompt on a TTY)
  instead of requiring passwordless; skip only when there's no TTY or auth fails,
  and then print the exact manual block plus a `passwd` hint — a stock SteamOS
  'deck' account has no password, so sudo can't work until one is set.
- update.sh: also retrofit the UDP-buffer sysctl for older/skipped installs, and
  loudly nag to reboot when the input group was just added (a --user restart does
  not pick up the new group; only a fresh login does).
- steamos-host.md: note this step prompts for the sudo password.

Root is unavoidable: the udev rule, input group, and vhci-hcd module are all
kernel operations with no user-space alternative.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:56:20 +02:00
enricobuehlerandClaude Opus 4.8 aacd610b68 fix(steamdeck/install): set up controller passthrough + surface the required reboot
ci / docs-site (push) Successful in 1m12s
ci / web (push) Successful in 1m13s
apple / swift (push) Successful in 1m18s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 14s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
ci / bench (push) Successful in 6m35s
apple / screenshots (push) Successful in 6m13s
deb / build-publish (push) Successful in 10m21s
docker / deploy-docs (push) Successful in 34s
deb / build-publish-host (push) Successful in 9m58s
android / android (push) Successful in 17m7s
arch / build-publish (push) Successful in 17m47s
ci / rust (push) Successful in 19m28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m59s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m2s
A fresh SteamOS host install failed on-glass two ways: KWin denied Desktop-mode
capture (zkde_screencast_unstable_v1) so every session died in the pipeline build,
and the native Steam Deck pad degraded to an Xbox 360 controller. Both stem from
setup that only a fresh login applies — the host user-service keeps the
pre-install group set and KWin reads its .desktop grant at session start — plus a
missing vhci-hcd autoload the pad's USB/IP transport needs (the deb/rpm/arch
packages ship it; the Deck installer did not).

- install.sh: install punktfunk-modules.conf (+ modprobe vhci-hcd now) for the
  native Deck controller transport, seed the KDE RemoteDesktop grant
  (kde-authorized) for Desktop-mode input, and print a loud "reboot before
  streaming" notice when a relogin is actually required (input group added or the
  .desktop grant first installed).
- update.sh: retrofit the udev rule, vhci-hcd autoload, input group, and grant so
  existing installs pick them up on the next update without a reinstall.
- steamos-host.md: document the first-install reboot and the native controller
  passthrough requirements (input group + vhci-hcd + client-side Steam Input Off).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:22:08 +02:00
enricobuehlerandClaude Opus 4.8 081ff64087 docs: explain how to actually run the VirtualHere passthrough example
apple / swift (push) Successful in 1m22s
ci / web (push) Successful in 1m11s
ci / docs-site (push) Successful in 1m11s
ci / bench (push) Successful in 5m39s
apple / screenshots (push) Successful in 6m38s
decky / build-publish (push) Successful in 20s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 57s
deb / build-publish (push) Successful in 9m0s
deb / build-publish-host (push) Successful in 9m20s
windows-host / package (push) Successful in 9m46s
android / android (push) Successful in 18m37s
arch / build-publish (push) Successful in 18m48s
docker / deploy-docs (push) Successful in 28s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m44s
ci / rust (push) Successful in 26m30s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 16m2s
The VirtualHere DualSense recipe linked the `virtualhere-dualsense.ts` SDK
example as a deployable recipe but skipped everything needed to run it: that
VirtualHere is a server (couch) + client (host) pair, and that the example —
like every example — imports `../src/index.js`, which only resolves inside the
SDK repo. A user copying it out had no way to know they must
`bun add @punktfunk/host` and swap that import.

- automation.md: rewrite the recipe into a full walkthrough — the two-sided
  VirtualHere setup, the `-t` verbs, the zero-code hooks version, and a new
  scripted section with exact deploy steps, env vars, and running it as a
  service (its own SIGTERM handler makes `systemctl stop` release the pad).
- sdk/README.md: say how to run an example in-repo vs deployed on a host.
- virtualhere-dualsense.ts: header note on the import swap + service setup,
  since that file is where the doc link lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 17:28:23 +02:00
enricobuehlerandClaude Opus 4.8 675fd24cce chore(release): bump workspace version to 0.17.1
audit / cargo-audit (push) Successful in 3m22s
audit / bun-audit (push) Failing after 13s
apple / swift (push) Successful in 1m23s
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m11s
ci / bench (push) Successful in 5m12s
apple / screenshots (push) Successful in 6m55s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m55s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m39s
ci / rust (push) Successful in 24m5s
decky / build-publish (push) Successful in 30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 17s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 40s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 13s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 11s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 10s
android-screenshots / screenshots (push) Successful in 3m34s
linux-client-screenshots / screenshots (push) Successful in 8m32s
release / apple (push) Successful in 11m16s
deb / build-publish-host (push) Successful in 12m51s
deb / build-publish (push) Successful in 12m57s
docker / deploy-docs (push) Successful in 27s
web-screenshots / screenshots (push) Successful in 3m13s
windows-host / package (push) Successful in 14m6s
android / android (push) Successful in 15m33s
arch / build-publish (push) Successful in 16m3s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m43s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m40s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m10s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 23m16s
flatpak / build-publish (push) Successful in 7m9s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 16:37:32 +02:00
enricobuehlerandClaude Fable 5 0e60e58cab perf(apple): windowed macOS presents — off-main transactional commit, surface prototype, safe-present setting
ci / docs-site (push) Successful in 52s
ci / web (push) Successful in 56s
decky / build-publish (push) Successful in 21s
apple / swift (push) Successful in 1m24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / bench (push) Successful in 5m53s
release / apple (push) Successful in 9m19s
deb / build-publish (push) Successful in 12m27s
arch / build-publish (push) Successful in 12m52s
docker / deploy-docs (push) Successful in 26s
deb / build-publish-host (push) Successful in 13m18s
apple / screenshots (push) Successful in 6m20s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 15m51s
android / android (push) Successful in 18m40s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m15s
ci / rust (push) Successful in 27m38s
Task 1 (latency): the transactional present now commits from the RENDER
thread inside an explicit CATransaction + flush() instead of hopping to
main. The present harness (2026-07-21, 240 Hz Studio, full-size window)
measured the main hop batching presents at runloop-iteration rate when
main is busy — an active implicit transaction NESTS the explicit one —
which is the field's presents=55 @ fps=240 / display_p50 18.6 ms.
Off-main commits measured immune to main churn: glass p50 ~10 ms,
cadence a clean 4.17 ms at 240. flush() is load-bearing: without it the
render thread's own implicit transaction (drawableSize/colour writes)
swallows the explicit commit and NOTHING reaches glass — the harness
reproduced the exact freeze (every present dropped). Legacy main-hop
kept as PUNKTFUNK_TXN_PRESENT=main for field A/B. New pf-windowed
Console line (subsystem io.unom.punktfunk, category presenter)
decomposes the issue side per second.

Task 1 lever D: the f407f418 IOSurface contents-swap path is
resurrected format-aware as WindowedPresentMode.surface — bgra8 SDR /
rgba16Float+PQ-tagged HDR pool, swap committed off-main from the
completion handler; EDR anchored by the metal layer underneath.
Env-only prototype (PUNKTFUNK_WINDOWED_PRESENT=surface) until the HDR
composite is eyeballed on glass.

Task 2 (setting): punktfunk.windowedSafePresent (default ON =
transactional mitigation; OFF = the fast async path whose panic the
saga is about — the caption says so in plain words). Rows in the touch
Settings (Presentation) and gamepad settings, macOS-only.
PUNKTFUNK_WINDOWED_PRESENT=async|transaction|surface overrides for dev
A/B. maximumDrawableCount stays 3 — the harness measured 2 starving the
render loop (172/s) and worsening glass p50.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:58:47 +02:00
enricobuehlerandClaude Fable 5 71faf38a85 fix(apple): stats os_log used %d for 64-bit Swift Int — macOS 26 drops the line
Swift `Int` is 64-bit (Foundation infers %lld); the stats mirror's format
string used %d (32-bit C int) for fps/presents/lost. macOS 26's strict
String(format:) validator rejects the mismatch and drops the whole line
(the error also cascades onto the float args, mis-blaming floor_p50). Use
%lld for the three Int fields. Pre-existing, unrelated to the DCP work;
surfaced while reading presenter logs during the panic fix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:14:41 +02:00
enricobuehlerandClaude Fable 5 f49d38826b fix(apple): windowed macOS DCP panic via presentsWithTransaction — keep real HDR, drop the SDR IOSurface path
The windowed-macOS "mismatched swapID's" @UnifiedPipeline.cpp kernel
panic is the CAMetalLayer ASYNC image queue (commandBuffer.present,
mandatory with displaySyncEnabled=false) diverging from WindowServer's
compositor on a high-refresh composited session — it survived glass
pacing (PyroWave, 2026-07-18) and hit windowed HEVC on the same 240 Hz
Mac Studio (2026-07-21), so it's the async queue itself, at any codec
or pacing.

f407f418's mitigation routed windowed PyroWave through a BGRA8 IOSurface
layer.contents pool, which cannot carry HDR — extending it to HEVC would
have tone-mapped windowed HDR down to SDR. Instead, present the drawable
through a Core Animation transaction (CAMetalLayer.presentsWithTransaction)
for every windowed session: commit, waitUntilScheduled, then drawable.present()
inside a CATransaction, so the swap commits with the layer tree and stays
in lockstep with the compositor instead of racing it. The full
rgba16Float / PQ / EDR render path is untouched — windowed HDR is real
HDR again. Fullscreen keeps the async path (direct scanout, lowest latency,
no panic there).

Removes the IOSurface surface pool, its surfaceLayer sibling, and the
macOS windowed PQ->SDR tone-map (tvOS keeps its no-headroom tone-map).
Net -150 lines. swift build + 169 tests green. Needs on-glass soak on
the 240 Hz Mac Studio (kernel race — only real hardware confirms).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:45:11 +02:00
enricobuehlerandClaude Fable 5 42198eb1b6 feat(linux/vulkan-encode): default-on native NV12 capture + RGB true-extent
ci / web (push) Successful in 48s
ci / docs-site (push) Successful in 1m5s
apple / swift (push) Successful in 1m20s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 25s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 42s
ci / bench (push) Successful in 5m59s
apple / screenshots (push) Successful in 6m17s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 6m29s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 5m12s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m16s
deb / build-publish (push) Successful in 12m58s
deb / build-publish-host (push) Successful in 13m40s
windows-host / package (push) Successful in 10m46s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m52s
android / android (push) Successful in 16m18s
arch / build-publish (push) Successful in 16m20s
docker / deploy-docs (push) Successful in 30s
ci / rust (push) Successful in 19m25s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 14m19s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m19s
Flip both zero-copy levers from opt-in to default:

- The capture negotiation now PREFERS gamescope's producer-side NV12 pod by
  default (PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation).
  Codec-aware gating rides a new OutputFormat::nv12_native ->
  ZeroCopyPolicy::native_nv12_session edge resolved by the host facade from
  the session plan (pf_encode::linux_native_nv12_ok): only H265/AV1 sessions
  whose backend can open the raw Vulkan Video encoder ever see the NV12 pod
  -- an H264/Moonlight session (libav VAAPI, which would misread the
  two-plane buffer) keeps today's BGRx negotiation, as do the
  GameStream-resolve and portal-mirror paths, PyroWave, and NVENC prefs.
- RGB-direct's unaligned modes default to the true-extent direct import
  (PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 restores the padded-copy staging).
  Guarded-tested on Van Gogh with the kernel journal watched: clean, and at
  5.38 ms p50 the fastest 1080p encode path measured on that hardware. The
  EFC only exists on Mesa >= 26, where the codedExtent-driven session_init
  padding is guaranteed (verified back to Mesa 24.2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehlerandClaude Fable 5 b4fbc94e36 feat(linux/vulkan-encode): PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT bring-up gate
Opt-in alternative to RGB-direct's padded-copy staging at unaligned modes:
direct-import the visible-size capture and declare the TRUE-SIZE source
codedExtent. RADV has derived the VCN session_init firmware padding from
srcPictureResource.codedExtent since Mesa 24.2, so the EFC is told the source
lacks the alignment rows and the hardware edge-extends them internally --
which also reframes the 2026-07-20 field GPU reset: that crash passed the
ALIGNED extent as the source codedExtent (firmware padding zero) over a
visible-size buffer. Off by default until a guarded live test proves the EFC
front-end honors the padding like the YUV fetch path does; the padded-copy
staging remains the shipped behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehlerandClaude Fable 5 004f0cacb8 perf(linux/vulkan-encode): true-size headers make native NV12 zero-copy at every mode
Drop the padded-copy staging for producer-native NV12 and direct-import the
visible-size buffer at unaligned modes (1080p) too. Safe by the driver's own
contract rather than by allocation luck: native sessions now author the H265
SPS / AV1 sequence header at the RENDER size and pass the matching codedExtent
on every picture resource. RADV rounds the bitstream SPS up itself (with a
conformance window -- radv_video_patch_encode_session_parameters, per the
VK_KHR_video_encode_h265 proposal's "implementations may override" clause) and
programs the VCN session_init with the true extent plus nonzero firmware
padding, so the hardware edge-extends the alignment rows internally and never
fetches past the source's real extent -- the exact mechanism VAAPI/radeonsi has
always used to encode 1080p. The driver-emitted header NALs already carry the
patched SPS, so the wire format is unchanged.

The CSC and RGB-direct paths keep the app-aligned convention (their sources
genuinely cover the aligned extent); RGB padded-copy staging is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 14:01:28 +02:00
enricobuehlerandClaude Fable 5 49c6dd00c9 fix(presenter): CPU frames never take the HDR10 passthrough surface
audit / bun-audit (push) Failing after 15s
ci / docs-site (push) Successful in 56s
ci / web (push) Successful in 58s
apple / swift (push) Successful in 1m22s
audit / cargo-audit (push) Successful in 2m20s
decky / build-publish (push) Successful in 32s
ci / bench (push) Successful in 7m0s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 13s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 14s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 7m18s
release / apple (push) Successful in 9m28s
deb / build-publish-host (push) Successful in 10m32s
deb / build-publish (push) Successful in 11m48s
docker / deploy-docs (push) Successful in 23s
android / android (push) Successful in 14m36s
arch / build-publish (push) Successful in 15m4s
windows-host / package (push) Successful in 16m11s
flatpak / build-publish (push) Failing after 8m19s
apple / screenshots (push) Successful in 6m40s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m11s
ci / rust (push) Successful in 19m38s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m29s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m39s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m39s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 5m36s
Software decode uploads swscale RGBA with no CSC/tonemap pass. On the new
desktop-compositor HDR10 surfaces (GNOME 48 / Plasma 6 + Mesa >= 25.1 offer
ST2084 even on SDR desktops) that sRGB-encoded content was composed as PQ —
the field-reported psychedelic picture (Fedora clients decode HEVC in
software because stock Mesa strips it; reproduced + fix live-validated on
GNOME Wayland: mode-0 soup -> HDR→SDR washed-out-but-correct).

The CPU lane keeps the SDR swapchain and its known untonemapped-PQ gap; a
real PQ→sRGB pass for CPU frames is the follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehlerandClaude Fable 5 e7c6c96e5f fix(presenter): set the Wayland app_id so the session window gets its icon
SDL_APP_ID = io.unom.Punktfunk — compositors match it against the shipped
.desktop for the window icon; without it KWin/mutter show the generic Wayland
icon (field-noticed on the Deck). Linux analog of win32 AppUserModelID.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehlerandClaude Fable 5 da134ad045 feat(presenter): PUNKTFUNK_HDR10=0 refuses the HDR10 swapchain (SDR tonemap pin)
GNOME 48 / Plasma 6 with Mesa >= 25.1 now offer HDR10_ST2084 surfaces even on
SDR desktops, silently flipping PQ streams into the mode-0 passthrough lane —
previously unreachable on desktop compositors and never validated there. The
hatch pins the shader tonemap for diagnosis and as the field escape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehlerandClaude Fable 5 428bd2519c test(qsv): converter→ring-profile-P010→encoder live e2e (the RTV-written seam)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehlerandClaude Fable 5 87d7eceabf fix(encode/qsv): write the colour VUI unconditionally + Intel-lane colour diagnostics
The native QSV encoder only attached mfxExtVideoSignalInfo on HDR sessions —
an SDR stream carried no colour description at all (NVENC writes 709-limited
unconditionally; qsv.rs now mirrors it).

Diagnostics for the field-reported blue/magenta Intel-host colours:
- hdr-p010-selftest takes an optional WxH + GPU vendor (intel|nvidia|amd) and
  prints which adapter it tested — 64x64 on the default GPU proved nothing on
  dual-GPU boxes, and the field heights (1080/1400) are not 16-aligned.
- pf-capture: ignored live test running the converter at 1920x1080 pinned to
  the Intel adapter.
- pf-encode: qsv_live_p010_1080_colorbars_dump — known P010 bars through the
  UNALIGNED-height ingest copy (1080 src -> align16 1088 pool, the seam no
  640x480 test exercises) to Main10 HEVC, dumped for off-box decode checks.
- dxgi.rs: drop the stale 'falls back to the R10 path' claim (no such path).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 13:30:35 +02:00
enricobuehlerandClaude Fable 5 1d4795666e feat(linux/vulkan-encode): opt-in gamescope producer-native NV12 encode source
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
decky / build-publish (push) Successful in 19s
apple / swift (push) Successful in 1m21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 10s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 8s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 10s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m53s
apple / screenshots (push) Successful in 6m16s
windows-host / package (push) Successful in 9m38s
deb / build-publish (push) Successful in 9m55s
docker / deploy-docs (push) Successful in 26s
arch / build-publish (push) Successful in 12m52s
android / android (push) Successful in 12m58s
deb / build-publish-host (push) Successful in 14m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m0s
ci / rust (push) Successful in 28m32s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 21m41s
With PUNKTFUNK_PIPEWIRE_NV12=1 (bring-up gate), the PipeWire negotiation
offers an NV12 LINEAR DMA-BUF pod (BT.709 limited pinned MANDATORY) ahead
of the BGRx one, and gamescope's producer-side RGB->YUV pass replaces the
host CSC entirely: the encoder imports the two-plane buffer as a profiled
VIDEO_ENCODE_SRC image and the VCN encodes it directly. Contributed
measurements: encode p99 2.9 ms (from ~4-4.5 ms via EFC RGB-direct),
60 fps capture, 0 send drops.

Hardening on top of the contributed patch:
- unaligned modes (1080p!) stage through a padded aligned NV12 copy (edge
  rows/columns duplicated, transfer-only) instead of direct-importing the
  visible-size buffer -- a direct import would make the VCN read past the
  producer allocation, the exact OOB class behind the 2026-07-20 field GPU
  reset; the encode extents return to the aligned coded extent everywhere
- the UV plane layout honors the producer's plane-1 chunk (offset/stride)
  when the SPA buffer carries one (same-BO verified by inode), with the
  contiguous-plane contract as fallback
- PyroWave sessions are excluded from the gate (their Vulkan compute CSC
  ingests packed RGB), and a native-NV12 session that resolves to libav
  VAAPI (H264 codec, PUNKTFUNK_VULKAN_ENCODE=0, feature off, or a failed
  Vulkan open) refuses at open instead of streaming garbage chroma
- pad staging images carry TRANSFER_SRC (the width-padding pass self-copies
  the staging image -- previously missing on 1366-wide modes)
- metadata-cursor one-shot warn (parity with RGB-direct) and a padded-NV12
  PUNKTFUNK_PERF split label; the padded RGB copy gets its timestamps too

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 11:57:42 +02:00
enricobuehlerandClaude Opus 4.8 58b27acbd5 fix(vdisplay/session): scrub the dead desktop's WAYLAND_DISPLAY from the user-manager env
ci / web (push) Successful in 1m2s
ci / docs-site (push) Successful in 1m14s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 27s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 19s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 17s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 12s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
ci / bench (push) Successful in 5m44s
apple / screenshots (push) Successful in 6m18s
android / android (push) Successful in 13m0s
docker / deploy-docs (push) Successful in 27s
deb / build-publish (push) Successful in 13m36s
deb / build-publish-host (push) Successful in 14m44s
windows-host / package (push) Successful in 16m32s
arch / build-publish (push) Successful in 17m49s
ci / rust (push) Successful in 19m58s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m35s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 19m42s
settle_desktop_portal pushes the live desktop's WAYLAND_DISPLAY into the
systemd --user manager (import-environment) so a re-activated portal inherits
the right session — but the import PERSISTS after that desktop dies. Every
later user unit then inherits the stale socket, including
gamescope-session.target: gamescope sees WAYLAND_DISPLAY, runs NESTED, and
aborts with "Failed to connect to wayland socket: wayland-0" — observed live
on a Deck, where Game Mode could not start at all (autologin crash-looped)
until the var was unset by hand.

Two-layer fix, mirroring the protection launch_session already gives its
transient unit:
- observe_session_instance: when a desktop compositor instance goes away,
  unset WAYLAND_DISPLAY/DISPLAY from the manager env (a desktop bounce is
  harmless — the next portal settle re-imports).
- write_steamos_dropin: UnsetEnvironment=DISPLAY WAYLAND_DISPLAY on the
  takeover drop-in, unit-scoped belt-and-suspenders for host-initiated
  restarts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:06:55 +02:00
enricobuehlerandClaude Opus 4.8 333c8979e9 fix(vdisplay/gamescope): capture-loss rebuild probes must not steal the box session
Observed live on a Deck host: the user switched Game Mode → Desktop, KDE came
up in 0.8 s — and the capture-loss rebuild's FIRST detection ran inside that
gap, still said Gaming, and its gamescope re-acquire restarted
gamescope-session.target. On SteamOS that steals the seat: the user was
yanked out of the KDE session they had just chosen, the two session managers
fought (job canceled / dependency failed), no gamescope node appeared within
30 s, and the stream died at the 40 s rebuild budget.

A rebuild probe acting on possibly-stale detection must never stop, relaunch,
or take over box sessions. New pf_vdisplay::rebuild_probe_scope (RAII,
counted): while held, the gamescope managed/takeover create paths only attach
to a live node and fail fast otherwise — no stop_autologin_sessions, no
session-plus relaunch, no gamescope-session.target restart. The capture-loss
loop holds it for the first 4 s after a loss (the detection-ambiguity
window); after that, destructive rebuilds are allowed again, so a genuine
switch INTO Game Mode still gets its headless takeover. Probe failures are
instant, so the loop now paces itself at ~2 Hz instead of spinning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:00:02 +02:00
enricobuehlerandClaude Opus 4.8 46d09ed973 fix(host/linux): stop burning retry budget on doomed pipeline builds
Two session-transition stalls found live on a SteamOS Deck host, one cause:
the pipeline retry loop couldn't tell a wait-it-out failure from a
retry-now-and-it-works one.

- Gamescope first-frame race: a PipeWire stream connected while gamescope
  re-initializes its headless takeover negotiates a format, reaches
  Streaming, and never receives a buffer — while a fresh connect delivers
  within ~0.5 s. Every gamescope bring-up ate the full 10 s first-frame
  budget on attempt 1 (17 s bring-ups; KWin: 1.2 s). The retry loop's FIRST
  attempt now waits 2.5 s (Capturer::next_frame_within), so the reconnect
  that fixes the race happens at ~3 s. Later attempts keep the patient 10 s
  — the documented 30-60 s Big Picture cold start still fits the budget.

- Capture-loss rebuild vs session switch: the rebuild loop re-detects the
  active session between build_pipeline_with_retry calls, but each call
  burned 8 attempts (~13 s) against a compositor that no longer exists (a
  Desktop→Gaming switch spent 13 of its 27 s retrying gone-KWin). The
  capture-loss path now passes max_attempts=2, turning the outer loop into
  ~1 s detect-and-retry cycles that follow the box to the new session.

The resize path's direct build_pipeline call keeps the default budget (no
retry wrapper there to absorb an early bail).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 10:34:49 +02:00
enricobuehlerandClaude Opus 4.8 946f1b3d69 fix(steamdeck): build the host with vulkan-encode, and wire up what it needs to actually engage
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 57s
apple / swift (push) Successful in 1m20s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 11s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 6m43s
apple / screenshots (push) Successful in 6m32s
deb / build-publish (push) Successful in 9m57s
docker / deploy-docs (push) Successful in 26s
arch / build-publish (push) Successful in 12m46s
android / android (push) Successful in 13m40s
deb / build-publish-host (push) Successful in 13m3s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 20m42s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 18m59s
ci / rust (push) Successful in 26m41s
The SteamOS scripts built the host featureless, so PlatformAuto could never
pick the Vulkan Video backend (default-on since 84329205) and every session
silently ran libav VAAPI — unlike the deb/arch/rpm/sysext builds, which all
pass the feature. Three gaps, found live on a Deck host:

- install.sh/update.sh: cargo build with --features punktfunk-host/vulkan-encode
  (matches the packaged builds; pure-Rust ash, no new system dep).
- host.env: RADV_PERFTEST=video_encode — Van Gogh RADV still gates
  VK_KHR_video_encode_* behind it, so without it the backend can't open and
  falls back to VAAPI. Harmless where encode is exposed by default.
- io.unom.Punktfunk.Host.desktop (Exec rewritten to this install's binary):
  KWin only grants zkde_screencast_unstable_v1/org_kde_kwin_fake_input to an
  exe a .desktop authorizes — without it Desktop-Mode capture fails outright
  and a mid-stream Game↔Desktop switch strands the stream on the old backend.

update.sh retrofits the last two idempotently so existing installs get them
without a reinstall.

Validated on a SteamOS 3.8.14 Deck (Van Gogh): Vulkan HEVC opened on both the
KWin and gamescope paths at 5120x1440 — in-place ABR reconfigure (no IDR, no
encoder rebuild) vs the VAAPI path's full teardown per ABR step, and Desktop
capture came up without a re-login.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 10:14:31 +02:00
169 changed files with 13756 additions and 2803 deletions
+12 -2
View File
@@ -46,13 +46,23 @@ jobs:
# SHA-pinned: this workflow's release job carries the signing keystore + Play service-account # SHA-pinned: this workflow's release job carries the signing keystore + Play service-account
# secrets, so a moved tag on a third-party action could exfiltrate them. v3 = 9fc6c4e. # secrets, so a moved tag on a third-party action could exfiltrate them. v3 = 9fc6c4e.
uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3 uses: android-actions/setup-android@9fc6c4e9069bf8d3d10b2204b1fb8f6ef7065407 # v3
with:
# Only platform-tools — NOT the action's default legacy `tools`, whose dependency chain
# drags in the ~250 MB emulator nobody here runs (instrumentation tests are deferred).
# That download was the single flakiest piece of this job: the shared runner fleet drops
# packets under parallel-job load and sdkmanager's streamed unzip turns a truncated
# stream into "Error on ZipFile unknown archive" (observed 2026-07-22, twice).
packages: platform-tools
- name: NDK r30 + platform 36 + build-tools + CMake (libopus cross-build) - name: NDK r30 + platform 36 + build-tools + CMake (libopus cross-build)
# cmake;3.22.1 installs cmake + ninja under $ANDROID_SDK/cmake/3.22.1/bin — the exact path # cmake;3.22.1 installs cmake + ninja under $ANDROID_SDK/cmake/3.22.1/bin — the exact path
# kit/build.gradle.kts prepends to PATH for cargo-ndk's audiopus_sys (libopus) CMake build. # kit/build.gradle.kts prepends to PATH for cargo-ndk's audiopus_sys (libopus) CMake build.
# Note: platforms;android-37 is sometimes missing from standard channels; AGP will # Note: platforms;android-37 is sometimes missing from standard channels; AGP will
# auto-download it if needed during the build. # auto-download it if needed during the build.
run: sdkmanager "platform-tools" "platforms;android-36" "build-tools;37.0.0" "ndk;30.0.14904198" "cmake;3.22.1" # retry.sh: sdkmanager is a single-shot multi-hundred-MB fetch, exactly the class the
# helper exists for (fleet-load packet drops truncate the stream mid-unzip); a failed
# attempt leaves no partial package behind, so a plain re-invoke is safe.
run: bash scripts/ci/retry.sh 4 sdkmanager "platform-tools" "platforms;android-36" "build-tools;37.0.0" "ndk;30.0.14904198" "cmake;3.22.1"
- name: Caches (cargo + gradle) - name: Caches (cargo + gradle)
uses: actions/cache@v4 uses: actions/cache@v4
+24 -30
View File
@@ -6,17 +6,12 @@
# #
# The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT # The plugin backend is PURE PYTHON (clients/decky/main.py — no compiled binary), so we do NOT
# need the Decky CLI (which requires Docker + rust-nightly only to compile native backends). # need the Decky CLI (which requires Docker + rust-nightly only to compile native backends).
# We build the frontend with pnpm and assemble the store-layout zip by hand: # We build the frontend with pnpm and stage the store-layout tree with the SAME script local
# # builds use (clients/decky/scripts/package.sh) — the plugin's file list lives in exactly ONE
# punktfunk.zip # place, so a file added there (bin/, assets/, controller_config/, …) can never be silently
# punktfunk/ <- single top-level dir == plugin.json "name" # missing from the published build. (Hand-assembling the zip here is how the shipped plugin
# plugin.json [required] # lost the shortcut artwork + Steam Input layout for a while.) CI only adds `update.json` on
# package.json [required; CI stamps "version" — Decky reads the installed version here] # top: the {channel, manifest} pointer the plugin's self-update check polls.
# main.py [required: python backend]
# dist/index.js [required: rollup output]
# update.json [CI-baked {channel, manifest}: where the plugin's self-update check polls]
# README.md (recommended)
# LICENSE [required by the plugin store]
# #
# SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel # SELF-UPDATE (no Decky store): alongside the zip we also publish a tiny per-channel
# `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed # `manifest.json` ({version, artifact=<immutable per-version zip URL>, sha256}). The installed
@@ -90,28 +85,27 @@ jobs:
- name: Assemble store-layout zip - name: Assemble store-layout zip
working-directory: ${{ gitea.workspace }} working-directory: ${{ gitea.workspace }}
run: | run: |
apt-get update && apt-get install -y --no-install-recommends zip >/dev/null # node:22-bookworm ships python3 (a package.sh dep) but not zip; install both anyway
STAGE="$RUNNER_TEMP/decky" # so an image change can't silently break the build.
DEST="$STAGE/$PLUGIN" apt-get update && apt-get install -y --no-install-recommends zip python3 >/dev/null
rm -rf "$STAGE"; mkdir -p "$DEST/dist" "$DEST/bin" # Stage the canonical plugin tree (dist/, main.py, bin/, assets/, controller_config/,
cp clients/decky/plugin.json "$DEST/" # LICENSE, …) with the same script local/sideload builds use — see the header comment.
cp clients/decky/package.json "$DEST/" # Runs AFTER the version stamp, so the staged package.json carries $VERSION.
cp clients/decky/main.py "$DEST/" bash clients/decky/scripts/package.sh
cp clients/decky/dist/index.js "$DEST/dist/" DEST="clients/decky/out/$PLUGIN"
cp clients/decky/README.md "$DEST/" # CI-only addition: the self-update channel pointer the backend reads (main.py
# The stream-launch wrapper (target of the Steam shortcut); keep it executable # check_update). It points at THIS channel's manifest.json (published below); that
# (runner_info() also re-chmods at runtime in case the zip/extract drops the bit). # manifest in turn points at the immutable per-version zip, so its sha256 stays valid
cp clients/decky/bin/punktfunkrun.sh "$DEST/bin/" # across future alias re-uploads.
chmod 0755 "$DEST/bin/punktfunkrun.sh"
# Store requires a LICENSE in the plugin root; the project is MIT OR Apache-2.0.
cp LICENSE-MIT "$DEST/LICENSE"
# Self-update channel pointer the backend reads (main.py check_update). It points at
# THIS channel's manifest.json (published below); that manifest in turn points at the
# immutable per-version zip, so its sha256 stays valid across future alias re-uploads.
printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json" printf '{"channel":"%s","manifest":"%s/%s/manifest.json"}\n' "$ALIAS" "$BASE" "$ALIAS" > "$DEST/update.json"
( cd "$STAGE" && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" ) ( cd clients/decky/out && zip -r "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN" )
ls -lh "$RUNNER_TEMP/punktfunk.zip" ls -lh "$RUNNER_TEMP/punktfunk.zip"
unzip -l "$RUNNER_TEMP/punktfunk.zip" unzip -l "$RUNNER_TEMP/punktfunk.zip"
# Backstop against packaging drift: the runtime-loaded pieces MUST be in the zip.
for f in main.py dist/index.js bin/punktfunkrun.sh assets/grid.png \
controller_config/punktfunk.vdf update.json; do
unzip -l "$RUNNER_TEMP/punktfunk.zip" "$PLUGIN/$f" >/dev/null || { echo "MISSING $f" >&2; exit 1; }
done
# The update manifest the plugin polls: the immutable per-version artifact + its # The update manifest the plugin polls: the immutable per-version artifact + its
# sha256 (Decky's installer verifies the download against this hash, aborting on # sha256 (Decky's installer verifies the download against this hash, aborting on
# mismatch — so it MUST be the per-version URL, never the mutable alias). # mismatch — so it MUST be the per-version URL, never the mutable alias).
+12 -25
View File
@@ -73,33 +73,20 @@ jobs:
# sufficient — the Tooling step's dnf install pulls a systemd package upgrade whose RPM # sufficient — the Tooling step's dnf install pulls a systemd package upgrade whose RPM
# trigger re-runs authselect and regenerates this file, undoing the fix. It's reapplied # trigger re-runs authselect and regenerates this file, undoing the fix. It's reapplied
# there, right before the first `flatpak` network call. # there, right before the first `flatpak` network call.
- name: Fix container DNS (drop nss-resolve, resolve over TCP) - name: Fix container DNS (drop nss-resolve)
run: | run: |
sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf
# Resolve over TCP instead of UDP. The documented root cause of the flathub # History: this step used to ALSO force glibc onto TCP DNS (`options use-vc`) because
# bootstrap failures (investigated 2026-07-11, see the Tooling step) is this box's # the runner fleet's Docker embedded resolver dropped UDP lookups under parallel-job
# Docker embedded resolver at 127.0.0.11 DROPPING UDP lookups while the shared # load (investigated 2026-07-11; v0.15.0/v0.16.0 each burned retry.sh's whole budget).
# runner fleet is saturated — a datagram nobody retransmits, so the lookup just # That root cause is now fixed at the infra level (2026-07-22): the runner host runs a
# times out. The answer then was to widen retry.sh's budget to 10 attempts (~9 min), # local dnsmasq cache on the docker bridge and daemon.json points every job container
# which is enough to outlast a main push's ~8-workflow fan-out but NOT a TAG push's # at it, so lookups terminate on-box instead of crossing the saturated uplink — the
# 13: v0.15.0 (twice) and v0.16.0 each burned all 10 attempts and failed the job, # UDP path is reliable again. The TCP path through the same chain proved FLAKY under
# each needing a manual re-run. # fleet concurrency (flatpak remote-add failed 10/10 with instant NXDOMAIN while dnf
# # in the same container resolved fine), so `use-vc` flipped from mitigation to sole
# `use-vc` makes glibc use TCP, where the kernel retransmits and the query cannot be # cause of this leg's failures — removed. retry.sh (10×) stays as the backstop for
# silently lost under load. Same resolver, same search path — only the transport # genuine upstream blips.
# changes, so internal names (git.unom.io) resolve exactly as before; deliberately
# NO extra nameservers, which would risk answering an internal name from a public
# resolver. Docker's embedded DNS serves TCP on 127.0.0.11:53 as well as UDP.
# retry.sh stays as the backstop for genuine upstream blips.
#
# Non-fatal: Docker bind-mounts /etc/resolv.conf and can present it read-only, and a
# DNS tuning that cannot be applied must not be what fails the release build — that
# would trade an occasional re-run for a hard stop. Falling back to UDP just restores
# today's behaviour, which retry.sh already covers.
if ! grep -q '^options .*use-vc' /etc/resolv.conf 2>/dev/null; then
echo 'options use-vc timeout:3 attempts:3' >> /etc/resolv.conf \
|| echo "::warning::could not set use-vc (read-only resolv.conf?); staying on UDP"
fi
cat /etc/resolv.conf || true cat /etc/resolv.conf || true
# fedora:43 has no node, but actions/checkout (a JS action) needs it. A plain `run:` step # fedora:43 has no node, but actions/checkout (a JS action) needs it. A plain `run:` step
Generated
+93 -27
View File
@@ -656,6 +656,30 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724" checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chacha20"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3613f74bd2eac03dad61bd53dbe620703d4371614fe0bc3b9f04dd36fe4e818"
dependencies = [
"cfg-if",
"cipher",
"cpufeatures",
]
[[package]]
name = "chacha20poly1305"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10cd79432192d1c0f4e1a0fef9527696cc039165d729fb41b3f4f4f354c2dc35"
dependencies = [
"aead",
"chacha20",
"cipher",
"poly1305",
"zeroize",
]
[[package]] [[package]]
name = "ciborium" name = "ciborium"
version = "0.2.2" version = "0.2.2"
@@ -691,6 +715,7 @@ checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad"
dependencies = [ dependencies = [
"crypto-common", "crypto-common",
"inout", "inout",
"zeroize",
] ]
[[package]] [[package]]
@@ -1434,6 +1459,16 @@ dependencies = [
"version_check", "version_check",
] ]
[[package]]
name = "gethostname"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1bd49230192a3797a9a4d6abe9b3eed6f7fa4c8a8a4947977c6f80025f92cbd8"
dependencies = [
"rustix",
"windows-link 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]] [[package]]
name = "getrandom" name = "getrandom"
version = "0.2.17" version = "0.2.17"
@@ -2159,7 +2194,7 @@ dependencies = [
[[package]] [[package]]
name = "latency-probe" name = "latency-probe"
version = "0.17.0" version = "0.18.0"
[[package]] [[package]]
name = "lazy_static" name = "lazy_static"
@@ -2264,7 +2299,7 @@ dependencies = [
[[package]] [[package]]
name = "libvpl-sys" name = "libvpl-sys"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
@@ -2299,7 +2334,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad"
[[package]] [[package]]
name = "loss-harness" name = "loss-harness"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"punktfunk-core", "punktfunk-core",
] ]
@@ -2788,7 +2823,7 @@ checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]] [[package]]
name = "pf-capture" name = "pf-capture"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2804,11 +2839,12 @@ dependencies = [
"tokio", "tokio",
"tracing", "tracing",
"windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)", "windows 0.62.2 (registry+https://github.com/rust-lang/crates.io-index)",
"x11rb",
] ]
[[package]] [[package]]
name = "pf-client-core" name = "pf-client-core"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2832,7 +2868,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-clipboard" name = "pf-clipboard"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2850,7 +2886,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-console-ui" name = "pf-console-ui"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2871,7 +2907,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-encode" name = "pf-encode"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2881,6 +2917,7 @@ dependencies = [
"libvpl-sys", "libvpl-sys",
"nvidia-video-codec-sdk", "nvidia-video-codec-sdk",
"openh264", "openh264",
"pf-capture",
"pf-frame", "pf-frame",
"pf-gpu", "pf-gpu",
"pf-host-config", "pf-host-config",
@@ -2894,7 +2931,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-ffvk" name = "pf-ffvk"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"ash", "ash",
"bindgen", "bindgen",
@@ -2903,7 +2940,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-frame" name = "pf-frame"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"libc", "libc",
@@ -2915,7 +2952,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-gpu" name = "pf-gpu"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-host-config", "pf-host-config",
@@ -2929,11 +2966,11 @@ dependencies = [
[[package]] [[package]]
name = "pf-host-config" name = "pf-host-config"
version = "0.17.0" version = "0.18.0"
[[package]] [[package]]
name = "pf-inject" name = "pf-inject"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -2961,14 +2998,14 @@ dependencies = [
[[package]] [[package]]
name = "pf-paths" name = "pf-paths"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"tracing", "tracing",
] ]
[[package]] [[package]]
name = "pf-presenter" name = "pf-presenter"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -2983,7 +3020,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-vdisplay" name = "pf-vdisplay"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ashpd", "ashpd",
@@ -3013,7 +3050,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-win-display" name = "pf-win-display"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-paths", "pf-paths",
@@ -3025,7 +3062,7 @@ dependencies = [
[[package]] [[package]]
name = "pf-zerocopy" name = "pf-zerocopy"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ash", "ash",
@@ -3136,6 +3173,17 @@ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
[[package]]
name = "poly1305"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8159bd90725d2df49889a078b54f4f79e87f1f8a8444194cdca81d38f5393abf"
dependencies = [
"cpufeatures",
"opaque-debug",
"universal-hash",
]
[[package]] [[package]]
name = "polyval" name = "polyval"
version = "0.6.2" version = "0.6.2"
@@ -3221,7 +3269,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-android" name = "punktfunk-client-android"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"android_logger", "android_logger",
"jni", "jni",
@@ -3237,7 +3285,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-linux" name = "punktfunk-client-linux"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-channel", "async-channel",
@@ -3253,7 +3301,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-session" name = "punktfunk-client-session"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"pf-client-core", "pf-client-core",
@@ -3268,7 +3316,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-client-windows" name = "punktfunk-client-windows"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"async-channel", "async-channel",
"ffmpeg-next", "ffmpeg-next",
@@ -3287,11 +3335,12 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-core" name = "punktfunk-core"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"aes-gcm", "aes-gcm",
"bytes", "bytes",
"cbindgen", "cbindgen",
"chacha20poly1305",
"criterion", "criterion",
"fec-rs", "fec-rs",
"hmac", "hmac",
@@ -3318,7 +3367,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-host" name = "punktfunk-host"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"aes", "aes",
"aes-gcm", "aes-gcm",
@@ -3402,7 +3451,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-probe" name = "punktfunk-probe"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"mdns-sd", "mdns-sd",
@@ -3416,7 +3465,7 @@ dependencies = [
[[package]] [[package]]
name = "punktfunk-tray" name = "punktfunk-tray"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"ksni", "ksni",
@@ -3439,7 +3488,7 @@ checksum = "d55d956fa96f5ec02be2e13af0e20391a5aa83d6a074e3ad368959d0fab299ea"
[[package]] [[package]]
name = "pyrowave-sys" name = "pyrowave-sys"
version = "0.17.0" version = "0.18.0"
dependencies = [ dependencies = [
"bindgen", "bindgen",
"cmake", "cmake",
@@ -5848,6 +5897,23 @@ version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4" checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4"
[[package]]
name = "x11rb"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9993aa5be5a26815fe2c3eacfc1fde061fc1a1f094bf1ad2a18bf9c495dd7414"
dependencies = [
"gethostname",
"rustix",
"x11rb-protocol",
]
[[package]]
name = "x11rb-protocol"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea6fc2961e4ef194dcbfe56bb845534d0dc8098940c7e5c012a258bfec6701bd"
[[package]] [[package]]
name = "x509-parser" name = "x509-parser"
version = "0.16.0" version = "0.16.0"
+1 -1
View File
@@ -48,7 +48,7 @@ exclude = [
ndk = { path = "clients/android/native/vendor/ndk" } ndk = { path = "clients/android/native/vendor/ndk" }
[workspace.package] [workspace.package]
version = "0.17.0" version = "0.18.0"
edition = "2021" edition = "2021"
rust-version = "1.82" rust-version = "1.82"
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
+1 -1
View File
@@ -10,7 +10,7 @@
"name": "MIT OR Apache-2.0", "name": "MIT OR Apache-2.0",
"identifier": "MIT OR Apache-2.0" "identifier": "MIT OR Apache-2.0"
}, },
"version": "0.17.0" "version": "0.18.0"
}, },
"paths": { "paths": {
"/api/v1/clients": { "/api/v1/clients": {
@@ -0,0 +1,107 @@
package io.unom.punktfunk
import android.content.ClipData
import android.content.ClipboardManager
import android.content.Context
import android.os.Handler
import android.os.Looper
import io.unom.punktfunk.kit.NativeBridge
/**
* Text clipboard sync for the active session (the desktop-client model, text-only v1):
* * **Device → host**: a local copy (the primary-clip listener, plus one probe at start) is
* announced as a lazy offer — the text crosses only when the host actually pastes (a
* `fetch:` event, answered with the clipboard's current content).
* * **Host → device**: a host copy arrives as an `offer:` event and is fetched eagerly into
* the system clipboard (Android apps can't lazily materialize a paste from the network
* without a content-provider round-trip that isn't worth it here).
*
* Loop guard: text set from a host fetch is remembered ([lastFromHost]) so the resulting
* primary-clip-changed callback doesn't bounce it straight back as a new offer. Clipboard reads
* happen while the stream is foreground (Android only allows focused-app reads). The native
* events are drained on a dedicated thread and applied on the main thread; [stop] joins it.
*/
class ClipboardSync(
private val context: Context,
private val handle: Long,
) {
private val main = Handler(Looper.getMainLooper())
private val cm = context.getSystemService(Context.CLIPBOARD_SERVICE) as ClipboardManager
@Volatile private var running = true
private var seq = 0
private var lastOffered: String? = null
private var lastFromHost: String? = null
private var pendingFetch = -1
private var thread: Thread? = null
private val clipListener = ClipboardManager.OnPrimaryClipChangedListener { offerLocal() }
fun start() {
NativeBridge.nativeClipControl(handle, true)
cm.addPrimaryClipChangedListener(clipListener)
thread = Thread({ pollLoop() }, "pf-clipboard").also { it.start() }
offerLocal() // whatever is already on the clipboard is pasteable host-side right away
}
fun stop() {
running = false
cm.removePrimaryClipChangedListener(clipListener)
thread?.join(600) // one poll timeout (250 ms) + slack
thread = null
}
/** Announce the current local text (if it's new and not an echo of a host copy). */
private fun offerLocal() {
if (!running) return
val text = currentClipText() ?: return
if (text == lastOffered || text == lastFromHost) return
lastOffered = text
seq += 1
NativeBridge.nativeClipOfferText(handle, seq)
}
private fun currentClipText(): String? = runCatching {
cm.primaryClip?.takeIf { it.itemCount > 0 }?.getItemAt(0)
?.coerceToText(context)?.toString()?.takeIf { it.isNotEmpty() }
}.getOrNull()
private fun pollLoop() {
while (running) {
val ev = NativeBridge.nativeNextClip(handle) ?: continue
if (ev == "closed") return
main.post { handleEvent(ev) }
}
}
private fun handleEvent(ev: String) {
if (!running) return
val parts = ev.split(":", limit = 3)
when (parts[0]) {
"offer" -> {
val offerSeq = parts.getOrNull(1)?.toIntOrNull() ?: return
if (parts.getOrNull(2) == "1") {
pendingFetch = NativeBridge.nativeClipFetchText(handle, offerSeq)
}
}
"fetch" -> {
val req = parts.getOrNull(1)?.toIntOrNull() ?: return
val text = currentClipText()
if (text != null) {
NativeBridge.nativeClipServeText(handle, req, text)
} else {
NativeBridge.nativeClipCancel(handle, req)
}
}
"data" -> {
val xfer = parts.getOrNull(1)?.toIntOrNull() ?: return
if (xfer != pendingFetch) return // stale/unknown transfer
pendingFetch = -1
val text = parts.getOrNull(2)?.takeIf { it.isNotEmpty() } ?: return
lastFromHost = text
runCatching { cm.setPrimaryClip(ClipData.newPlainText("Punktfunk", text)) }
}
// "state"/"cancel"/"error": nothing to drive in the text-only v1.
}
}
}
@@ -54,6 +54,21 @@ class MainActivity : ComponentActivity() {
var padKeyProbe: ((KeyEvent) -> Boolean)? = null var padKeyProbe: ((KeyEvent) -> Boolean)? = null
var padMotionProbe: ((MotionEvent) -> Boolean)? = null var padMotionProbe: ((MotionEvent) -> Boolean)? = null
/**
* Physical-mouse forwarder for the active session (built/released by StreamScreen, like
* [gamepadRouter]): uncaptured hover/click/wheel forwards as absolute cursor input, captured
* ([android.view.View.requestPointerCapture]) raw deltas as relative mouse-look. The dispatch
* overrides below route every SOURCE_MOUSE event here while streaming. Null while not streaming.
*/
var mouseForwarder: MouseForwarder? = null
/**
* TV remote-as-pointer for the active session (StreamScreen builds it on TV devices only):
* hold SELECT to toggle, then the D-pad glides the host cursor. Consulted first for
* non-gamepad keys while streaming. Null while not streaming or not a TV.
*/
var remotePointer: RemotePointer? = null
/** /**
* Set by [StreamScreen] to its disconnect action. The emergency-exit chord (below) invokes it so a * Set by [StreamScreen] to its disconnect action. The emergency-exit chord (below) invokes it so a
* couch user with no keyboard/Back can always leave a stream. * couch user with no keyboard/Back can always leave a stream.
@@ -324,9 +339,29 @@ class MainActivity : ComponentActivity() {
return true // consumed return true // consumed
} }
} }
// TV remote-as-pointer sees non-gamepad keys first (SELECT long-press toggles it;
// while active it owns the D-pad/SELECT/PLAY-PAUSE/BACK).
if (!event.isFromSource(InputDevice.SOURCE_GAMEPAD)) {
remotePointer?.let { if (it.onKey(event)) return true }
}
// Ctrl+Alt+Shift+Q — the cross-client pointer-capture toggle chord. Swallow both
// edges of the Q (the modifiers already went over the wire, exactly like desktop).
if (event.keyCode == KeyEvent.KEYCODE_Q &&
event.isCtrlPressed && event.isAltPressed && event.isShiftPressed
) {
if (event.action == KeyEvent.ACTION_DOWN && event.repeatCount == 0) {
mouseForwarder?.toggleCapture()
}
return true
}
when (event.keyCode) { when (event.keyCode) {
// A mouse back/forward button whose BUTTON_* press went unconsumed makes the
// framework synthesize a FALLBACK BACK — the button already went over the wire
// as X1/X2, and it must never yank the user out of the stream.
KeyEvent.KEYCODE_BACK ->
if (event.flags and KeyEvent.FLAG_FALLBACK != 0) return true
// Leave these to the system even while streaming. // Leave these to the system even while streaming.
KeyEvent.KEYCODE_BACK, // → BackHandler leaves the stream // (BACK above → BackHandler leaves the stream.)
KeyEvent.KEYCODE_VOLUME_UP, KeyEvent.KEYCODE_VOLUME_UP,
KeyEvent.KEYCODE_VOLUME_DOWN, KeyEvent.KEYCODE_VOLUME_DOWN,
KeyEvent.KEYCODE_VOLUME_MUTE, KeyEvent.KEYCODE_VOLUME_MUTE,
@@ -394,6 +429,10 @@ class MainActivity : ComponentActivity() {
override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean { override fun dispatchGenericMotionEvent(event: MotionEvent): Boolean {
if (streamHandle != 0L) { if (streamHandle != 0L) {
if (gamepadRouter?.onMotion(event) == true) return true if (gamepadRouter?.onMotion(event) == true) return true
// Physical mouse (uncaptured): hover motion, wheel, button edges.
if (event.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onGenericMotion(event)) return true }
}
return super.dispatchGenericMotionEvent(event) return super.dispatchGenericMotionEvent(event)
} }
// The Controllers debug screen sees pad motion before the stick→D-pad synthesis below. // The Controllers debug screen sees pad motion before the stick→D-pad synthesis below.
@@ -431,6 +470,24 @@ class MainActivity : ComponentActivity() {
return super.dispatchGenericMotionEvent(event) return super.dispatchGenericMotionEvent(event)
} }
/**
* Mouse clicks/drags ride the TOUCH stream (the pointer is "down"). While streaming they
* belong to the mouse forwarder, never to the Compose touch-gesture layer — a physical
* mouse click must be a real click at the cursor, not a synthesized trackpad tap.
*/
override fun dispatchTouchEvent(ev: MotionEvent): Boolean {
if (streamHandle != 0L && ev.isFromSource(InputDevice.SOURCE_MOUSE)) {
mouseForwarder?.let { if (it.onTouchEvent(ev)) return true }
}
return super.dispatchTouchEvent(ev)
}
/** The OS is the source of truth for pointer capture (it releases on focus loss). */
override fun onPointerCaptureChanged(hasCapture: Boolean) {
super.onPointerCaptureChanged(hasCapture)
mouseForwarder?.onCaptureChanged(hasCapture)
}
/** Keys that drive the console UI — D-pad + face buttons; used to classify the last input source. */ /** Keys that drive the console UI — D-pad + face buttons; used to classify the last input source. */
private fun isConsoleNavKey(kc: Int): Boolean = when (kc) { private fun isConsoleNavKey(kc: Int): Boolean = when (kc) {
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN, KeyEvent.KEYCODE_DPAD_LEFT, KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN, KeyEvent.KEYCODE_DPAD_LEFT,
@@ -0,0 +1,206 @@
package io.unom.punktfunk
import android.view.InputDevice
import android.view.MotionEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.roundToInt
/** True when any connected input device is a pointer (USB/BT mouse, or a touchpad driving one). */
fun hasPhysicalMouse(): Boolean = InputDevice.getDeviceIds().any { id ->
InputDevice.getDevice(id)?.supportsSource(InputDevice.SOURCE_MOUSE) == true
}
/**
* Physical mouse → wire, in two modes (the iPadOS/desktop model):
* * **uncaptured** (default): hover/drag positions forward as absolute cursor moves
* (`MouseMoveAbs`, host-normalized against the window size) — desktop-style pointing. The
* local cursor is hidden over the stream (StreamScreen sets a TYPE_NULL pointer icon); the
* host's own cursor, composited into the video, is the one you see.
* * **captured**: the OS pointer is grabbed ([android.view.View.requestPointerCapture]) and raw
* relative deltas forward as `MouseMove` — FPS mouse-look. Engaged at stream start / by
* clicking into the stream when the "Capture pointer for games" setting is on, and toggled
* any time by Ctrl+Alt+Shift+Q (the cross-client chord). Focus loss releases it (the OS
* guarantees that); a click re-engages.
*
* Buttons ride [MotionEvent.ACTION_BUTTON_PRESS]/RELEASE edges (left/middle/right/back/forward →
* wire 1/2/3/4/5), the wheel rides [MotionEvent.ACTION_SCROLL] with fractional accumulation so
* high-resolution wheels don't lose sub-notch travel. Held buttons are tracked and flushed on
* capture loss / stream exit so nothing sticks on the host. Events reach this class from
* MainActivity's dispatch overrides (uncaptured) and the capture view's captured-pointer listener.
*/
class MouseForwarder(
private val handle: Long,
private val invertScroll: Boolean,
private val captureWanted: Boolean,
private val surfaceSize: () -> Pair<Int, Int>,
) {
/** Capture plumbing, owned by StreamScreen (the focusable capture view). */
var onRequestCapture: (() -> Unit)? = null
var onReleaseCapture: (() -> Unit)? = null
/** Live capture state, updated from [android.app.Activity.onPointerCaptureChanged]. */
var captured = false
private set
/** Chord-released: no auto re-engage (start / click) until the user opts back in. */
private var userReleased = false
private val heldButtons = mutableSetOf<Int>()
private var scrollAccV = 0f
private var scrollAccH = 0f
private var moveAccX = 0f
private var moveAccY = 0f
/** Uncaptured mouse events on the TOUCH stream (position while a button is down). */
fun onTouchEvent(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_DOWN -> {
if (captureWanted && !captured && !userReleased) {
// The engaging click: grab the pointer and swallow the click (desktop
// parity — the click that captures never reaches the host). The paired
// BUTTON_RELEASE is dropped by the held-set guard in [button].
onRequestCapture?.invoke()
return true
}
sendAbs(ev)
}
MotionEvent.ACTION_MOVE -> sendAbs(ev)
// Button edges are documented on the generic stream, but be robust to either.
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
}
return true
}
/** Uncaptured mouse events on the GENERIC stream (hover motion, wheel, button edges). */
fun onGenericMotion(ev: MotionEvent): Boolean {
when (ev.actionMasked) {
MotionEvent.ACTION_HOVER_MOVE -> sendAbs(ev)
MotionEvent.ACTION_SCROLL -> wheel(ev)
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_HOVER_ENTER, MotionEvent.ACTION_HOVER_EXIT -> {}
else -> return false
}
return true
}
/**
* Captured-pointer events (the view holds [android.view.View.requestPointerCapture]): x/y ARE
* the relative deltas ([InputDevice.SOURCE_MOUSE_RELATIVE]), batched samples included. A
* captured touchpad reports absolute finger coordinates instead — not handled (the touch
* gesture layer is the touchpad story); returning false leaves those to the framework.
*/
fun onCapturedPointer(ev: MotionEvent): Boolean {
if (!ev.isFromSource(InputDevice.SOURCE_MOUSE_RELATIVE)) return false
when (ev.actionMasked) {
MotionEvent.ACTION_MOVE -> {
var dx = 0f
var dy = 0f
for (i in 0 until ev.historySize) {
dx += ev.getHistoricalX(i)
dy += ev.getHistoricalY(i)
}
dx += ev.x
dy += ev.y
moveAccX += dx
moveAccY += dy
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept w/ sign
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
MotionEvent.ACTION_BUTTON_PRESS -> button(ev.actionButton, true)
MotionEvent.ACTION_BUTTON_RELEASE -> button(ev.actionButton, false)
MotionEvent.ACTION_SCROLL -> wheel(ev)
}
return true
}
/** Ctrl+Alt+Shift+Q: release the grab, or (re-)engage it — works even when auto-capture is off. */
fun toggleCapture() {
if (captured) {
userReleased = true
onReleaseCapture?.invoke()
} else {
userReleased = false
onRequestCapture?.invoke()
}
}
/** Auto-engage at stream start (setting on + a mouse actually present). */
fun engageFromStart() {
if (captureWanted && !captured && !userReleased && hasPhysicalMouse()) {
onRequestCapture?.invoke()
}
}
/** From [android.app.Activity.onPointerCaptureChanged] — the OS is the source of truth. */
fun onCaptureChanged(has: Boolean) {
captured = has
// Losing the grab (focus loss, chord) must not leave buttons held on the host.
if (!has) flushButtons()
}
/** Stream teardown: lift anything held and let the grab go. */
fun release() {
flushButtons()
if (captured) onReleaseCapture?.invoke()
}
private fun sendAbs(ev: MotionEvent) {
val (w, h) = surfaceSize()
if (w <= 0 || h <= 0) return
NativeBridge.nativeSendPointerAbs(
handle,
ev.x.roundToInt().coerceIn(0, w - 1),
ev.y.roundToInt().coerceIn(0, h - 1),
w,
h,
)
}
private fun wheel(ev: MotionEvent) {
val dir = if (invertScroll) -1f else 1f
// Android: AXIS_VSCROLL + = up/away, AXIS_HSCROLL + = right — the wire's convention too.
scrollAccV += ev.getAxisValue(MotionEvent.AXIS_VSCROLL) * 120f * dir
scrollAccH += ev.getAxisValue(MotionEvent.AXIS_HSCROLL) * 120f * dir
val v = scrollAccV.toInt()
if (v != 0) {
NativeBridge.nativeSendScroll(handle, 0, v)
scrollAccV -= v
}
val h = scrollAccH.toInt()
if (h != 0) {
NativeBridge.nativeSendScroll(handle, 1, h)
scrollAccH -= h
}
}
private fun button(actionButton: Int, down: Boolean) {
val b = when (actionButton) {
MotionEvent.BUTTON_PRIMARY -> 1
MotionEvent.BUTTON_TERTIARY -> 2
MotionEvent.BUTTON_SECONDARY -> 3
MotionEvent.BUTTON_BACK -> 4
MotionEvent.BUTTON_FORWARD -> 5
else -> return
}
if (down) {
heldButtons.add(b)
NativeBridge.nativeSendPointerButton(handle, b, true)
} else if (heldButtons.remove(b)) {
// Only release what we pressed — drops the release of a swallowed engaging click
// and anything that raced a capture transition.
NativeBridge.nativeSendPointerButton(handle, b, false)
}
}
private fun flushButtons() {
heldButtons.forEach { NativeBridge.nativeSendPointerButton(handle, it, false) }
heldButtons.clear()
}
}
@@ -0,0 +1,193 @@
package io.unom.punktfunk
import android.os.Handler
import android.os.Looper
import android.view.Choreographer
import android.view.KeyEvent
import io.unom.punktfunk.kit.NativeBridge
import kotlin.math.hypot
// Hold this long on SELECT (pointer-mode toggle) / PLAY-PAUSE (keyboard toggle) for the long-press
// action instead of the tap action.
private const val LONG_PRESS_MS = 800L
// D-pad glide ballistics, in screen-widths per second: start slow enough to hit a close button,
// ramp over RAMP_S seconds of continuous hold so crossing the desktop doesn't take all day.
private const val SPEED_MIN = 0.14f
private const val SPEED_MAX = 0.70f
private const val RAMP_S = 1.2f
/**
* Android TV remote as a pointer — the Android analogue of the Apple client's Siri-remote pointer,
* adapted for D-pad-only remotes (most Android TV remotes have no touch surface). For the
* "TV as a desktop client" use case, where a plain remote is often the only thing in hand.
*
* While streaming on a TV, **hold SELECT ≈ 0.8 s** to toggle pointer mode. While active:
* * D-pad (held) glides the host cursor with ramping acceleration (relative `MouseMove`,
* Choreographer-paced, diagonal-normalized);
* * SELECT tap = left click; PLAY/PAUSE tap = right click (Siri-remote parity);
* * PLAY/PAUSE held = toggle the on-screen keyboard; BACK = leave pointer mode
* (a second BACK then leaves the stream as usual).
* While inactive, everything except the SELECT long-press passes through untouched (D-pad =
* arrow keys, SELECT tap = Enter — synthesized on release, since the down was held back to
* disambiguate the long-press).
*
* Only consulted for non-gamepad key events on TV devices (MainActivity gates the calls); all
* state lives on the main thread.
*/
class RemotePointer(
private val handle: Long,
private val surfaceWidth: () -> Int,
private val onActiveChanged: (Boolean) -> Unit,
private val onKeyboardToggle: () -> Unit,
) {
var active = false
private set
private val handler = Handler(Looper.getMainLooper())
private val held = mutableSetOf<Int>() // D-pad keycodes currently down
private var moveAccX = 0f
private var moveAccY = 0f
private var lastFrameNs = 0L
private var rampSec = 0f
private var tickerRunning = false
private var centerLongFired = false
private var playLongFired = false
private val centerLong = Runnable {
centerLongFired = true
toggle()
}
private val playLong = Runnable {
playLongFired = true
onKeyboardToggle()
}
private val frame = object : Choreographer.FrameCallback {
override fun doFrame(nowNs: Long) {
if (!tickerRunning) return
if (held.isEmpty() || !active) {
tickerRunning = false
return
}
val dt = if (lastFrameNs == 0L) {
1f / 60f
} else {
((nowNs - lastFrameNs) / 1e9f).coerceIn(0.001f, 0.1f)
}
lastFrameNs = nowNs
rampSec += dt
var vx = 0f
var vy = 0f
if (KeyEvent.KEYCODE_DPAD_LEFT in held) vx -= 1f
if (KeyEvent.KEYCODE_DPAD_RIGHT in held) vx += 1f
if (KeyEvent.KEYCODE_DPAD_UP in held) vy -= 1f
if (KeyEvent.KEYCODE_DPAD_DOWN in held) vy += 1f
val mag = hypot(vx, vy)
if (mag > 0f) {
val w = surfaceWidth().coerceAtLeast(640)
val speed = w * (SPEED_MIN + (SPEED_MAX - SPEED_MIN) * (rampSec / RAMP_S).coerceAtMost(1f))
moveAccX += vx / mag * speed * dt
moveAccY += vy / mag * speed * dt
val ox = moveAccX.toInt() // truncate toward zero — sub-pixel remainder kept
val oy = moveAccY.toInt()
if (ox != 0 || oy != 0) {
NativeBridge.nativeSendPointerMove(handle, ox, oy)
moveAccX -= ox
moveAccY -= oy
}
}
Choreographer.getInstance().postFrameCallback(this)
}
}
/** One remote key event; true = consumed. Ignore key repeats — the ticker owns motion. */
fun onKey(event: KeyEvent): Boolean {
val down = event.action == KeyEvent.ACTION_DOWN
when (event.keyCode) {
KeyEvent.KEYCODE_DPAD_CENTER -> {
if (down) {
if (event.repeatCount == 0) {
centerLongFired = false
handler.postDelayed(centerLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(centerLong)
if (!centerLongFired) {
if (active) {
click(1)
} else {
// The down was held back to disambiguate the long-press, so the
// normal path never saw it — synthesize the Enter here instead.
NativeBridge.nativeSendKey(handle, 0x0D, true, 0)
NativeBridge.nativeSendKey(handle, 0x0D, false, 0)
}
}
}
return true
}
KeyEvent.KEYCODE_DPAD_UP, KeyEvent.KEYCODE_DPAD_DOWN,
KeyEvent.KEYCODE_DPAD_LEFT, KeyEvent.KEYCODE_DPAD_RIGHT,
-> {
if (!active) return false
if (down) {
if (held.add(event.keyCode) && held.size == 1) startTicker()
} else {
held.remove(event.keyCode)
}
return true
}
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE -> {
if (!active) return false // inactive: the media-key VK path owns it
if (down) {
if (event.repeatCount == 0) {
playLongFired = false
handler.postDelayed(playLong, LONG_PRESS_MS)
}
} else {
handler.removeCallbacks(playLong)
if (!playLongFired) click(3)
}
return true
}
KeyEvent.KEYCODE_BACK -> {
if (!active) return false
if (!down) toggle() // leave pointer mode; the next BACK leaves the stream
return true
}
else -> return false
}
}
/** Stream teardown: stop timers/ticker; nothing wire-held to flush (clicks are edges). */
fun release() {
handler.removeCallbacks(centerLong)
handler.removeCallbacks(playLong)
active = false
held.clear()
tickerRunning = false
}
private fun toggle() {
active = !active
if (!active) {
held.clear()
tickerRunning = false
}
onActiveChanged(active)
}
private fun startTicker() {
rampSec = 0f
lastFrameNs = 0L
if (!tickerRunning) {
tickerRunning = true
Choreographer.getInstance().postFrameCallback(frame)
}
}
private fun click(button: Int) {
NativeBridge.nativeSendPointerButton(handle, button, true)
NativeBridge.nativeSendPointerButton(handle, button, false)
}
}
@@ -109,6 +109,27 @@ data class Settings(
* setup where the OS-level pad (lizard mode) is preferred. * setup where the OS-level pad (lizard mode) is preferred.
*/ */
val sc2Capture: Boolean = true, val sc2Capture: Boolean = true,
/**
* Lock a physical mouse to the stream ([android.view.View.requestPointerCapture]) and forward
* raw relative motion — FPS mouse-look, the iPad "Capture pointer for games" twin. Engages at
* stream start and on a click into the stream; Ctrl+Alt+Shift+Q toggles it live (the chord
* works even with this off). Off (default): a mouse points absolutely, desktop-style.
*/
val pointerCapture: Boolean = false,
/**
* Flip scroll direction — the mouse wheel and the two-finger touch scroll both. Parity with
* the Apple/GTK clients' "Invert scroll direction".
*/
val invertScroll: Boolean = false,
/**
* Sync text copied on this device to the host and vice versa while streaming (the desktop
* clients' shared clipboard, text-only here). Only effective when the host advertises the
* clipboard capability; the protocol is opt-in per session either way.
*/
val clipboardSync: Boolean = true,
) )
/** [Settings.touchMode] values; persisted by name. */ /** [Settings.touchMode] values; persisted by name. */
@@ -172,6 +193,9 @@ class SettingsStore(context: Context) {
autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true), autoWakeEnabled = prefs.getBoolean(K_AUTO_WAKE, true),
rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false), rumbleOnPhone = prefs.getBoolean(K_RUMBLE_ON_PHONE, false),
sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true), sc2Capture = prefs.getBoolean(K_SC2_CAPTURE, true),
pointerCapture = prefs.getBoolean(K_POINTER_CAPTURE, false),
invertScroll = prefs.getBoolean(K_INVERT_SCROLL, false),
clipboardSync = prefs.getBoolean(K_CLIPBOARD_SYNC, true),
) )
fun save(s: Settings) { fun save(s: Settings) {
@@ -195,6 +219,9 @@ class SettingsStore(context: Context) {
.putBoolean(K_AUTO_WAKE, s.autoWakeEnabled) .putBoolean(K_AUTO_WAKE, s.autoWakeEnabled)
.putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone) .putBoolean(K_RUMBLE_ON_PHONE, s.rumbleOnPhone)
.putBoolean(K_SC2_CAPTURE, s.sc2Capture) .putBoolean(K_SC2_CAPTURE, s.sc2Capture)
.putBoolean(K_POINTER_CAPTURE, s.pointerCapture)
.putBoolean(K_INVERT_SCROLL, s.invertScroll)
.putBoolean(K_CLIPBOARD_SYNC, s.clipboardSync)
.apply() .apply()
} }
@@ -233,6 +260,9 @@ class SettingsStore(context: Context) {
const val K_AUTO_WAKE = "auto_wake_enabled" const val K_AUTO_WAKE = "auto_wake_enabled"
const val K_RUMBLE_ON_PHONE = "rumble_on_phone" const val K_RUMBLE_ON_PHONE = "rumble_on_phone"
const val K_SC2_CAPTURE = "sc2_capture" const val K_SC2_CAPTURE = "sc2_capture"
const val K_POINTER_CAPTURE = "pointer_capture"
const val K_INVERT_SCROLL = "invert_scroll"
const val K_CLIPBOARD_SYNC = "clipboard_sync"
/** Legacy Boolean the enum replaced — read once as the migration default, never written. */ /** Legacy Boolean the enum replaced — read once as the migration default, never written. */
const val K_TRACKPAD = "trackpad_mode" const val K_TRACKPAD = "trackpad_mode"
@@ -412,6 +412,27 @@ private fun ControlsSettings(s: Settings, update: (Settings) -> Unit, onOpenCont
style = MaterialTheme.typography.bodySmall, style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant, color = MaterialTheme.colorScheme.onSurfaceVariant,
) )
ToggleRow(
title = "Capture pointer for games",
subtitle = "Lock a connected mouse to the stream and send raw relative motion " +
"(mouse-look). Ctrl+Alt+Shift+Q toggles it live; click the stream to re-capture. " +
"Off: the mouse points at the desktop directly",
checked = s.pointerCapture,
onCheckedChange = { on -> update(s.copy(pointerCapture = on)) },
)
ToggleRow(
title = "Invert scroll direction",
subtitle = "Flip the mouse wheel and two-finger touch scrolling",
checked = s.invertScroll,
onCheckedChange = { on -> update(s.copy(invertScroll = on)) },
)
ToggleRow(
title = "Shared clipboard",
subtitle = "Text copied here pastes on the host and vice versa (hosts with " +
"clipboard sharing enabled)",
checked = s.clipboardSync,
onCheckedChange = { on -> update(s.copy(clipboardSync = on)) },
)
} }
SettingsCard { SettingsCard {
SettingDropdown( SettingDropdown(
@@ -176,6 +176,14 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// "hold to quit" hint overlay. Set from the router's onExitArmed (main thread). // "hold to quit" hint overlay. Set from the router's onExitArmed (main thread).
var exitArming by remember { mutableStateOf(false) } var exitArming by remember { mutableStateOf(false) }
// True while the TV remote is acting as a pointer (hold SELECT toggles) — drives the mode hint.
var remotePointerOn by remember { mutableStateOf(false) }
// Focus anchor the soft keyboard is summoned onto AND the pointer-capture grab target (a grab
// needs a focusable view; captured-pointer events land on it). Declared before the effect
// below so the capture callbacks can reach the view once it exists.
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) }
DisposableEffect(handle) { DisposableEffect(handle) {
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON) window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
wifiLocks.forEach { lock -> wifiLocks.forEach { lock ->
@@ -221,6 +229,54 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Show a "hold to quit" hint the moment the chord completes (the router debounces the actual // Show a "hold to quit" hint the moment the chord completes (the router debounces the actual
// exit); it clears when the buttons release early or the hold elapses. Runs on the main thread. // exit); it clears when the buttons release early or the hold elapses. Runs on the main thread.
router.onExitArmed = { armed -> exitArming = armed } router.onExitArmed = { armed -> exitArming = armed }
// Physical mouse: uncaptured hover/click/wheel forwards as absolute pointing; captured
// (setting or the Ctrl+Alt+Shift+Q chord) raw deltas forward as relative mouse-look.
// The local cursor is hidden over the stream — the host's own cursor, composited into
// the video, is the one the user sees (twin of the desktop clients' hidden cursor).
val decor = window?.decorView
val priorPointerIcon = decor?.pointerIcon
decor?.pointerIcon = android.view.PointerIcon.getSystemIcon(
context,
android.view.PointerIcon.TYPE_NULL,
)
val mouse = MouseForwarder(
handle,
invertScroll = initialSettings.invertScroll,
captureWanted = initialSettings.pointerCapture,
surfaceSize = { (decor?.width ?: 0) to (decor?.height ?: 0) },
)
mouse.onRequestCapture = {
// The grab needs the (focusable) capture view: focus it, then ask. Posted so a
// request racing view attach/focus settles on the next frame.
keyCapture?.let { v ->
v.post {
v.requestFocus()
v.requestPointerCapture()
}
}
}
mouse.onReleaseCapture = { keyCapture?.releasePointerCapture() }
activity?.mouseForwarder = mouse
// TV remote-as-pointer: hold SELECT ≈ 0.8 s to toggle; the D-pad then glides the host
// cursor (see RemotePointer). TV only — a phone's remote-less keys stay on the VK path.
val remote = if (isTv) {
RemotePointer(
handle,
surfaceWidth = { decor?.width ?: 1920 },
onActiveChanged = { on -> remotePointerOn = on },
onKeyboardToggle = { keyCapture?.let { it.setImeVisible(!it.imeShown) } },
)
} else {
null
}
activity?.remotePointer = remote
// Shared clipboard (text v1): only when the user setting is on AND the host has a
// working clipboard service. Protocol-level opt-in + the poll thread live in the sync.
val clip = if (initialSettings.clipboardSync && NativeBridge.nativeClipSupported(handle)) {
ClipboardSync(context, handle).also { it.start() }
} else {
null
}
activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate activity?.setConsoleHighRefreshRate(false) // let the decoder's setFrameRate pick the panel rate
// Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad // Host→client feedback (rumble + DualSense lightbar/LEDs), routed to each controller by pad
// index via the router; poll threads stopped + joined before the router is released and the // index via the router; poll threads stopped + joined before the router is released and the
@@ -286,6 +342,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
} }
onDispose { onDispose {
closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it closed.set(true) // from here the handle gets freed; surfaceDestroyed must not touch it
clip?.stop() // stop + join the clipboard poll thread BEFORE the handle is freed
feedback.onHidRaw = null feedback.onHidRaw = null
feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed feedback.stop() // stop + join the poll threads BEFORE the router is released / handle freed
sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } } sc2UsbReceiver?.let { runCatching { context.unregisterReceiver(it) } }
@@ -293,6 +350,12 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down router.onExitArmed = null // don't poke Compose state from release()'s disarm while tearing down
router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener router.release() // flush every slot (nothing sticks host-side) + drop the hot-plug listener
activity?.gamepadRouter = null activity?.gamepadRouter = null
// Mouse/remote-pointer teardown: lift held buttons, drop the grab, restore the cursor.
mouse.release()
activity?.mouseForwarder = null
remote?.release()
activity?.remotePointer = null
decor?.pointerIcon = priorPointerIcon
activity?.streamHandle = 0L activity?.streamHandle = 0L
activity?.requestStreamExit = null activity?.requestStreamExit = null
// Back in the menus: the SC2 (if present) resumes driving the console UI. // Back in the menus: the SC2 (if present) resumes driving the console UI.
@@ -320,8 +383,12 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
// Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger). // Back gesture = a deliberate exit → signal the quit so the host tears down now (no linger).
BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() } BackHandler { NativeBridge.nativeDisconnectQuit(handle); onDisconnect() }
// Focus anchor the three-finger keyboard swipe summons the IME onto (see KeyCaptureView). // Auto-engage pointer capture at stream start (setting on + a mouse actually present).
var keyCapture by remember { mutableStateOf<KeyCaptureView?>(null) } // Delayed a beat: the grab needs window focus and the capture view attached.
LaunchedEffect(handle) {
delay(400)
activity?.mouseForwarder?.engageFromStart()
}
Box(modifier = Modifier.fillMaxSize()) { Box(modifier = Modifier.fillMaxSize()) {
AndroidView( AndroidView(
@@ -379,11 +446,26 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
if (exitArming) { if (exitArming) {
ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp)) ExitChordHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
} }
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe // Remote-pointer mode hint — the remote's keys are remapped while it's on, so say so.
// up in the mouse modes) — it never draws or takes touches, it just owns IME focus. if (remotePointerOn) {
RemotePointerHint(Modifier.align(Alignment.TopCenter).padding(top = 16.dp))
}
// Invisible 1-px focus anchor for the host-typing soft keyboard (three-finger swipe up
// in the mouse modes) AND the pointer-capture grab target — it never draws or takes
// touches, it just owns IME focus and receives captured-pointer events.
AndroidView( AndroidView(
modifier = Modifier.size(1.dp), modifier = Modifier.size(1.dp),
factory = { ctx -> KeyCaptureView(ctx).also { keyCapture = it } }, factory = { ctx ->
KeyCaptureView(ctx).also { v ->
keyCapture = v
// Real IME text path when the host types committed text (see KeyCaptureView).
v.textHandle =
if (NativeBridge.nativeTextInputSupported(handle)) handle else 0L
v.setOnCapturedPointerListener { _, ev ->
(ctx as? MainActivity)?.mouseForwarder?.onCapturedPointer(ev) ?: false
}
}
},
) )
// Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture // Touch input per the Settings model: trackpad/direct-pointer mouse (the shared gesture
// vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no // vocabulary) or real multi-touch passthrough — see TouchInput.kt. Passthrough gets no
@@ -396,6 +478,7 @@ fun StreamScreen(handle: Long, micEnabled: Boolean, onDisconnect: () -> Unit) {
else -> streamTouchInput( else -> streamTouchInput(
handle, handle,
trackpad = touchMode == TouchMode.TRACKPAD, trackpad = touchMode == TouchMode.TRACKPAD,
invertScroll = initialSettings.invertScroll,
onCycleStats = { statsVerbosity = statsVerbosity.next() }, onCycleStats = { statsVerbosity = statsVerbosity.next() },
onKeyboard = { show -> keyCapture?.setImeVisible(show) }, onKeyboard = { show -> keyCapture?.setImeVisible(show) },
) )
@@ -423,14 +506,35 @@ private fun ExitChordHint(modifier: Modifier = Modifier) {
) )
} }
/**
* The remote-pointer mode cue: while active the remote's keys are remapped (D-pad glides the host
* cursor, SELECT clicks), so the overlay both confirms the toggle and teaches the vocabulary.
*/
@Composable
private fun RemotePointerHint(modifier: Modifier = Modifier) {
Text(
"Remote pointer — SELECT click · play/pause right-click · hold SELECT to exit",
modifier = modifier
.background(Color.Black.copy(alpha = 0.55f), RoundedCornerShape(8.dp))
.padding(horizontal = 14.dp, vertical = 8.dp),
color = Color.White,
fontSize = 15.sp,
)
}
/** /**
* Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME * Invisible focus anchor for typing on the host: the three-finger swipe summons the device IME
* onto this view. `TYPE_NULL` puts the IME in "dumb keyboard" mode — it delivers raw [KeyEvent]s * onto this view. Two IME models, picked by the host's capabilities:
* (no composing text, no autocorrect), which flow through `MainActivity.dispatchKeyEvent` → * * **Text path** ([textHandle] set — the host advertised `HOST_CAP_TEXT_INPUT`): a real
* `Keymap.toVk` → the host, the exact path a hardware keyboard takes. Text an IME insists on * editable [HostTextConnection], so the IME gives autocorrect, gesture typing, non-Latin
* committing instead still arrives: the non-editable [BaseInputConnection] synthesizes KeyEvents * composition and emoji, all mirrored to the host as committed text + diffs.
* for it via `KeyCharacterMap` (with Shift carried as meta state — see the IME-shift wrap in * * **Fallback** (older host): `TYPE_NULL` puts the IME in "dumb keyboard" mode — raw
* `MainActivity.dispatchKeyEvent`). * [KeyEvent]s flow through `MainActivity.dispatchKeyEvent` → `Keymap.toVk` → the host, the
* exact path a hardware keyboard takes (with the IME-shift wrap documented there).
*
* Doubles as the pointer-capture grab target: a grab needs a focusable view, and captured-pointer
* events are delivered to it (routed to [MouseForwarder.onCapturedPointer] via the listener the
* stream screen installs).
*/ */
private class KeyCaptureView(context: Context) : View(context) { private class KeyCaptureView(context: Context) : View(context) {
init { init {
@@ -438,17 +542,32 @@ private class KeyCaptureView(context: Context) : View(context) {
isFocusableInTouchMode = true isFocusableInTouchMode = true
} }
/** The session handle when the host types committed text; `0` = VK-only fallback. */
var textHandle: Long = 0L
/** Whether [setImeVisible] last showed the IME — for toggle-style callers (remote pointer). */
var imeShown = false
private set
override fun onCheckIsTextEditor(): Boolean = true override fun onCheckIsTextEditor(): Boolean = true
override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection { override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection {
outAttrs.inputType = InputType.TYPE_NULL outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or
outAttrs.imeOptions = EditorInfo.IME_FLAG_NO_EXTRACT_UI or EditorInfo.IME_FLAG_NO_FULLSCREEN EditorInfo.IME_FLAG_NO_FULLSCREEN or EditorInfo.IME_FLAG_NO_ENTER_ACTION
return BaseInputConnection(this, false) return if (textHandle != 0L) {
outAttrs.inputType = InputType.TYPE_CLASS_TEXT or
InputType.TYPE_TEXT_FLAG_AUTO_CORRECT or InputType.TYPE_TEXT_FLAG_MULTI_LINE
HostTextConnection(this, textHandle)
} else {
outAttrs.inputType = InputType.TYPE_NULL
BaseInputConnection(this, false)
}
} }
fun setImeVisible(show: Boolean) { fun setImeVisible(show: Boolean) {
val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager val imm = context.getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager
?: return ?: return
imeShown = show
if (show) { if (show) {
requestFocus() requestFocus()
imm.showSoftInput(this, 0) imm.showSoftInput(this, 0)
@@ -457,3 +576,113 @@ private class KeyCaptureView(context: Context) : View(context) {
} }
} }
} }
/**
* IME → host text bridge (the `HOST_CAP_TEXT_INPUT` path): a real **editable** connection, so
* the IME runs its full machinery (autocorrect, gesture typing, non-Latin composition), mirrored
* to the host as it happens. The one piece of host-side state tracked is *what the host currently
* shows of the active composition* ([sentComposition]): composing updates send a common-prefix
* diff (backspaces + the new suffix) so corrections materialize live on the host; a commit
* settles it. [setComposingRegion] adopts already-committed text as the active composition
* (autocorrect-revert / backspace-into-word flows), so the next update diffs against it instead
* of retyping. Newlines become Enter taps; [deleteSurroundingText] becomes Backspace/Delete taps.
*
* Known approximation: diff lengths are counted in Unicode scalars, assuming one host Backspace
* deletes one scalar — true for the composition text IMEs actually produce (emoji and other
* multi-unit graphemes commit directly rather than composing).
*/
private class HostTextConnection(
view: KeyCaptureView,
private val handle: Long,
) : BaseInputConnection(view, true) {
/** What the host currently shows of the active composition ("" = none). */
private var sentComposition = ""
override fun commitText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
sentComposition = ""
val ok = super.commitText(text, newCursorPosition)
trimEditable()
return ok
}
override fun setComposingText(text: CharSequence, newCursorPosition: Int): Boolean {
retype(text.toString())
return super.setComposingText(text, newCursorPosition)
}
override fun finishComposingText(): Boolean {
// The composition text stands as committed — the host already shows it verbatim.
sentComposition = ""
return super.finishComposingText()
}
override fun setComposingRegion(start: Int, end: Int): Boolean {
val e = editable
if (e != null) {
val a = start.coerceIn(0, e.length)
val b = end.coerceIn(0, e.length)
sentComposition = e.subSequence(minOf(a, b), maxOf(a, b)).toString()
}
return super.setComposingRegion(start, end)
}
override fun deleteSurroundingText(beforeLength: Int, afterLength: Int): Boolean {
repeat(beforeLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_BACK) }
repeat(afterLength.coerceIn(0, MAX_TAPS)) { tapVk(VK_DELETE) }
return super.deleteSurroundingText(beforeLength, afterLength)
}
override fun performEditorAction(actionCode: Int): Boolean {
tapVk(VK_RETURN)
return true
}
/** Replace the host's view of the composition with [text] via a common-prefix diff. */
private fun retype(text: String) {
var common = sentComposition.commonPrefixWith(text)
// Never split a surrogate pair mid-diff — back off to the pair boundary.
if (common.isNotEmpty() && common.last().isHighSurrogate()) {
common = common.dropLast(1)
}
val stale = sentComposition.substring(common.length)
repeat(stale.codePointCount(0, stale.length).coerceAtMost(MAX_TAPS)) { tapVk(VK_BACK) }
sendText(text.substring(common.length))
sentComposition = text
}
/** Forward literal text, turning newlines into Enter taps (control chars never ride text). */
private fun sendText(s: String) {
var chunk = StringBuilder()
for (ch in s) {
if (ch == '\n') {
if (chunk.isNotEmpty()) {
NativeBridge.nativeSendText(handle, chunk.toString())
chunk = StringBuilder()
}
tapVk(VK_RETURN)
} else {
chunk.append(ch)
}
}
if (chunk.isNotEmpty()) NativeBridge.nativeSendText(handle, chunk.toString())
}
private fun tapVk(vk: Int) {
NativeBridge.nativeSendKey(handle, vk, true, 0)
NativeBridge.nativeSendKey(handle, vk, false, 0)
}
/** Bound the mirror buffer: once nothing is composing, old text serves no purpose. */
private fun trimEditable() {
val e = editable ?: return
if (getComposingSpanStart(e) == -1 && e.length > 4000) e.clear()
}
private companion object {
const val VK_BACK = 0x08
const val VK_RETURN = 0x0D
const val VK_DELETE = 0x2E
const val MAX_TAPS = 256
}
}
@@ -99,9 +99,11 @@ internal suspend fun PointerInputScope.streamTouchPassthrough(handle: Long) {
internal suspend fun PointerInputScope.streamTouchInput( internal suspend fun PointerInputScope.streamTouchInput(
handle: Long, handle: Long,
trackpad: Boolean, trackpad: Boolean,
invertScroll: Boolean,
onCycleStats: () -> Unit, onCycleStats: () -> Unit,
onKeyboard: (show: Boolean) -> Unit, onKeyboard: (show: Boolean) -> Unit,
) { ) {
val scrollDir = if (invertScroll) -1 else 1
var lastTapUp = 0L var lastTapUp = 0L
var lastTapX = 0f var lastTapX = 0f
var lastTapY = 0f var lastTapY = 0f
@@ -184,12 +186,12 @@ internal suspend fun PointerInputScope.streamTouchInput(
val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up val sy = ((prevCy - cy) / SCROLL_DIV).toInt() // finger up → wheel up
val sx = ((cx - prevCx) / SCROLL_DIV).toInt() val sx = ((cx - prevCx) / SCROLL_DIV).toInt()
if (sy != 0) { if (sy != 0) {
NativeBridge.nativeSendScroll(handle, 0, sy * 120) NativeBridge.nativeSendScroll(handle, 0, sy * 120 * scrollDir)
prevCy = cy prevCy = cy
moved = true moved = true
} }
if (sx != 0) { if (sx != 0) {
NativeBridge.nativeSendScroll(handle, 1, sx * 120) NativeBridge.nativeSendScroll(handle, 1, sx * 120 * scrollDir)
prevCx = cx prevCx = cx
moved = true moved = true
} }
@@ -106,6 +106,17 @@ object Keymap {
KeyEvent.KEYCODE_DPAD_UP -> 0x26 KeyEvent.KEYCODE_DPAD_UP -> 0x26
KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27 KeyEvent.KEYCODE_DPAD_RIGHT -> 0x27
KeyEvent.KEYCODE_DPAD_DOWN -> 0x28 KeyEvent.KEYCODE_DPAD_DOWN -> 0x28
// TV-remote SELECT = Enter (a gamepad's press routes via SOURCE_GAMEPAD before this).
KeyEvent.KEYCODE_DPAD_CENTER -> 0x0D
// Consumer/media keys — forwarded to the host while streaming (volume stays local:
// MainActivity's pass-through list wins before the map is consulted).
KeyEvent.KEYCODE_MEDIA_PLAY_PAUSE,
KeyEvent.KEYCODE_MEDIA_PLAY,
KeyEvent.KEYCODE_MEDIA_PAUSE -> 0xB3 // VK_MEDIA_PLAY_PAUSE
KeyEvent.KEYCODE_MEDIA_NEXT -> 0xB0 // VK_MEDIA_NEXT_TRACK
KeyEvent.KEYCODE_MEDIA_PREVIOUS -> 0xB1 // VK_MEDIA_PREV_TRACK
KeyEvent.KEYCODE_MEDIA_STOP -> 0xB2 // VK_MEDIA_STOP
// Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant) // Modifiers (L/R-specific VKs; the host folds the generic ones onto the left variant)
KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0 KeyEvent.KEYCODE_SHIFT_LEFT -> 0xA0
@@ -287,6 +287,50 @@ object NativeBridge {
/** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */ /** One key transition. vk: Windows VK (0 = dropped by Rust). mods: VK modifier mask (0 for now). */
external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int) external fun nativeSendKey(handle: Long, vk: Int, down: Boolean, mods: Int)
/**
* Whether the host advertised committed-text injection (`HOST_CAP_TEXT_INPUT`) — its inject
* backend can type Unicode text directly. Picks the real IME `InputConnection` (autocorrect,
* gesture typing, non-Latin scripts) over the TYPE_NULL raw-key fallback. False on `0`.
*/
external fun nativeTextInputSupported(handle: Long): Boolean
/**
* Committed IME text → one `TextInput` wire event per Unicode scalar, in order. Control
* characters are skipped natively (Enter/Backspace ride [nativeSendKey]). Only meaningful
* when [nativeTextInputSupported] returned true — older hosts ignore the events.
*/
external fun nativeSendText(handle: Long, text: String)
// ---- Shared clipboard (text v1): Kotlin drives ClipboardManager, Rust the protocol ----
// Opt-in per session (nativeClipControl). Local copies are announced as lazy offers; bytes
// cross only when the host pastes (a "fetch:" event answered by nativeClipServeText). Host
// copies arrive as "offer:" events, fetched eagerly into the system clipboard.
/** Whether the host advertised a working shared-clipboard service (HOST_CAP_CLIPBOARD). */
external fun nativeClipSupported(handle: Long): Boolean
/** Session-level clipboard opt-in/out; nothing happens until enabled=true crosses. */
external fun nativeClipControl(handle: Long, enabled: Boolean)
/** Announce "this device's clipboard now holds text". [seq]: monotonic, newest wins. */
external fun nativeClipOfferText(handle: Long, seq: Int)
/** Pull the text of the host's offer [seq] → transfer id echoed on "data:"/"error:", or -1. */
external fun nativeClipFetchText(handle: Long, seq: Int): Int
/** Answer a "fetch:" event with the clipboard's current text (the host is pasting). */
external fun nativeClipServeText(handle: Long, reqId: Int, text: String)
/** Abort a clipboard transfer by id (either direction). */
external fun nativeClipCancel(handle: Long, id: Int)
/**
* Block ≤250 ms for the next clipboard event, as a compact string: `state:<0|1>` ·
* `offer:<seq>:<hasText>` · `fetch:<reqId>` · `data:<xferId>:<text>` · `cancel:<id>` ·
* `error:<id>:<code>` · `closed` (session gone) — null on timeout. Dedicated poll thread.
*/
external fun nativeNextClip(handle: Long): String?
// ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ---- // ---- Gamepad: each controller forwarded on its own wire pad index (0..15, low byte of flags) ----
// The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0, // The pad index is assigned per Android device by GamepadRouter; a single controller lands on 0,
// so its wire is byte-identical to the old single-pad path. The core folds the per-transition // so its wire is byte-identical to the old single-pad path. The core folds the per-transition
@@ -0,0 +1,182 @@
//! Shared-clipboard plane (text-only v1): Kotlin drives the Android `ClipboardManager`, these
//! shims drive [`punktfunk_core::client::NativeClient`]'s clipboard surface.
//!
//! Model (mirrors the desktop clients): opt-in via `nativeClipControl(true)`; local copies are
//! announced lazily as format-list offers (`nativeClipOfferText`) and the bytes cross only when
//! the host pastes (a `fetch` event answered by `nativeClipServeText`); a host copy arrives as an
//! `offer` event, which the Kotlin side fetches eagerly (Android's clipboard has no lazy provider
//! path worth the complexity) and lands in the system clipboard on the `data` event.
//!
//! Events cross to Kotlin as compact strings from the blocking `nativeNextClip` poll (drained on
//! a dedicated thread, same pattern as `nativeNextRumble`):
//! `state:<0|1>` · `offer:<seq>:<has_text 0|1>` · `fetch:<req_id>` · `data:<xfer_id>:<text>` ·
//! `cancel:<id>` · `error:<id>:<code>` · `closed` — null on a poll timeout. Non-text fetch
//! requests are cancelled natively (only text is ever offered, so they shouldn't occur).
use std::time::Duration;
use jni::objects::{JObject, JString};
use jni::sys::{jboolean, jint, jlong, jstring};
use jni::JNIEnv;
use punktfunk_core::clipboard::ClipEventCore;
use punktfunk_core::error::PunktfunkError;
use punktfunk_core::quic::{ClipKind, CLIP_FILE_INDEX_NONE, HOST_CAP_CLIPBOARD};
use super::SessionHandle;
/// The portable wire MIME both ends map to their platform text type.
const TEXT_MIME: &str = "text/plain;charset=utf-8";
/// Deref the opaque handle (`0` → `None`).
///
/// SAFETY: live handle per the nativeConnect/nativeClose contract; every method used is `&self`
/// on the `Sync` connector.
fn client(handle: jlong) -> Option<&'static SessionHandle> {
if handle == 0 {
return None;
}
// SAFETY: see the function docs — the Kotlin side guarantees the handle outlives the call.
Some(unsafe { &*(handle as *const SessionHandle) })
}
/// `NativeBridge.nativeClipSupported(handle)` — the host advertised `HOST_CAP_CLIPBOARD`.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
client(handle).map_or(0, |h| {
u8::from(h.client.host_caps() & HOST_CAP_CLIPBOARD != 0)
})
}
/// `NativeBridge.nativeClipControl(handle, enabled)` — session-level opt-in/out. Nothing
/// clipboard-related happens on either side until an `enabled: true` crosses.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipControl(
_env: JNIEnv,
_this: JObject,
handle: jlong,
enabled: jboolean,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_control(enabled != 0, 0);
}
}
/// `NativeBridge.nativeClipOfferText(handle, seq)` — announce "the Android clipboard now holds
/// text" (format list only; bytes cross when the host fetches). `seq` is Kotlin's monotonic
/// counter, newest wins.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipOfferText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_offer(
seq as u32,
vec![ClipKind {
mime: TEXT_MIME.into(),
size_hint: 0,
}],
);
}
}
/// `NativeBridge.nativeClipFetchText(handle, seq)` — pull the text of the host's offer `seq`.
/// Returns the transfer id echoed on the matching `data:`/`error:` event, or 1.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipFetchText(
_env: JNIEnv,
_this: JObject,
handle: jlong,
seq: jint,
) -> jint {
client(handle)
.and_then(|h| {
h.client
.clip_fetch(seq as u32, TEXT_MIME.into(), CLIP_FILE_INDEX_NONE)
.ok()
})
.map_or(-1, |xfer| xfer as jint)
}
/// `NativeBridge.nativeClipServeText(handle, reqId, text)` — answer a `fetch:` event with the
/// clipboard's current text (the host is pasting our offer).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipServeText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
req_id: jint,
text: JString,
) {
let Some(h) = client(handle) else { return };
let Ok(s) = env.get_string(&text) else {
let _ = h.client.clip_cancel(req_id as u32);
return;
};
let _ = h
.client
.clip_serve(req_id as u32, String::from(s).into_bytes(), true);
}
/// `NativeBridge.nativeClipCancel(handle, id)` — abort a transfer (either direction).
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeClipCancel(
_env: JNIEnv,
_this: JObject,
handle: jlong,
id: jint,
) {
if let Some(h) = client(handle) {
let _ = h.client.clip_cancel(id as u32);
}
}
/// `NativeBridge.nativeNextClip(handle)` — block ≤250 ms for the next clipboard event, encoded
/// as a compact string (module docs); null on timeout, `"closed"` once the session is gone.
/// Call from a dedicated poll thread.
///
/// Text payloads ride `data:<xfer_id>:<text>` decoded lossily — safe because the phase-0
/// clipboard task delivers a whole payload in ONE event (`last = true`), so a chunk boundary
/// can never split a UTF-8 sequence.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeNextClip(
env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jstring {
let Some(h) = client(handle) else {
return std::ptr::null_mut();
};
let msg = match h.client.next_clip(Duration::from_millis(250)) {
Ok(ClipEventCore::State { enabled, .. }) => format!("state:{}", u8::from(enabled)),
Ok(ClipEventCore::RemoteOffer { seq, kinds }) => {
let has_text = kinds.iter().any(|k| k.mime.starts_with("text/plain"));
format!("offer:{seq}:{}", u8::from(has_text))
}
Ok(ClipEventCore::FetchRequest { req_id, mime, .. }) => {
if mime.starts_with("text/plain") {
format!("fetch:{req_id}")
} else {
// We only ever offer text; cancel anything else rather than stall the host.
let _ = h.client.clip_cancel(req_id);
return std::ptr::null_mut();
}
}
Ok(ClipEventCore::Data { xfer_id, bytes, .. }) => {
format!("data:{xfer_id}:{}", String::from_utf8_lossy(&bytes))
}
Ok(ClipEventCore::Cancelled { id }) => format!("cancel:{id}"),
Ok(ClipEventCore::Error { id, code }) => format!("error:{id}:{code}"),
Err(PunktfunkError::NoFrame) => return std::ptr::null_mut(),
Err(_) => "closed".into(),
};
env.new_string(msg)
.map(|s| s.into_raw())
.unwrap_or(std::ptr::null_mut())
}
@@ -201,6 +201,9 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
// No display-volume forwarding from Android yet (the panel tone-maps PQ itself via the // No display-volume forwarding from Android yet (the panel tone-maps PQ itself via the
// Surface dataspace + static metadata) — the host keeps its virtual-display EDID defaults. // Surface dataspace + static metadata) — the host keeps its virtual-display EDID defaults.
None, None,
// No non-video caps: this client does not render the host cursor locally (no shape/state
// planes in the jni surface), so advertising CLIENT_CAP_CURSOR would stream cursor-less.
0,
launch, // a store-qualified library id to boot into a game, or None for the desktop launch, // a store-qualified library id to boot into a game, or None for the desktop
pin, // Some → Crypto on host-fp mismatch pin, // Some → Crypto on host-fp mismatch
identity, // owned (cert, key) PEM, or None (anonymous) identity, // owned (cert, key) PEM, or None (anonymous)
+41 -2
View File
@@ -6,11 +6,11 @@
//! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal, //! conventions: buttons 1=left/2=middle/3=right/4=X1/5=X2; scroll axis 0=vertical/1=horizontal,
//! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side). //! signed 120-unit delta, +=up/right; keys are Windows VK (mapped from KEYCODE_* on the Kotlin side).
use jni::objects::{JByteBuffer, JObject}; use jni::objects::{JByteBuffer, JObject, JString};
use jni::sys::{jboolean, jint, jlong}; use jni::sys::{jboolean, jint, jlong};
use jni::JNIEnv; use jni::JNIEnv;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use punktfunk_core::quic::{RichInput, HID_REPORT_MAX}; use punktfunk_core::quic::{RichInput, HID_REPORT_MAX, HOST_CAP_TEXT_INPUT};
use super::SessionHandle; use super::SessionHandle;
@@ -145,6 +145,45 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendKey(
send_event(handle, kind, vk as u32, 0, 0, mods as u32); send_event(handle, kind, vk as u32, 0, 0, mods as u32);
} }
/// `NativeBridge.nativeTextInputSupported(handle)` — whether the host advertised
/// `HOST_CAP_TEXT_INPUT` (its inject backend types committed text), so the Kotlin side can pick
/// the real IME `InputConnection` over the TYPE_NULL raw-key fallback. `0` handle → false.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported(
_env: JNIEnv,
_this: JObject,
handle: jlong,
) -> jboolean {
if handle == 0 {
return 0;
}
// SAFETY: live handle per the nativeConnect/nativeClose contract; host_caps is &self.
let h = unsafe { &*(handle as *const SessionHandle) };
u8::from(h.client.host_caps() & HOST_CAP_TEXT_INPUT != 0)
}
/// `NativeBridge.nativeSendText(handle, text)` — committed IME text, one `TextInput` event per
/// Unicode scalar (`code` = the scalar; multi-char commits are consecutive events in order).
/// Control characters are skipped — Enter/Backspace/Tab ride the VK key path. Call only when
/// [`Java_io_unom_punktfunk_kit_NativeBridge_nativeTextInputSupported`] returned true.
#[no_mangle]
pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeSendText(
mut env: JNIEnv,
_this: JObject,
handle: jlong,
text: JString,
) {
if handle == 0 {
return;
}
let Ok(s) = env.get_string(&text) else {
return;
};
for ch in String::from(s).chars().filter(|c| !c.is_control()) {
send_event(handle, InputKind::TextInput, ch as u32, 0, 0, 0);
}
}
// ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input --------------- // ---- Gamepad: Kotlin captures (KeyEvent/MotionEvent) → NativeClient::send_input ---------------
// Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried // Multi-pad model: each physical controller is forwarded on its own wire pad index (0..15), carried
// in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a // in the low byte of `flags` on every per-pad event — the Kotlin side (`GamepadRouter`) assigns a
@@ -17,6 +17,7 @@
//! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode //! TODO(M4 Android stage 1): client→host DualSense rich input (`send_rich_input`), mode
//! renegotiation. Port the remaining orchestration from `clients/linux`. //! renegotiation. Port the remaining orchestration from `clients/linux`.
mod clipboard;
mod connect; mod connect;
mod input; mod input;
mod planes; mod planes;
@@ -302,13 +302,26 @@ final class SessionModel: ObservableObject {
if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported { if preferredCodec == PunktfunkConnection.codecPyroWave, MetalWaveletDecoder.supported {
videoCodecs |= PunktfunkConnection.codecPyroWave videoCodecs |= PunktfunkConnection.codecPyroWave
} }
// Cursor channel (remote-desktop-sweep M2, macOS): sessions STARTING in the desktop
// mouse model advertise local cursor rendering the host then stops compositing
// the pointer and forwards shape/state, which StreamView draws as the real
// NSCursor. Capture-mode sessions keep today's composited pointer.
#if os(macOS)
let clientCaps: UInt8 =
(MouseInputMode(
rawValue: UserDefaults.standard.string(forKey: DefaultsKey.mouseMode) ?? "")
?? .capture) == .desktop ? 0x01 : 0
#else
let clientCaps: UInt8 = 0
#endif
let result = Result { try PunktfunkConnection( let result = Result { try PunktfunkConnection(
host: host.address, port: host.port, host: host.address, port: host.port,
width: width, height: height, refreshHz: hz, width: width, height: height, refreshHz: hz,
pinSHA256: pin, identity: identity, compositor: compositor, pinSHA256: pin, identity: identity, compositor: compositor,
gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps, gamepad: gamepad, bitrateKbps: bitrateKbps, videoCaps: videoCaps,
audioChannels: audioChannels, audioChannels: audioChannels,
videoCodecs: videoCodecs, preferredCodec: preferredCodec, launchID: launchID, videoCodecs: videoCodecs, preferredCodec: preferredCodec,
clientCaps: clientCaps, launchID: launchID,
// Delegated approval: the host holds this connect open until the operator approves // Delegated approval: the host holds this connect open until the operator approves
// it (~180 s) outwait that window so a slow approval still lands here. Normal // it (~180 s) outwait that window so a slow approval still lands here. Normal
// connects keep the snappy default. // connects keep the snappy default.
@@ -705,8 +718,11 @@ final class SessionModel: ObservableObject {
// captured before the 2026-07 floor policy); the appended trio carries the // captured before the 2026-07 floor policy); the appended trio carries the
// measured OS present floor and the floor-shaved values the HUD displays. // measured OS present floor and the floor-shaved values the HUD displays.
let line = String( let line = String(
format: "fps=%d presents=%d e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f " // Swift Int is 64-bit %lld, NOT %d (which is a 32-bit C int); macOS 26's
+ "decode_p50=%.1f display_p50=%.1f lost=%d " // strict String(format:) validator rejects the %d/Int mismatch and drops
// the whole line (a cascade error that also mis-blames the float args).
format: "fps=%lld presents=%lld e2e_p50=%.1f e2e_p95=%.1f hostnet_p50=%.1f "
+ "decode_p50=%.1f display_p50=%.1f lost=%lld "
+ "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f", + "floor_p50=%.1f display_adj=%.1f e2e_adj=%.1f queue_p50=%.1f",
frames, frames,
displayWindow?.count ?? 0, displayWindow?.count ?? 0,
@@ -43,6 +43,9 @@ struct GamepadSettingsView: View {
@AppStorage(DefaultsKey.presentPriority) private var presentPriority = @AppStorage(DefaultsKey.presentPriority) private var presentPriority =
SettingsOptions.presentPriorityDefault SettingsOptions.presentPriorityDefault
@AppStorage(DefaultsKey.smoothBuffer) private var smoothBuffer = 0 @AppStorage(DefaultsKey.smoothBuffer) private var smoothBuffer = 0
#if os(macOS)
@AppStorage(DefaultsKey.windowedSafePresent) private var windowedSafePresent = true
#endif
#if os(iOS) #if os(iOS)
@AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false @AppStorage(DefaultsKey.rumbleOnDevice) private var rumbleOnDevice = false
#endif #endif
@@ -345,6 +348,22 @@ struct GamepadSettingsView: View {
detail: "Turn off to use the touch interface even with a controller connected.", detail: "Turn off to use the touch interface even with a controller connected.",
value: $gamepadUIEnabled), value: $gamepadUIEnabled),
] ]
#if os(macOS)
// The windowed safe-present toggle slots in after "Smoothness buffer" (staying inside
// the Video group) macOS only, mirroring the touch SettingsView's Presentation row
// (the DCP swapID-panic mitigation; see DefaultsKey.windowedSafePresent).
if let at = list.firstIndex(where: { $0.id == "smoothBuffer" }) {
list.insert(
toggleRow(
id: "windowedSafePresent", icon: "macwindow.badge.plus",
label: "Safe windowed presentation",
detail: "Windowed streams present in step with the compositor — avoids a "
+ "macOS display-driver crash on high-refresh displays, at a small "
+ "latency cost. Fullscreen always uses the fastest path.",
value: $windowedSafePresent),
at: at + 1)
}
#endif
#if os(iOS) #if os(iOS)
// The device-rumble mirror slots in after "Controller type" (staying inside the // The device-rumble mirror slots in after "Controller type" (staying inside the
// Controller group the next row carries the "Interface" header). iPhone only in // Controller group the next row carries the "Interface" header). iPhone only in
@@ -300,6 +300,18 @@ extension SettingsView {
+ "of added latency. Off shows frames as soon as they're ready.") { + "of added latency. Off shows frames as soon as they're ready.") {
Toggle("V-Sync", isOn: $vsync) Toggle("V-Sync", isOn: $vsync)
} }
// The DCP swapID-panic mitigation's user handle (see DefaultsKey.windowedSafePresent
// for the saga). Default ON: turning it off re-arms a WHOLE-MACHINE kernel panic on
// affected setups, so the caption says so in plain words.
described(windowedSafePresent
? "Windowed streams present in step with the system compositor — avoids a macOS "
+ "display-driver crash seen on high-refresh displays, at a small latency "
+ "cost. Fullscreen always uses the fastest path."
: "Windowed streams use the fastest present path. On some high-refresh setups "
+ "this can crash macOS itself (kernel panic) — turn back on if your Mac "
+ "restarts during windowed streaming.") {
Toggle("Safe windowed presentation", isOn: $windowedSafePresent)
}
#endif #endif
} }
} }
@@ -435,6 +447,14 @@ extension SettingsView {
/// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path). /// (always on macOS; an attached keyboard/mouse on iPad). Absent on tvOS (no such input path).
@ViewBuilder var inputSection: some View { @ViewBuilder var inputSection: some View {
Section("Keyboard & mouse") { Section("Keyboard & mouse") {
#if os(macOS)
described(mouseModeDescription) {
Picker("Mouse input", selection: $mouseMode) {
Text("Capture (games)").tag(MouseInputMode.capture.rawValue)
Text("Desktop (absolute)").tag(MouseInputMode.desktop.rawValue)
}
}
#endif
described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) { described((ModifierLayout(rawValue: modifierLayout) ?? .mac).detail) {
Picker("Modifier keys", selection: $modifierLayout) { Picker("Modifier keys", selection: $modifierLayout) {
ForEach(ModifierLayout.allCases, id: \.self) { layout in ForEach(ModifierLayout.allCases, id: \.self) { layout in
@@ -447,6 +467,20 @@ extension SettingsView {
} }
} }
} }
#if os(macOS)
/// The SELECTED mouse model explained dynamic, like the touch-mode caption.
private var mouseModeDescription: String {
switch MouseInputMode(rawValue: mouseMode) ?? .capture {
case .capture:
return "The pointer locks to the stream and sends relative motion — best for "
+ "games. ⌃⌥⇧M switches live; applies from the next capture otherwise."
case .desktop:
return "The pointer moves freely in and out of the stream and sends absolute "
+ "positions — best for remote desktop work. Unavailable on gamescope hosts."
}
}
#endif
#endif #endif
// MARK: - Audio // MARK: - Audio
@@ -40,6 +40,7 @@ struct SettingsView: View {
@AppStorage(DefaultsKey.smoothBuffer) var smoothBuffer = 0 @AppStorage(DefaultsKey.smoothBuffer) var smoothBuffer = 0
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.vsync) var vsync = false @AppStorage(DefaultsKey.vsync) var vsync = false
@AppStorage(DefaultsKey.windowedSafePresent) var windowedSafePresent = true
#endif #endif
#if !os(tvOS) #if !os(tvOS)
@AppStorage(DefaultsKey.allowVRR) var allowVRR = true @AppStorage(DefaultsKey.allowVRR) var allowVRR = true
@@ -88,6 +89,7 @@ struct SettingsView: View {
@State var customMode = false @State var customMode = false
#endif #endif
#if os(macOS) #if os(macOS)
@AppStorage(DefaultsKey.mouseMode) var mouseMode = MouseInputMode.capture.rawValue
@AppStorage(DefaultsKey.speakerUID) var speakerUID = "" @AppStorage(DefaultsKey.speakerUID) var speakerUID = ""
@AppStorage(DefaultsKey.micUID) var micUID = "" @AppStorage(DefaultsKey.micUID) var micUID = ""
@AppStorage(DefaultsKey.micChannel) var micChannel = 0 @AppStorage(DefaultsKey.micChannel) var micChannel = 0
@@ -221,6 +221,9 @@ public final class PunktfunkConnection {
/// core). The clip *sends* (`clipControl`/`clipOffer`/`clipServe`) share this lock too: /// core). The clip *sends* (`clipControl`/`clipOffer`/`clipServe`) share this lock too:
/// they're quick non-blocking enqueues, and a single lock keeps close() ordering simple. /// they're quick non-blocking enqueues, and a single lock keeps close() ordering simple.
private let clipboardLock = NSLock() private let clipboardLock = NSLock()
/// Serializes the (single) cursor pull thread against close() both cursor planes are
/// drained by ONE thread, so one lock covers them.
private let cursorLock = NSLock()
/// Negotiated session mode (host-confirmed). /// Negotiated session mode (host-confirmed).
public private(set) var width: UInt32 = 0 public private(set) var width: UInt32 = 0
@@ -381,6 +384,98 @@ public final class PunktfunkConnection {
public var hostSupportsClipboard: Bool { public var hostSupportsClipboard: Bool {
hostCaps & UInt8(PUNKTFUNK_HOST_CAP_CLIPBOARD) != 0 hostCaps & UInt8(PUNKTFUNK_HOST_CAP_CLIPBOARD) != 0
} }
/// The host answered `HOST_CAP_CURSOR`: it stopped compositing the pointer and forwards
/// shape/state on the cursor planes the client MUST draw the cursor locally.
public var hostSupportsCursor: Bool {
hostCaps & 0x04 != 0
}
/// One forwarded host-cursor shape (the cursor channel, ABI v11): straight-alpha RGBA,
/// `rgba.count == width * height * 4`, hotspot within the bitmap. Cache by `serial`
/// states reference shapes by it and a re-shown serial never resends pixels.
public struct CursorShapeEvent: Sendable {
public let serial: UInt32
public let width: Int
public let height: Int
public let hotX: Int
public let hotY: Int
public let rgba: Data
}
/// Per-host-tick cursor state: position (host video px, the pointer/hotspot point),
/// visibility, and the host-driven relative-mode hint (an app grabbed/hid the pointer
/// run captured relative; clear absolute, reappearing at `x`/`y`). Latest-wins.
public struct CursorStateEvent: Sendable {
public let serial: UInt32
public let visible: Bool
public let relativeHint: Bool
public let x: Int32
public let y: Int32
}
/// Pull the next forwarded cursor SHAPE (nil = timeout). Only a session connected with
/// `clientCaps` cursor bit against a `hostSupportsCursor` host receives any. Drain shape
/// AND state from ONE dedicated cursor thread (they share a lock).
public func nextCursorShape(timeoutMs: UInt32 = 0) throws -> CursorShapeEvent? {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkCursorShape()
let rc = punktfunk_connection_next_cursor_shape(h, &out, timeoutMs)
switch rc {
case statusOK:
// Copy out of the ABI borrow (valid until the next shape call) immediately.
let bytes = out.rgba.map { Data(bytes: $0, count: Int(out.len)) } ?? Data()
return CursorShapeEvent(
serial: out.serial, width: Int(out.w), height: Int(out.h),
hotX: Int(out.hot_x), hotY: Int(out.hot_y), rgba: bytes)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Pull the next cursor STATE (nil = timeout). Latest-wins drain the queue and apply
/// only the newest. Same thread + gate as [`nextCursorShape`].
public func nextCursorState(timeoutMs: UInt32 = 0) throws -> CursorStateEvent? {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { throw PunktfunkClientError.closed }
var out = PunktfunkCursorState()
let rc = punktfunk_connection_next_cursor_state(h, &out, timeoutMs)
switch rc {
case statusOK:
return CursorStateEvent(
serial: out.serial,
visible: out.flags & 0x01 != 0,
relativeHint: out.flags & 0x02 != 0,
x: out.x, y: out.y)
case statusNoFrame:
return nil
case statusClosed:
throw PunktfunkClientError.closed
default:
throw PunktfunkClientError.status(rc)
}
}
/// Tell the host who renders the pointer (the §8 mid-stream mouse-model flip, ABI v12):
/// `clientDraws = true` this client draws it locally (the desktop mouse model; the host
/// excludes the pointer from the video and forwards shape/state); `false` the host
/// composites it into the video (the capture model, full fidelity). Idempotent,
/// latest-wins; harmless against hosts without the cursor cap. Fire-and-forget errors
/// are swallowed (a closed session is the only failure and it moots the flip).
public func setCursorRender(clientDraws: Bool) {
cursorLock.lock()
defer { cursorLock.unlock() }
guard let h = liveHandle() else { return }
_ = punktfunk_connection_set_cursor_render(h, clientDraws)
}
/// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing /// The resolved codec as a `VideoCodec` (H.264 / HEVC / AV1) drives the bitstream framing
/// (Annex-B NAL parsing vs the AV1 OBU repack). /// (Annex-B NAL parsing vs the AV1 OBU repack).
public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) } public var videoCodec: VideoCodec { VideoCodec(wire: resolvedCodec) }
@@ -417,6 +512,7 @@ public final class PunktfunkConnection {
audioChannels: UInt8 = 2, audioChannels: UInt8 = 2,
videoCodecs: UInt8 = 0x02, // PUNKTFUNK_CODEC_HEVC the codecs this client can decode videoCodecs: UInt8 = 0x02, // PUNKTFUNK_CODEC_HEVC the codecs this client can decode
preferredCodec: UInt8 = 0, // 0 = auto; else PUNKTFUNK_CODEC_* soft preference preferredCodec: UInt8 = 0, // 0 = auto; else PUNKTFUNK_CODEC_* soft preference
clientCaps: UInt8 = 0, // ABI v11: PUNKTFUNK_CLIENT_CAP_CURSOR = render the host cursor locally
launchID: String? = nil, launchID: String? = nil,
timeoutMs: UInt32 = 10_000 timeoutMs: UInt32 = 10_000
) throws { ) throws {
@@ -436,18 +532,18 @@ public final class PunktfunkConnection {
withOptionalCString(launchID) { launch in withOptionalCString(launchID) { launch in
if let pin = pinSHA256 { if let pin = pinSHA256 {
return pin.withUnsafeBytes { p in return pin.withUnsafeBytes { p in
punktfunk_connect_ex8( punktfunk_connect_ex9(
cs, port, width, height, refreshHz, compositor.rawValue, cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, launch, videoCodecs, preferredCodec, clientCaps, launch,
p.bindMemory(to: UInt8.self).baseAddress, &observed, p.bindMemory(to: UInt8.self).baseAddress, &observed,
cert, key, timeoutMs, &connectStatus) cert, key, timeoutMs, &connectStatus)
} }
} }
return punktfunk_connect_ex8( return punktfunk_connect_ex9(
cs, port, width, height, refreshHz, compositor.rawValue, cs, port, width, height, refreshHz, compositor.rawValue,
gamepad.rawValue, bitrateKbps, videoCaps, audioChannels, gamepad.rawValue, bitrateKbps, videoCaps, audioChannels,
videoCodecs, preferredCodec, launch, videoCodecs, preferredCodec, clientCaps, launch,
nil, &observed, cert, key, timeoutMs, &connectStatus) nil, &observed, cert, key, timeoutMs, &connectStatus)
} }
} }
@@ -1059,10 +1155,12 @@ public final class PunktfunkConnection {
feedbackLock.lock() feedbackLock.lock()
statsLock.lock() statsLock.lock()
clipboardLock.lock() clipboardLock.lock()
cursorLock.lock()
abiLock.lock() abiLock.lock()
let h = handle let h = handle
handle = nil handle = nil
abiLock.unlock() abiLock.unlock()
cursorLock.unlock()
clipboardLock.unlock() clipboardLock.unlock()
statsLock.unlock() statsLock.unlock()
feedbackLock.unlock() feedbackLock.unlock()
@@ -110,11 +110,11 @@ public final class InputCapture {
/// event itself is swallowed). Main queue. /// event itself is swallowed). Main queue.
public var onToggleCapture: (() -> Void)? public var onToggleCapture: (() -> Void)?
/// Fired on C (the client-side-cursor toggle flips between the captured/disassociated /// Fired on M (the mouse-model flip, capture desktop cross-client parity with the
/// relative path and the visible-cursor absolute path; detected here, like , so it works /// SDL clients' Ctrl+Alt+Shift+M; detected here, like , so it works regardless of the
/// regardless of the current capture state and the event itself is swallowed). macOS only; /// current capture state and the event itself is swallowed). macOS only; the
/// the absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue. /// absolute-vs-relative forwarding lives entirely in StreamLayerView. Main queue.
public var onToggleCursor: (() -> Void)? public var onToggleMouseMode: (() -> Void)?
/// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS /// The cross-client combos (Windows/Linux parity: Ctrl+Alt+Shift+Q/D/S), fired from the macOS
/// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which /// keyDown monitor only WHILE FORWARDING that's the state in which the app's menu (which
@@ -245,13 +245,14 @@ public final class InputCapture {
self.onToggleCapture?() self.onToggleCapture?()
return nil return nil
} }
// C toggles the client-side cursor (visible-cursor absolute path vs the // M flips the mouse model (capture desktop the SDL clients' identical
// captured relative path). keyCode 8 = kVK_ANSI_C; layout-independent so it // chord). Detected in both capture states, like , so the model can be set
// fires the same on any keyboard. Suppress the C (latched like 's Esc) so it // before engaging. keyCode 46 = kVK_ANSI_M; layout-independent. Suppress the M
// doesn't type into the host, and swallow the event so it doesn't beep. // (latched like 's Esc) so it doesn't type into the host, and swallow the
if event.keyCode == 8 /* C */, flags == [.command, .shift] { // event so it doesn't beep.
self.suppressedVK = 0x43 // VK_C the same physical C is en route via GC if event.keyCode == 46 /* M */, flags == [.control, .option, .shift] {
self.onToggleCursor?() self.suppressedVK = 0x4D // VK_M the same physical M is en route via GC
self.onToggleMouseMode?()
return nil return nil
} }
// The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other // The cross-client combos (Ctrl+Alt+Shift+Q/D/S the same set every other
@@ -0,0 +1,12 @@
/// How a physical mouse drives the host the cross-client mouse model (the SDL clients'
/// `MouseMode` / `Settings::mouse_mode`, design/remote-desktop-sweep.md M1). Stored stringly
/// under `DefaultsKey.mouseMode`.
public enum MouseInputMode: String, CaseIterable, Sendable {
/// Pointer capture (disassociated, hidden cursor, relative deltas) the game model,
/// and the default: the only cursor you see is the host's.
case capture
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion is forwarded as absolute positions through the letterbox. The remote desktop
/// model. Requires a host injector with absolute support (not gamescope).
case desktop
}
@@ -23,6 +23,34 @@ import os
private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter") private let presenterLog = Logger(subsystem: "io.unom.punktfunk", category: "presenter")
#if os(macOS)
/// HOW a windowed (composited) macOS session pushes finished frames to glass the DCP
/// "mismatched swapID's" kernel-panic saga's mechanism picker. Fullscreen always presents
/// `async` (direct-scanout promotion, lowest latency, no panic reports there); the windowed
/// mechanism is resolved per session by SessionPresenter (user setting +
/// PUNKTFUNK_WINDOWED_PRESENT env override) and routed here via `setWindowedPresent`.
///
/// - `async`: the CAMetalLayer image queue (`commandBuffer.present`) the fastest composited
/// path and the PANIC TRIGGER on high-refresh displays (the out-of-band swaps race
/// WindowServer's compositor; it survived glass pacing and every codec).
/// - `transaction`: `CAMetalLayer.presentsWithTransaction` the swap commits WITH the layer
/// tree, in lockstep with the compositor (Apple's documented remedy; validated no-panic on
/// the 240 Hz repro machine). The present is committed from the RENDER thread inside an
/// explicit CATransaction + flush see `encodePresent` for why that beats the original
/// main-thread hop.
/// - `surface`: no image queue at all render into a pooled IOSurface and swap it into a plain
/// CALayer's `contents` (the f407f418 PyroWave mitigation, resurrected format-aware:
/// rgba16Float + PQ tagging keeps HDR). WindowServer treats it as ordinary layer damage on
/// its own composite cadence. PROTOTYPE: whether the compositor honors PQ/EDR for plain-layer
/// IOSurface contents still needs an on-glass eyeball the metal layer stays underneath with
/// `wantsExtendedDynamicRangeContent` as the EDR anchor.
enum WindowedPresentMode: String, Sendable {
case async
case transaction
case surface
}
#endif
/// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the /// HDR reference white (BT.2408 "HDR Reference White"): the absolute luminance, in nits, that the
/// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors /// PQ signal's diffuse white sits at. Passed to `CAEDRMetadata.hdr10(opticalOutputScale:)`, it anchors
/// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the /// 203-nit diffuse white at EDR 1.0 (the display's SDR-white level) and lets the system tone-map the
@@ -198,8 +226,8 @@ fragment float4 pf_frag_hdr(VOut in [[stage_in]],
// in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.) // in a genuine HDR10 output, PQ passthrough is the correct emission and the TV tone-maps.)
// The shared PQdisplay-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084 // The shared PQdisplay-referred-SDR tail (see pf_frag_hdr_tv's rationale above): ST 2084
// EOTF 203-nit-anchored scene light BT.2020709 primaries extended-Reinhard rolloff // EOTF 203-nit-anchored scene light BT.2020709 primaries extended-Reinhard rolloff
// BT.709 OETF. Used by the tvOS biplanar tone-map and the planar (PyroWave) tone-map the // BT.709 OETF. Used by the tvOS biplanar tone-map and the tvOS planar (PyroWave) tone-map (the
// latter also on macOS windowed sessions, whose IOSurface present path is BGRA8-only. // no-HDR-headroom fallback). macOS keeps real HDR windowed now see `WindowedPresentMode`.
static inline float3 pqToSdr(float3 pq) { static inline float3 pqToSdr(float3 pq) {
const float m1 = 2610.0/16384.0; const float m1 = 2610.0/16384.0;
const float m2 = 78.84375; const float m2 = 78.84375;
@@ -230,8 +258,7 @@ fragment float4 pf_frag_hdr_tv(VOut in [[stage_in]],
// PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows // PyroWave planar HDR tone-map: three separate R16 planes (P010-style studio codes; the rows
// fold in depth-10 MSB packing) PQ RGB the shared SDR tail. Used when a PQ pyrowave // fold in depth-10 MSB packing) PQ RGB the shared SDR tail. Used when a PQ pyrowave
// stream must land on an 8-bit surface: tvOS without HDR headroom, and macOS WINDOWED sessions // stream must land on an 8-bit surface: tvOS without HDR headroom. The passthrough planar
// (the IOSurface present path the DCP-panic mitigation is BGRA8). The passthrough planar
// HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math the // HDR pipeline reuses pf_frag_planar itself on an rgba16Float drawable (identical math the
// layer's itur_2100_PQ colour space + EDR metadata do the interpretation). // layer's itur_2100_PQ colour space + EDR metadata do the interpretation).
fragment float4 pf_frag_planar_tm(VOut in [[stage_in]], fragment float4 pf_frag_planar_tm(VOut in [[stage_in]],
@@ -259,21 +286,51 @@ public final class MetalVideoPresenter {
public let layer: CAMetalLayer public let layer: CAMetalLayer
#if os(macOS) #if os(macOS)
/// The WINDOWED-mode PyroWave present target: a plain CALayer sized like `layer` (installed /// WINDOWED-mode present coordination the macOS DCP KERNEL PANIC mitigation.
/// as a sibling ABOVE it), fed IOSurfaces via `contents` inside ordinary CATransactions.
/// ///
/// Why this exists the macOS DCP KERNEL PANIC ("mismatched swapID's" @UnifiedPipeline.cpp, /// The panic ("mismatched swapID's" @UnifiedPipeline.cpp, WindowServer dies, machine reboots):
/// WindowServer dies, machine reboots): out-of-band CAMetalLayer image-queue swaps into a /// the CAMetalLayer's ASYNCHRONOUS image queue (`commandBuffer.present(drawable)` an
/// COMPOSITED (windowed) session race WindowServer's own swap submissions on high-refresh /// out-of-band flip, mandatory with `displaySyncEnabled=false`) diverges from WindowServer's
/// displays, and the race survives glass pacing a fully serialized one-in-flight present /// compositor on a high-refresh COMPOSITED (windowed) session the compositor's notion of the
/// stream still panicked a 240 Hz Mac Studio (2026-07-18, twice). So in windowed mode we stop /// current swap and the layer's queued swap disagree, and the DCP asserts. It survived glass
/// using the image queue entirely and present the way video players do: render the planar CSC /// pacing: a fully serialized one-in-flight present stream still panicked a 240 Hz Mac Studio
/// into an IOSurface pool and swap `contents` on main WindowServer treats it as ordinary /// (2026-07-18, PyroWave), and a windowed HEVC session panicked the same machine 2026-07-21
/// damage on its own composite cadence, coalescing faster-than-refresh updates instead of /// so it is the async image queue itself, at any pacing or codec, not a present rate.
/// latching queue swaps mid-cycle. Fullscreen keeps the CAMetalLayer path (direct-scanout ///
/// promotion, no compositing, no panic reports). Contents updates are transparent to the /// The fix keeps the full render path (rgba16Float / PQ / EDR real HDR is preserved) and
/// layer below when nil, so flipping modes just covers/uncovers the metal layer. /// only changes HOW the drawable is presented: `CAMetalLayer.presentsWithTransaction`. With it
public let surfaceLayer: CALayer = { /// set, we don't hand the drawable to the command buffer; we commit, wait until scheduled, then
/// call `drawable.present()` INSIDE a CATransaction the present is enrolled in Core
/// Animation's transaction and committed together with the layer tree, so the swap stays in
/// lockstep with the compositor instead of racing it (Apple's documented remedy for Metal
/// presentation drifting out of sync with CA). Fullscreen keeps the async path (direct-scanout
/// promotion, lowest latency, no compositor and no panic reports there).
///
/// 2026-07-21 latency rework: the mitigation MECHANISM is now a three-way pick
/// (`WindowedPresentMode`) and the transactional present commits from the RENDER thread
/// see `encodePresent`. Staged under `stagingLock` (main pushes it via
/// `setComposited``setWindowedPresent`); the render thread drains it and toggles the layer
/// property + present style. `Active` is the render-thread copy so the layer property flips
/// exactly once per mode change.
private var windowedPresentStaged: WindowedPresentMode = .async
private var windowedPresentActive: WindowedPresentMode = .async
/// PUNKTFUNK_TXN_PRESENT=main the ORIGINAL transactional present (commit
/// waitUntilScheduled hop to the MAIN thread and present inside its CATransaction), kept
/// as a field A/B lever. The default is the render-thread commit: the present harness
/// (2026-07-21, this saga) measured the main hop landing a runloop turn late on a busy main
/// thread, and an ACTIVE implicit transaction there NESTS the explicit one presents batch
/// at runloop-iteration rate (the field's presents=55 @ fps=240, display_p50 18.6 ms).
/// Off-main commits measured immune to main-thread churn (~10 ms glass p50 at 240 Hz
/// full-size vs 14+ ms under a churned main hop).
private let txnPresentOnMain =
ProcessInfo.processInfo.environment["PUNKTFUNK_TXN_PRESENT"] == "main"
/// The WINDOWED-mode `surface` present target: a plain CALayer sized like `layer` (installed
/// as a sibling ABOVE it by SessionPresenter), fed IOSurfaces via `contents` inside explicit
/// CATransactions. Transparent (nil contents) whenever surface mode is off, so the metal
/// layer below shows through. See `WindowedPresentMode.surface`.
let surfaceLayer: CALayer = {
let l = CALayer() let l = CALayer()
l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout l.contentsGravity = .resize // frame is already aspect-fit + pixel-snapped by layout
l.isOpaque = true l.isOpaque = true
@@ -281,8 +338,8 @@ public final class MetalVideoPresenter {
return l return l
}() }()
/// One IOSurface-backed render target of the windowed present pool. All pool state is /// One IOSurface-backed render target of the windowed surface-present pool. All pool state
/// RENDER-THREAD confined; only the immutable surface refs cross to main (contents swap). /// is RENDER-THREAD confined; only the immutable surface refs cross threads (contents swap).
private struct SurfaceSlot { private struct SurfaceSlot {
let surface: IOSurfaceRef let surface: IOSurfaceRef
let texture: MTLTexture let texture: MTLTexture
@@ -292,15 +349,52 @@ public final class MetalVideoPresenter {
private var surfacePool: [SurfaceSlot] = [] private var surfacePool: [SurfaceSlot] = []
private var surfacePoolSize: CGSize = .zero private var surfacePoolSize: CGSize = .zero
private var surfacePoolHDR = false
private var surfaceSeq: UInt64 = 0 private var surfaceSeq: UInt64 = 0
/// Index of the slot most recently handed to the layer never rewritten next, even if its /// Index of the slot most recently handed to the layer never rewritten next, even if its
/// use count already dropped (the compositor may still be scanning out the previous frame). /// use count already dropped (the compositor may still be scanning out the previous frame).
private var lastHandedOff: Int? private var lastHandedOff: Int?
/// Staged (under `stagingLock`, like every cross-thread input): the hosting view's windowed
/// vs fullscreen state, pushed from main via `setSurfacePresents`. Drained in `renderPlanar`. /// Once-per-second decomposition of the ACTIVE windowed present path (the field-diagnosis
private var surfacePresentsStaged = false /// half of the DCP-latency work): scheduled/completed wait + commit/flush cost per present,
/// Render-thread copy, so pool teardown happens exactly once on a mode flip. /// and how many presents/swaps were issued. The pf-present line shows the GLASS side
private var surfacePresentsActive = false /// (latchMs / dropped); this shows the ISSUE side. Logged via `presenterLog` only while a
/// windowed mechanism is active (zero cost fullscreen). Lock-guarded: transaction mode
/// records from the render thread, surface mode from Metal completion threads.
private final class WindowedPresentDiag: @unchecked Sendable {
private let lock = NSLock()
private var presents = 0
private var schedMs: [Double] = []
private var commitMs: [Double] = []
private var last = CACurrentMediaTime()
func record(schedMs sched: Double, commitMs commit: Double, mode: WindowedPresentMode) {
lock.lock()
presents += 1
schedMs.append(sched)
commitMs.append(commit)
let now = CACurrentMediaTime()
guard now - last >= 1 else {
lock.unlock()
return
}
last = now
let sSched = schedMs.sorted()
let sCommit = commitMs.sorted()
let line = String(
format: "pf-windowed mode=%@ presents=%d schedMs p50=%.2f max=%.2f "
+ "commitMs p50=%.2f max=%.2f",
mode.rawValue, presents, sSched[sSched.count / 2], sSched.last ?? 0,
sCommit[sCommit.count / 2], sCommit.last ?? 0)
presents = 0
schedMs.removeAll(keepingCapacity: true)
commitMs.removeAll(keepingCapacity: true)
lock.unlock()
presenterLog.info("\(line, privacy: .public)")
}
}
private let windowedDiag = WindowedPresentDiag()
#endif #endif
private let device: MTLDevice private let device: MTLDevice
@@ -316,7 +410,7 @@ public final class MetalVideoPresenter {
private let pipelinePlanar: MTLRenderPipelineState private let pipelinePlanar: MTLRenderPipelineState
/// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour /// PyroWave planar HDR passthrough (pf_frag_planar rgba16Float; the layer's PQ colour
/// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm /// space + EDR interpret the samples) and the planar PQSDR tone-map (pf_frag_planar_tm
/// bgra8; tvOS without headroom + macOS windowed IOSurface presents). /// bgra8; tvOS without HDR headroom).
private let pipelinePlanarHDR: MTLRenderPipelineState private let pipelinePlanarHDR: MTLRenderPipelineState
private let pipelinePlanarToneMap: MTLRenderPipelineState private let pipelinePlanarToneMap: MTLRenderPipelineState
private var textureCache: CVMetalTextureCache? private var textureCache: CVMetalTextureCache?
@@ -591,13 +685,14 @@ public final class MetalVideoPresenter {
} }
#if os(macOS) #if os(macOS)
/// Park the windowed-vs-fullscreen present routing (MAIN thread the hosting view pushes its /// Park the windowed present mechanism (MAIN thread the hosting view pushes its window
/// window state on every layout). true = PyroWave frames present via `surfaceLayer` contents /// state on every layout; SessionPresenter resolves the mechanism per session). `.async` =
/// (the DCP swapID-panic mitigation see `surfaceLayer`); false = the CAMetalLayer path. /// FULLSCREEN (or the user opted out of the mitigation): the image queue. `.transaction` /
/// `.surface` = COMPOSITED (windowed) mitigation mechanisms see `WindowedPresentMode`.
/// Applied by the render thread on the next frame, like every other staged value here. /// Applied by the render thread on the next frame, like every other staged value here.
public func setSurfacePresents(_ on: Bool) { func setWindowedPresent(_ mode: WindowedPresentMode) {
stagingLock.lock() stagingLock.lock()
surfacePresentsStaged = on windowedPresentStaged = mode
stagingLock.unlock() stagingLock.unlock()
} }
#endif #endif
@@ -734,36 +829,12 @@ public final class MetalVideoPresenter {
) -> Bool { ) -> Bool {
stagingLock.lock() stagingLock.lock()
let targetFromLayout = drawableTarget let targetFromLayout = drawableTarget
#if os(macOS)
let surfaceMode = surfacePresentsStaged
#endif
stagingLock.unlock() stagingLock.unlock()
// A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path; // A PQ (HDR) pyrowave stream drives the same layer/EDR machinery as the biplanar path
// macOS WINDOWED sessions stay on the SDR layer (the IOSurface path tone-maps in-shader). // including macOS windowed sessions, which keep real HDR (the DCP mitigation is the
#if os(macOS) // transactional present in `encodePresent`, not a colour downgrade).
configure(hdr: planes.pq && !surfaceMode)
#else
configure(hdr: planes.pq) configure(hdr: planes.pq)
#endif
var csc = planes.csc var csc = planes.csc
#if os(macOS)
if surfaceMode != surfacePresentsActive {
surfacePresentsActive = surfaceMode
presenterLog.info(
"stage2: windowed surface presents \(surfaceMode ? "ON" : "OFF", privacy: .public) (PyroWave DCP-panic mitigation)")
if !surfaceMode {
// Back to the metal path (fullscreen): drop the pool at 5K it holds >100 MB,
// and re-entering windowed mode rebuilds it in one frame.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
}
if surfaceMode {
return renderPlanarToSurface(
planes, targetFromLayout: targetFromLayout, csc: &csc, onPresented: onPresented)
}
#endif
// PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still) // PQ passthrough needs the HDR drawable; a PQ frame while the drawable is (still)
// 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps // 8-bit tvOS without display headroom, or a not-yet-flipped layer tone-maps
// in-shader instead (the pipeline must match the drawable's pixel format). // in-shader instead (the pipeline must match the drawable's pixel format).
@@ -792,118 +863,6 @@ public final class MetalVideoPresenter {
} }
} }
#if os(macOS)
/// The windowed-mode present tail (see `surfaceLayer` for why this path exists): render the
/// planar CSC into a pooled IOSurface and hand it to `surfaceLayer.contents` on MAIN inside a
/// plain CATransaction an ordinary damaged-layer update on WindowServer's own composite
/// cadence, no CAMetalLayer image-queue swap anywhere. `presentAtMediaTime` doesn't apply
/// (the compositor paces); `onPresented` fires after the contents swap is committed, stamped
/// with CLOCK_REALTIME then the closest observable analogue of "reached glass" here (the
/// composite follows within a refresh, so the meters' display stage reads slightly optimistic).
private func renderPlanarToSurface(
_ planes: WaveletPlanes, targetFromLayout: CGSize, csc: inout CscUniform,
onPresented: ((Int64?) -> Void)?
) -> Bool {
let decodedSize = CGSize(width: planes.width, height: planes.height)
let targetSize = (targetFromLayout.width > 0 && targetFromLayout.height > 0)
? targetFromLayout : decodedSize
ensureSurfacePool(size: targetSize)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(planes.pq ? pipelinePlanarToneMap : pipelinePlanar)
encoder.setFragmentTexture(planes.y, index: 0)
encoder.setFragmentTexture(planes.cb, index: 1)
encoder.setFragmentTexture(planes.cr, index: 2)
encoder.setFragmentBytes(&csc, length: MemoryLayout<CscUniform>.stride, index: 0)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let keepAlive: [Any] = [planes.y, planes.cb, planes.cr]
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // ring textures pinned until the GPU finished sampling
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
onPresented?(
Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size` 4 BGRA8 IOSurface render targets (one on glass, one queued
/// in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the pool empty;
/// the caller returns false and the ring's putBack + display-link retry take over.
private func ensureSurfacePool(size: CGSize) {
guard size != surfacePoolSize else { return }
surfacePool.removeAll()
surfacePoolSize = size
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// 256-byte row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerRow = ((w * 4) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: 4,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one /// The shared present tail of `render`/`renderPlanar`: size the drawable, encode one
/// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule /// fullscreen triangle with `pipeline` (`bind` supplies the fragment resources), schedule
/// the present and the on-glass callback. /// the present and the on-glass callback.
@@ -936,6 +895,36 @@ public final class MetalVideoPresenter {
#if DEBUG #if DEBUG
logSizeIfChanged(decoded: decodedSize, drawable: targetSize) logSizeIfChanged(decoded: decodedSize, drawable: targetSize)
#endif #endif
#if os(macOS)
// Windowed (composited) the DCP swapID-panic mitigation mechanism (see
// `WindowedPresentMode`). Toggle the layer property BEFORE vending a drawable so the
// vend matches how it will be presented; drained here on the render thread, flipped
// exactly once per mode change.
stagingLock.lock()
let windowedMode = windowedPresentStaged
stagingLock.unlock()
if windowedMode != windowedPresentActive {
windowedPresentActive = windowedMode
layer.presentsWithTransaction = windowedMode == .transaction
if windowedMode != .surface, !surfacePool.isEmpty {
// Leaving surface mode (fullscreen entry / mechanism A/B): drop the pool at 5K
// it holds >100 MB, and re-entering rebuilds it in one frame. SessionPresenter
// clears the surface layer's contents on main.
surfacePool.removeAll()
surfacePoolSize = .zero
lastHandedOff = nil
}
presenterLog.info(
"stage2: windowed present mode \(windowedMode.rawValue, privacy: .public) (DCP swapID-panic mitigation)")
}
if windowedMode == .surface {
// No image queue at all: render into a pooled IOSurface and swap it into the
// sibling layer's contents. The drawable/queue tail below never runs.
return encodeToSurface(
targetSize: targetSize, pipeline: pipeline, onPresented: onPresented,
keepAlive: keepAlive, bind: bind)
}
#endif
if let providedDrawable, if let providedDrawable,
providedDrawable.texture.pixelFormat != layer.pixelFormat { providedDrawable.texture.pixelFormat != layer.pixelFormat {
return false // config outran the vend (HDR flip) next vend has the new format return false // config outran the vend (HDR flip) next vend has the new format
@@ -974,6 +963,61 @@ public final class MetalVideoPresenter {
} }
#endif #endif
} }
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
#if os(macOS)
if windowedPresentActive == .transaction {
// Windowed DCP mitigation: present the drawable THROUGH a Core Animation transaction
// (`presentsWithTransaction`, set above) instead of the async image queue, so the swap
// commits with the layer tree and stays in lockstep with the compositor (no out-of-band
// flip to race WindowServer's swaps). Wait until the GPU work is scheduled (contents
// will be ready p50 ~0.1 ms), then present inside an EXPLICIT CATransaction ON THIS
// RENDER THREAD and `flush()`. `presentAtMediaTime` does not apply the transaction
// paces.
//
// Threading history, because BOTH failure modes shipped or nearly shipped:
// A bare `present()` from this thread (no transaction) never flushes nothing
// commits a runloop-less thread's implicit transaction, so drawables are never
// released; after maximumDrawableCount vends `nextDrawable()` blocks forever and
// the stream FREEZES (the fullscreenwindowed switch did exactly this).
// The explicit begin/commit alone is NOT enough either: this thread has an ACTIVE
// implicit transaction (the layer mutations above drawableSize/colour created
// it), so the explicit transaction NESTS inside it and its commit defers to the
// implicit one that never comes. The harness reproduced the exact freeze: every
// present reported presentedTime=0, nothing reached glass. `CATransaction.flush()`
// pushes the implicit transaction (present included) to the render server NOW.
// The original fix hopped to MAIN and presented there correct, but slow in the
// field (presents=55 @ fps=240, display_p50 18.6 ms on the 240 Hz Studio): each
// present lands a runloop turn late, and main's own implicit transaction batches
// enrolled presents at runloop-iteration rate. Kept as PUNKTFUNK_TXN_PRESENT=main.
// The off-main commit measured immune to main-thread churn in the harness
// (2026-07-21: glass p50 ~10 ms at 240 Hz full-size, cadence a clean 4.17 ms).
commandBuffer.commit()
let schedStart = CACurrentMediaTime()
commandBuffer.waitUntilScheduled()
let schedMs = (CACurrentMediaTime() - schedStart) * 1000
let commitStart = CACurrentMediaTime()
if txnPresentOnMain {
let presentedDrawable = drawable
DispatchQueue.main.async {
CATransaction.begin()
CATransaction.setDisableActions(true)
presentedDrawable.present()
CATransaction.commit()
}
} else {
CATransaction.begin()
CATransaction.setDisableActions(true)
drawable.present()
CATransaction.commit()
CATransaction.flush()
}
windowedDiag.record(
schedMs: schedMs, commitMs: (CACurrentMediaTime() - commitStart) * 1000,
mode: .transaction)
return true
}
#endif
// Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment); // Scheduled on the vsync when the pipeline gave us the link's target (see the doc comment);
// immediate otherwise. A target already in the past presents immediately same thing. // immediate otherwise. A target already in the past presents immediately same thing.
if let presentAtMediaTime { if let presentAtMediaTime {
@@ -981,12 +1025,146 @@ public final class MetalVideoPresenter {
} else { } else {
commandBuffer.present(drawable) commandBuffer.present(drawable)
} }
// Keep the bound sources alive until the GPU finishes sampling (see the callers).
commandBuffer.addCompletedHandler { _ in _ = keepAlive }
commandBuffer.commit() commandBuffer.commit()
return true return true
} }
#if os(macOS)
/// The WINDOWED `surface` present tail (see `WindowedPresentMode.surface`): render with the
/// same per-frame pipeline into a pooled IOSurface and hand it to `surfaceLayer.contents`
/// from the command buffer's COMPLETION handler, inside an explicit CATransaction + flush
/// (the same off-main commit discipline as the transactional present an ordinary
/// damaged-layer update on WindowServer's own composite cadence, no image queue anywhere).
/// RENDER THREAD. `onPresented` is stamped right after the contents swap commits the
/// closest observable analogue of "reached glass" here (the composite follows within a
/// refresh, so the display-stage meters read slightly OPTIMISTIC in this mode).
///
/// The pool tracks `hdrActive`: bgra8 for SDR, rgba16Float tagged BT.2100 PQ for HDR
/// `configure` already ran, so the caller's `pipeline` attachment format always matches.
/// HDR OPEN RISK (why this whole mode is a prototype): whether the compositor honors the
/// PQ tag + EDR for plain-CALayer IOSurface contents needs an on-glass eyeball; the metal
/// layer underneath keeps `wantsExtendedDynamicRangeContent` as the EDR anchor (the harness
/// measured the display's EDR headroom engaging with this arrangement).
private func encodeToSurface(
targetSize: CGSize, pipeline: MTLRenderPipelineState,
onPresented: ((Int64?) -> Void)?,
keepAlive: [Any], bind: (MTLRenderCommandEncoder) -> Void
) -> Bool {
ensureSurfacePool(size: targetSize, hdr: hdrActive)
guard let slotIndex = takeSurfaceSlot(),
let commandBuffer = queue.makeCommandBuffer()
else { return false }
let slot = surfacePool[slotIndex]
let pass = MTLRenderPassDescriptor()
pass.colorAttachments[0].texture = slot.texture
pass.colorAttachments[0].loadAction = .clear
pass.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
pass.colorAttachments[0].storeAction = .store
guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: pass) else {
return false
}
encoder.setRenderPipelineState(pipeline)
bind(encoder)
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3)
encoder.endEncoding()
let surface = slot.surface
let surfaceLayer = surfaceLayer // captured directly the handler must not retain self
let diag = windowedDiag
let commitStamp = CACurrentMediaTime()
commandBuffer.addCompletedHandler { _ in
_ = keepAlive // sources pinned until the GPU finished sampling
let completedAt = CACurrentMediaTime()
// Swap on THIS Metal completion thread: explicit transaction + flush, so the commit
// reaches the render server now, independent of main (completion handlers for one
// queue fire in execution order, so swaps can't reorder).
CATransaction.begin()
CATransaction.setDisableActions(true)
surfaceLayer.contents = surface
CATransaction.commit()
CATransaction.flush()
diag.record(
schedMs: (completedAt - commitStamp) * 1000,
commitMs: (CACurrentMediaTime() - completedAt) * 1000, mode: .surface)
onPresented?(Stage2Pipeline.realtimeNs(forDisplayLinkTimestamp: CACurrentMediaTime()))
}
commandBuffer.commit()
lastHandedOff = slotIndex
return true
}
/// (Re)build the pool at `size`/`hdr` 4 IOSurface render targets (one on glass, one
/// committed in CA, one rendering, one spare). RENDER THREAD. A failed allocation leaves the
/// pool empty; the caller returns false and the ring's putBack + display-link retry take
/// over.
private func ensureSurfacePool(size: CGSize, hdr: Bool) {
guard size != surfacePoolSize || hdr != surfacePoolHDR else { return }
surfacePool.removeAll()
surfacePoolSize = size
surfacePoolHDR = hdr
lastHandedOff = nil
let w = Int(size.width)
let h = Int(size.height)
guard w > 0, h > 0 else { return }
// rgba16Float (8 B/px) carries the PQ-encoded HDR samples; bgra8 the SDR ones. 256-byte
// row alignment satisfies both IOSurface and Metal linear-texture rules.
let bytesPerElement = hdr ? 8 : 4
let bytesPerRow = ((w * bytesPerElement) + 255) & ~255
let props: [String: Any] = [
kIOSurfaceWidth as String: w,
kIOSurfaceHeight as String: h,
kIOSurfaceBytesPerElement as String: bytesPerElement,
kIOSurfaceBytesPerRow as String: bytesPerRow,
kIOSurfacePixelFormat as String: hdr
? kCVPixelFormatType_64RGBAHalf : kCVPixelFormatType_32BGRA,
]
let desc = MTLTextureDescriptor.texture2DDescriptor(
pixelFormat: hdr ? .rgba16Float : .bgra8Unorm, width: w, height: h, mipmapped: false)
desc.usage = [.renderTarget]
desc.storageMode = .shared
for _ in 0..<4 {
guard let surface = IOSurfaceCreate(props as CFDictionary),
let texture = device.makeTexture(descriptor: desc, iosurface: surface, plane: 0)
else {
surfacePool.removeAll()
return
}
if hdr, let name = CGColorSpace(name: CGColorSpace.itur_2100_PQ)?.name {
// Tag the surface BT.2100 PQ so the compositor interprets the half-float
// samples as PQ-encoded HDR (the CALayer-contents analogue of the metal
// layer's colorspace).
IOSurfaceSetValue(surface, "IOSurfaceColorSpace" as CFString, name)
}
surfacePool.append(SurfaceSlot(surface: surface, texture: texture))
}
// The EDR request rides the SURFACE layer too (its contents are what composite); the
// metal layer underneath keeps its own from configureColor as the anchor. Layer flags
// are committed by the next swap's transaction flush.
surfaceLayer.wantsExtendedDynamicRangeContent = hdr
}
/// Pick the slot to render into: never the one just handed to the layer (the compositor may
/// still scan it), prefer surfaces the window server isn't holding (`IOSurfaceIsInUse`), and
/// among those the least recently rendered. Falls back to the LRU busy slot rather than
/// stalling a visible glitch at worst, never a queue-up. RENDER THREAD.
private func takeSurfaceSlot() -> Int? {
guard !surfacePool.isEmpty else { return nil }
var free: Int?
var busy: Int?
for i in surfacePool.indices where i != lastHandedOff {
if !IOSurfaceIsInUse(surfacePool[i].surface) {
if free == nil || surfacePool[i].seq < surfacePool[free!].seq { free = i }
} else {
if busy == nil || surfacePool[i].seq < surfacePool[busy!].seq { busy = i }
}
}
guard let pick = free ?? busy else { return nil }
surfaceSeq += 1
surfacePool[pick].seq = surfaceSeq
return pick
}
#endif
/// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the /// Returns the CVMetalTexture (not just its MTLTexture) so the caller can keep it alive past the
/// draw the MTLTexture is only valid while its CVMetalTexture is retained. /// draw the MTLTexture is only valid while its CVMetalTexture is retained.
private func makeTexture( private func makeTexture(
@@ -142,20 +142,17 @@ enum PresentPriority: Equatable {
final class SessionPresenter { final class SessionPresenter {
/// Present pacing for this session. Stage-3 always means glass gating; under the stage-2 /// Present pacing for this session. Stage-3 always means glass gating; under the stage-2
/// default, macOS PyroWave sessions ALSO get glass gating a kernel-panic mitigation, not a /// default, macOS PyroWave sessions ALSO get glass gating for SMOOTHNESS, not as the panic
/// latency tweak. macOS's DCP panics ("mismatched swapID's" @UnifiedPipeline.cpp, the whole /// fix (that is the windowed transactional present see `setComposited`). PyroWave's wavelet
/// machine dies) when WindowServer's swap submissions race, and the reliable trigger is /// decode is near-instant Metal compute, so a network clump presents within the same
/// out-of-band CAMetalLayer presents (displaySyncEnabled=false mandatory for us, see /// millisecond, and it is the codec that sustains stream rates above the panel's refresh; the
/// MetalVideoPresenter's init) arriving faster than the compositor latches them in a /// glass gate admits one presented-but-undisplayed swap at a time (serialized on the on-glass
/// COMPOSITED (windowed) session. Arrival pacing does exactly that with PyroWave: the wavelet /// callback, 100 ms stale backstop) so those bursts coalesce in the newest-wins ring instead
/// decode is near-instant Metal compute, so a network clump of frames presents within the /// of flooding the queue. (Glass pacing was ALSO the original DCP-panic mitigation attempt
/// same millisecond, and PyroWave is the codec that sustains stream rates above the panel's /// disproven: a fully serialized stream still panicked, which is why the real fix moved to the
/// refresh. The glass gate admits one presented-but-undisplayed swap at a time (serialized on /// present mechanism.) An explicit stage-2 pick (setting/env) still forces arrival pacing
/// the on-glass callback, 100 ms stale backstop), which removes the racing pattern outright; /// that A/B lever must stay honest. VideoToolbox codecs keep arrival pacing: decode latency
/// frames the panel couldn't have shown anyway coalesce in the newest-wins ring. An explicit /// spaces their presents.
/// stage-2 pick (setting/env) still forces arrival pacing that A/B lever must stay honest.
/// VideoToolbox codecs keep arrival pacing: decode latency spaces their presents, and years
/// of stage-2 defaults there predate any panic report.
static func pacing( static func pacing(
for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec for choice: PresenterChoice, explicit: PresenterChoice?, codec: VideoCodec
) -> PresentPacing { ) -> PresentPacing {
@@ -178,10 +175,25 @@ final class SessionPresenter {
/// that doesn't exist after the first Wi-Fi clump. Sub-refresh display latency needs pacing /// that doesn't exist after the first Wi-Fi clump. Sub-refresh display latency needs pacing
/// that can't queue at all that's stage-4 (`PresentPacing.deadline`), not a deeper gate. /// that can't queue at all that's stage-4 (`PresentPacing.deadline`), not a deeper gate.
/// ///
#if os(macOS)
/// Resolve the windowed (composited) present MECHANISM for this session the DCP
/// swapID-panic mitigation picker (see `WindowedPresentMode`). The
/// `PUNKTFUNK_WINDOWED_PRESENT=async|transaction|surface` env lever wins (dev A/B);
/// otherwise the user's safe-present setting: ON/unset `.transaction` (the validated
/// mitigation), OFF `.async` (the fast pre-mitigation path the panic returns on
/// affected high-refresh setups; the Settings caption says so). `.surface` is currently
/// env-only (prototype HDR-composite verification owed). Fullscreen always presents
/// async regardless (`setComposited`). Internal (not private) for unit tests.
static func windowedPresentMode(setting: Bool?, env: String?) -> WindowedPresentMode {
if let env, let mode = WindowedPresentMode(rawValue: env) { return mode }
return (setting ?? true) ? .transaction : .async
}
#endif
/// `PUNKTFUNK_GATE_DEPTH` (13) still overrides on iOS/tvOS so the standing-queue ladder /// `PUNKTFUNK_GATE_DEPTH` (13) still overrides on iOS/tvOS so the standing-queue ladder
/// stays reproducible on-device; macOS is pinned to 1, env ignored glass pacing exists /// stays reproducible on-device; macOS is pinned to 1, env ignored a deeper gate only builds
/// there as the DCP swapID kernel-panic mitigation (see `pacing`), and STRICT present /// a standing queue (see above), and macOS glass pacing exists for PyroWave smoothness
/// serialization is its point. Internal (not private) for unit tests. /// (see `pacing`), where depth 1 is the point. Internal (not private) for unit tests.
static func gateDepth(env: String?) -> Int { static func gateDepth(env: String?) -> Int {
#if os(macOS) #if os(macOS)
return 1 return 1
@@ -196,10 +208,16 @@ final class SessionPresenter {
private var stage2Link: CADisplayLink? private var stage2Link: CADisplayLink?
private var metalLayer: CAMetalLayer? private var metalLayer: CAMetalLayer?
#if os(macOS) #if os(macOS)
/// The windowed-mode PyroWave present target (sibling above `metalLayer`) and the last /// The windowed present MECHANISM this session runs while composited (resolved once per
/// routing pushed to the pipeline see `setComposited`. Main-thread only, like all of this. /// session in `start` the user's safe-present setting + the PUNKTFUNK_WINDOWED_PRESENT
/// dev override) and the routing last pushed to the pipeline see `setComposited` (the DCP
/// swapID-panic mitigation). Main-thread only, like all of this.
private var windowedMode: WindowedPresentMode = .transaction
private var windowedPresentApplied: WindowedPresentMode = .async
/// The windowed `surface` present target (sibling above `metalLayer`, transparent while
/// unused) installed whenever stage-2 runs so a mechanism flip never has to mutate the
/// layer tree mid-session.
private var surfaceLayer: CALayer? private var surfaceLayer: CALayer?
private var surfacePresentsActive = false
#endif #endif
private var connection: PunktfunkConnection? private var connection: PunktfunkConnection?
/// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's /// The decoded frame's REAL pixel dimensions (ground truth, pushed by the view from the pump's
@@ -283,11 +301,17 @@ final class SessionPresenter {
baseLayer.addSublayer(metal) baseLayer.addSublayer(metal)
metalLayer = metal metalLayer = metal
#if os(macOS) #if os(macOS)
// The windowed-PyroWave present target sits ABOVE the metal layer: transparent (nil windowedPresentApplied = .async
// contents) while the metal path presents, covering it while surface presents run. // Resolve THIS session's windowed mechanism once (setting + dev env lever)
// `setComposited` routes between it and fullscreen-async from every layout.
windowedMode = Self.windowedPresentMode(
setting: UserDefaults.standard.object(
forKey: DefaultsKey.windowedSafePresent) as? Bool,
env: ProcessInfo.processInfo.environment["PUNKTFUNK_WINDOWED_PRESENT"])
// The surface present target sits ABOVE the metal layer: transparent (nil contents)
// unless the surface mechanism actually presents, covering it while it does.
baseLayer.addSublayer(pipeline.surfaceLayer) baseLayer.addSublayer(pipeline.surfaceLayer)
surfaceLayer = pipeline.surfaceLayer surfaceLayer = pipeline.surfaceLayer
surfacePresentsActive = false
#endif #endif
stage2 = pipeline stage2 = pipeline
// The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger // The link is the vsync CLOCK + putBack-retry nudge, not the presentation trigger
@@ -432,19 +456,23 @@ final class SessionPresenter {
#if os(macOS) #if os(macOS)
/// Route presents for the window's composited state (MAIN thread the view pushes it on /// Route presents for the window's composited state (MAIN thread the view pushes it on
/// every layout, which fullscreen transitions always trigger). PyroWave sessions in a /// every layout, which fullscreen transitions always trigger). A COMPOSITED (windowed)
/// COMPOSITED (windowed) session present via `surfaceLayer` contents instead of the /// session presents through this session's resolved mitigation mechanism (`windowedMode`
/// CAMetalLayer image queue the DCP "mismatched swapID's" kernel-panic mitigation (see /// transactional by default, see `windowedPresentMode`) instead of the async image queue
/// `MetalVideoPresenter.surfaceLayer`; the metal-swap race survives glass pacing, so pacing /// the DCP "mismatched swapID's" kernel-panic mitigation (see `MetalVideoPresenter`; the
/// alone was not enough). VT codecs keep the metal path: no panic reports there, and their /// async-swap race survives glass pacing, so pacing alone was not enough). ALL codecs:
/// HDR/EDR presentation has no surface-contents equivalent wired. /// PyroWave hit it 2026-07-18 and windowed HEVC hit the same 240 Hz Mac Studio 2026-07-21
/// it is the async image queue itself, not any codec or present rate. Fullscreen keeps the
/// async path (direct scanout, lowest latency, no panic there). The full HDR/EDR render
/// path is preserved in every mechanism.
func setComposited(_ composited: Bool) { func setComposited(_ composited: Bool) {
guard let stage2, let connection else { return } guard let stage2 else { return }
let wantsSurface = composited && connection.videoCodec == .pyrowave let mode: WindowedPresentMode = composited ? windowedMode : .async
guard wantsSurface != surfacePresentsActive else { return } guard mode != windowedPresentApplied else { return }
surfacePresentsActive = wantsSurface let wasSurface = windowedPresentApplied == .surface
stage2.setSurfacePresents(wantsSurface) windowedPresentApplied = mode
if !wantsSurface { stage2.setWindowedPresent(mode)
if wasSurface {
// Uncover the metal layer NOW (its last drawable is still attached, so fullscreen // Uncover the metal layer NOW (its last drawable is still attached, so fullscreen
// entry shows the previous frame until the next present no black flash). // entry shows the previous frame until the next present no black flash).
CATransaction.begin() CATransaction.begin()
@@ -471,7 +499,7 @@ final class SessionPresenter {
#if os(macOS) #if os(macOS)
surfaceLayer?.removeFromSuperlayer() surfaceLayer?.removeFromSuperlayer()
surfaceLayer = nil surfaceLayer = nil
surfacePresentsActive = false windowedPresentApplied = .async
#endif #endif
connection = nil connection = nil
} }
@@ -1114,15 +1114,15 @@ public final class Stage2Pipeline {
} }
#if os(macOS) #if os(macOS)
/// The windowed-mode PyroWave present target (see `MetalVideoPresenter.surfaceLayer` the /// Forward the windowed present mechanism (MAIN thread see
/// DCP swapID-panic mitigation). The hosting view installs it as a sibling above `layer`. /// `MetalVideoPresenter.setWindowedPresent`, the DCP swapID-panic mitigation).
public var surfaceLayer: CALayer { presenter.surfaceLayer } func setWindowedPresent(_ mode: WindowedPresentMode) {
presenter.setWindowedPresent(mode)
/// Forward the windowed-vs-fullscreen present routing (MAIN thread see
/// `MetalVideoPresenter.setSurfacePresents`).
public func setSurfacePresents(_ on: Bool) {
presenter.setSurfacePresents(on)
} }
/// The windowed `surface` present target the hosting SessionPresenter installs as a sibling
/// ABOVE `layer` (transparent while unused see `MetalVideoPresenter.surfaceLayer`).
var surfaceLayer: CALayer { presenter.surfaceLayer }
#endif #endif
/// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen` /// Forward the display's current EDR headroom to the presenter (MAIN thread a `UIScreen`
@@ -38,10 +38,11 @@ private let streamInputDebug =
/// dragged deltas become the relative motion StreamLayerView forwards), and hide it. /// dragged deltas become the relative motion StreamLayerView forwards), and hide it.
/// hide/unhide and associate are balanced via `captured`. /// hide/unhide and associate are balanced via `captured`.
/// ///
/// In CLIENT-SIDE-CURSOR mode (gamescope, whose capture carries no host cursor) this is a /// In the DESKTOP mouse model (absolute pointer, remote-desktop-sweep M1) this is a no-op:
/// no-op: the local cursor stays visible and free, and StreamLayerView forwards ABSOLUTE /// the pointer stays free (entering and leaving the stream at will) and StreamLayerView
/// positions instead the visible system cursor IS the on-screen cursor. `disassociate` /// forwards ABSOLUTE positions instead; the local cursor is hidden only while over the view
/// selects between the two; `release()` only undoes what `capture` actually did. /// (cursor rects). `disassociate` selects between the two; `release()` only undoes what
/// `capture` actually did.
private final class CursorCapture { private final class CursorCapture {
private var captured = false private var captured = false
/// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode), /// Whether the engaged capture actually disassociated+hid (false in cursor-visible mode),
@@ -207,14 +208,31 @@ public final class StreamLayerView: NSView {
/// forwarded). Main-thread only. /// forwarded). Main-thread only.
public private(set) var captured = false public private(set) var captured = false
/// Client-side-cursor mode: when true the local system cursor stays VISIBLE over the /// Desktop (absolute) mouse model remote-desktop-sweep M1: when true the pointer is
/// stream and the mouse monitor forwards ABSOLUTE positions (the visible cursor is the /// never disassociated (it enters and leaves the stream freely) and the mouse monitor
/// on-screen cursor gamescope draws none, so no double cursor); when false the existing /// forwards ABSOLUTE positions through the letterbox; the local cursor is hidden only
/// captured/disassociated relative path runs unchanged. Initialized at session start from /// while over this view (cursor rects the host's composited cursor, tracking our
/// the `cursorMode` setting + the host's resolved compositor, toggled live by C. A live /// sends, is the one you see) and reappears the moment it leaves. When false the
/// flip re-engages capture in the new mode so disassociation + the abs/rel choice swap /// captured/disassociated relative path runs unchanged. Initialized at session start
/// atomically. Main-thread only. /// from the `mouseMode` setting gated by the host's resolved compositor (gamescope's
private var cursorVisible = false /// EIS is relative-only absolute sends would be dropped, so it pins to capture);
/// flipped live by M. A live flip re-engages capture in the new model so
/// disassociation + the abs/rel choice swap atomically. Main-thread only.
private var desktopMouse = false
/// Cursor channel (M2): the host forwards shape/state and WE draw the pointer. Active
/// when the Welcome carried `HOST_CAP_CURSOR` (only sessions that advertised the client
/// cap get it). Shapes cache by serial; state is latest-wins. Main-thread only.
private var cursorChannelActive = false
private var hostCursors: [UInt32: NSCursor] = [:]
private var cursorState: PunktfunkConnection.CursorStateEvent?
/// Last `CursorRenderMode.clientDraws` told to the host (the §8 mid-stream render flip);
/// nil = nothing sent yet. Edge-detected by [`reconcileCursorRender`] from the live mouse
/// model, so the chord, engage/release, and session start all reconcile through one path.
private var sentClientDraws: Bool?
/// M3 hint tracking: edge-triggered so a manual M isn't fought the override latch
/// holds until the HOST's intent next changes.
private var lastHint: Bool?
private var hintOverride = false
/// One-shot auto-engage request (stream start, trust confirmed) attempted as soon /// One-shot auto-engage request (stream start, trust confirmed) attempted as soon
/// as the view is in a window with real bounds, then dropped, so it can never fire /// as the view is in a window with real bounds, then dropped, so it can never fire
/// surprisingly later (e.g. on a resize). /// surprisingly later (e.g. on a resize).
@@ -440,9 +458,9 @@ public final class StreamLayerView: NSView {
// If the cursor grab is refused (e.g. the reactivating click arrives before the app is // If the cursor grab is refused (e.g. the reactivating click arrives before the app is
// frontmost), stay released so the NEXT click retries never latch captured=true over // frontmost), stay released so the NEXT click retries never latch captured=true over
// a free cursor, which would make mouseDown's `!captured` guard reject every later click. // a free cursor, which would make mouseDown's `!captured` guard reject every later click.
// In client-side-cursor mode there is no grab (the cursor stays visible) capture // In the desktop mouse model there is no grab (the pointer stays free) capture
// always engages and the monitor forwards absolute positions instead. // always engages and the monitor forwards absolute positions instead.
guard cursorCapture.capture(in: self, disassociate: !cursorVisible) else { return } guard cursorCapture.capture(in: self, disassociate: !desktopMouse) else { return }
inputCapture?.setForwarding(true, suppressClick: fromClick) inputCapture?.setForwarding(true, suppressClick: fromClick)
// Install AFTER the warp + setForwarding: the engage warp generates no forwarded // Install AFTER the warp + setForwarding: the engage warp generates no forwarded
// delta (the monitor isn't up yet), and the engage click's suppression latch is // delta (the monitor isn't up yet), and the engage click's suppression latch is
@@ -450,7 +468,9 @@ public final class StreamLayerView: NSView {
installMouseMonitor() installMouseMonitor()
captured = true captured = true
window?.makeFirstResponder(self) window?.makeFirstResponder(self)
window?.invalidateCursorRects(for: self) // desktop model: hide-over-view engages
notifyCaptureChange(true) notifyCaptureChange(true)
reconcileCursorRender()
} }
private func releaseCapture() { private func releaseCapture() {
@@ -459,7 +479,162 @@ public final class StreamLayerView: NSView {
cursorCapture.release() cursorCapture.release()
inputCapture?.setForwarding(false) inputCapture?.setForwarding(false)
captured = false captured = false
window?.invalidateCursorRects(for: self)
notifyCaptureChange(false) notifyCaptureChange(false)
reconcileCursorRender() // released the host composites the pointer again
}
/// A fully transparent cursor for the desktop mouse model's hide-over-view rect
/// an empty 1×1 image draws nothing.
private static let invisibleCursor = NSCursor(
image: NSImage(size: NSSize(width: 1, height: 1)), hotSpot: .zero)
/// Desktop mouse model: the local cursor is hidden while over the stream (the host's
/// composited cursor, tracking our absolute sends, is the one you see) and reappears
/// the moment it leaves the view AppKit applies/removes the rect's cursor for us,
/// so there is no hide/unhide balancing to get wrong. Capture model instead hides
/// globally via `CursorCapture` (the pointer can't leave the view there).
override public func resetCursorRects() {
if captured && desktopMouse {
// Cursor channel active: wear the HOST's pointer shape (it is no longer in the
// video); hidden host pointer (or no shape yet) = invisible. Without the channel,
// M1 behavior: invisible local cursor, the composited host cursor is the visible one.
if cursorChannelActive, let st = cursorState, st.visible,
let host = hostCursors[st.serial] {
addCursorRect(bounds, cursor: host)
} else {
addCursorRect(bounds, cursor: Self.invisibleCursor)
}
} else {
super.resetCursorRects()
}
}
/// Tell the host who renders the pointer (the §8 mid-stream render flip): we draw it only
/// while the DESKTOP model is engaged (the local OS cursor wears the host shape); under
/// the capture model and while released the host composites it into the video (full
/// fidelity, the pre-channel look). One edge-detected reconciler, called from every
/// transition (chord, engage/release, session start).
private func reconcileCursorRender() {
guard cursorChannelActive, let connection else { return }
let clientDraws = captured && desktopMouse
guard sentClientDraws != clientDraws else { return }
sentClientDraws = clientDraws
connection.setCursorRender(clientDraws: clientDraws)
}
/// Flip the mouse model with the atomic release/re-engage swap; `reappearAt` (host video
/// px the M3 hand-back position) warps the local pointer so leaving relative lands the
/// cursor exactly where the host last had it.
private func setDesktopMouse(_ on: Bool, reappearAt: (x: Int32, y: Int32)?) {
guard desktopMouse != on else { return }
let wasCaptured = captured
if wasCaptured { releaseCapture() }
desktopMouse = on
if wasCaptured { engageCapture(fromClick: false) }
window?.invalidateCursorRects(for: self)
if on, let p = reappearAt, let sp = cgScreenPoint(forHostX: p.x, p.y) {
CGWarpMouseCursorPosition(sp)
}
reconcileCursorRender()
}
/// The single cursor pull thread (both planes share the connection's cursor lock):
/// latest-wins state at a short timeout + a non-blocking shape poll per iteration.
/// Exits when the connection closes; events hop to main where all cursor state lives.
private func startCursorPump(_ connection: PunktfunkConnection) {
let thread = Thread { [weak self] in
while true {
do {
var newest: PunktfunkConnection.CursorStateEvent?
if let st = try connection.nextCursorState(timeoutMs: 100) {
newest = st
while let more = try connection.nextCursorState(timeoutMs: 0) {
newest = more // drain latest wins
}
}
while let shape = try connection.nextCursorShape(timeoutMs: 0) {
DispatchQueue.main.async { self?.applyCursorShape(shape) }
}
if let st = newest {
DispatchQueue.main.async { self?.applyCursorState(st) }
}
} catch {
return // connection closed the session is over
}
if self == nil { return }
}
}
thread.name = "pf-cursor-pump"
thread.start()
}
private func applyCursorShape(_ ev: PunktfunkConnection.CursorShapeEvent) {
guard let cursor = Self.makeCursor(ev) else {
streamInputLog.warning("cursor shape rejected (\(ev.width)x\(ev.height)) — keeping the previous cursor")
return
}
if hostCursors.count >= 64 { hostCursors.removeAll() } // degenerate host: reset
hostCursors[ev.serial] = cursor
if cursorState?.serial == ev.serial {
window?.invalidateCursorRects(for: self)
}
}
private func applyCursorState(_ ev: PunktfunkConnection.CursorStateEvent) {
let prev = cursorState
cursorState = ev
if prev?.visible != ev.visible || prev?.serial != ev.serial {
window?.invalidateCursorRects(for: self)
}
// M3 host-driven auto-flip is DISABLED: `relative_hint` is derived from host cursor
// VISIBILITY, and Windows hides the pointer for ordinary desktop activity (clicking,
// typing) not just when a game grabs it. Acting on those transients flipped
// desktopcapturedesktop, which warped the cursor to view-centre and flushed held
// buttons (a spurious button-up ~200 ms into every press broke window drags). Until
// the host exposes a real pointer-LOCK signal (ClipCursor/raw-input, not visibility),
// the mouse model is user-driven only (M). The hint still rides the wire, unused.
_ = (lastHint, hintOverride)
}
/// Build an `NSCursor` from a forwarded straight-alpha RGBA shape.
private static func makeCursor(_ ev: PunktfunkConnection.CursorShapeEvent) -> NSCursor? {
let (w, h) = (ev.width, ev.height)
guard w > 0, h > 0, ev.rgba.count >= w * h * 4,
let provider = CGDataProvider(data: ev.rgba as CFData),
let cg = CGImage(
width: w, height: h, bitsPerComponent: 8, bitsPerPixel: 32,
bytesPerRow: w * 4, space: CGColorSpaceCreateDeviceRGB(),
bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue),
provider: provider, decode: nil, shouldInterpolate: false,
intent: .defaultIntent)
else { return nil }
let image = NSImage(cgImage: cg, size: NSSize(width: w, height: h))
return NSCursor(
image: image,
hotSpot: NSPoint(x: min(ev.hotX, w - 1), y: min(ev.hotY, h - 1)))
}
/// Host video px CG GLOBAL screen coordinates (top-left origin, the
/// `CGWarpMouseCursorPosition` convention `CursorCapture` established) through the
/// aspect-fit letterbox the inverse direction of `hostPoint(from:)`.
private func cgScreenPoint(forHostX hx: Int32, _ hy: Int32) -> CGPoint? {
guard let connection, let window else { return nil }
let mode = connection.currentMode()
guard mode.width > 0, mode.height > 0 else { return nil }
let fit = AVMakeRect(
aspectRatio: CGSize(width: Int(mode.width), height: Int(mode.height)),
insideRect: bounds)
guard fit.width > 0, fit.height > 0 else { return nil }
let u = (CGFloat(hx) / CGFloat(mode.width)).clamped(to: 0...1)
let v = (CGFloat(hy) / CGFloat(mode.height)).clamped(to: 0...1)
let videoMinYTop = bounds.height - fit.maxY
let pTop = CGPoint(x: fit.minX + u * fit.width, y: videoMinYTop + v * fit.height)
let inView = CGPoint(x: pTop.x, y: bounds.height - pTop.y)
let inWindow = convert(inView, to: nil)
let onScreen = window.convertPoint(toScreen: inWindow)
let primaryHeight = NSScreen.screens.first?.frame.height ?? 0
return CGPoint(x: onScreen.x, y: primaryHeight - onScreen.y)
} }
/// A single local monitor for motion + buttons, installed only while captured. A local /// A single local monitor for motion + buttons, installed only while captured. A local
@@ -473,12 +648,12 @@ public final class StreamLayerView: NSView {
/// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're /// via IOHID. Events are returned (not swallowed): the cursor is frozen, so they're
/// inert locally. /// inert locally.
/// ///
/// In client-side-cursor mode the cursor is NOT frozen, so bare `.mouseMoved` events are /// In the desktop mouse model the cursor is NOT frozen, so bare `.mouseMoved` events are
/// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and /// only generated while `window.acceptsMouseMovedEvents` is true we enable it here and
/// restore it on removal so absolute hover-motion keeps flowing without a click held. /// restore it on removal so absolute hover-motion keeps flowing without a click held.
private func installMouseMonitor() { private func installMouseMonitor() {
guard mouseEventMonitor == nil else { return } guard mouseEventMonitor == nil else { return }
if cursorVisible { if desktopMouse {
savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents savedAcceptsMouseMoved = window?.acceptsMouseMovedEvents
window?.acceptsMouseMovedEvents = true window?.acceptsMouseMovedEvents = true
} }
@@ -490,8 +665,8 @@ public final class StreamLayerView: NSView {
guard let self, self.captured, let ic = self.inputCapture else { return event } guard let self, self.captured, let ic = self.inputCapture else { return event }
switch event.type { switch event.type {
case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged: case .mouseMoved, .leftMouseDragged, .rightMouseDragged, .otherMouseDragged:
if self.cursorVisible { if self.desktopMouse {
// Client-side cursor: forward the ABSOLUTE position (mapped through the // Desktop mouse model: forward the ABSOLUTE position (mapped through the
// aspect-fit letterbox into host pixels), the same path the iPad pointer // aspect-fit letterbox into host pixels), the same path the iPad pointer
// fallback uses. Events in the letterbox bars are dropped (nil host point). // fallback uses. Events in the letterbox bars are dropped (nil host point).
if let p = self.hostPoint(from: event) { if let p = self.hostPoint(from: event) {
@@ -609,14 +784,25 @@ public final class StreamLayerView: NSView {
// be a cursor trap with dead input. // be a cursor trap with dead input.
self?.releaseCapture() self?.releaseCapture()
} }
// C flips the client-side cursor live. Only the key window's stream owns it (same // M flips the mouse model (capture desktop) live the SDL clients' identical
// guard as the capture toggle). Re-engage capture in the new mode so disassociation // chord. Only the key window's stream owns it (same guard as the capture toggle).
// and the absolute/relative forwarding choice swap atomically releaseCapture restores // Re-engage capture in the new model so disassociation and the absolute/relative
// the old mode's grab (if any), engageCapture installs the new one. // forwarding choice swap atomically releaseCapture restores the old model's grab
// C would flip the client-side cursor live NEUTERED while the feature is disabled // (if any), engageCapture installs the new one. On a gamescope host the chord is a
// (see the cursorVisible resolution below): toggling it on under gamescope's relative-only // no-op: its EIS grants only a relative pointer, so the desktop model's absolute
// input traps the pointer. Restore this body when absolute/synthetic-cursor support lands. // sends would be silently dropped (pointer stuck = "all input dead").
capture.onToggleCursor = {} capture.onToggleMouseMode = { [weak self] in
guard let self, self.window?.isKeyWindow == true,
let conn = self.connection else { return }
guard conn.resolvedCompositor != .gamescope else {
streamInputLog.info("mouse-mode chord ignored: gamescope host is relative-only")
return
}
// A manual flip outranks the standing host hint until the hint next CHANGES.
self.hintOverride = true
self.setDesktopMouse(!self.desktopMouse, reappearAt: nil)
streamInputLog.info("chord: mouse mode \(self.desktopMouse ? "desktop" : "capture", privacy: .public)")
}
// The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by // The cross-client combos (Q/D/S Ctrl+Alt+Shift on the other clients), delivered by
// the monitor only while captured; the same key-window ownership rule as throughout. // the monitor only while captured; the same key-window ownership rule as throughout.
capture.onReleaseCapture = { [weak self] in capture.onReleaseCapture = { [weak self] in
@@ -643,15 +829,26 @@ public final class StreamLayerView: NSView {
capture.start() capture.start()
inputCapture = capture inputCapture = capture
// Client-side cursor is TEMPORARILY DISABLED. It positions the host cursor with ABSOLUTE // Desktop (absolute) mouse model resolved at session start from the mouseMode
// events, but gamescope's input socket (EIS) grants only a relative pointer, so those are // setting, gated by the host's compositor: gamescope's input socket (EIS) grants
// silently dropped the pointer never moves and clicks/scroll land on the stuck position // only a relative pointer, so absolute sends would be silently dropped there
// (looks like "all input dead"). gamescope is exactly the compositor Auto enabled it for. // (pointer stuck = "all input dead") pinned to capture. M flips it live.
// Forced off until per-compositor gating (KWin/GNOME/Sway have absolute) or a synthetic- let mode = MouseInputMode(
// cursor-over-relative path lands; the resolution logic below is kept for that. See the rawValue: UserDefaults.standard.string(forKey: DefaultsKey.mouseMode) ?? ""
// C handler (also neutered) and the cursorMode setting (hidden). ) ?? .capture
cursorVisible = false let absOK = connection.resolvedCompositor != .gamescope
_ = connection.resolvedCompositor // (was: Auto gamescope; kept to document intent) desktopMouse = mode == .desktop && absOK
if mode == .desktop && !absOK {
streamInputLog.info("desktop mouse mode unavailable on a gamescope host (relative-only) — using capture")
}
// Cursor channel (M2): the host stopped compositing the pointer drain its shape/
// state planes and draw the pointer as the real NSCursor (plus the M3 auto-flip).
if connection.hostSupportsCursor {
cursorChannelActive = true
streamInputLog.info("cursor channel negotiated — host cursor renders locally")
startCursorPump(connection)
reconcileCursorRender() // initial render mode (a capture-model start composites)
}
// Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2 // Presenter choice + lifecycle live in SessionPresenter (shared with iOS/tvOS): stage-2
// (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by // (explicit VTDecompressionSession decode + a CAMetalLayer/display-link present) by
@@ -700,9 +897,9 @@ public final class StreamLayerView: NSView {
private func layoutPresenter() { private func layoutPresenter() {
presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1) presenter.layout(in: bounds, contentsScale: window?.backingScaleFactor ?? 1)
// Present routing tracks the window's composited state (fullscreen transitions always // Present routing tracks the window's composited state (fullscreen transitions always
// re-layout, so this stays current): windowed PyroWave presents via surface contents // re-layout, so this stays current): a windowed session presents through a Core Animation
// the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited). A view // transaction the DCP swapID kernel-panic mitigation (see SessionPresenter.setComposited).
// not yet in a window counts as composited (the safe default). // A view not yet in a window counts as composited (the safe default).
presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false)) presenter.setComposited(!(window?.styleMask.contains(.fullScreen) ?? false))
// Feed the follower only once in a window (backing scale is real then) and with real // Feed the follower only once in a window (backing scale is real then) and with real
// bounds a pre-window layout would report point-sized dimensions. // bounds a pre-window layout would report point-sized dimensions.
@@ -741,6 +938,14 @@ public final class StreamLayerView: NSView {
matchFollower = nil matchFollower = nil
lastDecodedContentSize = nil // the next session re-derives it from its first frame lastDecodedContentSize = nil // the next session re-derives it from its first frame
connection = nil connection = nil
// Cursor-channel state is per-session: without this reset a next session against a
// host WITHOUT the cap would wear this session's stale shapes (`cursorChannelActive`
// stayed latched true across sessions).
cursorChannelActive = false
cursorState = nil
hostCursors.removeAll()
sentClientDraws = nil
window?.invalidateCursorRects(for: self)
} }
deinit { deinit {
@@ -70,6 +70,16 @@ public enum DefaultsKey {
/// (lowest latency the default, OFF). Resolved once per session; /// (lowest latency the default, OFF). Resolved once per session;
/// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header. /// PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B. See Stage2Pipeline's header.
public static let vsync = "punktfunk.vsync" public static let vsync = "punktfunk.vsync"
/// macOS: present WINDOWED sessions in lockstep with the system compositor (the DCP
/// "mismatched swapID's" kernel-panic mitigation see SessionPresenter.windowedPresentMode
/// and the MetalVideoPresenter saga notes). ON/unset (the default): windowed presents ride
/// a Core Animation transaction validated panic-free on the 240 Hz repro machine, at a
/// small display-latency cost vs the raw path. OFF: windowed sessions keep the fast async
/// image queue ON AFFECTED SETUPS (high-refresh displays) THAT PATH KERNEL-PANICS THE
/// WHOLE MAC, which is why the default is ON. Fullscreen always presents async (fast path)
/// regardless. Resolved once per session; PUNKTFUNK_WINDOWED_PRESENT=async|transaction|
/// surface overrides it for dev A/B.
public static let windowedSafePresent = "punktfunk.windowedSafePresent"
/// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor, /// Allow variable refresh rate: hand the display link a wide frame-rate RANGE (low floor,
/// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical /// preferred = stream rate) so a ProMotion / adaptive-sync display can vary its physical
/// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off, /// refresh to match the stream. On by default; a no-op on fixed-refresh displays. When off,
@@ -84,8 +94,11 @@ public enum DefaultsKey {
/// stays 4:2:0). Sharper text/UI at the cost of more bandwidth. /// stays 4:2:0). Sharper text/UI at the cost of more bandwidth.
public static let enable444 = "punktfunk.enable444" public static let enable444 = "punktfunk.enable444"
public static let hosts = "punktfunk.hosts" public static let hosts = "punktfunk.hosts"
/// Client-side cursor mode: "auto" (shown only in gamescope sessions), "always", "never". /// Physical-mouse model (macOS): "capture" (pointer lock + relative, the default) or
public static let cursorMode = "punktfunk.cursorMode" /// "desktop" (uncaptured absolute pointer) the cross-client `mouse_mode`. Replaces the
/// never-shipped "punktfunk.cursorMode" (auto/always/never client-side-cursor setting,
/// which was hidden while disabled and had no readers).
public static let mouseMode = "punktfunk.mouseMode"
/// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by /// Invert the scroll-wheel / two-finger-scroll direction sent to the host (both axes). Off by
/// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is /// default: the local (natural-scrolling) sign passes through untouched. When on, the sign is
/// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across /// negated at the single scroll sink (`InputCapture.sendScroll`), so it flips consistently across
@@ -316,6 +316,41 @@ final class PresentPacingTests: XCTestCase {
SessionPresenter.pacing(for: .stage4, explicit: .stage4, codec: .pyrowave), .deadline) SessionPresenter.pacing(for: .stage4, explicit: .stage4, codec: .pyrowave), .deadline)
} }
// MARK: - Windowed present mechanism (the macOS DCP swapID-panic mitigation picker)
#if os(macOS)
/// The safe-present setting: ON/unset the validated transactional mitigation; an explicit
/// OFF the fast async path (the user accepted the affected-setup panic risk). The
/// PUNKTFUNK_WINDOWED_PRESENT env lever overrides both ways, `surface` is env-only (the
/// prototype mechanism), and garbage/empty env values are "unset", not an override.
func testWindowedPresentModeResolution() {
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: nil), .transaction,
"unset defaults to the panic mitigation")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: nil), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: nil), .async,
"an explicit opt-out gets the fast async path")
// The dev env lever wins over the setting, both directions.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "async"), .async)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "transaction"),
.transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: true, env: "surface"), .surface,
"the surface prototype is reachable via env only")
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: "surface"), .surface)
// Garbage/empty env = unset.
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: nil, env: "garbage"), .transaction)
XCTAssertEqual(
SessionPresenter.windowedPresentMode(setting: false, env: ""), .async)
}
#endif
// MARK: - Glass-gate depth // MARK: - Glass-gate depth
/// The in-flight present budget is 1 EVERYWHERE: any deeper gate keeps a standing queue /// The in-flight present budget is 1 EVERYWHERE: any deeper gate keeps a standing queue
+11 -2
View File
@@ -16,6 +16,8 @@
# PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games) # PF_LAUNCH library id to launch on connect (optional, e.g. steam:570 — pinned games)
# PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect) # PF_BROWSE non-empty = open the gamepad library (optional; --browse instead of --connect)
# PF_MGMT management-API port for --browse (optional; client defaults to 47990) # PF_MGMT management-API port for --browse (optional; client defaults to 47990)
# PF_CONNECT_TIMEOUT connect budget in seconds (optional; the plugin stretches it after
# firing Wake-on-LAN so the connect survives the host's resume)
# PF_APPID flatpak app id (default io.unom.Punktfunk) # PF_APPID flatpak app id (default io.unom.Punktfunk)
# PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH) # PF_FLATPAK override the flatpak binary path (default: `flatpak` on PATH)
# #
@@ -61,10 +63,17 @@ if [ -z "${PF_HOST:-}" ]; then
echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2 echo "punktfunkrun: PF_HOST is not set (the plugin sets it as a launch option)" >&2
exit 2 exit 2
fi fi
# Trailing args shared by both streaming execs. A stretched connect budget rides along when the
# plugin set one (it just fired Wake-on-LAN, so the host may still be resuming); an older flatpak
# without --connect-timeout ignores the flag harmlessly (hand-scanned argv).
set -- --fullscreen
if [ -n "${PF_CONNECT_TIMEOUT:-}" ]; then
set -- --connect-timeout "$PF_CONNECT_TIMEOUT" "$@"
fi
if [ -n "${PF_LAUNCH:-}" ]; then if [ -n "${PF_LAUNCH:-}" ]; then
# A pinned game: the id rides the session Hello and the host launches that title. # A pinned game: the id rides the session Hello and the host launches that title.
echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2 echo "punktfunkrun: streaming $APPID --connect $PF_HOST --launch $PF_LAUNCH" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" --fullscreen exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --launch "$PF_LAUNCH" "$@"
fi fi
echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2 echo "punktfunkrun: streaming $APPID --connect $PF_HOST" >&2
exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" --fullscreen exec "$FLATPAK" run --arch=x86_64 "$APPID" --connect "$PF_HOST" "$@"
+33 -3
View File
@@ -10,9 +10,17 @@ import {
showModal, showModal,
staticClasses, staticClasses,
} from "@decky/ui"; } from "@decky/ui";
import { definePlugin, routerHook } from "@decky/api"; import { definePlugin, routerHook, toaster } from "@decky/api";
import { FC } from "react"; import { FC } from "react";
import { FaDownload, FaLock, FaLockOpen, FaPlay, FaSyncAlt, FaTv } from "react-icons/fa"; import {
FaDownload,
FaLock,
FaLockOpen,
FaPlay,
FaPlus,
FaSyncAlt,
FaTv,
} from "react-icons/fa";
import { PluginErrorBoundary } from "./boundary"; import { PluginErrorBoundary } from "./boundary";
import { import {
applyUpdate, applyUpdate,
@@ -31,7 +39,19 @@ import {
import { streamPin } from "./library"; import { streamPin } from "./library";
import { PunktfunkRoute, ROUTE } from "./page"; import { PunktfunkRoute, ROUTE } from "./page";
import { PairModal } from "./pair"; import { PairModal } from "./pair";
import { ensureGamepadUiShortcut } from "./steam"; import { ensureGamepadUiShortcut, recreateShortcuts } from "./steam";
// Recovery action for "the Punktfunk library entry vanished" — recreates the visible shortcut.
// Deleting the shortcut (optionally + reinstalling the plugin) leaves a stale appId in Steam's
// CEF localStorage that self-heal fixes on the next mount, but this gives an in-session button
// that works even without a reload. Always ends in a toast so the tap has feedback.
async function recreatePunktfunkShortcut(): Promise<void> {
const appId = await recreateShortcuts();
toaster.toast({
title: "Punktfunk",
body: appId != null ? "Shortcut restored to your library" : "Couldn't create the shortcut",
});
}
// ---------------------------------------------------------------------------------------- // ----------------------------------------------------------------------------------------
// QAM panel — quick status + entry into the full page + one-tap stream for known hosts // QAM panel — quick status + entry into the full page + one-tap stream for known hosts
@@ -190,6 +210,16 @@ const QamPanel: FC = () => {
{checking ? "Checking…" : "Check for updates"} {checking ? "Checking…" : "Check for updates"}
</ButtonItem> </ButtonItem>
</PanelSectionRow> </PanelSectionRow>
<PanelSectionRow>
<ButtonItem
layout="below"
description="Missing the Punktfunk entry in your library? This puts it back."
onClick={() => void recreatePunktfunkShortcut()}
>
<FaPlus style={{ marginRight: "0.5em" }} />
Recreate library shortcut
</ButtonItem>
</PanelSectionRow>
</PanelSection> </PanelSection>
</> </>
); );
+87 -10
View File
@@ -55,6 +55,29 @@ declare const collectionStore:
| { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void } | { SetAppsAsHidden?: (appIds: number[], hidden: boolean) => void }
| undefined; | undefined;
// SteamUI's appStore indexes every registered app/shortcut by appId; a remembered appId whose
// overview is gone was deleted out from under us (the user removed the library entry). We must
// verify this because the remembered appId lives in Steam's CEF localStorage — which survives a
// plugin UNINSTALL/REINSTALL — so a manually-deleted shortcut otherwise leaves a dangling appId
// that the reuse path below silently repoints (SetShortcut* on a dead id is a no-op), and the
// entry never comes back.
declare const appStore:
| { GetAppOverviewByAppID?: (appId: number) => unknown | null }
| undefined;
/** True if a remembered appId still maps to a live Steam shortcut. When appStore is unavailable
* we can't tell, so assume it exists better to keep reusing than risk a duplicate library
* entry from a false "missing". A confident null means the shortcut was deleted recreate. */
function shortcutStillExists(appId: number): boolean {
try {
const get = appStore?.GetAppOverviewByAppID;
if (!get) return true; // no way to verify — preserve the reuse path
return get(appId) != null;
} catch {
return true;
}
}
/** Set a shortcut's library visibility (best-effort, deferred the overview registers a moment /** Set a shortcut's library visibility (best-effort, deferred the overview registers a moment
* after AddShortcut). Hides the stateful stream shortcut; keeps the gamepad-UI one visible. */ * after AddShortcut). Hides the stateful stream shortcut; keeps the gamepad-UI one visible. */
function setShortcutHidden(appId: number, hidden: boolean): void { function setShortcutHidden(appId: number, hidden: boolean): void {
@@ -70,7 +93,9 @@ function setShortcutHidden(appId: number, hidden: boolean): void {
}; };
// Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId). // Bump when the shipped artwork changes so existing shortcuts re-apply it once (per appId).
const ART_VERSION = 2; // v3: CI zips through 0.17.1 shipped no assets/ at all, yet v2 was still recorded as applied
// on those installs — the bump makes them re-apply once on the first build that has the files.
const ART_VERSION = 3;
function artKey(appId: number): string { function artKey(appId: number): string {
return `punktfunk:shortcutArt:${appId}`; return `punktfunk:shortcutArt:${appId}`;
} }
@@ -79,7 +104,7 @@ function artKey(appId: number): string {
* Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per * Apply the plugin's grid/hero/logo/icon to a shortcut (idempotent, once per ART_VERSION per
* appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call. * appId). Cosmetic and fully best-effort: any failure is swallowed and retried on the next call.
*/ */
async function applyArtwork(appId: number): Promise<void> { async function applyArtwork(appId: number, isRetry = false): Promise<void> {
try { try {
if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) { if (localStorage.getItem(artKey(appId)) === `${ART_VERSION}`) {
return; return;
@@ -91,16 +116,29 @@ async function applyArtwork(appId: number): Promise<void> {
[art.logo, 2], [art.logo, 2],
[art.gridwide, 3], [art.gridwide, 3],
]; ];
let applied = false;
for (const [data, assetType] of assets) { for (const [data, assetType] of assets) {
if (data) { if (data) {
await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType); await SteamClient.Apps.SetCustomArtworkForApp(appId, data, "png", assetType);
applied = true;
} }
} }
if (art.icon_path) { if (art.icon_path) {
SteamClient.Apps.SetShortcutIcon(appId, art.icon_path); SteamClient.Apps.SetShortcutIcon(appId, art.icon_path);
applied = true;
}
// Only record "done" when something actually landed — a plugin build whose assets/ is
// missing/empty must keep retrying on later mounts instead of poisoning the marker.
if (applied) {
localStorage.setItem(artKey(appId), `${ART_VERSION}`);
} }
localStorage.setItem(artKey(appId), `${ART_VERSION}`);
} catch (e) { } catch (e) {
// A shortcut fresh out of AddShortcut may not be registered yet (the same race
// setShortcutHidden defers around) — one deferred second attempt, then leave it to
// the next mount.
if (!isRetry) {
setTimeout(() => void applyArtwork(appId, true), 2500);
}
console.warn("punktfunk: shortcut artwork not applied", e); console.warn("punktfunk: shortcut artwork not applied", e);
} }
} }
@@ -157,7 +195,9 @@ async function ensureControllerConfig(): Promise<void> {
return; return;
} }
const r = await applyControllerConfig(SHORTCUT_NAME); const r = await applyControllerConfig(SHORTCUT_NAME);
if (r?.ok) { // `ok` alone isn't done: with zero account configset dirs (fresh Steam) the backend
// succeeds without pointing any account at the template — keep retrying until one lands.
if (r?.ok && (r.applied ?? []).some((a) => a.startsWith("configset:"))) {
localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`); localStorage.setItem(CONFIG_KEY, `${CONFIG_VERSION}`);
} else { } else {
console.warn("punktfunk: controller config not fully applied", r); console.warn("punktfunk: controller config not fully applied", r);
@@ -182,8 +222,10 @@ async function ensureStreamShortcut(): Promise<{ appId: number; runner: string }
const startDir = info.runner.replace(/\/[^/]*$/, ""); // the plugin's bin/ dir const startDir = info.runner.replace(/\/[^/]*$/, ""); // the plugin's bin/ dir
void ensureControllerConfig(); // fire-and-forget — never blocks the launch void ensureControllerConfig(); // fire-and-forget — never blocks the launch
// Reuse the remembered shortcut only if it still exists — a stale appId (shortcut deleted, key
// outlived it across a reinstall) must fall through to AddShortcut, not be silently repointed.
const remembered = recall(STORAGE_KEY_STREAM); const remembered = recall(STORAGE_KEY_STREAM);
if (remembered != null) { if (remembered != null && shortcutStillExists(remembered)) {
SteamClient.Apps.SetShortcutExe(remembered, SHELL); SteamClient.Apps.SetShortcutExe(remembered, SHELL);
SteamClient.Apps.SetShortcutStartDir(remembered, startDir); SteamClient.Apps.SetShortcutStartDir(remembered, startDir);
SteamClient.Apps.SetShortcutName(remembered, SHORTCUT_NAME); SteamClient.Apps.SetShortcutName(remembered, SHORTCUT_NAME);
@@ -219,8 +261,11 @@ export async function ensureGamepadUiShortcut(): Promise<number | null> {
// home). %command% expands to the shortcut exe (/bin/sh); the wrapper rides behind as an arg. // home). %command% expands to the shortcut exe (/bin/sh); the wrapper rides behind as an arg.
const launchOpts = `PF_BROWSE=1 %command% "${info.runner}"`; const launchOpts = `PF_BROWSE=1 %command% "${info.runner}"`;
// Reuse the remembered entry only if it still exists; a stale appId (deleted shortcut whose
// localStorage key survived a plugin reinstall) falls through to AddShortcut so the visible
// library entry actually comes back instead of repointing a dead id.
let appId = recall(STORAGE_KEY_UI); let appId = recall(STORAGE_KEY_UI);
if (appId != null) { if (appId != null && shortcutStillExists(appId)) {
SteamClient.Apps.SetShortcutExe(appId, SHELL); SteamClient.Apps.SetShortcutExe(appId, SHELL);
SteamClient.Apps.SetShortcutStartDir(appId, startDir); SteamClient.Apps.SetShortcutStartDir(appId, startDir);
SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME); SteamClient.Apps.SetShortcutName(appId, SHORTCUT_NAME);
@@ -239,6 +284,30 @@ export async function ensureGamepadUiShortcut(): Promise<number | null> {
} }
} }
/**
* Force the visible "Punktfunk" library entry back into existence the recovery button for
* "my shortcut disappeared". Drops any remembered appId that no longer maps to a live shortcut
* (so it can't shadow a fresh AddShortcut), then re-ensures. Safe to press anytime: a shortcut
* that still exists is left in place (no duplicate); a missing one is recreated. Covers the case
* self-heal-on-mount can't deleting the shortcut WITHOUT reinstalling (no mount no ensure).
* Returns the (new or existing) visible appId, or null on failure.
*/
export async function recreateShortcuts(): Promise<number | null> {
for (const key of [STORAGE_KEY_STREAM, STORAGE_KEY_UI]) {
const id = recall(key);
if (id != null && !shortcutStillExists(id)) {
try {
localStorage.removeItem(artKey(id)); // stale art marker for the dead appId
localStorage.removeItem(key);
} catch {
/* ignore */
}
}
}
// Recreate the visible entry now; the hidden stream shortcut re-registers lazily on next launch.
return ensureGamepadUiShortcut();
}
/** Launch the stateless gamepad-UI shortcut (console home) from the plugin, e.g. a QAM button. */ /** Launch the stateless gamepad-UI shortcut (console home) from the plugin, e.g. a QAM button. */
export async function launchGamepadUi(): Promise<void> { export async function launchGamepadUi(): Promise<void> {
const appId = await ensureGamepadUiShortcut(); const appId = await ensureGamepadUiShortcut();
@@ -283,13 +352,21 @@ export async function launchStream(
opts: LaunchOpts = {}, opts: LaunchOpts = {},
): Promise<void> { ): Promise<void> {
// Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now // Wake-on-LAN: if this host is asleep, nudge it awake before the stream connects. Kicked off now
// so it races with the shortcut setup (near-zero added latency), and awaited just before RunGame. // so it races with the shortcut setup (near-zero added latency); its outcome is needed below
// (the connect budget), and RunGame follows the await either way, so nothing is slower for it.
// Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is // Best-effort — the flatpak client's --wake looks up the host's learned MAC (a no-op if none is
// known), and the connect that follows has its own retry window, so a failure never blocks launch. // known), and the connect that follows has its own retry window, so a failure never blocks launch.
const waking = wake(host, port).catch(() => ({ ok: false })); const waking = wake(host, port).catch(() => ({ ok: false }));
const { appId, runner } = await ensureStreamShortcut(); const [{ appId, runner }, woke] = await Promise.all([ensureStreamShortcut(), waking]);
const target = port && port !== 9777 ? `${host}:${port}` : host; const target = port && port !== 9777 ? `${host}:${port}` : host;
const env = [`PF_HOST=${target}`]; const env = [`PF_HOST=${target}`];
// A magic packet actually went out (a MAC was known), so the host may be mid-resume from
// suspend — that takes far longer than the client's default 15 s connect budget. Stretch the
// budget so the client's wake-tolerant dial keeps retrying across the resume; against an
// already-awake host the connect still lands in under a second, so this costs nothing.
if (woke.ok) {
env.push("PF_CONNECT_TIMEOUT=75");
}
if (opts.browse) { if (opts.browse) {
env.push("PF_BROWSE=1"); env.push("PF_BROWSE=1");
if (opts.mgmt) { if (opts.mgmt) {
@@ -303,9 +380,9 @@ export async function launchStream(
env.push(`PF_LAUNCH=${opts.launchId}`); env.push(`PF_LAUNCH=${opts.launchId}`);
} }
// KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper // KEY=value ... %command% args — %command% expands to the shortcut exe (/bin/sh); the wrapper
// script rides behind it as an argument and reads PF_* from the environment. // script rides behind it as an argument and reads PF_* from the environment. The wake was
// awaited above, so the magic packet is out before the connect attempt.
SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`); SteamClient.Apps.SetAppLaunchOptions(appId, `${env.join(" ")} %command% "${runner}"`);
await waking; // ensure the magic packet is out before the connect attempt
SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100); SteamClient.Apps.RunGame(gameIdFromAppId(appId), "", -1, 100);
} }
+1
View File
@@ -547,6 +547,7 @@ impl AppModel {
crate::video::decodable_codecs(), // codecs (unused by the probe, but honest) crate::video::decodable_codecs(), // codecs (unused by the probe, but honest)
0, // preferred_codec: no preference 0, // preferred_codec: no preference
None, // display_hdr: probe connect, nothing presents None, // display_hdr: probe connect, nothing presents
0, // client_caps: probe connect, nothing renders a cursor
None, // launch: probe connect, no game None, // launch: probe connect, no game
pin, pin,
Some(identity), Some(identity),
+31
View File
@@ -69,6 +69,14 @@ const TOUCH_MODE_CAPTIONS: &[&str] = &[
"The cursor jumps to your finger — a tap clicks there", "The cursor jumps to your finger — a tap clicks there",
"Real multi-touch reaches the host — for touch-native apps", "Real multi-touch reaches the host — for touch-native apps",
]; ];
/// Physical-mouse model values (persisted) + labels + dynamic captions — same idiom as
/// the touch rows. Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[&str] = &["capture", "desktop"];
const MOUSE_MODE_LABELS: &[&str] = &["Capture (games)", "Desktop (absolute)"];
const MOUSE_MODE_CAPTIONS: &[&str] = &[
"Pointer locks to the stream — relative motion, best for games",
"Pointer moves freely in and out — best for remote desktop work",
];
/// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page. /// punktfunk's own license (MIT OR Apache-2.0), shown on the About dialog's Legal page.
const APP_LICENSE: &str = concat!( const APP_LICENSE: &str = concat!(
@@ -542,6 +550,20 @@ pub fn show(
set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]); set_row_subtitle(&w, TOUCH_MODE_CAPTIONS[i]);
}); });
} }
let mouse_row = ChoiceRow::new(
&dialog,
inline,
"Mouse input",
MOUSE_MODE_CAPTIONS[0],
MOUSE_MODE_LABELS,
);
{
let w = mouse_row.widget().clone();
mouse_row.connect_changed(move |i| {
let i = (i as usize).min(MOUSE_MODE_CAPTIONS.len() - 1);
set_row_subtitle(&w, MOUSE_MODE_CAPTIONS[i]);
});
}
let inhibit_row = adw::SwitchRow::builder() let inhibit_row = adw::SwitchRow::builder()
.title("Capture system shortcuts") .title("Capture system shortcuts")
.subtitle("Forward Alt+Tab, Super, … to the host while input is captured") .subtitle("Forward Alt+Tab, Super, … to the host while input is captured")
@@ -718,6 +740,12 @@ pub fn show(
touch_row.set_selected(touch_i as u32); touch_row.set_selected(touch_i as u32);
// set_selected never fires the changed hook, so seed the dynamic caption directly. // set_selected never fires the changed hook, so seed the dynamic caption directly.
set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]); set_row_subtitle(touch_row.widget(), TOUCH_MODE_CAPTIONS[touch_i]);
let mouse_i = MOUSE_MODES
.iter()
.position(|&m| m == s.mouse_mode)
.unwrap_or(0);
mouse_row.set_selected(mouse_i as u32);
set_row_subtitle(mouse_row.widget(), MOUSE_MODE_CAPTIONS[mouse_i]);
let comp_i = COMPOSITORS let comp_i = COMPOSITORS
.iter() .iter()
.position(|&c| c == s.compositor) .position(|&c| c == s.compositor)
@@ -788,6 +816,7 @@ pub fn show(
touch_group.add(touch_row.widget()); touch_group.add(touch_row.widget());
// Group titles are Pango markup — the ampersand must be an entity. // Group titles are Pango markup — the ampersand must be an entity.
let kbm_group = group("Keyboard &amp; mouse", ""); let kbm_group = group("Keyboard &amp; mouse", "");
kbm_group.add(mouse_row.widget());
kbm_group.add(&inhibit_row); kbm_group.add(&inhibit_row);
kbm_group.add(&invert_row); kbm_group.add(&invert_row);
input.add(&touch_group); input.add(&touch_group);
@@ -867,6 +896,8 @@ pub fn show(
} }
s.touch_mode = s.touch_mode =
TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string(); TOUCH_MODES[(touch_row.selected() as usize).min(TOUCH_MODES.len() - 1)].to_string();
s.mouse_mode =
MOUSE_MODES[(mouse_row.selected() as usize).min(MOUSE_MODES.len() - 1)].to_string();
s.forward_pad = chosen_pin.borrow().clone(); s.forward_pad = chosen_pin.borrow().clone();
s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)] s.compositor = COMPOSITORS[(compositor_row.selected() as usize).min(COMPOSITORS.len() - 1)]
.to_string(); .to_string();
+15
View File
@@ -498,6 +498,13 @@ async fn session(args: Args) -> Result<()> {
if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() { if std::env::var_os("PUNKTFUNK_CLIENT_444").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_444; caps |= punktfunk_core::quic::VIDEO_CAP_444;
} }
// PUNKTFUNK_CLIENT_CHACHA20=1 advertises VIDEO_CAP_CHACHA20 — drives the
// host's ChaCha20-Poly1305 session-cipher resolution (the soft-AES armv7
// negotiation, design/chacha20-session-cipher.md §7) without a webOS build;
// the negotiated cipher is reported in the welcome log line below.
if std::env::var_os("PUNKTFUNK_CLIENT_CHACHA20").is_some() {
caps |= punktfunk_core::quic::VIDEO_CAP_CHACHA20;
}
caps caps
}, },
// `--audio-channels` (default stereo); the probe multistream-decodes + validates the // `--audio-channels` (default stereo); the probe multistream-decodes + validates the
@@ -516,6 +523,9 @@ async fn session(args: Args) -> Result<()> {
// writes it into the virtual display's EDID (CTA HDR block), so the EDID-forwarding // writes it into the virtual display's EDID (CTA HDR block), so the EDID-forwarding
// path can be validated headlessly (check the host's monitor caps / ADD log line). // path can be validated headlessly (check the host's monitor caps / ADD log line).
display_hdr: punktfunk_core::client::display_hdr_env_override(), display_hdr: punktfunk_core::client::display_hdr_env_override(),
// No CLIENT_CAP_CURSOR: this headless tool renders nothing — advertising it would
// just strip the pointer from the dumped bitstream.
client_caps: 0,
} }
.encode(), .encode(),
) )
@@ -535,6 +545,11 @@ async fn session(args: Args) -> Result<()> {
chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444, chroma_444 = welcome.chroma_format == punktfunk_core::quic::CHROMA_IDC_444,
chroma_format_idc = welcome.chroma_format, chroma_format_idc = welcome.chroma_format,
codec = codec_ext(welcome.codec), codec = codec_ext(welcome.codec),
cipher = if welcome.cipher == punktfunk_core::quic::CIPHER_CHACHA20_POLY1305 {
"chacha20-poly1305"
} else {
"aes-128-gcm"
},
"session offer" "session offer"
); );
+1
View File
@@ -158,6 +158,7 @@ pub fn run(target: Option<&str>) -> u8 {
v => v, v => v,
}, },
touch_mode: settings_at_start.touch_mode(), touch_mode: settings_at_start.touch_mode(),
mouse_mode: settings_at_start.mouse_mode(),
invert_scroll: settings_at_start.invert_scroll, invert_scroll: settings_at_start.invert_scroll,
json_status, json_status,
on_connected: Some(Box::new(move |fingerprint: [u8; 32]| { on_connected: Some(Box::new(move |fingerprint: [u8; 32]| {
+6
View File
@@ -172,6 +172,11 @@ mod session_main {
// defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session // defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session
// pump) pins one manually. // pump) pins one manually.
display_hdr: None, display_hdr: None,
// The presenter renders the host cursor locally in desktop mouse mode (M2 cursor
// channel); capture-mode sessions keep the composited cursor, so only advertise
// when the session STARTS in desktop mode. The host gates further (Linux portal
// compositors only).
cursor_forward: settings.mouse_mode() == trust::MouseMode::Desktop,
mic_enabled: settings.mic_enabled, mic_enabled: settings.mic_enabled,
clipboard, clipboard,
// The Settings preference (auto → VAAPI where it exists; the presenter // The Settings preference (auto → VAAPI where it exists; the presenter
@@ -429,6 +434,7 @@ mod session_main {
v => v, v => v,
}, },
touch_mode: settings.touch_mode(), touch_mode: settings.touch_mode(),
mouse_mode: settings.mouse_mode(),
invert_scroll: settings.invert_scroll, invert_scroll: settings.invert_scroll,
json_status: true, json_status: true,
on_connected: Some(Box::new(|fingerprint: [u8; 32]| { on_connected: Some(Box::new(|fingerprint: [u8; 32]| {
+18
View File
@@ -90,6 +90,13 @@ const TOUCH_MODES: &[(&str, &str)] = &[
("pointer", "Direct pointer"), ("pointer", "Direct pointer"),
("touch", "Touch passthrough"), ("touch", "Touch passthrough"),
]; ];
/// Physical-mouse presets: `(stored value, display label)` — capture (pointer lock,
/// relative, for games) vs desktop (uncaptured absolute pointer, for remote desktop
/// work). Ctrl+Alt+Shift+M flips the model live in-stream.
const MOUSE_MODES: &[(&str, &str)] = &[
("capture", "Capture (games)"),
("desktop", "Desktop (absolute)"),
];
/// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to /// Host compositor presets: `(stored value, display label)`. Advisory — the host falls back to
/// auto-detect when the choice is unavailable. Only meaningful against a Linux host. /// auto-detect when the choice is unavailable. Only meaningful against a Linux host.
const COMPOSITORS: &[(&str, &str)] = &[ const COMPOSITORS: &[(&str, &str)] = &[
@@ -394,6 +401,10 @@ pub(crate) fn settings_page(
let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| { let touch_combo = setting_combo(ctx, "Touch input", touch_names, touch_i, |s, i| {
s.touch_mode = TOUCH_MODES[i].0.to_string(); s.touch_mode = TOUCH_MODES[i].0.to_string();
}); });
let (mouse_names, mouse_i) = presets(MOUSE_MODES, |v| *v == s.mouse_mode);
let mouse_combo = setting_combo(ctx, "Mouse input", mouse_names, mouse_i, |s, i| {
s.mouse_mode = MOUSE_MODES[i].0.to_string();
});
let invert_scroll_toggle = let invert_scroll_toggle =
setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| { setting_toggle(ctx, "Invert scroll direction", s.invert_scroll, |s, on| {
s.invert_scroll = on s.invert_scroll = on
@@ -542,6 +553,13 @@ pub(crate) fn settings_page(
out.extend(group( out.extend(group(
Some("Keyboard & mouse"), Some("Keyboard & mouse"),
vec![ vec![
described(
mouse_combo,
"Capture locks the pointer to the stream and sends relative motion — \
best for games. Desktop leaves the pointer free to enter and leave \
the stream and sends absolute positions best for remote desktop \
work. Ctrl+Alt+Shift+M switches live.",
),
described( described(
shortcuts_toggle, shortcuts_toggle,
"Alt+Tab, the Windows key and friends reach the host while the stream \ "Alt+Tab, the Windows key and friends reach the host while the stream \
+1
View File
@@ -56,6 +56,7 @@ pub fn run_speed_probe(
decodable_codecs(), decodable_codecs(),
0, // preferred_codec: no preference 0, // preferred_codec: no preference
None, // display_hdr: probe connect, nothing presents None, // display_hdr: probe connect, nothing presents
0, // client_caps: probe connect, nothing renders a cursor
None, // launch: no game None, // launch: no game
pin, pin,
Some(identity), Some(identity),
+8
View File
@@ -30,6 +30,12 @@ pipewire = "0.9"
libc = "0.2" libc = "0.2"
# ashpd 0.13 uses the tokio runtime for the one-time portal handshake (control plane). # ashpd 0.13 uses the tokio runtime for the one-time portal handshake (control plane).
tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] } tokio = { version = "1", features = ["rt", "rt-multi-thread", "net", "time"] }
# XFixes cursor source for gamescope (remote-desktop-sweep Phase C): gamescope paints no
# `SPA_META_Cursor`, so the pointer never reaches the PipeWire node. We read the shape/hotspot/
# visibility from gamescope's nested Xwayland via XFixes instead and feed the existing cursor slot.
# `RustConnection` is the pure-Rust default (no libxcb link → no new C dependency on the host); the
# `xfixes` feature (auto-pulls `render` + `shape`) is what exposes GetCursorImage/SelectCursorInput.
x11rb = { version = "0.13", default-features = false, features = ["xfixes"] }
[target.'cfg(target_os = "windows")'.dependencies] [target.'cfg(target_os = "windows")'.dependencies]
# The host<->driver wire contract for the sealed frame channel (control IOCTL structs + frame header). # The host<->driver wire contract for the sealed frame channel (control IOCTL structs + frame header).
@@ -43,7 +49,9 @@ windows = { version = "0.62", features = [
"Win32_Graphics_Direct3D_Fxc", "Win32_Graphics_Direct3D_Fxc",
"Win32_Graphics_Dxgi", "Win32_Graphics_Dxgi",
"Win32_Graphics_Dxgi_Common", "Win32_Graphics_Dxgi_Common",
"Win32_Graphics_Gdi",
"Win32_System_LibraryLoader", "Win32_System_LibraryLoader",
"Win32_System_StationsAndDesktops",
"Win32_System_Memory", "Win32_System_Memory",
"Win32_System_Threading", "Win32_System_Threading",
"Win32_UI_HiDpi", "Win32_UI_HiDpi",
+58
View File
@@ -24,6 +24,16 @@ use pf_frame::DmabufFrame;
pub trait Capturer: Send { pub trait Capturer: Send {
fn next_frame(&mut self) -> Result<CapturedFrame>; fn next_frame(&mut self) -> Result<CapturedFrame>;
/// [`next_frame`](Self::next_frame) with a caller-chosen first-frame budget instead of the
/// backend's default. The pipeline retry loop shortens its FIRST attempt's wait: a PipeWire
/// stream connected while gamescope re-inits its headless takeover can negotiate a format,
/// reach `Streaming`, and still never receive a buffer — a fresh connect then delivers within
/// ~0.5 s, so waiting out the full default budget on a doomed stream just delays the retry
/// that fixes it. Backends without an internal wait budget ignore it (the default delegates).
fn next_frame_within(&mut self, _budget: std::time::Duration) -> Result<CapturedFrame> {
self.next_frame()
}
/// Non-blocking: the freshest frame available since the last call, or `None` if none has /// Non-blocking: the freshest frame available since the last call, or `None` if none has
/// arrived (the caller reuses its last frame to hold a steady output rate). The default /// arrived (the caller reuses its last frame to hold a steady output rate). The default
/// just produces a frame each call — fine for instant synthetic sources; the portal /// just produces a frame each call — fine for instant synthetic sources; the portal
@@ -59,6 +69,34 @@ pub trait Capturer: Send {
/// SDR / a backend that doesn't expose it (the default — Linux capture has no HDR path yet). /// SDR / a backend that doesn't expose it (the default — Linux capture has no HDR path yet).
/// The stream loop forwards this to the encoder (in-band SEI) and the client (`0xCE` datagram), /// The stream loop forwards this to the encoder (in-band SEI) and the client (`0xCE` datagram),
/// so the two stay a single source of truth. May change mid-session if the source is regraded. /// so the two stay a single source of truth. May change mid-session if the source is regraded.
/// The capture source's LIVE cursor state, when it arrives out-of-band from the frames
/// (the Windows IddCx hardware-cursor channel). Polled by the encode loop every tick and
/// preferred over `CapturedFrame::cursor` — with a hardware cursor, pointer-only moves
/// produce NO new frame, so the frame-attached overlay would go stale on a static desktop.
/// Default `None`: the Linux portal path attaches its cursor to frames instead.
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
None
}
/// LIVE cursor-render flip for a cursor-forward session (design/remote-desktop-sweep.md §8):
/// `on = true` — the client draws the pointer, keep it OUT of the video; `on = false` —
/// the capture mouse model, the pointer must be IN the video again. The Windows IDD
/// capturer implements the composite side ITSELF (slot-copy + alpha-blended quad from the
/// GDI poller) — a declared IddCx hardware cursor is irrevocable, so DWM can never be
/// handed the job back. Called every encode tick (implementations cache; steady state is
/// one compare). Default no-op: the Linux portal never bakes the pointer into frames —
/// the encode loop blends its overlay instead.
fn set_cursor_forward(&mut self, _on: bool) {}
/// Attach a gamescope cursor source (remote-desktop-sweep Phase C). gamescope paints no
/// `SPA_META_Cursor`, so [`cursor`](Self::cursor)'s slot stays empty — this hands the Linux
/// portal capturer gamescope's nested Xwayland `(DISPLAY, XAUTHORITY)` targets (it may run
/// several — one per `--xwayland-count`) so it reads the pointer shape/position over X11
/// (XFixes + QueryPointer), following whichever display is focused, and publishes it into that
/// same slot. Called once, after the capturer is built, only for gamescope sessions. Default
/// no-op: every non-gamescope capturer already has a cursor source.
fn attach_gamescope_cursor(&mut self, _targets: Vec<(String, Option<String>)>) {}
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> { fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
None None
} }
@@ -249,6 +287,12 @@ pub struct ZeroCopyPolicy {
/// passthrough (like the VAAPI backend) instead of the EGL→CUDA import whose payloads only /// passthrough (like the VAAPI backend) instead of the EGL→CUDA import whose payloads only
/// NVENC can consume. Per-session (the codec is negotiated), unlike `backend_is_vaapi`. /// NVENC can consume. Per-session (the codec is negotiated), unlike `backend_is_vaapi`.
pub pyrowave_session: bool, pub pyrowave_session: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (the Linux raw Vulkan
/// Video backend on an H265/AV1 session — resolved by the host facade via
/// `pf_encode::linux_native_nv12_ok`). Gates whether the negotiation PREFERS gamescope's
/// producer-side NV12 pod: libav VAAPI (H264's backend) would misread the two-plane buffer,
/// so H264/GameStream/PyroWave sessions must never see NV12 frames.
pub native_nv12_session: bool,
/// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc, /// The PyroWave encoder's Vulkan-importable dmabuf modifiers for the capture's packed-RGB fourcc,
/// resolved when the session encodes PyroWave (the passthrough advertises them so Mutter+NVIDIA, /// resolved when the session encodes PyroWave (the passthrough advertises them so Mutter+NVIDIA,
/// which allocates tiled-only, still negotiates zero-copy). Empty otherwise. /// which allocates tiled-only, still negotiates zero-copy). Empty otherwise.
@@ -351,6 +395,16 @@ pub type FrameChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetFrameChannelRequest) -> Result<()> + Send + Sync, dyn Fn(&pf_driver_proto::control::SetFrameChannelRequest) -> Result<()> + Send + Sync,
>; >;
/// Delivery closure for the v5 hardware-cursor channel (`IOCTL_SET_CURSOR_CHANNEL`) — same
/// facade contract as [`FrameChannelSender`]. `Some` also OPTS THE SESSION IN: the capturer
/// creates + delivers the cursor section only when the host hands it a sender (the negotiated
/// cursor-forward sessions), and the driver only declares the hardware cursor once that
/// delivery lands — so a plain session keeps DWM's composited pointer untouched.
#[cfg(target_os = "windows")]
pub type CursorChannelSender = std::sync::Arc<
dyn Fn(&pf_driver_proto::control::SetCursorChannelRequest) -> Result<()> + Send + Sync,
>;
// One-time PipeWire library init, shared by the video (portal) and audio capture threads. // One-time PipeWire library init, shared by the video (portal) and audio capture threads.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub mod pwinit; pub mod pwinit;
@@ -408,6 +462,7 @@ pub fn open_virtual_output(
allow_zerocopy: bool, allow_zerocopy: bool,
want_444: bool, want_444: bool,
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
expect_exact_dims: bool,
) -> Result<Box<dyn Capturer>> { ) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::from_virtual_output( linux::PortalCapturer::from_virtual_output(
remote_fd, remote_fd,
@@ -417,6 +472,7 @@ pub fn open_virtual_output(
allow_zerocopy, allow_zerocopy,
want_444, want_444,
policy, policy,
expect_exact_dims,
) )
.map(|c| Box::new(c) as Box<dyn Capturer>) .map(|c| Box::new(c) as Box<dyn Capturer>)
} }
@@ -434,6 +490,7 @@ pub fn open_idd_push(
pyrowave: bool, pyrowave: bool,
keepalive: Box<dyn Send>, keepalive: Box<dyn Send>,
sender: FrameChannelSender, sender: FrameChannelSender,
cursor_sender: Option<CursorChannelSender>,
) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> { ) -> std::result::Result<Box<dyn Capturer>, (anyhow::Error, Box<dyn Send>)> {
idd_push::IddPushCapturer::open( idd_push::IddPushCapturer::open(
target, target,
@@ -443,6 +500,7 @@ pub fn open_idd_push(
pyrowave, pyrowave,
keepalive, keepalive,
sender, sender,
cursor_sender,
) )
.map(|c| Box::new(c) as Box<dyn Capturer>) .map(|c| Box::new(c) as Box<dyn Capturer>)
} }
+349 -62
View File
@@ -22,6 +22,10 @@
use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy}; use super::{CapturedFrame, Capturer, DmabufFrame, FramePayload, PixelFormat, ZeroCopyPolicy};
use anyhow::{anyhow, Context, Result}; use anyhow::{anyhow, Context, Result};
// gamescope cursor source (remote-desktop-sweep Phase C) — feeds `cursor_live` from XFixes when
// the PipeWire node carries no `SPA_META_Cursor` (gamescope's does not).
mod xfixes_cursor;
use std::os::fd::OwnedFd; use std::os::fd::OwnedFd;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, TryRecvError}; use std::sync::mpsc::{sync_channel, Receiver, RecvTimeoutError, TryRecvError};
@@ -57,6 +61,12 @@ pub struct PortalCapturer {
/// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream /// renegotiation before declaring the source lost. Cleared whenever a frame arrives or the stream
/// is `Streaming`. /// is `Streaming`.
stall_since: Option<std::time::Instant>, stall_since: Option<std::time::Instant>,
/// The LIVE cursor overlay, published by the PipeWire thread from every buffer's
/// `SPA_META_Cursor` — including the cursor-only "corrupted" buffers that never become
/// frames. [`Capturer::cursor`] serves it so the encode loop's forwarder tracks pointer-only
/// motion on a static desktop; the frame-attached overlay alone goes stale between damage
/// frames (the same gap the Windows IddCx channel fills, and why the tick prefers LIVE).
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
/// True when this capture runs the VAAPI dmabuf passthrough (a LINEAR-dmabuf-only offer). If /// True when this capture runs the VAAPI dmabuf passthrough (a LINEAR-dmabuf-only offer). If
/// that offer never negotiates, [`next_frame`](Capturer::next_frame)'s timeout branch latches /// that offer never negotiates, [`next_frame`](Capturer::next_frame)'s timeout branch latches
/// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline /// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
@@ -82,6 +92,12 @@ pub struct PortalCapturer {
/// is, releasing the compositor-side output via the keepalive's own `Drop`. `None` for the /// is, releasing the compositor-side output via the keepalive's own `Drop`. `None` for the
/// portal source (its session ends with the portal thread's zbus connection). /// portal source (its session ends with the portal thread's zbus connection).
_keepalive: Option<Box<dyn Send>>, _keepalive: Option<Box<dyn Send>>,
/// The gamescope XFixes cursor reader (remote-desktop-sweep Phase C), when this capturer
/// serves a gamescope node. `Some` after
/// [`attach_gamescope_cursor`](Capturer::attach_gamescope_cursor); its `Drop` stops the reader
/// thread, so it lives exactly as long as the capturer. `None` on the portal path (its cursor
/// comes from `SPA_META_Cursor`).
_gs_cursor: Option<xfixes_cursor::XFixesCursorSource>,
} }
impl PortalCapturer { impl PortalCapturer {
@@ -114,10 +130,17 @@ impl PortalCapturer {
"ScreenCast portal session started; connecting PipeWire" "ScreenCast portal session started; connecting PipeWire"
); );
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before. // This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
Ok( Ok(spawn_pipewire(
spawn_pipewire(Some(fd), node_id, None, true, false, want_hdr, policy)? Some(fd),
.into_capturer(node_id, None), node_id,
) None,
true,
false,
want_hdr,
policy,
false,
)?
.into_capturer(node_id, None))
} }
/// Build a capturer from an already-created virtual output's PipeWire node. The host facade /// Build a capturer from an already-created virtual output's PipeWire node. The host facade
@@ -137,11 +160,13 @@ impl PortalCapturer {
allow_zerocopy: bool, allow_zerocopy: bool,
want_444: bool, want_444: bool,
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
expect_exact_dims: bool,
) -> Result<PortalCapturer> { ) -> Result<PortalCapturer> {
tracing::info!( tracing::info!(
node_id, node_id,
allow_zerocopy, allow_zerocopy,
want_444, want_444,
expect_exact_dims,
"connecting PipeWire to virtual output" "connecting PipeWire to virtual output"
); );
// Virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise 8-bit // Virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise 8-bit
@@ -154,6 +179,7 @@ impl PortalCapturer {
want_444, want_444,
false, false,
policy, policy,
expect_exact_dims,
)? )?
.into_capturer(node_id, Some(keepalive))) .into_capturer(node_id, Some(keepalive)))
} }
@@ -176,6 +202,8 @@ struct PwHandles {
hdr_offer: bool, hdr_offer: bool,
/// See [`PortalCapturer::hdr_negotiated`]. /// See [`PortalCapturer::hdr_negotiated`].
hdr_negotiated: Arc<AtomicBool>, hdr_negotiated: Arc<AtomicBool>,
/// See [`PortalCapturer::cursor_live`].
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
quit: ::pipewire::channel::Sender<()>, quit: ::pipewire::channel::Sender<()>,
join: thread::JoinHandle<()>, join: thread::JoinHandle<()>,
} }
@@ -195,10 +223,12 @@ impl PwHandles {
vaapi_dmabuf: self.vaapi_dmabuf, vaapi_dmabuf: self.vaapi_dmabuf,
hdr_offer: self.hdr_offer, hdr_offer: self.hdr_offer,
hdr_negotiated: self.hdr_negotiated, hdr_negotiated: self.hdr_negotiated,
cursor_live: self.cursor_live,
node_id, node_id,
quit: Some(self.quit), quit: Some(self.quit),
join: Some(self.join), join: Some(self.join),
_keepalive: keepalive, _keepalive: keepalive,
_gs_cursor: None,
} }
} }
} }
@@ -206,6 +236,7 @@ impl PwHandles {
/// Spawn the PipeWire consumer thread for `node_id` (fd `Some` = portal remote, `None` = /// Spawn the PipeWire consumer thread for `node_id` (fd `Some` = portal remote, `None` =
/// default daemon) and return its [`PwHandles`]. `preferred` seeds the format negotiation's /// default daemon) and return its [`PwHandles`]. `preferred` seeds the format negotiation's
/// default size/framerate — for Mutter virtual monitors this is what actually sizes the monitor. /// default size/framerate — for Mutter virtual monitors this is what actually sizes the monitor.
#[allow(clippy::too_many_arguments)]
fn spawn_pipewire( fn spawn_pipewire(
fd: Option<OwnedFd>, fd: Option<OwnedFd>,
node_id: u32, node_id: u32,
@@ -224,6 +255,12 @@ fn spawn_pipewire(
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way // Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6). // capture→encode edge (plan §W6).
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
// The producer's FIRST negotiation is for a sacrificial mode and a renegotiation to
// `preferred`'s dims is guaranteed to follow (KWin virtual outputs — see kwin.rs `create`):
// skip whole buffers until the negotiated size matches, so the pipeline never builds against
// the doomed birth mode. `false` everywhere else (Mutter SIZES the monitor from negotiation,
// gamescope fixates its own — gating those would starve legitimate first frames).
expect_exact_dims: bool,
) -> Result<PwHandles> { ) -> Result<PwHandles> {
// Frames flow from the pipewire thread over a small bounded channel. // Frames flow from the pipewire thread over a small bounded channel.
let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8); let (frame_tx, frame_rx) = sync_channel::<CapturedFrame>(8);
@@ -237,6 +274,8 @@ fn spawn_pipewire(
let broken_cb = broken.clone(); let broken_cb = broken.clone();
let hdr_negotiated = Arc::new(AtomicBool::new(false)); let hdr_negotiated = Arc::new(AtomicBool::new(false));
let hdr_negotiated_cb = hdr_negotiated.clone(); let hdr_negotiated_cb = hdr_negotiated.clone();
let cursor_live = Arc::new(std::sync::Mutex::new(None::<pf_frame::CursorOverlay>));
let cursor_live_cb = cursor_live.clone();
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the // pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the // sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope. // inner `mod pipewire` shadows the crate name at this scope.
@@ -272,12 +311,14 @@ fn spawn_pipewire(
streaming_cb, streaming_cb,
broken_cb, broken_cb,
hdr_negotiated_cb, hdr_negotiated_cb,
cursor_live_cb,
zerocopy, zerocopy,
want_444, want_444,
want_hdr, want_hdr,
preferred, preferred,
quit_rx, quit_rx,
policy, policy,
expect_exact_dims,
) { ) {
tracing::error!(error = %format!("{e:#}"), "pipewire capture thread failed"); tracing::error!(error = %format!("{e:#}"), "pipewire capture thread failed");
} }
@@ -292,6 +333,7 @@ fn spawn_pipewire(
vaapi_dmabuf, vaapi_dmabuf,
hdr_offer: want_hdr, hdr_offer: want_hdr,
hdr_negotiated, hdr_negotiated,
cursor_live,
quit: quit_tx, quit: quit_tx,
join, join,
}) })
@@ -299,29 +341,29 @@ fn spawn_pipewire(
impl Capturer for PortalCapturer { impl Capturer for PortalCapturer {
fn next_frame(&mut self) -> Result<CapturedFrame> { fn next_frame(&mut self) -> Result<CapturedFrame> {
// First frame can lag behind format negotiation; later frames arrive at ~fps. Wait in self.frame_within(Duration::from_secs(10))
// short slices so a GPU-import poison (worker death) fails the capture within ~0.5 s }
// instead of sitting out the full first-frame budget.
let deadline = std::time::Instant::now() + Duration::from_secs(10); fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
loop { // The PipeWire thread's live cursor slot (fed by every buffer's meta, frames or not) —
if self.broken.load(Ordering::Relaxed) { // lets the forwarder track pointer-only motion on a static desktop. See `cursor_live`.
return Err(anyhow!( // On a gamescope node the meta never arrives; the XFixes source (attached below) fills
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \ // the same slot instead.
failed repeatedly rebuilding capture", self.cursor_live.lock().ok().and_then(|slot| slot.clone())
self.node_id }
));
} fn attach_gamescope_cursor(&mut self, targets: Vec<(String, Option<String>)>) {
if let Some(f) = self.pending.take() { // gamescope paints no `SPA_META_Cursor`, so `cursor_live` would stay empty. Spawn the
return Ok(f); // a wait_arrival stash outranks the channel (it's older) // XFixes reader to publish gamescope's pointer into that SAME slot — `cursor()` above then
} // serves it and the encode loop composites it, exactly like the portal path. It connects
let slice = Duration::from_millis(500) // to every nested Xwayland and follows the focused one's pointer. A failure (no Xwayland /
.min(deadline.saturating_duration_since(std::time::Instant::now())); // no XFixes) logs and leaves the slot empty = today's cursorless gamescope.
match self.frames.recv_timeout(slice) { self._gs_cursor =
Ok(frame) => return Ok(frame), xfixes_cursor::XFixesCursorSource::spawn(targets, Arc::clone(&self.cursor_live));
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue, }
Err(e) => return self.next_frame_timed_out(e),
} fn next_frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
} self.frame_within(budget)
} }
fn supports_arrival_wait(&self) -> bool { fn supports_arrival_wait(&self) -> bool {
@@ -417,9 +459,41 @@ impl Capturer for PortalCapturer {
} }
impl PortalCapturer { impl PortalCapturer {
/// The [`Capturer::next_frame`] budget expired (or the thread ended) — turn it into the /// The blocking first-frame wait behind [`Capturer::next_frame`] /
/// diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged. /// [`Capturer::next_frame_within`]. First frame can lag behind format negotiation; later
fn next_frame_timed_out(&self, err: RecvTimeoutError) -> Result<CapturedFrame> { /// frames arrive at ~fps. Wait in short slices so a GPU-import poison (worker death) fails
/// the capture within ~0.5 s instead of sitting out the full first-frame budget.
fn frame_within(&mut self, budget: Duration) -> Result<CapturedFrame> {
let deadline = std::time::Instant::now() + budget;
loop {
if self.broken.load(Ordering::Relaxed) {
return Err(anyhow!(
"zero-copy GPU import lost (node {}): the import worker died or tiled imports \
failed repeatedly rebuilding capture",
self.node_id
));
}
if let Some(f) = self.pending.take() {
return Ok(f); // a wait_arrival stash outranks the channel (it's older)
}
let slice = Duration::from_millis(500)
.min(deadline.saturating_duration_since(std::time::Instant::now()));
match self.frames.recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(RecvTimeoutError::Timeout) if std::time::Instant::now() < deadline => continue,
Err(e) => return self.next_frame_timed_out(e, budget),
}
}
}
/// The [`frame_within`](Self::frame_within) budget expired (or the thread ended) — turn it
/// into the diagnosis-bearing error. Split out of the slicing loop above; behavior unchanged.
fn next_frame_timed_out(
&self,
err: RecvTimeoutError,
budget: Duration,
) -> Result<CapturedFrame> {
let within = budget.as_secs_f32();
match err { match err {
RecvTimeoutError::Timeout => { RecvTimeoutError::Timeout => {
// Split the two black-screen root causes apart so the operator gets a cause, not // Split the two black-screen root causes apart so the operator gets a cause, not
@@ -427,9 +501,10 @@ impl PortalCapturer {
// not (no acceptable format / node never emitted a param)? // not (no acceptable format / node never emitted a param)?
if self.negotiated.load(Ordering::Relaxed) { if self.negotiated.load(Ordering::Relaxed) {
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiated but no buffers \ "no PipeWire frame within {within}s (node {}): format negotiated but no \
arrived the compositor produced no frames (virtual output idle/unmapped, \ buffers arrived the compositor produced no frames (virtual output \
or capture never started)", idle/unmapped, capture never started, or a stream bound during a \
compositor (re)start that will never deliver a reconnect fixes that)",
self.node_id self.node_id
)) ))
} else if self.hdr_offer { } else if self.hdr_offer {
@@ -440,10 +515,10 @@ impl PortalCapturer {
// auto-reconnects) negotiates SDR instead of re-running this same timeout. // auto-reconnects) negotiates SDR instead of re-running this same timeout.
super::note_hdr_capture_failed(); super::note_hdr_capture_failed();
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \ "no PipeWire frame within {within}s (node {}): the compositor never \
the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored monitor in \ accepted the HDR (10-bit PQ/BT.2020 dmabuf) offer is the mirrored \
HDR mode on GNOME 50+? Downgrading this host to SDR capture; reconnect \ monitor in HDR mode on GNOME 50+? Downgrading this host to SDR capture; \
to stream SDR", reconnect to stream SDR",
self.node_id self.node_id
)) ))
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() { } else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
@@ -452,14 +527,15 @@ impl PortalCapturer {
// retries on the CPU offer instead of failing this same negotiation forever. // retries on the CPU offer instead of failing this same negotiation forever.
pf_zerocopy::note_vaapi_dmabuf_failed(); pf_zerocopy::note_vaapi_dmabuf_failed();
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \ "no PipeWire frame within {within}s (node {}): the compositor never \
the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this host to the \ accepted the LINEAR-dmabuf offer (VAAPI zero-copy) downgrading this \
CPU capture path; the pipeline rebuild will renegotiate without dmabuf", host to the CPU capture path; the pipeline rebuild will renegotiate \
without dmabuf",
self.node_id self.node_id
)) ))
} else { } else {
Err(anyhow!( Err(anyhow!(
"no PipeWire frame within 10s (node {}): format negotiation never \ "no PipeWire frame within {within}s (node {}): format negotiation never \
completed the compositor offered no format this consumer accepts \ completed the compositor offered no format this consumer accepts \
(pixel-format/modifier mismatch) or the node never emitted a Format param", (pixel-format/modifier mismatch) or the node never emitted a Format param",
self.node_id self.node_id
@@ -824,6 +900,7 @@ mod pipewire {
VideoFormat::RGBA => PixelFormat::Rgba, VideoFormat::RGBA => PixelFormat::Rgba,
VideoFormat::RGB => PixelFormat::Rgb, VideoFormat::RGB => PixelFormat::Rgb,
VideoFormat::BGR => PixelFormat::Bgr, VideoFormat::BGR => PixelFormat::Bgr,
VideoFormat::NV12 => PixelFormat::Nv12,
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by // The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by
// the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020). // the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020).
VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10, VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10,
@@ -851,13 +928,21 @@ mod pipewire {
/// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves, /// Bumps whenever the bitmap (`rgba`/`bw`/`bh`) changes — stable across position-only moves,
/// so the GPU encoder re-uploads its cursor texture only on change. /// so the GPU encoder re-uploads its cursor texture only on change.
serial: u64, serial: u64,
/// The compositor-reported hotspot — carried on the overlay for the cursor-forward
/// channel (the blend path uses the pre-adjusted `x`/`y` and never reads it).
hot_x: i32,
hot_y: i32,
} }
impl CursorState { impl CursorState {
/// A shareable overlay for the GPU encode paths (blended at encode time), or `None` when /// A shareable overlay for the encode/forward paths, or `None` before the first bitmap
/// there is nothing to draw. Cheap: clones an `Arc` + a few scalars. /// arrived. A HIDDEN pointer still yields `Some` (with `visible: false`): the
/// cursor-forward channel needs "known but hidden" — an app grabbed the pointer, the
/// client's relative-mode hint (M3) — which is a different fact from "no cursor yet".
/// The encode loop strips invisible overlays before any blend path sees the frame.
/// Cheap: clones an `Arc` + a few scalars.
fn overlay(&self) -> Option<pf_frame::CursorOverlay> { fn overlay(&self) -> Option<pf_frame::CursorOverlay> {
if !self.visible || self.rgba.is_empty() { if self.rgba.is_empty() {
return None; return None;
} }
Some(pf_frame::CursorOverlay { Some(pf_frame::CursorOverlay {
@@ -867,6 +952,9 @@ mod pipewire {
h: self.bh, h: self.bh,
rgba: self.rgba.clone(), rgba: self.rgba.clone(),
serial: self.serial, serial: self.serial,
hot_x: self.hot_x.max(0) as u32,
hot_y: self.hot_y.max(0) as u32,
visible: self.visible,
}) })
} }
} }
@@ -917,6 +1005,21 @@ mod pipewire {
dbg_log_n: u64, dbg_log_n: u64,
/// Cursor-as-metadata state, composited into the CPU de-pad path (see `consume_frame`). /// Cursor-as-metadata state, composited into the CPU de-pad path (see `consume_frame`).
cursor: CursorState, cursor: CursorState,
/// LIVE overlay slot shared with [`super::PortalCapturer::cursor_live`] — refreshed after
/// every `update_cursor_meta`, including from cursor-only buffers that never become frames.
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
/// `Some((w, h))` while the producer's negotiated size is a sacrificial birth mode and a
/// renegotiation to these dims is guaranteed (KWin virtual outputs — kwin.rs `create`):
/// `.process` skips whole buffers until the negotiated size matches, then clears this
/// (self-disarming — later legitimate resizes are unaffected). `None` = no gating.
expect_dims: Option<(u32, u32)>,
/// Buffers skipped by the `expect_dims` gate (rate-limits its log).
gate_skips: u64,
/// When the gate first held a buffer — after [`GATE_DEADLINE`] with no renegotiation the
/// gate disarms and accepts what the producer serves (degraded dims beat a session wedged
/// into the first-frame-timeout retry loop; the promised renegotiation normally lands
/// within a frame or two).
gate_since: Option<std::time::Instant>,
} }
/// Consecutive tiled-import failures (worker alive, e.g. a per-buffer `EGL_BAD_MATCH`) before /// Consecutive tiled-import failures (worker alive, e.g. a per-buffer `EGL_BAD_MATCH`) before
@@ -989,10 +1092,10 @@ mod pipewire {
.into_inner()) .into_inner())
} }
/// Build a BGRx dmabuf `EnumFormat` pod advertising the EGL-importable `modifiers` as a /// Build a LINEAR/modifier DMA-BUF `EnumFormat` pod. Packed BGRx is the existing import path;
/// mandatory enum Choice; the compositor fixates to one of them that it can allocate, which /// NV12 is gamescope's producer-side RGB→YUV path (opt-in during bring-up).
/// we read back in `param_changed`.
fn build_dmabuf_format( fn build_dmabuf_format(
format: VideoFormat,
modifiers: &[u64], modifiers: &[u64],
preferred: Option<(u32, u32, u32)>, preferred: Option<(u32, u32, u32)>,
) -> Result<Vec<u8>> { ) -> Result<Vec<u8>> {
@@ -1003,7 +1106,7 @@ mod pipewire {
pw::spa::param::ParamType::EnumFormat, pw::spa::param::ParamType::EnumFormat,
pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video), pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video),
pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw), pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw),
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, VideoFormat::BGRx), pw::spa::pod::property!(FormatProperties::VideoFormat, Id, format),
pw::spa::pod::property!( pw::spa::pod::property!(
FormatProperties::VideoSize, FormatProperties::VideoSize,
Choice, Choice,
@@ -1032,6 +1135,22 @@ mod pipewire {
pw::spa::utils::Fraction { num: 240, denom: 1 } pw::spa::utils::Fraction { num: 240, denom: 1 }
), ),
); );
if format == VideoFormat::NV12 {
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorMatrix,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_MATRIX_BT709,
)),
});
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorRange,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_RANGE_16_235,
)),
});
}
obj.properties.push(pw::spa::pod::Property { obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier, key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier,
flags: pw::spa::pod::PropertyFlags::MANDATORY, flags: pw::spa::pod::PropertyFlags::MANDATORY,
@@ -1279,10 +1398,17 @@ mod pipewire {
value: pw::spa::pod::Value::Choice(pw::spa::pod::ChoiceValue::Int( value: pw::spa::pod::Value::Choice(pw::spa::pod::ChoiceValue::Int(
pw::spa::utils::Choice( pw::spa::utils::Choice(
pw::spa::utils::ChoiceFlags::empty(), pw::spa::utils::ChoiceFlags::empty(),
// The max must cover the producer's offer or the Meta param silently
// fails to negotiate and NO buffer ever carries the meta region:
// Mutter offers a FIXED `SPA_POD_Int(CURSOR_META_SIZE(384, 384))`
// (meta-screen-cast-stream-src.c, GNOME 50) — a 256² max made the
// intersection empty, which cost the whole Linux cursor channel
// on-glass. 1024² is headroom, not an allocation: the negotiated
// region follows the producer's value.
pw::spa::utils::ChoiceEnum::Range { pw::spa::utils::ChoiceEnum::Range {
default: meta_size(64, 64), default: meta_size(64, 64),
min: meta_size(1, 1), min: meta_size(1, 1),
max: meta_size(256, 256), max: meta_size(1024, 1024),
}, },
), ),
)), )),
@@ -1340,13 +1466,19 @@ mod pipewire {
) )
}; };
if id == 0 { if id == 0 {
// Compositor reports no visible pointer (e.g. a game grabbed/hid it). // SPA contract: id 0 = "no cursor information", NOT "cursor hidden". Mutter only
cursor.visible = false; // REWRITES a buffer's meta region when the cursor changed, so recycled buffers
// between damage frames carry a stale id-0 meta — treating that as hidden flickered
// the cursor off between hovers (on-glass round 5). Keep the last-known state; a
// pointer that really left/hid simply stops producing updates. (The M3 hidden hint
// loses its Mutter signal — Windows has its own CURSOR_SUPPRESSED source.)
return; return;
} }
cursor.visible = true; cursor.visible = true;
cursor.x = pos_x - hot_x; cursor.x = pos_x - hot_x;
cursor.y = pos_y - hot_y; cursor.y = pos_y - hot_y;
cursor.hot_x = hot_x;
cursor.hot_y = hot_y;
if bmp_off == 0 { if bmp_off == 0 {
// Position-only update — keep the cached bitmap. // Position-only update — keep the cached bitmap.
return; return;
@@ -1372,8 +1504,9 @@ mod pipewire {
(*bmp).offset as usize, (*bmp).offset as usize,
) )
}; };
// Ignore empty or implausibly large bitmaps (we requested <= 256×256). // Ignore empty or implausibly large bitmaps (the meta-size request covers <= 1024×1024;
if bw == 0 || bh == 0 || bw > 256 || bh > 256 { // real cursors are ≤96px — the cursor channel downscales >120px for the wire anyway).
if bw == 0 || bh == 0 || bw > 1024 || bh > 1024 {
return; return;
} }
let row = bw as usize * 4; let row = bw as usize * 4;
@@ -1604,8 +1737,8 @@ mod pipewire {
} }
} }
// VAAPI zero-copy passthrough: hand the raw dmabuf straight to the encoder, which imports // Raw DMA-BUF passthrough: packed RGB is imported for GPU CSC; producer-native NV12 can
// it into a VA surface and does RGB→NV12 on the GPU video engine. No CUDA importer here. // be consumed by the Vulkan Video encoder without another color conversion.
if ud.vaapi_passthrough { if ud.vaapi_passthrough {
if let Some(fmt) = ud.format { if let Some(fmt) = ud.format {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf { if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
@@ -1613,9 +1746,41 @@ mod pipewire {
let chunk = datas[0].chunk(); let chunk = datas[0].chunk();
let offset = chunk.offset(); let offset = chunk.offset();
let stride = chunk.stride().max(0) as u32; let stride = chunk.stride().max(0) as u32;
// Native NV12 usually arrives as a two-plane SPA buffer over ONE buffer
// object; plane 1's chunk carries the REAL UV offset/stride (compositors
// may align the Y plane before UV). Pass it through instead of assuming
// contiguity. Each spa_data holds its own (dup'd) fd, so BO identity is
// by inode, not fd number; a genuinely two-BO frame cannot travel through
// the single-fd import — drop it with a diagnosis instead of streaming
// garbage chroma.
let plane1 =
if fmt == PixelFormat::Nv12 && datas.len() >= 2 && datas[1].fd() > 0 {
// SAFETY: zeroed `libc::stat` is a valid POD initializer; both fds are
// owned by the live PipeWire buffer for this callback, and `fstat`
// only writes the out-param structs, whose fields are read only after
// the `== 0` success checks.
let same_bo = unsafe {
let mut s0: libc::stat = std::mem::zeroed();
let mut s1: libc::stat = std::mem::zeroed();
libc::fstat(datas[0].fd() as i32, &mut s0) == 0
&& libc::fstat(datas[1].fd() as i32, &mut s1) == 0
&& (s0.st_dev, s0.st_ino) == (s1.st_dev, s1.st_ino)
};
if !same_bo {
warn_once(
"NV12 planes live in different buffer objects — frames \
dropped (single-fd import only)",
);
return;
}
let c1 = datas[1].chunk();
Some((c1.offset(), c1.stride().max(0) as u32))
} else {
None
};
// dup the fd so it survives the SPA buffer recycle — the encode thread // dup the fd so it survives the SPA buffer recycle — the encode thread
// imports it. (Content stability across the brief map+CSC window relies on // imports it. Content stability across the brief import/encode window relies
// the compositor's buffer-pool depth, like any zero-copy capture.) // on the compositor's buffer-pool depth, like any zero-copy capture.
// SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid // SAFETY: `datas[0].fd()` is the dmabuf fd owned by the live PipeWire buffer (valid
// for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd, // for this callback). `fcntl(fd, F_DUPFD_CLOEXEC, 0)` reads only the integer fd,
// touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1). // touches no Rust memory, and returns a fresh independent CLOEXEC duplicate (or -1).
@@ -1642,9 +1807,10 @@ mod pipewire {
modifier: ud.modifier, modifier: ud.modifier,
offset, offset,
stride, stride,
plane1,
}), }),
// Cursor-as-metadata: the encoder blends this into its owned VA // Cursor-as-metadata is blended only by RGB→NV12 backends. Gamescope
// surface (raw dmabuf never touched). // embeds its pointer in the produced pixels, so native NV12 has none.
cursor: ud.cursor.overlay(), cursor: ud.cursor.overlay(),
}); });
static ONCE: std::sync::atomic::AtomicBool = static ONCE: std::sync::atomic::AtomicBool =
@@ -1655,7 +1821,12 @@ mod pipewire {
h, h,
modifier = ud.modifier, modifier = ud.modifier,
fourcc = format_args!("{:#010x}", fourcc), fourcc = format_args!("{:#010x}", fourcc),
"zero-copy: handing the raw dmabuf to the encoder (GPU import + CSC)" source = if fmt == PixelFormat::Nv12 {
"producer-native NV12"
} else {
"packed RGB (encoder GPU CSC)"
},
"zero-copy: handing the raw DMA-BUF to the encoder"
); );
} }
return; return;
@@ -1935,6 +2106,9 @@ mod pipewire {
streaming: Arc<AtomicBool>, streaming: Arc<AtomicBool>,
broken: Arc<AtomicBool>, broken: Arc<AtomicBool>,
hdr_negotiated: Arc<AtomicBool>, hdr_negotiated: Arc<AtomicBool>,
// LIVE cursor publisher (see `PortalCapturer::cursor_live`): refreshed from every
// dequeued buffer's cursor meta, frames or not.
cursor_live: Arc<std::sync::Mutex<Option<pf_frame::CursorOverlay>>>,
zerocopy: bool, zerocopy: bool,
// 4:4:4 session: tiled dmabufs take the worker's planar-YUV444 GPU convert. // 4:4:4 session: tiled dmabufs take the worker's planar-YUV444 GPU convert.
want_444: bool, want_444: bool,
@@ -1946,6 +2120,9 @@ mod pipewire {
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way // Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6). // capture→encode edge (plan §W6).
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
// See `spawn_pipewire`: the first negotiation is for a sacrificial mode; hold frames
// until the producer renegotiates to `preferred`'s dims.
expect_exact_dims: bool,
) -> Result<()> { ) -> Result<()> {
crate::pwinit::ensure_init(); crate::pwinit::ensure_init();
@@ -2015,6 +2192,28 @@ mod pipewire {
// PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw // PyroWave session (the wavelet encoder's own Vulkan device, any vendor) → hand the raw
// dmabuf straight to the encoder. // dmabuf straight to the encoder.
let vaapi_passthrough = zerocopy && !force_shm && importer.is_none() && raw_passthrough; let vaapi_passthrough = zerocopy && !force_shm && importer.is_none() && raw_passthrough;
// Producer-side NV12 (default-on; PUNKTFUNK_PIPEWIRE_NV12=0 escapes): gamescope offers a
// one-fd LINEAR NV12 image when the consumer asks — its compositor pass does the RGB→YUV,
// and the Vulkan Video encoder imports the buffer as its encode source directly (no host
// CSC at all). `native_nv12_session` restricts this to sessions whose encoder can ingest
// it (Linux vulkan-encode H265/AV1 — never H264/libav-VAAPI, GameStream-resolve, or
// PyroWave, whose Vulkan compute CSC ingests packed RGB only). Raw passthrough is
// required because the CUDA importer expects packed RGB, and 4:4:4/HDR must not be
// silently subsampled/downconverted. Non-NV12 compositors (KWin/GNOME) simply match the
// packed-RGB fallback pod.
let prefer_native_nv12 = std::env::var("PUNKTFUNK_PIPEWIRE_NV12").as_deref() != Ok("0")
&& policy.native_nv12_session
&& backend_is_vaapi
&& vaapi_passthrough
&& !policy.pyrowave_session
&& !want_444
&& !want_hdr;
if prefer_native_nv12 {
tracing::info!(
"zero-copy: preferring gamescope producer-side NV12 LINEAR DMA-BUF (no host \
RGB CSC; PUNKTFUNK_PIPEWIRE_NV12=0 restores the packed-RGB negotiation)"
);
}
// Modifiers our import stack handles for BGRx: the EGL-importable (tiled) set, plus LINEAR // Modifiers our import stack handles for BGRx: the EGL-importable (tiled) set, plus LINEAR
// (0) — NVIDIA's EGL won't list it, but LINEAR dmabufs (gamescope's only offer) import via // (0) — NVIDIA's EGL won't list it, but LINEAR dmabufs (gamescope's only offer) import via
// CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only: // CUDA external memory instead. For the VAAPI passthrough path we advertise LINEAR only:
@@ -2054,7 +2253,9 @@ mod pipewire {
tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path"); tracing::warn!("zero-copy: no importable dmabuf modifiers — using CPU path");
} else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() { } else if vaapi_passthrough && policy.pyrowave_modifiers.is_empty() {
tracing::info!( tracing::info!(
"zero-copy: advertising LINEAR dmabuf for direct VAAPI import (GPU CSC)" native_nv12_preferred = prefer_native_nv12,
"zero-copy: advertising LINEAR DMA-BUF for encoder import (native NV12 first \
when enabled, packed RGB fallback)"
); );
} else if want_dmabuf && !vaapi_passthrough { } else if want_dmabuf && !vaapi_passthrough {
tracing::info!( tracing::info!(
@@ -2106,6 +2307,14 @@ mod pipewire {
linear_nv12_failed: false, linear_nv12_failed: false,
dbg_log_n: 0, dbg_log_n: 0,
cursor: CursorState::default(), cursor: CursorState::default(),
cursor_live,
expect_dims: if expect_exact_dims {
preferred.map(|(w, h, _)| (w, h))
} else {
None
},
gate_skips: 0,
gate_since: None,
}; };
let stream = pw::stream::StreamBox::new( let stream = pw::stream::StreamBox::new(
@@ -2217,6 +2426,60 @@ mod pipewire {
newest = next; newest = next;
drained += 1; drained += 1;
} }
// Sacrificial-mode gate (kwin.rs `create`): until the producer renegotiates to the
// expected dims, every buffer — frame AND cursor meta, whose positions are in the
// doomed mode's space — belongs to the birth mode; consuming one would build the
// pipeline at the wrong size. Self-disarms on the first matching negotiation, or
// after `GATE_DEADLINE` without one — degraded dims beat wedging the session into
// the first-frame-timeout retry loop when the promised renegotiation never comes.
if let Some((ew, eh)) = ud.expect_dims {
/// The renegotiation normally lands within a frame or two of recording; well
/// past that, the producer is not going to deliver it (the on-glass case: the
/// real mode never actually applied) — stop starving the pipeline.
const GATE_DEADLINE: std::time::Duration = std::time::Duration::from_secs(3);
let sz = ud.info.size();
if sz.width == ew && sz.height == eh {
tracing::info!(
skipped = ud.gate_skips,
width = ew,
height = eh,
"producer renegotiated to the expected mode — frames flow"
);
ud.expect_dims = None;
} else if ud
.gate_since
.get_or_insert_with(std::time::Instant::now)
.elapsed()
> GATE_DEADLINE
{
tracing::warn!(
negotiated_w = sz.width,
negotiated_h = sz.height,
expected_w = ew,
expected_h = eh,
skipped = ud.gate_skips,
"producer never renegotiated to the expected mode — accepting its \
dims (session runs degraded rather than wedged)"
);
ud.expect_dims = None;
} else {
ud.gate_skips += 1;
if ud.gate_skips == 1 || ud.gate_skips.is_power_of_two() {
tracing::info!(
negotiated_w = sz.width,
negotiated_h = sz.height,
expected_w = ew,
expected_h = eh,
n = ud.gate_skips,
"holding frames until the producer renegotiates to the expected mode"
);
}
// SAFETY: `newest` was dequeued from this stream and not yet requeued;
// requeued exactly once here, then never touched (mirrors the null path).
unsafe { stream.queue_raw_buffer(newest) };
return;
}
}
// PipeWire dispatches from a C trampoline with no catch_unwind; a panic crossing that FFI // PipeWire dispatches from a C trampoline with no catch_unwind; a panic crossing that FFI
// boundary would abort the whole host. Contain the inspect/consume work — the only Rust // boundary would abort the whole host. Contain the inspect/consume work — the only Rust
// code here that can panic — and requeue `newest` unconditionally after it. // code here that can panic — and requeue `newest` unconditionally after it.
@@ -2230,6 +2493,19 @@ mod pipewire {
// pointer-only movements as metadata-only "corrupted" buffers we drop for their // pointer-only movements as metadata-only "corrupted" buffers we drop for their
// frame, but their cursor meta is fresh and must still move our overlay. // frame, but their cursor meta is fresh and must still move our overlay.
update_cursor_meta(&mut ud.cursor, spa_buf); update_cursor_meta(&mut ud.cursor, spa_buf);
// Publish the LIVE overlay (frames or not) so the encode loop's forwarder
// tracks pointer-only motion on a static desktop — the frame-attached overlay
// alone stales between damage frames. ONLY when we actually have one: a
// gamescope node carries no `SPA_META_Cursor`, so `overlay()` is always `None`
// here, and writing that would clobber — at frame rate — the `Some` the
// attached XFixes source publishes into this SAME slot, strobing the
// composited pointer on/off. Portal cursors are `None` only before the first
// bitmap (nothing to drop), and a HIDDEN pointer is still `Some(visible:false)`.
if let Some(overlay) = ud.cursor.overlay() {
if let Ok(mut slot) = ud.cursor_live.lock() {
*slot = Some(overlay);
}
}
// Inspect the newest buffer's header + first chunk for the diagnostic and the // Inspect the newest buffer's header + first chunk for the diagnostic and the
// CORRUPTED skip. SPA_META_Header is optional — `hdr` may be null. // CORRUPTED skip. SPA_META_Header is optional — `hdr` may be null.
@@ -2394,7 +2670,18 @@ mod pipewire {
build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?, build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?,
] ]
} else if want_dmabuf { } else if want_dmabuf {
vec![build_dmabuf_format(&modifiers, preferred)?] let mut pods = Vec::with_capacity(if prefer_native_nv12 { 2 } else { 1 });
if prefer_native_nv12 {
// First compatible consumer pod wins. Gamescope advertises NV12 and BGRx; pinning
// BT.709 limited here selects its RGB→NV12 shader with our bitstream colorimetry.
pods.push(build_dmabuf_format(VideoFormat::NV12, &[0], preferred)?);
}
pods.push(build_dmabuf_format(
VideoFormat::BGRx,
&modifiers,
preferred,
)?);
pods
} else { } else {
vec![serialize_pod(obj)?] vec![serialize_pod(obj)?]
}; };
@@ -0,0 +1,366 @@
//! XFixes cursor source for the gamescope capture path (remote-desktop-sweep Phase C).
//!
//! gamescope draws the pointer on a DRM hardware-cursor plane and its `paint_pipewire()`
//! deliberately excludes the cursor from the frame it feeds its built-in PipeWire node — so
//! `SPA_META_Cursor` never arrives and the ordinary [`cursor_live`](super::PortalCapturer) slot
//! stays empty (a KWin/GNOME session gets its cursor from that meta; gamescope can't embed one
//! either, its `set_hw_cursor` is inert). We instead read the pointer from gamescope's nested
//! Xwayland via X11 — the trick Sunshine uses — and publish a [`CursorOverlay`] into that same
//! slot, so the encoder blend composites the pointer into the video exactly like the portal path.
//!
//! **Multiple Xwaylands.** gamescope runs one Xwayland per `--xwayland-count` (Steam Gaming Mode
//! uses 2: one for Big Picture, one for the game). The pointer lives on whichever is FOCUSED — an
//! inactive display's pointer is frozen. So the source connects to ALL of them and each tick
//! follows the one whose pointer actually moved (gamescope routes input to the focused surface, so
//! exactly one moves at a time). It reads that display's shape too, since each Xwayland has its own
//! current cursor. This is why a single-display read froze the pointer the moment a game on the
//! OTHER Xwayland took focus.
//!
//! Two X sources per display, split by cost (Sunshine's split):
//! * **Position** — core `QueryPointer` on the root, polled fast. Cheap (a few-byte reply, no
//! bitmap), so it can out-pace the stream fps and keep the composited pointer smooth. It also
//! doubles as the focus signal (the display whose pointer moves is the active one).
//! * **Shape / hotspot / visibility** — `XFixesGetCursorImage`, refreshed only after an XFixes
//! `CursorNotify` (a real cursor change). A game hiding the pointer IS a cursor change → the
//! image comes back fully transparent → `visible: false`, which the encode loop strips before
//! any blend path draws it (so a grabbed pointer shows nothing, matching native gamescope).
use std::sync::{
atomic::{AtomicBool, Ordering},
Arc, Mutex,
};
use std::time::Duration;
use pf_frame::CursorOverlay;
use x11rb::connection::Connection;
use x11rb::errors::ReplyError;
use x11rb::protocol::xfixes::{self, ConnectionExt as _, GetCursorImageReply};
use x11rb::protocol::xproto::{ConnectionExt as _, QueryPointerReply, Window};
use x11rb::rust_connection::RustConnection;
/// Serializes the brief `XAUTHORITY` env swap around a connect (the var is process-global). Only
/// ever contended if two gamescope sessions start at once — rare, and the swap is microseconds.
static XAUTH_LOCK: Mutex<()> = Mutex::new(());
/// Position out-paces the stream fps (`POLL`); shape rides `CursorNotify` events drained each tick.
/// 4 ms ≈ 250 Hz matches the Windows GDI poller — the polled position IS the composited position
/// and must out-run a 240 fps session or the pointer stutters.
const POLL: Duration = Duration::from_millis(4);
/// A running XFixes cursor reader. Dropping it stops the worker thread and joins it, releasing the
/// X connections — so it lives exactly as long as the capturer that owns it.
pub(super) struct XFixesCursorSource {
stop: Arc<AtomicBool>,
join: Option<std::thread::JoinHandle<()>>,
}
impl XFixesCursorSource {
/// Connect to every gamescope nested Xwayland in `targets` (`(DISPLAY, XAUTHORITY)`) and start
/// publishing cursor overlays into `slot`, following the focused display's pointer. Returns
/// `None` — and logs — if NONE can be used (no X connection / no XFixes), so the caller
/// degrades to no gamescope cursor (today's behaviour) instead of failing the session.
pub(super) fn spawn(
targets: Vec<(String, Option<String>)>,
slot: Arc<Mutex<Option<CursorOverlay>>>,
) -> Option<Self> {
// Connect on the caller's thread so failures degrade cleanly and the displays are validated
// before we commit a thread.
let mut displays = Vec::new();
for (dpy, xauth) in targets {
match connect(&dpy, xauth.as_deref()) {
Ok((conn, root)) => displays.push(XDisplay::new(dpy, conn, root)),
Err(e) => tracing::warn!(
dpy = %dpy,
error = %e,
"gamescope cursor: skipping a nested Xwayland we can't use"
),
}
}
if displays.is_empty() {
tracing::warn!(
"gamescope cursor: no usable nested Xwayland — no in-video pointer this session \
(falls back to today's cursorless gamescope stream)"
);
return None;
}
let names: Vec<&str> = displays.iter().map(|d| d.name.as_str()).collect();
tracing::info!(
displays = ?names,
"gamescope cursor: XFixes source live — following the focused Xwayland's pointer"
);
let stop = Arc::new(AtomicBool::new(false));
let stop_worker = Arc::clone(&stop);
let join = std::thread::Builder::new()
.name("pf-gs-cursor".into())
.spawn(move || run(displays, slot, stop_worker))
.ok()?;
Some(XFixesCursorSource {
stop,
join: Some(join),
})
}
}
impl Drop for XFixesCursorSource {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(j) = self.join.take() {
let _ = j.join();
}
}
}
/// Open the X connection, negotiate XFixes, and select cursor-change events — returning the
/// connection + root window. `RustConnection` reads `XAUTHORITY` from the env at connect time only,
/// so set it under the lock (the host isn't a gamescope child), connect, then restore.
fn connect(dpy: &str, xauthority: Option<&str>) -> Result<(RustConnection, Window), String> {
let (conn, screen_num) = {
let _g = XAUTH_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let prev = std::env::var_os("XAUTHORITY");
if let Some(x) = xauthority {
std::env::set_var("XAUTHORITY", x);
}
let out = RustConnection::connect(Some(dpy));
match (&prev, xauthority) {
(Some(p), _) => std::env::set_var("XAUTHORITY", p),
(None, Some(_)) => std::env::remove_var("XAUTHORITY"),
(None, None) => {}
}
out.map_err(|e| format!("connect: {e}"))?
};
// XFixes ≥ 1 gives GetCursorImage / SelectCursorInput; ask for a modern minor, take what we get.
conn.xfixes_query_version(5, 0)
.map_err(ReplyError::from)
.and_then(|c| c.reply())
.map_err(|e| format!("XFixes unavailable: {e}"))?;
let root = conn
.setup()
.roots
.get(screen_num)
.ok_or_else(|| format!("no X screen {screen_num}"))?
.root;
// Wake the worker's event drain whenever the cursor shape changes (incl. hide/show).
conn.xfixes_select_cursor_input(root, xfixes::CursorNotifyMask::DISPLAY_CURSOR)
.map_err(ReplyError::from)
.and_then(|c| c.check())
.map_err(|e| format!("SelectCursorInput: {e}"))?;
let _ = conn.flush();
Ok((conn, root))
}
/// One gamescope Xwayland the source tracks.
struct XDisplay {
name: String,
conn: RustConnection,
root: Window,
/// Last polled pointer position — a change since the previous tick marks this display FOCUSED.
last_pos: Option<(i32, i32)>,
/// Cached cursor shape, refreshed only after this display's XFixes `CursorNotify`.
shape: Shape,
/// A `CursorNotify` (or first read) is pending — fetch the shape when this display is active.
need_shape: bool,
/// The X connection died (game/Xwayland exited) — skip it.
dead: bool,
}
impl XDisplay {
fn new(name: String, conn: RustConnection, root: Window) -> Self {
XDisplay {
name,
conn,
root,
last_pos: None,
shape: Shape::default(),
need_shape: true,
dead: false,
}
}
}
/// Cached cursor shape for one display.
#[derive(Default)]
struct Shape {
/// Straight-alpha RGBA (`w*h*4`, bytes R,G,B,A); empty before the first image arrives.
rgba: Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
/// XFixes' own per-display cursor serial — bumps on every shape change.
serial: u64,
/// A hidden pointer arrives as an all-transparent image; kept so a position-only tick preserves
/// the last known visibility.
visible: bool,
}
fn run(
mut displays: Vec<XDisplay>,
slot: Arc<Mutex<Option<CursorOverlay>>>,
stop: Arc<AtomicBool>,
) {
let mut active = 0usize;
// The overlay serial must bump whenever the DRAWN cursor changes — either the active display's
// shape OR which display is active (per-display XFixes serials aren't comparable across
// displays, so switching could reuse a number and the encoder would keep the old texture).
let mut out_serial = 0u64;
let mut last_key = (usize::MAX, u64::MAX);
let mut warned_image = false;
while !stop.load(Ordering::Relaxed) {
// 1) Poll every display's pointer; note which moved since last tick (the focus signal).
let mut active_moved = false;
let mut other_moved: Option<usize> = None;
for (i, d) in displays.iter_mut().enumerate() {
if d.dead {
continue;
}
// Drain pending events; the only ones selected are CursorNotify, so ANY event means
// "re-read this display's shape". poll_for_event never blocks.
loop {
match d.conn.poll_for_event() {
Ok(Some(_)) => d.need_shape = true,
Ok(None) => break,
Err(_) => {
d.dead = true;
break;
}
}
}
match fetch_pointer(&d.conn, d.root) {
Ok(p) if p.same_screen => {
let pos = (i32::from(p.root_x), i32::from(p.root_y));
let moved = d.last_pos.is_some_and(|lp| lp != pos);
d.last_pos = Some(pos);
if moved {
if i == active {
active_moved = true;
} else if other_moved.is_none() {
other_moved = Some(i);
}
}
}
Ok(_) => {} // pointer on another screen — keep the last position.
Err(_) => d.dead = true,
}
}
// 2) Switch focus: sticky to the active display while it moves (no flapping); otherwise
// follow another display that moved. If the active one died, fall to any live display.
if !active_moved {
if let Some(j) = other_moved {
active = j;
}
}
if displays.get(active).is_none_or(|d| d.dead) {
match displays.iter().position(|d| !d.dead) {
Some(k) => active = k,
None => {
std::thread::sleep(POLL); // all connections dead — idle until Drop.
continue;
}
}
}
// 3) Fetch the active display's shape if a CursorNotify (or a focus switch) left it stale.
if displays[active].need_shape {
match fetch_cursor_image(&displays[active].conn) {
Ok(img) => {
update_shape(&mut displays[active].shape, &img);
displays[active].need_shape = false;
}
Err(e) => {
if !warned_image {
warned_image = true;
tracing::warn!(error = %e, "gamescope cursor: GetCursorImage failed — retrying");
}
}
}
}
// 4) Publish the ACTIVE display's pointer + shape (or clear the slot when it has no cursor
// of its own — so a focus switch never leaves the other display's stale pointer showing).
let d = &displays[active];
let overlay = match (d.last_pos, d.shape.rgba.is_empty()) {
(Some((px, py)), false) => {
let key = (active, d.shape.serial);
if key != last_key {
out_serial += 1;
last_key = key;
}
Some(CursorOverlay {
// Top-left = pointer position hotspot (the overlay contract).
x: px - d.shape.hot_x as i32,
y: py - d.shape.hot_y as i32,
w: d.shape.w,
h: d.shape.h,
rgba: Arc::clone(&d.shape.rgba),
serial: out_serial,
hot_x: d.shape.hot_x,
hot_y: d.shape.hot_y,
visible: d.shape.visible,
})
}
_ => None,
};
if let Ok(mut s) = slot.lock() {
*s = overlay;
}
std::thread::sleep(POLL);
}
}
/// Update `shape` from a fresh `GetCursorImage` reply. A hidden pointer (all-transparent) keeps the
/// last bitmap (instant re-show) but flips visibility; the serial still bumps so the change shows.
fn update_shape(shape: &mut Shape, img: &GetCursorImageReply) {
let visible =
img.width > 0 && img.height > 0 && img.cursor_image.iter().any(|&p| (p >> 24) & 0xff != 0);
if visible {
shape.rgba = Arc::new(argb_premul_to_straight_rgba(&img.cursor_image));
shape.w = u32::from(img.width);
shape.h = u32::from(img.height);
shape.hot_x = u32::from(img.xhot);
shape.hot_y = u32::from(img.yhot);
}
shape.visible = visible;
shape.serial = u64::from(img.cursor_serial);
}
/// One request+reply — x11rb splits errors (the request is `ConnectionError`, `reply()` is
/// `ReplyError` which is `From<ConnectionError>`), so the request `?` converts into the reply error.
fn fetch_cursor_image(conn: &RustConnection) -> Result<GetCursorImageReply, ReplyError> {
conn.xfixes_get_cursor_image()?.reply()
}
fn fetch_pointer(conn: &RustConnection, root: Window) -> Result<QueryPointerReply, ReplyError> {
conn.query_pointer(root)?.reply()
}
/// XFixes cursor pixels are packed `0xAARRGGBB` with **premultiplied** alpha (the Xrender / Xcursor
/// convention). The overlay + both blend paths want **straight** alpha RGBA (R,G,B,A bytes), like
/// the `SPA_META_Cursor` path — so un-premultiply here. (If on-glass shows over-bright fringes the
/// source wasn't premultiplied after all; drop the divide.)
fn argb_premul_to_straight_rgba(argb: &[u32]) -> Vec<u8> {
let mut out = Vec::with_capacity(argb.len() * 4);
for &px in argb {
let a = (px >> 24) & 0xff;
let r = (px >> 16) & 0xff;
let g = (px >> 8) & 0xff;
let b = px & 0xff;
let (r, g, b) = match a {
0 => (0, 0, 0),
255 => (r, g, b),
a => (
((r * 255 + a / 2) / a).min(255),
((g * 255 + a / 2) / a).min(255),
((b * 255 + a / 2) / a).min(255),
),
};
out.extend_from_slice(&[r as u8, g as u8, b as u8, a as u8]);
}
out
}
+211 -14
View File
@@ -161,7 +161,7 @@ pub fn install_gpu_pref_hook() {
}); });
} }
unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u8>> { pub(crate) unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u8>> {
let mut blob: Option<ID3DBlob> = None; let mut blob: Option<ID3DBlob> = None;
let mut errs: Option<ID3DBlob> = None; let mut errs: Option<ID3DBlob> = None;
let r = D3DCompile( let r = D3DCompile(
@@ -194,7 +194,7 @@ unsafe fn compile_shader(src: &str, entry: PCSTR, target: PCSTR) -> Result<Vec<u
} }
/// Fullscreen-triangle vertex shader for the HDR conversion pass (3 verts, no input layout). /// Fullscreen-triangle vertex shader for the HDR conversion pass (3 verts, no input layout).
const HDR_VS: &str = r" pub(crate) const HDR_VS: &str = r"
struct VOut { float4 pos : SV_POSITION; float2 uv : TEXCOORD0; }; struct VOut { float4 pos : SV_POSITION; float2 uv : TEXCOORD0; };
VOut main(uint vid : SV_VertexID) { VOut main(uint vid : SV_VertexID) {
float2 uv = float2((vid << 1) & 2, vid & 2); float2 uv = float2((vid << 1) & 2, vid & 2);
@@ -308,8 +308,9 @@ float2 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
/// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV /// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV
/// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This /// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This
/// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`] /// planar-RTV mechanism needs a D3D11.3+ runtime + driver support; [`HdrP010Converter::convert`]
/// surfaces a clear error if `CreateRenderTargetView` rejects the plane format so the caller can fall /// surfaces a clear error if `CreateRenderTargetView` rejects the plane format. (There is no runtime
/// back to the existing R10 path. /// fallback — the error propagates through `try_consume` and ends the session; the "R10 path" the
/// original design referenced was never kept.)
pub(crate) struct HdrP010Converter { pub(crate) struct HdrP010Converter {
vs: ID3D11VertexShader, vs: ID3D11VertexShader,
ps_y: ID3D11PixelShader, ps_y: ID3D11PixelShader,
@@ -737,14 +738,157 @@ fn p010_reference(r: f64, g: f64, b: f64) -> (f64, f64, f64) {
/// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL. /// Y ≤ 4 codes, U/V ≤ 5 codes (rounding + chroma averaging). Prints a per-colour table + PASS/FAIL.
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
pub fn hdr_p010_selftest() -> Result<()> { pub fn hdr_p010_selftest() -> Result<()> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_HARDWARE; hdr_p010_selftest_at(64, 64, None)
use windows::Win32::Graphics::Dxgi::IDXGIAdapter; }
// 64x64, even dims. A 4x4 grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform → /// [`hdr_p010_selftest`] at an arbitrary even size and (optionally) on a specific GPU vendor
// exact chroma comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus /// (PCI vendor id, e.g. `0x8086` Intel / `0x10de` NVIDIA / `0x1002` AMD). The size matters on
// a couple of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only). /// top of the 64×64 default because the field sessions run at capture resolutions whose height
const W: u32 = 64; /// is NOT 16-aligned (1080 → the encoder's align16 pool seam) and a driver may treat the planar
const H: u32 = 64; /// RTVs differently at real sizes; the vendor pin matters on dual-GPU boxes where the default
/// adapter is not the one the session encodes on.
/// Test support (used by pf-encode's live e2e): the 8 sRGB colour bars (white/yellow/cyan/green/
/// magenta/red/blue/black, sRGB 1.0 = scRGB 1.0 = 80 nits) as a w×h FP16 scRGB texture on the
/// adapter with `luid`, converted through the REAL [`HdrP010Converter`] into a P010 texture with
/// **`BIND_RENDER_TARGET` only, `MiscFlags` 0 — the exact bind profile of the IDD out-ring** (the
/// CPU-upload encoder tests can't use that profile, so only this path exercises "RTV-written P010
/// → encoder ingest copy"). Returns `(device, p010)`; expected decoded codes per bar are the
/// bars_pq2020 fixture's: (490,512,512) (478,423,518) (464,525,473) (450,432,476) (350,584,585)
/// (325,448,598) (226,650,535) (64,512,512).
#[cfg(target_os = "windows")]
#[doc(hidden)]
pub fn hdr_p010_convert_bars_on_luid(
luid: [u8; 8],
w: u32,
h: u32,
) -> Result<(ID3D11Device, ID3D11Texture2D)> {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("bars pattern needs even non-zero dimensions, got {w}x{h}");
}
// sRGB primaries at full/zero channels: sRGB EOTF(1.0)=1.0, (0)=0 → the scRGB pattern is
// pure 0/1 floats and the PQ/BT.2020 reference codes above are exact.
const BARS: [(f32, f32, f32); 8] = [
(1.0, 1.0, 1.0),
(1.0, 1.0, 0.0),
(0.0, 1.0, 1.0),
(0.0, 1.0, 0.0),
(1.0, 0.0, 1.0),
(1.0, 0.0, 0.0),
(0.0, 0.0, 1.0),
(0.0, 0.0, 0.0),
];
let bar_w = (w / 8).max(1) as usize;
let mut fp16 = vec![0u16; (w * h * 4) as usize];
for y in 0..h as usize {
for x in 0..w as usize {
let (r, g, b) = BARS[(x / bar_w).min(7)];
let i = (y * w as usize + x) * 4;
fp16[i] = f32_to_f16(r);
fp16[i + 1] = f32_to_f16(g);
fp16[i + 2] = f32_to_f16(b);
fp16[i + 3] = f32_to_f16(1.0);
}
}
// SAFETY: same single-device/single-thread contract as `hdr_p010_selftest_at`; the FP16
// initial-data Vec outlives the synchronous CreateTexture2D; the returned COM handles own
// their references.
unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().context("dxgi factory")?;
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).context("adapter by luid")?;
let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]),
D3D11_SDK_VERSION,
Some(&mut device),
None,
Some(&mut context),
)
.context("D3D11CreateDevice(luid) for bars convert")?;
let device = device.context("null device")?;
let context = context.context("null context")?;
let src_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R16G16B16A16_FLOAT,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: fp16.as_ptr() as *const c_void,
SysMemPitch: w * 8,
SysMemSlicePitch: 0,
};
let mut src_tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&src_desc, Some(&init), Some(&mut src_tex))
.context("CreateTexture2D(fp16 bars)")?;
let src_tex = src_tex.context("null src tex")?;
let mut src_srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&src_tex, None, Some(&mut src_srv))
.context("CreateShaderResourceView(fp16 bars)")?;
let src_srv = src_srv.context("null src srv")?;
// The IDD out-ring's exact profile: P010, RENDER_TARGET only, MiscFlags 0.
let p010_desc = D3D11_TEXTURE2D_DESC {
Width: w,
Height: h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_RENDER_TARGET.0 as u32,
..Default::default()
};
let mut p010: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&p010_desc, None, Some(&mut p010))
.context("CreateTexture2D(P010 bars dst)")?;
let p010 = p010.context("null p010 tex")?;
let conv = HdrP010Converter::new(&device)?;
conv.convert(&device, &context, &src_srv, &p010, w, h)?;
Ok((device, p010))
}
}
#[cfg(target_os = "windows")]
pub fn hdr_p010_selftest_at(w: u32, h: u32, vendor: Option<u32>) -> Result<()> {
use windows::Win32::Graphics::Direct3D::{D3D_DRIVER_TYPE_HARDWARE, D3D_DRIVER_TYPE_UNKNOWN};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter, IDXGIFactory1};
if w == 0 || h == 0 || w % 2 != 0 || h % 2 != 0 {
bail!("hdr-p010-selftest needs even non-zero dimensions, got {w}x{h}");
}
// A grid of 16x16 flat scRGB blocks (each 2x2 chroma footprint uniform → exact chroma
// comparison) covering pure R/G/B/white/black/gray at plausible HDR nit levels, plus a couple
// of bright (>1.0 scRGB) colours, then the rest is a gradient (compared on Y only).
#[allow(non_snake_case)]
let (W, H) = (w, h);
const BLK: u32 = 16; const BLK: u32 = 16;
// (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values. // (name, r, g, b) scRGB linear (1.0 = 80 nits). Mix of SDR-ish and HDR (>1.0) values.
let named: [(&str, f32, f32, f32); 8] = [ let named: [(&str, f32, f32, f32); 8] = [
@@ -797,12 +941,36 @@ pub fn hdr_p010_selftest() -> Result<()> {
// `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are // `fp16` outlives the synchronous `CreateTexture2D` that reads it. The mapped-pointer reads are
// proven individually at the `read_u16` closure below. // proven individually at the `read_u16` closure below.
unsafe { unsafe {
// Hardware D3D11 device (no adapter pin — the default GPU is fine for the self-test). // Device on the requested vendor's adapter (dual-GPU boxes encode on a specific one), else
// the default hardware GPU. Always says which adapter ran — a PASS is only meaningful for
// the GPU it actually tested.
let adapter: Option<IDXGIAdapter> = match vendor {
None => None,
Some(want) => {
let factory: IDXGIFactory1 = CreateDXGIFactory1().context("dxgi factory")?;
let mut found = None;
for i in 0.. {
let Ok(a) = factory.EnumAdapters(i) else {
break;
};
let desc = a.GetDesc().context("adapter desc")?;
if desc.VendorId == want {
found = Some(a);
break;
}
}
Some(found.with_context(|| format!("no adapter with vendor id {want:#x}"))?)
}
};
let mut device: Option<ID3D11Device> = None; let mut device: Option<ID3D11Device> = None;
let mut context: Option<ID3D11DeviceContext> = None; let mut context: Option<ID3D11DeviceContext> = None;
D3D11CreateDevice( D3D11CreateDevice(
None::<&IDXGIAdapter>, adapter.as_ref(),
D3D_DRIVER_TYPE_HARDWARE, if adapter.is_some() {
D3D_DRIVER_TYPE_UNKNOWN
} else {
D3D_DRIVER_TYPE_HARDWARE
},
HMODULE::default(), HMODULE::default(),
D3D11_CREATE_DEVICE_BGRA_SUPPORT, D3D11_CREATE_DEVICE_BGRA_SUPPORT,
Some(&[D3D_FEATURE_LEVEL_11_0]), Some(&[D3D_FEATURE_LEVEL_11_0]),
@@ -814,6 +982,22 @@ pub fn hdr_p010_selftest() -> Result<()> {
.context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?; .context("D3D11CreateDevice(hardware) for hdr-p010-selftest")?;
let device = device.context("null device")?; let device = device.context("null device")?;
let context = context.context("null context")?; let context = context.context("null context")?;
{
let dxgi: windows::Win32::Graphics::Dxgi::IDXGIDevice =
device.cast().context("device -> IDXGIDevice")?;
let desc = dxgi.GetAdapter().context("GetAdapter")?.GetDesc()?;
let name = String::from_utf16_lossy(
&desc.Description[..desc
.Description
.iter()
.position(|&c| c == 0)
.unwrap_or(desc.Description.len())],
);
println!(
"adapter: {name} (vendor {:#06x}, luid {:08x}:{:08x})",
desc.VendorId, desc.AdapterLuid.HighPart, desc.AdapterLuid.LowPart
);
}
// Source FP16 texture (initialized) + SRV. // Source FP16 texture (initialized) + SRV.
let src_desc = D3D11_TEXTURE2D_DESC { let src_desc = D3D11_TEXTURE2D_DESC {
@@ -1175,3 +1359,16 @@ impl VideoConverter {
blt.context("VideoProcessorBlt") blt.context("VideoProcessorBlt")
} }
} }
#[cfg(test)]
mod hdr_selftests {
/// LIVE (needs the GPU): [`super::hdr_p010_selftest_at`] at the field capture size — 1080 is
/// NOT 16-aligned, and the planar-RTV write path is driver-specific per vendor. Pinned to the
/// Intel adapter (`0x8086`), so it runs on the Intel validation boxes and errors out cleanly
/// ("no adapter") elsewhere. `cargo test -p pf-capture -- --ignored hdr_p010 --nocapture`.
#[test]
#[ignore]
fn hdr_p010_selftest_intel_1080_live() {
super::hdr_p010_selftest_at(1920, 1080, Some(0x8086)).expect("hdr p010 selftest @1080");
}
}
+393 -22
View File
@@ -365,6 +365,12 @@ pub unsafe fn verify_is_wudfhost(process: HANDLE, wudf_pid: u32, what: &str) ->
#[path = "idd_push/channel.rs"] #[path = "idd_push/channel.rs"]
mod channel; mod channel;
#[path = "idd_push/cursor.rs"]
mod cursor;
#[path = "idd_push/cursor_blend.rs"]
mod cursor_blend;
#[path = "idd_push/cursor_poll.rs"]
mod cursor_poll;
#[path = "idd_push/descriptor.rs"] #[path = "idd_push/descriptor.rs"]
mod descriptor; mod descriptor;
#[path = "idd_push/stall.rs"] #[path = "idd_push/stall.rs"]
@@ -386,6 +392,54 @@ pub struct IddPushCapturer {
/// The sealed channel's handle-duplication broker (WUDFHost process + control device); used at open /// The sealed channel's handle-duplication broker (WUDFHost process + control device); used at open
/// and again on every ring recreate to deliver fresh duplicates. /// and again on every ring recreate to deliver fresh duplicates.
broker: ChannelBroker, broker: ChannelBroker,
/// The v5 hardware-cursor channel's host end (`Some` = delivered; the driver declared the
/// hardware cursor and seqlock-publishes into it). Survives ring recreates — the section is
/// independent of the frame ring's generation. With the channel delivered, the driver's
/// hardware cursor keeps DWM from compositing ANY cursor into the frame; the SHAPE now comes
/// from [`cursor_poll::CursorPoller`] (the IddCx query is alpha-only — see cursor_poll.rs),
/// and this shm read is the fallback if that poller dies.
cursor_shared: Option<cursor::CursorShared>,
/// The GDI cursor-shape poller (design §8): the overlay source while alive. `Some` when
/// `cursor_shared` is (both ride the negotiated cursor channel + successful delivery) — or
/// when `composite_forced` (no channel, but the target's sticky declare needs a blend source).
cursor_poll: Option<cursor_poll::CursorPoller>,
/// Retained delivery sender (`IOCTL_SET_CURSOR_CHANNEL`) for RE-delivery: a driver-side
/// monitor re-arrival (match-window re-arrival resize, a sibling session recreating the
/// shared slot) destroys the driver's cursor worker — the section here survives, so the
/// channel is re-delivered on ring recreates.
cursor_sender: Option<crate::CursorChannelSender>,
/// The CAPTURE mouse model is active — the HOST composites the pointer into the frame
/// (see cursor_blend.rs for why DWM cannot: a declared IddCx hardware cursor is forever).
composite_cursor: bool,
/// This session never negotiated the cursor channel but its target carries an IRREVOCABLE
/// hardware-cursor declare from an earlier session (`WinCaptureTarget::cursor_excluded`,
/// §8.6): DWM delivers pointer-free frames and no client draws the cursor, so the ONLY path
/// to a visible pointer is compositing here. Pins `composite_cursor` on — nothing may turn
/// it off (there is no channel to hand the pointer to).
composite_forced: bool,
/// The cursor-quad blend pass (lazy; per capture device). `None` after a build failure —
/// composite mode then degrades to pointer-less frames (warned once).
cursor_blend: Option<cursor_blend::CursorBlendPass>,
cursor_blend_failed: bool,
/// The frame-sized blend scratch (slot copy + cursor quad): texture + SRV + (w, h, fmt)
/// it was built for — rebuilt when the ring geometry changes.
blend_scratch: Option<(
ID3D11Texture2D,
ID3D11ShaderResourceView,
u32,
u32,
DXGI_FORMAT,
)>,
/// The (serial, x, y, visible) of the LAST blended pointer — the composite-regen change
/// key: pointer-only motion produces no driver publish (the declared hardware cursor
/// doesn't dirty frames), so `try_consume` regenerates from the last slot when this moves.
last_blend_key: Option<(u64, i32, i32, bool)>,
/// The ring slot of the last FRESH publish — the regen source.
last_slot: Option<usize>,
/// The target's SDR-white scale (vs 80 nits) for HDR cursor compositing — refreshed on
/// each blend-scratch rebuild (first use + ring geometry changes). 2.5 ≈ the Windows
/// SDR-brightness default; without it the composited cursor renders visibly dark on HDR.
sdr_white_scale: f32,
width: u32, width: u32,
height: u32, height: u32,
slots: Vec<HostSlot>, slots: Vec<HostSlot>,
@@ -603,6 +657,7 @@ impl IddPushCapturer {
/// instead of tearing the display down (audit §5.1 — no more 20 s black bail). "Failure" includes the /// instead of tearing the display down (audit §5.1 — no more 20 s black bail). "Failure" includes the
/// driver not attaching to the ring within a few seconds (e.g. a hybrid-GPU render mismatch). /// driver not attaching to the ring within a few seconds (e.g. a hybrid-GPU render mismatch).
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
#[allow(clippy::too_many_arguments)]
pub fn open( pub fn open(
target: WinCaptureTarget, target: WinCaptureTarget,
preferred: Option<(u32, u32, u32)>, preferred: Option<(u32, u32, u32)>,
@@ -611,11 +666,20 @@ impl IddPushCapturer {
pyrowave: bool, pyrowave: bool,
keepalive: Box<dyn Send>, keepalive: Box<dyn Send>,
sender: crate::FrameChannelSender, sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> { ) -> std::result::Result<Self, (anyhow::Error, Box<dyn Send>)> {
// The stall-attribution listener (idempotent): started with the first IDD-push capturer so // The stall-attribution listener (idempotent): started with the first IDD-push capturer so
// the stall log can correlate DWM holes with OS display events for the session's lifetime. // the stall log can correlate DWM holes with OS display events for the session's lifetime.
pf_win_display::display_events::spawn_once(); pf_win_display::display_events::spawn_once();
match Self::open_inner(target, preferred, client_10bit, want_444, pyrowave, sender) { match Self::open_inner(
target,
preferred,
client_10bit,
want_444,
pyrowave,
sender,
cursor_sender,
) {
Ok(mut me) => { Ok(mut me) => {
me._keepalive = keepalive; me._keepalive = keepalive;
Ok(me) Ok(me)
@@ -632,6 +696,7 @@ impl IddPushCapturer {
want_444: bool, want_444: bool,
pyrowave: bool, pyrowave: bool,
sender: crate::FrameChannelSender, sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
) -> Result<Self> { ) -> Result<Self> {
// The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the // The ring MUST live on the adapter the driver's swap-chain renders on. Primary: the
// selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor // selected render GPU — the same pick SET_RENDER_ADAPTER pinned the driver to at monitor
@@ -654,6 +719,7 @@ impl IddPushCapturer {
pyrowave, pyrowave,
luid, luid,
sender.clone(), sender.clone(),
cursor_sender.clone(),
) { ) {
Ok(me) => Ok(me), Ok(me) => Ok(me),
Err(e) => { Err(e) => {
@@ -687,6 +753,7 @@ impl IddPushCapturer {
pyrowave, pyrowave,
drv, drv,
sender, sender,
cursor_sender,
) )
.context("IDD-push rebind to the driver's reported render adapter") .context("IDD-push rebind to the driver's reported render adapter")
} }
@@ -702,6 +769,7 @@ impl IddPushCapturer {
pyrowave: bool, pyrowave: bool,
luid: LUID, luid: LUID,
sender: crate::FrameChannelSender, sender: crate::FrameChannelSender,
cursor_sender: Option<crate::CursorChannelSender>,
) -> Result<Self> { ) -> Result<Self> {
let (pw, ph, _hz) = preferred let (pw, ph, _hz) = preferred
.context("IDD push needs the negotiated mode (WxH) to size the shared ring")?; .context("IDD push needs the negotiated mode (WxH) to size the shared ring")?;
@@ -935,6 +1003,50 @@ impl IddPushCapturer {
) )
.context("deliver IDD-push frame channel to the driver")?; .context("deliver IDD-push frame channel to the driver")?;
// v5 hardware-cursor channel (M2c): create + deliver the CursorShm section. Failure
// is NON-fatal — the driver never declares the hardware cursor without this delivery,
// so the session degrades to today's composited pointer (and the forwarder simply
// never sees a live overlay).
let cursor_shared = cursor_sender.as_ref().and_then(|send_cursor| {
match cursor::CursorShared::create(target.target_id) {
Ok(cs) => {
// Deliver via the shared helper (also used for RE-delivery after a
// driver-side monitor re-arrival destroyed the worker).
deliver_cursor_channel(&broker, target.target_id, &cs, send_cursor)
.then_some(cs)
}
Err(e) => {
tracing::warn!(
"cursor section creation failed (composited cursor stays): {e:#}"
);
None
}
}
});
// No channel this session, but the target's sticky declare (an EARLIER session's —
// irrevocable, §8.6) keeps DWM's frames pointer-free with no client drawing either:
// the only visible pointer is the one composited here, so force composite mode on.
let composite_forced = target.cursor_excluded && cursor_sender.is_none();
if composite_forced {
tracing::info!(
target_id = target.target_id,
"target carries an irrevocable hardware-cursor declare from an earlier \
desktop-mode session and this session has no cursor channel the host \
composites the pointer into frames (forced, for the session's life)"
);
}
// The GDI shape poller rides the SAME gate as the delivered channel: with the driver's
// hardware cursor keeping the frame cursor-free, the poller supplies the full-fidelity
// shape (masked/monochrome included — the IddCx query can't; see cursor_poll.rs).
// Forced-composite sessions need it too — it is their only shape/position source.
let cursor_poll = (cursor_shared.is_some() || composite_forced).then(|| {
// Safety of the CCD call: read-only QueryDisplayConfig over owned locals (same
// call CursorShared::create makes) — already inside open_on's unsafe region.
let rect = pf_win_display::win_display::source_desktop_rect(target.target_id)
.unwrap_or((0, 0, i32::MAX, i32::MAX));
cursor_poll::CursorPoller::spawn(target.target_id, rect)
});
tracing::info!( tracing::info!(
target_id = target.target_id, target_id = target.target_id,
wudf_pid = target.wudf_pid, wudf_pid = target.wudf_pid,
@@ -993,6 +1105,17 @@ impl IddPushCapturer {
last_seq: 0, last_seq: 0,
last_present: None, last_present: None,
status_logged: false, status_logged: false,
cursor_shared,
cursor_poll,
cursor_sender,
composite_cursor: composite_forced,
composite_forced,
cursor_blend: None,
cursor_blend_failed: false,
blend_scratch: None,
last_blend_key: None,
last_slot: None,
sdr_white_scale: 1.0,
// Held from BEFORE the first-frame gate (the display must not idle off while we // Held from BEFORE the first-frame gate (the display must not idle off while we
// wait for the first compose) until the capturer drops with the session. // wait for the first compose) until the capturer drops with the session.
_display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(), _display_wake: pf_frame::session_tuning::DisplayWakeRequest::new(),
@@ -1337,6 +1460,14 @@ impl IddPushCapturer {
"IDD push: frame-channel re-delivery failed after ring recreate" "IDD push: frame-channel re-delivery failed after ring recreate"
); );
} }
// Ring recreates ride display churn that can also have re-arrived the MONITOR driver-side
// (destroying its cursor worker with it) — re-deliver the surviving cursor section so the
// hardware-cursor declaration follows the CURRENT monitor generation.
if let (Some(cs), Some(send)) = (self.cursor_shared.as_ref(), self.cursor_sender.as_ref()) {
let _ = deliver_cursor_channel(&self.broker, self.target_id, cs, send);
}
self.blend_scratch = None; // ring geometry/format changed — rebuild at next blend
self.last_slot = None; // old-ring slot indices are meaningless now
self.last_seq = 0; self.last_seq = 0;
self.out_ring.clear(); // the output format changed → rebuild lazily at the new format self.out_ring.clear(); // the output format changed → rebuild lazily at the new format
self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode
@@ -1622,6 +1753,136 @@ impl IddPushCapturer {
Ok(Some((self.pyro_fence_handle, value))) Ok(Some((self.pyro_fence_handle, value)))
} }
/// The (serial, x, y, visible) of the CURRENT polled cursor — the composite-regen change
/// key. `None` while the poller has no shape yet (or isn't running).
fn cursor_blend_key(&self) -> Option<(u64, i32, i32, bool)> {
self.cursor_poll
.as_ref()
.and_then(|p| p.read())
.map(|o| (o.serial, o.x, o.y, o.visible))
}
/// Composite the pointer for this convert: ensure the frame-sized blend scratch, copy the
/// slot into it, and alpha-blend the GDI poller's shape at its polled position. Returns the
/// scratch (texture + SRV) the conversion should read INSTEAD of the slot; `None` degrades
/// to the pointer-less slot (scratch/pass creation failed — warned once). A hidden pointer
/// blends nothing (the plain copy is the correct frame).
///
/// # Safety
/// D3D11 calls on the owning capture/encode thread's device + immediate context, called
/// while holding the slot's keyed mutex (the copy reads the slot).
unsafe fn prepare_blend_scratch(
&mut self,
slot_tex: &ID3D11Texture2D,
) -> Option<(ID3D11Texture2D, ID3D11ShaderResourceView)> {
let fmt = self.ring_format();
// (Re)build the scratch at the current ring geometry.
let stale = self
.blend_scratch
.as_ref()
.is_none_or(|(_, _, w, h, f)| (*w, *h, *f) != (self.width, self.height, fmt));
if stale {
self.blend_scratch = None;
let desc = D3D11_TEXTURE2D_DESC {
Width: self.width,
Height: self.height,
MipLevels: 1,
ArraySize: 1,
Format: fmt,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
..Default::default()
};
let mut tex: Option<ID3D11Texture2D> = None;
let built = self
.device
.CreateTexture2D(&desc, None, Some(&mut tex))
.ok()
.and(tex)
.and_then(|t| {
let mut srv: Option<ID3D11ShaderResourceView> = None;
self.device
.CreateShaderResourceView(&t, None, Some(&mut srv))
.ok()
.and(srv)
.map(|v| (t, v))
});
match built {
Some((t, v)) => {
self.blend_scratch = Some((t, v, self.width, self.height, fmt));
if self.display_hdr {
// Where DWM places SDR white on this HDR desktop — the composited
// cursor must match or it reads dark (~2.5x at the Windows default).
// Queried only here: scratch rebuilds are rare, and the CCD query
// contends on the display-config lock, which must stay OFF the
// per-frame path.
// Safety: read-only CCD query over owned locals (within unsafe fn).
let queried =
pf_win_display::win_display::sdr_white_level_scale(self.target_id);
self.sdr_white_scale = queried.unwrap_or(self.sdr_white_scale);
tracing::info!(
target_id = self.target_id,
queried = ?queried,
applied = self.sdr_white_scale,
"cursor composite: HDR SDR-white scale (1.0 = 80 nits; None = \
query failed keeping the prior value)"
);
}
}
None => {
if !self.cursor_blend_failed {
self.cursor_blend_failed = true;
tracing::warn!(
"cursor blend scratch creation failed — capture-model frames stay \
pointer-less this session"
);
}
return None;
}
}
}
let (tex, srv, ..) = self.blend_scratch.as_ref().expect("just ensured");
let (tex, srv) = (tex.clone(), srv.clone());
self.context.CopyResource(&tex, slot_tex);
// Blend the pointer (visible shapes only; hidden = the copy alone is the frame).
let overlay = self.cursor_poll.as_ref().and_then(|p| p.read());
self.last_blend_key = overlay.as_ref().map(|o| (o.serial, o.x, o.y, o.visible));
if let Some(ov) = overlay.filter(|o| o.visible) {
if self.cursor_blend.is_none() && !self.cursor_blend_failed {
match cursor_blend::CursorBlendPass::new(&self.device) {
Ok(p) => self.cursor_blend = Some(p),
Err(e) => {
self.cursor_blend_failed = true;
tracing::warn!(
"cursor blend pass build failed — capture-model frames stay \
pointer-less this session: {e:#}"
);
}
}
}
if let Some(pass) = self.cursor_blend.as_mut() {
// FP16 ring = scRGB linear composition (HDR): linearize the sRGB shape and
// scale it to the target's SDR white so it matches the desktop around it.
let scale = if self.display_hdr {
self.sdr_white_scale
} else {
0.0
};
if let Err(e) = pass.blend(&self.device, &self.context, &tex, &ov, scale) {
if !self.cursor_blend_failed {
self.cursor_blend_failed = true;
tracing::warn!("cursor blend draw failed — pointer-less frames: {e:#}");
}
}
}
}
Some((tex, srv))
}
fn try_consume(&mut self) -> Result<Option<CapturedFrame>> { fn try_consume(&mut self) -> Result<Option<CapturedFrame>> {
self.log_driver_status_once(); self.log_driver_status_once();
// Follow the display: a "Use HDR" flip recreates the ring at the matching format. // Follow the display: a "Use HDR" flip recreates the ring at the matching format.
@@ -1682,9 +1943,25 @@ impl IddPushCapturer {
return Ok(None); return Ok(None);
} }
let seq = u64::from(tok.seq); let seq = u64::from(tok.seq);
let slot = tok.slot as usize; let mut slot = tok.slot as usize;
if seq == self.last_seq || slot >= self.slots.len() { let fresh = seq != self.last_seq && slot < self.slots.len();
return Ok(None); let mut regen = false;
if !fresh {
// Composite cursor model: pointer-only motion produces NO new publish (the declared
// hardware cursor never dirties the frame), so a static desktop would freeze the
// blended pointer. Regenerate from the LAST slot whenever the polled cursor state
// changed — the re-converted out-ring frame carries the pointer's new position.
let moved = self.composite_cursor
&& self.last_slot.is_some()
&& self.cursor_blend_key() != self.last_blend_key;
if !moved {
return Ok(None);
}
slot = self.last_slot.expect("checked above");
if slot >= self.slots.len() {
return Ok(None); // ring shrank across a recreate — wait for a fresh publish
}
regen = true;
} }
// Build the ring + converter BEFORE acquiring the slot so nothing between Acquire and Release // Build the ring + converter BEFORE acquiring the slot so nothing between Acquire and Release
// can `?`-return and leak the keyed-mutex lock (which would stall the driver on that slot). // can `?`-return and leak the keyed-mutex lock (which would stall the driver on that slot).
@@ -1718,18 +1995,31 @@ impl IddPushCapturer {
// Hold the slot's keyed mutex only across the convert/copy into the host out-ring (NOT across the // Hold the slot's keyed mutex only across the convert/copy into the host out-ring (NOT across the
// ~3 ms encode — NVENC reads the host out-ring slot, not the keyed-mutex slot), so the driver gets // ~3 ms encode — NVENC reads the host out-ring slot, not the keyed-mutex slot), so the driver gets
// the slot back immediately and the encode of the PREVIOUS frame overlaps this convert. // the slot back immediately and the encode of the PREVIOUS frame overlaps this convert.
let s = &self.slots[slot]; // Clone the slot's COM interfaces (an AddRef each) so the guard borrows LOCALS, leaving
// `self` free for the composite blend prep inside the lock.
let (slot_tex, slot_srv, slot_mutex) = {
let s = &self.slots[slot];
(s.tex.clone(), s.srv.clone(), s.mutex.clone())
};
// Acquire the slot's keyed mutex via a RAII guard, scoped to JUST the convert/copy below so it // Acquire the slot's keyed mutex via a RAII guard, scoped to JUST the convert/copy below so it
// releases at the same point as the old hand-written `ReleaseSync` (the driver gets the slot back // releases at the same point as the old hand-written `ReleaseSync` (the driver gets the slot back
// immediately, NOT held across the rest of `try_consume`) — but now leak-proof on any early return. // immediately, NOT held across the rest of `try_consume`) — but now leak-proof on any early return.
{ {
let Some(_lock) = KeyedMutexGuard::acquire(&s.mutex, 0, 8) else { let Some(_lock) = KeyedMutexGuard::acquire(&slot_mutex, 0, 8) else {
return Ok(None); return Ok(None);
}; };
// SAFETY: convert on the owning (encode) thread's immediate context, holding the slot lock. // SAFETY: convert on the owning (encode) thread's immediate context, holding the slot lock.
// A `?` here is leak-safe: `_lock` (the KeyedMutexGuard) drops on the early return, releasing // A `?` here is leak-safe: `_lock` (the KeyedMutexGuard) drops on the early return, releasing
// the slot back to the driver. // the slot back to the driver.
unsafe { unsafe {
// Composite cursor model: divert the convert input through the blend scratch —
// a slot copy with the pointer quad alpha-blended on top. `None` = compositing
// off or degraded (the conversion then reads the slot as always).
let blended = if self.composite_cursor {
self.prepare_blend_scratch(&slot_tex)
} else {
None
};
if self.pyrowave { if self.pyrowave {
// PyroWave: ring slot SRV (BGRA for SDR, scRGB FP16 for HDR) → the two separate // PyroWave: ring slot SRV (BGRA for SDR, scRGB FP16 for HDR) → the two separate
// plane textures via the mode-aware CSC; the shared fence signalled just after // plane textures via the mode-aware CSC; the shared fence signalled just after
@@ -1737,22 +2027,17 @@ impl IddPushCapturer {
// convert. The composition format is pinned to the negotiated depth. // convert. The composition format is pinned to the negotiated depth.
let (_, y_rtv, _, cbcr_rtv) = pyro_slot.as_ref().expect("pyro slot"); let (_, y_rtv, _, cbcr_rtv) = pyro_slot.as_ref().expect("pyro slot");
if let Some(conv) = self.pyro_conv.as_ref() { if let Some(conv) = self.pyro_conv.as_ref() {
conv.convert( let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
&self.context, conv.convert(&self.context, src, y_rtv, cbcr_rtv, self.width, self.height)?;
&s.srv,
y_rtv,
cbcr_rtv,
self.width,
self.height,
)?;
} }
} else if self.display_hdr { } else if self.display_hdr {
// HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010. // HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010.
if let Some(conv) = self.hdr_p010_conv.as_ref() { if let Some(conv) = self.hdr_p010_conv.as_ref() {
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
conv.convert( conv.convert(
&self.device, &self.device,
&self.context, &self.context,
&s.srv, src,
out.as_ref().expect("out ring"), out.as_ref().expect("out ring"),
self.width, self.width,
self.height, self.height,
@@ -1762,12 +2047,14 @@ impl IddPushCapturer {
// SDR 4:4:4: pass the BGRA slot through untouched — NVENC ingests full-chroma // SDR 4:4:4: pass the BGRA slot through untouched — NVENC ingests full-chroma
// RGB and CSCs to YUV 4:4:4 itself (per the always-written BT.709 VUI). Plain // RGB and CSCs to YUV 4:4:4 itself (per the always-written BT.709 VUI). Plain
// copy-engine move; the slot releases back to the driver immediately. // copy-engine move; the slot releases back to the driver immediately.
let src = blended.as_ref().map(|(t, _)| t).unwrap_or(&slot_tex);
self.context self.context
.CopyResource(out.as_ref().expect("out ring"), &s.tex); .CopyResource(out.as_ref().expect("out ring"), src);
} else { } else {
// SDR: BGRA slot → NV12 on the VIDEO engine; NVENC takes native NV12, no SM-side CSC. // SDR: BGRA slot → NV12 on the VIDEO engine; NVENC takes native NV12, no SM-side CSC.
if let Some(conv) = self.video_conv.as_ref() { if let Some(conv) = self.video_conv.as_ref() {
conv.convert(&s.tex, out.as_ref().expect("out ring"))?; let src = blended.as_ref().map(|(t, _)| t).unwrap_or(&slot_tex);
conv.convert(src, out.as_ref().expect("out ring"))?;
} }
} }
} }
@@ -1775,13 +2062,20 @@ impl IddPushCapturer {
} }
self.out_idx = (i + 1) % ring_len; self.out_idx = (i + 1) % ring_len;
self.last_seq = seq; self.last_seq = seq;
if fresh {
self.last_slot = Some(slot);
}
if let Some((y, _, cbcr, _)) = pyro_slot.as_ref() { if let Some((y, _, cbcr, _)) = pyro_slot.as_ref() {
self.pyro_last = Some((y.clone(), cbcr.clone())); self.pyro_last = Some((y.clone(), cbcr.clone()));
} else { } else {
self.last_present = Some((out.as_ref().expect("out ring").clone(), pf)); self.last_present = Some((out.as_ref().expect("out ring").clone(), pf));
} }
let now = Instant::now(); let now = Instant::now();
if self.recovering_since.take().is_some() { if regen {
// A regen re-encodes OLD desktop content at a new pointer position — it is not a
// fresh driver frame; feeding the freshness/stall bookkeeping would mask a dead
// driver and pollute stall attribution.
} else if self.recovering_since.take().is_some() {
// A fresh frame resumed → recovered. The recovery gap is self-inflicted (ring // A fresh frame resumed → recovered. The recovery gap is self-inflicted (ring
// recreate, already logged by the recreate path) — reset the stall watch so it // recreate, already logged by the recreate path) — reset the stall watch so it
// doesn't read as a DWM stall. // doesn't read as a DWM stall.
@@ -1853,10 +2147,12 @@ impl IddPushCapturer {
} }
} }
} }
self.last_fresh = now; // feeds the driver-death watch if !regen {
// Build the frame. For PyroWave the encode input is the Y plane self.last_fresh = now; // feeds the driver-death watch
// (`texture`) + the CbCr plane & fence in `pyro`; signal the shared fence }
// after the convert above. SAFETY: on the owning capture/encode thread. // Build the frame. For PyroWave the encode input is the Y plane
// (`texture`) + the CbCr plane & fence in `pyro`; signal the shared fence
// after the convert above. SAFETY: on the owning capture/encode thread.
let (texture, pyro) = if let Some((y, _, cbcr, _)) = pyro_slot { let (texture, pyro) = if let Some((y, _, cbcr, _)) = pyro_slot {
// SAFETY: on the owning capture/encode thread holding the immediate context. // SAFETY: on the owning capture/encode thread holding the immediate context.
let (fence_handle, fence_value) = let (fence_handle, fence_value) =
@@ -1981,7 +2277,82 @@ impl std::fmt::Display for AttachTexFail {
impl std::error::Error for AttachTexFail {} impl std::error::Error for AttachTexFail {}
/// Duplicate `cs`'s section into the driver's WUDFHost and send `IOCTL_SET_CURSOR_CHANNEL`.
/// `true` = the driver adopted it (worker declared per its `cursor_forward_on` state). Shared by
/// the open-time delivery and every RE-delivery (ring recreate / flip NOT_FOUND) — the request is
/// idempotent driver-side (a replaced worker is stopped + joined).
fn deliver_cursor_channel(
broker: &ChannelBroker,
target_id: u32,
cs: &cursor::CursorShared,
send_cursor: &crate::CursorChannelSender,
) -> bool {
// SAFETY: `cs.section_handle()` borrows the section mapping `cs` owns (live across this
// synchronous call); the broker's WUDFHost process handle is live for the broker's lifetime.
let value = match unsafe { broker.dup_into_public(cs.section_handle()) } {
Ok(v) => v,
Err(e) => {
tracing::warn!("cursor section duplication failed (composited cursor stays): {e:#}");
return false;
}
};
let req = pf_driver_proto::control::SetCursorChannelRequest {
target_id,
_pad: 0,
header_handle: value,
};
match send_cursor(&req) {
Ok(()) => {
tracing::info!(
target_id,
"IDD push(host): cursor channel delivered — driver declares the hardware cursor"
);
true
}
Err(e) => {
broker.close_remote_public(value);
tracing::warn!("cursor channel delivery failed (composited cursor stays): {e:#}");
false
}
}
}
impl Capturer for IddPushCapturer { impl Capturer for IddPushCapturer {
fn cursor(&mut self) -> Option<pf_frame::CursorOverlay> {
// A LIVE poller is the sole source — even while it still reports `None` (pre-first-shape):
// falling back to the shm mid-session would interleave two serial namespaces and poison
// the client's shape cache. The shm read only serves a poller that failed to start/died.
if let Some(p) = &self.cursor_poll {
if p.alive() {
return p.read();
}
}
self.cursor_shared.as_mut().and_then(|c| c.read())
}
fn set_cursor_forward(&mut self, on: bool) {
// The composite (capture) model is implemented HOST-side: the driver's hardware cursor
// stays declared for the session's whole life — the only dependable state (there is NO
// working un-declare; see cursor_blend.rs) — keeping every frame pointer-free, and the
// capturer blends the GDI poller's shape into the frame itself. No driver round-trip.
// `composite_forced` (a channel-less session on a sticky-declared target) is pinned ON:
// with no client drawing, un-compositing would erase the pointer entirely.
let composite = (!on && self.cursor_shared.is_some()) || self.composite_forced;
if self.composite_cursor != composite {
self.composite_cursor = composite;
self.last_blend_key = None; // regenerate immediately at the current pointer state
tracing::info!(
composite,
"cursor render model: host compositing {}",
if composite {
"ON (capture model — blending the pointer into frames)"
} else {
"OFF (client draws locally)"
}
);
}
}
fn next_frame(&mut self) -> Result<CapturedFrame> { fn next_frame(&mut self) -> Result<CapturedFrame> {
let deadline = Instant::now() + Duration::from_secs(20); let deadline = Instant::now() + Duration::from_secs(20);
loop { loop {
@@ -105,6 +105,22 @@ impl ChannelBroker {
Ok(out.0 as usize as u64) Ok(out.0 as usize as u64)
} }
/// Duplicate the cursor section into WUDFHost (v5 cursor channel) with the same
/// least-privilege section rights as the frame header. Thin `pub(super)` face over
/// [`dup_into`](Self::dup_into) for the cursor-delivery path in `open_on`.
///
/// # Safety
/// `h` must be a live handle of the current process.
pub(super) unsafe fn dup_into_public(&self, h: HANDLE) -> Result<u64> {
// SAFETY: forwarded contract — `h` is live per this fn's own contract.
unsafe { self.dup_into(h, Some(SECTION_MAP_RW)) }
}
/// [`close_remote`](Self::close_remote) for the cursor-delivery failure path.
pub(super) fn close_remote_public(&self, value: u64) {
self.close_remote(value);
}
/// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE` /// Close a handle VALUE inside the WUDFHost table (the failure-path reaper): `DUPLICATE_CLOSE_SOURCE`
/// with no target closes the source handle regardless of the (ignored) result. /// with no target closes the source handle regardless of the (ignored) result.
fn close_remote(&self, value: u64) { fn close_remote(&self, value: u64) {
@@ -0,0 +1,194 @@
//! Host side of the v5 hardware-cursor channel (remote-desktop-sweep M2c): the capturer creates
//! an unnamed [`CursorShm`] section, delivers it to the pf-vdisplay driver (which declares an
//! IddCx hardware cursor — DWM then EXCLUDES the pointer from the frames we consume), and reads
//! the driver's seqlock publishes here at encode-tick pace, converting them into the same
//! [`pf_frame::CursorOverlay`] the Linux portal path produces — everything downstream (the
//! cursor forwarder, the wire, the client renderer) is shared.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it.
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use pf_driver_proto::cursor::{
CursorShm, CURSOR_MAGIC, CURSOR_SHAPE_BYTES, CURSOR_SHAPE_MAX, CURSOR_SHAPE_OFFSET,
CURSOR_SHM_SIZE, CURSOR_TYPE_MASKED_COLOR,
};
use std::sync::atomic::AtomicU32;
/// The host end of one monitor's cursor channel: the section (we created it — the mapping stays
/// valid for the capturer's life) plus the reader's conversion cache.
pub(super) struct CursorShared {
section: MappedSection,
/// The monitor's desktop origin — IddCx reports positions in DESKTOP coordinates; the
/// overlay wants frame-relative. Fetched at attach (the virtual monitor's placement is
/// stable for the session; a topology change recreates the pipeline anyway).
origin: (i32, i32),
/// Conversion cache: the last `shape_id` whose pixels were converted, and the result.
/// Position-only updates (the common case) reuse it — a refcount bump, no pixel work.
cached_id: u32,
cached: Option<ConvertedShape>,
}
struct ConvertedShape {
rgba: std::sync::Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
}
impl CursorShared {
/// Create + initialize the section (magic stamped, seq even/zero). The returned handle is
/// the section itself (owned by `self`); the caller duplicates it into the WUDFHost.
pub(super) fn create(target_id: u32) -> Result<CursorShared> {
// SAFETY: plain FFI. Unnamed pagefile-backed section, host-lifetime owned; the view is
// mapped once and unmapped never (the capturer's life = the session's life).
let section = unsafe {
let map = CreateFileMappingW(
INVALID_HANDLE_VALUE,
None,
PAGE_READWRITE,
0,
CURSOR_SHM_SIZE as u32,
PCWSTR::null(),
)
.context("CreateFileMapping(cursor)")?;
let map = OwnedHandle::from_raw_handle(map.0 as _);
let view = MapViewOfFile(
HANDLE(map.as_raw_handle()),
FILE_MAP_ALL_ACCESS,
0,
0,
CURSOR_SHM_SIZE,
);
if view.Value.is_null() {
bail!("MapViewOfFile failed for the cursor section");
}
let shm = view.Value.cast::<CursorShm>();
std::ptr::write_bytes(view.Value.cast::<u8>(), 0, CURSOR_SHM_SIZE);
// Magic LAST-ish (the driver validates it at adopt; seq 0 = even = consistent).
std::sync::atomic::fence(Ordering::Release);
(*shm).magic = CURSOR_MAGIC;
MappedSection { handle: map, view }
};
// Desktop origin of this monitor's source — for the desktop→frame coordinate shift.
// SAFETY: `source_desktop_rect` only runs the CCD QueryDisplayConfig FFI over owned
// locals (same call the compose-kick path makes).
let rect = unsafe { pf_win_display::win_display::source_desktop_rect(target_id) };
let origin = rect.map(|(x, y, _w, _h)| (x, y)).unwrap_or((0, 0));
Ok(CursorShared {
section,
origin,
cached_id: 0,
cached: None,
})
}
/// The section handle for the broker's duplication into the WUDFHost.
pub(super) fn section_handle(&self) -> HANDLE {
HANDLE(self.section.handle.as_raw_handle())
}
/// Seqlock-read the driver's latest publish → a frame-relative [`pf_frame::CursorOverlay`].
/// `None` until the first publish lands (or while the pointer has never been seen). A hidden
/// pointer returns `Some` with `visible: false` — the forwarder turns that into the client's
/// relative-mode hint, exactly like the Linux path.
pub(super) fn read(&mut self) -> Option<pf_frame::CursorOverlay> {
let shm = self.section.ptr::<CursorShm>();
// SAFETY: the view spans CURSOR_SHM_SIZE for self's lifetime; seq is 4-aligned in the
// fixed layout (offset 4).
let seq = unsafe { &*std::ptr::addr_of!((*shm).seq).cast::<AtomicU32>() };
for _ in 0..64 {
let s1 = seq.load(Ordering::Acquire);
if s1 == 0 {
return None; // no publish yet
}
if s1 & 1 != 0 {
std::hint::spin_loop();
continue; // writer mid-update
}
// SAFETY: header reads within the mapped view; consistency is validated by the
// seq re-check below (a torn read is discarded and retried).
let hdr = unsafe { std::ptr::read_volatile(shm) };
// Shape pixels: convert only when the OS minted a new shape id.
if hdr.visible != 0 && hdr.shape_id != self.cached_id {
let rows = hdr.height.min(CURSOR_SHAPE_MAX) as usize;
let width = hdr.width.min(CURSOR_SHAPE_MAX) as usize;
let pitch = (hdr.pitch as usize).min(CURSOR_SHAPE_BYTES / rows.max(1));
let mut raw = vec![0u8; rows * pitch];
// SAFETY: the shape region spans CURSOR_SHAPE_BYTES from CURSOR_SHAPE_OFFSET
// inside the mapped view; `rows * pitch` is clamped to it above.
unsafe {
std::ptr::copy_nonoverlapping(
self.section.ptr::<u8>().add(CURSOR_SHAPE_OFFSET),
raw.as_mut_ptr(),
rows * pitch,
);
}
// Discard the copy if the writer raced us mid-shape (seq moved) — retry.
if seq.load(Ordering::Acquire) != s1 {
continue;
}
self.cached = Some(convert_shape(&hdr, &raw, width, rows, pitch));
self.cached_id = hdr.shape_id;
} else if seq.load(Ordering::Acquire) != s1 {
continue;
}
let shape = self.cached.as_ref()?;
return Some(pf_frame::CursorOverlay {
x: hdr.x - self.origin.0,
y: hdr.y - self.origin.1,
w: shape.w,
h: shape.h,
rgba: shape.rgba.clone(),
serial: u64::from(hdr.shape_id),
hot_x: shape.hot_x,
hot_y: shape.hot_y,
visible: hdr.visible != 0,
});
}
None // persistent tearing (writer wedged mid-seq) — skip this tick
}
}
/// Convert the OS's 32-bpp pitch-strided shape rows into the overlay's packed straight RGBA.
/// ALPHA cursors are BGRA with straight per-pixel alpha (swap R↔B). MASKED_COLOR approximates:
/// alpha 0x00 = opaque color pixel; 0xFF = an XOR pixel we cannot honor client-side — rendered
/// as a translucent mid-gray so inversion cursors stay visible instead of vanishing.
fn convert_shape(
hdr: &CursorShm,
raw: &[u8],
width: usize,
rows: usize,
pitch: usize,
) -> ConvertedShape {
let masked = hdr.cursor_type == CURSOR_TYPE_MASKED_COLOR;
let mut rgba = Vec::with_capacity(width * rows * 4);
for y in 0..rows {
let row = &raw[y * pitch..];
for x in 0..width {
let o = x * 4;
if o + 4 > row.len() {
rgba.extend_from_slice(&[0, 0, 0, 0]);
continue;
}
let (b, g, r, a) = (row[o], row[o + 1], row[o + 2], row[o + 3]);
if masked {
if a == 0 {
rgba.extend_from_slice(&[r, g, b, 0xFF]);
} else {
rgba.extend_from_slice(&[0x80, 0x80, 0x80, 0xB4]);
}
} else {
rgba.extend_from_slice(&[r, g, b, a]);
}
}
}
ConvertedShape {
rgba: std::sync::Arc::new(rgba),
w: width as u32,
h: rows as u32,
hot_x: hdr.hot_x.min(width.saturating_sub(1) as u32),
hot_y: hdr.hot_y.min(rows.saturating_sub(1) as u32),
}
}
@@ -0,0 +1,216 @@
//! Host-side cursor compositing for the CAPTURE mouse model (design/remote-desktop-sweep.md §8).
//!
//! Why the host draws it: once a monitor has ever declared an IddCx hardware cursor, DWM will
//! not composite the software cursor back into its frames — there is no un-declare DDI (the
//! empty-caps re-setup is rejected `STATUS_INVALID_PARAMETER`), and a successful same-mode
//! re-commit with the driver's re-declare provably suppressed still leaves the pointer excluded
//! (all observed on-glass, 26100). So the driver keeps its hardware cursor declared for the
//! session's whole life — the state that works — and when the client flips to the capture model
//! the HOST composites the pointer into the frame itself: a slot→scratch copy plus one
//! alpha-blended quad (the GDI poller's full-fidelity shape at its polled position), entirely
//! GPU-side on the capture device, before the normal conversion runs from the scratch.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use windows::core::s;
use windows::Win32::Graphics::Direct3D::D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST;
use windows::Win32::Graphics::Direct3D11::{
ID3D11BlendState, ID3D11Buffer, ID3D11PixelShader, ID3D11SamplerState, ID3D11VertexShader,
D3D11_BIND_CONSTANT_BUFFER, D3D11_BLEND_DESC, D3D11_BLEND_INV_SRC_ALPHA, D3D11_BLEND_ONE,
D3D11_BLEND_OP_ADD, D3D11_BLEND_SRC_ALPHA, D3D11_BUFFER_DESC, D3D11_COMPARISON_NEVER,
D3D11_CPU_ACCESS_WRITE, D3D11_FILTER_MIN_MAG_MIP_LINEAR, D3D11_MAPPED_SUBRESOURCE,
D3D11_MAP_WRITE_DISCARD, D3D11_RENDER_TARGET_BLEND_DESC, D3D11_SAMPLER_DESC,
D3D11_SUBRESOURCE_DATA, D3D11_TEXTURE_ADDRESS_CLAMP, D3D11_USAGE_DYNAMIC, D3D11_VIEWPORT,
};
use windows::Win32::Graphics::Dxgi::Common::DXGI_FORMAT_R8G8B8A8_UNORM;
/// Straight-alpha sample of the cursor bitmap. `linear_scale` = 0 passes sRGB through (SDR
/// ring); non-zero linearizes sRGB→scRGB AND multiplies by the target's SDR-white scale
/// (`sdr_white_level_scale` — 1.0 would put cursor-white at 80 nits, visibly DARKER than the
/// surrounding SDR desktop content DWM composes at the user's SDR-brightness setting).
const CURSOR_PS: &str = r"
Texture2D<float4> tx : register(t0);
SamplerState sm : register(s0);
cbuffer C : register(b0) { float linear_scale; float3 pad; };
float4 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_Target {
float4 c = tx.Sample(sm, uv);
if (linear_scale != 0.0) {
c.rgb = pow(abs(c.rgb), 2.2) * linear_scale;
}
return c;
}
";
/// The cursor-quad blend pass + its shape-texture cache. One per capturer (device-scoped).
pub(super) struct CursorBlendPass {
vs: ID3D11VertexShader,
ps: ID3D11PixelShader,
sampler: ID3D11SamplerState,
blend: ID3D11BlendState,
cbuf: ID3D11Buffer,
cbuf_scale: Option<f32>,
/// The uploaded shape (serial-keyed): SRV + dims in host pixels.
shape: Option<(u64, ID3D11ShaderResourceView, u32, u32)>,
}
impl CursorBlendPass {
pub(super) unsafe fn new(device: &ID3D11Device) -> Result<Self> {
let vsb = crate::dxgi::compile_shader(crate::dxgi::HDR_VS, s!("main"), s!("vs_5_0"))?;
let psb = crate::dxgi::compile_shader(CURSOR_PS, s!("main"), s!("ps_5_0"))?;
let mut vs = None;
device.CreateVertexShader(&vsb, None, Some(&mut vs))?;
let mut ps = None;
device.CreatePixelShader(&psb, None, Some(&mut ps))?;
let sd = D3D11_SAMPLER_DESC {
// LINEAR: the quad is drawn 1:1 in frame pixels, so this only matters at the
// half-texel edges; linear keeps them soft instead of ringing.
Filter: D3D11_FILTER_MIN_MAG_MIP_LINEAR,
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
ComparisonFunc: D3D11_COMPARISON_NEVER,
MaxLOD: f32::MAX,
..Default::default()
};
let mut sampler = None;
device.CreateSamplerState(&sd, Some(&mut sampler))?;
// Straight-alpha over: dst.rgb = src.rgb*a + dst.rgb*(1-a); keep dst alpha.
let mut bd = D3D11_BLEND_DESC::default();
bd.RenderTarget[0] = D3D11_RENDER_TARGET_BLEND_DESC {
BlendEnable: true.into(),
SrcBlend: D3D11_BLEND_SRC_ALPHA,
DestBlend: D3D11_BLEND_INV_SRC_ALPHA,
BlendOp: D3D11_BLEND_OP_ADD,
SrcBlendAlpha: D3D11_BLEND_ONE,
DestBlendAlpha: D3D11_BLEND_ONE,
BlendOpAlpha: D3D11_BLEND_OP_ADD,
RenderTargetWriteMask: 0x0F,
};
let mut blend = None;
device.CreateBlendState(&bd, Some(&mut blend))?;
let cbd = D3D11_BUFFER_DESC {
ByteWidth: 16, // float to_linear + float3 pad
Usage: D3D11_USAGE_DYNAMIC,
BindFlags: D3D11_BIND_CONSTANT_BUFFER.0 as u32,
CPUAccessFlags: D3D11_CPU_ACCESS_WRITE.0 as u32,
..Default::default()
};
let mut cbuf = None;
device.CreateBuffer(&cbd, None, Some(&mut cbuf))?;
Ok(Self {
vs: vs.context("cursor blend vs")?,
ps: ps.context("cursor blend ps")?,
sampler: sampler.context("cursor blend sampler")?,
blend: blend.context("cursor blend state")?,
cbuf: cbuf.context("cursor blend cbuf")?,
cbuf_scale: None,
shape: None,
})
}
/// Upload `ov`'s bitmap if its serial is new; reuse the cached SRV otherwise.
unsafe fn ensure_shape(
&mut self,
device: &ID3D11Device,
ov: &pf_frame::CursorOverlay,
) -> Result<()> {
if self.shape.as_ref().is_some_and(|(s, ..)| *s == ov.serial) {
return Ok(());
}
if ov.rgba.len() < (ov.w as usize) * (ov.h as usize) * 4 || ov.w == 0 || ov.h == 0 {
bail!("malformed cursor overlay ({}x{})", ov.w, ov.h);
}
let desc = D3D11_TEXTURE2D_DESC {
Width: ov.w,
Height: ov.h,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_R8G8B8A8_UNORM,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: D3D11_BIND_SHADER_RESOURCE.0 as u32,
..Default::default()
};
let init = D3D11_SUBRESOURCE_DATA {
pSysMem: ov.rgba.as_ptr().cast(),
SysMemPitch: ov.w * 4,
SysMemSlicePitch: 0,
};
let mut tex: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, Some(&init), Some(&mut tex))
.context("CreateTexture2D(cursor shape)")?;
let tex = tex.context("null cursor shape texture")?;
let mut srv: Option<ID3D11ShaderResourceView> = None;
device
.CreateShaderResourceView(&tex, None, Some(&mut srv))
.context("CreateShaderResourceView(cursor shape)")?;
self.shape = Some((ov.serial, srv.context("null cursor shape srv")?, ov.w, ov.h));
Ok(())
}
/// Alpha-blend `ov` onto `dst` (frame-sized, RENDER_TARGET-capable — the blend scratch).
/// `linear_scale`: 0 = SDR passthrough; non-zero = the frame is FP16 scRGB (HDR
/// composition) — linearize and scale to the target's SDR white. The quad is placed purely
/// via the viewport (the fullscreen-triangle VS fills whatever viewport is set), clipped by
/// the target automatically.
pub(super) unsafe fn blend(
&mut self,
device: &ID3D11Device,
ctx: &ID3D11DeviceContext,
dst: &ID3D11Texture2D,
ov: &pf_frame::CursorOverlay,
linear_scale: f32,
) -> Result<()> {
self.ensure_shape(device, ov)?;
let (_, srv, w, h) = self.shape.as_ref().expect("shape just ensured");
if self.cbuf_scale != Some(linear_scale) {
let cb: [f32; 4] = [linear_scale, 0.0, 0.0, 0.0];
let mut mapped = D3D11_MAPPED_SUBRESOURCE::default();
if ctx
.Map(&self.cbuf, 0, D3D11_MAP_WRITE_DISCARD, 0, Some(&mut mapped))
.is_ok()
{
std::ptr::copy_nonoverlapping(cb.as_ptr(), mapped.pData as *mut f32, cb.len());
ctx.Unmap(&self.cbuf, 0);
}
self.cbuf_scale = Some(linear_scale);
}
let mut rtv: Option<ID3D11RenderTargetView> = None;
device
.CreateRenderTargetView(dst, None, Some(&mut rtv))
.context("CreateRenderTargetView(cursor blend scratch)")?;
let rtv = rtv.context("null cursor blend rtv")?;
ctx.OMSetRenderTargets(Some(&[Some(rtv)]), None);
ctx.OMSetBlendState(&self.blend, None, 0xffff_ffff);
ctx.VSSetShader(&self.vs, None);
ctx.PSSetShader(&self.ps, None);
ctx.PSSetShaderResources(0, Some(&[Some(srv.clone())]));
ctx.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
ctx.PSSetConstantBuffers(0, Some(&[Some(self.cbuf.clone())]));
ctx.IASetInputLayout(None);
ctx.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
// Placement IS the viewport: the VS fills it, the OS clips it to the target.
let vp = D3D11_VIEWPORT {
TopLeftX: ov.x as f32,
TopLeftY: ov.y as f32,
Width: *w as f32,
Height: *h as f32,
MinDepth: 0.0,
MaxDepth: 1.0,
};
ctx.RSSetViewports(Some(&[vp]));
ctx.Draw(3, 0);
// Unbind so the scratch can be bound as a conversion INPUT without a hazard warning.
ctx.OMSetRenderTargets(None, None);
let none_srv: [Option<ID3D11ShaderResourceView>; 1] = [None];
ctx.PSSetShaderResources(0, Some(&none_srv));
Ok(())
}
}
@@ -0,0 +1,445 @@
//! GDI cursor poller — the Windows cursor-SHAPE source for the cursor-forward channel
//! (design/remote-desktop-sweep.md §8, the M2c redesign).
//!
//! Why not the IddCx hardware-cursor query (the v5 `CursorShm` path, now the fallback): it is
//! alpha-only BY DESIGN — `IDDCX_CURSOR_SHAPE_TYPE` has no monochrome value, the OS pre-converts
//! monochrome to masked-color (IDDCX_CURSOR_CAPS docs), and masked-color delivery is dead code on
//! modern builds (proven on-glass at every `ColorXorCursorSupport` level; no public evidence of a
//! MASKED_COLOR delivery anywhere). The driver keeps its hardware cursor declared at XOR FULL
//! purely so DWM EXCLUDES every cursor type from the IDD frame.
//!
//! Why not DXGI Desktop Duplication `GetFramePointerShape`: its `PointerPosition.Visible` goes
//! stale when the cursor moves only via injected input on current Win11 (Sunshine #5293 — exactly
//! a Punktfunk session's topology), it burns one of the session's four duplication slots, and its
//! per-output metadata on IDD monitors has conflicting field reports. The GDI path below is the
//! metadata-forwarding-remote-desktop pattern (RustDesk, WebRTC/Chrome Remote Desktop, OBS): the
//! cursor is per-session global state in win32k, readable cross-process, and `CURSOR_SHOWING` is
//! the logical visibility — immune to all of the above.
//!
//! Works because the capture host runs as SYSTEM *inside the interactive session* on
//! `winsta0\default` (the service supervisor retargets the token — `windows/service.rs`
//! `spawn_host`), so the poller thread sees the session's cursor directly; no helper process.
// Every `unsafe` block in this file carries a `// SAFETY:` proof; enforce it (unsafe-proof program).
#![deny(clippy::undocumented_unsafe_blocks)]
use super::*;
use windows::Win32::Graphics::Gdi::{
DeleteObject, GetDC, GetDIBits, GetObjectW, ReleaseDC, BITMAP, BITMAPINFO, BITMAPINFOHEADER,
BI_RGB, DIB_RGB_COLORS, HBITMAP, HDC,
};
use windows::Win32::System::StationsAndDesktops::{
CloseDesktop, OpenInputDesktop, SetThreadDesktop, DESKTOP_ACCESS_FLAGS, DESKTOP_CONTROL_FLAGS,
HDESK,
};
use windows::Win32::UI::HiDpi::{
SetThreadDpiAwarenessContext, DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2,
};
use windows::Win32::UI::WindowsAndMessaging::{
CopyIcon, DestroyIcon, GetCursorInfo, GetIconInfo, CURSORINFO, HICON, ICONINFO,
};
/// `CURSORINFO.flags` bits (WindowsAndMessaging): the pointer is logically shown /
/// touch-or-pen-suppressed. Named locally so the visibility rule below reads as the docs do.
const CURSOR_SHOWING: u32 = 0x1;
const CURSOR_SUPPRESSED: u32 = 0x2;
/// A converted shape: the cache the per-tick overlay is assembled from. `rgba` is `Arc` so the
/// slot publish (and every downstream frame attach) is a refcount bump.
struct Shape {
rgba: std::sync::Arc<Vec<u8>>,
w: u32,
h: u32,
hot_x: u32,
hot_y: u32,
serial: u64,
}
/// Off-thread GDI cursor poller. Samples `GetCursorInfo` at ~60 Hz, rasterises the `HCURSOR` only
/// when its handle value changes, and publishes a ready [`pf_frame::CursorOverlay`] snapshot; the
/// capture thread's per-tick cost is one uncontended mutex read + an `Arc` clone
/// (same split as [`DescriptorPoller`], and for the same reason: user32/gdi32 calls have no place
/// on the capture/encode thread).
pub(super) struct CursorPoller {
slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>>,
stop: Arc<AtomicBool>,
thread: Option<std::thread::JoinHandle<()>>,
}
impl CursorPoller {
/// ~250 Hz: the polled position is ALSO the composite-blend position (capture model), so
/// it must out-pace the fastest session — at 16 ms a 240 fps stream re-used a stale
/// position for ~4 consecutive frames and the composited pointer visibly stuttered
/// against the video. A tick is one `GetCursorInfo` syscall (rasterisation only on shape
/// change), so 250 Hz is still negligible CPU.
const INTERVAL: Duration = Duration::from_millis(4);
/// Unconditional input-desktop reattach cadence — catches secure-desktop (UAC/lock) switches
/// without a failure signal (`GetCursorInfo` on a stale desktop *succeeds* with stale data).
const REATTACH: Duration = Duration::from_secs(2);
/// Spawn the poller for the virtual display `target_id`. `rect` = the target's desktop rect
/// (`source_desktop_rect` order: x, y, w, h) — cursor positions are desktop-global; the
/// overlay wants frame-relative, and a pointer outside the rect reports `visible: false`
/// (per-output semantics, matching the driver shm path and the Linux portal).
pub(super) fn spawn(target_id: u32, rect: (i32, i32, i32, i32)) -> Self {
let slot: Arc<Mutex<Option<pf_frame::CursorOverlay>>> = Arc::new(Mutex::new(None));
let stop = Arc::new(AtomicBool::new(false));
let (slot_t, stop_t) = (slot.clone(), stop.clone());
let thread = std::thread::Builder::new()
.name("pf-cursor-poll".into())
.spawn(move || run(target_id, rect, &slot_t, &stop_t))
.ok();
if thread.is_none() {
tracing::warn!("cursor poller thread spawn failed — cursor falls back to driver shm");
}
Self { slot, stop, thread }
}
/// The latest overlay snapshot (`None` until the first successful shape rasterisation).
pub(super) fn read(&self) -> Option<pf_frame::CursorOverlay> {
self.slot.lock().unwrap_or_else(|p| p.into_inner()).clone()
}
/// Whether the worker thread is (still) alive — `false` degrades the capturer to the shm read.
pub(super) fn alive(&self) -> bool {
self.thread.as_ref().is_some_and(|t| !t.is_finished())
}
}
impl Drop for CursorPoller {
fn drop(&mut self) {
self.stop.store(true, Ordering::Relaxed);
if let Some(t) = self.thread.take() {
let _ = t.join(); // worker sleeps ≤ INTERVAL — a bounded join
}
}
}
/// The poll loop. Owns the thread's input-desktop binding and the shape cache.
fn run(
target_id: u32,
rect: (i32, i32, i32, i32),
slot: &Mutex<Option<pf_frame::CursorOverlay>>,
stop: &AtomicBool,
) {
// Physical-pixel coordinates on this thread regardless of the process's DPI awareness:
// `rect` comes from CCD (always physical), and a DPI-virtualized `GetCursorInfo` position
// would land in the wrong frame pixel on any scaled display. Thread-scoped, so the rest of
// the host is untouched.
// SAFETY: takes and returns only a by-value context handle; affects this thread only.
let _ = unsafe { SetThreadDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2) };
let mut desktop = DesktopBinding::default();
desktop.reattach(); // best-effort: already on winsta0\default if this fails
let mut last_attach = Instant::now();
let mut shape: Option<Shape> = None;
let mut cached_handle: isize = 0;
let mut failed_handle: isize = 0; // don't re-rasterise a failing handle every tick
let mut serial: u64 = 0;
let mut logged_live = false;
while !stop.load(Ordering::Relaxed) {
std::thread::sleep(CursorPoller::INTERVAL);
if last_attach.elapsed() >= CursorPoller::REATTACH {
last_attach = Instant::now();
desktop.reattach();
}
let mut ci = CURSORINFO {
cbSize: std::mem::size_of::<CURSORINFO>() as u32,
..Default::default()
};
// SAFETY: `ci` is a live, correctly-sized out-param for this synchronous call; no pointer
// escapes it.
if unsafe { GetCursorInfo(&mut ci) }.is_err() {
// Desktop went away under us (secure-desktop switch mid-call) — rebind and retry
// next tick; the slot keeps its last snapshot meanwhile.
desktop.reattach();
last_attach = Instant::now();
continue;
}
let flags = ci.flags.0;
let showing = flags & CURSOR_SHOWING != 0 && flags & CURSOR_SUPPRESSED == 0;
// Rasterise on handle change only (position-only ticks are a header update). Hidden
// cursors keep the cached shape — the forwarder's hidden-but-known contract needs the
// bitmap to have been seen. v1: animated cursors publish their first frame (the OBS
// behavior); frame cycling via DrawIconEx istep is a known follow-up.
let handle = ci.hCursor.0 as isize;
if showing && handle != 0 && handle != cached_handle && handle != failed_handle {
match rasterize(ci.hCursor) {
Some((rgba, w, h, hot_x, hot_y)) => {
serial += 1;
shape = Some(Shape {
rgba: std::sync::Arc::new(rgba),
w,
h,
hot_x,
hot_y,
serial,
});
cached_handle = handle;
failed_handle = 0;
if !logged_live {
logged_live = true;
tracing::info!(
target_id,
"cursor poller live — GDI shape source publishing (serial 1: {w}x{h})"
);
}
}
None => {
// The owning app may have destroyed the cursor mid-read; keep the previous
// shape and don't hammer this handle again until it changes.
failed_handle = handle;
}
}
}
let overlay = shape.as_ref().map(|s| {
let (px, py) = (ci.ptScreenPos.x - rect.0, ci.ptScreenPos.y - rect.1);
let in_rect = px >= 0 && py >= 0 && px < rect.2 && py < rect.3;
pf_frame::CursorOverlay {
// Overlay x/y = bitmap top-left (reported position hotspot), frame pixels.
x: px - s.hot_x as i32,
y: py - s.hot_y as i32,
w: s.w,
h: s.h,
rgba: s.rgba.clone(),
serial: s.serial,
hot_x: s.hot_x,
hot_y: s.hot_y,
visible: showing && in_rect,
}
});
*slot.lock().unwrap_or_else(|p| p.into_inner()) = overlay;
}
}
/// The thread's owned input-desktop handle — the [`SendInputInjector`] reattach model
/// (`pf-inject` sendinput.rs): keep the current binding, swap on demand, close exactly once.
#[derive(Default)]
struct DesktopBinding(Option<HDESK>);
impl DesktopBinding {
fn reattach(&mut self) {
const GENERIC_ALL: u32 = 0x1000_0000;
// SAFETY: `OpenInputDesktop`/`SetThreadDesktop`/`CloseDesktop` take only by-value args.
// `OpenInputDesktop` yields an owned `HDESK` only on `Ok`; it is either installed (and the
// previously-owned handle closed exactly once) or closed on failure — no handle is leaked
// or used after close. `SetThreadDesktop` rebinds only this calling thread (which owns
// no windows/hooks, so the rebind cannot fail on that account).
unsafe {
match OpenInputDesktop(
DESKTOP_CONTROL_FLAGS(0),
false,
DESKTOP_ACCESS_FLAGS(GENERIC_ALL),
) {
Ok(h) => {
if SetThreadDesktop(h).is_ok() {
if let Some(old) = self.0.replace(h) {
let _ = CloseDesktop(old);
}
} else {
let _ = CloseDesktop(h);
}
}
Err(_) => { /* not privileged for this desktop; stay put */ }
}
}
}
}
impl Drop for DesktopBinding {
fn drop(&mut self) {
if let Some(h) = self.0.take() {
// SAFETY: `h` is our owned desktop handle, closed exactly once here.
let _ = unsafe { CloseDesktop(h) };
}
}
}
/// Rasterise `hcursor` to straight-alpha RGBA: `(rgba, w, h, hot_x, hot_y)`. `None` on any
/// failure (caller keeps the previous shape).
fn rasterize(hcursor: windows::Win32::UI::WindowsAndMessaging::HCURSOR) -> RasterOut {
// CopyIcon first: the owning process can destroy its HCURSOR between GetCursorInfo and the
// reads below; the copy is ours (the OBS/WebRTC guard).
// SAFETY: `HICON(hcursor.0)` reinterprets the cursor handle as an icon handle (cursors ARE
// icons in user32); CopyIcon yields an owned HICON we destroy below.
let Ok(icon) = (unsafe { CopyIcon(HICON(hcursor.0)) }) else {
return None;
};
let mut ii = ICONINFO::default();
// SAFETY: `ii` is a live out-param. On Ok it hands us COPIES of the mask/color bitmaps —
// both deleted below (GDI-handle leak otherwise).
let got = unsafe { GetIconInfo(icon, &mut ii) };
let out = if got.is_ok() { convert(&ii) } else { None };
// SAFETY: deleting the two bitmap copies GetIconInfo returned (null-safe: DeleteObject on a
// null HGDIOBJ fails harmlessly) and the icon copy — each exactly once.
unsafe {
let _ = DeleteObject(ii.hbmColor.into());
let _ = DeleteObject(ii.hbmMask.into());
let _ = DestroyIcon(icon);
}
out.map(|(rgba, w, h)| {
let hot_x = ii.xHotspot.min(w.saturating_sub(1));
let hot_y = ii.yHotspot.min(h.saturating_sub(1));
(rgba, w, h, hot_x, hot_y)
})
}
type RasterOut = Option<(Vec<u8>, u32, u32, u32, u32)>;
/// Convert the ICONINFO bitmaps to straight RGBA. Two families:
/// - color (`hbmColor` set): 32bpp BGRA; if the alpha channel is entirely empty (old-style
/// cursors) the AND mask supplies it (mask bit 1 = transparent).
/// - monochrome (`hbmColor` null): `hbmMask` is DOUBLE height — AND plane over XOR plane, the
/// WebRTC truth table: (0,0) black, (0,1) white, (1,0) transparent, (1,1) invert. Invert
/// pixels — unrepresentable in straight alpha — become opaque black with a white outline
/// grown into adjacent transparency (the WebRTC approximation; keeps the I-beam legible on
/// any background, which the old translucent-gray stand-in did not).
fn convert(ii: &ICONINFO) -> Option<(Vec<u8>, u32, u32)> {
// SAFETY: GetDC(None) yields the screen DC, released below on every path; it is only used
// as the GetDIBits reference DC.
let dc = unsafe { GetDC(None) };
let result = (|| {
if !ii.hbmColor.is_invalid() {
let color = read_bitmap_32(dc, ii.hbmColor)?;
let (w, h) = (color.w as u32, color.h as u32);
let mut rgba = bgra_to_rgba(&color.bgra);
if rgba.chunks_exact(4).all(|p| p[3] == 0) {
// Alpha-less color cursor: transparency lives in the AND mask.
let mask = read_bitmap_32(dc, ii.hbmMask)?;
if mask.w != color.w || mask.h < color.h {
return None;
}
for (px, m) in rgba.chunks_exact_mut(4).zip(mask.bgra.chunks_exact(4)) {
px[3] = if m[0] != 0 { 0 } else { 0xFF }; // mask white (AND=1) = transparent
}
}
Some((rgba, w, h))
} else {
let mask = read_bitmap_32(dc, ii.hbmMask)?;
if mask.h < 2 || mask.h % 2 != 0 {
return None;
}
let (w, h) = (mask.w as usize, (mask.h / 2) as usize);
let row = w * 4;
let (and_plane, xor_plane) = mask.bgra.split_at(h * row);
let mut rgba = vec![0u8; w * h * 4];
let mut invert = vec![false; w * h];
for i in 0..w * h {
let (a, x) = (and_plane[i * 4] != 0, xor_plane[i * 4] != 0);
let px = &mut rgba[i * 4..i * 4 + 4];
match (a, x) {
(false, false) => px.copy_from_slice(&[0, 0, 0, 0xFF]),
(false, true) => px.copy_from_slice(&[0xFF, 0xFF, 0xFF, 0xFF]),
(true, false) => {} // transparent (already zeroed)
(true, true) => {
px.copy_from_slice(&[0, 0, 0, 0xFF]);
invert[i] = true;
}
}
}
// White outline around invert regions so the (now black) shape survives dark
// backgrounds: any transparent 8-neighbor of an invert pixel turns opaque white.
for y in 0..h as i32 {
for x in 0..w as i32 {
if !invert[(y * w as i32 + x) as usize] {
continue;
}
for (dx, dy) in NEIGHBORS {
let (nx, ny) = (x + dx, y + dy);
if nx < 0 || ny < 0 || nx >= w as i32 || ny >= h as i32 {
continue;
}
let o = (ny * w as i32 + nx) as usize * 4;
if rgba[o + 3] == 0 {
rgba[o..o + 4].copy_from_slice(&[0xFF, 0xFF, 0xFF, 0xFF]);
}
}
}
}
Some((rgba, w as u32, h as u32))
}
})();
// SAFETY: releasing the screen DC obtained above, exactly once.
unsafe {
ReleaseDC(None, dc);
}
result
}
const NEIGHBORS: [(i32, i32); 8] = [
(-1, -1),
(0, -1),
(1, -1),
(-1, 0),
(1, 0),
(-1, 1),
(0, 1),
(1, 1),
];
struct RawBitmap {
w: i32,
h: i32,
/// 32bpp top-down BGRA rows, `w*h*4` (monochrome sources arrive expanded: 0x00/0xFF channels).
bgra: Vec<u8>,
}
/// Read any GDI bitmap as 32bpp top-down via `GetDIBits` (which performs the 1bpp→32bpp
/// expansion for the mask planes).
fn read_bitmap_32(dc: HDC, hbm: HBITMAP) -> Option<RawBitmap> {
let mut bm = BITMAP::default();
// SAFETY: `bm` is a live out-param sized exactly as passed; GetObjectW only writes into it.
let n = unsafe {
GetObjectW(
hbm.into(),
std::mem::size_of::<BITMAP>() as i32,
Some((&mut bm as *mut BITMAP).cast()),
)
};
if n == 0 || bm.bmWidth <= 0 || bm.bmHeight <= 0 || bm.bmWidth > 512 || bm.bmHeight > 1024 {
return None; // 512/1024: sanity caps (256² is the wire max; XL accessibility ≤ that)
}
let (w, h) = (bm.bmWidth, bm.bmHeight);
let mut info = BITMAPINFO {
bmiHeader: BITMAPINFOHEADER {
biSize: std::mem::size_of::<BITMAPINFOHEADER>() as u32,
biWidth: w,
biHeight: -h, // top-down
biPlanes: 1,
biBitCount: 32,
biCompression: BI_RGB.0,
..Default::default()
},
..Default::default()
};
let mut buf = vec![0u8; (w as usize) * (h as usize) * 4];
// SAFETY: `buf` spans exactly `h` rows of `w` 32bpp pixels as described by `info`; both are
// live locals for this synchronous call, `hbm` is a live bitmap not selected into any DC
// (fresh GetIconInfo copies).
let rows = unsafe {
GetDIBits(
dc,
hbm,
0,
h as u32,
Some(buf.as_mut_ptr().cast()),
&mut info,
DIB_RGB_COLORS,
)
};
(rows != 0).then_some(RawBitmap { w, h, bgra: buf })
}
fn bgra_to_rgba(bgra: &[u8]) -> Vec<u8> {
let mut out = bgra.to_vec();
for px in out.chunks_exact_mut(4) {
px.swap(0, 2);
}
out
}
+10 -4
View File
@@ -876,10 +876,16 @@ impl Worker {
); );
return; return;
}; };
let pref = self let pref = match self.pad_info(id) {
.pad_info(id) // Steam Input's virtual pad standing in front of the Deck's built-in controls (the
.map(|p| p.pref) // only-pad-forwarded case, [`Self::forwarded_ids`]): declare the DECK kind, not the
.unwrap_or(GamepadPref::Xbox360); // wrapper's Xbox 360 identity. [`Self::auto_pref`] already resolves the SESSION
// default this way, but a current host honors the per-pad arrival over the session
// default — so without this the host builds an X-Box 360 pad on a real Deck.
Some(p) if p.steam_virtual && is_steam_deck() => GamepadPref::SteamDeck,
Some(p) => p.pref,
None => GamepadPref::Xbox360,
};
match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) { match self.subsystem.open(sdl3::sys::joystick::SDL_JoystickID(id)) {
Ok(pad) => { Ok(pad) => {
let mut slot = Slot::new(id, index, pref, pad); let mut slot = Slot::new(id, index, pref, pad);
+12
View File
@@ -44,6 +44,13 @@ pub struct SessionParams {
/// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The /// Share the clipboard with this host (the per-host `KnownHost::clipboard_sync`). The
/// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`. /// bridge additionally needs the host to advertise `HOST_CAP_CLIPBOARD`.
pub clipboard: bool, pub clipboard: bool,
/// Advertise `quic::CLIENT_CAP_CURSOR`: this embedder renders the host cursor locally
/// (the presenter's cursor channel, design/remote-desktop-sweep.md M2), so the host may
/// stop compositing the pointer into the video. Only set when the embedder actually
/// draws it (the SDL presenter in desktop mouse mode) — a session that advertises it
/// without rendering streams with NO visible cursor. The host answers `HOST_CAP_CURSOR`
/// when its capture can forward (Linux portal, not gamescope/Windows).
pub cursor_forward: bool,
/// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see /// Video decoder preference (Settings; `PUNKTFUNK_DECODER` overrides — see
/// `video::Decoder::new`). /// `video::Decoder::new`).
pub decoder: String, pub decoder: String,
@@ -255,6 +262,11 @@ fn pump(
// This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an // This display's HDR volume → the host's virtual-display EDID. The env hatch wins so an
// A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600). // A/B run can pin an exact peak (PUNKTFUNK_CLIENT_PEAK_NITS=600).
punktfunk_core::client::display_hdr_env_override().or(params.display_hdr), punktfunk_core::client::display_hdr_env_override().or(params.display_hdr),
if params.cursor_forward {
punktfunk_core::quic::CLIENT_CAP_CURSOR
} else {
0
},
params.launch.clone(), params.launch.clone(),
params.pin, params.pin,
Some(params.identity), Some(params.identity),
+62
View File
@@ -325,6 +325,11 @@ pub fn connect_reject_message(reason: punktfunk_core::reject::RejectReason) -> S
"Client and host versions don't match — update both to the same release.".into() "Client and host versions don't match — update both to the same release.".into()
} }
R::Busy => "The host is busy with another session.".into(), R::Busy => "The host is busy with another session.".into(),
R::SetupFailed => {
"The host accepted the connection but couldn't start the stream — the host's log \
(web console Log) has the cause."
.into()
}
} }
} }
@@ -456,6 +461,48 @@ impl TouchMode {
} }
} }
/// How a physical mouse drives the host — the desktop-sweep mouse model
/// (design/remote-desktop-sweep.md M1). Stored stringly in [`Settings::mouse_mode`] so the
/// file stays readable; parsed with [`MouseMode::from_name`].
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum MouseMode {
/// Pointer lock (relative deltas, hidden cursor) — the game model, and the default:
/// the only cursor you see is the host's.
Capture,
/// Absolute pointer, uncaptured: the cursor enters and leaves the stream freely and
/// motion goes on the wire as absolute positions through the letterbox. The remote
/// desktop model. Requires a host injector with absolute support (not gamescope).
Desktop,
}
impl MouseMode {
/// Cycle/picker order (also the settings pickers' option order).
pub const ALL: [MouseMode; 2] = [MouseMode::Capture, MouseMode::Desktop];
/// Parse the persisted name, defaulting to `Capture` for unset/unknown values.
pub fn from_name(s: &str) -> MouseMode {
match s {
"desktop" => MouseMode::Desktop,
_ => MouseMode::Capture,
}
}
/// The persisted name (the inverse of [`from_name`](Self::from_name)).
pub fn as_name(self) -> &'static str {
match self {
MouseMode::Capture => "capture",
MouseMode::Desktop => "desktop",
}
}
pub fn label(self) -> &'static str {
match self {
MouseMode::Capture => "Capture (games)",
MouseMode::Desktop => "Desktop (absolute)",
}
}
}
/// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file /// App settings, persisted as JSON. Stringly-typed gamepad/compositor prefs so the file
/// stays readable; parsed with `*Pref::from_name` at connect time. /// stays readable; parsed with `*Pref::from_name` at connect time.
#[derive(Clone, Serialize, Deserialize)] #[derive(Clone, Serialize, Deserialize)]
@@ -490,6 +537,12 @@ pub struct Settings {
/// stores load as trackpad. /// stores load as trackpad.
#[serde(default = "default_touch_mode")] #[serde(default = "default_touch_mode")]
pub touch_mode: String, pub touch_mode: String,
/// How a physical mouse drives the host: a [`MouseMode`] name — `"capture"` (default,
/// pointer lock + relative) or `"desktop"` (uncaptured absolute pointer). Read at
/// connect via [`Settings::mouse_mode`]. `default` so pre-existing stores load as
/// capture — today's behavior.
#[serde(default = "default_mouse_mode")]
pub mouse_mode: String,
/// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured. /// Grab compositor shortcuts (Alt+Tab, Super…) while input is captured.
pub inhibit_shortcuts: bool, pub inhibit_shortcuts: bool,
/// Stream the default microphone to the host's virtual mic source. /// Stream the default microphone to the host's virtual mic source.
@@ -577,6 +630,10 @@ fn default_touch_mode() -> String {
"trackpad".into() "trackpad".into()
} }
fn default_mouse_mode() -> String {
"capture".into()
}
fn default_true() -> bool { fn default_true() -> bool {
true true
} }
@@ -604,6 +661,10 @@ impl Settings {
TouchMode::from_name(&self.touch_mode) TouchMode::from_name(&self.touch_mode)
} }
pub fn mouse_mode(&self) -> MouseMode {
MouseMode::from_name(&self.mouse_mode)
}
/// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto). /// The `codec` setting as a `quic::CODEC_*` preference bit (`0` = auto).
pub fn preferred_codec(&self) -> u8 { pub fn preferred_codec(&self) -> u8 {
match self.codec.as_str() { match self.codec.as_str() {
@@ -631,6 +692,7 @@ impl Default for Settings {
forward_pad: String::new(), forward_pad: String::new(),
compositor: "auto".into(), compositor: "auto".into(),
touch_mode: "trackpad".into(), touch_mode: "trackpad".into(),
mouse_mode: "capture".into(),
inhibit_shortcuts: true, inhibit_shortcuts: true,
mic_enabled: false, mic_enabled: false,
audio_channels: 2, audio_channels: 2,
+42 -2
View File
@@ -10,7 +10,7 @@ use crate::screens::{Ctx, Outbox};
use crate::theme::{Fonts, DIM, W}; use crate::theme::{Fonts, DIM, W};
use crate::widgets::{ListMsg, MenuList, RowSpec}; use crate::widgets::{ListMsg, MenuList, RowSpec};
use pf_client_core::gamepad::{MenuEvent, MenuPulse}; use pf_client_core::gamepad::{MenuEvent, MenuPulse};
use pf_client_core::trust::{StatsVerbosity, TouchMode}; use pf_client_core::trust::{MouseMode, StatsVerbosity, TouchMode};
use skia_safe::{Canvas, Rect}; use skia_safe::{Canvas, Rect};
/// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale /// Stable row identity — adjust/activate dispatch by id so nothing acts on a stale
@@ -29,10 +29,11 @@ enum RowId {
Pad, Pad,
PadType, PadType,
Touch, Touch,
Mouse,
Stats, Stats,
} }
const ROWS: [RowId; 13] = [ const ROWS: [RowId; 14] = [
RowId::Resolution, RowId::Resolution,
RowId::Refresh, RowId::Refresh,
RowId::Bitrate, RowId::Bitrate,
@@ -45,6 +46,7 @@ const ROWS: [RowId; 13] = [
RowId::Pad, RowId::Pad,
RowId::PadType, RowId::PadType,
RowId::Touch, RowId::Touch,
RowId::Mouse,
RowId::Stats, RowId::Stats,
]; ];
@@ -251,6 +253,7 @@ fn row_spec(id: RowId, ctx: &Ctx) -> RowSpec {
"Touch mode", "Touch mode",
s.touch_mode().label().into(), s.touch_mode().label().into(),
), ),
RowId::Mouse => (None, "Mouse mode", s.mouse_mode().label().into()),
RowId::Stats => ( RowId::Stats => (
Some("Interface"), Some("Interface"),
"Statistics overlay", "Statistics overlay",
@@ -292,6 +295,11 @@ fn detail(id: RowId) -> &'static str {
"How the touchscreen drives the host: Trackpad (relative cursor), \ "How the touchscreen drives the host: Trackpad (relative cursor), \
Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)." Direct pointer (cursor jumps to your finger), or Touch passthrough (raw contacts)."
} }
RowId::Mouse => {
"How a physical mouse drives the host: Capture locks the pointer (relative, \
for games), Desktop leaves it free and sends absolute positions. \
Ctrl+Alt+Shift+M switches live while streaming."
}
RowId::Stats => { RowId::Stats => {
"How much the overlay shows: Compact (one line) → Normal → Detailed. \ "How much the overlay shows: Compact (one line) → Normal → Detailed. \
Ctrl+Alt+Shift+S cycles it live while streaming." Ctrl+Alt+Shift+S cycles it live while streaming."
@@ -367,6 +375,11 @@ fn adjust(id: RowId, delta: i32, wrap: bool, ctx: &mut Ctx) -> bool {
step_option(cur, TouchMode::ALL.len(), delta, wrap) step_option(cur, TouchMode::ALL.len(), delta, wrap)
.map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string()) .map(|i| s.touch_mode = TouchMode::ALL[i].as_name().to_string())
} }
RowId::Mouse => {
let cur = MouseMode::ALL.iter().position(|m| *m == s.mouse_mode());
step_option(cur, MouseMode::ALL.len(), delta, wrap)
.map(|i| s.mouse_mode = MouseMode::ALL[i].as_name().to_string())
}
RowId::Stats => { RowId::Stats => {
let cur = StatsVerbosity::ALL let cur = StatsVerbosity::ALL
.iter() .iter()
@@ -510,6 +523,33 @@ mod tests {
assert_eq!(ctx.settings.touch_mode, "trackpad"); assert_eq!(ctx.settings.touch_mode, "trackpad");
} }
#[test]
fn mouse_mode_steps_and_wraps() {
let (mut settings, pads) = ctx_parts();
assert_eq!(settings.mouse_mode, "capture");
let library = crate::library::LibraryShared::default();
let mut ctx = Ctx {
hosts: &[],
library: &library,
settings: &mut settings,
pads: &pads,
deck: false,
device_name: "t",
t: 0.0,
};
// Capture → Desktop, then a step past the end is a boundary.
assert!(
!adjust(RowId::Mouse, -1, false, &mut ctx),
"already first = thud"
);
assert!(adjust(RowId::Mouse, 1, false, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "desktop");
assert!(!adjust(RowId::Mouse, 1, false, &mut ctx), "last = thud");
// A wraps back to the first.
assert!(adjust(RowId::Mouse, 1, true, &mut ctx));
assert_eq!(ctx.settings.mouse_mode, "capture");
}
#[test] #[test]
fn unknown_value_snaps_to_first() { fn unknown_value_snaps_to_first() {
let (mut settings, pads) = ctx_parts(); let (mut settings, pads) = ctx_parts();
+211 -9
View File
@@ -66,7 +66,30 @@ pub const fn interface_guid_fields() -> (u32, u16, u16, [u8; 8]) {
/// too ([`MIN_DRIVER_PROTOCOL_VERSION`]) and simply falls back to the re-arrival resize against it; /// too ([`MIN_DRIVER_PROTOCOL_VERSION`]) and simply falls back to the re-arrival resize against it;
/// a v4 driver serving an older (v3-asserting) host fails that host's strict handshake — ship /// a v4 driver serving an older (v3-asserting) host fails that host's strict handshake — ship
/// driver+host together, as ever. /// driver+host together, as ever.
pub const PROTOCOL_VERSION: u32 = 4; /// v5: ADDITIVE — the IddCx HARDWARE CURSOR channel (remote-desktop-sweep M2c):
/// [`control::AddRequest::hw_cursor`] (the former tail `_reserved` — same size, same offsets)
/// asks the driver to declare a hardware cursor for this monitor (DWM then EXCLUDES the pointer
/// from the desktop frame and delivers shape/position out-of-band), and
/// [`control::IOCTL_SET_CURSOR_CHANNEL`] delivers the host-created [`cursor::CursorShm`] section
/// the driver's cursor thread seqlock-publishes into. Nothing existing changed; the host gates
/// the feature on the handshake-reported version (`>= 5`) and keeps composited-cursor behavior
/// against older drivers.
/// v6: ADDITIVE — [`control::IOCTL_SET_CURSOR_FORWARD`] (the mid-stream cursor-render flip,
/// remote-desktop-sweep §8): the client's mouse-model flip (un)declares the hardware cursor on a
/// LIVE monitor, so the capture mouse model gets DWM's composited pointer back (full fidelity)
/// and the desktop model gets exclusion + forwarding. Nothing existing changed; against a v5
/// driver the unknown IOCTL fails and the host logs + keeps the declared-at-ADD behavior.
/// v6 tail ext (no bump, the `AddRequest` luminance-tail discipline):
/// [`control::AddReply::cursor_excluded`] — the driver reports whether the ADDed monitor's OS
/// target already carries a hardware-cursor declare from an earlier session. A declare is
/// IRREVOCABLE for the target's life (remote-desktop-sweep §8.6, proven on-glass): DWM never
/// composites the software cursor back into that target's frames, and the sticky state survives
/// monitor REMOVE→ADD because the host hands every client a STABLE target id. The host uses the
/// flag to self-composite the pointer (GDI poller + blend) in sessions that never negotiate the
/// cursor channel — without it those sessions are silently cursor-less. Both skews degrade
/// cleanly: an old driver writes only the 20-byte reply prefix (host reads `0` = unknown/clean),
/// an old host retrieves a 20-byte buffer (driver writes just the prefix).
pub const PROTOCOL_VERSION: u32 = 6;
/// The OLDEST driver protocol this host still drives (v4 is additive over v3 — see the v4 note on /// The OLDEST driver protocol this host still drives (v4 is additive over v3 — see the v4 note on
/// [`PROTOCOL_VERSION`]): a v3 driver lacks only `IOCTL_UPDATE_MODES`, which the host gates on the /// [`PROTOCOL_VERSION`]): a v3 driver lacks only `IOCTL_UPDATE_MODES`, which the host gates on the
@@ -110,6 +133,22 @@ pub mod control {
/// identity (saved per-monitor DPI) and the driver's swap-chain/stash machinery survive. A v3 /// identity (saved per-monitor DPI) and the driver's swap-chain/stash machinery survive. A v3
/// driver fails this unknown IOCTL → the host falls back to the re-arrival resize. /// driver fails this unknown IOCTL → the host falls back to the re-arrival resize.
pub const IOCTL_UPDATE_MODES: u32 = ctl_code(0x907); pub const IOCTL_UPDATE_MODES: u32 = ctl_code(0x907);
/// Deliver a monitor's hardware-cursor channel (v5): the handle VALUE of the unnamed
/// [`cursor::CursorShm`](crate::cursor) file mapping the host duplicated into the WUDFHost
/// (same delivery model as [`IOCTL_SET_FRAME_CHANNEL`], no event — the host polls the
/// seqlock at its encode-tick pace). Sent once after ADD, only for a monitor whose
/// [`AddRequest::hw_cursor`] was set. The driver maps it, calls
/// `IddCxMonitorSetupHardwareCursor`, and starts its cursor-query thread. Input
/// [`SetCursorChannelRequest`].
pub const IOCTL_SET_CURSOR_CHANNEL: u32 = ctl_code(0x908);
/// Flip a LIVE monitor's hardware-cursor declaration (v6, the mid-stream cursor-render
/// flip): `enable = 1` re-declares (`IddCxMonitorSetupHardwareCursor` — DWM excludes the
/// pointer, the query/shape machinery resumes), `enable = 0` un-declares (the driver stops
/// re-declaring on mode commits and asks the OS to revert to the software cursor — DWM
/// composites the pointer into the frame, the pre-channel behavior the capture mouse
/// model wants). Only meaningful for a monitor whose cursor channel was delivered. Input
/// [`SetCursorForwardRequest`].
pub const IOCTL_SET_CURSOR_FORWARD: u32 = ctl_code(0x909);
/// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns /// `IOCTL_ADD` input. A monotonic `session_id` keys the monitor (the host's refcount manager owns
/// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this /// collision safety — no more SudoVDA's 16-byte GUID + pid-mangling). The driver advertises this
@@ -147,9 +186,13 @@ pub mod control {
/// The client display's min luminance in MILLI-nits (0.001 cd/m² — the CTA min-luminance /// The client display's min luminance in MILLI-nits (0.001 cd/m² — the CTA min-luminance
/// range lives well below 1 nit) → Desired Content Min Luminance. `0` = unknown. /// range lives well below 1 nit) → Desired Content Min Luminance. `0` = unknown.
pub min_luminance_millinits: u32, pub min_luminance_millinits: u32,
/// Pads the `u64`-aligned struct to a multiple of 8 (Pod forbids implicit tail padding); /// Non-zero = declare an IddCx HARDWARE CURSOR for this monitor (v5, remote-desktop-sweep
/// free expansion room for the next appended field. /// M2c): DWM stops compositing the pointer into the frame and the driver publishes
pub _reserved: u32, /// shape/position into the [`cursor::CursorShm`](crate::cursor) section delivered by
/// [`IOCTL_SET_CURSOR_CHANNEL`]. Byte-compatible with the old tail `_reserved` (offset 36):
/// an un-upgraded driver ignores it (cursor stays composited — the host already gates on
/// the handshake version, this is defense in depth), an un-upgraded host sends `0` (off).
pub hw_cursor: u32,
} }
/// [`AddRequest`]'s size before the client-HDR luminance tail — the prefix an un-upgraded /// [`AddRequest`]'s size before the client-HDR luminance tail — the prefix an un-upgraded
@@ -174,8 +217,21 @@ pub mod control {
/// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a /// `DuplicateHandle`, then [`IOCTL_SET_FRAME_CHANNEL`]). Reported per-ADD, not per-open, so a
/// WUDFHost restart between sessions can never leave the host duplicating into a dead process. /// WUDFHost restart between sessions can never leave the host duplicating into a dead process.
pub wudf_pid: u32, pub wudf_pid: u32,
/// Non-zero = this monitor's OS target already carries an IRREVOCABLE hardware-cursor
/// declare from an earlier session (remote-desktop-sweep §8.6): DWM excludes the pointer
/// from every frame on this target, forever, and a session without the cursor channel must
/// self-composite (GDI poller + blend) or stream a cursor-less desktop. Appended after
/// [`ADD_REPLY_LEGACY_SIZE`] under the same dual-size discipline as the `AddRequest`
/// luminance tail: an un-upgraded driver writes only the legacy prefix (the host's
/// zero-initialized buffer then reads `0` = unknown/clean), an un-upgraded host retrieves a
/// legacy-size buffer (the driver writes just the prefix).
pub cursor_excluded: u32,
} }
/// [`AddReply`]'s size before the `cursor_excluded` tail — the prefix an un-upgraded driver
/// writes and an un-upgraded host retrieves (see the field docs).
pub const ADD_REPLY_LEGACY_SIZE: usize = 20;
/// `IOCTL_REMOVE` input. /// `IOCTL_REMOVE` input.
#[repr(C)] #[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)] #[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
@@ -253,6 +309,29 @@ pub mod control {
/// at the compile-time maximum; `ring_len` says how many entries are live). /// at the compile-time maximum; `ring_len` says how many entries are live).
pub const RING_LEN_USIZE: usize = RING_LEN as usize; pub const RING_LEN_USIZE: usize = RING_LEN as usize;
/// `IOCTL_SET_CURSOR_CHANNEL` input (v5): the hardware-cursor section for one monitor.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct SetCursorChannelRequest {
/// The OS target id from [`AddReply`] — which monitor this channel belongs to.
pub target_id: u32,
pub _pad: u32,
/// The [`cursor::CursorShm`](crate::cursor) file-mapping handle VALUE, already duplicated
/// into the driver's WUDFHost process ([`AddReply::wudf_pid`]).
pub header_handle: u64,
}
/// `IOCTL_SET_CURSOR_FORWARD` input (v6): the mid-stream cursor-render flip for one monitor.
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct SetCursorForwardRequest {
/// The OS target id from [`AddReply`] — which monitor to flip.
pub target_id: u32,
/// `1` = declare the hardware cursor (exclude + forward), `0` = un-declare (DWM
/// composites — the capture mouse model).
pub enable: u32,
}
// Layout is load-bearing across the process boundary — pin it. (bytemuck's Pod derive already // Layout is load-bearing across the process boundary — pin it. (bytemuck's Pod derive already
// rejects any internal padding; these assert the externally-visible sizes too.) The `offset_of!` // rejects any internal padding; these assert the externally-visible sizes too.) The `offset_of!`
// asserts additionally catch a SAME-SIZE field reorder, which the size+Pod checks alone miss. // asserts additionally catch a SAME-SIZE field reorder, which the size+Pod checks alone miss.
@@ -269,13 +348,18 @@ pub mod control {
assert!(offset_of!(AddRequest, max_luminance_nits) == ADD_REQUEST_LEGACY_SIZE); assert!(offset_of!(AddRequest, max_luminance_nits) == ADD_REQUEST_LEGACY_SIZE);
assert!(offset_of!(AddRequest, max_frame_avg_nits) == 28); assert!(offset_of!(AddRequest, max_frame_avg_nits) == 28);
assert!(offset_of!(AddRequest, min_luminance_millinits) == 32); assert!(offset_of!(AddRequest, min_luminance_millinits) == 32);
// v5: the former tail `_reserved` — same offset, same total size (rename-only).
assert!(offset_of!(AddRequest, hw_cursor) == 36);
assert!(size_of::<AddRequest>() == 40);
assert!(size_of::<AddReply>() == 20); assert!(size_of::<AddReply>() == 24);
assert!(offset_of!(AddReply, adapter_luid_low) == 0); assert!(offset_of!(AddReply, adapter_luid_low) == 0);
assert!(offset_of!(AddReply, adapter_luid_high) == 4); assert!(offset_of!(AddReply, adapter_luid_high) == 4);
assert!(offset_of!(AddReply, target_id) == 8); assert!(offset_of!(AddReply, target_id) == 8);
assert!(offset_of!(AddReply, resolved_monitor_id) == 12); assert!(offset_of!(AddReply, resolved_monitor_id) == 12);
assert!(offset_of!(AddReply, wudf_pid) == 16); assert!(offset_of!(AddReply, wudf_pid) == 16);
// The cursor-excluded tail starts exactly at the legacy boundary (prefix-compat).
assert!(offset_of!(AddReply, cursor_excluded) == ADD_REPLY_LEGACY_SIZE);
assert!(size_of::<SetFrameChannelRequest>() == 32 + 8 * RING_LEN_USIZE); assert!(size_of::<SetFrameChannelRequest>() == 32 + 8 * RING_LEN_USIZE);
assert!(offset_of!(SetFrameChannelRequest, target_id) == 0); assert!(offset_of!(SetFrameChannelRequest, target_id) == 0);
@@ -288,6 +372,13 @@ pub mod control {
assert!(size_of::<RemoveRequest>() == 8); assert!(size_of::<RemoveRequest>() == 8);
assert!(offset_of!(RemoveRequest, session_id) == 0); assert!(offset_of!(RemoveRequest, session_id) == 0);
assert!(size_of::<SetCursorChannelRequest>() == 16);
assert!(offset_of!(SetCursorChannelRequest, target_id) == 0);
assert!(offset_of!(SetCursorChannelRequest, header_handle) == 8);
assert!(size_of::<SetCursorForwardRequest>() == 8);
assert!(offset_of!(SetCursorForwardRequest, target_id) == 0);
assert!(offset_of!(SetCursorForwardRequest, enable) == 4);
assert!(size_of::<UpdateModesRequest>() == 24); assert!(size_of::<UpdateModesRequest>() == 24);
assert!(offset_of!(UpdateModesRequest, session_id) == 0); assert!(offset_of!(UpdateModesRequest, session_id) == 0);
assert!(offset_of!(UpdateModesRequest, width) == 8); assert!(offset_of!(UpdateModesRequest, width) == 8);
@@ -960,6 +1051,84 @@ pub mod mouse {
}; };
} }
/// The v5 hardware-cursor channel (remote-desktop-sweep M2c): one unnamed file mapping per
/// monitor, host-created, delivered by handle value ([`control::IOCTL_SET_CURSOR_CHANNEL`]).
/// The DRIVER's cursor thread (woken by its IddCx `hNewCursorDataAvailable` event) seqlock-writes
/// shape + position + visibility; the HOST reads at its encode-tick pace — no event crosses the
/// boundary. Writer: bump [`CursorShm::seq`] to ODD, write fields (+ shape bytes when the OS said
/// the shape changed), bump to EVEN. Reader: read seq (retry while odd), copy, re-read seq —
/// unchanged ⇒ consistent snapshot. Position-only updates never touch the shape bytes, so a
/// reader that skips unchanged `shape_id`s never copies torn pixels.
pub mod cursor {
use bytemuck::{Pod, Zeroable};
/// First field of [`CursorShm`] — `b"PFCU"` little-endian; anything else = not attached yet.
pub const CURSOR_MAGIC: u32 = u32::from_le_bytes(*b"PFCU");
/// Max cursor side (px) the driver declares to the OS (`IDDCX_CURSOR_CAPS::MaxX/MaxY`) and
/// the section's shape buffer is sized for. Windows XL accessibility cursors top out here;
/// the host's wire forwarder downscales to its own cap anyway.
pub const CURSOR_SHAPE_MAX: u32 = 256;
/// Shape-buffer bytes: 32-bpp at the declared max.
pub const CURSOR_SHAPE_BYTES: usize = (CURSOR_SHAPE_MAX * CURSOR_SHAPE_MAX * 4) as usize;
/// Byte offset of the shape pixels inside the section (one cache-line-ish header).
pub const CURSOR_SHAPE_OFFSET: usize = 64;
/// Total section size.
pub const CURSOR_SHM_SIZE: usize = CURSOR_SHAPE_OFFSET + CURSOR_SHAPE_BYTES;
/// `IDDCX_CURSOR_SHAPE_TYPE` values mirrored for the host (the driver writes the OS value
/// verbatim into [`CursorShm::cursor_type`]).
pub const CURSOR_TYPE_MASKED_COLOR: u32 = 1;
pub const CURSOR_TYPE_ALPHA: u32 = 2;
/// The section header (the shape pixels follow at [`CURSOR_SHAPE_OFFSET`]). `x`/`y` are the
/// shape's TOP-LEFT in desktop coordinates (the IddCx `IDARG_OUT_QUERY_HWCURSOR::X/Y`
/// convention — position hotspot, can be negative); `shape_id` is the OS's per-set counter
/// (bumps on every shape set, the overlay serial); pixels are the OS's 32-bpp rows at
/// `pitch` bytes (BGRA for ALPHA; color+mask for MASKED_COLOR — the host converts).
#[repr(C)]
#[derive(Clone, Copy, Pod, Zeroable, Debug, PartialEq, Eq)]
pub struct CursorShm {
pub magic: u32,
/// Seqlock: odd = writer mid-update.
pub seq: u32,
pub visible: u32,
pub cursor_type: u32,
pub x: i32,
pub y: i32,
pub shape_id: u32,
pub width: u32,
pub height: u32,
pub pitch: u32,
pub hot_x: u32,
pub hot_y: u32,
/// Reserved expansion room up to [`CURSOR_SHAPE_OFFSET`].
pub _reserved: [u32; 4],
}
// Layout is load-bearing across the process boundary — pin it.
const _: () = {
use core::mem::{offset_of, size_of};
assert!(size_of::<CursorShm>() == 64);
assert!(size_of::<CursorShm>() <= CURSOR_SHAPE_OFFSET);
assert!(offset_of!(CursorShm, magic) == 0);
assert!(offset_of!(CursorShm, seq) == 4);
assert!(offset_of!(CursorShm, visible) == 8);
assert!(offset_of!(CursorShm, cursor_type) == 12);
assert!(offset_of!(CursorShm, x) == 16);
assert!(offset_of!(CursorShm, y) == 20);
assert!(offset_of!(CursorShm, shape_id) == 24);
assert!(offset_of!(CursorShm, width) == 28);
assert!(offset_of!(CursorShm, height) == 32);
assert!(offset_of!(CursorShm, pitch) == 36);
assert!(offset_of!(CursorShm, hot_x) == 40);
assert!(offset_of!(CursorShm, hot_y) == 44);
};
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -1079,7 +1248,7 @@ mod tests {
max_luminance_nits: 800, max_luminance_nits: 800,
max_frame_avg_nits: 400, max_frame_avg_nits: 400,
min_luminance_millinits: 50, // 0.05 nits min_luminance_millinits: 50, // 0.05 nits
_reserved: 0, hw_cursor: 1,
}; };
let bytes = bytemuck::bytes_of(&req); let bytes = bytemuck::bytes_of(&req);
assert_eq!(bytes.len(), 40); assert_eq!(bytes.len(), 40);
@@ -1109,14 +1278,19 @@ mod tests {
target_id: 262, target_id: 262,
resolved_monitor_id: 7, resolved_monitor_id: 7,
wudf_pid: 4242, wudf_pid: 4242,
cursor_excluded: 1,
}; };
let rbytes = bytemuck::bytes_of(&reply); let rbytes = bytemuck::bytes_of(&reply);
assert_eq!(rbytes.len(), 20); assert_eq!(rbytes.len(), 24);
assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply); assert_eq!(*bytemuck::from_bytes::<control::AddReply>(rbytes), reply);
// resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible. // resolved_monitor_id occupies the old `_reserved` slot at offset 12 — byte-compatible.
assert_eq!(rbytes[12..16], 7u32.to_le_bytes()); assert_eq!(rbytes[12..16], 7u32.to_le_bytes());
// The v2 duplication-target pid trails at offset 16. // The v2 duplication-target pid trails at offset 16.
assert_eq!(rbytes[16..20], 4242u32.to_le_bytes()); assert_eq!(rbytes[16..20], 4242u32.to_le_bytes());
// The cursor-excluded tail rides after the legacy boundary; an un-upgraded driver writes
// only the prefix, so a zero-filled tail reads as "unknown/clean" (see the field docs).
assert_eq!(rbytes[20..24], 1u32.to_le_bytes());
assert_eq!(control::ADD_REPLY_LEGACY_SIZE, 20);
} }
#[test] #[test]
@@ -1161,11 +1335,39 @@ mod tests {
req req
); );
assert_eq!(bytes[8..12], 2560u32.to_le_bytes()); assert_eq!(bytes[8..12], 2560u32.to_le_bytes());
// The compat window: v4 is additive over v3, so the host floor stays one below. // The compat window: v4v6 are additive over v3, so the host floor stays at 3.
assert_eq!(PROTOCOL_VERSION, 4); assert_eq!(PROTOCOL_VERSION, 6);
assert_eq!(MIN_DRIVER_PROTOCOL_VERSION, 3); assert_eq!(MIN_DRIVER_PROTOCOL_VERSION, 3);
} }
#[test]
fn cursor_shm_layout_is_pinned() {
use cursor::*;
// The header must leave the shape offset intact whatever grows inside `_reserved`.
assert_eq!(core::mem::size_of::<CursorShm>(), 64);
assert_eq!(CURSOR_SHM_SIZE, 64 + 256 * 256 * 4);
assert_eq!(CURSOR_MAGIC, u32::from_le_bytes(*b"PFCU"));
// Seqlock snapshot discipline survives a bytemuck roundtrip.
let hdr = CursorShm {
magic: CURSOR_MAGIC,
seq: 2,
visible: 1,
cursor_type: CURSOR_TYPE_ALPHA,
x: -3,
y: 7,
shape_id: 42,
width: 32,
height: 32,
pitch: 128,
hot_x: 4,
hot_y: 5,
_reserved: [0; 4],
};
let bytes = bytemuck::bytes_of(&hdr);
assert_eq!(*bytemuck::from_bytes::<CursorShm>(bytes), hdr);
assert_eq!(bytes[16..20], (-3i32).to_le_bytes());
}
#[test] #[test]
fn gamepad_names_and_magics_are_stable() { fn gamepad_names_and_magics_are_stable() {
assert_eq!(gamepad::xusb_boot_name(0), "Global\\pfxusb-boot-0"); assert_eq!(gamepad::xusb_boot_name(0), "Global\\pfxusb-boot-0");
+5
View File
@@ -25,6 +25,11 @@ tracing = "0.1"
# A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`). # A test writer for the NVENC backend's unit tests (`with_test_writer().try_init()`).
tracing-subscriber = { version = "0.3", features = ["env-filter"] } tracing-subscriber = { version = "0.3", features = ["env-filter"] }
[target.'cfg(target_os = "windows")'.dev-dependencies]
# The QSV live e2e drives the REAL HdrP010Converter output (an RTV-written, ring-profile P010
# texture) into the encoder — the one seam the CPU-upload tests can't reach.
pf-capture = { path = "../pf-capture" }
[target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies] [target.'cfg(any(target_os = "linux", target_os = "windows"))'.dependencies]
# Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms. # Software H.264 (openh264, BSD-2) — the GPU-less encode path on both platforms.
openh264 = "0.9" openh264 = "0.9"
@@ -1,105 +0,0 @@
// Cursor-overlay blend kernels for the CUDA/NVENC path (cursor-as-metadata). The cursor bitmap is
// straight-alpha RGBA, row-packed (stride = curW*4). Blended into the encoder-OWNED NVENC input
// surface — never the compositor's dmabuf. One thread per cursor pixel (ARGB / YUV444) or per 2x2
// chroma block (NV12). Coefficients are BT.709 limited, matching rgb2yuv.comp so the cursor colour
// matches the rest of the frame regardless of which zero-copy backend encodes it.
//
// Build (regenerate cursor_blend.ptx after editing):
// nvcc -ptx -arch=compute_75 cursor_blend.cu -o cursor_blend.ptx
// PTX is JIT'd by the driver forward to the actual GPU, so a compute_75 (Turing) baseline runs on
// every Turing-or-newer NVENC GPU. (CUDA 13's nvcc no longer targets pre-Turing archs.)
typedef unsigned char u8;
__device__ __forceinline__ u8 blend8(int dst, int src, int a) {
return (u8)((src * a + dst * (255 - a)) / 255);
}
// Packed 4-byte surface. NVENC's ARGB format stores bytes B,G,R,A in memory; the cursor is R,G,B,A.
extern "C" __global__ void blend_argb(
u8* surf, int pitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int cx = blockIdx.x * blockDim.x + threadIdx.x;
int cy = blockIdx.y * blockDim.y + threadIdx.y;
if (cx >= curW || cy >= curH) return;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) return;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) return;
u8* d = surf + (size_t)py * pitch + (size_t)px * 4;
d[0] = blend8(d[0], s[2], a); // B <- cursor B
d[1] = blend8(d[1], s[1], a); // G <- cursor G
d[2] = blend8(d[2], s[0], a); // R <- cursor R
}
// Planar YUV444: three full-res planes stacked at base, base+plane, base+2*plane (plane=pitch*surfH).
extern "C" __global__ void blend_yuv444(
u8* base, int pitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int cx = blockIdx.x * blockDim.x + threadIdx.x;
int cy = blockIdx.y * blockDim.y + threadIdx.y;
if (cx >= curW || cy >= curH) return;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) return;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) return;
float R = s[0], G = s[1], B = s[2];
int Y = (int)(16.0f + 0.1826f * R + 0.6142f * G + 0.0620f * B + 0.5f);
int U = (int)(128.0f - 0.1006f * R - 0.3386f * G + 0.4392f * B + 0.5f);
int V = (int)(128.0f + 0.4392f * R - 0.3989f * G - 0.0403f * B + 0.5f);
size_t plane = (size_t)pitch * surfH;
u8* yp = base + (size_t)py * pitch + px;
u8* up = base + plane + (size_t)py * pitch + px;
u8* vp = base + 2 * plane + (size_t)py * pitch + px;
*yp = blend8(*yp, Y, a);
*up = blend8(*up, U, a);
*vp = blend8(*vp, V, a);
}
// NV12: full-res Y plane + interleaved half-res UV plane. One thread per 2x2 luma block; each blends
// up to four Y samples and one (alpha-weighted) UV sample.
extern "C" __global__ void blend_nv12(
u8* yb, int yPitch, u8* uvb, int uvPitch, int surfW, int surfH,
const u8* cur, int curW, int curH, int ox, int oy)
{
int bx = blockIdx.x * blockDim.x + threadIdx.x;
int by = blockIdx.y * blockDim.y + threadIdx.y;
int base_cx = bx * 2, base_cy = by * 2;
if (base_cx >= curW || base_cy >= curH) return;
float ua = 0.0f, va = 0.0f, wa = 0.0f;
int cnt = 0;
for (int j = 0; j < 2; j++) {
for (int i = 0; i < 2; i++) {
int cx = base_cx + i, cy = base_cy + j;
if (cx >= curW || cy >= curH) continue;
int px = ox + cx, py = oy + cy;
if (px < 0 || py < 0 || px >= surfW || py >= surfH) continue;
const u8* s = cur + (size_t)(cy * curW + cx) * 4;
int a = s[3];
if (a == 0) continue;
float R = s[0], G = s[1], B = s[2];
int Y = (int)(16.0f + 0.1826f * R + 0.6142f * G + 0.0620f * B + 0.5f);
u8* yp = yb + (size_t)py * yPitch + px;
*yp = blend8(*yp, Y, a);
ua += (128.0f - 0.1006f * R - 0.3386f * G + 0.4392f * B) * a;
va += (128.0f + 0.4392f * R - 0.3989f * G - 0.0403f * B) * a;
wa += a;
cnt++;
}
}
if (wa <= 0.0f || cnt == 0) return;
// The chroma sample covering this block's top-left surface pixel.
int uvx = (ox + base_cx) / 2;
int uvy = (oy + base_cy) / 2;
if (uvx < 0 || uvy < 0 || uvx * 2 >= surfW || uvy * 2 >= surfH) return;
int U = (int)(ua / wa + 0.5f);
int V = (int)(va / wa + 0.5f);
int amean = (int)(wa / cnt + 0.5f);
u8* uv = uvb + (size_t)uvy * uvPitch + (size_t)uvx * 2;
uv[0] = blend8(uv[0], U, amean);
uv[1] = blend8(uv[1], V, amean);
}
@@ -1,576 +0,0 @@
//
// Generated by NVIDIA NVVM Compiler
//
// Compiler Build ID: CL-38244171
// Cuda compilation tools, release 13.3, V13.3.73
// Based on NVVM 7.0.1
//
.version 9.3
.target sm_75
.address_size 64
// .globl blend_argb
.visible .entry blend_argb(
.param .u64 blend_argb_param_0,
.param .u32 blend_argb_param_1,
.param .u32 blend_argb_param_2,
.param .u32 blend_argb_param_3,
.param .u64 blend_argb_param_4,
.param .u32 blend_argb_param_5,
.param .u32 blend_argb_param_6,
.param .u32 blend_argb_param_7,
.param .u32 blend_argb_param_8
)
{
.reg .pred %p<10>;
.reg .b16 %rs<2>;
.reg .b32 %r<34>;
.reg .b64 %rd<17>;
ld.param.u64 %rd2, [blend_argb_param_0];
ld.param.u32 %r5, [blend_argb_param_1];
ld.param.u32 %r6, [blend_argb_param_2];
ld.param.u32 %r7, [blend_argb_param_3];
ld.param.u64 %rd3, [blend_argb_param_4];
ld.param.u32 %r8, [blend_argb_param_5];
ld.param.u32 %r11, [blend_argb_param_6];
ld.param.u32 %r9, [blend_argb_param_7];
ld.param.u32 %r10, [blend_argb_param_8];
mov.u32 %r12, %ntid.x;
mov.u32 %r13, %ctaid.x;
mov.u32 %r14, %tid.x;
mad.lo.s32 %r1, %r13, %r12, %r14;
mov.u32 %r15, %ntid.y;
mov.u32 %r16, %ctaid.y;
mov.u32 %r17, %tid.y;
mad.lo.s32 %r2, %r16, %r15, %r17;
setp.ge.s32 %p1, %r1, %r8;
setp.ge.s32 %p2, %r2, %r11;
or.pred %p3, %p1, %p2;
@%p3 bra $L__BB0_4;
add.s32 %r3, %r1, %r9;
add.s32 %r4, %r2, %r10;
or.b32 %r18, %r4, %r3;
setp.lt.s32 %p4, %r18, 0;
setp.ge.s32 %p5, %r3, %r6;
or.pred %p6, %p5, %p4;
setp.ge.s32 %p7, %r4, %r7;
or.pred %p8, %p7, %p6;
@%p8 bra $L__BB0_4;
mad.lo.s32 %r19, %r2, %r8, %r1;
mul.wide.s32 %rd4, %r19, 4;
cvta.to.global.u64 %rd5, %rd3;
add.s64 %rd1, %rd5, %rd4;
ld.global.u8 %rs1, [%rd1+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB0_4;
cvt.u32.u16 %r20, %rs1;
mul.wide.s32 %rd6, %r4, %r5;
mul.wide.s32 %rd7, %r3, 4;
add.s64 %rd8, %rd6, %rd7;
cvta.to.global.u64 %rd9, %rd2;
add.s64 %rd10, %rd9, %rd8;
ld.global.u8 %r21, [%rd10];
ld.global.u8 %r22, [%rd1+2];
xor.b32 %r23, %r20, 255;
mul.lo.s32 %r24, %r23, %r21;
mad.lo.s32 %r25, %r22, %r20, %r24;
mul.wide.u32 %rd11, %r25, -2139062143;
shr.u64 %rd12, %rd11, 39;
st.global.u8 [%rd10], %rd12;
ld.global.u8 %r26, [%rd10+1];
ld.global.u8 %r27, [%rd1+1];
mul.lo.s32 %r28, %r23, %r26;
mad.lo.s32 %r29, %r27, %r20, %r28;
mul.wide.u32 %rd13, %r29, -2139062143;
shr.u64 %rd14, %rd13, 39;
st.global.u8 [%rd10+1], %rd14;
ld.global.u8 %r30, [%rd10+2];
ld.global.u8 %r31, [%rd1];
mul.lo.s32 %r32, %r23, %r30;
mad.lo.s32 %r33, %r31, %r20, %r32;
mul.wide.u32 %rd15, %r33, -2139062143;
shr.u64 %rd16, %rd15, 39;
st.global.u8 [%rd10+2], %rd16;
$L__BB0_4:
ret;
}
// .globl blend_yuv444
.visible .entry blend_yuv444(
.param .u64 blend_yuv444_param_0,
.param .u32 blend_yuv444_param_1,
.param .u32 blend_yuv444_param_2,
.param .u32 blend_yuv444_param_3,
.param .u64 blend_yuv444_param_4,
.param .u32 blend_yuv444_param_5,
.param .u32 blend_yuv444_param_6,
.param .u32 blend_yuv444_param_7,
.param .u32 blend_yuv444_param_8
)
{
.reg .pred %p<10>;
.reg .b16 %rs<5>;
.reg .f32 %f<16>;
.reg .b32 %r<49>;
.reg .b64 %rd<14>;
ld.param.u64 %rd2, [blend_yuv444_param_0];
ld.param.u32 %r5, [blend_yuv444_param_1];
ld.param.u32 %r6, [blend_yuv444_param_2];
ld.param.u32 %r7, [blend_yuv444_param_3];
ld.param.u64 %rd3, [blend_yuv444_param_4];
ld.param.u32 %r8, [blend_yuv444_param_5];
ld.param.u32 %r11, [blend_yuv444_param_6];
ld.param.u32 %r9, [blend_yuv444_param_7];
ld.param.u32 %r10, [blend_yuv444_param_8];
mov.u32 %r12, %ntid.x;
mov.u32 %r13, %ctaid.x;
mov.u32 %r14, %tid.x;
mad.lo.s32 %r1, %r13, %r12, %r14;
mov.u32 %r15, %ntid.y;
mov.u32 %r16, %ctaid.y;
mov.u32 %r17, %tid.y;
mad.lo.s32 %r2, %r16, %r15, %r17;
setp.ge.s32 %p1, %r1, %r8;
setp.ge.s32 %p2, %r2, %r11;
or.pred %p3, %p1, %p2;
@%p3 bra $L__BB1_4;
add.s32 %r3, %r1, %r9;
add.s32 %r4, %r2, %r10;
or.b32 %r18, %r4, %r3;
setp.lt.s32 %p4, %r18, 0;
setp.ge.s32 %p5, %r3, %r6;
or.pred %p6, %p5, %p4;
setp.ge.s32 %p7, %r4, %r7;
or.pred %p8, %p7, %p6;
@%p8 bra $L__BB1_4;
mad.lo.s32 %r19, %r2, %r8, %r1;
mul.wide.s32 %rd4, %r19, 4;
cvta.to.global.u64 %rd5, %rd3;
add.s64 %rd1, %rd5, %rd4;
ld.global.u8 %rs1, [%rd1+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB1_4;
cvt.u32.u16 %r20, %rs1;
ld.global.u8 %rs2, [%rd1];
cvt.rn.f32.u16 %f1, %rs2;
ld.global.u8 %rs3, [%rd1+1];
cvt.rn.f32.u16 %f2, %rs3;
ld.global.u8 %rs4, [%rd1+2];
cvt.rn.f32.u16 %f3, %rs4;
fma.rn.f32 %f4, %f1, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f5, %f2, 0f3F1D3C36, %f4;
fma.rn.f32 %f6, %f3, 0f3D7DF3B6, %f5;
add.f32 %f7, %f6, 0f3F000000;
cvt.rzi.s32.f32 %r21, %f7;
fma.rn.f32 %f8, %f1, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f9, %f2, 0fBEAD5CFB, %f8;
fma.rn.f32 %f10, %f3, 0f3EE0DED3, %f9;
add.f32 %f11, %f10, 0f3F000000;
cvt.rzi.s32.f32 %r22, %f11;
fma.rn.f32 %f12, %f1, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f13, %f2, 0fBECC3C9F, %f12;
fma.rn.f32 %f14, %f3, 0fBD25119D, %f13;
add.f32 %f15, %f14, 0f3F000000;
cvt.rzi.s32.f32 %r23, %f15;
mul.wide.s32 %rd6, %r4, %r5;
cvt.s64.s32 %rd7, %r3;
add.s64 %rd8, %rd6, %rd7;
cvta.to.global.u64 %rd9, %rd2;
add.s64 %rd10, %rd9, %rd8;
ld.global.u8 %r24, [%rd10];
mul.lo.s32 %r25, %r21, %r20;
xor.b32 %r26, %r20, 255;
mad.lo.s32 %r27, %r26, %r24, %r25;
mul.hi.s32 %r28, %r27, -2139062143;
add.s32 %r29, %r28, %r27;
shr.u32 %r30, %r29, 31;
shr.u32 %r31, %r29, 7;
add.s32 %r32, %r31, %r30;
st.global.u8 [%rd10], %r32;
mul.wide.s32 %rd11, %r7, %r5;
add.s64 %rd12, %rd10, %rd11;
ld.global.u8 %r33, [%rd12];
mul.lo.s32 %r34, %r22, %r20;
mad.lo.s32 %r35, %r26, %r33, %r34;
mul.hi.s32 %r36, %r35, -2139062143;
add.s32 %r37, %r36, %r35;
shr.u32 %r38, %r37, 31;
shr.u32 %r39, %r37, 7;
add.s32 %r40, %r39, %r38;
st.global.u8 [%rd12], %r40;
add.s64 %rd13, %rd12, %rd11;
ld.global.u8 %r41, [%rd13];
mul.lo.s32 %r42, %r23, %r20;
mad.lo.s32 %r43, %r26, %r41, %r42;
mul.hi.s32 %r44, %r43, -2139062143;
add.s32 %r45, %r44, %r43;
shr.u32 %r46, %r45, 31;
shr.u32 %r47, %r45, 7;
add.s32 %r48, %r47, %r46;
st.global.u8 [%rd13], %r48;
$L__BB1_4:
ret;
}
// .globl blend_nv12
.visible .entry blend_nv12(
.param .u64 blend_nv12_param_0,
.param .u32 blend_nv12_param_1,
.param .u64 blend_nv12_param_2,
.param .u32 blend_nv12_param_3,
.param .u32 blend_nv12_param_4,
.param .u32 blend_nv12_param_5,
.param .u64 blend_nv12_param_6,
.param .u32 blend_nv12_param_7,
.param .u32 blend_nv12_param_8,
.param .u32 blend_nv12_param_9,
.param .u32 blend_nv12_param_10
)
{
.reg .pred %p<43>;
.reg .b16 %rs<17>;
.reg .f32 %f<108>;
.reg .b32 %r<123>;
.reg .b64 %rd<35>;
ld.param.u64 %rd11, [blend_nv12_param_0];
ld.param.u32 %r21, [blend_nv12_param_1];
ld.param.u64 %rd10, [blend_nv12_param_2];
ld.param.u32 %r22, [blend_nv12_param_3];
ld.param.u32 %r23, [blend_nv12_param_4];
ld.param.u32 %r24, [blend_nv12_param_5];
ld.param.u64 %rd12, [blend_nv12_param_6];
ld.param.u32 %r25, [blend_nv12_param_7];
ld.param.u32 %r26, [blend_nv12_param_8];
ld.param.u32 %r27, [blend_nv12_param_9];
ld.param.u32 %r28, [blend_nv12_param_10];
cvta.to.global.u64 %rd1, %rd11;
cvta.to.global.u64 %rd2, %rd12;
mov.u32 %r29, %ntid.x;
mov.u32 %r30, %ctaid.x;
mov.u32 %r31, %tid.x;
mad.lo.s32 %r32, %r30, %r29, %r31;
mov.u32 %r33, %ntid.y;
mov.u32 %r34, %ctaid.y;
mov.u32 %r35, %tid.y;
mad.lo.s32 %r36, %r34, %r33, %r35;
shl.b32 %r1, %r32, 1;
shl.b32 %r2, %r36, 1;
setp.ge.s32 %p1, %r1, %r25;
setp.ge.s32 %p2, %r2, %r26;
or.pred %p3, %p1, %p2;
mov.f32 %f102, 0f00000000;
mov.f32 %f103, 0f00000000;
mov.f32 %f104, 0f00000000;
@%p3 bra $L__BB2_19;
cvt.s64.s32 %rd3, %r21;
add.s32 %r3, %r2, %r28;
setp.ge.s32 %p4, %r3, %r24;
mul.lo.s32 %r4, %r2, %r25;
mul.wide.s32 %rd4, %r3, %r21;
add.s32 %r5, %r1, %r27;
or.b32 %r38, %r5, %r3;
setp.lt.s32 %p5, %r38, 0;
mov.u32 %r121, 0;
setp.ge.s32 %p6, %r5, %r23;
or.pred %p7, %p6, %p5;
or.pred %p8, %p4, %p7;
@%p8 bra $L__BB2_4;
add.s32 %r40, %r1, %r4;
mul.wide.s32 %rd13, %r40, 4;
add.s64 %rd5, %rd2, %rd13;
ld.global.u8 %rs1, [%rd5+3];
setp.eq.s16 %p9, %rs1, 0;
@%p9 bra $L__BB2_4;
cvt.u32.u16 %r42, %rs1;
ld.global.u8 %rs5, [%rd5];
cvt.rn.f32.u16 %f31, %rs5;
ld.global.u8 %rs6, [%rd5+1];
cvt.rn.f32.u16 %f32, %rs6;
ld.global.u8 %rs7, [%rd5+2];
cvt.rn.f32.u16 %f33, %rs7;
fma.rn.f32 %f34, %f31, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f35, %f32, 0f3F1D3C36, %f34;
fma.rn.f32 %f36, %f33, 0f3D7DF3B6, %f35;
add.f32 %f37, %f36, 0f3F000000;
cvt.rzi.s32.f32 %r43, %f37;
cvt.s64.s32 %rd14, %r5;
add.s64 %rd15, %rd4, %rd14;
add.s64 %rd16, %rd1, %rd15;
ld.global.u8 %r44, [%rd16];
mul.lo.s32 %r45, %r43, %r42;
xor.b32 %r46, %r42, 255;
mad.lo.s32 %r47, %r46, %r44, %r45;
mul.hi.s32 %r48, %r47, -2139062143;
add.s32 %r49, %r48, %r47;
shr.u32 %r50, %r49, 31;
shr.u32 %r51, %r49, 7;
add.s32 %r52, %r51, %r50;
st.global.u8 [%rd16], %r52;
fma.rn.f32 %f38, %f31, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f39, %f32, 0fBEAD5CFB, %f38;
fma.rn.f32 %f40, %f33, 0f3EE0DED3, %f39;
cvt.rn.f32.u16 %f104, %rs1;
fma.rn.f32 %f102, %f40, %f104, 0f00000000;
fma.rn.f32 %f41, %f31, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f42, %f32, 0fBECC3C9F, %f41;
fma.rn.f32 %f43, %f33, 0fBD25119D, %f42;
fma.rn.f32 %f103, %f43, %f104, 0f00000000;
mov.u32 %r121, 1;
$L__BB2_4:
add.s32 %r7, %r1, 1;
setp.ge.s32 %p10, %r7, %r25;
@%p10 bra $L__BB2_8;
add.s32 %r8, %r7, %r27;
or.b32 %r53, %r8, %r3;
setp.lt.s32 %p12, %r53, 0;
setp.ge.s32 %p13, %r8, %r23;
or.pred %p14, %p13, %p12;
or.pred %p15, %p4, %p14;
@%p15 bra $L__BB2_8;
add.s32 %r54, %r7, %r4;
mul.wide.s32 %rd17, %r54, 4;
add.s64 %rd6, %rd2, %rd17;
ld.global.u8 %rs2, [%rd6+3];
setp.eq.s16 %p16, %rs2, 0;
@%p16 bra $L__BB2_8;
cvt.u32.u16 %r55, %rs2;
ld.global.u8 %rs8, [%rd6];
cvt.rn.f32.u16 %f44, %rs8;
ld.global.u8 %rs9, [%rd6+1];
cvt.rn.f32.u16 %f45, %rs9;
ld.global.u8 %rs10, [%rd6+2];
cvt.rn.f32.u16 %f46, %rs10;
fma.rn.f32 %f47, %f44, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f48, %f45, 0f3F1D3C36, %f47;
fma.rn.f32 %f49, %f46, 0f3D7DF3B6, %f48;
add.f32 %f50, %f49, 0f3F000000;
cvt.rzi.s32.f32 %r56, %f50;
cvt.s64.s32 %rd18, %r8;
add.s64 %rd19, %rd4, %rd18;
add.s64 %rd20, %rd1, %rd19;
ld.global.u8 %r57, [%rd20];
mul.lo.s32 %r58, %r56, %r55;
xor.b32 %r59, %r55, 255;
mad.lo.s32 %r60, %r59, %r57, %r58;
mul.hi.s32 %r61, %r60, -2139062143;
add.s32 %r62, %r61, %r60;
shr.u32 %r63, %r62, 31;
shr.u32 %r64, %r62, 7;
add.s32 %r65, %r64, %r63;
st.global.u8 [%rd20], %r65;
fma.rn.f32 %f51, %f44, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f52, %f45, 0fBEAD5CFB, %f51;
fma.rn.f32 %f53, %f46, 0f3EE0DED3, %f52;
cvt.rn.f32.u16 %f54, %rs2;
fma.rn.f32 %f102, %f53, %f54, %f102;
fma.rn.f32 %f55, %f44, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f56, %f45, 0fBECC3C9F, %f55;
fma.rn.f32 %f57, %f46, 0fBD25119D, %f56;
fma.rn.f32 %f103, %f57, %f54, %f103;
add.f32 %f104, %f104, %f54;
add.s32 %r121, %r121, 1;
$L__BB2_8:
add.s32 %r11, %r2, 1;
setp.ge.s32 %p17, %r11, %r26;
add.s32 %r12, %r11, %r28;
add.s32 %r13, %r4, %r25;
cvt.s64.s32 %rd21, %r12;
mul.lo.s64 %rd7, %rd21, %rd3;
@%p17 bra $L__BB2_12;
setp.ge.s32 %p18, %r12, %r24;
or.b32 %r66, %r5, %r12;
setp.lt.s32 %p19, %r66, 0;
or.pred %p21, %p6, %p19;
or.pred %p22, %p18, %p21;
@%p22 bra $L__BB2_12;
add.s32 %r67, %r1, %r13;
mul.wide.s32 %rd22, %r67, 4;
add.s64 %rd8, %rd2, %rd22;
ld.global.u8 %rs3, [%rd8+3];
setp.eq.s16 %p23, %rs3, 0;
@%p23 bra $L__BB2_12;
cvt.u32.u16 %r68, %rs3;
ld.global.u8 %rs11, [%rd8];
cvt.rn.f32.u16 %f58, %rs11;
ld.global.u8 %rs12, [%rd8+1];
cvt.rn.f32.u16 %f59, %rs12;
ld.global.u8 %rs13, [%rd8+2];
cvt.rn.f32.u16 %f60, %rs13;
fma.rn.f32 %f61, %f58, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f62, %f59, 0f3F1D3C36, %f61;
fma.rn.f32 %f63, %f60, 0f3D7DF3B6, %f62;
add.f32 %f64, %f63, 0f3F000000;
cvt.rzi.s32.f32 %r69, %f64;
cvt.s64.s32 %rd23, %r5;
add.s64 %rd24, %rd7, %rd23;
add.s64 %rd25, %rd1, %rd24;
ld.global.u8 %r70, [%rd25];
mul.lo.s32 %r71, %r69, %r68;
xor.b32 %r72, %r68, 255;
mad.lo.s32 %r73, %r72, %r70, %r71;
mul.hi.s32 %r74, %r73, -2139062143;
add.s32 %r75, %r74, %r73;
shr.u32 %r76, %r75, 31;
shr.u32 %r77, %r75, 7;
add.s32 %r78, %r77, %r76;
st.global.u8 [%rd25], %r78;
fma.rn.f32 %f65, %f58, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f66, %f59, 0fBEAD5CFB, %f65;
fma.rn.f32 %f67, %f60, 0f3EE0DED3, %f66;
cvt.rn.f32.u16 %f68, %rs3;
fma.rn.f32 %f102, %f67, %f68, %f102;
fma.rn.f32 %f69, %f58, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f70, %f59, 0fBECC3C9F, %f69;
fma.rn.f32 %f71, %f60, 0fBD25119D, %f70;
fma.rn.f32 %f103, %f71, %f68, %f103;
add.f32 %f104, %f104, %f68;
add.s32 %r121, %r121, 1;
$L__BB2_12:
or.pred %p26, %p17, %p10;
@%p26 bra $L__BB2_16;
setp.ge.s32 %p27, %r12, %r24;
add.s32 %r16, %r7, %r27;
or.b32 %r79, %r16, %r12;
setp.lt.s32 %p28, %r79, 0;
setp.ge.s32 %p29, %r16, %r23;
or.pred %p30, %p29, %p28;
or.pred %p31, %p27, %p30;
@%p31 bra $L__BB2_16;
add.s32 %r80, %r7, %r13;
mul.wide.s32 %rd26, %r80, 4;
add.s64 %rd9, %rd2, %rd26;
ld.global.u8 %rs4, [%rd9+3];
setp.eq.s16 %p32, %rs4, 0;
@%p32 bra $L__BB2_16;
cvt.u32.u16 %r81, %rs4;
ld.global.u8 %rs14, [%rd9];
cvt.rn.f32.u16 %f72, %rs14;
ld.global.u8 %rs15, [%rd9+1];
cvt.rn.f32.u16 %f73, %rs15;
ld.global.u8 %rs16, [%rd9+2];
cvt.rn.f32.u16 %f74, %rs16;
fma.rn.f32 %f75, %f72, 0f3E3AFB7F, 0f41800000;
fma.rn.f32 %f76, %f73, 0f3F1D3C36, %f75;
fma.rn.f32 %f77, %f74, 0f3D7DF3B6, %f76;
add.f32 %f78, %f77, 0f3F000000;
cvt.rzi.s32.f32 %r82, %f78;
cvt.s64.s32 %rd27, %r16;
add.s64 %rd28, %rd7, %rd27;
add.s64 %rd29, %rd1, %rd28;
ld.global.u8 %r83, [%rd29];
mul.lo.s32 %r84, %r82, %r81;
xor.b32 %r85, %r81, 255;
mad.lo.s32 %r86, %r85, %r83, %r84;
mul.hi.s32 %r87, %r86, -2139062143;
add.s32 %r88, %r87, %r86;
shr.u32 %r89, %r88, 31;
shr.u32 %r90, %r88, 7;
add.s32 %r91, %r90, %r89;
st.global.u8 [%rd29], %r91;
fma.rn.f32 %f79, %f72, 0fBDCE075F, 0f43000000;
fma.rn.f32 %f80, %f73, 0fBEAD5CFB, %f79;
fma.rn.f32 %f81, %f74, 0f3EE0DED3, %f80;
cvt.rn.f32.u16 %f82, %rs4;
fma.rn.f32 %f102, %f81, %f82, %f102;
fma.rn.f32 %f83, %f72, 0f3EE0DED3, 0f43000000;
fma.rn.f32 %f84, %f73, 0fBECC3C9F, %f83;
fma.rn.f32 %f85, %f74, 0fBD25119D, %f84;
fma.rn.f32 %f103, %f85, %f82, %f103;
add.f32 %f104, %f104, %f82;
add.s32 %r121, %r121, 1;
$L__BB2_16:
setp.eq.s32 %p33, %r121, 0;
setp.le.f32 %p34, %f104, 0f00000000;
or.pred %p35, %p34, %p33;
@%p35 bra $L__BB2_19;
shr.u32 %r92, %r5, 31;
add.s32 %r93, %r5, %r92;
shr.s32 %r19, %r93, 1;
setp.lt.s32 %p36, %r3, -1;
setp.lt.s32 %p37, %r5, -1;
or.pred %p38, %p37, %p36;
and.b32 %r94, %r93, -2;
setp.ge.s32 %p39, %r94, %r23;
or.pred %p40, %p38, %p39;
shr.u32 %r95, %r3, 31;
add.s32 %r96, %r3, %r95;
shr.s32 %r20, %r96, 1;
and.b32 %r97, %r96, -2;
setp.ge.s32 %p41, %r97, %r24;
or.pred %p42, %p40, %p41;
@%p42 bra $L__BB2_19;
div.rn.f32 %f86, %f102, %f104;
add.f32 %f87, %f86, 0f3F000000;
cvt.rzi.s32.f32 %r98, %f87;
div.rn.f32 %f88, %f103, %f104;
add.f32 %f89, %f88, 0f3F000000;
cvt.rzi.s32.f32 %r99, %f89;
cvt.rn.f32.s32 %f90, %r121;
div.rn.f32 %f91, %f104, %f90;
add.f32 %f92, %f91, 0f3F000000;
cvt.rzi.s32.f32 %r100, %f92;
mul.wide.s32 %rd30, %r20, %r22;
mul.wide.s32 %rd31, %r19, 2;
add.s64 %rd32, %rd30, %rd31;
cvta.to.global.u64 %rd33, %rd10;
add.s64 %rd34, %rd33, %rd32;
ld.global.u8 %r101, [%rd34];
mul.lo.s32 %r102, %r100, %r98;
mov.u32 %r103, 255;
sub.s32 %r104, %r103, %r100;
mad.lo.s32 %r105, %r104, %r101, %r102;
mul.hi.s32 %r106, %r105, -2139062143;
add.s32 %r107, %r106, %r105;
shr.u32 %r108, %r107, 31;
shr.u32 %r109, %r107, 7;
add.s32 %r110, %r109, %r108;
st.global.u8 [%rd34], %r110;
ld.global.u8 %r111, [%rd34+1];
mul.lo.s32 %r112, %r100, %r99;
mad.lo.s32 %r113, %r104, %r111, %r112;
mul.hi.s32 %r114, %r113, -2139062143;
add.s32 %r115, %r114, %r113;
shr.u32 %r116, %r115, 31;
shr.u32 %r117, %r115, 7;
add.s32 %r118, %r117, %r116;
st.global.u8 [%rd34+1], %r118;
$L__BB2_19:
ret;
}
+225 -99
View File
@@ -69,6 +69,7 @@ use super::{AuChunk, ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
use anyhow::{anyhow, bail, Context, Result}; use anyhow::{anyhow, bail, Context, Result};
use pf_frame::{CapturedFrame, FramePayload}; use pf_frame::{CapturedFrame, FramePayload};
use pf_zerocopy::cuda::{self, InputSurface}; use pf_zerocopy::cuda::{self, InputSurface};
use pf_zerocopy::vkslot::{SlotFormat, VkSlotBlend, VkSlotRef};
use std::collections::VecDeque; use std::collections::VecDeque;
use std::ffi::c_void; use std::ffi::c_void;
use std::ptr; use std::ptr;
@@ -76,12 +77,6 @@ use std::sync::mpsc;
use nvidia_video_codec_sdk::sys::nvEncodeAPI as nv; use nvidia_video_codec_sdk::sys::nvEncodeAPI as nv;
/// Prebuilt PTX for the cursor-overlay blend kernels (cursor-as-metadata). Source is
/// `cursor_blend.cu` beside this file; regenerate with
/// `nvcc -ptx -arch=compute_75 cursor_blend.cu -o cursor_blend.ptx` after editing. JIT'd by the
/// driver, so it runs on any Turing-or-newer GPU.
const CURSOR_PTX: &[u8] = include_bytes!("cursor_blend.ptx");
// --------------------------------------------------------------------------------------------- // ---------------------------------------------------------------------------------------------
// Runtime-loaded NVENC entry table (Linux). Same shape as the Windows backend's `EncodeApi`, minus // Runtime-loaded NVENC entry table (Linux). Same shape as the Windows backend's `EncodeApi`, minus
// the async-event entry points (Windows-only). Resolved once from `libnvidia-encode.so.1` — the two // the async-event entry points (Windows-only). Resolved once from `libnvidia-encode.so.1` — the two
@@ -428,13 +423,54 @@ fn buffer_format(buf: &cuda::DeviceBuffer) -> nv::NV_ENC_BUFFER_FORMAT {
} }
} }
/// One encoder-owned CUDA input surface + its NVENC registration. The surface is copied into each /// One encoder-owned input surface + its NVENC registration. The surface is copied into each
/// use (device→device) and the registration is created once at session init, unregistered at teardown. /// use (device→device) and the registration is created once at session init, unregistered at teardown.
struct RingSlot { struct RingSlot {
surface: InputSurface, surface: SlotSurface,
reg: nv::NV_ENC_REGISTERED_PTR, reg: nv::NV_ENC_REGISTERED_PTR,
} }
/// The ring slot's backing allocation: Vulkan external memory CUDA-imported (the normal case —
/// blendable by the SPIR-V cursor pass, see `vkslot.rs`) or a plain pitched CUDA allocation (the
/// fallback when Vulkan bring-up fails: sessions still encode, composite mode just has no
/// cursor). Both present the same `(ptr, pitch, height)` NVENC-registration vocabulary.
enum SlotSurface {
Cuda(InputSurface),
/// Backing objects live in the encoder's [`VkSlotBlend`] (freed by its `free_slots`); the
/// ref itself is Copy and carries the registered geometry.
Vk(VkSlotRef),
}
/// The [`SlotFormat`] for an NVENC buffer format (the ring-build + blend vocabulary).
fn slot_fmt_of(fmt: nv::NV_ENC_BUFFER_FORMAT) -> SlotFormat {
match fmt {
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => SlotFormat::Yuv444,
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_NV12 => SlotFormat::Nv12,
_ => SlotFormat::Argb,
}
}
impl SlotSurface {
fn ptr(&self) -> pf_zerocopy::cuda::CUdeviceptr {
match self {
SlotSurface::Cuda(s) => s.ptr,
SlotSurface::Vk(r) => r.ptr,
}
}
fn pitch(&self) -> usize {
match self {
SlotSurface::Cuda(s) => s.pitch,
SlotSurface::Vk(r) => r.pitch,
}
}
fn height(&self) -> u32 {
match self {
SlotSurface::Cuda(s) => s.height,
SlotSurface::Vk(r) => r.height,
}
}
}
/// `doNotWait` sampling cadence inside [`Encoder::poll_chunk`] — the probe measured ~200 µs /// `doNotWait` sampling cadence inside [`Encoder::poll_chunk`] — the probe measured ~200 µs
/// between slice completions on the 5070 Ti, so 50 µs keeps the added per-chunk delivery delay /// between slice completions on the 5070 Ti, so 50 µs keeps the added per-chunk delivery delay
/// well under one slice time without hammering the driver. /// well under one slice time without hammering the driver.
@@ -530,16 +566,23 @@ pub struct NvencCudaEncoder {
split_mode: u32, split_mode: u32,
/// The last reference-frame range we invalidated — dedupes repeated RFI requests for one loss. /// The last reference-frame range we invalidated — dedupes repeated RFI requests for one loss.
last_rfi_range: Option<(i64, i64)>, last_rfi_range: Option<(i64, i64)>,
/// Cursor-as-metadata GPU blend (loaded lazily on the first frame that carries a cursor, once the /// Cursor-as-metadata GPU blend: the Vulkan device + SPIR-V compute pass the ring's
/// CUDA context is current). `None` until then or if the module load fails; `cursor_tried` stops /// external-memory slots are allocated through (`vkslot.rs`) — the driver-portable
/// re-attempting a failed load every frame. `cursor_serial` tracks the uploaded bitmap. /// replacement for the retired PTX kernels. Brought up once at session init (`cursor_tried`
cursor: Option<cuda::CursorBlend>, /// stops re-attempts); `None` = bring-up failed, the ring fell back to plain CUDA
/// allocations and composite mode degrades to no cursor. `cursor_serial` tracks the
/// uploaded bitmap.
vk_blend: Option<VkSlotBlend>,
/// The session may hand this encoder cursor overlays (`SessionPlan.cursor_blend` — only
/// cursor-channel sessions since Phase B). Off = skip the Vulkan bring-up entirely and ring
/// on plain CUDA surfaces: embedded-pointer sessions never carry an overlay, so they pay
/// zero blend cost, per-session or per-frame.
blend_wanted: bool,
cursor_tried: bool, cursor_tried: bool,
cursor_serial: u64, cursor_serial: u64,
/// Suppress-until-success latches for the per-frame cursor upload/blend warns: a persistent /// Suppress-until-success latch for the per-frame blend warn: a persistent failure sits in
/// failure sits in the submit() hot path, so warn once per failure streak (reset on success) /// the submit() hot path, so warn once per failure streak (reset on success) rather than on
/// rather than on every cursor-bearing frame, which would evict the log ring. /// every cursor-bearing frame, which would evict the log ring.
cursor_upload_warned: bool,
cursor_blend_warned: bool, cursor_blend_warned: bool,
/// One-shot latch for [`diagnose_failed_open`](Self::diagnose_failed_open) so a rebuild-retry /// One-shot latch for [`diagnose_failed_open`](Self::diagnose_failed_open) so a rebuild-retry
/// burst (the session loop's bounded encoder resets) logs the diagnosis once, not per attempt. /// burst (the session loop's bounded encoder resets) logs the diagnosis once, not per attempt.
@@ -607,6 +650,7 @@ impl NvencCudaEncoder {
_cuda: bool, _cuda: bool,
bit_depth: u8, bit_depth: u8,
chroma: ChromaFormat, chroma: ChromaFormat,
cursor_blend: bool,
) -> Result<Self> { ) -> Result<Self> {
// The runtime `.so` load is the real "is NVENC possible here" gate: fail the open with a // The runtime `.so` load is the real "is NVENC possible here" gate: fail the open with a
// clear reason instead of an opaque session error on the first frame. // clear reason instead of an opaque session error on the first frame.
@@ -645,10 +689,10 @@ impl NvencCudaEncoder {
frame_idx: 0, frame_idx: 0,
force_kf: false, force_kf: false,
pending_anchor: false, pending_anchor: false,
cursor: None, vk_blend: None,
blend_wanted: cursor_blend,
cursor_tried: false, cursor_tried: false,
cursor_serial: u64::MAX, cursor_serial: u64::MAX,
cursor_upload_warned: false,
cursor_blend_warned: false, cursor_blend_warned: false,
diagnosed: false, diagnosed: false,
inited: false, inited: false,
@@ -739,7 +783,12 @@ impl NvencCudaEncoder {
// (the forfeit contract), and the next session re-latches the arming at init. // (the forfeit contract), and the next session re-latches the arming at init.
self.subframe_chunks = false; self.subframe_chunks = false;
self.chunk = None; self.chunk = None;
self.ring.clear(); // drops the InputSurfaces, freeing their CUDA allocations self.ring.clear(); // drops the CUDA InputSurfaces; Vk slots are freed just below
if let Some(vk) = &mut self.vk_blend {
// The Vulkan-backed slots' memory (and its CUDA mapping) — the device itself stays
// up for the next session's ring (`cursor_tried` keeps bring-up one-shot).
vk.free_slots();
}
self.bitstreams.clear(); self.bitstreams.clear();
self.pending.clear(); self.pending.clear();
self.encoder = ptr::null_mut(); self.encoder = ptr::null_mut();
@@ -1128,36 +1177,102 @@ impl NvencCudaEncoder {
// Encoder-owned input-surface ring: allocate + register POOL surfaces in the negotiated // Encoder-owned input-surface ring: allocate + register POOL surfaces in the negotiated
// format. Registered once here, mapped per submit, unregistered at teardown. // format. Registered once here, mapped per submit, unregistered at teardown.
for _ in 0..POOL { // Preferred backing = Vulkan external memory CUDA-imported (`vkslot.rs`), so the
let surface = match self.buffer_fmt { // SPIR-V cursor blend can composite into the very bytes NVENC encodes; any bring-up
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => { // or per-slot failure falls back to plain pitched CUDA allocations (sessions always
InputSurface::alloc_yuv444(self.width, self.height) // encode — composite mode just loses the cursor, warned below).
} if !self.cursor_tried && self.blend_wanted {
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_NV12 => { self.cursor_tried = true;
InputSurface::alloc_nv12(self.width, self.height) match VkSlotBlend::new() {
} Ok(v) => self.vk_blend = Some(v),
_ => InputSurface::alloc_rgb(self.width, self.height), Err(e) => tracing::warn!(
error = %format!("{e:#}"),
"NVENC (Linux): Vulkan slot-blend bring-up failed — plain CUDA input \
surfaces, cursor compositing unavailable"
),
} }
.context("alloc NVENC input surface")?; }
let mut rr = nv::NV_ENC_REGISTER_RESOURCE { let slot_fmt = slot_fmt_of(self.buffer_fmt);
version: nv::NV_ENC_REGISTER_RESOURCE_VER, // Two attempts: the full ring on Vulkan slots, else (any failure) the full ring on
resourceType: // plain CUDA — never a mixed ring (it would blend on some slots only: a flickering
nv::NV_ENC_INPUT_RESOURCE_TYPE::NV_ENC_INPUT_RESOURCE_TYPE_CUDADEVICEPTR, // cursor) and never a short one.
width: self.width, 'ring: for use_vk in [self.vk_blend.is_some(), false] {
height: self.height, if !use_vk && self.vk_blend.is_some() {
pitch: surface.pitch as u32, // Second attempt: retire the Vulkan side wholesale first.
resourceToRegister: surface.ptr as *mut c_void, for s in self.ring.drain(..) {
bufferFormat: self.buffer_fmt, let _ = (api().unregister_resource)(self.encoder, s.reg);
bufferUsage: nv::NV_ENC_BUFFER_USAGE::NV_ENC_INPUT_IMAGE, }
..Default::default() if let Some(vk) = &mut self.vk_blend {
}; vk.free_slots();
(api().register_resource)(self.encoder, &mut rr) }
.nv_ok() self.vk_blend = None;
.map_err(|e| nvenc_status::call_err("register_resource (CUDADEVICEPTR)", e))?; }
self.ring.push(RingSlot { for _ in 0..POOL {
surface, let surface = if use_vk {
reg: rr.registeredResource, let vk = self.vk_blend.as_mut().expect("use_vk implies Some");
}); match vk.alloc_slot(slot_fmt, self.width, self.height) {
Ok(r) => SlotSurface::Vk(r),
Err(e) => {
tracing::warn!(
error = %format!("{e:#}"),
"NVENC (Linux): Vulkan slot alloc failed — rebuilding the \
ring on plain CUDA surfaces (cursor compositing \
unavailable)"
);
continue 'ring;
}
}
} else {
SlotSurface::Cuda(
match self.buffer_fmt {
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => {
InputSurface::alloc_yuv444(self.width, self.height)
}
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_NV12 => {
InputSurface::alloc_nv12(self.width, self.height)
}
_ => InputSurface::alloc_rgb(self.width, self.height),
}
.context("alloc NVENC input surface")?,
)
};
let mut rr = nv::NV_ENC_REGISTER_RESOURCE {
version: nv::NV_ENC_REGISTER_RESOURCE_VER,
resourceType:
nv::NV_ENC_INPUT_RESOURCE_TYPE::NV_ENC_INPUT_RESOURCE_TYPE_CUDADEVICEPTR,
width: self.width,
height: self.height,
pitch: surface.pitch() as u32,
resourceToRegister: surface.ptr() as *mut c_void,
bufferFormat: self.buffer_fmt,
bufferUsage: nv::NV_ENC_BUFFER_USAGE::NV_ENC_INPUT_IMAGE,
..Default::default()
};
match (api().register_resource)(self.encoder, &mut rr).nv_ok() {
Ok(()) => {}
Err(e) if use_vk => {
// NVENC refusing the imported pointer is a Vulkan-side condition
// too — same wholesale fallback.
tracing::warn!(
error = ?e,
"NVENC (Linux): registering a Vulkan-imported slot failed — \
rebuilding the ring on plain CUDA surfaces"
);
continue 'ring;
}
Err(e) => {
return Err(nvenc_status::call_err(
"register_resource (CUDADEVICEPTR)",
e,
))
}
}
self.ring.push(RingSlot {
surface,
reg: rr.registeredResource,
});
}
break 'ring; // full ring built
} }
self.inited = true; self.inited = true;
@@ -1242,9 +1357,9 @@ impl NvencCudaEncoder {
/// IO-stream binding (stream-ordered submit — see the gate in [`Encoder::submit`]). /// IO-stream binding (stream-ordered submit — see the gate in [`Encoder::submit`]).
fn copy_into_slot(&self, buf: &cuda::DeviceBuffer, slot: usize, sync: bool) -> Result<()> { fn copy_into_slot(&self, buf: &cuda::DeviceBuffer, slot: usize, sync: bool) -> Result<()> {
let s = &self.ring[slot].surface; let s = &self.ring[slot].surface;
let base = s.ptr; let base = s.ptr();
let pitch = s.pitch; let pitch = s.pitch();
let hh = s.height as u64; let hh = s.height() as u64;
match self.buffer_fmt { match self.buffer_fmt {
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => { nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => {
if !buf.yuv444 { if !buf.yuv444 {
@@ -1396,72 +1511,83 @@ impl Encoder for NvencCudaEncoder {
// `async_rt` must be absent too: in two-thread mode the frame may be recycled right after // `async_rt` must be absent too: in two-thread mode the frame may be recycled right after
// submit returns while the stream still holds its copy (belt-and-braces — an escalated // submit returns while the stream still holds its copy (belt-and-braces — an escalated
// session was rebuilt without the binding, so `stream_ordered` is false there anyway). // session was rebuilt without the binding, so `stream_ordered` is false there anyway).
let ordered = self.stream_ordered && self.async_rt.is_none() && self.pending.is_empty(); // Cursor-bearing frames additionally force the CPU-synced path: the Vulkan blend sits
// between the CUDA copy and the encode, and its cross-API ordering is fence/CPU-
// established, not stream-ordered. Frames without a cursor (games hide it; client-draws
// sessions strip it) keep the stream-ordered fast path untouched.
let ordered = self.stream_ordered
&& self.async_rt.is_none()
&& self.pending.is_empty()
&& captured.cursor.is_none();
let t0 = std::time::Instant::now(); let t0 = std::time::Instant::now();
// Copy the captured buffer into this slot's input surface before encoding it. // Copy the captured buffer into this slot's input surface before encoding it.
self.copy_into_slot(buf, slot, !ordered)?; self.copy_into_slot(buf, slot, !ordered)?;
let t_copy = t0.elapsed(); let t_copy = t0.elapsed();
// Cursor-as-metadata: blend the overlay into this slot's OWNED input surface (a tiny kernel // Cursor-as-metadata: blend the overlay into this slot's OWNED input surface via the
// over the cursor's rect — never the compositor's dmabuf). The PTX module loads lazily on the // SPIR-V compute pass (a dispatch over the cursor's rect — never the compositor's
// first cursor frame now that the CUDA context is current; any failure degrades to no cursor, // dmabuf). Cursor-bearing frames forced `ordered = false` above, so the CUDA copy has
// never a dropped frame. // completed before the Vulkan dispatch and the fence-waited dispatch completes before
// the encode below — the cross-API ordering is CPU-established. Any failure degrades to
// no cursor, never a dropped frame.
if let Some(ov) = &captured.cursor { if let Some(ov) = &captured.cursor {
if !self.cursor_tried { if let (Some(vk), SlotSurface::Vk(vref)) =
self.cursor_tried = true; (self.vk_blend.as_mut(), &self.ring[slot].surface)
match cuda::CursorBlend::new(CURSOR_PTX) { {
Ok(cb) => self.cursor = Some(cb),
Err(e) => tracing::warn!(
error = %format!("{e:#}"),
"NVENC (Linux): cursor blend module load failed — cursor not composited"
),
}
}
if let Some(cb) = &self.cursor {
if self.cursor_serial != ov.serial { if self.cursor_serial != ov.serial {
match cb.upload(ov.rgba.as_slice(), ov.w, ov.h) { vk.upload_cursor(ov.rgba.as_slice(), ov.w, ov.h);
Ok(()) => { self.cursor_serial = ov.serial;
self.cursor_serial = ov.serial;
self.cursor_upload_warned = false;
}
Err(e) => {
if !self.cursor_upload_warned {
self.cursor_upload_warned = true;
tracing::warn!(
error = %format!("{e:#}"),
serial = ov.serial,
"NVENC (Linux): cursor upload failed — cursor not composited"
);
}
}
}
} }
let s = &self.ring[slot].surface; // surfW = content width; the blend derives plane strides from the slot's luma
// surfW = content width; surfH = the surface's allocated height (matches // height. Cursor pixels past the content land in cropped padding rows — harmless.
// `copy_into_slot`'s plane math). Cursor pixels past the content are in cropped let r = vk.blend_ref(
// padding rows — harmless. vref,
let (w, h) = (self.width, s.height); slot_fmt_of(self.buffer_fmt),
let r = match self.buffer_fmt { self.width,
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_YUV444 => { ov.w,
cb.blend_yuv444(s.ptr, s.pitch, w, h, ov.w, ov.h, ov.x, ov.y, !ordered) ov.h,
} ov.x,
nv::NV_ENC_BUFFER_FORMAT::NV_ENC_BUFFER_FORMAT_NV12 => { ov.y,
cb.blend_nv12(s.ptr, s.pitch, w, h, ov.w, ov.h, ov.x, ov.y, !ordered) );
}
_ => cb.blend_argb(s.ptr, s.pitch, w, h, ov.w, ov.h, ov.x, ov.y, !ordered),
};
if let Err(e) = r { if let Err(e) = r {
if !self.cursor_blend_warned { if !self.cursor_blend_warned {
self.cursor_blend_warned = true; self.cursor_blend_warned = true;
tracing::warn!( tracing::warn!(
error = %format!("{e:#}"), error = %format!("{e:#}"),
"NVENC (Linux): cursor blend launch failed — cursor not composited" "NVENC (Linux): cursor blend dispatch failed — cursor not composited"
); );
} }
} else { } else {
self.cursor_blend_warned = false; self.cursor_blend_warned = false;
// TEMP KWin composite probe (DROP BEFORE MERGE): prove the blend dispatches
// and with what geometry.
{
use std::sync::atomic::{AtomicU64, Ordering as ProbeOrd};
static PROBE_BLEND: AtomicU64 = AtomicU64::new(0);
let n = PROBE_BLEND.fetch_add(1, ProbeOrd::Relaxed) + 1;
if n == 1 || n % 512 == 0 {
tracing::info!(
n,
fmt = ?self.buffer_fmt,
ov_x = ov.x,
ov_y = ov.y,
ov_w = ov.w,
ov_h = ov.h,
visible = ov.visible,
"cursor blend probe: vulkan dispatch submitted"
);
}
}
} }
} else if !self.cursor_blend_warned {
self.cursor_blend_warned = true;
tracing::warn!(
blend_wanted = self.blend_wanted,
"NVENC (Linux): cursor overlay present but no Vulkan blend (bring-up failed, \
or a non-blend session unexpectedly carried an overlay) cursor not \
composited"
);
} }
} }
@@ -1509,7 +1635,7 @@ impl Encoder for NvencCudaEncoder {
version: nv::NV_ENC_PIC_PARAMS_VER, version: nv::NV_ENC_PIC_PARAMS_VER,
inputWidth: self.width, inputWidth: self.width,
inputHeight: self.height, inputHeight: self.height,
inputPitch: self.ring[slot].surface.pitch as u32, inputPitch: self.ring[slot].surface.pitch() as u32,
inputBuffer: mp.mappedResource, inputBuffer: mp.mappedResource,
bufferFmt: mp.mappedBufferFmt, bufferFmt: mp.mappedBufferFmt,
outputBitstream: self.bitstreams[slot], outputBitstream: self.bitstreams[slot],
+25 -7
View File
@@ -37,9 +37,11 @@ pub(crate) fn fourcc_to_vk(fourcc: u32) -> Option<vk::Format> {
const AR24: u32 = 0x3432_5241; // ARGB8888 const AR24: u32 = 0x3432_5241; // ARGB8888
const XB24: u32 = 0x3432_4258; // XBGR8888 const XB24: u32 = 0x3432_4258; // XBGR8888
const AB24: u32 = 0x3432_4241; // ABGR8888 const AB24: u32 = 0x3432_4241; // ABGR8888
const NV12: u32 = 0x3231_564e; // DRM_FORMAT_NV12
match fourcc { match fourcc {
XR24 | AR24 => Some(vk::Format::B8G8R8A8_UNORM), XR24 | AR24 => Some(vk::Format::B8G8R8A8_UNORM),
XB24 | AB24 => Some(vk::Format::R8G8B8A8_UNORM), XB24 | AB24 => Some(vk::Format::R8G8B8A8_UNORM),
NV12 => Some(vk::Format::G8_B8R8_2PLANE_420_UNORM),
_ => None, _ => None,
} }
} }
@@ -90,9 +92,10 @@ pub(crate) unsafe fn import_rgb_dmabuf(
) )
} }
/// [`import_rgb_dmabuf`] with the image usage explicit and an optional video-profile list /// [`import_rgb_dmabuf`] with the image usage explicit and an optional video-profile list.
/// (chained into the image create) — the RGB-direct encode path imports the captured buffer /// Despite the historical name, this also imports gamescope's one-fd LINEAR NV12: the UV
/// as a profiled `VIDEO_ENCODE_SRC` image instead of a sampled one. /// subresource layout comes from the producer's plane-1 chunk when it reported one, falling
/// back to the shared-stride contiguous-plane contract.
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
pub(crate) unsafe fn import_rgb_dmabuf_as( pub(crate) unsafe fn import_rgb_dmabuf_as(
device: &ash::Device, device: &ash::Device,
@@ -108,12 +111,27 @@ pub(crate) unsafe fn import_rgb_dmabuf_as(
use std::os::fd::IntoRawFd; use std::os::fd::IntoRawFd;
let fmt = fourcc_to_vk(d.fourcc) let fmt = fourcc_to_vk(d.fourcc)
.with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?; .with_context(|| format!("unsupported dmabuf fourcc {:#x}", d.fourcc))?;
let plane = [vk::SubresourceLayout::default() let planes: Vec<vk::SubresourceLayout> = if fmt == vk::Format::G8_B8R8_2PLANE_420_UNORM {
.offset(d.offset as u64) let (uv_offset, uv_stride) = d.plane1.map(|(o, s)| (o as u64, s as u64)).unwrap_or((
.row_pitch(d.stride as u64)]; d.offset as u64 + d.stride as u64 * ch as u64,
d.stride as u64,
));
vec![
vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64),
vk::SubresourceLayout::default()
.offset(uv_offset)
.row_pitch(uv_stride),
]
} else {
vec![vk::SubresourceLayout::default()
.offset(d.offset as u64)
.row_pitch(d.stride as u64)]
};
let mut drm = vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default() let mut drm = vk::ImageDrmFormatModifierExplicitCreateInfoEXT::default()
.drm_format_modifier(d.modifier) .drm_format_modifier(d.modifier)
.plane_layouts(&plane); .plane_layouts(&planes);
let mut ext = vk::ExternalMemoryImageCreateInfo::default() let mut ext = vk::ExternalMemoryImageCreateInfo::default()
.handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT); .handle_types(vk::ExternalMemoryHandleTypeFlags::DMA_BUF_EXT);
let mut ci = vk::ImageCreateInfo::default() let mut ci = vk::ImageCreateInfo::default()
+498 -137
View File
@@ -14,7 +14,7 @@ use super::vk_util::{color_range, find_mem, make_plain_image, make_view, pixel_t
use crate::{Codec, EncodedFrame, Encoder, EncoderCaps}; use crate::{Codec, EncodedFrame, Encoder, EncoderCaps};
use anyhow::{bail, Context, Result}; use anyhow::{bail, Context, Result};
use ash::vk; use ash::vk;
use pf_frame::{CapturedFrame, FramePayload}; use pf_frame::{CapturedFrame, FramePayload, PixelFormat};
use std::collections::VecDeque; use std::collections::VecDeque;
use std::ffi::c_void; use std::ffi::c_void;
use std::os::fd::AsRawFd; use std::os::fd::AsRawFd;
@@ -84,17 +84,36 @@ fn rgb_request() -> Option<bool> {
} }
} }
/// True-extent RGB-direct at unaligned modes (default ON; `PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0`
/// restores the padded-copy staging): direct-import the visible-size capture with the TRUE-SIZE
/// source `codedExtent` — RADV derives nonzero VCN firmware padding from it, so the EFC is told
/// the source lacks the alignment rows (see [`RgbDirect::true_extent`]). Guarded-tested on Van
/// Gogh 2026-07-21 (kernel clean, and the fastest 1080p encode path measured); the EFC only
/// exists on Mesa ≥ 26, where the `codedExtent`-driven `session_init` is guaranteed.
fn rgb_true_extent_request() -> bool {
std::env::var("PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT").as_deref() != Ok("0")
}
/// Live RGB-direct session config: the chroma-siting bits the session was created with /// Live RGB-direct session config: the chroma-siting bits the session was created with
/// (chosen from what the driver advertises — see [`probe_rgb_direct`]). /// (chosen from what the driver advertises — see [`probe_rgb_direct`]).
struct RgbDirect { struct RgbDirect {
x_offset: u32, // vk_valve_rgb::CHROMA_OFFSET_* x_offset: u32, // vk_valve_rgb::CHROMA_OFFSET_*
y_offset: u32, y_offset: u32,
/// The mode is not 64x16-aligned, so the captured buffer cannot be the encode source /// The mode is not 64x16-aligned, so the captured buffer cannot be the encode source
/// directly (the EFC would read past it — the 2026-07-20 field GPU hang). Instead each /// under the session's ALIGNED source extent (the EFC read past it — the 2026-07-20 field
/// frame is copied into a per-slot ALIGNED BGRA staging image with the edge rows/columns /// GPU hang, when the source `codedExtent` was the aligned size and RADV therefore derived
/// duplicated into the padding (transfer-only, no shader) and encoded from there. Aligned /// ZERO firmware padding). Each frame is copied into a per-slot ALIGNED BGRA staging image
/// modes keep the true zero-copy import. /// with the edge rows/columns duplicated into the padding (transfer-only, no shader) and
/// encoded from there. Aligned modes keep the true zero-copy import.
padded: bool, padded: bool,
/// The default unaligned-mode source strategy (`PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0` falls
/// back to `padded`): direct-import the visible-size buffer and pass the TRUE-SIZE source
/// `codedExtent` — RADV then programs nonzero firmware padding from it (Mesa ≥ 24.2
/// derives `session_init` padding from `srcPictureResource.codedExtent`; see
/// [`VulkanVideoEncoder::native_nv12`]), telling the VCN the source lacks the alignment
/// rows, which the hardware edge-extends internally. The session/SPS/DPB stay app-aligned.
/// Guarded-tested on Van Gogh (kernel clean; fastest 1080p path measured).
true_extent: bool,
} }
/// Stack storage for a complete rgb-chained video profile. Profiled image creation AFTER open /// Stack storage for a complete rgb-chained video profile. Profiled image creation AFTER open
@@ -155,6 +174,51 @@ impl RgbProfileStack {
} }
} }
/// Non-RGB video profile rebuilt for native NV12 DMA-BUF imports. Image creation after `open`
/// must carry a profile identical by value to the session profile.
struct NativeProfileStack {
usage: vk::VideoEncodeUsageInfoKHR<'static>,
h265: vk::VideoEncodeH265ProfileInfoKHR<'static>,
av1: super::vk_av1_encode::VideoEncodeAV1ProfileInfoKHR,
profile: vk::VideoProfileInfoKHR<'static>,
}
impl NativeProfileStack {
fn new(codec_op: vk::VideoCodecOperationFlagsKHR) -> Self {
use super::vk_av1_encode as av1b;
Self {
usage: vk::VideoEncodeUsageInfoKHR::default()
.video_usage_hints(vk::VideoEncodeUsageFlagsKHR::STREAMING)
.video_content_hints(vk::VideoEncodeContentFlagsKHR::RENDERED)
.tuning_mode(vk::VideoEncodeTuningModeKHR::ULTRA_LOW_LATENCY),
h265: vk::VideoEncodeH265ProfileInfoKHR::default().std_profile_idc(
vk::native::StdVideoH265ProfileIdc_STD_VIDEO_H265_PROFILE_IDC_MAIN,
),
av1: av1b::VideoEncodeAV1ProfileInfoKHR {
s_type: av1b::stype(av1b::ST_PROFILE_INFO),
p_next: std::ptr::null(),
std_profile: vk::native::StdVideoAV1Profile_STD_VIDEO_AV1_PROFILE_MAIN,
},
profile: vk::VideoProfileInfoKHR::default()
.video_codec_operation(codec_op)
.chroma_subsampling(vk::VideoChromaSubsamplingFlagsKHR::TYPE_420)
.luma_bit_depth(vk::VideoComponentBitDepthFlagsKHR::TYPE_8)
.chroma_bit_depth(vk::VideoComponentBitDepthFlagsKHR::TYPE_8),
}
}
fn wire(&mut self, av1: bool) -> &vk::VideoProfileInfoKHR<'static> {
if av1 {
self.av1.p_next = &self.usage as *const _ as *const c_void;
self.profile.p_next = &self.av1 as *const _ as *const c_void;
} else {
self.h265.p_next = &self.usage as *const _ as *const c_void;
self.profile.p_next = &self.h265 as *const _ as *const c_void;
}
&self.profile
}
}
/// The Vulkan codec-operation bit for our codec selection (shared by open and the per-import /// The Vulkan codec-operation bit for our codec selection (shared by open and the per-import
/// profile rebuilds — the two must agree, profile identity is by value). /// profile rebuilds — the two must agree, profile identity is by value).
fn codec_op_for(av1: bool) -> vk::VideoCodecOperationFlagsKHR { fn codec_op_for(av1: bool) -> vk::VideoCodecOperationFlagsKHR {
@@ -173,12 +237,12 @@ enum SrcAcquire {
/// CSC path: `nv12_src` was written by this frame's compute batch (GENERAL layout; the /// CSC path: `nv12_src` was written by this frame's compute batch (GENERAL layout; the
/// csc_sem orders the queues). /// csc_sem orders the queues).
CscGeneral, CscGeneral,
/// RGB-direct, first use of a dmabuf import: acquire from the foreign producer /// First use of a DMA-BUF imported directly as the video source: acquire from the foreign
/// (UNDEFINED preserves the modifier-tiled bytes) with a FOREIGN→encode-family transfer. /// producer (UNDEFINED preserves modifier-backed bytes) with a FOREIGN→encode-family transfer.
RgbFresh, DmabufFresh,
/// RGB-direct, cached import: the image is already VIDEO_ENCODE_SRC; visibility-only /// Cached direct-source import: already VIDEO_ENCODE_SRC; visibility-only barrier for the
/// barrier for the producer's out-of-band rewrite of the bytes. /// producer's out-of-band rewrite of the bytes.
RgbCached, DmabufCached,
/// RGB-direct CPU upload: the compute queue copied the staging buffer in (semaphore /// RGB-direct CPU upload: the compute queue copied the staging buffer in (semaphore
/// ordered); transition TRANSFER_DST → VIDEO_ENCODE_SRC. /// ordered); transition TRANSFER_DST → VIDEO_ENCODE_SRC.
CpuUpload, CpuUpload,
@@ -376,18 +440,33 @@ pub struct VulkanVideoEncoder {
/// `ENCODE_QUALITY_LEVEL` control and baked into the session-parameters object (the spec /// `ENCODE_QUALITY_LEVEL` control and baked into the session-parameters object (the spec
/// requires the two to match). /// requires the two to match).
quality_level: u32, quality_level: u32,
/// `PUNKTFUNK_PERF` CSC/encode split: >0 ⇒ per-frame GPU timestamps are recorded around the /// `PUNKTFUNK_PERF` pre-encode split: >0 ⇒ per-frame GPU timestamps bracket either the
/// compute batch and a sampled `csc_us` line is logged; the value is the device timestamp /// RGB→NV12 compute batch or the native-NV12 padded copy. The measured duration is logged
/// period in ns/tick. 0.0 ⇒ off (env unset, or the compute family has no timestamp support). /// separately from the host's fence wait; 0.0 means disabled or unsupported.
ts_period_ns: f64, ts_period_ns: f64,
perf_at: std::time::Instant, // last sampled csc_us log (2 s cadence) perf_at: std::time::Instant,
/// RGB-direct (EFC) session config — `Some` ⇒ the session's picture format is BGRA, frames /// RGB-direct (EFC) session config — `Some` ⇒ the session's picture format is BGRA, frames
/// are handed to the encoder as RGB (dmabuf import or CPU upload) and the VCN front-end does /// are handed to the encoder as RGB (dmabuf import or CPU upload) and the VCN front-end does
/// the CSC; `None` ⇒ the compute-CSC path. Fixed per session (the picture format is baked /// the CSC; `None` ⇒ the compute-CSC path. Fixed per session (the picture format is baked
/// into the video session). /// into the video session).
rgb: Option<RgbDirect>, rgb: Option<RgbDirect>,
/// One-shot warning latch: a cursor bitmap arrived on an RGB-direct session (EFC cannot /// Producer supplied native NV12 rather than packed RGB. EVERY mode encodes the imported
/// composite it — the cursor will be missing from the stream until the CSC path is used). /// visible-size buffer directly — safely, because native sessions use TRUE-SIZE headers:
/// the SPS/sequence header is authored at the render size and every picture resource's
/// `codedExtent` matches it, so RADV programs the VCN with `session_init` extent = the true
/// size and a nonzero `padding_width/height`, and the FIRMWARE edge-extends the alignment
/// padding internally (radv_video_enc.c `radv_enc_session_init`; the driver also rounds the
/// bitstream SPS up itself and compensates with a conformance window —
/// `radv_video_patch_encode_session_parameters`, per the VK_KHR_video_encode_h265 proposal's
/// "implementations may override" clause). The source is never read past its extent — unlike
/// the app-aligned-SPS convention the CSC/RGB paths use, where the coded extent is 64x16-
/// aligned and an undersized direct source is the OOB-read class behind the 2026-07-20 field
/// GPU reset (those paths keep their aligned-size sources/staging).
native_nv12: bool,
/// One-shot warning latch: a cursor bitmap arrived on an RGB-direct or native-NV12 session
/// (neither has a compositing stage — the cursor will be missing from the stream until the
/// CSC path is used).
warned_cursor: bool, warned_cursor: bool,
/// A [`reconfigure_bitrate`](Encoder::reconfigure_bitrate) rate not yet installed in the video /// A [`reconfigure_bitrate`](Encoder::reconfigure_bitrate) rate not yet installed in the video
/// session. The next `record_submit` emits an `ENCODE_RATE_CONTROL` control command carrying it /// session. The next `record_submit` emits an `ENCODE_RATE_CONTROL` control command carrying it
@@ -422,14 +501,24 @@ impl VulkanVideoEncoder {
/// (B2). `PUNKTFUNK_VULKAN_RGB_DIRECT` overrides both ways (see [`rgb_request`]). /// (B2). `PUNKTFUNK_VULKAN_RGB_DIRECT` overrides both ways (see [`rgb_request`]).
pub fn open( pub fn open(
codec: Codec, codec: Codec,
format: PixelFormat,
width: u32, width: u32,
height: u32, height: u32,
fps: u32, fps: u32,
bitrate_bps: u64, bitrate_bps: u64,
cursor_blend: bool, cursor_blend: bool,
) -> Result<Self> { ) -> Result<Self> {
let want_rgb = rgb_request().unwrap_or(!cursor_blend); let native_nv12 = format == PixelFormat::Nv12;
Self::open_opts(codec, width, height, fps, bitrate_bps, want_rgb) let want_rgb = !native_nv12 && rgb_request().unwrap_or(!cursor_blend);
Self::open_opts_inner(
codec,
width,
height,
fps,
bitrate_bps,
want_rgb,
native_nv12,
)
} }
/// `open` with the RGB-direct request explicit instead of read from the env — the smoke /// `open` with the RGB-direct request explicit instead of read from the env — the smoke
@@ -443,6 +532,18 @@ impl VulkanVideoEncoder {
fps: u32, fps: u32,
bitrate_bps: u64, bitrate_bps: u64,
want_rgb: bool, want_rgb: bool,
) -> Result<Self> {
Self::open_opts_inner(codec, width, height, fps, bitrate_bps, want_rgb, false)
}
fn open_opts_inner(
codec: Codec,
width: u32,
height: u32,
fps: u32,
bitrate_bps: u64,
want_rgb: bool,
native_nv12: bool,
) -> Result<Self> { ) -> Result<Self> {
if !matches!(codec, Codec::H265 | Codec::Av1) { if !matches!(codec, Codec::H265 | Codec::Av1) {
bail!("vulkan-encode backend supports HEVC + AV1 only (got {codec:?})"); bail!("vulkan-encode backend supports HEVC + AV1 only (got {codec:?})");
@@ -464,6 +565,7 @@ impl VulkanVideoEncoder {
fps.max(1), fps.max(1),
bitrate_bps.max(1_000_000), bitrate_bps.max(1_000_000),
want_rgb, want_rgb,
native_nv12,
) )
} }
} }
@@ -478,6 +580,7 @@ impl VulkanVideoEncoder {
fps: u32, fps: u32,
bitrate: u64, bitrate: u64,
want_rgb: bool, want_rgb: bool,
native_nv12: bool,
) -> Result<Self> { ) -> Result<Self> {
use super::vk_av1_encode as av1b; use super::vk_av1_encode as av1b;
use super::vk_valve_rgb as vrgb; use super::vk_valve_rgb as vrgb;
@@ -551,46 +654,63 @@ impl VulkanVideoEncoder {
0.0 0.0
}; };
// RGB-direct (EFC) resolution — BEFORE the profile is built, because an active rgb // Resolve the encode source before building the profile: EFC RGB conversion changes
// session changes the profile identity (the rgb-conversion struct rides the chain) and // profile identity; producer-native NV12 uses the ordinary 4:2:0 profile.
// the session's picture format. The probe runs unconditionally: its verdict is the
// field telemetry that decides where B2 can default this on.
let rgb_probe = probe_rgb_direct(&instance, &vq_inst, pd, codec_op, av1);
// ALIGNMENT GATE (field GPU-hang, 2026-07-20): the coded extent is 64x16-aligned but the
// captured dmabuf is only the REAL mode size — handing it to the encoder as the direct
// source makes the VCN's EFC read the alignment-padding rows PAST the end of the buffer.
// At 1920x1080 (coded 1088) that is 8 rows = 61 KB of out-of-bounds reads per frame:
// deterministic VM protection faults, vcn_enc ring timeouts, and — through the stall
// watchdog's rebuild-and-refault loop — a full MODE1 GPU reset with VRAM loss. The CSC
// shader absorbs the padding by clamping reads and duplicating the edge; RGB-direct has
// no such stage. Mode select: an aligned mode (720p/1440p/4K) encodes the imported
// buffer directly (true zero-copy); an unaligned one (1080p!) goes through the
// padded-copy staging image (see [`RgbDirect::padded`]) — transfer-only, still no
// compute CSC.
let aligned = rw == w && rh == h; let aligned = rw == w && rh == h;
let rgb_probe = if native_nv12 {
Err("not-probed(native NV12 source selected)")
} else {
probe_rgb_direct(&instance, &vq_inst, pd, codec_op, av1)
};
let rgb_cfg: Option<RgbDirect> = match (&rgb_probe, want_rgb) { let rgb_cfg: Option<RgbDirect> = match (&rgb_probe, want_rgb) {
(Ok((x, y)), true) => Some(RgbDirect { (Ok((x, y)), true) => {
x_offset: *x, let true_extent = !aligned && rgb_true_extent_request();
y_offset: *y, Some(RgbDirect {
padded: !aligned, x_offset: *x,
}), y_offset: *y,
padded: !aligned && !true_extent,
true_extent,
})
}
_ => None, _ => None,
}; };
tracing::info!( if native_nv12 {
rgb_direct = match (&rgb_probe, want_rgb, &rgb_cfg) { tracing::info!(
(_, _, Some(RgbDirect { padded: false, .. })) => "active", native_nv12 = "active(direct-import)",
(_, _, Some(RgbDirect { padded: true, .. })) => source_width = rw,
"active(padded-copy: mode is not 64x16-aligned — staging blit + edge \ source_height = rh,
duplication instead of the direct import)", fw_padding_width = w - rw,
(Ok(_), false, None) => fw_padding_height = h - rh,
"available(off: PUNKTFUNK_VULKAN_RGB_DIRECT=0, or a cursor-blend session \ "vulkan-encode: producer-native NV12 encode source (true-size headers: the \
=1 forces)", driver aligns the bitstream SPS itself and the firmware edge-extends the \
(Err(e), _, None) => e, padding the source is never read past its extent)"
// (Ok, wanted) always builds Some above. );
(Ok(_), true, None) => unreachable!("rgb gate and cfg disagree"), } else {
}, tracing::info!(
"vulkan-encode: EFC RGB-direct encode source (design/vulkan-rgb-direct-encode.md)" rgb_direct = match (&rgb_probe, want_rgb, &rgb_cfg) {
); (
_,
_,
Some(RgbDirect {
true_extent: true, ..
}),
) =>
"active(true-extent: unaligned mode, direct import with the true-size \
source codedExtent RADV firmware padding covers the alignment rows; \
PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 restores the padded copy)",
(_, _, Some(RgbDirect { padded: false, .. })) => "active",
(_, _, Some(RgbDirect { padded: true, .. })) =>
"active(padded-copy: mode is not 64x16-aligned — staging blit + edge \
duplication instead of the direct import)",
(Ok(_), false, None) =>
"available(off: PUNKTFUNK_VULKAN_RGB_DIRECT=0, or a cursor-blend session \
=1 forces)",
(Err(e), _, None) => e,
(Ok(_), true, None) => unreachable!("rgb gate and cfg disagree"),
},
"vulkan-encode: EFC RGB-direct encode source (design/vulkan-rgb-direct-encode.md)"
);
}
// the encode profile — H265 Main, or AV1 Main; chained raw and uniformly (vendored AV1 + // the encode profile — H265 Main, or AV1 Main; chained raw and uniformly (vendored AV1 +
// rgb structs can't `push_next`, and the chain must match [`RgbProfileStack::wire`]'s // rgb structs can't `push_next`, and the chain must match [`RgbProfileStack::wire`]'s
@@ -820,13 +940,19 @@ impl VulkanVideoEncoder {
// ---- session parameters + header framing (HEVC: VPS/SPS/PPS on keyframes; AV1: a // ---- session parameters + header framing (HEVC: VPS/SPS/PPS on keyframes; AV1: a
// temporal-delimiter OBU per frame + a sequence-header OBU on keyframes) ---- // temporal-delimiter OBU per frame + a sequence-header OBU on keyframes) ----
// Native NV12 authors TRUE-SIZE headers (SPS/seq at the render size, no app-side
// conformance window): RADV rounds the bitstream SPS up itself and, keyed off the
// matching true-size codedExtent, programs the VCN with firmware padding so the source
// is never read past its extent. The CSC/RGB paths keep the app-aligned convention
// (their sources genuinely cover the aligned extent).
let (hdr_w, hdr_h) = if native_nv12 { (rw, rh) } else { (w, h) };
let (params, header, frame_prefix) = if av1 { let (params, header, frame_prefix) = if av1 {
build_parameters_av1( build_parameters_av1(
&device, &device,
&vq_dev, &vq_dev,
session, session,
w, hdr_w,
h, hdr_h,
rw, rw,
rh, rh,
av1_caps.max_level, av1_caps.max_level,
@@ -839,8 +965,8 @@ impl VulkanVideoEncoder {
&vq_dev, &vq_dev,
&venc_dev, &venc_dev,
session, session,
w, hdr_w,
h, hdr_h,
rw, rw,
rh, rh,
quality_level, quality_level,
@@ -996,9 +1122,14 @@ impl VulkanVideoEncoder {
compute_pool, compute_pool,
bs_size, bs_size,
sampler, sampler,
ts_period_ns > 0.0 && rgb_cfg.is_none(), ts_period_ns > 0.0
rgb_cfg.is_none(), && ((rgb_cfg.is_none() && !native_nv12)
rgb_cfg.as_ref().is_some_and(|c| c.padded), || rgb_cfg.as_ref().is_some_and(|c| c.padded)),
rgb_cfg.is_none() && !native_nv12,
rgb_cfg
.as_ref()
.is_some_and(|c| c.padded)
.then_some(vk::Format::B8G8R8A8_UNORM),
guard.frames.last_mut().expect("frame just pushed"), guard.frames.last_mut().expect("frame just pushed"),
)?; )?;
} }
@@ -1052,6 +1183,7 @@ impl VulkanVideoEncoder {
ts_period_ns, ts_period_ns,
perf_at: std::time::Instant::now(), perf_at: std::time::Instant::now(),
rgb: rgb_cfg, rgb: rgb_cfg,
native_nv12,
warned_cursor: false, warned_cursor: false,
pending_bitrate: None, pending_bitrate: None,
width: w, width: w,
@@ -1200,15 +1332,31 @@ impl VulkanVideoEncoder {
} }
} }
/// Import a packed-RGB dmabuf as a VkImage (explicit DRM modifier). CSC sessions import it /// Import a DMA-BUF VkImage with usage/profile matching this session's source mode. Native
/// SAMPLED (the compute shader reads it); RGB-direct sessions import it as a profiled /// NV12 and aligned RGB-direct are profiled `VIDEO_ENCODE_SRC` images. Padded RGB-direct
/// `VIDEO_ENCODE_SRC` — the buffer IS the encode source. Caller destroys. /// imports the producer allocation as transfer-source only.
unsafe fn import_dmabuf( unsafe fn import_dmabuf(
&self, &self,
d: &pf_frame::DmabufFrame, d: &pf_frame::DmabufFrame,
cw: u32, cw: u32,
ch: u32, ch: u32,
) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> { ) -> Result<(vk::Image, vk::DeviceMemory, vk::ImageView)> {
if self.native_nv12 {
let mut ps = NativeProfileStack::new(codec_op_for(self.codec == Codec::Av1));
let profile = *ps.wire(self.codec == Codec::Av1);
let arr = [profile];
let mut plist = vk::VideoProfileListInfoKHR::default().profiles(&arr);
return super::vk_util::import_rgb_dmabuf_as(
&self.device,
&self.ext_fd,
&self.mem_props,
d,
cw,
ch,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR,
Some(&mut plist),
);
}
if self.rgb.as_ref().is_some_and(|r| r.padded) { if self.rgb.as_ref().is_some_and(|r| r.padded) {
// Padded-copy mode: the import is only ever a transfer SOURCE (the blit into the // Padded-copy mode: the import is only ever a transfer SOURCE (the blit into the
// aligned staging image) — plain TRANSFER_SRC, no video profile involved. // aligned staging image) — plain TRANSFER_SRC, no video profile involved.
@@ -1503,8 +1651,21 @@ impl VulkanVideoEncoder {
setup_idx = (setup_idx + 1) % DPB_SLOTS as usize; setup_idx = (setup_idx + 1) % DPB_SLOTS as usize;
} }
// ---- 2..4 diverge by encode source; the RGB-direct twin returns through the shared // ---- 2..4 diverge by encode source; native NV12 and RGB-direct return through the
// bookkeeping tail (design/vulkan-rgb-direct-encode.md B1) ---- // shared bookkeeping tail ----
if self.native_nv12 {
self.record_submit_nv12(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?;
self.post_submit_bookkeeping(
slot,
frame.pts_ns,
wire,
is_idr,
recovery,
setup_idx,
poc,
);
return Ok(());
}
if self.rgb.is_some() { if self.rgb.is_some() {
self.record_submit_rgb(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?; self.record_submit_rgb(slot, frame, is_idr, recovery, ref_slot, setup_idx, poc)?;
self.post_submit_bookkeeping( self.post_submit_bookkeeping(
@@ -1831,12 +1992,20 @@ impl VulkanVideoEncoder {
Ok(()) Ok(())
} }
/// Padded-copy blit (unaligned-mode RGB-direct): record — into `compute_cmd` — the visible /// Padded-copy blit (unaligned-mode RGB-direct or native NV12): record — into `compute_cmd`
/// frame copy from the imported capture image into the aligned staging image, plus the edge /// — the visible frame copy from the imported capture image into the aligned staging image,
/// duplication into the 64x16 padding (the same edge semantics the CSC shader implements /// plus the edge duplication into the 64x16 padding (the same edge semantics the CSC shader
/// with clamped reads). Transfer-only, no shader. The staging image lives in GENERAL — it /// implements with clamped reads). Transfer-only, no shader. The staging image lives in
/// is both copy dst and, for the right-column pass, copy src — and the encode acquires it /// GENERAL — it is both copy dst and, for the right-column pass, copy src — and the encode
/// via [`SrcAcquire::CscGeneral`] (content produced on the compute queue, csc_sem-ordered). /// acquires it via [`SrcAcquire::CscGeneral`] (content produced on the compute queue,
/// csc_sem-ordered).
///
/// `planes` lists the copy aspects with their subsampling divisor — `[(COLOR, 1)]` for
/// packed RGB, `[(PLANE_0, 1), (PLANE_1, 2)]` for NV12 (multi-planar copy regions are in
/// each plane's own coordinate space; barriers on non-disjoint images stay COLOR-aspect).
/// Every divisor must divide the visible and aligned extents (4:2:0 frames are even, the
/// coded extent is 64x16-aligned).
#[allow(clippy::too_many_arguments)]
unsafe fn record_pad_blit( unsafe fn record_pad_blit(
&self, &self,
dev: &ash::Device, dev: &ash::Device,
@@ -1844,6 +2013,8 @@ impl VulkanVideoEncoder {
src: vk::Image, src: vk::Image,
src_fresh: bool, src_fresh: bool,
pad: vk::Image, pad: vk::Image,
ts_pool: vk::QueryPool,
planes: &[(vk::ImageAspectFlags, u32)],
) -> Result<()> { ) -> Result<()> {
let (rw, rh) = (self.render_w, self.render_h); let (rw, rh) = (self.render_w, self.render_h);
let (w, h) = (self.width, self.height); let (w, h) = (self.width, self.height);
@@ -1852,6 +2023,10 @@ impl VulkanVideoEncoder {
&vk::CommandBufferBeginInfo::default() &vk::CommandBufferBeginInfo::default()
.flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT), .flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT),
)?; )?;
if self.ts_period_ns > 0.0 {
dev.cmd_reset_query_pool(compute_cmd, ts_pool, 0, 2);
dev.cmd_write_timestamp2(compute_cmd, vk::PipelineStageFlags2::NONE, ts_pool, 0);
}
// Acquire the imported capture buffer for transfer reads (FOREIGN hand-off on first // Acquire the imported capture buffer for transfer reads (FOREIGN hand-off on first
// import — UNDEFINED preserves the modifier-tiled bytes — visibility-only afterwards), // import — UNDEFINED preserves the modifier-tiled bytes — visibility-only afterwards),
// and the staging image for transfer writes (prior contents discarded). // and the staging image for transfer writes (prior contents discarded).
@@ -1893,26 +2068,32 @@ impl VulkanVideoEncoder {
compute_cmd, compute_cmd,
&vk::DependencyInfo::default().image_memory_barriers(&[src_acq, pad_dst]), &vk::DependencyInfo::default().image_memory_barriers(&[src_acq, pad_dst]),
); );
let layers = vk::ImageSubresourceLayers::default() let region =
.aspect_mask(vk::ImageAspectFlags::COLOR) |aspect: vk::ImageAspectFlags, sx: i32, sy: i32, dx: i32, dy: i32, cw: u32, ch: u32| {
.layer_count(1); let layers = vk::ImageSubresourceLayers::default()
let region = |sx: i32, sy: i32, dx: i32, dy: i32, cw: u32, ch: u32| { .aspect_mask(aspect)
vk::ImageCopy::default() .layer_count(1);
.src_subresource(layers) vk::ImageCopy::default()
.dst_subresource(layers) .src_subresource(layers)
.src_offset(vk::Offset3D { x: sx, y: sy, z: 0 }) .dst_subresource(layers)
.dst_offset(vk::Offset3D { x: dx, y: dy, z: 0 }) .src_offset(vk::Offset3D { x: sx, y: sy, z: 0 })
.extent(vk::Extent3D { .dst_offset(vk::Offset3D { x: dx, y: dy, z: 0 })
width: cw, .extent(vk::Extent3D {
height: ch, width: cw,
depth: 1, height: ch,
}) depth: 1,
}; })
// Pass 1 — from the capture: the visible region, then each bottom padding row as a };
// copy of the last visible row (1080p: 8 such rows). One call, disjoint regions. // Pass 1 — from the capture, per plane: the visible region, then each bottom padding row
let mut regions = vec![region(0, 0, 0, 0, rw, rh)]; // as a copy of the last visible row (1080p: 8 luma + 4 chroma rows). One call, disjoint
for y in rh..h { // regions.
regions.push(region(0, rh as i32 - 1, 0, y as i32, rw, 1)); let mut regions = Vec::new();
for &(aspect, div) in planes {
let (rw, rh, h) = (rw / div, rh / div, h / div);
regions.push(region(aspect, 0, 0, 0, 0, rw, rh));
for y in rh..h {
regions.push(region(aspect, 0, rh as i32 - 1, 0, y as i32, rw, 1));
}
} }
dev.cmd_copy_image( dev.cmd_copy_image(
compute_cmd, compute_cmd,
@@ -1942,9 +2123,13 @@ impl VulkanVideoEncoder {
compute_cmd, compute_cmd,
&vk::DependencyInfo::default().image_memory_barriers(&[self_dep]), &vk::DependencyInfo::default().image_memory_barriers(&[self_dep]),
); );
let cols: Vec<vk::ImageCopy> = (rw..w) let mut cols = Vec::new();
.map(|x| region(rw as i32 - 1, 0, x as i32, 0, 1, h)) for &(aspect, div) in planes {
.collect(); let (rw, w, h) = (rw / div, w / div, h / div);
for x in rw..w {
cols.push(region(aspect, rw as i32 - 1, 0, x as i32, 0, 1, h));
}
}
dev.cmd_copy_image( dev.cmd_copy_image(
compute_cmd, compute_cmd,
pad, pad,
@@ -1954,10 +2139,112 @@ impl VulkanVideoEncoder {
&cols, &cols,
); );
} }
if self.ts_period_ns > 0.0 {
dev.cmd_write_timestamp2(
compute_cmd,
vk::PipelineStageFlags2::ALL_COMMANDS,
ts_pool,
1,
);
}
dev.end_command_buffer(compute_cmd)?; dev.end_command_buffer(compute_cmd)?;
Ok(()) Ok(())
} }
/// Producer-native NV12 submit: import the producer's visible-size buffer directly as the
/// encode source. Safe at every mode because native sessions run true-size headers — the
/// picture resources' codedExtent equals the source extent and the VCN firmware edge-extends
/// the alignment padding internally (see the [`Self::native_nv12`] field docs).
#[allow(clippy::too_many_arguments)]
unsafe fn record_submit_nv12(
&mut self,
slot: usize,
frame: &CapturedFrame,
is_idr: bool,
recovery: bool,
ref_slot: usize,
setup_idx: usize,
poc: i32,
) -> Result<()> {
if frame.format != PixelFormat::Nv12 {
bail!(
"vulkan-encode (native NV12): negotiated NV12 but received {:?}",
frame.format
);
}
if frame.width != self.render_w || frame.height != self.render_h {
bail!(
"vulkan-encode (native NV12): frame {}x{} != mode {}x{}",
frame.width,
frame.height,
self.render_w,
self.render_h
);
}
if frame.width % 2 != 0 || frame.height % 2 != 0 {
bail!("vulkan-encode (native NV12): 4:2:0 frame dimensions must be even");
}
let FramePayload::Dmabuf(d) = &frame.payload else {
bail!("vulkan-encode (native NV12): producer frame is not a DMA-BUF");
};
if d.fourcc != pf_frame::drm_fourcc(PixelFormat::Nv12).expect("NV12 FourCC") {
bail!(
"vulkan-encode (native NV12): DMA-BUF FourCC {:#x} is not NV12",
d.fourcc
);
}
if d.modifier != 0 {
bail!(
"vulkan-encode (native NV12): only LINEAR is supported, got modifier {:#x}",
d.modifier
);
}
// No compositing stage exists here (like RGB-direct/EFC): gamescope embeds its pointer
// in the produced pixels, but any other NV12 producer's metadata cursor would be lost —
// say so once instead of silently.
if frame.cursor.is_some() && !self.warned_cursor {
self.warned_cursor = true;
tracing::warn!(
"cursor bitmap on a native-NV12 session — nothing composites it; the cursor \
will be missing from the stream (unset PUNKTFUNK_PIPEWIRE_NV12 for \
metadata-cursor captures)"
);
}
let dev = self.device.clone();
let cmd = self.frames[slot].cmd;
let fence = self.frames[slot].fence;
let query_pool = self.frames[slot].query_pool;
let bs_buf = self.frames[slot].bs_buf;
// The frame-size check above proved the buffer covers the (true-size) coded extent —
// the direct import is the encode source at every mode.
let (src_img, src_view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let acquire = if fresh {
SrcAcquire::DmabufFresh
} else {
SrcAcquire::DmabufCached
};
if self.codec == Codec::Av1 {
self.record_coding_av1(
&dev, cmd, query_pool, bs_buf, src_img, src_view, acquire, is_idr, recovery,
ref_slot, setup_idx, poc,
)?;
} else {
self.record_coding_h265(
&dev, cmd, query_pool, bs_buf, src_img, src_view, acquire, is_idr, ref_slot,
setup_idx, poc,
)?;
}
dev.reset_fences(&[fence])?;
// The whole frame is one submit: the encoder reads the imported NV12 directly.
let ecmds = [cmd];
dev.queue_submit(
self.encode_queue,
&[vk::SubmitInfo::default().command_buffers(&ecmds)],
fence,
)?;
Ok(())
}
/// RGB-direct twin of [`record_submit`]'s steps 24 (step 1 and the bookkeeping tail are /// RGB-direct twin of [`record_submit`]'s steps 24 (step 1 and the bookkeeping tail are
/// shared): resolve the RGB encode source — the imported capture dmabuf itself, or the CPU /// shared): resolve the RGB encode source — the imported capture dmabuf itself, or the CPU
/// staging upload — record the encode, and submit. There is no compute CSC: the VCN EFC /// staging upload — record the encode, and submit. There is no compute CSC: the VCN EFC
@@ -1981,6 +2268,7 @@ impl VulkanVideoEncoder {
let fence = self.frames[slot].fence; let fence = self.frames[slot].fence;
let query_pool = self.frames[slot].query_pool; let query_pool = self.frames[slot].query_pool;
let bs_buf = self.frames[slot].bs_buf; let bs_buf = self.frames[slot].bs_buf;
let ts_pool = self.frames[slot].ts_pool;
// EFC cannot composite the cursor bitmap the metadata-cursor captures hand us — say so // EFC cannot composite the cursor bitmap the metadata-cursor captures hand us — say so
// once instead of silently losing the pointer (gamescope, the flagship, embeds it). // once instead of silently losing the pointer (gamescope, the flagship, embeds it).
if frame.cursor.is_some() && !self.warned_cursor { if frame.cursor.is_some() && !self.warned_cursor {
@@ -1995,25 +2283,33 @@ impl VulkanVideoEncoder {
let (src_img, src_view, acquire, compute_active) = match &frame.payload { let (src_img, src_view, acquire, compute_active) = match &frame.payload {
FramePayload::Dmabuf(d) if !padded => { FramePayload::Dmabuf(d) if !padded => {
// Defense in depth for the OOB class the alignment gate closes at open: the // Defense in depth for the OOB class the alignment gate closes at open: the
// imported buffer must cover the FULL coded extent, or the EFC reads past it // imported buffer must cover the source extent the encode declares — the FULL
// (VM faults → VCN ring hang → GPU reset, the 2026-07-20 field report). A // aligned coded extent normally (or the EFC reads past it: VM faults → VCN
// mismatched frame (mid-flight mode change, odd capture) errors out here and // ring hang → GPU reset, the 2026-07-20 field report), the render size in
// takes the encoder-rebuild path instead of faulting the GPU. // true-extent mode (where the declared source codedExtent shrinks with it and
if frame.width != self.width || frame.height != self.height { // RADV's firmware padding covers the alignment rows). A mismatched frame
// (mid-flight mode change, odd capture) errors out here and takes the
// encoder-rebuild path instead of faulting the GPU.
let (need_w, need_h) = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
(self.render_w, self.render_h)
} else {
(self.width, self.height)
};
if frame.width != need_w || frame.height != need_h {
bail!( bail!(
"vulkan-encode (rgb-direct): frame {}x{} does not cover the coded \ "vulkan-encode (rgb-direct): frame {}x{} does not cover the declared \
extent {}x{} refusing an out-of-bounds encode source", source extent {}x{} refusing an out-of-bounds encode source",
frame.width, frame.width,
frame.height, frame.height,
self.width, need_w,
self.height need_h
); );
} }
let (img, view, fresh) = self.import_cached(d, frame.width, frame.height)?; let (img, view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let acq = if fresh { let acq = if fresh {
SrcAcquire::RgbFresh SrcAcquire::DmabufFresh
} else { } else {
SrcAcquire::RgbCached SrcAcquire::DmabufCached
}; };
(img, view, acq, false) (img, view, acq, false)
} }
@@ -2036,7 +2332,15 @@ impl VulkanVideoEncoder {
let (img, _view, fresh) = self.import_cached(d, frame.width, frame.height)?; let (img, _view, fresh) = self.import_cached(d, frame.width, frame.height)?;
let pad_img = self.frames[slot].pad_img; let pad_img = self.frames[slot].pad_img;
let pad_view = self.frames[slot].pad_view; let pad_view = self.frames[slot].pad_view;
self.record_pad_blit(&dev, compute_cmd, img, fresh, pad_img)?; self.record_pad_blit(
&dev,
compute_cmd,
img,
fresh,
pad_img,
ts_pool,
&[(vk::ImageAspectFlags::COLOR, 1)],
)?;
// The staging image ends the blit in GENERAL with the csc_sem ordering the // The staging image ends the blit in GENERAL with the csc_sem ordering the
// hand-off — exactly the CscGeneral acquire contract. // hand-off — exactly the CscGeneral acquire contract.
(pad_img, pad_view, SrcAcquire::CscGeneral, true) (pad_img, pad_view, SrcAcquire::CscGeneral, true)
@@ -2240,13 +2544,13 @@ impl VulkanVideoEncoder {
.src_stage_mask(vk::PipelineStageFlags2::ALL_COMMANDS) .src_stage_mask(vk::PipelineStageFlags2::ALL_COMMANDS)
.src_access_mask(vk::AccessFlags2::NONE) .src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::GENERAL), .old_layout(vk::ImageLayout::GENERAL),
SrcAcquire::RgbFresh => src_base SrcAcquire::DmabufFresh => src_base
.src_stage_mask(vk::PipelineStageFlags2::NONE) .src_stage_mask(vk::PipelineStageFlags2::NONE)
.src_access_mask(vk::AccessFlags2::NONE) .src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::UNDEFINED) .old_layout(vk::ImageLayout::UNDEFINED)
.src_queue_family_index(vk::QUEUE_FAMILY_FOREIGN_EXT) .src_queue_family_index(vk::QUEUE_FAMILY_FOREIGN_EXT)
.dst_queue_family_index(self.encode_family), .dst_queue_family_index(self.encode_family),
SrcAcquire::RgbCached => src_base SrcAcquire::DmabufCached => src_base
.src_stage_mask(vk::PipelineStageFlags2::NONE) .src_stage_mask(vk::PipelineStageFlags2::NONE)
.src_access_mask(vk::AccessFlags2::NONE) .src_access_mask(vk::AccessFlags2::NONE)
.old_layout(vk::ImageLayout::VIDEO_ENCODE_SRC_KHR), .old_layout(vk::ImageLayout::VIDEO_ENCODE_SRC_KHR),
@@ -2284,9 +2588,29 @@ impl VulkanVideoEncoder {
poc: i32, poc: i32,
) -> Result<()> { ) -> Result<()> {
use ash::vk::native as h; use ash::vk::native as h;
let ext2d = vk::Extent2D { // Setup/reference extent: the aligned size for app-aligned sessions (CSC, RGB — it
width: self.width, // pairs with their aligned SPS), the render size for native NV12's true-size headers.
height: self.height, let ext2d = if self.native_nv12 {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
vk::Extent2D {
width: self.width,
height: self.height,
}
};
// Source extent additionally drops to the render size in RGB true-extent mode: RADV
// derives the VCN firmware padding from srcPictureResource.codedExtent (Mesa ≥ 24.2),
// so the visible-size import is never read past its extent (see RgbDirect::true_extent).
let src_extent = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
ext2d
}; };
let ref_poc = if is_idr { 0 } else { self.slot_poc[ref_slot] }; let ref_poc = if is_idr { 0 } else { self.slot_poc[ref_slot] };
@@ -2458,7 +2782,7 @@ impl VulkanVideoEncoder {
} }
dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty()); dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty());
let src_res = vk::VideoPictureResourceInfoKHR::default() let src_res = vk::VideoPictureResourceInfoKHR::default()
.coded_extent(ext2d) .coded_extent(src_extent)
.image_view_binding(src_view); .image_view_binding(src_view);
let mut enc = vk::VideoEncodeInfoKHR::default() let mut enc = vk::VideoEncodeInfoKHR::default()
.dst_buffer(bs_buf) .dst_buffer(bs_buf)
@@ -2502,9 +2826,29 @@ impl VulkanVideoEncoder {
) -> Result<()> { ) -> Result<()> {
use super::vk_av1_encode as av1; use super::vk_av1_encode as av1;
use ash::vk::native as h; use ash::vk::native as h;
let ext2d = vk::Extent2D { // Setup/reference extent: the aligned size for app-aligned sessions (CSC, RGB — it
width: self.width, // pairs with their aligned SPS), the render size for native NV12's true-size headers.
height: self.height, let ext2d = if self.native_nv12 {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
vk::Extent2D {
width: self.width,
height: self.height,
}
};
// Source extent additionally drops to the render size in RGB true-extent mode: RADV
// derives the VCN firmware padding from srcPictureResource.codedExtent (Mesa ≥ 24.2),
// so the visible-size import is never read past its extent (see RgbDirect::true_extent).
let src_extent = if self.rgb.as_ref().is_some_and(|r| r.true_extent) {
vk::Extent2D {
width: self.render_w,
height: self.render_h,
}
} else {
ext2d
}; };
// ---- required AV1 frame sub-structs (single tile; no CDEF/LR/segmentation/global-motion) ---- // ---- required AV1 frame sub-structs (single tile; no CDEF/LR/segmentation/global-motion) ----
@@ -2726,7 +3070,7 @@ impl VulkanVideoEncoder {
} }
dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty()); dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty());
let src_res = vk::VideoPictureResourceInfoKHR::default() let src_res = vk::VideoPictureResourceInfoKHR::default()
.coded_extent(ext2d) .coded_extent(src_extent)
.image_view_binding(src_view); .image_view_binding(src_view);
let mut enc = vk::VideoEncodeInfoKHR::default() let mut enc = vk::VideoEncodeInfoKHR::default()
.dst_buffer(bs_buf) .dst_buffer(bs_buf)
@@ -2784,10 +3128,13 @@ impl VulkanVideoEncoder {
); );
} }
let (off, len) = (off64 as usize, len64 as usize); let (off, len) = (off64 as usize, len64 as usize);
// PUNKTFUNK_PERF CSC split (best-effort): the fence signaled, so the compute batch that // PUNKTFUNK_PERF pre-encode split (best-effort): the fence signaled, so the compute/transfer
// wrote these timestamps completed long ago — WAIT is a formality. Sampled to one log // timestamps are available. This is a device duration, not permission to label the
// line per ~2 s; `wait_us` in the pump's stage perf minus this ≈ the ASIC encode. // remaining host fence wait as pure VCN time; queueing and synchronization remain in it.
if self.ts_period_ns > 0.0 && f.ts_pool != vk::QueryPool::null() { if self.ts_period_ns > 0.0
&& f.ts_pool != vk::QueryPool::null()
&& self.perf_at.elapsed() >= std::time::Duration::from_secs(2)
{
let mut ts = [0u64; 2]; let mut ts = [0u64; 2];
if dev if dev
.get_query_pool_results( .get_query_pool_results(
@@ -2797,16 +3144,26 @@ impl VulkanVideoEncoder {
vk::QueryResultFlags::TYPE_64 | vk::QueryResultFlags::WAIT, vk::QueryResultFlags::TYPE_64 | vk::QueryResultFlags::WAIT,
) )
.is_ok() .is_ok()
&& self.perf_at.elapsed() >= std::time::Duration::from_secs(2)
{ {
self.perf_at = std::time::Instant::now(); self.perf_at = std::time::Instant::now();
let csc_us = (ts[1].saturating_sub(ts[0]) as f64 * self.ts_period_ns) / 1000.0; let pre_encode_us =
tracing::info!( (ts[1].saturating_sub(ts[0]) as f64 * self.ts_period_ns) / 1000.0;
csc_us = format!("{csc_us:.0}"), if self.rgb.as_ref().is_some_and(|r| r.padded) {
au_bytes = len, tracing::info!(
"vulkan-encode split (sampled): csc=GPU compute batch (import barriers + \ rgb_copy_us = format!("{pre_encode_us:.0}"),
CSC + plane copies); ASIC encode stage-perf wait_us csc" au_bytes = len,
); "vulkan-encode split (sampled): padded RGB copy device time before EFC; \
remaining fence wait still includes queue synchronization + RGBYUV EFC \
+ video encode"
);
} else {
tracing::info!(
csc_us = format!("{pre_encode_us:.0}"),
au_bytes = len,
"vulkan-encode split (sampled): RGB→NV12 compute batch device time; \
remaining fence wait still includes queue synchronization + video encode"
);
}
} }
} }
let f = &self.frames[slot]; let f = &self.frames[slot];
@@ -3412,26 +3769,30 @@ unsafe fn make_frame(
sampler: vk::Sampler, sampler: vk::Sampler,
with_ts: bool, with_ts: bool,
csc: bool, csc: bool,
rgb_pad: bool, pad_fmt: Option<vk::Format>,
f: &mut Frame, f: &mut Frame,
) -> Result<()> { ) -> Result<()> {
// "no cursor uploaded yet" sentinel — a real serial may be 0 (see `prep_cursor`). // "no cursor uploaded yet" sentinel — a real serial may be 0 (see `prep_cursor`).
f.cursor_serial = u64::MAX; f.cursor_serial = u64::MAX;
// Padded-copy RGB staging (unaligned-mode RGB-direct): aligned BGRA encode-src filled by a // Padded-copy staging (unaligned-mode RGB-direct or native NV12): an aligned encode-src in
// transfer blit each frame — concurrent compute (copy) + encode (source read). // the session's picture format, filled by a transfer blit each frame — concurrent compute
if rgb_pad { // (copy) + encode (source read). TRANSFER_SRC because the width-padding pass self-copies the
// staging image's own last visible column (see `record_pad_blit`).
if let Some(fmt) = pad_fmt {
(f.pad_img, f.pad_mem) = make_video_image( (f.pad_img, f.pad_mem) = make_video_image(
device, device,
mem_props, mem_props,
vk::Format::B8G8R8A8_UNORM, fmt,
w, w,
h, h,
1, 1,
vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR | vk::ImageUsageFlags::TRANSFER_DST, vk::ImageUsageFlags::VIDEO_ENCODE_SRC_KHR
| vk::ImageUsageFlags::TRANSFER_DST
| vk::ImageUsageFlags::TRANSFER_SRC,
profile_list, profile_list,
fams, fams,
)?; )?;
f.pad_view = make_view(device, f.pad_img, vk::Format::B8G8R8A8_UNORM, 0)?; f.pad_view = make_view(device, f.pad_img, fmt, 0)?;
} }
// RGB-direct sessions never touch the CSC pipeline: no NV12 encode-src, no Y/UV scratch, no // RGB-direct sessions never touch the CSC pipeline: no NV12 encode-src, no Y/UV scratch, no
// cursor overlay, no descriptor set — the encode source is the imported RGB itself (or the // cursor overlay, no descriptor set — the encode source is the imported RGB itself (or the
+260 -8
View File
@@ -478,10 +478,14 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
b b
}); });
// HDR signalling (10-bit sessions are the HDR path on Windows — same coupling as NVENC): // Colour signalling, written UNCONDITIONALLY (mirrors nvenc_core.rs): the input is already
// BT.2020/PQ colour description + the source's mastering/CLL grade at every IDR. // CSC'd to a specific matrix — BT.709 limited for SDR (the capture-side VideoConverter),
// BT.2020 PQ for HDR (HdrP010Converter) — so the stream must say so. An SDR stream without a
// colour description leaves the choice to the decoder's "unspecified" default, and
// Moonlight/third-party/Android-vendor decoders default to 601 at sub-HD → mis-rendered
// colours. (10-bit sessions are the HDR path on Windows — same coupling as NVENC.)
let hdr = cfg.ten_bit && cfg.codec != Codec::H264; let hdr = cfg.ten_bit && cfg.codec != Codec::H264;
let vsi = hdr.then(|| { let vsi = {
// SAFETY: all-zero is valid; header stamped below. // SAFETY: all-zero is valid; header stamped below.
let mut b: Box<vpl::mfxExtVideoSignalInfo> = Box::new(unsafe { std::mem::zeroed() }); let mut b: Box<vpl::mfxExtVideoSignalInfo> = Box::new(unsafe { std::mem::zeroed() });
b.Header.BufferId = vpl::MFX_EXTBUFF_VIDEO_SIGNAL_INFO as u32; b.Header.BufferId = vpl::MFX_EXTBUFF_VIDEO_SIGNAL_INFO as u32;
@@ -489,11 +493,17 @@ fn build_params(cfg: &EncodeConfig) -> ParamSet {
b.VideoFormat = 5; // unspecified b.VideoFormat = 5; // unspecified
b.VideoFullRange = 0; b.VideoFullRange = 0;
b.ColourDescriptionPresent = 1; b.ColourDescriptionPresent = 1;
b.ColourPrimaries = 9; // BT.2020 if hdr {
b.TransferCharacteristics = 16; // SMPTE ST 2084 (PQ) b.ColourPrimaries = 9; // BT.2020
b.MatrixCoefficients = 9; // BT.2020 non-constant b.TransferCharacteristics = 16; // SMPTE ST 2084 (PQ)
b b.MatrixCoefficients = 9; // BT.2020 non-constant
}); } else {
b.ColourPrimaries = 1; // BT.709
b.TransferCharacteristics = 1; // BT.709
b.MatrixCoefficients = 1; // BT.709
}
Some(b)
};
let mastering = cfg.hdr_meta.filter(|_| hdr).map(|m| { let mastering = cfg.hdr_meta.filter(|_| hdr).map(|m| {
// SAFETY: all-zero is valid; header stamped below. // SAFETY: all-zero is valid; header stamped below.
let mut b: Box<vpl::mfxExtMasteringDisplayColourVolume> = let mut b: Box<vpl::mfxExtMasteringDisplayColourVolume> =
@@ -1994,4 +2004,246 @@ mod tests {
"the bitrate retarget emitted a keyframe (StartNewSequence leak)" "the bitrate retarget emitted a keyframe (StartNewSequence leak)"
); );
} }
/// FULL-CHAIN colour check at the field capture size: a known P010 colour-bar source at
/// 1920x1080 — whose height is NOT 16-aligned, so the ingest `CopySubresourceRegion` copies
/// into a 1920x1088 runtime pool surface whose chroma plane sits at a DIFFERENT row offset
/// than the source's (the seam no 640x480 test exercises) — encoded to Main10 HEVC and
/// dumped to `%TEMP%\pf_qsv_1080_bars.h265` for off-box decode verification against the
/// same codes. On-box this asserts stream shape; the pixel verdict needs a decoder.
#[test]
fn qsv_live_p010_1080_colorbars_dump() {
use windows::Win32::Graphics::Direct3D::D3D_DRIVER_TYPE_UNKNOWN;
use windows::Win32::Graphics::Direct3D11::{
D3D11CreateDevice, D3D11_BIND_RENDER_TARGET, D3D11_BIND_SHADER_RESOURCE,
D3D11_SDK_VERSION, D3D11_SUBRESOURCE_DATA, D3D11_USAGE_DEFAULT,
};
use windows::Win32::Graphics::Dxgi::Common::{DXGI_FORMAT_P010, DXGI_SAMPLE_DESC};
use windows::Win32::Graphics::Dxgi::{CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4};
// (Y, Cb, Cr) 10-bit limited codes for the 8 sRGB bars white/yellow/cyan/green/magenta/
// red/blue/black at 80-nit SDR white under PQ/BT.2020 — the same math as pf-capture's
// `p010_reference` (and the bars_pq2020 client fixture). Stored MSB-aligned (`<<6`).
const BARS: [(u16, u16, u16); 8] = [
(490, 512, 512),
(478, 423, 518),
(464, 525, 473),
(450, 432, 476),
(350, 584, 585),
(325, 448, 598),
(226, 650, 535),
(64, 512, 512),
];
const W: u32 = 1920;
const H: u32 = 1080;
init_tracing();
let Ok((_l, impls)) = intel_loader() else {
eprintln!("skipping: no VPL loader");
return;
};
let Some(imp) = impls.iter().find(|i| i.luid_valid) else {
eprintln!("skipping: no Intel VPL implementation on this box");
return;
};
if !probe_can_encode_10bit(Codec::H265) {
eprintln!("skipping: this GPU declines 10-bit HEVC");
return;
}
// P010 initial data: plane 0 = H rows of W u16 luma; plane 1 = H/2 rows of W u16
// (interleaved Cb,Cr pairs), same pitch. Bars are vertical: bar index = x / (W/8).
let bar_w = (W / 8) as usize;
let mut init = vec![0u16; (W as usize) * (H as usize + H as usize / 2)];
for y in 0..H as usize {
for x in 0..W as usize {
init[y * W as usize + x] = BARS[(x / bar_w).min(7)].0 << 6;
}
}
let chroma_base = (W as usize) * (H as usize);
for cy in 0..(H as usize / 2) {
for cx in 0..(W as usize / 2) {
let (_, cb, cr) = BARS[((cx * 2) / bar_w).min(7)];
init[chroma_base + cy * W as usize + cx * 2] = cb << 6;
init[chroma_base + cy * W as usize + cx * 2 + 1] = cr << 6;
}
}
// SAFETY: self-contained harness on one thread/device (same contract as `drive_live`);
// the initial-data pointer outlives the synchronous CreateTexture2D that reads it.
let (device, tex) = unsafe {
let luid = windows::Win32::Foundation::LUID {
LowPart: u32::from_le_bytes(imp.luid[..4].try_into().unwrap()),
HighPart: i32::from_le_bytes(imp.luid[4..].try_into().unwrap()),
};
let factory: IDXGIFactory4 = CreateDXGIFactory1().expect("dxgi factory");
let adapter: IDXGIAdapter1 = factory.EnumAdapterByLuid(luid).expect("intel adapter");
let mut device = None;
D3D11CreateDevice(
&adapter,
D3D_DRIVER_TYPE_UNKNOWN,
windows::Win32::Foundation::HMODULE::default(),
Default::default(),
None,
D3D11_SDK_VERSION,
Some(&mut device),
None,
None,
)
.expect("d3d11 device on intel adapter");
let device: ID3D11Device = device.expect("device");
let desc = D3D11_TEXTURE2D_DESC {
Width: W,
Height: H,
MipLevels: 1,
ArraySize: 1,
Format: DXGI_FORMAT_P010,
SampleDesc: DXGI_SAMPLE_DESC {
Count: 1,
Quality: 0,
},
Usage: D3D11_USAGE_DEFAULT,
BindFlags: (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
CPUAccessFlags: 0,
MiscFlags: 0,
};
let data = D3D11_SUBRESOURCE_DATA {
pSysMem: init.as_ptr() as *const std::ffi::c_void,
SysMemPitch: W * 2,
SysMemSlicePitch: 0,
};
let mut t: Option<ID3D11Texture2D> = None;
device
.CreateTexture2D(&desc, Some(&data), Some(&mut t))
.expect("bar texture");
(device.clone(), t.expect("texture"))
};
let mut enc = QsvEncoder::open(
Codec::H265,
PixelFormat::P010,
W,
H,
30,
10_000_000,
10,
ChromaFormat::Yuv420,
)
.expect("open");
enc.set_hdr_meta(Some(test_hdr_meta()));
let mut stream = Vec::new();
let mut aus = 0usize;
let mut keyframes = 0usize;
for i in 0..12u32 {
let frame = CapturedFrame {
width: W,
height: H,
pts_ns: i as u64 * 33_333_333,
format: PixelFormat::P010,
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
enc.submit_indexed(&frame, i).expect("submit");
if let Some(au) = enc.poll().expect("poll") {
aus += 1;
keyframes += au.keyframe as usize;
stream.extend_from_slice(&au.data);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("drain") {
aus += 1;
keyframes += au.keyframe as usize;
stream.extend_from_slice(&au.data);
}
assert!(aus >= 10, "expected ≥10 AUs, got {aus}");
assert!(keyframes >= 1, "expected an IDR in the dump");
let path = std::env::temp_dir().join("pf_qsv_1080_bars.h265");
std::fs::write(&path, &stream).expect("write dump");
println!(
"wrote {} AUs ({} bytes, {keyframes} keyframes) to {}",
aus,
stream.len(),
path.display()
);
}
/// The PRODUCTION host chain minus the IDD ring: the REAL `HdrP010Converter` renders the 8
/// sRGB bars into a ring-profile P010 texture (`BIND_RENDER_TARGET` only — RTV-written, not
/// CPU-uploaded) on the VPL implementation's own adapter, and THAT texture goes through the
/// unaligned-height ingest copy into a Main10 encode. Dumped to
/// `%TEMP%\pf_qsv_conv_1080_bars.h265`; expected decode codes = the bars_pq2020 fixture set
/// (see `hdr_p010_convert_bars_on_luid`).
#[test]
fn qsv_live_hdr_converter_e2e_1080_dump() {
const W: u32 = 1920;
const H: u32 = 1080;
init_tracing();
let Ok((_l, impls)) = intel_loader() else {
eprintln!("skipping: no VPL loader");
return;
};
let Some(imp) = impls.iter().find(|i| i.luid_valid) else {
eprintln!("skipping: no Intel VPL implementation on this box");
return;
};
if !probe_can_encode_10bit(Codec::H265) {
eprintln!("skipping: this GPU declines 10-bit HEVC");
return;
}
let (device, tex) = pf_capture::dxgi::hdr_p010_convert_bars_on_luid(imp.luid, W, H)
.expect("converter bars");
let mut enc = QsvEncoder::open(
Codec::H265,
PixelFormat::P010,
W,
H,
30,
10_000_000,
10,
ChromaFormat::Yuv420,
)
.expect("open");
enc.set_hdr_meta(Some(test_hdr_meta()));
let mut stream = Vec::new();
let mut aus = 0usize;
for i in 0..12u32 {
let frame = CapturedFrame {
width: W,
height: H,
pts_ns: i as u64 * 33_333_333,
format: PixelFormat::P010,
payload: FramePayload::D3d11(pf_frame::dxgi::D3d11Frame {
texture: tex.clone(),
device: device.clone(),
pyro: None,
}),
cursor: None,
};
enc.submit_indexed(&frame, i).expect("submit");
if let Some(au) = enc.poll().expect("poll") {
aus += 1;
stream.extend_from_slice(&au.data);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("drain") {
aus += 1;
stream.extend_from_slice(&au.data);
}
assert!(aus >= 10, "expected ≥10 AUs, got {aus}");
let path = std::env::temp_dir().join("pf_qsv_conv_1080_bars.h265");
std::fs::write(&path, &stream).expect("write dump");
println!(
"wrote {aus} AUs ({} bytes) to {}",
stream.len(),
path.display()
);
}
} }
+60 -6
View File
@@ -270,7 +270,7 @@ fn open_video_backend(
chroma: ChromaFormat, chroma: ChromaFormat,
cursor_blend: bool, cursor_blend: bool,
) -> Result<(Box<dyn Encoder>, &'static str)> { ) -> Result<(Box<dyn Encoder>, &'static str)> {
let _ = cursor_blend; // consumed only by the Linux vulkan-encode arm below let _ = cursor_blend; // consumed only by the Linux vulkan-encode + direct-NVENC arms below
validate_dimensions(codec, width, height)?; validate_dimensions(codec, width, height)?;
// Refresh/fps must be positive and sane: fps feeds the encoder time_base (`Rational(1, fps)`) // Refresh/fps must be positive and sane: fps feeds the encoder time_base (`Rational(1, fps)`)
// and the pts→ns conversion (`pts * 1e9 / fps`), so 0 builds a 1/0 rational / divides by zero. // and the pts→ns conversion (`pts * 1e9 / fps`), so 0 builds a 1/0 rational / divides by zero.
@@ -330,6 +330,7 @@ fn open_video_backend(
{ {
match vulkan_video::VulkanVideoEncoder::open( match vulkan_video::VulkanVideoEncoder::open(
codec, codec,
format,
width, width,
height, height,
fps, fps,
@@ -344,12 +345,32 @@ fn open_video_backend(
); );
return Ok((Box::new(e) as Box<dyn Encoder>, "vulkan")); return Ok((Box::new(e) as Box<dyn Encoder>, "vulkan"));
} }
// Native NV12 (PUNKTFUNK_PIPEWIRE_NV12 capture) has no VAAPI fallback:
// libav's dmabuf lane would import the two-plane buffer as packed RGB
// (silent garbage) and its CPU lane bails per frame — die crisply instead.
Err(e) if format == PixelFormat::Nv12 => {
return Err(e.context(
"Vulkan Video open failed on a native-NV12 capture \
no VAAPI fallback exists; set PUNKTFUNK_PIPEWIRE_NV12=0 to \
restore the packed-RGB negotiation",
));
}
Err(e) => tracing::warn!( Err(e) => tracing::warn!(
error = %format!("{e:#}"), error = %format!("{e:#}"),
"Vulkan Video encode open failed — falling back to libav VAAPI" "Vulkan Video encode open failed — falling back to libav VAAPI"
), ),
} }
} }
// Same rule when the Vulkan backend was never eligible (H264 session,
// PUNKTFUNK_VULKAN_ENCODE=0, or a build without the feature).
if format == PixelFormat::Nv12 {
anyhow::bail!(
"native NV12 capture requires the Vulkan Video encoder (HEVC/AV1 \
session, --features vulkan-encode, PUNKTFUNK_VULKAN_ENCODE not 0) this \
session resolved to libav VAAPI; set PUNKTFUNK_PIPEWIRE_NV12=0 to restore \
the packed-RGB negotiation"
);
}
vaapi::VaapiEncoder::open( vaapi::VaapiEncoder::open(
codec, codec,
format, format,
@@ -373,6 +394,7 @@ fn open_video_backend(
cuda, cuda,
bit_depth, bit_depth,
chroma, chroma,
cursor_blend,
) )
.map(|e| (e, "nvenc")) .map(|e| (e, "nvenc"))
}; };
@@ -390,6 +412,7 @@ fn open_video_backend(
} }
vulkan_video::VulkanVideoEncoder::open( vulkan_video::VulkanVideoEncoder::open(
codec, codec,
format,
width, width,
height, height,
fps, fps,
@@ -707,12 +730,15 @@ fn open_nvenc_probed(
cuda: bool, cuda: bool,
bit_depth: u8, bit_depth: u8,
chroma: ChromaFormat, chroma: ChromaFormat,
cursor_blend: bool,
) -> Result<Box<dyn Encoder>> { ) -> Result<Box<dyn Encoder>> {
// Direct-SDK NVENC (design/linux-direct-nvenc.md): the DEFAULT on NVIDIA, and only for a CUDA #[cfg(not(feature = "nvenc"))]
// capture payload (it registers CUDADEVICEPTR inputs — a CPU/dmabuf frame can't feed it, so those let _ = cursor_blend; // consumed by the direct-SDK arm below
// keep the libav path; and `cuda` is false on AMD/Intel, so they stay on VAAPI). Set // Direct-SDK NVENC (design/linux-direct-nvenc.md): the DEFAULT on NVIDIA, and only for a CUDA
// PUNKTFUNK_NVENC_DIRECT=0 to fall back to libav. It self-clamps the bitrate internally (its own // capture payload (it registers CUDADEVICEPTR inputs — a CPU/dmabuf frame can't feed it, so those
// level-ceiling binary search at session open), so it skips the probe-loop stepping below. // keep the libav path; and `cuda` is false on AMD/Intel, so they stay on VAAPI). Set
// PUNKTFUNK_NVENC_DIRECT=0 to fall back to libav. It self-clamps the bitrate internally (its own
// level-ceiling binary search at session open), so it skips the probe-loop stepping below.
#[cfg(feature = "nvenc")] #[cfg(feature = "nvenc")]
if cuda && nvenc_direct_enabled() { if cuda && nvenc_direct_enabled() {
tracing::info!( tracing::info!(
@@ -729,6 +755,7 @@ fn open_nvenc_probed(
cuda, cuda,
bit_depth, bit_depth,
chroma, chroma,
cursor_blend,
)?) as Box<dyn Encoder>); )?) as Box<dyn Encoder>);
} }
// The silent-degrade trap: a build without `--features nvenc` compiles the direct-SDK // The silent-degrade trap: a build without `--features nvenc` compiles the direct-SDK
@@ -819,6 +846,33 @@ fn vulkan_encode_enabled() -> bool {
.unwrap_or(true) .unwrap_or(true)
} }
/// Whether THIS session's encoder can ingest a producer-native NV12 capture: only the raw
/// Vulkan Video backend does (libav VAAPI would misread the two-plane buffer as packed RGB —
/// [`open_video`] refuses the combination), so the session's codec must be one it encodes and
/// the backend must be eligible to open. The host facade threads the verdict into the capture
/// negotiation (`OutputFormat::nv12_native` → `ZeroCopyPolicy::native_nv12_session`), which
/// then PREFERS gamescope's producer-side NV12 pod (default-on; `PUNKTFUNK_PIPEWIRE_NV12=0`
/// escapes at the capture gate).
#[cfg(target_os = "linux")]
pub fn linux_native_nv12_ok(codec: Codec) -> bool {
#[cfg(feature = "vulkan-encode")]
{
matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
// NVENC/PyroWave prefs never open the Vulkan Video backend; every other pref
// (auto/vaapi/amd/intel/vulkan) tries it first on AMD/Intel — see [`open_video`].
&& !matches!(
pf_host_config::config().encoder_pref.as_str(),
"nvenc" | "nvidia" | "cuda" | "pyrowave"
)
}
#[cfg(not(feature = "vulkan-encode"))]
{
let _ = codec;
false
}
}
/// Cheap, side-effect-free NVIDIA-presence probe for the `auto` backend selector: the NVIDIA /// Cheap, side-effect-free NVIDIA-presence probe for the `auto` backend selector: the NVIDIA
/// kernel driver exposes these device nodes, AMD/Intel boxes have neither. Deliberately does NOT /// kernel driver exposes these device nodes, AMD/Intel boxes have neither. Deliberately does NOT
/// create a CUDA context (that would allocate GPU state on every host that merely *might* be /// create a CUDA context (that would allocate GPU state on every host that merely *might* be
+5
View File
@@ -32,6 +32,11 @@ pub struct WinCaptureTarget {
/// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown /// duplicates the sealed frame channel's handles INTO (`idd_push::ChannelBroker`). `0` = unknown
/// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse). /// (a pre-v2 pairing can't occur — the version handshake is hard — so this only guards misuse).
pub wudf_pid: u32, pub wudf_pid: u32,
/// The ADD reply flagged this target as carrying an IRREVOCABLE IddCx hardware-cursor declare
/// from an earlier session (remote-desktop-sweep §8.6): DWM excludes the pointer from its
/// frames forever, so a session WITHOUT the cursor channel must self-composite (the IDD-push
/// capturer's forced-composite gate) or the streamed desktop has no cursor at all.
pub cursor_excluded: bool,
} }
/// The PyroWave (Windows) zero-copy sharing payload attached to a captured frame: the SECOND plane /// The PyroWave (Windows) zero-copy sharing payload attached to a captured frame: the SECOND plane
+49 -10
View File
@@ -105,13 +105,15 @@ pub fn drm_fourcc(format: PixelFormat) -> Option<u32> {
Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888 Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888
Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888 Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888
Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888 Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888
// Linux native NV12 capture (gamescope PipeWire): one LINEAR dmabuf with contiguous Y then
// interleaved UV, exposed under DRM_FORMAT_NV12.
Nv12 => drm_fourcc_code(b"NV12"),
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020). // The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020).
X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010 X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010
X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010 X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010
// 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path. // 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path.
// Rgb10a2/Nv12/P010 are the Windows HDR / video-processor formats — never produced on // Rgb10a2/P010 are Windows formats; Yuv444 is OUR convert output, never a capture source.
// Linux; Yuv444 is OUR convert's OUTPUT, never a capture source format. Rgb | Bgr | Rgb10a2 | P010 | Yuv444 => return None,
Rgb | Bgr | Rgb10a2 | Nv12 | P010 | Yuv444 => return None,
}) })
} }
@@ -144,6 +146,18 @@ pub struct OutputFormat {
/// (never BGRA-passthrough / P010). `false` on every non-PyroWave session and on Linux (the /// (never BGRA-passthrough / P010). `false` on every non-PyroWave session and on Linux (the
/// wavelet encoder ingests dmabufs / CPU RGB there, not a D3D11 texture). /// wavelet encoder ingests dmabufs / CPU RGB there, not a D3D11 texture).
pub pyrowave: bool, pub pyrowave: bool,
/// THIS session's encoder can ingest a producer-native NV12 capture (Linux raw Vulkan Video
/// backend on an H265/AV1 session — see `pf_encode::linux_native_nv12_ok`). The Linux capture
/// negotiation only offers gamescope the NV12 pod when this is set: libav VAAPI (the H264
/// codec's backend, and the fallback family) would misread the two-plane buffer as packed
/// RGB. Always `false` on Windows (the IDD-push capturer owns its own formats).
pub nv12_native: bool,
/// The session negotiated the cursor-forward channel (remote-desktop-sweep M2c): on Windows
/// the IDD-push capturer creates + delivers the driver's hardware-cursor section, so DWM
/// stops compositing the pointer into the frames and the capturer surfaces it via
/// `Capturer::cursor()` instead. Ignored on Linux (the portal's `SPA_META_Cursor` already
/// separates the pointer; the session plan's `cursor_blend` gate handles the rest).
pub hw_cursor: bool,
} }
impl OutputFormat { impl OutputFormat {
@@ -161,6 +175,13 @@ impl OutputFormat {
chroma_444: false, chroma_444: false,
// GameStream never negotiates PyroWave (native punktfunk/1 only). // GameStream never negotiates PyroWave (native punktfunk/1 only).
pyrowave: false, pyrowave: false,
// GameStream/spike sessions never negotiate the cursor channel.
hw_cursor: false,
// Conservative: the GameStream + spike paths don't resolve the codec here, and a
// Moonlight client may negotiate H264 (whose VAAPI backend can't ingest NV12) — so
// they never prefer the producer-native NV12 pod. The punktfunk/1 plane opts in via
// `SessionPlan::output_format()`, which knows the codec.
nv12_native: false,
} }
} }
} }
@@ -183,6 +204,16 @@ pub struct CursorOverlay {
pub rgba: std::sync::Arc<Vec<u8>>, pub rgba: std::sync::Arc<Vec<u8>>,
/// Bumps whenever `rgba`/`w`/`h` change; stable across position-only moves. /// Bumps whenever `rgba`/`w`/`h` change; stable across position-only moves.
pub serial: u64, pub serial: u64,
/// Hotspot (the pixel that IS the pointer position) within `w`×`h`. The blend paths ignore
/// it (`x`/`y` are already hotspot-adjusted); the cursor-forward channel ships it to the
/// client so a locally-drawn OS cursor points with the right pixel.
pub hot_x: u32,
pub hot_y: u32,
/// Compositor-reported pointer visibility. `false` = an app on the host grabbed/hid the
/// pointer — the cursor-forward channel turns that into the client's relative-mode hint
/// (remote-desktop-sweep M3). The encode loop STRIPS invisible overlays before the frame
/// reaches any blend path, so encoders may keep treating `Some` as "draw it".
pub visible: bool,
} }
/// A captured frame. [`format`](Self::format)/dimensions describe the pixels regardless of /// A captured frame. [`format`](Self::format)/dimensions describe the pixels regardless of
@@ -201,18 +232,26 @@ pub struct CapturedFrame {
pub cursor: Option<CursorOverlay>, pub cursor: Option<CursorOverlay>,
} }
/// A captured frame still living in a single-plane packed-RGB dmabuf (the VAAPI zero-copy path). /// A captured frame still living in a DMA-BUF. Packed RGB uses one plane. Native Linux NV12
/// (gamescope PipeWire) travels in ONE fd: Y starts at `offset`, and the interleaved UV plane
/// lives at `plane1`'s offset/stride when the producer reported them — else at the contiguous
/// fallback `offset + stride * frame_height` with the shared `stride`.
///
/// Owns a *dup* of the PipeWire buffer's fd, so the frame can travel to the encode thread and be /// Owns a *dup* of the PipeWire buffer's fd, so the frame can travel to the encode thread and be
/// imported into a VA surface there without the compositor's buffer being closed underneath it. /// imported there without the compositor's buffer being closed underneath it. Content stability
/// (Content stability across the brief import window relies on the compositor's buffer pool depth, /// across the brief import window relies on the compositor's buffer pool depth, like any zero-copy
/// same as any zero-copy capture — the VAAPI importer copies into its own NV12 surface promptly.) /// capture.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub struct DmabufFrame { pub struct DmabufFrame {
pub fd: std::os::fd::OwnedFd, pub fd: std::os::fd::OwnedFd,
/// DRM FourCC of the packed-RGB plane (e.g. `XR24` for BGRx). /// DRM FourCC (`XR24` for BGRx, `NV12` for native 4:2:0).
pub fourcc: u32, pub fourcc: u32,
/// DRM format modifier the compositor allocated (0 = LINEAR). /// DRM format modifier the compositor allocated (0 = LINEAR).
pub modifier: u64, pub modifier: u64,
/// Second-plane `(offset, stride)` within the SAME fd, when the producer reported one (the
/// PipeWire buffer's plane-1 chunk — NV12's interleaved UV). `None` falls back to the
/// contiguous-plane contract above. Always `None` for single-plane packed RGB.
pub plane1: Option<(u32, u32)>,
pub offset: u32, pub offset: u32,
pub stride: u32, pub stride: u32,
} }
@@ -225,8 +264,8 @@ pub enum FramePayload {
/// The dmabuf has already been imported + copied into this owned device buffer. /// The dmabuf has already been imported + copied into this owned device buffer.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
Cuda(pf_zerocopy::DeviceBuffer), Cuda(pf_zerocopy::DeviceBuffer),
/// A raw packed-RGB dmabuf — the AMD/Intel (VAAPI) zero-copy path. The encoder imports it into /// A raw DMA-BUF: packed RGB for the existing GPU CSC paths, or native NV12 from a producer
/// a VA surface and does RGB→NV12 on the GPU video engine (no host CSC, no upload). /// such as gamescope. The encoder imports it without a host copy.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
Dmabuf(DmabufFrame), Dmabuf(DmabufFrame),
/// A GPU-resident D3D11 texture (Windows zero-copy path for NVENC). Owns the copied frame. /// A GPU-resident D3D11 texture (Windows zero-copy path for NVENC). Owns the copied frame.
+17
View File
@@ -63,6 +63,13 @@ pub struct HostConfig {
/// deliver full chroma, and the GPU/driver passed the encode probe — otherwise 4:2:0. /// deliver full chroma, and the GPU/driver passed the encode probe — otherwise 4:2:0.
/// `PUNKTFUNK_444=0`/`false`/`off`/`no` disables. Independent of `ten_bit` (chroma vs depth). /// `PUNKTFUNK_444=0`/`false`/`off`/`no` disables. Independent of `ten_bit` (chroma vs depth).
pub four_four_four: bool, pub four_four_four: bool,
/// `PUNKTFUNK_CHACHA20` — host policy gate for the negotiated ChaCha20-Poly1305 session
/// cipher (design/chacha20-session-cipher.md). **Default ON** (pure rollout safety — perf-only,
/// both AEADs are full-strength): the host merely *allows* it — a session only seals with
/// ChaCha when the client advertised `VIDEO_CAP_CHACHA20` (set by soft-AES armv7 clients,
/// e.g. webOS TVs, whose GCM decrypt caps at ~100 Mbps); everyone else stays AES-128-GCM.
/// `PUNKTFUNK_CHACHA20=0`/`false`/`off`/`no` disables.
pub chacha20: bool,
/// `PUNKTFUNK_PERF` — per-stage timing instrumentation. /// `PUNKTFUNK_PERF` — per-stage timing instrumentation.
pub perf: bool, pub perf: bool,
/// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic). /// `PUNKTFUNK_VIDEO_SOURCE` — GameStream video source select (`virtual` / `portal` / unset → synthetic).
@@ -147,6 +154,16 @@ impl HostConfig {
) )
}) })
.unwrap_or(true), .unwrap_or(true),
// Default ON, explicit-off grammar (the client's VIDEO_CAP_CHACHA20 bit is the real
// per-session switch; see the field doc).
chacha20: val("PUNKTFUNK_CHACHA20")
.map(|s| {
!matches!(
s.trim().to_ascii_lowercase().as_str(),
"0" | "false" | "off" | "no"
)
})
.unwrap_or(true),
perf: flag("PUNKTFUNK_PERF"), perf: flag("PUNKTFUNK_PERF"),
video_source: val("PUNKTFUNK_VIDEO_SOURCE"), video_source: val("PUNKTFUNK_VIDEO_SOURCE"),
compositor: val("PUNKTFUNK_COMPOSITOR"), compositor: val("PUNKTFUNK_COMPOSITOR"),
+6
View File
@@ -36,6 +36,12 @@ pub fn vk_to_evdev(vk: u8) -> Option<u16> {
0x2D => Some(110), // VK_INSERT -> KEY_INSERT 0x2D => Some(110), // VK_INSERT -> KEY_INSERT
0x2E => Some(111), // VK_DELETE -> KEY_DELETE 0x2E => Some(111), // VK_DELETE -> KEY_DELETE
// --- Consumer/media keys (Android TV remotes, keyboard media rows) ---
0xB0 => Some(163), // VK_MEDIA_NEXT_TRACK -> KEY_NEXTSONG
0xB1 => Some(165), // VK_MEDIA_PREV_TRACK -> KEY_PREVIOUSSONG
0xB2 => Some(166), // VK_MEDIA_STOP -> KEY_STOPCD
0xB3 => Some(164), // VK_MEDIA_PLAY_PAUSE -> KEY_PLAYPAUSE
// --- Generic modifiers --- // --- Generic modifiers ---
0x10 => Some(42), // VK_SHIFT -> KEY_LEFTSHIFT 0x10 => Some(42), // VK_SHIFT -> KEY_LEFTSHIFT
0x11 => Some(29), // VK_CONTROL -> KEY_LEFTCTRL 0x11 => Some(29), // VK_CONTROL -> KEY_LEFTCTRL
@@ -424,6 +424,9 @@ impl InputInjector for KwinFakeInjector {
self.fake.touch_up(event.code); self.fake.touch_up(event.code);
self.fake.touch_frame(); self.fake.touch_frame();
} }
// fake_input can only press host-layout keycodes — no committed-text path (the
// HOST_CAP_TEXT_INPUT cap is not advertised on this backend).
InputKind::TextInput => {}
// Gamepads are injected through uinput, not the compositor. // Gamepads are injected through uinput, not the compositor.
InputKind::GamepadState InputKind::GamepadState
| InputKind::GamepadButton | InputKind::GamepadButton
+205 -30
View File
@@ -109,7 +109,7 @@ async fn session_main(mut rx: UnboundedReceiver<InputEvent>, source: EiSource) {
// Keep `_rd`/`_session` bound for the whole loop — dropping the portal session closes the // Keep `_rd`/`_session` bound for the whole loop — dropping the portal session closes the
// EIS connection. Bound the setup so a headless approval dialog (un-bypassed grant) can't // EIS connection. Bound the setup so a headless approval dialog (un-bypassed grant) can't
// hang the worker forever. // hang the worker forever.
let (_keepalive, context, mut events) = match tokio::time::timeout( let (_keepalive, context, mut events, output_hint) = match tokio::time::timeout(
Duration::from_secs(30), Duration::from_secs(30),
connect(source), connect(source),
) )
@@ -130,6 +130,7 @@ async fn session_main(mut rx: UnboundedReceiver<InputEvent>, source: EiSource) {
tracing::info!("libei: EIS connected — awaiting devices"); tracing::info!("libei: EIS connected — awaiting devices");
let mut state = EiState::new(); let mut state = EiState::new();
state.output_hint = output_hint;
// Watchdog: a healthy EIS server adds + resumes an input device within a beat of the handshake. // Watchdog: a healthy EIS server adds + resumes an input device within a beat of the handshake.
// If none has resumed by this deadline, the connection is dead-on-arrival (stale/half-ready // If none has resumed by this deadline, the connection is dead-on-arrival (stale/half-ready
// gamescope socket the handshake passed but no real server is behind) — exit so the next // gamescope socket the handshake passed but no real server is behind) — exit so the next
@@ -177,21 +178,29 @@ type Connected = (
Box<dyn Send>, Box<dyn Send>,
ei::Context, ei::Context,
reis::tokio::EiConvertEventStream, reis::tokio::EiConvertEventStream,
// The compositor's output size ("WxH" relay-file hint) — the scale target for absolute
// coordinates when the EIS advertises only a degenerate region (gamescope). `None` for
// the portal/Mutter paths, whose regions are real.
Option<(u32, u32)>,
); );
/// Reach an EIS server per `source` and run the EI sender handshake. /// Reach an EIS server per `source` and run the EI sender handshake.
async fn connect(source: EiSource) -> Result<Connected> { async fn connect(source: EiSource) -> Result<Connected> {
let (keepalive, stream): (Box<dyn Send>, UnixStream) = match source { let (keepalive, stream, output_hint): (Box<dyn Send>, UnixStream, Option<(u32, u32)>) =
EiSource::Portal => { match source {
let (rd, session, fd) = connect_portal().await?; EiSource::Portal => {
(Box::new((rd, session)), UnixStream::from(fd)) let (rd, session, fd) = connect_portal().await?;
} (Box::new((rd, session)), UnixStream::from(fd), None)
EiSource::MutterEis => { }
let (keepalive, fd) = connect_mutter().await?; EiSource::MutterEis => {
(keepalive, UnixStream::from(fd)) let (keepalive, fd) = connect_mutter().await?;
} (keepalive, UnixStream::from(fd), None)
EiSource::SocketPathFile(file) => (Box::new(()), connect_socket_file(&file).await?), }
}; EiSource::SocketPathFile(file) => {
let (stream, hint) = connect_socket_file(&file).await?;
(Box::new(()), stream, hint)
}
};
let context = ei::Context::new(stream).map_err(|e| anyhow!("reis EI context: {e}"))?; let context = ei::Context::new(stream).map_err(|e| anyhow!("reis EI context: {e}"))?;
// Bound the handshake. `UnixStream::connect` to a socket *file* succeeds the moment the path // Bound the handshake. `UnixStream::connect` to a socket *file* succeeds the moment the path
// exists, but a stale/half-ready gamescope (its socket created early in startup, or left behind // exists, but a stale/half-ready gamescope (its socket created early in startup, or left behind
@@ -206,7 +215,7 @@ async fn connect(source: EiSource) -> Result<Connected> {
anyhow!("EI handshake timed out (EIS server not responding — stale/half-ready socket?)") anyhow!("EI handshake timed out (EIS server not responding — stale/half-ready socket?)")
})? })?
.map_err(|e| anyhow!("EI handshake: {e}"))?; .map_err(|e| anyhow!("EI handshake: {e}"))?;
Ok((keepalive, context, events)) Ok((keepalive, context, events, output_hint))
} }
/// Open a RemoteDesktop portal session (pointer + keyboard) and obtain the EIS socket fd. /// Open a RemoteDesktop portal session (pointer + keyboard) and obtain the EIS socket fd.
@@ -294,8 +303,10 @@ async fn connect_mutter() -> Result<(Box<dyn Send>, std::os::fd::OwnedFd)> {
/// Poll `file` for the EIS socket path (the gamescope backend relays `LIBEI_SOCKET` there once /// Poll `file` for the EIS socket path (the gamescope backend relays `LIBEI_SOCKET` there once
/// the nested app launches), then connect. A bare name is resolved against `XDG_RUNTIME_DIR`, /// the nested app launches), then connect. A bare name is resolved against `XDG_RUNTIME_DIR`,
/// mirroring libei's own `LIBEI_SOCKET` semantics. /// mirroring libei's own `LIBEI_SOCKET` semantics. Line 2 of the relay file, when present,
async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> { /// carries the compositor's output size as `WxH` — returned as the absolute-coordinate scale
/// hint (gamescope's EIS region is degenerate, so the geometry can't come from the protocol).
async fn connect_socket_file(file: &std::path::Path) -> Result<(UnixStream, Option<(u32, u32)>)> {
// The relay file is rewritten each session with the CURRENT gamescope's `LIBEI_SOCKET`, and the // The relay file is rewritten each session with the CURRENT gamescope's `LIBEI_SOCKET`, and the
// socket may not be `listen()`ing the instant its name appears — or the file may briefly still // socket may not be `listen()`ing the instant its name appears — or the file may briefly still
// hold a prior, now-dead session's name (the host-lifetime injector reconnecting between // hold a prior, now-dead session's name (the host-lifetime injector reconnecting between
@@ -319,7 +330,12 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
)); ));
} }
if let Ok(s) = std::fs::read_to_string(file) { if let Ok(s) = std::fs::read_to_string(file) {
let name = s.trim(); let mut file_lines = s.lines();
let name = file_lines.next().unwrap_or("").trim();
let hint = file_lines.next().and_then(|l| {
let (w, h) = l.trim().split_once('x')?;
Some((w.parse::<u32>().ok()?, h.parse::<u32>().ok()?))
});
if !name.is_empty() { if !name.is_empty() {
let full = if name.starts_with('/') { let full = if name.starts_with('/') {
std::path::PathBuf::from(name) std::path::PathBuf::from(name)
@@ -334,7 +350,7 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
logged = name.to_string(); logged = name.to_string();
} }
match UnixStream::connect(&full) { match UnixStream::connect(&full) {
Ok(stream) => return Ok(stream), Ok(stream) => return Ok((stream, hint)),
// Refused = socket file exists but no listener yet (or a dead session); // Refused = socket file exists but no listener yet (or a dead session);
// NotFound = path not created yet. Both heal once the live gamescope's EIS is // NotFound = path not created yet. Both heal once the live gamescope's EIS is
// up — retry. Anything else (e.g. permission) is a real failure. // up — retry. Anything else (e.g. permission) is a real failure.
@@ -358,6 +374,25 @@ async fn connect_socket_file(file: &std::path::Path) -> Result<UnixStream> {
} }
/// One EI device and its emulation state. /// One EI device and its emulation state.
/// Pick the region to map absolute coordinates into: the one whose logical size matches the
/// streamed mode (the session's virtual output). The device advertises one region per logical
/// monitor, and blindly taking `first()` next to a physical monitor put the pointer — and every
/// click — on whichever output the compositor happened to announce first (on-glass: GNOME with a
/// dummy HDMI beside the virtual primary; the seat cursor never entered the streamed monitor, so
/// neither embedded nor metadata cursor capture could see it). Size is the only key available
/// today: regions carry no output name, and matching the screencast stream's `mapping_id` needs
/// the stream id plumbed across crates (follow-up). Two same-sized monitors stay ambiguous.
fn region_for_mode(
regions: &[reis::event::Region],
w: f32,
h: f32,
) -> Option<&reis::event::Region> {
regions
.iter()
.find(|r| r.width as f32 == w && r.height as f32 == h)
.or_else(|| regions.first())
}
struct DeviceSlot { struct DeviceSlot {
device: reis::event::Device, device: reis::event::Device,
/// The device is resumed (allowed to emit). Devices arrive paused and may pause again. /// The device is resumed (allowed to emit). Devices arrive paused and may pause again.
@@ -386,6 +421,22 @@ struct EiState {
held_keys: Vec<u32>, held_keys: Vec<u32>,
held_buttons: Vec<u32>, held_buttons: Vec<u32>,
held_touches: Vec<u32>, held_touches: Vec<u32>,
/// The touch id currently degraded to the absolute pointer ([`EiState::degrade_touch`]) —
/// the "primary finger" while the EIS has no touchscreen device. `None` between touches.
degraded_touch: Option<u32>,
/// The compositor's output size (relay-file "WxH" hint) — scale target for absolute
/// coordinates when the device's region is degenerate/absent (gamescope). Without it the
/// fallback is raw client pixels, correct only when the stream runs at the output's size.
output_hint: Option<(u32, u32)>,
}
/// Is this EIS region a plausible OUTPUT geometry — something to map normalized coordinates
/// into? gamescope advertises a degenerate `(0,0,INT32_MAX,INT32_MAX)` "everything" region on
/// its virtual input device, meaning "absolute coordinates are raw"; normalizing into it
/// explodes a center tap to x≈1e9, which the compositor clamps to the far corner. 16384
/// comfortably covers real multi-monitor layouts while rejecting the sentinel.
fn sane_region(r: &reis::event::Region) -> bool {
r.width > 0 && r.height > 0 && r.width <= 16_384 && r.height <= 16_384
} }
/// Stable small index per [`InputKind`] for the `seen_kinds` bitmask. /// Stable small index per [`InputKind`] for the `seen_kinds` bitmask.
@@ -406,6 +457,7 @@ fn kind_bit(kind: InputKind) -> u32 {
InputKind::GamepadState => 12, InputKind::GamepadState => 12,
InputKind::GamepadRemove => 13, InputKind::GamepadRemove => 13,
InputKind::GamepadArrival => 14, InputKind::GamepadArrival => 14,
InputKind::TextInput => 15,
}; };
1 << i 1 << i
} }
@@ -422,6 +474,8 @@ impl EiState {
held_keys: Vec::new(), held_keys: Vec::new(),
held_buttons: Vec::new(), held_buttons: Vec::new(),
held_touches: Vec::new(), held_touches: Vec::new(),
degraded_touch: None,
output_hint: None,
} }
} }
@@ -430,6 +484,10 @@ impl EiState {
/// normal [`EiState::inject`] path so the compositor sees proper key-up / button-up / /// normal [`EiState::inject`] path so the compositor sees proper key-up / button-up /
/// touch-up frames before the devices disappear. /// touch-up frames before the devices disappear.
fn release_all(&mut self, ctx: &ei::Context) { fn release_all(&mut self, ctx: &ei::Context) {
// A degraded touch is held as a synthesized left button (in `held_buttons`), so the
// loop below releases it — but the primary-finger latch must clear too, or the NEXT
// session's first TouchDown reads as a second finger and is ignored.
self.degraded_touch = None;
let (keys, buttons, touches) = ( let (keys, buttons, touches) = (
std::mem::take(&mut self.held_keys), std::mem::take(&mut self.held_keys),
std::mem::take(&mut self.held_buttons), std::mem::take(&mut self.held_buttons),
@@ -506,6 +564,14 @@ impl EiState {
keyboard = dev.has_capability(DeviceCapability::Keyboard), keyboard = dev.has_capability(DeviceCapability::Keyboard),
button = dev.has_capability(DeviceCapability::Button), button = dev.has_capability(DeviceCapability::Button),
scroll = dev.has_capability(DeviceCapability::Scroll), scroll = dev.has_capability(DeviceCapability::Scroll),
// One region per logical monitor — which one absolute coords map into is
// resolved per event (`region_for_mode`); log them so a mis-mapped pointer
// (cursor/clicks on the wrong output) is diagnosable from the journal.
regions = ?dev
.regions()
.iter()
.map(|r| format!("{}x{}+{}+{}", r.width, r.height, r.x, r.y))
.collect::<Vec<_>>(),
"libei: device RESUMED (now emittable)" "libei: device RESUMED (now emittable)"
); );
} }
@@ -537,8 +603,85 @@ impl EiState {
} }
} }
/// Degrade touch to a single-finger ABSOLUTE POINTER on compositors whose EIS never
/// creates a touchscreen device (gamescope's "Gamescope Virtual Input" advertises
/// pointer/pointer_abs/button but no touch — observed live; headless KWin the same).
/// Down = abs-move + left press, move = abs-move, up = left release — synthesized through
/// the normal [`EiState::inject`] Mouse* paths so region mapping, held-state tracking and
/// [`EiState::release_all`] all apply. Only the FIRST finger drives the pointer; later
/// fingers are ignored (a pointer has no second contact — a pinch degrades to a drag).
fn degrade_touch(&mut self, ev: &InputEvent, ctx: &ei::Context) {
const GS_BUTTON_LEFT: u32 = 1;
match ev.kind {
InputKind::TouchDown => {
if self.degraded_touch.is_some() {
return; // secondary finger — single-pointer degradation
}
self.degraded_touch = Some(ev.code);
static NOTED: std::sync::atomic::AtomicBool =
std::sync::atomic::AtomicBool::new(false);
if !NOTED.swap(true, std::sync::atomic::Ordering::Relaxed) {
tracing::info!(
"compositor's EIS has no touchscreen device — degrading touch to a \
single-finger absolute pointer (tap = left click; multi-touch \
gestures unavailable)"
);
}
self.inject(
&InputEvent {
kind: InputKind::MouseMoveAbs,
..*ev
},
ctx,
);
self.inject(
&InputEvent {
kind: InputKind::MouseButtonDown,
code: GS_BUTTON_LEFT,
..*ev
},
ctx,
);
}
InputKind::TouchMove if self.degraded_touch == Some(ev.code) => {
self.inject(
&InputEvent {
kind: InputKind::MouseMoveAbs,
..*ev
},
ctx,
);
}
InputKind::TouchUp if self.degraded_touch == Some(ev.code) => {
self.degraded_touch = None;
self.inject(
&InputEvent {
kind: InputKind::MouseButtonUp,
code: GS_BUTTON_LEFT,
..*ev
},
ctx,
);
}
_ => {}
}
}
/// Translate and emit one client input event, committing it as a single `frame`. /// Translate and emit one client input event, committing it as a single `frame`.
fn inject(&mut self, ev: &InputEvent, ctx: &ei::Context) { fn inject(&mut self, ev: &InputEvent, ctx: &ei::Context) {
// No ei_touchscreen device but an absolute pointer exists → degrade rather than drop
// (the game-mode "touch simply does not work" trap). Checked per event, not latched:
// a touchscreen device appearing later (compositor restart, capability change) takes
// over seamlessly on the next touch.
if matches!(
ev.kind,
InputKind::TouchDown | InputKind::TouchMove | InputKind::TouchUp
) && self.device_for(DeviceCapability::Touch).is_none()
&& self.device_for(DeviceCapability::PointerAbsolute).is_some()
{
self.degrade_touch(ev, ctx);
return;
}
let cap = match ev.kind { let cap = match ev.kind {
InputKind::MouseMove => DeviceCapability::Pointer, InputKind::MouseMove => DeviceCapability::Pointer,
InputKind::MouseMoveAbs => DeviceCapability::PointerAbsolute, InputKind::MouseMoveAbs => DeviceCapability::PointerAbsolute,
@@ -553,6 +696,9 @@ impl EiState {
| InputKind::GamepadAxis | InputKind::GamepadAxis
| InputKind::GamepadRemove | InputKind::GamepadRemove
| InputKind::GamepadArrival => return, // uinput path (later) | InputKind::GamepadArrival => return, // uinput path (later)
// libei presses keycodes against the server's negotiated keymap — no committed-text
// path (the HOST_CAP_TEXT_INPUT cap is not advertised on this backend).
InputKind::TextInput => return,
}; };
self.injected += 1; self.injected += 1;
let n = self.injected; let n = self.injected;
@@ -605,16 +751,33 @@ impl EiState {
InputKind::MouseMoveAbs => { InputKind::MouseMoveAbs => {
let w = ((ev.flags >> 16) & 0xffff) as f32; let w = ((ev.flags >> 16) & 0xffff) as f32;
let h = (ev.flags & 0xffff) as f32; let h = (ev.flags & 0xffff) as f32;
match ( match slot.interface::<ei::PointerAbsolute>() {
slot.interface::<ei::PointerAbsolute>(), Some(p) if w > 0.0 && h > 0.0 => {
slot.regions().first(), // Map the normalized client position into the region matching the streamed
) { // output's mode (`region_for_mode` picks the right one on a multi-monitor
(Some(p), Some(region)) if w > 0.0 && h > 0.0 => { // EIS). gamescope's "Gamescope Virtual Input" advertises a degenerate
// Map the normalized client position into the device's first region. // (0,0,INT32_MAX,INT32_MAX) region meaning "coordinates are raw" —
// `sane_region` rejects it (normalizing into it explodes a center tap to
// x≈1e9, clamped to the far corner), so a non-matching / insane region
// falls to the output hint (correct across a resolution mismatch), then
// raw client pixels as the last resort.
let nx = (ev.x as f32 / w).clamp(0.0, 1.0); let nx = (ev.x as f32 / w).clamp(0.0, 1.0);
let ny = (ev.y as f32 / h).clamp(0.0, 1.0); let ny = (ev.y as f32 / h).clamp(0.0, 1.0);
let x = region.x as f32 + nx * region.width as f32; let (x, y) = match region_for_mode(slot.regions(), w, h)
let y = region.y as f32 + ny * region.height as f32; .filter(|r| sane_region(r))
{
Some(region) => (
region.x as f32 + nx * region.width as f32,
region.y as f32 + ny * region.height as f32,
),
// Degenerate/absent region: scale into the relay-file output hint
// (correct even when the client streams at a different resolution
// than the session runs); raw client pixels as the last resort.
None => match self.output_hint {
Some((ow, oh)) => (nx * ow as f32, ny * oh as f32),
None => (ev.x as f32, ev.y as f32),
},
};
p.motion_absolute(x, y); p.motion_absolute(x, y);
} }
_ => emitted = false, _ => emitted = false,
@@ -680,12 +843,23 @@ impl EiState {
InputKind::TouchDown | InputKind::TouchMove => { InputKind::TouchDown | InputKind::TouchMove => {
let w = ((ev.flags >> 16) & 0xffff) as f32; let w = ((ev.flags >> 16) & 0xffff) as f32;
let h = (ev.flags & 0xffff) as f32; let h = (ev.flags & 0xffff) as f32;
match (slot.interface::<ei::Touchscreen>(), slot.regions().first()) { match slot.interface::<ei::Touchscreen>() {
(Some(t), Some(region)) if w > 0.0 && h > 0.0 => { Some(t) if w > 0.0 && h > 0.0 => {
let nx = (ev.x as f32 / w).clamp(0.0, 1.0); let nx = (ev.x as f32 / w).clamp(0.0, 1.0);
let ny = (ev.y as f32 / h).clamp(0.0, 1.0); let ny = (ev.y as f32 / h).clamp(0.0, 1.0);
let x = region.x as f32 + nx * region.width as f32; // Same region-selection + degenerate fallback ladder as MouseMoveAbs.
let y = region.y as f32 + ny * region.height as f32; let (x, y) = match region_for_mode(slot.regions(), w, h)
.filter(|r| sane_region(r))
{
Some(region) => (
region.x as f32 + nx * region.width as f32,
region.y as f32 + ny * region.height as f32,
),
None => match self.output_hint {
Some((ow, oh)) => (nx * ow as f32, ny * oh as f32),
None => (ev.x as f32, ev.y as f32),
},
};
if ev.kind == InputKind::TouchDown { if ev.kind == InputKind::TouchDown {
t.down(ev.code, x, y); t.down(ev.code, x, y);
} else { } else {
@@ -703,7 +877,8 @@ impl EiState {
| InputKind::GamepadButton | InputKind::GamepadButton
| InputKind::GamepadAxis | InputKind::GamepadAxis
| InputKind::GamepadRemove | InputKind::GamepadRemove
| InputKind::GamepadArrival => emitted = false, | InputKind::GamepadArrival
| InputKind::TextInput => emitted = false,
} }
if emitted { if emitted {
+96
View File
@@ -98,9 +98,26 @@ pub struct WlrootsInjector {
keyboard: ZwpVirtualKeyboardV1, keyboard: ZwpVirtualKeyboardV1,
xkb_state: xkb::State, xkb_state: xkb::State,
_keymap_file: std::fs::File, // keep the memfd alive for the compositor's mmap _keymap_file: std::fs::File, // keep the memfd alive for the compositor's mmap
/// Dedicated committed-text device ([`InputKind::TextInput`]), created on first use.
text: Option<TextKeyboard>,
start: Instant, start: Instant,
} }
/// Cap on distinct characters the dynamic text keymap holds before it restarts from scratch
/// (keycodes grow upward from 9; xkb tops out at 255, so stay well under).
const TEXT_KEYMAP_MAX: usize = 200;
/// The dedicated **text** virtual keyboard: types committed IME text (`InputKind::TextInput`,
/// one Unicode scalar per event) by growing a keymap of Unicode keysyms on demand and pressing
/// the character's keycode — the `wtype` model. A separate `zwp_virtual_keyboard` so keymap
/// re-uploads never disturb the main device's layout/modifier state that VK key events ride on.
struct TextKeyboard {
keyboard: ZwpVirtualKeyboardV1,
/// Characters in keycode order: `chars[i]` types on wire keycode `i + 1` (xkb `i + 9`).
chars: Vec<char>,
_keymap_file: Option<std::fs::File>, // keep the memfd alive for the compositor's mmap
}
impl WlrootsInjector { impl WlrootsInjector {
pub fn open() -> Result<Self> { pub fn open() -> Result<Self> {
let conn = Connection::connect_to_env() let conn = Connection::connect_to_env()
@@ -171,6 +188,7 @@ impl WlrootsInjector {
keyboard, keyboard,
xkb_state, xkb_state,
_keymap_file: file, _keymap_file: file,
text: None,
start: Instant::now(), start: Instant::now(),
}) })
} }
@@ -179,6 +197,54 @@ impl WlrootsInjector {
self.start.elapsed().as_millis() as u32 self.start.elapsed().as_millis() as u32
} }
/// Type one committed-text Unicode scalar on the dedicated text device (created lazily),
/// growing its keymap when the character is new. Control characters are dropped — Enter,
/// Backspace and Tab ride the VK key-event path.
fn type_text(&mut self, cp: u32) -> Result<()> {
let Some(ch) = char::from_u32(cp) else {
return Ok(()); // lone surrogate / out of range
};
if ch.is_control() {
return Ok(());
}
if self.text.is_none() {
let (Some(mgr), Some(seat)) =
(self.globals.keyboard_mgr.clone(), self.globals.seat.clone())
else {
return Ok(());
};
let kb = mgr.create_virtual_keyboard(&seat, &self.queue.handle(), ());
self.text = Some(TextKeyboard {
keyboard: kb,
chars: Vec::new(),
_keymap_file: None,
});
}
let t = self.now_ms();
let text = self.text.as_mut().expect("created above");
let code = match text.chars.iter().position(|&c| c == ch) {
Some(i) => (i + 1) as u32,
None => {
if text.chars.len() >= TEXT_KEYMAP_MAX {
text.chars.clear(); // restart the map; old codes are re-assigned lazily
}
text.chars.push(ch);
let keymap_str = text_keymap(&text.chars);
let file = memfd_with(&keymap_str)?;
text.keyboard.keymap(
1, /* XKB_V1 */
file.as_fd(),
keymap_str.len() as u32 + 1,
);
text._keymap_file = Some(file);
text.chars.len() as u32
}
};
text.keyboard.key(t, code, 1);
text.keyboard.key(t, code, 0);
Ok(())
}
/// Update xkb state for a key and tell the compositor the resulting modifier mask. /// Update xkb state for a key and tell the compositor the resulting modifier mask.
fn send_modifiers(&mut self, evdev: u16, down: bool) { fn send_modifiers(&mut self, evdev: u16, down: bool) {
let kc = xkb::Keycode::new(evdev as u32 + 8); // evdev -> xkb keycode let kc = xkb::Keycode::new(evdev as u32 + 8); // evdev -> xkb keycode
@@ -254,6 +320,9 @@ impl InputInjector for WlrootsInjector {
tracing::debug!(vk = event.code, "unmapped VK keycode — dropped"); tracing::debug!(vk = event.code, "unmapped VK keycode — dropped");
} }
} }
InputKind::TextInput => {
self.type_text(event.code)?;
}
InputKind::GamepadState InputKind::GamepadState
| InputKind::GamepadButton | InputKind::GamepadButton
| InputKind::GamepadAxis | InputKind::GamepadAxis
@@ -271,6 +340,33 @@ impl InputInjector for WlrootsInjector {
} }
} }
/// Build a minimal xkb keymap whose keycode `i + 9` (wire code `i + 1`) types `chars[i]`, using
/// Unicode keysym names (`U<hex>` — xkbcommon resolves them for any scalar, emoji included).
/// Types/compat `include "complete"` mirrors `wtype`'s generated keymap — proven on wlroots
/// compositors, and the system XKB data is present (the main keymap compiled from it in `open`).
fn text_keymap(chars: &[char]) -> String {
use std::fmt::Write as _;
let mut keycodes = String::new();
let mut symbols = String::new();
for (i, ch) in chars.iter().enumerate() {
let _ = writeln!(keycodes, " <T{i}> = {};", i + 9);
let _ = writeln!(symbols, " key <T{i}> {{ [ U{:04X} ] }};", *ch as u32);
}
format!(
"xkb_keymap {{\n\
xkb_keycodes \"punktfunk-text\" {{\n\
minimum = 8;\n\
maximum = {};\n\
{keycodes}\
}};\n\
xkb_types \"punktfunk-text\" {{ include \"complete\" }};\n\
xkb_compatibility \"punktfunk-text\" {{ include \"complete\" }};\n\
xkb_symbols \"punktfunk-text\" {{\n{symbols} }};\n\
}};\n",
chars.len() + 9,
)
}
/// Create an anonymous in-memory file holding `s` + a trailing NUL (for the keymap fd). /// Create an anonymous in-memory file holding `s` + a trailing NUL (for the keymap fd).
fn memfd_with(s: &str) -> Result<std::fs::File> { fn memfd_with(s: &str) -> Result<std::fs::File> {
let name = b"punktfunk-keymap\0"; let name = b"punktfunk-keymap\0";
@@ -27,10 +27,10 @@ use windows::Win32::System::StationsAndDesktops::{
use windows::Win32::UI::Input::KeyboardAndMouse::{ use windows::Win32::UI::Input::KeyboardAndMouse::{
GetKeyboardLayout, MapVirtualKeyExW, SendInput, HKL, INPUT, INPUT_0, INPUT_KEYBOARD, GetKeyboardLayout, MapVirtualKeyExW, SendInput, HKL, INPUT, INPUT_0, INPUT_KEYBOARD,
INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY, KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE, INPUT_MOUSE, KEYBDINPUT, KEYEVENTF_EXTENDEDKEY, KEYEVENTF_KEYUP, KEYEVENTF_SCANCODE,
MAPVK_VK_TO_VSC_EX, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL, MOUSEEVENTF_LEFTDOWN, KEYEVENTF_UNICODE, MAPVK_VK_TO_VSC_EX, MOUSEEVENTF_ABSOLUTE, MOUSEEVENTF_HWHEEL,
MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP, MOUSEEVENTF_MOVE, MOUSEEVENTF_LEFTDOWN, MOUSEEVENTF_LEFTUP, MOUSEEVENTF_MIDDLEDOWN, MOUSEEVENTF_MIDDLEUP,
MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK, MOUSEEVENTF_WHEEL, MOUSEEVENTF_MOVE, MOUSEEVENTF_RIGHTDOWN, MOUSEEVENTF_RIGHTUP, MOUSEEVENTF_VIRTUALDESK,
MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY, MOUSEEVENTF_WHEEL, MOUSEEVENTF_XDOWN, MOUSEEVENTF_XUP, MOUSEINPUT, VIRTUAL_KEY,
}; };
use windows::Win32::UI::WindowsAndMessaging::{ use windows::Win32::UI::WindowsAndMessaging::{
GetForegroundWindow, GetSystemMetrics, GetWindowThreadProcessId, SM_CXVIRTUALSCREEN, GetForegroundWindow, GetSystemMetrics, GetWindowThreadProcessId, SM_CXVIRTUALSCREEN,
@@ -297,6 +297,33 @@ impl InputInjector for SendInputInjector {
}; };
self.send(&[key(ki)]) self.send(&[key(ki)])
} }
InputKind::TextInput => {
// Committed IME text: one Unicode scalar per event, injected as
// `KEYEVENTF_UNICODE` packets (wScan = UTF-16 unit, no scancode/layout involved
// — the receiving app gets the character verbatim via WM_CHAR). An astral-plane
// scalar (emoji) is its surrogate pair, each unit down+up in order — exactly how
// Windows expects supplementary characters from unicode injection.
let Some(ch) = char::from_u32(event.code) else {
return Ok(()); // lone surrogate / out of range — drop
};
if ch.is_control() {
return Ok(()); // control chars ride the VK path (Enter/Backspace/Tab)
}
let mut units = [0u16; 2];
let mut inputs: Vec<INPUT> = Vec::with_capacity(4);
for &unit in ch.encode_utf16(&mut units).iter() {
for flags in [KEYEVENTF_UNICODE, KEYEVENTF_UNICODE | KEYEVENTF_KEYUP] {
inputs.push(key(KEYBDINPUT {
wVk: VIRTUAL_KEY(0),
wScan: unit,
dwFlags: flags,
time: 0,
dwExtraInfo: 0,
}));
}
}
self.send(&inputs)
}
// Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op. // Gamepad goes through the XUSB backend. Touch: no SendInput equivalent -> no-op.
InputKind::GamepadButton InputKind::GamepadButton
| InputKind::GamepadAxis | InputKind::GamepadAxis
+23
View File
@@ -174,6 +174,29 @@ pub fn default_backend() -> Backend {
Backend::Unsupported Backend::Unsupported
} }
/// Whether the session's inject backend can type **committed text**
/// ([`InputKind::TextInput`] — see `HOST_CAP_TEXT_INPUT`): Windows always (`KEYEVENTF_UNICODE`);
/// Linux only on the wlroots backend (a dedicated virtual keyboard with a dynamically-grown
/// Unicode keymap) — KWin fake-input/libei/gamescope can only press keycodes of the host layout.
/// Consulted at Welcome time to advertise the cap; a mid-session backend switch away from a
/// capable one just degrades to dropped text events (input is lossy by design).
#[cfg(target_os = "windows")]
pub fn text_input_supported() -> bool {
true
}
/// See the Windows variant: Linux types text only through the wlroots virtual-keyboard backend.
#[cfg(target_os = "linux")]
pub fn text_input_supported() -> bool {
matches!(default_backend(), Backend::WlrVirtual)
}
/// No injector ⇒ no text.
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn text_input_supported() -> bool {
false
}
#[path = "inject/service.rs"] #[path = "inject/service.rs"]
mod service; mod service;
pub use service::InjectorService; pub use service::InjectorService;
+125
View File
@@ -0,0 +1,125 @@
//! Client-side cursor rendering (design/remote-desktop-sweep.md M2): the host forwards the
//! pointer's SHAPE (reliable control stream, cached by serial) and per-frame STATE (lossy
//! `0xD0` — position/visibility), and WE draw it as a real OS cursor — pointer feel stops
//! paying the video round-trip (the Parsec/RDP model). Active only when the session
//! negotiated it (`HOST_CAP_CURSOR` in the Welcome — the host stopped compositing then) and
//! only applied while the DESKTOP mouse model is engaged: under capture the pointer is
//! relative-locked (SDL hides it) and games draw their own cursor in-frame.
use punktfunk_core::client::NativeClient;
use punktfunk_core::quic::{CursorState, HOST_CAP_CURSOR};
use sdl3::mouse::{Cursor, MouseUtil, SystemCursor};
use std::collections::HashMap;
use std::time::Duration;
/// Shape serials cached at most — cursors cycle through a handful of shapes (arrow, I-beam,
/// resize…); a runaway host can't grow the map past this (the cache resets, shapes re-arrive
/// on the reliable stream via the serial-miss path).
const SHAPE_CACHE_MAX: usize = 64;
pub struct CursorChannel {
/// The Welcome carried `HOST_CAP_CURSOR` — the host forwards instead of compositing.
negotiated: bool,
/// Serial → built OS cursor. An SDL `Cursor` must outlive its `set()`, so the cache owns
/// every shape ever applied this session (bounded by [`SHAPE_CACHE_MAX`]).
shapes: HashMap<u32, Cursor>,
/// The serial currently installed via `Cursor::set` (`None` = default/system cursor).
installed: Option<u32>,
/// Latest `0xD0` state (latest-wins across a drained batch).
state: Option<CursorState>,
}
impl CursorChannel {
pub fn new(connector: &NativeClient) -> CursorChannel {
let negotiated = connector.host_caps() & HOST_CAP_CURSOR != 0;
if negotiated {
tracing::info!("cursor channel negotiated — host cursor renders locally");
}
CursorChannel {
negotiated,
shapes: HashMap::new(),
installed: None,
state: None,
}
}
/// Whether the host forwards the cursor this session (it no longer composites one).
pub fn negotiated(&self) -> bool {
self.negotiated
}
/// The latest drained `0xD0` state — the run loop reads `relative_hint` off it for the
/// M3 host-driven mode flip (and `x`/`y` as the reappear position when leaving relative).
pub fn state(&self) -> Option<CursorState> {
self.state
}
/// Drain the two planes and apply the newest state — once per run-loop iteration.
/// `desktop_active` = the desktop mouse model is engaged (captured + desktop): only then
/// do we own the local cursor's shape/visibility; under capture SDL's relative mode owns
/// it, and released the system cursor must look normal.
pub fn pump(&mut self, connector: &NativeClient, mouse: &MouseUtil, desktop_active: bool) {
if !self.negotiated {
return;
}
while let Ok(shape) = connector.next_cursor_shape(Duration::ZERO) {
if self.shapes.len() >= SHAPE_CACHE_MAX {
// Degenerate host: reset — live shapes re-install via the serial-miss path.
self.shapes.clear();
self.installed = None;
}
let mut data = shape.rgba;
let built = sdl3::surface::Surface::from_data(
&mut data,
shape.w as u32,
shape.h as u32,
shape.w as u32 * 4,
sdl3::pixels::PixelFormat::RGBA32,
)
.map_err(|e| e.to_string())
.and_then(|surf| {
Cursor::from_surface(&surf, shape.hot_x as i32, shape.hot_y as i32)
.map_err(|e| e.to_string())
});
match built {
Ok(cursor) => {
// A re-sent serial replaces its entry; force re-install if it's current.
if self.installed == Some(shape.serial) {
self.installed = None;
}
self.shapes.insert(shape.serial, cursor);
}
Err(e) => tracing::warn!(error = %e, w = shape.w, h = shape.h,
"cursor shape rejected by SDL — keeping the previous cursor"),
}
}
while let Ok(st) = connector.next_cursor_state(Duration::ZERO) {
self.state = Some(st); // latest wins
}
if !desktop_active {
// Capture mode / released: hand the cursor back to the system default so a
// released pointer over the window doesn't wear the host's shape.
if self.installed.take().is_some() {
Cursor::from_system(SystemCursor::Arrow)
.map(|c| c.set())
.ok();
}
return;
}
let Some(st) = self.state else { return };
if st.visible() && self.installed != Some(st.serial) {
if let Some(cursor) = self.shapes.get(&st.serial) {
cursor.set();
self.installed = Some(st.serial);
}
// Serial miss: the (reliable) shape hasn't landed yet — keep the previous
// cursor for the RTT rather than flashing default.
}
// Visibility follows the host (a host app hid its pointer ⇒ ours hides too). Queried,
// not shadowed, so apply_capture's own show/hide calls can never desync us.
if mouse.is_cursor_showing() != st.visible() {
mouse.show_cursor(st.visible());
}
}
}
+77 -3
View File
@@ -14,10 +14,17 @@
//! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Motion deltas are //! Keys are SDL scancodes → VK via `keymap_sdl`, layout-independent. Motion deltas are
//! COALESCED: one summed `MouseMove` per loop iteration (a 1000 Hz mouse would //! COALESCED: one summed `MouseMove` per loop iteration (a 1000 Hz mouse would
//! otherwise send a datagram per event). //! otherwise send a datagram per event).
//!
//! The DESKTOP mouse model (design/remote-desktop-sweep.md M1) reuses this same engage/
//! release state but never locks the pointer: the local cursor moves freely (hidden over
//! the window — the host's composited cursor is the one you see) and motion goes on the
//! wire as absolute positions through the letterbox (`MouseMoveAbs`, latest-wins per loop
//! iteration). Requires a host injector with absolute support — gamescope's EIS is
//! relative-only, so sessions there are pinned to capture ([`Capture::new`] `abs_ok`).
use crate::keymap_sdl; use crate::keymap_sdl;
use crate::touch::{Abs, Act, Gestures}; use crate::touch::{Abs, Act, Gestures};
use pf_client_core::trust::TouchMode; use pf_client_core::trust::{MouseMode, TouchMode};
use punktfunk_core::client::NativeClient; use punktfunk_core::client::NativeClient;
use punktfunk_core::input::{InputEvent, InputKind}; use punktfunk_core::input::{InputEvent, InputKind};
use std::collections::{HashMap, HashSet}; use std::collections::{HashMap, HashSet};
@@ -41,6 +48,13 @@ pub struct Capture {
held_buttons: HashSet<u32>, held_buttons: HashSet<u32>,
/// Relative motion not yet on the wire, summed per loop iteration. /// Relative motion not yet on the wire, summed per loop iteration.
pending_rel: (i32, i32), pending_rel: (i32, i32),
/// Desktop-model position not yet on the wire, latest-wins per loop iteration.
pending_abs: Option<Abs>,
/// The desktop (absolute, uncaptured) mouse model is active. Flipped live by the
/// Ctrl+Alt+Shift+M chord; never true unless `abs_ok`.
desktop: bool,
/// The host injector accepts `MouseMoveAbs` (any compositor but gamescope).
abs_ok: bool,
/// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space — /// Fractional wheel remainder per axis (x, y) in 120-unit WHEEL_DELTA space —
/// precision surfaces deliver sub-unit deltas; truncating each event drops the tail. /// precision surfaces deliver sub-unit deltas; truncating each event drops the tail.
scroll_acc: (f64, f64), scroll_acc: (f64, f64),
@@ -70,10 +84,14 @@ fn send(connector: &NativeClient, kind: InputKind, code: u32, x: i32, y: i32, fl
} }
impl Capture { impl Capture {
/// `abs_ok` = the host injector accepts absolute pointer events; without it the
/// desktop model is unavailable and `mouse_mode` silently resolves to capture.
pub fn new( pub fn new(
connector: Arc<NativeClient>, connector: Arc<NativeClient>,
touch_mode: TouchMode, touch_mode: TouchMode,
invert_scroll: bool, invert_scroll: bool,
mouse_mode: MouseMode,
abs_ok: bool,
) -> Capture { ) -> Capture {
Capture { Capture {
connector, connector,
@@ -82,6 +100,9 @@ impl Capture {
held_keys: HashSet::new(), held_keys: HashSet::new(),
held_buttons: HashSet::new(), held_buttons: HashSet::new(),
pending_rel: (0, 0), pending_rel: (0, 0),
pending_abs: None,
desktop: abs_ok && mouse_mode == MouseMode::Desktop,
abs_ok,
scroll_acc: (0.0, 0.0), scroll_acc: (0.0, 0.0),
touch_slots: HashMap::new(), touch_slots: HashMap::new(),
touch_mode, touch_mode,
@@ -94,6 +115,38 @@ impl Capture {
self.captured self.captured
} }
/// The desktop (absolute, uncaptured) mouse model is active.
pub fn desktop(&self) -> bool {
self.desktop
}
/// Flip capture ⇄ desktop (the Ctrl+Alt+Shift+M chord). `None` = the host can't take
/// absolute pointer events (gamescope), so the chord has nothing to offer; otherwise
/// the new desktop state. Motion gathered under the old model never crosses modes.
pub fn toggle_desktop(&mut self) -> Option<bool> {
if !self.abs_ok {
return None;
}
self.desktop = !self.desktop;
self.pending_rel = (0, 0);
self.pending_abs = None;
Some(self.desktop)
}
/// Set the mouse model directly (the M3 host-driven flip — `relative_hint` says a host
/// app grabbed/hid the pointer, so run relative; hint clear = back to absolute). Same
/// gating and motion hygiene as [`toggle_desktop`](Self::toggle_desktop); returns whether
/// the model actually changed.
pub fn set_desktop(&mut self, on: bool) -> bool {
if !self.abs_ok || self.desktop == on {
return false;
}
self.desktop = on;
self.pending_rel = (0, 0);
self.pending_abs = None;
true
}
/// Whether a regained focus should re-engage: yes unless the user released /// Whether a regained focus should re-engage: yes unless the user released
/// deliberately (the chord keeps its meaning across an Alt-Tab). /// deliberately (the chord keeps its meaning across an Alt-Tab).
pub fn should_reengage(&self) -> bool { pub fn should_reengage(&self) -> bool {
@@ -117,6 +170,7 @@ impl Capture {
return false; return false;
} }
self.pending_rel = (0, 0); // never flush motion gathered while captured self.pending_rel = (0, 0); // never flush motion gathered while captured
self.pending_abs = None;
for vk in self.held_keys.drain() { for vk in self.held_keys.drain() {
send(&self.connector, InputKind::KeyUp, vk as u32, 0, 0, 0); send(&self.connector, InputKind::KeyUp, vk as u32, 0, 0, 0);
} }
@@ -132,22 +186,42 @@ impl Capture {
true true
} }
/// Forward the coalesced motion delta, if any — one datagram per loop iteration. /// Forward the coalesced motion, if any — one datagram per loop iteration. Only one
/// of the two stores is ever populated (the run loop routes by [`desktop`](Self::desktop)).
pub fn flush_motion(&mut self) { pub fn flush_motion(&mut self) {
let (dx, dy) = std::mem::take(&mut self.pending_rel); let (dx, dy) = std::mem::take(&mut self.pending_rel);
if dx != 0 || dy != 0 { if dx != 0 || dy != 0 {
send(&self.connector, InputKind::MouseMove, 0, dx, dy, 0); send(&self.connector, InputKind::MouseMove, 0, dx, dy, 0);
} }
if let Some(a) = self.pending_abs.take() {
send(
&self.connector,
InputKind::MouseMoveAbs,
0,
a.x,
a.y,
Self::touch_flags(a.w, a.h),
);
}
} }
/// Relative motion (SDL relative mouse mode delivers raw deltas while locked). /// Relative motion (SDL relative mouse mode delivers raw deltas while locked).
pub fn on_motion(&mut self, xrel: f32, yrel: f32) { pub fn on_motion(&mut self, xrel: f32, yrel: f32) {
if self.captured { if self.captured && !self.desktop {
self.pending_rel.0 += xrel as i32; self.pending_rel.0 += xrel as i32;
self.pending_rel.1 += yrel as i32; self.pending_rel.1 += yrel as i32;
} }
} }
/// Desktop-model motion: the cursor's position mapped into the letterboxed content
/// rect. Latest-wins — intermediate positions carry no information the final one
/// doesn't (unlike deltas, which must sum).
pub fn on_motion_abs(&mut self, abs: Abs) {
if self.captured && self.desktop {
self.pending_abs = Some(abs);
}
}
pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode) { pub fn on_key_down(&mut self, sc: sdl3::keyboard::Scancode) {
if !self.captured { if !self.captured {
return; return;
+2
View File
@@ -16,6 +16,8 @@
#[cfg(any(target_os = "linux", windows))] #[cfg(any(target_os = "linux", windows))]
pub mod csc; pub mod csc;
#[cfg(any(target_os = "linux", windows))]
pub mod cursor;
#[cfg(windows)] #[cfg(windows)]
pub mod d3d11; pub mod d3d11;
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
+241 -21
View File
@@ -20,11 +20,11 @@ use crate::vk::{FrameInput, Presenter};
use anyhow::{Context as _, Result}; use anyhow::{Context as _, Result};
use pf_client_core::gamepad::GamepadService; use pf_client_core::gamepad::GamepadService;
use pf_client_core::session::{self, SessionEvent, SessionHandle, SessionParams, Stats}; use pf_client_core::session::{self, SessionEvent, SessionHandle, SessionParams, Stats};
use pf_client_core::trust::{StatsVerbosity, TouchMode}; use pf_client_core::trust::{MouseMode, StatsVerbosity, TouchMode};
use pf_client_core::video::VulkanDecodeDevice; use pf_client_core::video::VulkanDecodeDevice;
use pf_client_core::video::{DecodedFrame, DecodedImage}; use pf_client_core::video::{DecodedFrame, DecodedImage};
use punktfunk_core::client::NativeClient; use punktfunk_core::client::NativeClient;
use punktfunk_core::config::Mode; use punktfunk_core::config::{CompositorPref, Mode};
use sdl3::event::{Event, WindowEvent}; use sdl3::event::{Event, WindowEvent};
use sdl3::keyboard::Mod; use sdl3::keyboard::Mod;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
@@ -48,6 +48,11 @@ pub struct SessionOpts {
/// `Pointer` (absolute cursor), or `Touch` (real multi-touch passthrough). Latched per /// `Pointer` (absolute cursor), or `Touch` (real multi-touch passthrough). Latched per
/// session — a mouse-only client leaves this at the default and never sees a finger. /// session — a mouse-only client leaves this at the default and never sees a finger.
pub touch_mode: TouchMode, pub touch_mode: TouchMode,
/// Physical-mouse model: `Capture` (pointer lock + relative, the default) or `Desktop`
/// (uncaptured absolute pointer — design/remote-desktop-sweep.md M1). Ctrl+Alt+Shift+M
/// flips it live; silently resolves to capture on hosts without absolute injection
/// (gamescope).
pub mouse_mode: MouseMode,
/// Reverse the scroll direction sent to the host ([`Settings::invert_scroll`]). /// Reverse the scroll direction sent to the host ([`Settings::invert_scroll`]).
pub invert_scroll: bool, pub invert_scroll: bool,
/// Emit the `{"ready":true}` stdout line after the first presented frame. /// Emit the `{"ready":true}` stdout line after the first presented frame.
@@ -228,6 +233,19 @@ struct StreamState {
/// window-normalized position must be re-based onto the content rect). `None` until /// window-normalized position must be re-based onto the content rect). `None` until
/// the first frame; touches before then have nothing to map onto and are dropped. /// the first frame; touches before then have nothing to map onto and are dropped.
last_video: Option<(u32, u32)>, last_video: Option<(u32, u32)>,
/// Client-side cursor rendering (M2 cursor channel) — created with the connector; inert
/// when the host didn't negotiate the channel.
cursor_chan: Option<crate::cursor::CursorChannel>,
/// Last observed `relative_hint` (M3): the auto-flip fires on CHANGES only, so it never
/// fights a user who chorded away from the hinted model.
last_hint: Option<bool>,
/// The user flipped the model manually (⌃⌥⇧M) — the standing hint stops driving until
/// the HOST's intent next changes (a fresh hint edge clears this and applies).
hint_override: bool,
/// Last `CursorRenderMode.client_draws` told to the host (§8 mid-stream render flip);
/// `None` = nothing sent yet. Edge-detected each iteration from the live mouse model, so
/// the chord, the M3 auto-flip, and engage/release all reconcile through one path.
sent_client_draws: Option<bool>,
} }
impl StreamState { impl StreamState {
@@ -258,6 +276,10 @@ impl StreamState {
frames: wake_rx, frames: wake_rx,
connector: None, connector: None,
capture: None, capture: None,
cursor_chan: None,
last_hint: None,
hint_override: false,
sent_client_draws: None,
force_software, force_software,
canceled: false, canceled: false,
ready_announced: false, ready_announced: false,
@@ -333,6 +355,11 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
// bottom-right corner (the reported bug). The menu/library is keyboard+gamepad-driven // bottom-right corner (the reported bug). The menu/library is keyboard+gamepad-driven
// and consumes no mouse, so nothing wanted these synthetic events anyway. // and consumes no mouse, so nothing wanted these synthetic events anyway.
sdl3::hint::set("SDL_TOUCH_MOUSE_EVENTS", "0"); sdl3::hint::set("SDL_TOUCH_MOUSE_EVENTS", "0");
// The Wayland `app_id` (and X11 WM_CLASS) — compositors match it against
// io.unom.Punktfunk.desktop for the window/taskbar icon. Without it SDL uses a generic
// identity and the session window gets the default-Wayland icon (the Linux analog of
// the AppUserModelID adoption above).
sdl3::hint::set("SDL_APP_ID", "io.unom.Punktfunk");
let sdl = sdl3::init().context("SDL init")?; let sdl = sdl3::init().context("SDL init")?;
let video = sdl.video().context("SDL video")?; let video = sdl.video().context("SDL video")?;
let events = sdl.event().context("SDL events")?; let events = sdl.event().context("SDL events")?;
@@ -485,7 +512,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
WindowEvent::FocusLost => { WindowEvent::FocusLost => {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(false) { if cap.release(false) {
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
tracing::info!("focus lost — input released"); tracing::info!("focus lost — input released");
} }
} }
@@ -496,7 +523,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.should_reengage() { if cap.should_reengage() {
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
tracing::info!("focus gained — input recaptured"); tracing::info!("focus gained — input recaptured");
} }
} }
@@ -532,20 +559,48 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.captured() { if cap.captured() {
cap.release(true); cap.release(true);
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} else { } else {
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
} }
tracing::info!(captured = cap.captured(), "chord: release/engage"); tracing::info!(captured = cap.captured(), "chord: release/engage");
} }
continue; continue;
} }
// Mouse model flip (capture ⇄ desktop) — applies immediately when
// engaged; a released stream just changes what the next engage does.
if chord && sc == Scancode::M {
if let Some(st) = stream.as_mut() {
let mut flipped = false;
if let Some(cap) = st.capture.as_mut() {
match cap.toggle_desktop() {
Some(desktop) => {
if cap.captured() {
apply_capture(&mut window, &mouse, true, desktop);
}
flipped = true;
tracing::info!(desktop, "chord: mouse mode");
}
None => tracing::info!(
"chord: mouse mode — host has no absolute pointer \
(gamescope), staying captured"
),
}
}
// A manual flip outranks the standing hint until the host's
// intent next CHANGES (M3 — the hint edge clears this).
if flipped {
st.hint_override = true;
}
}
continue;
}
if chord && sc == Scancode::D { if chord && sc == Scancode::D {
if let Some(st) = &mut stream { if let Some(st) = &mut stream {
tracing::info!("chord: disconnect"); tracing::info!("chord: disconnect");
st.request_quit(); st.request_quit();
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
// The pump emits Ended(None); the end path routes per mode. // The pump emits Ended(None); the end path routes per mode.
} }
continue; continue;
@@ -578,9 +633,34 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
cap.on_key_up(sc); cap.on_key_up(sc);
} }
} }
Event::MouseMotion { xrel, yrel, .. } => { Event::MouseMotion {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { x, y, xrel, yrel, ..
cap.on_motion(xrel, yrel); } => {
if let Some(st) = stream.as_mut() {
let video = st.last_video;
if let Some(cap) = st.capture.as_mut() {
if cap.desktop() {
// Desktop model: the cursor's window position through the
// letterbox (same mapping as a pointer-mode finger).
// Before the first decoded frame there is nothing to map
// onto — dropped, like touch.
if let Some(video) = video {
let (lw, lh) = window.size();
let nx = x / lw.max(1) as f32;
let ny = y / lh.max(1) as f32;
let (ax, ay, aw, ah) =
finger_to_content(window.size_in_pixels(), video, nx, ny);
cap.on_motion_abs(Abs {
x: ax,
y: ay,
w: aw,
h: ah,
});
}
} else {
cap.on_motion(xrel, yrel);
}
}
} }
} }
Event::MouseButtonDown { mouse_btn, .. } => { Event::MouseButtonDown { mouse_btn, .. } => {
@@ -588,7 +668,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if !cap.captured() { if !cap.captured() {
// The engaging click is suppressed toward the host. // The engaging click is suppressed toward the host.
cap.engage(); cap.engage();
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
} else { } else {
cap.on_button_down(mouse_btn); cap.on_button_down(mouse_btn);
} }
@@ -690,6 +770,67 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
cap.flush_motion(); cap.flush_motion();
} }
// Cursor channel (M2): drain forwarded shape/state and drive the local OS cursor —
// only meaningful in the desktop mouse model (capture's relative lock hides it).
if let Some(st) = stream.as_mut() {
if let (Some(chan), Some(c)) = (st.cursor_chan.as_mut(), st.connector.as_ref()) {
let desktop_active = st
.capture
.as_ref()
.is_some_and(|cap| cap.captured() && cap.desktop());
chan.pump(c, &mouse, desktop_active);
// §8 mid-stream render flip: tell the host who renders the pointer whenever
// the local model changes. Desktop-active = we draw it (host excludes +
// forwards); anything else — the capture model OR a released pointer — the
// host composites it into the video (full fidelity, the pre-channel look).
// One edge-detected reconciler covers the chord, the M3 auto-flip, and
// engage/release alike.
if chan.negotiated() && st.sent_client_draws != Some(desktop_active) {
st.sent_client_draws = Some(desktop_active);
let _ = c.set_cursor_render(desktop_active);
}
}
// M3 — host-driven mode flip: `relative_hint` set = a host app grabbed/hid the
// pointer (run captured relative, like a game expects); clear = the desktop is
// back (return to absolute, local cursor reappearing at the host's position).
// Edge-triggered so a user's manual chord isn't fought: the override latch
// holds until the HOST's intent next changes.
let hint_state = st.cursor_chan.as_ref().and_then(|ch| ch.state());
if let Some(hs) = hint_state {
let hint = hs.relative_hint();
if st.last_hint != Some(hint) {
st.last_hint = Some(hint);
st.hint_override = false;
}
if !st.hint_override {
let video = st.last_video;
if let Some(cap) = st.capture.as_mut() {
// Desired model: hint ⇒ capture (desktop off); clear ⇒ desktop on.
if cap.captured() && cap.set_desktop(!hint) {
apply_capture(&mut window, &mouse, true, cap.desktop());
if cap.desktop() {
// Reappear where the host last had the pointer, so the
// hand-back is seamless (Parsec's positionX/Y idea).
if let Some(video) = video {
let (wx, wy) = content_to_window(
window.size(),
window.size_in_pixels(),
video,
hs.x,
hs.y,
);
mouse.warp_mouse_in_window(&window, wx, wy);
}
}
tracing::info!(
desktop = cap.desktop(),
"host cursor hint: mouse model flipped"
);
}
}
}
}
}
// Text input follows the overlay's editing state (edge-triggered). // Text input follows the overlay's editing state (edge-triggered).
let want_text = overlay.as_ref().is_some_and(|o| o.text_input_active()); let want_text = overlay.as_ref().is_some_and(|o| o.text_input_active());
@@ -709,7 +850,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
while escape_rx.try_recv().is_ok() { while escape_rx.try_recv().is_ok() {
if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) { if let Some(cap) = stream.as_mut().and_then(|s| s.capture.as_mut()) {
if cap.release(true) { if cap.release(true) {
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} }
} }
if fullscreen && !opts.fullscreen { if fullscreen && !opts.fullscreen {
@@ -722,7 +863,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = &mut stream { if let Some(st) = &mut stream {
tracing::info!("controller chord: disconnect"); tracing::info!("controller chord: disconnect");
st.request_quit(); st.request_quit();
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
} }
} }
@@ -813,10 +954,28 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
.ok(); .ok();
gamepad.attach(c.clone()); gamepad.attach(c.clone());
st.clock_offset = Some(c.clock_offset_shared()); st.clock_offset = Some(c.clock_offset_shared());
let mut cap = Capture::new(c.clone(), opts.touch_mode, opts.invert_scroll); // gamescope's EIS grants only a relative pointer — absolute sends
// would be dropped, so the desktop model is pinned off there. Auto
// (an older host that didn't say) stays allowed: Windows hosts and
// pre-Welcome-compositor Linux hosts both take absolute.
let abs_ok = c.resolved_compositor != CompositorPref::Gamescope;
if opts.mouse_mode == MouseMode::Desktop && !abs_ok {
tracing::info!(
"desktop mouse mode unavailable on a gamescope host \
(relative-only input) using capture"
);
}
let mut cap = Capture::new(
c.clone(),
opts.touch_mode,
opts.invert_scroll,
opts.mouse_mode,
abs_ok,
);
cap.engage(); // capture engages when the stream starts (ui_stream parity) cap.engage(); // capture engages when the stream starts (ui_stream parity)
apply_capture(&mut window, &mouse, true); apply_capture(&mut window, &mouse, true, cap.desktop());
st.capture = Some(cap); st.capture = Some(cap);
st.cursor_chan = Some(crate::cursor::CursorChannel::new(&c));
st.connector = Some(c); st.connector = Some(c);
if let Some(f) = opts.on_connected.as_mut() { if let Some(f) = opts.on_connected.as_mut() {
f(fingerprint); f(fingerprint);
@@ -865,7 +1024,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(st) = stream.take() { if let Some(st) = stream.take() {
st.shutdown(); st.shutdown();
} }
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
if let Some(o) = overlay.as_mut() { if let Some(o) = overlay.as_mut() {
// A user-canceled dial ends silently — no error scene. // A user-canceled dial ends silently — no error scene.
if canceled { if canceled {
@@ -882,7 +1041,7 @@ fn run_inner(mut opts: SessionOpts, mut mode: ModeCtl) -> Result<Option<Outcome>
if let Some(cap) = &mut st.capture { if let Some(cap) = &mut st.capture {
cap.release(true); cap.release(true);
} }
apply_capture(&mut window, &mouse, false); apply_capture(&mut window, &mouse, false, false);
match &mode { match &mode {
ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)), ModeCtl::Single(_) => break 'main Some(Outcome::Ended(reason)),
ModeCtl::Browse(_) => { ModeCtl::Browse(_) => {
@@ -1472,11 +1631,22 @@ impl ResizeIndicator {
/// with a low-level keyboard hook, the same mechanism the WinUI shell's in-process /// with a low-level keyboard hook, the same mechanism the WinUI shell's in-process
/// client used its own WH_KEYBOARD_LL hooks for. Not engaged on Linux: the compositor /// client used its own WH_KEYBOARD_LL hooks for. Not engaged on Linux: the compositor
/// shortcut-inhibit story stays the shells' concern (Settings.inhibit_shortcuts). /// shortcut-inhibit story stays the shells' concern (Settings.inhibit_shortcuts).
fn apply_capture(window: &mut sdl3::video::Window, mouse: &sdl3::mouse::MouseUtil, on: bool) { ///
mouse.set_relative_mouse_mode(window, on); /// The `desktop` mouse model never locks: the pointer roams (and leaves the window)
/// freely, the local cursor is hidden over the window — the host's composited cursor,
/// tracking our absolute sends, is the one you see (until the M2 cursor channel flips
/// who draws it) — and system chords stay local (a remote desktop is something you
/// Alt-Tab away from, not into). `desktop` only matters while `on`.
fn apply_capture(
window: &mut sdl3::video::Window,
mouse: &sdl3::mouse::MouseUtil,
on: bool,
desktop: bool,
) {
mouse.set_relative_mouse_mode(window, on && !desktop);
mouse.show_cursor(!on); mouse.show_cursor(!on);
#[cfg(windows)] #[cfg(windows)]
window.set_keyboard_grab(on); window.set_keyboard_grab(on && !desktop);
} }
/// Is this SDL touch device a real touchscreen (DIRECT, window-relative coordinates)? /// Is this SDL touch device a real touchscreen (DIRECT, window-relative coordinates)?
@@ -1584,6 +1754,32 @@ fn finger_to_content(
(cx.round() as i32, cy.round() as i32, dw as u32, dh as u32) (cx.round() as i32, cy.round() as i32, dw as u32, dh as u32)
} }
/// The inverse direction of [`finger_to_content`] for the M3 reappear warp: a HOST-frame pixel
/// (`video` space — what `CursorState` carries) → LOGICAL window coordinates (what
/// `warp_mouse_in_window` takes). Maps through the aspect-fit letterbox (physical), then
/// physical → logical via the window's pixel density; out-of-range host coords clamp into the
/// content rect so the warp always lands on the video.
fn content_to_window(
logical: (u32, u32),
surface: (u32, u32),
video: (u32, u32),
x: i32,
y: i32,
) -> (f32, f32) {
let (pw, ph) = (f64::from(surface.0), f64::from(surface.1));
let (vw, vh) = (f64::from(video.0.max(1)), f64::from(video.1.max(1)));
let scale = (pw / vw).min(ph / vh);
let (dw, dh) = ((vw * scale).max(1.0), (vh * scale).max(1.0));
let ox = (pw - dw) / 2.0;
let oy = (ph - dh) / 2.0;
let px = ox + (f64::from(x).clamp(0.0, vw - 1.0)) * scale;
let py = oy + (f64::from(y).clamp(0.0, vh - 1.0)) * scale;
// Physical → logical (HiDPI): the ratio of the window's logical size to its pixel size.
let lx = px * f64::from(logical.0) / pw.max(1.0);
let ly = py * f64::from(logical.1) / ph.max(1.0);
(lx as f32, ly as f32)
}
/// The presenter's share of the unified stats window — folded into each printed line. /// The presenter's share of the unified stats window — folded into each printed line.
#[derive(Default)] #[derive(Default)]
struct PresentedWindow { struct PresentedWindow {
@@ -1594,7 +1790,7 @@ struct PresentedWindow {
/// The capture hints (`ui_stream` parity — the words the user reads while released). /// The capture hints (`ui_stream` parity — the words the user reads while released).
const HINT_KEYBOARD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \ const HINT_KEYBOARD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \
Ctrl+Alt+Shift+D disconnects · Ctrl+Alt+Shift+S stats"; Ctrl+Alt+Shift+M mouse mode · Ctrl+Alt+Shift+D disconnects · Ctrl+Alt+Shift+S stats";
const HINT_WITH_PAD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \ const HINT_WITH_PAD: &str = "Click the stream to capture input · Ctrl+Alt+Shift+Q releases · \
Ctrl+Alt+Shift+D disconnects · hold L1 + R1 + Start + Select to leave"; Ctrl+Alt+Shift+D disconnects · hold L1 + R1 + Start + Select to leave";
@@ -1681,6 +1877,30 @@ fn stats_text(
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn content_to_window_inverts_the_letterbox() {
// 1920×1080 video letterboxed in a 1600×1200 (4:3) window at 2× HiDPI: pillarless
// top/bottom bars — scale = 1600/1920, dh = 900, oy = 150 (physical).
let logical = (800u32, 600u32);
let surface = (1600u32, 1200u32);
let video = (1920u32, 1080u32);
// The host-frame center must land at the logical window center.
let (wx, wy) = content_to_window(logical, surface, video, 960, 540);
assert!((wx - 400.0).abs() < 1.0, "wx = {wx}");
assert!((wy - 300.0).abs() < 1.0, "wy = {wy}");
// Roundtrip through the forward mapping: normalized window pos → the same host
// content-rect pixel (finger_to_content returns content-RECT coords, i.e. the
// host pixel scaled by the letterbox factor).
let (nx, ny) = (wx / logical.0 as f32, wy / logical.1 as f32);
let (cx, cy, dw, dh) = finger_to_content(surface, video, nx, ny);
assert_eq!((dw, dh), (1600, 900));
assert!((cx - 800).abs() <= 1, "cx = {cx}"); // 960 * (1600/1920)
assert!((cy - 450).abs() <= 1, "cy = {cy}"); // 540 * ( 900/1080)
// Out-of-range host coords clamp into the video, never the bars.
let (_, wy_clamped) = content_to_window(logical, surface, video, 0, 10_000);
assert!(wy_clamped <= 300.0 + 225.0 + 1.0, "wy = {wy_clamped}"); // ≤ bottom of content
}
#[test] #[test]
fn resize_decision_follows_the_d2_discipline() { fn resize_decision_follows_the_d2_discipline() {
let t0 = Instant::now(); let t0 = Instant::now();
+9 -1
View File
@@ -30,9 +30,17 @@ impl Presenter {
// switch modes before anything touches this frame. Only where the surface // switch modes before anything touches this frame. Only where the surface
// offers HDR10 — otherwise PQ stays on the SDR swapchain and the CSC shader // offers HDR10 — otherwise PQ stays on the SDR swapchain and the CSC shader
// tonemaps (mode 1). // tonemaps (mode 1).
//
// CPU frames NEVER take the HDR10 surface: software decode uploads swscale RGBA with
// no CSC/tonemap pass, so on a mode-0 swapchain that sRGB-encoded content would be
// composed as PQ — the field-reported psychedelic cyan/magenta picture (reproduced
// 2026-07-21: Fedora-class client, no hw HEVC decode, GNOME/Mesa offering HDR10 even
// on an SDR desktop). On the SDR swapchain the same frames are merely untonemapped
// (washed out) — wrong in the known, benign way until the CPU lane grows a real
// PQ→sRGB pass.
let frame_pq = match &input { let frame_pq = match &input {
FrameInput::Redraw => None, FrameInput::Redraw => None,
FrameInput::Cpu(f) => Some(f.color.is_pq()), FrameInput::Cpu(_) => Some(false),
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
FrameInput::Dmabuf(d) => Some(d.color.is_pq()), FrameInput::Dmabuf(d) => Some(d.color.is_pq()),
FrameInput::VkFrame(v) => Some(v.color.is_pq()), FrameInput::VkFrame(v) => Some(v.color.is_pq()),
+9
View File
@@ -617,6 +617,15 @@ pub(super) fn pick_formats(
surface: vk::SurfaceKHR, surface: vk::SurfaceKHR,
colorspace_ext: bool, colorspace_ext: bool,
) -> Result<(vk::SurfaceFormatKHR, Option<vk::SurfaceFormatKHR>)> { ) -> Result<(vk::SurfaceFormatKHR, Option<vk::SurfaceFormatKHR>)> {
// `PUNKTFUNK_HDR10=0` (explicit-off grammar) refuses the HDR10/ST.2084 swapchain outright,
// pinning PQ streams to the shader tonemap on an SDR surface. Two reasons this exists:
// desktop compositors newly offer HDR10 even on SDR desktops (GNOME 48 / Plasma 6 with
// Mesa ≥ 25.1 — a lane that otherwise engages silently), and it is the A/B lever that
// splits "HDR10 passthrough composes wrong" from "the decoded planes are wrong" in the
// field without rebuilding anything.
let colorspace_ext = colorspace_ext
&& !std::env::var("PUNKTFUNK_HDR10")
.is_ok_and(|v| matches!(v.as_str(), "0" | "false" | "off" | "no"));
let formats = unsafe { surface_i.get_physical_device_surface_formats(pdev, surface) }?; let formats = unsafe { surface_i.get_physical_device_surface_formats(pdev, surface) }?;
let mut sdr = None; let mut sdr = None;
for want in [vk::Format::B8G8R8A8_UNORM, vk::Format::R8G8B8A8_UNORM] { for want in [vk::Format::B8G8R8A8_UNORM, vk::Format::R8G8B8A8_UNORM] {
+33 -4
View File
@@ -72,13 +72,14 @@ pub use session::{session_epoch, try_recover_session};
#[path = "vdisplay/routing.rs"] #[path = "vdisplay/routing.rs"]
pub(crate) mod routing; pub(crate) mod routing;
pub use routing::{ pub use routing::{
apply_input_env, restore_managed_session, restore_takeover_on_startup, start_restore_worker, apply_input_env, managed_session_available, restore_managed_session,
wants_dedicated_game_session, restore_takeover_on_startup, start_restore_worker, wants_dedicated_game_session,
}; };
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub use routing::{ pub use routing::{
cancel_pending_tv_restore, dedicated_game_exited, launch_into_gamescope_session, cancel_pending_tv_restore, dedicated_game_exited, gamescope_xwayland_cursor_targets,
launch_is_nested, steam_appid_from_launch, watch_steam_game_exit, launch_into_gamescope_session, launch_is_nested, steam_appid_from_launch,
watch_steam_game_exit,
}; };
/// Compositors punktfunk knows how to drive (plan §6). /// Compositors punktfunk knows how to drive (plan §6).
@@ -254,6 +255,34 @@ pub fn detect() -> Result<Compositor> {
} }
} }
/// Attach-only probes: while any scope is held, backend `create` paths must not stop, relaunch,
/// or take over box sessions — they may only attach to an already-live output, and fail fast
/// otherwise. The capture-loss rebuild holds one for its first seconds: right after a capture
/// loss the active-session detection can be STALE (a Game→Desktop switch observed live: the
/// probe's gamescope re-acquire restarted `gamescope-session.target` and yanked the user out of
/// the KDE session they had just switched to). A counter, so overlapping scopes compose.
static REBUILD_PROBES: std::sync::atomic::AtomicU32 = std::sync::atomic::AtomicU32::new(0);
/// RAII scope marking pipeline builds as attach-only probes (see [`rebuild_probe_active`]).
pub struct RebuildProbeScope(());
pub fn rebuild_probe_scope() -> RebuildProbeScope {
REBUILD_PROBES.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
RebuildProbeScope(())
}
impl Drop for RebuildProbeScope {
fn drop(&mut self) {
REBUILD_PROBES.fetch_sub(1, std::sync::atomic::Ordering::SeqCst);
}
}
/// Is any [`rebuild_probe_scope`] active? Destructive session operations (stop/relaunch/
/// takeover-restart) must be skipped while true.
pub fn rebuild_probe_active() -> bool {
REBUILD_PROBES.load(std::sync::atomic::Ordering::SeqCst) > 0
}
/// Open the virtual-display driver for `compositor`. /// Open the virtual-display driver for `compositor`.
pub fn open(compositor: Compositor) -> Result<Box<dyn VirtualDisplay>> { pub fn open(compositor: Compositor) -> Result<Box<dyn VirtualDisplay>> {
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
@@ -70,6 +70,12 @@ pub struct VirtualOutput {
/// Linux-only (the keep-alive pool is Linux). /// Linux-only (the keep-alive pool is Linux).
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub pool_gen: Option<u64>, pub pool_gen: Option<u64>,
/// The backend created the output at a SACRIFICIAL mode and the producer will renegotiate the
/// live stream to `preferred_mode`'s dims (KWin's screencast only rebuilds its format offer —
/// and thus its refresh cap — on a size change while recording; see kwin.rs `create`). The
/// capturer must hold frames until that renegotiation lands. Linux-only.
#[cfg(target_os = "linux")]
pub expect_exact_dims: bool,
} }
impl VirtualOutput { impl VirtualOutput {
@@ -93,6 +99,8 @@ impl VirtualOutput {
reused_gen: None, reused_gen: None,
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pool_gen: None, pool_gen: None,
#[cfg(target_os = "linux")]
expect_exact_dims: false,
} }
} }
} }
@@ -134,6 +142,25 @@ pub trait VirtualDisplay: Send {
/// the backend's default EDID. Default: no-op — only the Windows pf-vdisplay backend can mint /// the backend's default EDID. Default: no-op — only the Windows pf-vdisplay backend can mint
/// per-monitor EDIDs today (the Linux compositors' virtual outputs take no EDID from us). /// per-monitor EDIDs today (the Linux compositors' virtual outputs take no EDID from us).
fn set_client_hdr(&mut self, _hdr: Option<punktfunk_core::quic::HdrMeta>) {} fn set_client_hdr(&mut self, _hdr: Option<punktfunk_core::quic::HdrMeta>) {}
/// Ask the backend for an OUT-OF-BAND cursor on the created output (the cursor channel):
/// the compositor/OS stops compositing the pointer into captured frames and the capture
/// layer surfaces shape/position separately. Carried on the backend instance; set once
/// before [`create`](Self::create) (both session paths pass `cursor_forward`). Off = the
/// compositor EMBEDS the pointer into frames — zero host-side cursor work, the pre-channel
/// path — which is what every session without the negotiated cursor cap gets (Moonlight /
/// GameStream / legacy clients / capture-mode starts), mirroring the Windows no-regression
/// gate. Implementations: Windows pf-vdisplay (IddCx hardware cursor, driver proto v5);
/// KWin (zkde `pointer` metadata vs embedded); Mutter (`cursor-mode` metadata vs embedded);
/// wlroots/hyprland (portal `CursorMode`). Default: no-op (gamescope has no cursor either
/// way — see the Phase C source).
fn set_hw_cursor(&mut self, _on: bool) {}
/// The out-of-band-cursor request currently set (see [`set_hw_cursor`](Self::set_hw_cursor)).
/// The registry includes it in the keep-alive REUSE key: a kept embedded-pointer display can
/// never serve a cursor-channel session (its stream has no cursor metadata to forward) nor
/// vice versa (the pointer would be missing from frames).
fn hw_cursor(&self) -> bool {
false
}
/// The stable identity slot the backend resolved for the most recent [`create`](Self::create) — /// The stable identity slot the backend resolved for the most recent [`create`](Self::create) —
/// the per-client id the identity policy assigned (`Some`), or `None` for shared/anonymous. The /// the per-client id the identity policy assigned (`Some`), or `None` for shared/anonymous. The
/// registry reads it right after `create` to key the display's group **arrangement** (manual /// registry reads it right after `create` to key the display's group **arrangement** (manual
@@ -253,6 +253,7 @@ impl VirtualDisplay for GamescopeDisplay {
ownership: DisplayOwnership::External, ownership: DisplayOwnership::External,
reused_gen: None, reused_gen: None,
pool_gen: None, pool_gen: None,
expect_exact_dims: false,
}); });
} }
check_gamescope_version(); // diagnostic only — warns on known-deadlock-prone versions check_gamescope_version(); // diagnostic only — warns on known-deadlock-prone versions
@@ -325,6 +326,29 @@ fn create_managed_session(client: &str, mode: Mode) -> Result<VirtualOutput> {
if steamos_session_present() { if steamos_session_present() {
return create_managed_session_steamos(mode); return create_managed_session_steamos(mode);
} }
// Attach-only rebuild probe: reuse a live same-mode session, but NEVER stop/relaunch box
// sessions — right after a capture loss the caller's session detection can be stale, and a
// destructive rebuild here would fight the session the user just switched to.
if crate::rebuild_probe_active() {
let guard = MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner());
let same_mode = guard.as_ref().is_some_and(|s| {
s.width == mode.width && s.height == mode.height && s.refresh_hz == mode.refresh_hz
});
if same_mode {
if let Some(node_id) = find_gamescope_node() {
point_injector_at_eis();
tracing::info!(
node_id,
"gamescope session: attach-only probe reusing live node"
);
return Ok(managed_output(node_id, mode));
}
}
return Err(anyhow!(
"gamescope session has no attachable live node — attach-only rebuild probe refuses \
to stop/relaunch box sessions (re-detection follows the live session)"
));
}
// Steam is single-instance: if the box autologged into gaming mode on a physical display (the // Steam is single-instance: if the box autologged into gaming mode on a physical display (the
// Bazzite default — `gamescope-session-plus@ogui-steam` on the TV), that session holds Steam and // Bazzite default — `gamescope-session-plus@ogui-steam` on the TV), that session holds Steam and
// renders to the TV's native mode, which we'd capture instead of the client's. Free Steam by // renders to the TV's native mode, which we'd capture instead of the client's. Free Steam by
@@ -386,6 +410,7 @@ fn managed_output(node_id: u32, mode: Mode) -> VirtualOutput {
ownership: DisplayOwnership::SessionManaged, ownership: DisplayOwnership::SessionManaged,
reused_gen: None, reused_gen: None,
pool_gen: None, pool_gen: None,
expect_exact_dims: false,
} }
} }
@@ -483,7 +508,7 @@ pub fn launch_into_session(cmd: &str) -> Result<std::process::Child> {
let mut c = Command::new("sh"); let mut c = Command::new("sh");
c.arg("-c").arg(cmd); c.arg("-c").arg(cmd);
match discover_session_display_env() { match discover_session_display_env() {
Some((x11, wayland)) => { Some((x11, wayland, _xauth)) => {
tracing::info!( tracing::info!(
command = %cmd, command = %cmd,
x11_display = x11.as_deref().unwrap_or("-"), x11_display = x11.as_deref().unwrap_or("-"),
@@ -508,11 +533,71 @@ pub fn launch_into_session(cmd: &str) -> Result<std::process::Child> {
.context("spawn launch command into gamescope session") .context("spawn launch command into gamescope session")
} }
/// Find the live gamescope session's `(DISPLAY, WAYLAND_DISPLAY)` by scanning same-uid processes /// EVERY nested Xwayland the running gamescope session exposes, as `(DISPLAY, XAUTHORITY)` pairs
/// for one whose environment carries `GAMESCOPE_WAYLAND_DISPLAY` (gamescope sets it for everything /// for the XFixes cursor source (remote-desktop-sweep Phase C). gamescope can run several
/// it runs — Steam, the game, our own nested `sh`). The Wayland value returned is that gamescope /// (`--xwayland-count N` — Steam Gaming Mode uses 2: one for Big Picture, one for the game), and
/// socket; `DISPLAY` is the nested Xwayland. Either can be individually absent. /// the pointer lives on whichever is FOCUSED — so the source connects to all and follows the one
fn discover_session_display_env() -> Option<(Option<String>, Option<String>)> { /// whose pointer moves. The host is not a gamescope child, so gamescope's auth cookie rides along
/// when a process exposes it. Empty when no gamescope session is running / none exposes a `DISPLAY`.
#[cfg(target_os = "linux")]
pub(crate) fn xwayland_cursor_targets() -> Vec<(String, Option<String>)> {
// SAFETY: `getuid()` is a parameterless POSIX call that always succeeds and touches no memory.
let uid = unsafe { libc::getuid() };
let mut out: Vec<(String, Option<String>)> = Vec::new();
let Ok(entries) = std::fs::read_dir("/proc") else {
return out;
};
for e in entries.flatten() {
let name = e.file_name();
let Some(pid_str) = name.to_str() else {
continue;
};
if !pid_str.bytes().all(|b| b.is_ascii_digit()) {
continue;
}
let Ok(md) = std::fs::metadata(e.path()) else {
continue;
};
use std::os::unix::fs::MetadataExt;
if md.uid() != uid {
continue;
}
let Ok(raw) = std::fs::read(e.path().join("environ")) else {
continue;
};
let (mut display, mut is_gamescope, mut xauth) = (None, false, None);
for kv in raw.split(|&b| b == 0) {
let kv = String::from_utf8_lossy(kv);
if kv.starts_with("GAMESCOPE_WAYLAND_DISPLAY=") {
is_gamescope = true;
} else if let Some(v) = kv.strip_prefix("DISPLAY=") {
if !v.is_empty() {
display = Some(v.to_string());
}
} else if let Some(v) = kv.strip_prefix("XAUTHORITY=") {
if !v.is_empty() {
xauth = Some(v.to_string());
}
}
}
if let (true, Some(d)) = (is_gamescope, display) {
// Distinct DISPLAY only; prefer the first non-empty XAUTHORITY seen for it.
match out.iter_mut().find(|(dd, _)| *dd == d) {
Some((_, xa)) if xa.is_none() => *xa = xauth,
Some(_) => {}
None => out.push((d, xauth)),
}
}
}
out
}
/// Find the live gamescope session's `(DISPLAY, WAYLAND_DISPLAY, XAUTHORITY)` by scanning same-uid
/// processes for one whose environment carries `GAMESCOPE_WAYLAND_DISPLAY` (gamescope sets it for
/// everything it runs — Steam, the game, our own nested `sh`). The Wayland value returned is that
/// gamescope socket; `DISPLAY` is the nested Xwayland; `XAUTHORITY` is its auth file (for X
/// clients that aren't gamescope children). Any one can be individually absent.
fn discover_session_display_env() -> Option<(Option<String>, Option<String>, Option<String>)> {
// SAFETY: `getuid()` is a parameterless POSIX call that always succeeds and touches no memory. // SAFETY: `getuid()` is a parameterless POSIX call that always succeeds and touches no memory.
let uid = unsafe { libc::getuid() }; let uid = unsafe { libc::getuid() };
for e in std::fs::read_dir("/proc").ok()?.flatten() { for e in std::fs::read_dir("/proc").ok()?.flatten() {
@@ -535,6 +620,7 @@ fn discover_session_display_env() -> Option<(Option<String>, Option<String>)> {
}; };
let mut display = None; let mut display = None;
let mut gs_wayland = None; let mut gs_wayland = None;
let mut xauth = None;
for kv in raw.split(|&b| b == 0) { for kv in raw.split(|&b| b == 0) {
let kv = String::from_utf8_lossy(kv); let kv = String::from_utf8_lossy(kv);
if let Some(v) = kv.strip_prefix("GAMESCOPE_WAYLAND_DISPLAY=") { if let Some(v) = kv.strip_prefix("GAMESCOPE_WAYLAND_DISPLAY=") {
@@ -545,11 +631,15 @@ fn discover_session_display_env() -> Option<(Option<String>, Option<String>)> {
if !v.is_empty() { if !v.is_empty() {
display = Some(v.to_string()); display = Some(v.to_string());
} }
} else if let Some(v) = kv.strip_prefix("XAUTHORITY=") {
if !v.is_empty() {
xauth = Some(v.to_string());
}
} }
} }
// Only a process INSIDE a gamescope session (it has the marker var) is a valid source. // Only a process INSIDE a gamescope session (it has the marker var) is a valid source.
if gs_wayland.is_some() { if gs_wayland.is_some() {
return Some((display, gs_wayland)); return Some((display, gs_wayland, xauth));
} }
} }
None None
@@ -607,12 +697,17 @@ fn write_steamos_dropin(shim_dir: &std::path::Path, mode: Mode) -> Result<()> {
if let Some(parent) = path.parent() { if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).with_context(|| format!("mkdir {}", parent.display()))?; std::fs::create_dir_all(parent).with_context(|| format!("mkdir {}", parent.display()))?;
} }
// UnsetEnvironment: the same headless-must-not-attach armor `launch_session` gives its
// transient unit — the manager env can carry a stale desktop DISPLAY/WAYLAND_DISPLAY (from a
// portal settle), and gamescope would abort trying to attach to it instead of becoming the
// display server. Unit-scoped belt-and-suspenders on top of the observe_session_instance scrub.
let body = format!( let body = format!(
"[Service]\n\ "[Service]\n\
Environment=PATH={shim}:/usr/bin:/bin:/usr/local/bin\n\ Environment=PATH={shim}:/usr/bin:/bin:/usr/local/bin\n\
Environment=PF_W={w}\n\ Environment=PF_W={w}\n\
Environment=PF_H={h}\n\ Environment=PF_H={h}\n\
Environment=PF_HZ={hz}\n", Environment=PF_HZ={hz}\n\
UnsetEnvironment=DISPLAY WAYLAND_DISPLAY\n",
shim = shim_dir.display(), shim = shim_dir.display(),
w = mode.width, w = mode.width,
h = mode.height, h = mode.height,
@@ -650,6 +745,16 @@ fn create_managed_session_steamos(mode: Mode) -> Result<VirtualOutput> {
} }
*guard = None; // tracked session lost its node — fall through to a clean restart *guard = None; // tracked session lost its node — fall through to a clean restart
} }
// Attach-only rebuild probe: the reuse path above may attach, but a restart of the session
// target is out of bounds — observed live on a Deck: a stale post-capture-loss detection made
// this restart steal the seat back from the KDE session the user had just switched to.
if crate::rebuild_probe_active() {
return Err(anyhow!(
"gamescope has no live node and this is an attach-only rebuild probe — refusing to \
restart {STEAMOS_SESSION_TARGET} (the box may be mid-switch to another session; \
re-detection follows it)"
));
}
let shim_dir = write_headless_shim()?; let shim_dir = write_headless_shim()?;
write_steamos_dropin(&shim_dir, mode)?; write_steamos_dropin(&shim_dir, mode)?;
systemctl_user(&["daemon-reload"]); systemctl_user(&["daemon-reload"]);
@@ -1078,6 +1183,26 @@ pub fn schedule_restore_tv_session() {
} }
} }
/// Does any DRM connector report a physically `connected` display? Scans
/// `/sys/class/drm/*/status` — only connector nodes (`card0-eDP-1`, `card0-HDMI-A-1`, …) have a
/// `status` file, so the bare `cardN` device dirs and `renderD*` nodes filter themselves out. A
/// headless box (VM, panel-less mini PC) has none — in which case a "restore to the physical
/// panel" can only fail, gamescope having no output to drive. Errors (no DRM at all, sysfs
/// unreadable) read as headless: the safe direction is keeping the working session.
fn physical_display_connected() -> bool {
connected_connector_under(std::path::Path::new("/sys/class/drm"))
}
/// [`physical_display_connected`] against an arbitrary sysfs root (the unit-testable core).
fn connected_connector_under(base: &std::path::Path) -> bool {
let Ok(entries) = std::fs::read_dir(base) else {
return false;
};
entries.flatten().any(|e| {
std::fs::read_to_string(e.path().join("status")).is_ok_and(|s| s.trim() == "connected")
})
}
/// Tear down our host-managed session (freeing Steam) and restart the autologin gaming session(s) /// Tear down our host-managed session (freeing Steam) and restart the autologin gaming session(s)
/// we stopped on connect — so the TV returns to gaming mode when no one is streaming. Invoked by /// we stopped on connect — so the TV returns to gaming mode when no one is streaming. Invoked by
/// [`start_restore_worker`] once the debounce deadline passes; takes the stopped-unit list so a /// [`start_restore_worker`] once the debounce deadline passes; takes the stopped-unit list so a
@@ -1089,6 +1214,19 @@ fn do_restore_tv_session() {
{ {
let mut took = STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner()); let mut took = STEAMOS_TOOK_OVER.lock().unwrap_or_else(|e| e.into_inner());
if *took { if *took {
// A box with no physically connected display (a VM, a panel-less mini PC) has no
// "physical gaming session" to restore TO: removing the drop-in and restarting the
// target just crash-loops gamescope (no output to drive) and strands every later
// connect on "no usable compositor". Keep the headless session — and the takeover
// state, so a same-mode reconnect reuses it warm — instead. Checked at restore time
// (not connect time) so plugging a panel in later restores normally.
if !physical_display_connected() {
tracing::info!(
"gamescope (SteamOS): no physical display connected — keeping the headless \
session (nothing to restore to)"
);
return;
}
*took = false; *took = false;
clear_takeover(); // A3: takeover undone — drop the persisted crash-restore marker clear_takeover(); // A3: takeover undone — drop the persisted crash-restore marker
*MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None; *MANAGED_SESSION.lock().unwrap_or_else(|e| e.into_inner()) = None;
@@ -1188,15 +1326,31 @@ pub fn start_restore_worker() -> std::sync::Arc<()> {
/// session). Shared by the attach and host-managed-session paths. /// session). Shared by the attach and host-managed-session paths.
fn point_injector_at_eis() { fn point_injector_at_eis() {
match find_gamescope_eis_socket() { match find_gamescope_eis_socket() {
Some(sock) => match std::fs::write(ei_socket_file(), &sock) { Some(sock) => {
Ok(()) => { // Relay format: line 1 = socket, optional line 2 = the session's CURRENT output
tracing::info!(socket = %sock, "gamescope: pointed injector at the session's EIS socket") // size as "WxH". gamescope's EIS advertises only a degenerate INT32_MAX region, so
// the injector can't learn the output geometry from the protocol — the hint lets
// it scale normalized client positions correctly even when the client streams at
// a different resolution than the session runs (foreign attach, supersample).
let size = current_gamescope_output_size();
let body = match size {
Some((w, h)) => format!("{sock}\n{w}x{h}"),
None => sock.clone(),
};
match std::fs::write(ei_socket_file(), body) {
Ok(()) => {
tracing::info!(
socket = %sock,
output = ?size,
"gamescope: pointed injector at the session's EIS socket"
)
}
Err(e) => tracing::warn!(
error = %e,
"gamescope: could not write the EIS relay file — input may not reach the session"
),
} }
Err(e) => tracing::warn!( }
error = %e,
"gamescope: could not write the EIS relay file — input may not reach the session"
),
},
None => tracing::warn!( None => tracing::warn!(
"gamescope: no connectable gamescope EIS socket found — input won't reach the session" "gamescope: no connectable gamescope EIS socket found — input won't reach the session"
), ),
@@ -1486,7 +1640,35 @@ impl Drop for GamescopeProc {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{cgroup_is_punktfunk_owned, is_steam_launch, shape_dedicated_command}; use super::{
cgroup_is_punktfunk_owned, connected_connector_under, is_steam_launch,
shape_dedicated_command,
};
#[test]
fn connector_status_scan() {
let base = std::env::temp_dir().join(format!("pf-drm-scan-{}", std::process::id()));
let mk = |name: &str, status: Option<&str>| {
let dir = base.join(name);
std::fs::create_dir_all(&dir).unwrap();
if let Some(s) = status {
std::fs::write(dir.join("status"), s).unwrap();
}
};
// Headless layout: device + render nodes only (no status files) → not connected.
mk("card0", None);
mk("renderD128", None);
assert!(!connected_connector_under(&base));
// Connectors present but nothing plugged in → still not connected.
mk("card0-HDMI-A-1", Some("disconnected\n"));
assert!(!connected_connector_under(&base));
// A live panel → connected.
mk("card0-eDP-1", Some("connected\n"));
assert!(connected_connector_under(&base));
// A missing base dir (no DRM at all) reads as headless.
assert!(!connected_connector_under(&base.join("nope")));
std::fs::remove_dir_all(&base).unwrap();
}
#[test] #[test]
fn steam_launch_detection() { fn steam_launch_detection() {
@@ -92,11 +92,19 @@ fn next_output_name() -> String {
/// The Hyprland virtual-display driver. Stateless — each [`create`](VirtualDisplay::create) adds one /// The Hyprland virtual-display driver. Stateless — each [`create`](VirtualDisplay::create) adds one
/// named headless output and spins up a portal thread owning the cast on it. /// named headless output and spins up a portal thread owning the cast on it.
pub struct HyprlandDisplay; pub struct HyprlandDisplay {
/// Out-of-band cursor request (`set_hw_cursor`, the negotiated cursor channel): portal
/// `CursorMode::Metadata` — shapes/positions ride `SPA_META_Cursor` for the channel + the
/// composite blend. Off (every non-channel session): `Embedded` — the compositor paints the
/// pointer into frames, zero host-side cursor work (the pre-channel default this backend
/// always had). ⚠️ Metadata is UNTESTED on-glass for this backend (Phase B wired it so the
/// channel isn't silently dead here; KWin/Mutter are the validated legs).
hw_cursor: bool,
}
impl HyprlandDisplay { impl HyprlandDisplay {
pub fn new() -> Result<Self> { pub fn new() -> Result<Self> {
Ok(HyprlandDisplay) Ok(HyprlandDisplay { hw_cursor: false })
} }
} }
@@ -141,6 +149,14 @@ impl VirtualDisplay for HyprlandDisplay {
"hyprland" "hyprland"
} }
fn set_hw_cursor(&mut self, on: bool) {
self.hw_cursor = on;
}
fn hw_cursor(&self) -> bool {
self.hw_cursor
}
fn create(&mut self, mode: Mode) -> Result<VirtualOutput> { fn create(&mut self, mode: Mode) -> Result<VirtualOutput> {
// Log the permission-system caveat once per process (silent black frames otherwise). // Log the permission-system caveat once per process (silent black frames otherwise).
preflight_once(); preflight_once();
@@ -167,9 +183,10 @@ impl VirtualDisplay for HyprlandDisplay {
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>(); let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>();
let stop = Arc::new(AtomicBool::new(false)); let stop = Arc::new(AtomicBool::new(false));
let stop_thread = stop.clone(); let stop_thread = stop.clone();
let hw_cursor = self.hw_cursor;
thread::Builder::new() thread::Builder::new()
.name("punktfunk-hypr-vout".into()) .name("punktfunk-hypr-vout".into())
.spawn(move || portal_thread(setup_tx, stop_thread)) .spawn(move || portal_thread(setup_tx, stop_thread, hw_cursor))
.context("spawn hyprland portal thread")?; .context("spawn hyprland portal thread")?;
let (fd, node_id) = match setup_rx.recv_timeout(Duration::from_secs(20)) { let (fd, node_id) = match setup_rx.recv_timeout(Duration::from_secs(20)) {
@@ -199,6 +216,7 @@ impl VirtualDisplay for HyprlandDisplay {
ownership: DisplayOwnership::Owned, ownership: DisplayOwnership::Owned,
reused_gen: None, reused_gen: None,
pool_gen: None, pool_gen: None,
expect_exact_dims: false,
}) })
} }
} }
@@ -490,7 +508,17 @@ fn ensure_xdph_config() -> Result<()> {
/// stopped (the zbus connection is the cast's lifetime). xdph answers source selection via our /// stopped (the zbus connection is the cast's lifetime). xdph answers source selection via our
/// custom picker, no dialog. (Kept separate from wlroots' copy so each wlr-family backend stays /// custom picker, no dialog. (Kept separate from wlroots' copy so each wlr-family backend stays
/// self-owned per D1; unify if they ever diverge no further.) /// self-owned per D1; unify if they ever diverge no further.)
fn portal_thread(setup_tx: Sender<Result<(OwnedFd, u32), String>>, stop: Arc<AtomicBool>) { fn portal_thread(
setup_tx: Sender<Result<(OwnedFd, u32), String>>,
stop: Arc<AtomicBool>,
hw_cursor: bool,
) {
// Portal cursor mode per the session's channel negotiation (see the struct doc).
let cursor_mode = if hw_cursor {
CursorMode::Metadata
} else {
CursorMode::Embedded
};
use ashpd::desktop::screencast::{CursorMode, Screencast, SelectSourcesOptions, SourceType}; use ashpd::desktop::screencast::{CursorMode, Screencast, SelectSourcesOptions, SourceType};
use ashpd::desktop::PersistMode; use ashpd::desktop::PersistMode;
use ashpd::enumflags2::BitFlags; use ashpd::enumflags2::BitFlags;
@@ -523,7 +551,7 @@ fn portal_thread(setup_tx: Sender<Result<(OwnedFd, u32), String>>, stop: Arc<Ato
.select_sources( .select_sources(
&session, &session,
SelectSourcesOptions::default() SelectSourcesOptions::default()
.set_cursor_mode(CursorMode::Embedded) .set_cursor_mode(cursor_mode)
// xdph offers MONITOR; the custom picker selects our output. // xdph offers MONITOR; the custom picker selects our output.
.set_sources(BitFlags::from_flag(SourceType::Monitor)) .set_sources(BitFlags::from_flag(SourceType::Monitor))
.set_multiple(false) .set_multiple(false)
+257 -89
View File
@@ -56,8 +56,13 @@ use zkde::zkde_screencast_stream_unstable_v1::{
}; };
use zkde::zkde_screencast_unstable_v1::ZkdeScreencastUnstableV1 as Screencast; use zkde::zkde_screencast_unstable_v1::ZkdeScreencastUnstableV1 as Screencast;
/// `pointer` attachment mode (the protocol enum): render the cursor into the stream so the /// `pointer` attachment modes (the protocol enum), chosen per session by `set_hw_cursor`
/// remote sees it move with injected input. /// (Phase B — the Windows no-regression gate mirrored): a CURSOR-CHANNEL session gets METADATA
/// (`SPA_META_Cursor` on the stream — shapes forwarded to the client, the composite flip blends
/// host-side; embedded would leave both with nothing, the round-1 mutter trap), every other
/// session gets EMBEDDED — KWin composites the pointer into frames itself, zero host-side
/// cursor work, the pre-channel path Moonlight/legacy clients always had.
const POINTER_METADATA: u32 = 4;
const POINTER_EMBEDDED: u32 = 2; const POINTER_EMBEDDED: u32 = 2;
/// The name we give the created output; KWin exposes it to output-management as `Virtual-<name>`. /// The name we give the created output; KWin exposes it to output-management as `Virtual-<name>`.
@@ -79,14 +84,19 @@ pub struct KwinDisplay {
/// `None` for shared/anonymous) — reported to the registry via [`last_identity_slot`] so it can key /// `None` for shared/anonymous) — reported to the registry via [`last_identity_slot`] so it can key
/// the group arrangement + `/display/state` slot to the same id this backend named the output with. /// the group arrangement + `/display/state` slot to the same id this backend named the output with.
last_slot: Option<u32>, last_slot: Option<u32>,
/// The base output name the last `create` used (`punktfunk` / `punktfunk-<id>`) — so /// The RESOLVED kscreen address of the last `create`'s output — the numeric kscreen output id
/// [`apply_position`](VirtualDisplay::apply_position) can address the KWin output `Virtual-<name>`. /// when [`resolve_kscreen_addr`] found it, else the `Virtual-<name>` fallback — so
/// [`apply_position`](VirtualDisplay::apply_position) addresses OUR output even while a
/// superseded same-name sibling is still alive.
last_name: Option<String>, last_name: Option<String>,
/// The topology-restore action the last `create` prepared (re-enable the outputs an `exclusive` /// The topology-restore action the last `create` prepared (re-enable the outputs an `exclusive`
/// topology disabled), pending pickup by the registry via [`take_topology_restore`] — so the /// topology disabled), pending pickup by the registry via [`take_topology_restore`] — so the
/// physical is re-enabled only when the display GROUP's last member drops (§6.1), not this session's. /// physical is re-enabled only when the display GROUP's last member drops (§6.1), not this session's.
/// A backstop [`Drop`] runs it if the registry never took it (so a physical is never left dark). /// A backstop [`Drop`] runs it if the registry never took it (so a physical is never left dark).
pending_restore: Option<Box<dyn FnOnce() + Send>>, pending_restore: Option<Box<dyn FnOnce() + Send>>,
/// Out-of-band cursor request (`set_hw_cursor`, i.e. the session negotiated the cursor
/// channel): METADATA pointer mode at creation; off = EMBEDDED (see the consts above).
hw_cursor: bool,
} }
impl Drop for KwinDisplay { impl Drop for KwinDisplay {
@@ -123,12 +133,22 @@ impl VirtualDisplay for KwinDisplay {
self.pending_restore.take() self.pending_restore.take()
} }
fn set_hw_cursor(&mut self, on: bool) {
self.hw_cursor = on;
}
fn hw_cursor(&self) -> bool {
self.hw_cursor
}
fn apply_position(&mut self, x: i32, y: i32) { fn apply_position(&mut self, x: i32, y: i32) {
let Some(name) = self.last_name.clone() else { // `last_name` holds the RESOLVED kscreen address (numeric output id, or the
// `Virtual-<name>` fallback) — never re-derive from the name: during a supersede two
// outputs share it and the command would hit the old one (see `create`).
let Some(output) = self.last_name.clone() else {
return; return;
}; };
let output = format!("Virtual-{name}"); // kscreen-doctor position syntax: `output.<name-or-id>.position.<x>,<y>`.
// kscreen-doctor position syntax: `output.<name>.position.<x>,<y>`.
let ok = std::process::Command::new("kscreen-doctor") let ok = std::process::Command::new("kscreen-doctor")
.arg(format!("output.{output}.position.{x},{y}")) .arg(format!("output.{output}.position.{x},{y}"))
.status() .status()
@@ -158,32 +178,100 @@ impl VirtualDisplay for KwinDisplay {
None => VOUT_NAME.to_string(), None => VOUT_NAME.to_string(),
}; };
self.last_name = Some(name.clone()); // for apply_position (registry-driven §6.2 layout) self.last_name = Some(name.clone()); // for apply_position (registry-driven §6.2 layout)
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<u32, String>>();
let stop = Arc::new(AtomicBool::new(false));
let stop_thread = stop.clone();
let (width, height) = (mode.width, mode.height); let (width, height) = (mode.width, mode.height);
let name_thread = name.clone(); let pointer_mode = if self.hw_cursor {
thread::Builder::new() POINTER_METADATA
.name("punktfunk-kwin-vout".into()) } else {
.spawn(move || virtual_output_thread(width, height, name_thread, setup_tx, stop_thread)) POINTER_EMBEDDED
.context("spawn KWin virtual-output thread")?;
let node_id = match setup_rx.recv_timeout(Duration::from_secs(20)) {
Ok(Ok(v)) => v,
Ok(Err(e)) => bail!("KWin virtual output failed: {e}"),
Err(_) => bail!("timed out creating the KWin virtual output"),
}; };
tracing::info!(node_id, width, height, "KWin virtual output ready"); let spawn_vout = |w: u32, h: u32| -> Result<(u32, Arc<AtomicBool>)> {
// KWin creates virtual outputs at a hardcoded 60 Hz and `stream_virtual_output` has no let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<u32, String>>();
// refresh argument, so above 60 Hz we install + select a custom mode (supported on virtual let stop = Arc::new(AtomicBool::new(false));
// outputs since KWin 6.6) before capture connects PipeWire, so the stream negotiates at the let stop_thread = stop.clone();
// higher rate. First cut shells out to kscreen-doctor; the in-process let name_thread = name.clone();
// kde_output_management_v2 client is a follow-up. `set_custom_refresh` reads back and thread::Builder::new()
// returns what KWin *actually* achieved so the encoder paces to the real source rate (a .name("punktfunk-kwin-vout".into())
// rejected custom mode leaves the output at 60 Hz). At ≤60 Hz there's nothing to install — .spawn(move || {
// the source runs 60 Hz and the encoder downsamples — so carry the requested rate through. virtual_output_thread(w, h, name_thread, pointer_mode, setup_tx, stop_thread)
let achieved_hz = if mode.refresh_hz > 60 { })
set_custom_refresh(width, height, mode.refresh_hz, &name) .context("spawn KWin virtual-output thread")?;
match setup_rx.recv_timeout(Duration::from_secs(20)) {
Ok(Ok(v)) => Ok((v, stop)),
Ok(Err(e)) => bail!("KWin virtual output failed: {e}"),
Err(_) => bail!("timed out creating the KWin virtual output"),
}
};
// KWin creates virtual outputs at a hardcoded 60 Hz, `stream_virtual_output` has no
// refresh argument — and its screencast stream builds its PipeWire format offer, INCLUDING
// the `maxFramerate` cap it actively throttles delivery to, ONCE at stream creation
// (screencaststream.cpp `buildFormats`, verified against the live offer with `pw-dump`:
// the offer stays `max=60` after a kscreen mode change, and consumers connecting later
// still negotiate against it). The ONLY path that rebuilds the offer is the stream's own
// resize handling: when the source's texture size changes while recording, KWin re-runs
// `buildFormats` — picking up the output's CURRENT refresh — and renegotiates the live
// stream via `pw_stream_update_params`. So above 60 Hz the output is born at a
// SACRIFICIAL height: installing + selecting the real `WxH@hz` custom mode (supported on
// virtual outputs since KWin 6.6) then changes the SIZE, and the first buffers recorded
// after the consumer connects trigger KWin's resize → a renegotiation to `WxH@hz`. The
// capturer holds frames until that lands (`expect_exact_dims`), so the pipeline never
// builds against the birth mode. First cut shells out to kscreen-doctor; the in-process
// kde_output_management_v2 client is a follow-up. `set_custom_refresh` reads back what
// KWin *actually* gave so the encoder paces to the real source rate. At ≤60 Hz there's
// nothing to install — the output is born at the real size and 60 Hz is the offer anyway.
let want_high = mode.refresh_hz > 60;
let birth_h = if want_high { height + 16 } else { height };
let (mut node_id, mut stop) = spawn_vout(width, birth_h)?;
tracing::info!(
node_id,
width,
height,
birth_h,
embedded_pointer = !self.hw_cursor,
"KWin virtual output ready"
);
// ⚠️ ADDRESS BY NUMERIC KSCREEN ID, NEVER BY NAME: a supersede (mode switch) creates the
// replacement output — SAME per-slot name, deliberately, for KWin's per-name config
// persistence — while the superseded sibling is still alive (create-before-drop). Every
// name-addressed kscreen command then hits the FIRST match = the OLD output: on-glass this
// resized the LIVE session's display out from under it (wrong-res/black), read back the
// OLD output as "custom mode applied", made the OLD output primary, and positioned it —
// while the new output never left its birth mode and the capturer's dims gate starved.
// Resolve OUR output's kscreen id once (match: managed-prefix name AND current mode ==
// the just-created birth size; newest id wins), and use it for every kscreen operation.
let mut addr = resolve_kscreen_addr(&name, width, birth_h);
self.last_name = Some(addr.clone()); // for apply_position (registry-driven §6.2 layout)
let mut expect_exact_dims = false;
let achieved_hz = if want_high {
let (achieved, size_applied) =
set_custom_refresh(width, height, mode.refresh_hz, &addr);
if size_applied {
// Real mode selected: the recording stream will renegotiate to it (see above).
expect_exact_dims = true;
achieved
} else {
// Custom-mode install/select rejected (pre-6.6 KWin / stale kscreen-doctor): the
// output is STUCK at the sacrificial birth size — unusable. Recreate plain at the
// real size (the pre-sacrifice behavior: correct size, KWin's native 60 Hz).
tracing::warn!(
"KWin rejected the custom mode — recreating the virtual output at the real \
size (60 Hz ceiling on this KWin)"
);
stop.store(true, Ordering::Relaxed);
// Let KWin retire the doomed output before re-using its name.
std::thread::sleep(Duration::from_millis(300));
let (nid, st) = spawn_vout(width, height)?;
node_id = nid;
stop = st;
addr = resolve_kscreen_addr(&name, width, height);
self.last_name = Some(addr.clone());
tracing::info!(
node_id,
width,
height,
"KWin virtual output ready (fallback)"
);
60
}
} else { } else {
mode.refresh_hz mode.refresh_hz
}; };
@@ -194,9 +282,9 @@ impl VirtualDisplay for KwinDisplay {
// bootstrap output. Read from the policy (replacing the PUNKTFUNK_KWIN_VIRTUAL_PRIMARY boolean). // bootstrap output. Read from the policy (replacing the PUNKTFUNK_KWIN_VIRTUAL_PRIMARY boolean).
use crate::policy::Topology; use crate::policy::Topology;
let disabled = match crate::effective_topology() { let disabled = match crate::effective_topology() {
Topology::Exclusive => apply_virtual_primary(&name), Topology::Exclusive => apply_virtual_primary(&addr),
Topology::Primary => { Topology::Primary => {
apply_virtual_primary_only(&name); apply_virtual_primary_only(&addr);
Vec::new() // nothing disabled → nothing to restore Vec::new() // nothing disabled → nothing to restore
} }
Topology::Extend | Topology::Auto => Vec::new(), Topology::Extend | Topology::Auto => Vec::new(),
@@ -212,11 +300,13 @@ impl VirtualDisplay for KwinDisplay {
}); });
// Layout position (§6.2) is applied by the registry via `apply_position` right after create // Layout position (§6.2) is applied by the registry via `apply_position` right after create
// (it owns the display group, so it computes auto-row / manual placement over the whole group). // (it owns the display group, so it computes auto-row / manual placement over the whole group).
Ok(VirtualOutput::owned( let mut out = VirtualOutput::owned(
node_id, node_id,
Some((mode.width, mode.height, achieved_hz)), Some((mode.width, mode.height, achieved_hz)),
Box::new(StopGuard { stop }), Box::new(StopGuard { stop }),
)) );
out.expect_exact_dims = expect_exact_dims;
Ok(out)
} }
} }
@@ -254,14 +344,88 @@ fn reenable_outputs(outputs: &[(String, String)]) {
tracing::info!(reenabled = ?outputs, "KWin: restored the physical/bootstrap outputs at their captured modes (group empty)"); tracing::info!(reenabled = ?outputs, "KWin: restored the physical/bootstrap outputs at their captured modes (group empty)");
} }
/// Best-effort: raise the just-created virtual output's refresh above KWin's default 60 Hz by /// Resolve the kscreen address of the virtual output the host JUST created: the managed-prefix
/// installing + selecting a custom mode via `kscreen-doctor` (the output is `Virtual-<VOUT_NAME>`, /// name alone is ambiguous during a supersede (the replacement deliberately reuses the per-slot
/// refresh given in mHz), then **read back the active mode** and return the refresh KWin actually /// name while the superseded sibling is still alive), so match on the birth mode's size too —
/// gave us. The apply command can report success yet leave the output at 60 Hz (mode rejected), /// only the just-created output sits at the sacrificial `(w, h)` — and prefer the HIGHEST output
/// and a silent rate mismatch surfaces downstream as judder / duplicated frames — so the caller /// id (the newest) if several match. Returns the numeric id as a string (kscreen-doctor accepts
/// paces the encoder to the *achieved* rate, not the requested one. /// `output.<id>.…`), falling back to the ambiguous `Virtual-<name>` if the output hasn't reached
fn set_custom_refresh(width: u32, height: u32, hz: u32, name: &str) -> u32 { /// kscreen's model yet after a few tries (single-output sessions are unambiguous anyway).
let output = format!("Virtual-{name}"); fn resolve_kscreen_addr(name: &str, w: u32, h: u32) -> String {
let fallback = format!("Virtual-{name}");
for attempt in 0..3 {
if attempt > 0 {
std::thread::sleep(Duration::from_millis(150));
}
let Some(doc) = kscreen_json() else { continue };
let Some(outputs) = doc.get("outputs").and_then(|o| o.as_array()) else {
continue;
};
let best = outputs
.iter()
.filter(|o| {
o.get("name")
.and_then(|n| n.as_str())
.is_some_and(|n| n.starts_with(&fallback))
&& output_active_size(o) == Some((w, h))
})
.filter_map(|o| o.get("id").and_then(|i| i.as_u64()))
.max();
if let Some(id) = best {
tracing::info!(id, name, w, h, "KWin: resolved the new output's kscreen id");
return id.to_string();
}
}
tracing::warn!(
name,
w,
h,
"KWin: could not resolve the new output's kscreen id — falling back to name addressing \
(ambiguous during a mode-switch supersede)"
);
fallback
}
/// `kscreen-doctor -j` parsed, `None` on any failure.
fn kscreen_json() -> Option<serde_json::Value> {
let out = std::process::Command::new("kscreen-doctor")
.arg("-j")
.output()
.ok()?;
serde_json::from_slice(&out.stdout).ok()
}
/// The `(width, height)` of an output's CURRENT mode from its `kscreen-doctor -j` entry.
fn output_active_size(o: &serde_json::Value) -> Option<(u32, u32)> {
let as_id = |v: &serde_json::Value| -> Option<String> {
v.as_str()
.map(|s| s.to_string())
.or_else(|| v.as_u64().map(|n| n.to_string()))
};
let current = o.get("currentModeId").and_then(as_id)?;
let mode = o
.get("modes")?
.as_array()?
.iter()
.find(|m| m.get("id").and_then(as_id).as_deref() == Some(current.as_str()))?;
let size = mode.get("size")?;
Some((
size.get("width").and_then(|v| v.as_u64())? as u32,
size.get("height").and_then(|v| v.as_u64())? as u32,
))
}
/// Best-effort: install + select the `width`x`height`@`hz` custom mode on the just-created
/// virtual output via `kscreen-doctor` (`output` is the RESOLVED kscreen address — numeric id or
/// name, see [`resolve_kscreen_addr`] — refresh given in mHz), then **read back the active mode**
/// and return `(achieved_hz, size_applied)`. The apply command can report success yet leave the
/// output on its old mode (rejected), and a silent rate mismatch surfaces downstream as judder /
/// duplicated frames — so the caller paces the encoder to the *achieved* rate, not the requested
/// one. `size_applied` tells the sacrificial-birth caller (see `create`) whether the SIZE half of
/// the mode actually landed — that, not the refresh, is what triggers KWin's stream
/// renegotiation.
fn set_custom_refresh(width: u32, height: u32, hz: u32, output: &str) -> (u32, bool) {
let output = output.to_string();
let mhz = hz.saturating_mul(1000); let mhz = hz.saturating_mul(1000);
let run = |arg: String| { let run = |arg: String| {
std::process::Command::new("kscreen-doctor") std::process::Command::new("kscreen-doctor")
@@ -275,26 +439,29 @@ fn set_custom_refresh(width: u32, height: u32, hz: u32, name: &str) -> u32 {
"output.{output}.addCustomMode.{width}.{height}.{mhz}.full" "output.{output}.addCustomMode.{width}.{height}.{mhz}.full"
)); ));
let applied = run(format!("output.{output}.mode.{width}x{height}@{hz}")); let applied = run(format!("output.{output}.mode.{width}x{height}@{hz}"));
match read_active_refresh(&output) { match read_active_mode(&output) {
Some(achieved) if achieved >= hz => { Some((w, h, achieved)) => {
tracing::info!( let size_applied = (w, h) == (width, height);
output, if achieved >= hz && size_applied {
requested = hz, tracing::info!(
achieved, output,
"KWin virtual output: custom refresh applied" requested = hz,
); achieved,
achieved "KWin virtual output: custom refresh applied"
} );
Some(achieved) => { } else {
tracing::warn!( tracing::warn!(
output, output,
requested = hz, requested = hz,
achieved, achieved,
applied, active_w = w,
"KWin virtual output refresh below requested — pacing the encoder to the achieved \ active_h = h,
rate (custom-mode install rejected? is kscreen-doctor up to date?)" applied,
); "KWin virtual output mode below requested — pacing the encoder to the \
achieved.max(1) achieved rate (custom-mode install rejected? is kscreen-doctor up to date?)"
);
}
(achieved.max(1), size_applied)
} }
None => { None => {
tracing::warn!( tracing::warn!(
@@ -304,38 +471,37 @@ fn set_custom_refresh(width: u32, height: u32, hz: u32, name: &str) -> u32 {
"could not read back KWin virtual output refresh — assuming 60 Hz (is \ "could not read back KWin virtual output refresh — assuming 60 Hz (is \
kscreen-doctor installed?)" kscreen-doctor installed?)"
); );
60 (60, false)
} }
} }
} }
/// Read the active refresh (Hz, rounded) of `output` from `kscreen-doctor -j`. `None` if the /// Read the active mode (`(width, height, refresh_hz)`, Hz rounded) of `output` — a RESOLVED
/// tool, the output, or its current mode can't be found. Mode/output ids come through as either /// kscreen address (numeric id or name, see [`resolve_kscreen_addr`]) — from `kscreen-doctor -j`.
/// JSON strings or numbers depending on the KWin version, so both are accepted. /// `None` if the tool, the output, or its current mode can't be found. Mode/output ids come
fn read_active_refresh(output: &str) -> Option<u32> { /// through as either JSON strings or numbers depending on the KWin version, so both are accepted.
let out = std::process::Command::new("kscreen-doctor") fn read_active_mode(output: &str) -> Option<(u32, u32, u32)> {
.arg("-j") let doc = kscreen_json()?;
.output()
.ok()?;
let doc: serde_json::Value = serde_json::from_slice(&out.stdout).ok()?;
let as_id = |v: &serde_json::Value| -> Option<String> { let as_id = |v: &serde_json::Value| -> Option<String> {
v.as_str() v.as_str()
.map(|s| s.to_string()) .map(|s| s.to_string())
.or_else(|| v.as_u64().map(|n| n.to_string())) .or_else(|| v.as_u64().map(|n| n.to_string()))
}; };
let o = doc let o = doc.get("outputs")?.as_array()?.iter().find(|o| {
.get("outputs")? o.get("name").and_then(|n| n.as_str()) == Some(output)
.as_array()? || o.get("id").and_then(as_id).as_deref() == Some(output)
.iter() })?;
.find(|o| o.get("name").and_then(|n| n.as_str()) == Some(output))?;
let current = o.get("currentModeId").and_then(as_id)?; let current = o.get("currentModeId").and_then(as_id)?;
let mode = o let mode = o
.get("modes")? .get("modes")?
.as_array()? .as_array()?
.iter() .iter()
.find(|m| m.get("id").and_then(as_id).as_deref() == Some(current.as_str()))?; .find(|m| m.get("id").and_then(as_id).as_deref() == Some(current.as_str()))?;
let size = mode.get("size")?;
let w = size.get("width").and_then(|v| v.as_u64())? as u32;
let h = size.get("height").and_then(|v| v.as_u64())? as u32;
let hz = mode.get("refreshRate").and_then(|r| r.as_f64())?; let hz = mode.get("refreshRate").and_then(|r| r.as_f64())?;
Some(hz.round() as u32) Some((w, h, hz.round() as u32))
} }
/// The prefix EVERY managed KWin output shares — Stage 3 names them `punktfunk` / `punktfunk-<id>`, /// The prefix EVERY managed KWin output shares — Stage 3 names them `punktfunk` / `punktfunk-<id>`,
@@ -374,7 +540,7 @@ fn output_current_mode_spec(o: &serde_json::Value) -> Option<String> {
/// bare re-enable drops a 120 Hz panel to KWin's default ~60 Hz). /// bare re-enable drops a 120 Hz panel to KWin's default ~60 Hz).
/// **Group-aware (§6.1):** excludes the WHOLE managed family (the [`MANAGED_PREFIX`]), not just this /// **Group-aware (§6.1):** excludes the WHOLE managed family (the [`MANAGED_PREFIX`]), not just this
/// session's own output — so a 2nd `exclusive` session (with a distinct per-slot name) never disables /// session's own output — so a 2nd `exclusive` session (with a distinct per-slot name) never disables
/// the 1st session's live output. Parsed from `kscreen-doctor -j` (same source as [`read_active_refresh`]). /// the 1st session's live output. Parsed from `kscreen-doctor -j` (same source as [`read_active_mode`]).
fn other_enabled_outputs() -> Vec<(String, String)> { fn other_enabled_outputs() -> Vec<(String, String)> {
let out = match std::process::Command::new("kscreen-doctor") let out = match std::process::Command::new("kscreen-doctor")
.arg("-j") .arg("-j")
@@ -437,12 +603,12 @@ fn a_managed_output_is_primary() -> bool {
.unwrap_or(false) .unwrap_or(false)
} }
/// Set `Virtual-punktfunk` primary and disable the bootstrap output(s) so the managed group becomes /// Set our output primary and disable the bootstrap output(s) so the managed group becomes
/// the sole desktop (KWin re-homes plasmashell + windows onto it). Returns the disabled outputs for /// the sole desktop (KWin re-homes plasmashell + windows onto it). `ours` is the RESOLVED kscreen
/// address (numeric id or name, see [`resolve_kscreen_addr`]). Returns the disabled outputs for
/// the keepalive to re-enable on teardown. Best-effort: on failure, streaming continues (just possibly /// the keepalive to re-enable on teardown. Best-effort: on failure, streaming continues (just possibly
/// showing only the wallpaper) rather than failing the session. /// showing only the wallpaper) rather than failing the session.
fn apply_virtual_primary(name: &str) -> Vec<(String, String)> { fn apply_virtual_primary(ours: &str) -> Vec<(String, String)> {
let ours = format!("Virtual-{name}");
let kscreen = |args: &[String]| { let kscreen = |args: &[String]| {
std::process::Command::new("kscreen-doctor") std::process::Command::new("kscreen-doctor")
.args(args) .args(args)
@@ -480,8 +646,7 @@ fn apply_virtual_primary(name: &str) -> Vec<(String, String)> {
/// **Primary** (Stage 2): make the streamed output the primary but KEEP the other outputs enabled /// **Primary** (Stage 2): make the streamed output the primary but KEEP the other outputs enabled
/// (don't disable the bootstrap/physical) — so the shell re-homes onto the streamed surface while a /// (don't disable the bootstrap/physical) — so the shell re-homes onto the streamed surface while a
/// physical screen stays usable. Nothing to restore on teardown (we disabled nothing). /// physical screen stays usable. Nothing to restore on teardown (we disabled nothing).
fn apply_virtual_primary_only(name: &str) { fn apply_virtual_primary_only(ours: &str) {
let ours = format!("Virtual-{name}");
let ok = std::process::Command::new("kscreen-doctor") let ok = std::process::Command::new("kscreen-doctor")
.arg(format!("output.{ours}.primary")) .arg(format!("output.{ours}.primary"))
.status() .status()
@@ -578,10 +743,11 @@ fn virtual_output_thread(
width: u32, width: u32,
height: u32, height: u32,
name: String, name: String,
pointer_mode: u32,
setup_tx: Sender<Result<u32, String>>, setup_tx: Sender<Result<u32, String>>,
stop: Arc<AtomicBool>, stop: Arc<AtomicBool>,
) { ) {
if let Err(e) = run(width, height, &name, &setup_tx, &stop) { if let Err(e) = run(width, height, &name, pointer_mode, &setup_tx, &stop) {
// If we never delivered a node id, report the failure to the waiting opener. // If we never delivered a node id, report the failure to the waiting opener.
let _ = setup_tx.send(Err(format!("{e:#}"))); let _ = setup_tx.send(Err(format!("{e:#}")));
} }
@@ -622,6 +788,7 @@ fn run(
width: u32, width: u32,
height: u32, height: u32,
name: &str, name: &str,
pointer_mode: u32,
setup_tx: &Sender<Result<u32, String>>, setup_tx: &Sender<Result<u32, String>>,
stop: &AtomicBool, stop: &AtomicBool,
) -> Result<()> { ) -> Result<()> {
@@ -642,13 +809,14 @@ fn run(
) )
})?; })?;
// Create the virtual output sized to the client, cursor composited into the stream. // Create the virtual output sized to the client; the pointer rides as stream metadata
// (cursor-channel session) or KWin embeds it into frames (everyone else — see the consts).
let stream = screencast.stream_virtual_output( let stream = screencast.stream_virtual_output(
name.to_string(), name.to_string(),
width as i32, width as i32,
height as i32, height as i32,
1.0, // scale (logical == physical) 1.0, // scale (logical == physical)
POINTER_EMBEDDED, pointer_mode,
&qh, &qh,
(), (),
); );
@@ -46,7 +46,17 @@ const BUS_DC: &str = "org.gnome.Mutter.DisplayConfig";
/// e.g. when our virtual output is torn down — so we never persist a layout to monitors.xml). /// e.g. when our virtual output is torn down — so we never persist a layout to monitors.xml).
const APPLY_TEMPORARY: u32 = 1; const APPLY_TEMPORARY: u32 = 1;
/// Mutter cursor mode: render the cursor into the stream (matches the KWin/gamescope backends). /// Mutter cursor mode: ship the pointer as `SPA_META_Cursor` metadata instead of burning it into
/// the frames. The capturer always negotiates the meta (pf-capture `meta_param`) and the encoder
/// blend composites it for sessions where the client does not draw the cursor itself — while a
/// cursor-forwarding session strips the overlay and sends shape/state over the cursor channel.
/// Embedded mode would leave BOTH paths blind: no metadata means nothing to forward AND nothing
/// to blend, and Mutter's own embedded painting is what the pre-channel path relied on.
const CURSOR_METADATA: u32 = 2;
/// `cursor-mode` embedded (mutter enum 1): Mutter composites the pointer into frames itself —
/// zero host-side cursor work, the pre-channel path. Chosen for every session WITHOUT the
/// negotiated cursor channel (`set_hw_cursor` off — Phase B, the Windows no-regression gate
/// mirrored); metadata stays the cursor-channel sessions' mode (shapes forwarded / host blend).
const CURSOR_EMBEDDED: u32 = 1; const CURSOR_EMBEDDED: u32 = 1;
/// Serializes, process-wide, every Mutter operation that adds/removes a virtual monitor or applies /// Serializes, process-wide, every Mutter operation that adds/removes a virtual monitor or applies
@@ -69,6 +79,9 @@ pub struct MutterDisplay {
/// sole-monitor config (which would disable the first session's virtual). Defaults true (a lone /// sole-monitor config (which would disable the first session's virtual). Defaults true (a lone
/// session establishes topology as before). /// session establishes topology as before).
first_in_group: bool, first_in_group: bool,
/// Out-of-band cursor request (`set_hw_cursor`): metadata cursor-mode at creation; off =
/// embedded (see [`CURSOR_EMBEDDED`]).
hw_cursor: bool,
/// The connecting client's cert fingerprint (set before [`create`](VirtualDisplay::create)) — /// The connecting client's cert fingerprint (set before [`create`](VirtualDisplay::create)) —
/// keys the per-client persisted **scale** (GNOME can't persist it itself: Mutter mints a fresh /// keys the per-client persisted **scale** (GNOME can't persist it itself: Mutter mints a fresh
/// EDID serial per `RecordVirtual` monitor, so `monitors.xml` never rematches; see /// EDID serial per `RecordVirtual` monitor, so `monitors.xml` never rematches; see
@@ -84,6 +97,7 @@ impl MutterDisplay {
pub fn new() -> Result<Self> { pub fn new() -> Result<Self> {
Ok(MutterDisplay { Ok(MutterDisplay {
first_in_group: true, first_in_group: true,
hw_cursor: false,
client_fp: None, client_fp: None,
last_slot: None, last_slot: None,
}) })
@@ -112,6 +126,14 @@ impl VirtualDisplay for MutterDisplay {
self.client_fp = fingerprint; self.client_fp = fingerprint;
} }
fn set_hw_cursor(&mut self, on: bool) {
self.hw_cursor = on;
}
fn hw_cursor(&self) -> bool {
self.hw_cursor
}
fn last_identity_slot(&self) -> Option<u32> { fn last_identity_slot(&self) -> Option<u32> {
self.last_slot self.last_slot
} }
@@ -141,6 +163,7 @@ impl VirtualDisplay for MutterDisplay {
let stop = Arc::new(AtomicBool::new(false)); let stop = Arc::new(AtomicBool::new(false));
let stop_thread = stop.clone(); let stop_thread = stop.clone();
let first_in_group = self.first_in_group; let first_in_group = self.first_in_group;
let hw_cursor = self.hw_cursor;
thread::Builder::new() thread::Builder::new()
.name("punktfunk-mutter-vout".into()) .name("punktfunk-mutter-vout".into())
.spawn(move || { .spawn(move || {
@@ -149,6 +172,7 @@ impl VirtualDisplay for MutterDisplay {
stop_thread, stop_thread,
mode, mode,
first_in_group, first_in_group,
hw_cursor,
scale_key, scale_key,
remembered_scale, remembered_scale,
) )
@@ -203,6 +227,7 @@ fn session_thread(
stop: Arc<AtomicBool>, stop: Arc<AtomicBool>,
mode: Mode, mode: Mode,
first_in_group: bool, first_in_group: bool,
hw_cursor: bool,
scale_key: String, scale_key: String,
remembered_scale: Option<f64>, remembered_scale: Option<f64>,
) { ) {
@@ -262,7 +287,7 @@ fn session_thread(
} }
}; };
let session = match connect(mode, remembered_scale).await { let session = match connect(mode, hw_cursor, remembered_scale).await {
Ok(s) => s, Ok(s) => s,
Err(e) => { Err(e) => {
let _ = setup_tx.send(Err(format!("{e:#}"))); let _ = setup_tx.send(Err(format!("{e:#}")));
@@ -363,7 +388,11 @@ struct MutterSession {
/// desktop scale, passed as the virtual mode's `preferred-scale` so Mutter creates the monitor /// desktop scale, passed as the virtual mode's `preferred-scale` so Mutter creates the monitor
/// already scaled (Mutter ≥ 48; older Mutter ignores unknown mode keys) — this covers the /// already scaled (Mutter ≥ 48; older Mutter ignores unknown mode keys) — this covers the
/// `extend` topology, where we never issue our own ApplyMonitorsConfig. /// `extend` topology, where we never issue our own ApplyMonitorsConfig.
async fn connect(mode: Mode, preferred_scale: Option<f64>) -> Result<MutterSession> { async fn connect(
mode: Mode,
hw_cursor: bool,
preferred_scale: Option<f64>,
) -> Result<MutterSession> {
let conn = zbus::Connection::session() let conn = zbus::Connection::session()
.await .await
.context("connect session D-Bus")?; .context("connect session D-Bus")?;
@@ -424,7 +453,14 @@ async fn connect(mode: Mode, preferred_scale: Option<f64>) -> Result<MutterSessi
// once gated behind PUNKTFUNK_MUTTER_VIRTUAL_REFRESH; the stop-screencast-before-any-monitor- // once gated behind PUNKTFUNK_MUTTER_VIRTUAL_REFRESH; the stop-screencast-before-any-monitor-
// reconfig teardown below fixed the crash, so pinning the client's refresh is now the default.) // reconfig teardown below fixed the crash, so pinning the client's refresh is now the default.)
let mut rec: HashMap<&str, Value> = HashMap::new(); let mut rec: HashMap<&str, Value> = HashMap::new();
rec.insert("cursor-mode", Value::from(CURSOR_EMBEDDED)); rec.insert(
"cursor-mode",
Value::from(if hw_cursor {
CURSOR_METADATA
} else {
CURSOR_EMBEDDED
}),
);
if mode.refresh_hz > 60 || preferred_scale.is_some() { if mode.refresh_hz > 60 || preferred_scale.is_some() {
let mut vmode: HashMap<&str, Value> = HashMap::new(); let mut vmode: HashMap<&str, Value> = HashMap::new();
vmode.insert("size", Value::from((mode.width, mode.height))); vmode.insert("size", Value::from((mode.width, mode.height)));
@@ -54,11 +54,19 @@ chooser_cmd=cat {} 2>/dev/null || echo 'Monitor: HEADLESS-1'\n",
/// The wlroots/Sway virtual-display driver. Stateless — each [`create`](VirtualDisplay::create) /// The wlroots/Sway virtual-display driver. Stateless — each [`create`](VirtualDisplay::create)
/// adds one headless output and spins up a portal thread owning the cast on it. /// adds one headless output and spins up a portal thread owning the cast on it.
pub struct WlrootsDisplay; pub struct WlrootsDisplay {
/// Out-of-band cursor request (`set_hw_cursor`, the negotiated cursor channel): portal
/// `CursorMode::Metadata` — shapes/positions ride `SPA_META_Cursor` for the channel + the
/// composite blend. Off (every non-channel session): `Embedded` — the compositor paints the
/// pointer into frames, zero host-side cursor work (the pre-channel default this backend
/// always had). ⚠️ Metadata is UNTESTED on-glass for this backend (Phase B wired it so the
/// channel isn't silently dead here; KWin/Mutter are the validated legs).
hw_cursor: bool,
}
impl WlrootsDisplay { impl WlrootsDisplay {
pub fn new() -> Result<Self> { pub fn new() -> Result<Self> {
Ok(WlrootsDisplay) Ok(WlrootsDisplay { hw_cursor: false })
} }
} }
@@ -73,6 +81,14 @@ impl VirtualDisplay for WlrootsDisplay {
"wlroots" "wlroots"
} }
fn set_hw_cursor(&mut self, on: bool) {
self.hw_cursor = on;
}
fn hw_cursor(&self) -> bool {
self.hw_cursor
}
fn create(&mut self, mode: Mode) -> Result<VirtualOutput> { fn create(&mut self, mode: Mode) -> Result<VirtualOutput> {
let before = output_names() let before = output_names()
.context("swaymsg get_outputs (is the host inside the sway session env — SWAYSOCK?)")?; .context("swaymsg get_outputs (is the host inside the sway session env — SWAYSOCK?)")?;
@@ -104,9 +120,10 @@ impl VirtualDisplay for WlrootsDisplay {
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>(); let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>();
let stop = Arc::new(AtomicBool::new(false)); let stop = Arc::new(AtomicBool::new(false));
let stop_thread = stop.clone(); let stop_thread = stop.clone();
let hw_cursor = self.hw_cursor;
thread::Builder::new() thread::Builder::new()
.name("punktfunk-wlr-vout".into()) .name("punktfunk-wlr-vout".into())
.spawn(move || portal_thread(setup_tx, stop_thread)) .spawn(move || portal_thread(setup_tx, stop_thread, hw_cursor))
.context("spawn wlroots portal thread")?; .context("spawn wlroots portal thread")?;
let (fd, node_id) = match setup_rx.recv_timeout(Duration::from_secs(20)) { let (fd, node_id) = match setup_rx.recv_timeout(Duration::from_secs(20)) {
@@ -136,6 +153,7 @@ impl VirtualDisplay for WlrootsDisplay {
ownership: DisplayOwnership::Owned, ownership: DisplayOwnership::Owned,
reused_gen: None, reused_gen: None,
pool_gen: None, pool_gen: None,
expect_exact_dims: false,
}) })
} }
} }
@@ -254,7 +272,17 @@ fn ensure_xdpw_config() -> Result<()> {
/// The ScreenCast portal handshake (same shape as the capture module's portal thread, but it /// The ScreenCast portal handshake (same shape as the capture module's portal thread, but it
/// reports the fd + node id and parks until stopped — the zbus connection is the cast's /// reports the fd + node id and parks until stopped — the zbus connection is the cast's
/// lifetime). xdpw answers the source selection via the chooser, no dialog. /// lifetime). xdpw answers the source selection via the chooser, no dialog.
fn portal_thread(setup_tx: Sender<Result<(OwnedFd, u32), String>>, stop: Arc<AtomicBool>) { fn portal_thread(
setup_tx: Sender<Result<(OwnedFd, u32), String>>,
stop: Arc<AtomicBool>,
hw_cursor: bool,
) {
// Portal cursor mode per the session's channel negotiation (see the struct doc).
let cursor_mode = if hw_cursor {
CursorMode::Metadata
} else {
CursorMode::Embedded
};
use ashpd::desktop::screencast::{CursorMode, Screencast, SelectSourcesOptions, SourceType}; use ashpd::desktop::screencast::{CursorMode, Screencast, SelectSourcesOptions, SourceType};
use ashpd::desktop::PersistMode; use ashpd::desktop::PersistMode;
use ashpd::enumflags2::BitFlags; use ashpd::enumflags2::BitFlags;
@@ -287,7 +315,7 @@ fn portal_thread(setup_tx: Sender<Result<(OwnedFd, u32), String>>, stop: Arc<Ato
.select_sources( .select_sources(
&session, &session,
SelectSourcesOptions::default() SelectSourcesOptions::default()
.set_cursor_mode(CursorMode::Embedded) .set_cursor_mode(cursor_mode)
// xdpw offers MONITOR only; the chooser picks our output. // xdpw offers MONITOR only; the chooser picks our output.
.set_sources(BitFlags::from_flag(SourceType::Monitor)) .set_sources(BitFlags::from_flag(SourceType::Monitor))
.set_multiple(false) .set_multiple(false)
+39 -5
View File
@@ -84,20 +84,28 @@ fn topology_str() -> String {
/// `quit` is the session's deliberate-quit flag: when the session ends with it set (the client closed /// `quit` is the session's deliberate-quit flag: when the session ends with it set (the client closed
/// with the quit application code — a user "stop", not a network drop), the display is torn down /// with the quit application code — a user "stop", not a network drop), the display is torn down
/// **immediately**, skipping the keep-alive linger. A bare disconnect leaves it `false` → normal linger. /// **immediately**, skipping the keep-alive linger. A bare disconnect leaves it `false` → normal linger.
///
/// `supersedes`: the pool gen of a display this acquire REPLACES (a mid-stream mode switch creates
/// the new display before retiring the old — create-before-drop). The replacement inherits group
/// topology ownership: without this, the dying predecessor counts as a live sibling and the new
/// display "extends" behind it, losing a Primary/Exclusive topology on every resize. `None`
/// everywhere else.
pub fn acquire( pub fn acquire(
vd: &mut Box<dyn super::VirtualDisplay>, vd: &mut Box<dyn super::VirtualDisplay>,
mode: super::Mode, mode: super::Mode,
quit: std::sync::Arc<std::sync::atomic::AtomicBool>, quit: std::sync::Arc<std::sync::atomic::AtomicBool>,
supersedes: Option<u64>,
) -> Result<super::VirtualOutput> { ) -> Result<super::VirtualOutput> {
let backend = vd.name(); let backend = vd.name();
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
let out = linux::acquire(vd, mode, quit); let out = linux::acquire(vd, mode, quit, supersedes);
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
let out = { let out = {
// Windows leases in the manager (its own linger); its deliberate-quit skip is wired through // Windows leases in the manager (its own linger); its deliberate-quit skip is wired through
// `VirtualDisplay::set_quit_flag` on the backend instance (set by the session before any // `VirtualDisplay::set_quit_flag` on the backend instance (set by the session before any
// `create`, so the retry-hold lease gets it too) — not through this parameter. // `create`, so the retry-hold lease gets it too) — not through this parameter. The
let _ = quit; // supersede handoff is Linux-pool-only too (the manager resizes in place).
let _ = (quit, supersedes);
vd.create(mode) vd.create(mode)
}; };
if out.is_ok() { if out.is_ok() {
@@ -260,6 +268,11 @@ mod linux {
/// Generation stamp: a [`DisplayLease`] only releases if its gen still matches (a stale lease /// Generation stamp: a [`DisplayLease`] only releases if its gen still matches (a stale lease
/// — its entry was reused + re-stamped — is a no-op). /// — its entry was reused + re-stamped — is a no-op).
gen: u64, gen: u64,
/// The out-of-band-cursor mode this display was CREATED with (Phase B): metadata-pointer
/// (cursor-channel session) vs compositor-embedded. Reuse requires an exact match — a kept
/// embedded display has no cursor metadata for a channel session to forward, and a kept
/// metadata display would leave a channel-less session with no pointer in its frames.
hw_cursor: bool,
} }
/// A per-group topology-restore action (see [`Entry::topology_restore`]). /// A per-group topology-restore action (see [`Entry::topology_restore`]).
@@ -421,6 +434,7 @@ mod linux {
vd: &mut Box<dyn VirtualDisplay>, vd: &mut Box<dyn VirtualDisplay>,
mode: Mode, mode: Mode,
quit: Arc<AtomicBool>, quit: Arc<AtomicBool>,
supersedes: Option<u64>,
) -> Result<VirtualOutput> { ) -> Result<VirtualOutput> {
ensure_timer(); ensure_timer();
let backend = vd.name(); let backend = vd.name();
@@ -464,6 +478,7 @@ mod linux {
) && e.backend == backend ) && e.backend == backend
&& e.mode == mode && e.mode == mode
&& e.launch == launch && e.launch == launch
&& e.hw_cursor == vd.hw_cursor()
&& epoch_matches(e.backend, e.epoch, cur_epoch) && epoch_matches(e.backend, e.epoch, cur_epoch)
}) })
.map(|e| (e.gen, e.node_id)) .map(|e| (e.gen, e.node_id))
@@ -535,9 +550,19 @@ mod linux {
// §6.1) — so a topology-establishing backend (Mutter exclusive) extends into an already-exclusive // §6.1) — so a topology-establishing backend (Mutter exclusive) extends into an already-exclusive
// desktop rather than re-clobbering the first session's virtual. Best-effort (a concurrent create // desktop rather than re-clobbering the first session's virtual. Best-effort (a concurrent create
// is a narrow race); single-session is always `first == true` → today's behavior. // is a narrow race); single-session is always `first == true` → today's behavior.
// Siblings that don't demote the newcomer:
// * the display this acquire SUPERSEDES (mode switch, create-before-drop) — still Active
// here because the old lease drops only after the new pipeline is up, but it's leaving,
// and deferring to it loses the group's Primary/Exclusive topology on every resize;
// * kept (Lingering/Pinned) entries — no session owns them, so there is no live desktop to
// clobber; a new session next to an unclaimed leftover should still establish topology.
let first_in_group = { let first_in_group = {
let es = r.entries.lock().unwrap(); let es = r.entries.lock().unwrap();
!es.iter().any(|e| e.backend == backend) !es.iter().any(|e| {
e.backend == backend
&& Some(e.gen) != supersedes
&& matches!(e.life, lifecycle::State::Active { .. })
})
}; };
vd.set_first_in_group(first_in_group); vd.set_first_in_group(first_in_group);
@@ -565,6 +590,11 @@ mod linux {
let node_id = real.node_id; let node_id = real.node_id;
let preferred_mode = real.preferred_mode; let preferred_mode = real.preferred_mode;
// Fresh creates only: the backend may have birthed the output at a sacrificial mode whose
// stream must renegotiate before frames count (KWin >60 Hz — see backend.rs). A REUSED kept
// display already renegotiated in its prior session (the producer's rebuilt offer persists
// across consumer reconnects), so the reuse path above correctly leaves the flag off.
let expect_exact_dims = real.expect_exact_dims;
// The backend's topology-restore action (KWin `exclusive` → re-enable the disabled physicals), // The backend's topology-restore action (KWin `exclusive` → re-enable the disabled physicals),
// lifted into the group so it runs once when the group's last member drops (§6.1), not at this // lifted into the group so it runs once when the group's last member drops (§6.1), not at this
// session's teardown. `None` for non-exclusive / non-first / backends whose topology auto-reverts. // session's teardown. `None` for non-exclusive / non-first / backends whose topology auto-reverts.
@@ -584,6 +614,7 @@ mod linux {
launch: launch.clone(), launch: launch.clone(),
epoch: cur_epoch, epoch: cur_epoch,
gen, gen,
hw_cursor: vd.hw_cursor(),
}; };
// Compute this new display's position in its group (design §6.2) BEFORE pushing, then push // Compute this new display's position in its group (design §6.2) BEFORE pushing, then push
@@ -628,7 +659,9 @@ mod linux {
if (position.x, position.y) != (0, 0) { if (position.x, position.y) != (0, 0) {
vd.apply_position(position.x, position.y); vd.apply_position(position.x, position.y);
} }
Ok(output_for(node_id, preferred_mode, gen, quit, false)) let mut out = output_for(node_id, preferred_mode, gen, quit, false);
out.expect_exact_dims = expect_exact_dims;
Ok(out)
} }
/// The linger a releasing session actually gets. A deliberate quit (`force_immediate` — the /// The linger a releasing session actually gets. A deliberate quit (`force_immediate` — the
@@ -1003,6 +1036,7 @@ mod linux {
launch: None, launch: None,
epoch: 0, epoch: 0,
gen, gen,
hw_cursor: false,
} }
} }
@@ -162,6 +162,15 @@ pub fn launch_into_gamescope_session(cmd: &str) -> Result<std::process::Child> {
gamescope::launch_into_session(cmd) gamescope::launch_into_session(cmd)
} }
/// Every nested Xwayland `(DISPLAY, XAUTHORITY)` of the running gamescope session for the XFixes
/// cursor source (remote-desktop-sweep Phase C) — gamescope can run several, and the pointer is on
/// whichever is focused. Empty when no gamescope session is running / it exposes no Xwayland (the
/// host then leaves gamescope cursorless, today's behaviour).
#[cfg(target_os = "linux")]
pub fn gamescope_xwayland_cursor_targets() -> Vec<(String, Option<String>)> {
gamescope::xwayland_cursor_targets()
}
/// B2: has a **dedicated** gamescope game session's game exited (its `node_id` doesn't reappear within a /// B2: has a **dedicated** gamescope game session's game exited (its `node_id` doesn't reappear within a
/// short window after capture loss)? The dedicated-spawn session ends cleanly on `true` instead of the /// short window after capture loss)? The dedicated-spawn session ends cleanly on `true` instead of the
/// capture-loss rebuild. Scoped to the session's OWN node so a coexisting gamescope doesn't mask the /// capture-loss rebuild. Scoped to the session's OWN node so a coexisting gamescope doesn't mask the
@@ -207,6 +216,20 @@ pub fn cancel_pending_tv_restore() {
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
pub fn cancel_pending_tv_restore() {} pub fn cancel_pending_tv_restore() {}
/// Can the MANAGED gamescope path stand a session up from nothing on this box (SteamOS's
/// `gamescope-session` launcher or Bazzite's `gamescope-session-plus` present)? Lets the connect
/// path route a "no live graphical session" box to the gamescope takeover — which rebuilds the
/// session at the client's mode — instead of failing the connect. Always `false` off Linux.
#[cfg(target_os = "linux")]
pub fn managed_session_available() -> bool {
gamescope::managed_session_available()
}
#[cfg(not(target_os = "linux"))]
pub fn managed_session_available() -> bool {
false
}
/// Call when a client session ends: if the host-managed gamescope path took over a box's autologin /// Call when a client session ends: if the host-managed gamescope path took over a box's autologin
/// gaming session (stopped its single-instance Steam to stream at the client's mode), **schedule** a /// gaming session (stopped its single-instance Steam to stream at the client's mode), **schedule** a
/// debounced restore so the TV returns to gaming mode — unless a client reconnects within the window /// debounced restore so the TV returns to gaming mode — unless a client reconnects within the window
@@ -57,6 +57,13 @@ pub fn observe_session_instance(active: &ActiveSession) {
if let Some(old) = compositor_for_kind(prev.0) { if let Some(old) = compositor_for_kind(prev.0) {
registry::invalidate_backend(old.id()); registry::invalidate_backend(old.id());
} }
// The dead desktop's socket vars may still sit in the systemd --user manager env
// ([`settle_desktop_portal`]'s import-environment) — scrub them NOW, or the next
// `gamescope-session.target` start inherits a stale WAYLAND_DISPLAY and gamescope
// runs NESTED against the dead desktop socket instead of becoming the display
// server ("Failed to connect to wayland socket: wayland-0" — kept a Deck's Game
// Mode from starting at all, observed live 2026-07-21).
scrub_desktop_manager_env();
} }
let epoch = bump_session_epoch(); let epoch = bump_session_epoch();
tracing::info!( tracing::info!(
@@ -70,6 +77,23 @@ pub fn observe_session_instance(active: &ActiveSession) {
*last = Some(cur); *last = Some(cur);
} }
/// Counterpart to [`settle_desktop_portal`]'s `import-environment`: drop the desktop session's
/// socket vars from the systemd `--user` manager env once that desktop instance is GONE. They
/// persist in the manager otherwise, and every later user unit inherits them — including
/// `gamescope-session.target`, whose gamescope then aborts trying to attach to the dead desktop
/// socket. Best-effort; the D-Bus activation env has no unset op, but gamescope-session is
/// systemd-started, so the manager scrub is the one that matters. (A desktop restart re-imports
/// via the next [`settle_desktop_portal`], so scrubbing on a bounce is harmless.)
#[cfg(target_os = "linux")]
fn scrub_desktop_manager_env() {
let _ = std::process::Command::new("systemctl")
.args(["--user", "unset-environment", "WAYLAND_DISPLAY", "DISPLAY"])
.status();
}
#[cfg(not(target_os = "linux"))]
fn scrub_desktop_manager_env() {}
/// Is `kind` a **desktop** compositor (KWin / Mutter / wlroots) — one whose kept PipeWire outputs die /// Is `kind` a **desktop** compositor (KWin / Mutter / wlroots) — one whose kept PipeWire outputs die
/// with the compositor instance, so the session epoch tracks it? `Gaming` (gamescope) and `None` are /// with the compositor instance, so the session epoch tracks it? `Gaming` (gamescope) and `None` are
/// not (gamescope spawns are independent nested sessions — see [`observe_session_instance`]). /// not (gamescope spawns are independent nested sessions — see [`observe_session_instance`]).
@@ -71,6 +71,15 @@ struct Monitor {
/// selectable). The pin is never re-issued on reuse, so this is what the driver still renders /// selectable). The pin is never re-issued on reuse, so this is what the driver still renders
/// on — [`warn_if_pick_moved`] compares the CURRENT pick against it. /// on — [`warn_if_pick_moved`] compares the CURRENT pick against it.
render_pin: Option<LUID>, render_pin: Option<LUID>,
/// This monitor was ADDed with the v5 hardware-cursor flag (already driver-proto-gated) —
/// preserved across a re-arrival resize so the recreated monitor keeps the cursor channel.
hw_cursor: bool,
/// The ADD reply flagged this monitor's OS target as carrying an IRREVOCABLE hardware-cursor
/// declare from an earlier session (§8.6): DWM excludes the pointer from its frames forever
/// (the sticky state survives monitor REMOVE→ADD via the stable per-client target id), so a
/// session WITHOUT the cursor channel must self-composite — carried into
/// [`WinCaptureTarget`] for the IDD-push capturer's forced-composite gate.
cursor_excluded: bool,
/// The driver's WUDFHost pid (from the ADD reply) — carried into [`WinCaptureTarget`] so the /// The driver's WUDFHost pid (from the ADD reply) — carried into [`WinCaptureTarget`] so the
/// IDD-push capturer knows where to duplicate the sealed frame channel's handles. The SAME /// IDD-push capturer knows where to duplicate the sealed frame channel's handles. The SAME
/// process for every parallel monitor (one devnode → one WUDFHost hosts all publishers), which /// process for every parallel monitor (one devnode → one WUDFHost hosts all publishers), which
@@ -98,6 +107,7 @@ impl Monitor {
gdi_name: n, gdi_name: n,
target_id: self.target_id, target_id: self.target_id,
wudf_pid: self.wudf_pid, wudf_pid: self.wudf_pid,
cursor_excluded: self.cursor_excluded,
}) })
} }
} }
@@ -267,6 +277,25 @@ pub fn vdm() -> &'static VirtualDisplayManager {
/// for the process lifetime — a dead one is RETIRED (kept alive, see [`DeviceSlot`]), so a stale copy /// for the process lifetime — a dead one is RETIRED (kept alive, see [`DeviceSlot`]), so a stale copy
/// can only fail IOCTLs, never dangle. `None` before the first backend open — impossible for a /// can only fail IOCTLs, never dangle. `None` before the first backend open — impossible for a
/// capturer, which only exists on a monitor the manager created. /// capturer, which only exists on a monitor the manager created.
/// Can this host's pf-vdisplay driver run the v5 hardware-cursor channel? Reads the
/// handshake-latched protocol version, opening the control device once if no session has
/// opened it yet this service run (the same open every session performs anyway) — so the
/// Welcome-time capability decision never guesses. `false` when the driver is missing/stale.
///
/// The FIRST session's Welcome precedes any backend construction (`vdisplay::open` runs at
/// display prep, after the Welcome), so this must not assume an initialised manager —
/// `init` is idempotent and constructing the driver facade is free (on-glass finding: the
/// `vdm()` expect panicked the very first handshake of a fresh service).
pub fn hw_cursor_capable() -> bool {
let m = init(Box::new(crate::driver::PfVdisplayDriver));
let v = m.driver_proto.load(Ordering::Relaxed);
if v != 0 {
return v >= 5;
}
let _ = m.ensure_device();
m.driver_proto.load(Ordering::Relaxed) >= 5
}
pub fn control_device_handle() -> Option<HANDLE> { pub fn control_device_handle() -> Option<HANDLE> {
VDM.get().and_then(VirtualDisplayManager::device_handle) VDM.get().and_then(VirtualDisplayManager::device_handle)
} }
@@ -390,6 +419,7 @@ impl VirtualDisplayManager {
mode: Mode, mode: Mode,
client_fp: Option<[u8; 32]>, client_fp: Option<[u8; 32]>,
client_hdr: Option<punktfunk_core::quic::HdrMeta>, client_hdr: Option<punktfunk_core::quic::HdrMeta>,
hw_cursor: bool,
quit: Option<Arc<AtomicBool>>, quit: Option<Arc<AtomicBool>>,
) -> Result<VirtualOutput> { ) -> Result<VirtualOutput> {
// Console-session guard: a host outside the ACTIVE console session cannot drive the display // Console-session guard: a host outside the ACTIVE console session cannot drive the display
@@ -622,7 +652,9 @@ impl VirtualDisplayManager {
// SAFETY: `create_monitor` requires `dev` to be a valid control handle; `dev` is the handle // SAFETY: `create_monitor` requires `dev` to be a valid control handle; `dev` is the handle
// `ensure_device()` returned above (cached handles are never closed — a dead one is retired, // `ensure_device()` returned above (cached handles are never closed — a dead one is retired,
// kept alive; see `DeviceSlot`), and we hold the `state` lock. // kept alive; see `DeviceSlot`), and we hold the `state` lock.
let mon = match unsafe { self.create_monitor(dev, mode, slot, client_hdr, &mut inner) } { let mon = match unsafe {
self.create_monitor(dev, mode, slot, client_hdr, hw_cursor, &mut inner)
} {
// The cached device died under us (driver upgrade / WUDFHost restart, detected only // The cached device died under us (driver upgrade / WUDFHost restart, detected only
// now — e.g. the host sat idle past the pinger-less window). Retire it, reopen, and // now — e.g. the host sat idle past the pinger-less window). Retire it, reopen, and
// retry ONCE so the reconnect-after-driver-restart succeeds first try instead of // retry ONCE so the reconnect-after-driver-restart succeeds first try instead of
@@ -635,7 +667,7 @@ impl VirtualDisplayManager {
); );
// SAFETY: as above — `dev` is the handle the reopening `ensure_device` just // SAFETY: as above — `dev` is the handle the reopening `ensure_device` just
// returned, and the `state` lock is still held. // returned, and the `state` lock is still held.
unsafe { self.create_monitor(dev, mode, slot, client_hdr, &mut inner)? } unsafe { self.create_monitor(dev, mode, slot, client_hdr, hw_cursor, &mut inner)? }
} }
r => r?, r => r?,
}; };
@@ -861,6 +893,7 @@ impl VirtualDisplayManager {
mode: Mode, mode: Mode,
slot: u32, slot: u32,
client_hdr: Option<punktfunk_core::quic::HdrMeta>, client_hdr: Option<punktfunk_core::quic::HdrMeta>,
hw_cursor: bool,
inner: &mut MgrInner, inner: &mut MgrInner,
) -> Result<Monitor> { ) -> Result<Monitor> {
// The slot id doubles as the driver-preferred monitor id (EDID serial / ConnectorIndex), so // The slot id doubles as the driver-preferred monitor id (EDID serial / ConnectorIndex), so
@@ -868,13 +901,17 @@ impl VirtualDisplayManager {
// `0` (anonymous) = the driver auto-allocates the lowest-free id. // `0` (anonymous) = the driver auto-allocates the lowest-free id.
let preferred_id = slot; let preferred_id = slot;
let render_pin = resolve_render_pin(); let render_pin = resolve_render_pin();
// Hardware cursor only against a driver that implements the v5 channel: an older driver
// ignores the AddRequest field anyway (composited cursor), but gating here keeps the
// capture layer from creating + delivering a section nobody will ever publish into.
let hw_cursor = hw_cursor && self.driver_proto.load(Ordering::Relaxed) >= 5;
// SAFETY: `create_monitor`'s own `# Safety` contract guarantees `dev` is the live control // SAFETY: `create_monitor`'s own `# Safety` contract guarantees `dev` is the live control
// handle; we forward it unchanged to `add_monitor`, whose precondition is exactly that. // handle; we forward it unchanged to `add_monitor`, whose precondition is exactly that.
// `render_pin` is an `Option<LUID>` by value (plain `Copy`), so no borrowed memory // `render_pin` is an `Option<LUID>` by value (plain `Copy`), so no borrowed memory
// crosses the call. // crosses the call.
let added = unsafe { let added = unsafe {
self.driver self.driver
.add_monitor(dev, mode, render_pin, preferred_id, client_hdr)? .add_monitor(dev, mode, render_pin, preferred_id, client_hdr, hw_cursor)?
}; };
// Mandatory keepalive: ping inside the watchdog window or the driver tears all displays down. // Mandatory keepalive: ping inside the watchdog window or the driver tears all displays down.
@@ -1061,6 +1098,8 @@ impl VirtualDisplayManager {
resolved_monitor_id: added.resolved_monitor_id, resolved_monitor_id: added.resolved_monitor_id,
position: (0, 0), position: (0, 0),
gen: self.gen.fetch_add(1, Ordering::Relaxed), gen: self.gen.fetch_add(1, Ordering::Relaxed),
hw_cursor,
cursor_excluded: added.cursor_excluded,
}) })
} }
@@ -1231,7 +1270,7 @@ impl VirtualDisplayManager {
// values passed by value — no borrow crosses the call. // values passed by value — no borrow crosses the call.
let added = unsafe { let added = unsafe {
self.driver self.driver
.add_monitor(dev, mode, render_pin, slot, client_hdr) .add_monitor(dev, mode, render_pin, slot, client_hdr, old.hw_cursor)
.context("re-arrival ADD at the new mode")? .context("re-arrival ADD at the new mode")?
}; };
self.ensure_pinger(); self.ensure_pinger();
@@ -1283,6 +1322,10 @@ impl VirtualDisplayManager {
resolved_monitor_id: added.resolved_monitor_id, resolved_monitor_id: added.resolved_monitor_id,
position: old.position, position: old.position,
gen: old.gen, gen: old.gen,
hw_cursor: old.hw_cursor,
// Fresh from THIS reply, not `old`: the driver's per-target declare registry is the
// ground truth (this session may itself have declared since the original ADD).
cursor_excluded: added.cursor_excluded,
}) })
} }
@@ -22,6 +22,11 @@ pub(crate) struct AddedMonitor {
pub luid: LUID, pub luid: LUID,
pub wudf_pid: u32, pub wudf_pid: u32,
pub resolved_monitor_id: u32, pub resolved_monitor_id: u32,
/// The driver reports the OS target already carries an IRREVOCABLE hardware-cursor declare
/// from an earlier session (`AddReply::cursor_excluded`, remote-desktop-sweep §8.6): DWM
/// excludes the pointer from this target's frames forever, so a session without the cursor
/// channel must self-composite (GDI poller + blend) or stream a cursor-less desktop.
pub cursor_excluded: bool,
} }
/// The backend-specific IOCTL surface — the *only* thing that differs between SudoVDA and pf-vdisplay. /// The backend-specific IOCTL surface — the *only* thing that differs between SudoVDA and pf-vdisplay.
@@ -56,6 +61,7 @@ pub(crate) trait VdisplayDriver: Send + Sync {
render_luid: Option<LUID>, render_luid: Option<LUID>,
preferred_monitor_id: u32, preferred_monitor_id: u32,
client_hdr: Option<punktfunk_core::quic::HdrMeta>, client_hdr: Option<punktfunk_core::quic::HdrMeta>,
hw_cursor: bool,
) -> Result<AddedMonitor>; ) -> Result<AddedMonitor>;
/// Refresh the LIVE monitor `key`'s advertised mode list to lead with `mode` (the in-place /// Refresh the LIVE monitor `key`'s advertised mode list to lead with `mode` (the in-place
/// mid-stream resize, latency plan P2 — pf-vdisplay `IOCTL_UPDATE_MODES`, driver protocol v4). /// mid-stream resize, latency plan P2 — pf-vdisplay `IOCTL_UPDATE_MODES`, driver protocol v4).
@@ -234,6 +234,55 @@ pub unsafe fn send_frame_channel(dev: HANDLE, req: &control::SetFrameChannelRequ
.context("pf-vdisplay SET_FRAME_CHANNEL") .context("pf-vdisplay SET_FRAME_CHANNEL")
} }
/// Deliver a monitor's hardware-cursor section (`IOCTL_SET_CURSOR_CHANNEL`, proto v5) — the
/// cursor sibling of [`send_frame_channel`], same delivery/ownership contract.
///
/// # Safety
/// `dev` must be a live pf-vdisplay control handle (see [`super::manager::control_device_handle`]).
pub unsafe fn send_cursor_channel(
dev: HANDLE,
req: &control::SetCursorChannelRequest,
) -> Result<()> {
let mut none: [u8; 0] = [];
// SAFETY: per this fn's contract `dev` is the live control handle; `bytes_of(req)` borrows the
// caller's request across this synchronous call; no output buffer.
unsafe {
ioctl(
dev,
control::IOCTL_SET_CURSOR_CHANNEL,
bytemuck::bytes_of(req),
&mut none,
)
}
.map(|_| ())
.context("pf-vdisplay SET_CURSOR_CHANNEL")
}
/// Flip a LIVE monitor's hardware-cursor declaration (`IOCTL_SET_CURSOR_FORWARD`, proto v6) —
/// the mid-stream cursor-render flip. Fails against a pre-v6 driver (unknown IOCTL); callers
/// log and keep the declared-at-ADD behavior.
///
/// # Safety
/// `dev` must be a live pf-vdisplay control handle (see [`super::manager::control_device_handle`]).
pub unsafe fn send_cursor_forward(
dev: HANDLE,
req: &control::SetCursorForwardRequest,
) -> Result<()> {
let mut none: [u8; 0] = [];
// SAFETY: per this fn's contract `dev` is the live control handle; `bytes_of(req)` borrows the
// caller's request across this synchronous call; no output buffer.
unsafe {
ioctl(
dev,
control::IOCTL_SET_CURSOR_FORWARD,
bytemuck::bytes_of(req),
&mut none,
)
}
.map(|_| ())
.context("pf-vdisplay SET_CURSOR_FORWARD")
}
/// RAII over a SetupAPI device-info list: every exit path of [`open_device`] destroys it (the error /// RAII over a SetupAPI device-info list: every exit path of [`open_device`] destroys it (the error
/// paths used to leak one `HDEVINFO` per failed open — and a driverless / mid-upgrade box probes /// paths used to leak one `HDEVINFO` per failed open — and a driverless / mid-upgrade box probes
/// repeatedly). /// repeatedly).
@@ -460,6 +509,7 @@ impl VdisplayDriver for PfVdisplayDriver {
render_luid: Option<LUID>, render_luid: Option<LUID>,
preferred_monitor_id: u32, preferred_monitor_id: u32,
client_hdr: Option<punktfunk_core::quic::HdrMeta>, client_hdr: Option<punktfunk_core::quic::HdrMeta>,
hw_cursor: bool,
) -> Result<AddedMonitor> { ) -> Result<AddedMonitor> {
let session_id = next_session_id(); let session_id = next_session_id();
// The client display's volume rides into the monitor's EDID CTA HDR block; all-zero = // The client display's volume rides into the monitor's EDID CTA HDR block; all-zero =
@@ -485,7 +535,11 @@ impl VdisplayDriver for PfVdisplayDriver {
max_luminance_nits, max_luminance_nits,
max_frame_avg_nits, max_frame_avg_nits,
min_luminance_millinits, min_luminance_millinits,
_reserved: 0, // v5 cursor channel: the driver declares an IddCx hardware cursor for this monitor
// (DWM stops compositing the pointer into the frame); the capture layer delivers the
// CursorShm section right after its ring. Zero toward older drivers is harmless —
// the host only sets this when the handshake-reported proto is >= 5.
hw_cursor: hw_cursor as u32,
}; };
// SET_RENDER_ADAPTER (opt-in; pf-vdisplay IMPLEMENTS it). Non-fatal on failure: the driver reports // SET_RENDER_ADAPTER (opt-in; pf-vdisplay IMPLEMENTS it). Non-fatal on failure: the driver reports
// its real render LUID in the shared header, so the host binds correctly even if this is ignored. // its real render LUID in the shared header, so the host binds correctly even if this is ignored.
@@ -551,10 +605,13 @@ impl VdisplayDriver for PfVdisplayDriver {
})?; })?;
// Fail closed on a short reply — `target_id`/`wudf_pid`/`luid` below feed OpenProcess + the // Fail closed on a short reply — `target_id`/`wudf_pid`/`luid` below feed OpenProcess + the
// WUDFHost verification, so don't decode a partially-written (zeroed) reply as authoritative. // WUDFHost verification, so don't decode a partially-written (zeroed) reply as authoritative.
if (n as usize) < size_of::<control::AddReply>() { // The LEGACY size, not the full struct: an un-upgraded driver writes only the prefix before
// the `cursor_excluded` tail; `out` is zero-initialized, so the missing tail reads `0`
// (= unknown/clean — exactly what a driver that can't track declares should report).
if (n as usize) < control::ADD_REPLY_LEGACY_SIZE {
anyhow::bail!( anyhow::bail!(
"pf-vdisplay ADD returned {n} bytes, expected {}", "pf-vdisplay ADD returned {n} bytes, expected at least {}",
size_of::<control::AddReply>() control::ADD_REPLY_LEGACY_SIZE
); );
} }
// `pod_read_unaligned` (NOT `from_bytes`): `out` is a stack `[u8; N]` with no guaranteed 4-byte // `pod_read_unaligned` (NOT `from_bytes`): `out` is a stack `[u8; N]` with no guaranteed 4-byte
@@ -569,6 +626,7 @@ impl VdisplayDriver for PfVdisplayDriver {
target_id = reply.target_id, target_id = reply.target_id,
adapter_luid = %format_args!("{:#x}", luid.LowPart), adapter_luid = %format_args!("{:#x}", luid.LowPart),
wudf_pid = reply.wudf_pid, wudf_pid = reply.wudf_pid,
cursor_excluded = reply.cursor_excluded != 0,
"pf-vdisplay monitor created {}x{}@{}", "pf-vdisplay monitor created {}x{}@{}",
mode.width, mode.width,
mode.height, mode.height,
@@ -605,6 +663,7 @@ impl VdisplayDriver for PfVdisplayDriver {
luid, luid,
wudf_pid: reply.wudf_pid, wudf_pid: reply.wudf_pid,
resolved_monitor_id: reply.resolved_monitor_id, resolved_monitor_id: reply.resolved_monitor_id,
cursor_excluded: reply.cursor_excluded != 0,
}) })
} }
@@ -682,6 +741,10 @@ pub struct PfVdisplayDisplay {
/// freshly created monitor's EDID advertises this volume so host apps tone-map to the client's /// freshly created monitor's EDID advertises this volume so host apps tone-map to the client's
/// real panel. /// real panel.
client_hdr: Option<punktfunk_core::quic::HdrMeta>, client_hdr: Option<punktfunk_core::quic::HdrMeta>,
/// Declare an IddCx hardware cursor on the created monitor (the M2c cursor channel). Set by
/// [`set_hw_cursor`](VirtualDisplay::set_hw_cursor) before `create`; only honored when the
/// driver handshake reported proto >= 5.
hw_cursor: bool,
/// The session's deliberate-quit flag (`None` = no signal → the linger policy applies). Set by /// The session's deliberate-quit flag (`None` = no signal → the linger policy applies). Set by
/// [`set_quit_flag`](VirtualDisplay::set_quit_flag) before `create`; rides into every lease this /// [`set_quit_flag`](VirtualDisplay::set_quit_flag) before `create`; rides into every lease this
/// backend mints so a user "stop" tears the monitor down immediately instead of lingering. /// backend mints so a user "stop" tears the monitor down immediately instead of lingering.
@@ -694,6 +757,7 @@ impl PfVdisplayDisplay {
Ok(Self { Ok(Self {
client_fp: None, client_fp: None,
client_hdr: None, client_hdr: None,
hw_cursor: false,
quit: None, quit: None,
}) })
} }
@@ -712,12 +776,26 @@ impl VirtualDisplay for PfVdisplayDisplay {
self.client_hdr = hdr; self.client_hdr = hdr;
} }
fn set_hw_cursor(&mut self, on: bool) {
self.hw_cursor = on;
}
fn hw_cursor(&self) -> bool {
self.hw_cursor
}
fn set_quit_flag(&mut self, quit: std::sync::Arc<std::sync::atomic::AtomicBool>) { fn set_quit_flag(&mut self, quit: std::sync::Arc<std::sync::atomic::AtomicBool>) {
self.quit = Some(quit); self.quit = Some(quit);
} }
fn create(&mut self, mode: Mode) -> Result<VirtualOutput> { fn create(&mut self, mode: Mode) -> Result<VirtualOutput> {
super::manager::vdm().acquire(mode, self.client_fp, self.client_hdr, self.quit.clone()) super::manager::vdm().acquire(
mode,
self.client_fp,
self.client_hdr,
self.hw_cursor,
self.quit.clone(),
)
} }
} }
@@ -804,17 +882,12 @@ mod tests {
// Live-run diagnostics: surface the manager/backend tracing (activation ladder, settle // Live-run diagnostics: surface the manager/backend tracing (activation ladder, settle
// waits, UPDATE_MODES) on stdout — a bare test harness has no subscriber, which made the // waits, UPDATE_MODES) on stdout — a bare test harness has no subscriber, which made the
// first on-glass run blind. // first on-glass run blind.
let _ = tracing_subscriber::fmt() // (tracing-subscriber is not a dep of this crate — run the host binary for traced runs.)
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| "debug".into()),
)
.try_init();
// Context probe: can this process see the CCD active-path set at all? (`None` = the query // Context probe: can this process see the CCD active-path set at all? (`None` = the query
// itself fails in this session/window-station — the whole ladder would be blind, and a // itself fails in this session/window-station — the whole ladder would be blind, and a
// "monitor never activated" verdict would be an artifact of the test context.) // "monitor never activated" verdict would be an artifact of the test context.)
// SAFETY: CCD query over an owned empty slice (test-only diagnostics). // SAFETY: CCD query over an owned empty slice (test-only diagnostics).
let active0 = unsafe { crate::win_display::count_other_active(&[]) }; let active0 = unsafe { pf_win_display::win_display::count_other_active(&[]) };
println!("spike: CCD active paths visible before create: {active0:?}"); println!("spike: CCD active paths visible before create: {active0:?}");
let mut vd = PfVdisplayDisplay::new().expect("open pf-vdisplay"); let mut vd = PfVdisplayDisplay::new().expect("open pf-vdisplay");
let first = vd let first = vd
@@ -847,7 +920,7 @@ mod tests {
.target_id; .target_id;
let in_place = t1 == t2; let in_place = t1 == t2;
// SAFETY: CCD query over a Copy target id (test-only diagnostics). // SAFETY: CCD query over a Copy target id (test-only diagnostics).
let active = unsafe { crate::win_display::active_resolution(t2) }; let active = unsafe { pf_win_display::win_display::active_resolution(t2) };
println!( println!(
"in-place resize spike: in_place={in_place} (target {t1} -> {t2}) took {resize_ms} ms, \ "in-place resize spike: in_place={in_place} (target {t1} -> {t2}) took {resize_ms} ms, \
active resolution now {active:?}" active resolution now {active:?}"
+67 -9
View File
@@ -23,10 +23,10 @@ use windows::core::PCWSTR;
use windows::Win32::Devices::Display::{ use windows::Win32::Devices::Display::{
DisplayConfigGetDeviceInfo, DisplayConfigSetDeviceInfo, GetDisplayConfigBufferSizes, DisplayConfigGetDeviceInfo, DisplayConfigSetDeviceInfo, GetDisplayConfigBufferSizes,
QueryDisplayConfig, SetDisplayConfig, DISPLAYCONFIG_DEVICE_INFO_GET_ADVANCED_COLOR_INFO, QueryDisplayConfig, SetDisplayConfig, DISPLAYCONFIG_DEVICE_INFO_GET_ADVANCED_COLOR_INFO,
DISPLAYCONFIG_DEVICE_INFO_GET_SOURCE_NAME, DISPLAYCONFIG_DEVICE_INFO_GET_TARGET_NAME, DISPLAYCONFIG_DEVICE_INFO_GET_SDR_WHITE_LEVEL, DISPLAYCONFIG_DEVICE_INFO_GET_SOURCE_NAME,
DISPLAYCONFIG_DEVICE_INFO_SET_ADVANCED_COLOR_STATE, DISPLAYCONFIG_GET_ADVANCED_COLOR_INFO, DISPLAYCONFIG_DEVICE_INFO_GET_TARGET_NAME, DISPLAYCONFIG_DEVICE_INFO_SET_ADVANCED_COLOR_STATE,
DISPLAYCONFIG_MODE_INFO, DISPLAYCONFIG_MODE_INFO_TYPE_SOURCE, DISPLAYCONFIG_GET_ADVANCED_COLOR_INFO, DISPLAYCONFIG_MODE_INFO,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_COMPONENT_VIDEO, DISPLAYCONFIG_MODE_INFO_TYPE_SOURCE, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_COMPONENT_VIDEO,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_COMPOSITE_VIDEO, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_COMPOSITE_VIDEO,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DISPLAYPORT_EMBEDDED, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DISPLAYPORT_EMBEDDED,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DISPLAYPORT_EXTERNAL, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DVI, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DISPLAYPORT_EXTERNAL, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_DVI,
@@ -35,10 +35,11 @@ use windows::Win32::Devices::Display::{
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SDI, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SDTVDONGLE, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SDI, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SDTVDONGLE,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SVIDEO, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_UDI_EMBEDDED, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_SVIDEO, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_UDI_EMBEDDED,
DISPLAYCONFIG_OUTPUT_TECHNOLOGY_UDI_EXTERNAL, DISPLAYCONFIG_PATH_INFO, DISPLAYCONFIG_OUTPUT_TECHNOLOGY_UDI_EXTERNAL, DISPLAYCONFIG_PATH_INFO,
DISPLAYCONFIG_SET_ADVANCED_COLOR_STATE, DISPLAYCONFIG_SOURCE_DEVICE_NAME, DISPLAYCONFIG_SDR_WHITE_LEVEL, DISPLAYCONFIG_SET_ADVANCED_COLOR_STATE,
DISPLAYCONFIG_TARGET_DEVICE_NAME, DISPLAYCONFIG_VIDEO_OUTPUT_TECHNOLOGY, QDC_ALL_PATHS, DISPLAYCONFIG_SOURCE_DEVICE_NAME, DISPLAYCONFIG_TARGET_DEVICE_NAME,
QDC_ONLY_ACTIVE_PATHS, SDC_ALLOW_CHANGES, SDC_APPLY, SDC_FORCE_MODE_ENUMERATION, DISPLAYCONFIG_VIDEO_OUTPUT_TECHNOLOGY, QDC_ALL_PATHS, QDC_ONLY_ACTIVE_PATHS, SDC_ALLOW_CHANGES,
SDC_SAVE_TO_DATABASE, SDC_TOPOLOGY_EXTEND, SDC_USE_SUPPLIED_DISPLAY_CONFIG, SDC_APPLY, SDC_FORCE_MODE_ENUMERATION, SDC_SAVE_TO_DATABASE, SDC_TOPOLOGY_EXTEND,
SDC_USE_SUPPLIED_DISPLAY_CONFIG,
}; };
use windows::Win32::Foundation::POINTL; use windows::Win32::Foundation::POINTL;
use windows::Win32::Graphics::Gdi::{ use windows::Win32::Graphics::Gdi::{
@@ -481,6 +482,52 @@ pub unsafe fn advanced_color_enabled(target_id: u32) -> Option<bool> {
None None
} }
/// The target's SDR white level as a SCALE relative to 80 nits (`1.0` = 80 nits): where DWM
/// places SDR-white when composing SDR content onto this HDR desktop. An SDR-authored overlay
/// (the composited cursor) must be multiplied by this in scRGB space or it renders visibly
/// darker than the surrounding SDR desktop content (the Windows "SDR content brightness"
/// slider default alone is ~2.5x). `None` = query failed / target not active (callers keep
/// their last value or 1.0).
///
/// # Safety
/// Runs the read-only CCD query FFI over owned locals (same shape as [`advanced_color_enabled`]).
pub unsafe fn sdr_white_level_scale(target_id: u32) -> Option<f32> {
let mut np = 0u32;
let mut nm = 0u32;
if GetDisplayConfigBufferSizes(QDC_ONLY_ACTIVE_PATHS, &mut np, &mut nm).is_err() {
return None;
}
let mut paths = vec![DISPLAYCONFIG_PATH_INFO::default(); np as usize];
let mut modes = vec![DISPLAYCONFIG_MODE_INFO::default(); nm as usize];
if QueryDisplayConfig(
QDC_ONLY_ACTIVE_PATHS,
&mut np,
paths.as_mut_ptr(),
&mut nm,
modes.as_mut_ptr(),
None,
)
.is_err()
{
return None;
}
for p in paths.iter().take(np as usize) {
if p.targetInfo.id == target_id {
let mut info = DISPLAYCONFIG_SDR_WHITE_LEVEL::default();
info.header.r#type = DISPLAYCONFIG_DEVICE_INFO_GET_SDR_WHITE_LEVEL;
info.header.size = size_of::<DISPLAYCONFIG_SDR_WHITE_LEVEL>() as u32;
info.header.adapterId = p.targetInfo.adapterId;
info.header.id = p.targetInfo.id;
if DisplayConfigGetDeviceInfo(&mut info.header) == 0 && info.SDRWhiteLevel > 0 {
// Contract: SDRWhiteLevel/1000 * 80 = nits, i.e. the /1000 IS the 80-nit scale.
return Some(info.SDRWhiteLevel as f32 / 1000.0);
}
return None;
}
}
None
}
/// Force the freshly-added virtual monitor to the client's exact `WxH@Hz`. The ADD IOCTL only /// Force the freshly-added virtual monitor to the client's exact `WxH@Hz`. The ADD IOCTL only
/// ADVERTISES the mode; Windows otherwise activates an IDD target at a 1280x720 default, so the /// ADVERTISES the mode; Windows otherwise activates an IDD target at a 1280x720 default, so the
/// ACTIVE mode (what DXGI Desktop Duplication captures) must be set explicitly. CDS_TEST first so a /// ACTIVE mode (what DXGI Desktop Duplication captures) must be set explicitly. CDS_TEST first so a
@@ -861,7 +908,18 @@ pub unsafe fn isolate_displays_ccd(keep_target_ids: &[u32]) -> Option<SavedConfi
continue; continue;
} }
if p.flags & DISPLAYCONFIG_PATH_ACTIVE != 0 { if p.flags & DISPLAYCONFIG_PATH_ACTIVE != 0 {
p.flags &= !DISPLAYCONFIG_PATH_ACTIVE; // mark this path inactive // Mark the path inactive AND unpin its modes: per the SetDisplayConfig
// contract a path being turned OFF needs BOTH mode indexes marked invalid,
// and leaving them referencing the queried mode entries gets the whole
// supplied config rejected with 0x57 ERROR_INVALID_PARAMETER on some
// driver/topology combinations (field-reported: exclusive mode left the
// physical panel lit, every retry failing 0x57). Writing the all-ones
// sentinel to the whole union is also correct under the virtual-mode-aware
// interpretation (cloneGroupId/sourceModeInfoIdx both become their 0xffff
// INVALID values).
p.flags &= !DISPLAYCONFIG_PATH_ACTIVE;
p.sourceInfo.Anonymous.modeInfoIdx = DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
p.targetInfo.Anonymous.modeInfoIdx = DISPLAYCONFIG_PATH_MODE_IDX_INVALID;
others += 1; others += 1;
} }
} }
+6 -255
View File
@@ -7,8 +7,12 @@
//! and ffmpeg's `hevc_nvenc` (encode thread) — each thread makes it current before use; //! and ffmpeg's `hevc_nvenc` (encode thread) — each thread makes it current before use;
//! * device memory: pitched allocations, the reusable `BufferPool`/`DeviceBuffer`, IPC //! * device memory: pitched allocations, the reusable `BufferPool`/`DeviceBuffer`, IPC
//! export/import, host readback, and the plane copies; //! export/import, host readback, and the plane copies;
//! * GL / external-memory interop (`RegisteredTexture`, `ExternalDmabuf`); and //! * GL / external-memory interop (`RegisteredTexture`, `ExternalDmabuf`).
//! * the CUDA cursor-blend kernel (`CursorBlend`). //!
//! (The CUDA cursor-blend PTX kernel that used to live here is retired: vendored PTX is JIT'd
//! against the driver's ISA ceiling and silently dies on older drivers. The NVENC cursor blend
//! is now the SPIR-V compute pass in [`super::vkslot`], dispatched over Vulkan-allocated,
//! CUDA-imported input slots.)
//! //!
//! (We use GL interop, not EGL interop: `cuGraphicsEGLRegisterImage` is Tegra-only on the desktop //! (We use GL interop, not EGL interop: `cuGraphicsEGLRegisterImage` is Tegra-only on the desktop
//! driver — see [`super::egl`].) //! driver — see [`super::egl`].)
@@ -18,7 +22,6 @@
#![deny(clippy::undocumented_unsafe_blocks)] #![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{bail, Result}; use anyhow::{bail, Result};
use std::ffi::CStr;
use std::os::raw::{c_uint, c_void}; use std::os::raw::{c_uint, c_void};
use std::sync::{Arc, Mutex, OnceLock}; use std::sync::{Arc, Mutex, OnceLock};
@@ -280,258 +283,6 @@ pub fn copy_stream_handle() -> *mut c_void {
/// Max cursor-overlay bitmap edge (px) uploaded to the device blend buffer — matches the Vulkan path. /// Max cursor-overlay bitmap edge (px) uploaded to the device blend buffer — matches the Vulkan path.
pub const CURSOR_MAX: u32 = 256; pub const CURSOR_MAX: u32 = 256;
/// GPU cursor-overlay compositor for the NVENC path (cursor-as-metadata): loads the `cursor_blend`
/// PTX module once and blends a straight-alpha RGBA cursor into an encoder-OWNED NVENC input surface
/// (ARGB / NV12 / YUV444) with a small kernel launched over the cursor's rectangle — no full-frame
/// pass, and the compositor's dmabuf is never touched. The cursor bitmap lives in a device buffer
/// re-uploaded only when it changes. Requires `context()` to have succeeded (driver present).
pub struct CursorBlend {
module: CUmodule,
f_argb: CUfunction,
f_nv12: CUfunction,
f_yuv444: CUfunction,
cur_buf: CUdeviceptr, // device RGBA staging (CURSOR_MAX²·4, tight rows)
}
// SAFETY: process-lifetime driver handles used only from the encode thread with the shared context
// current — like [`DeviceBuffer`], moving the struct between threads cannot dangle or race.
unsafe impl Send for CursorBlend {}
impl CursorBlend {
/// Load the embedded PTX image and resolve the three blend kernels + a device cursor buffer.
pub fn new(ptx: &[u8]) -> Result<CursorBlend> {
// cuModuleLoadData reads a PTX image as a NUL-terminated string; the embedded .ptx is not,
// so append a terminator.
let mut image = ptx.to_vec();
image.push(0);
let mut module: CUmodule = std::ptr::null_mut();
// SAFETY: `&mut module` is a live out-param the driver fills; `image` is a NUL-terminated PTX
// byte image that outlives the synchronous load. `ck` bails on error before `module` is used.
unsafe {
ck(
cuModuleLoadData(&mut module, image.as_ptr() as *const c_void),
"cuModuleLoadData(cursor_blend)",
)?;
}
let getf = |name: &CStr| -> Result<CUfunction> {
let mut f: CUfunction = std::ptr::null_mut();
// SAFETY: `module` loaded above; each name is a valid NUL-terminated symbol present in
// the module (verified in the .ptx `.entry` list); `&mut f` is a live out-param.
unsafe {
ck(
cuModuleGetFunction(&mut f, module, name.as_ptr()),
"cuModuleGetFunction",
)?;
}
Ok(f)
};
let f_argb = getf(c"blend_argb")?;
let f_nv12 = getf(c"blend_nv12")?;
let f_yuv444 = getf(c"blend_yuv444")?;
let mut cur_buf: CUdeviceptr = 0;
// SAFETY: `&mut cur_buf` is a live out-param; the size fits the CURSOR_MAX² RGBA buffer.
unsafe {
ck(
cuMemAlloc_v2(&mut cur_buf, (CURSOR_MAX * CURSOR_MAX * 4) as usize),
"cuMemAlloc(cursor)",
)?;
}
Ok(CursorBlend {
module,
f_argb,
f_nv12,
f_yuv444,
cur_buf,
})
}
/// Upload the cursor RGBA (`cw*ch*4`, tight rows) into the device blend buffer. Call only when
/// the bitmap changes; position moves are just kernel args.
pub fn upload(&self, rgba: &[u8], cw: u32, ch: u32) -> Result<()> {
let cw = cw.min(CURSOR_MAX);
let ch = ch.min(CURSOR_MAX);
let row = cw as usize * 4;
let copy = CUDA_MEMCPY2D {
srcMemoryType: 1, // HOST
srcHost: rgba.as_ptr() as *const c_void,
srcPitch: row,
dstMemoryType: CU_MEMORYTYPE_DEVICE,
dstDevice: self.cur_buf,
dstPitch: row,
WidthInBytes: row,
Height: ch as usize,
..Default::default()
};
// SAFETY: HOST→DEVICE 2D copy of `row*ch` bytes; `rgba` covers at least that (caller passes
// `cw*ch*4`), `cur_buf` is the CURSOR_MAX²·4 device alloc (row ≤ CURSOR_MAX·4, ch ≤ CURSOR_MAX).
// Synchronous via `copy_blocking`. Requires the context current (caller's contract).
unsafe { copy_blocking(&copy, "cursor HtoD") }
}
/// Blend into a packed 4-byte (NVENC ARGB) owned surface at `(ox,oy)`.
#[allow(clippy::too_many_arguments)] // surface geometry + cursor size + offset — a struct would just be unpacked at the call
pub fn blend_argb(
&self,
surf: CUdeviceptr,
pitch: usize,
w: u32,
h: u32,
cw: u32,
ch: u32,
ox: i32,
oy: i32,
sync: bool,
) -> Result<()> {
let (mut a_surf, mut a_cur) = (surf, self.cur_buf);
let (mut a_pitch, mut a_w, mut a_h) = (pitch as i32, w as i32, h as i32);
let (mut a_cw, mut a_ch) = (cw.min(CURSOR_MAX) as i32, ch.min(CURSOR_MAX) as i32);
let (mut a_ox, mut a_oy) = (ox, oy);
let mut args: [*mut c_void; 9] = [
&mut a_surf as *mut _ as *mut c_void,
&mut a_pitch as *mut _ as *mut c_void,
&mut a_w as *mut _ as *mut c_void,
&mut a_h as *mut _ as *mut c_void,
&mut a_cur as *mut _ as *mut c_void,
&mut a_cw as *mut _ as *mut c_void,
&mut a_ch as *mut _ as *mut c_void,
&mut a_ox as *mut _ as *mut c_void,
&mut a_oy as *mut _ as *mut c_void,
];
self.launch(self.f_argb, a_cw as u32, a_ch as u32, &mut args, sync)
}
/// Blend into an owned planar YUV444 surface (3 stacked full-res planes) at `(ox,oy)`.
#[allow(clippy::too_many_arguments)] // surface geometry + cursor size + offset — a struct would just be unpacked at the call
pub fn blend_yuv444(
&self,
base: CUdeviceptr,
pitch: usize,
w: u32,
h: u32,
cw: u32,
ch: u32,
ox: i32,
oy: i32,
sync: bool,
) -> Result<()> {
let (mut a_base, mut a_cur) = (base, self.cur_buf);
let (mut a_pitch, mut a_w, mut a_h) = (pitch as i32, w as i32, h as i32);
let (mut a_cw, mut a_ch) = (cw.min(CURSOR_MAX) as i32, ch.min(CURSOR_MAX) as i32);
let (mut a_ox, mut a_oy) = (ox, oy);
let mut args: [*mut c_void; 9] = [
&mut a_base as *mut _ as *mut c_void,
&mut a_pitch as *mut _ as *mut c_void,
&mut a_w as *mut _ as *mut c_void,
&mut a_h as *mut _ as *mut c_void,
&mut a_cur as *mut _ as *mut c_void,
&mut a_cw as *mut _ as *mut c_void,
&mut a_ch as *mut _ as *mut c_void,
&mut a_ox as *mut _ as *mut c_void,
&mut a_oy as *mut _ as *mut c_void,
];
self.launch(self.f_yuv444, a_cw as u32, a_ch as u32, &mut args, sync)
}
/// Blend into an owned NV12 surface (Y plane at `base`, interleaved UV at `base + pitch*h`).
#[allow(clippy::too_many_arguments)] // surface geometry + cursor size + offset — a struct would just be unpacked at the call
pub fn blend_nv12(
&self,
base: CUdeviceptr,
pitch: usize,
w: u32,
h: u32,
cw: u32,
ch: u32,
ox: i32,
oy: i32,
sync: bool,
) -> Result<()> {
let (mut a_yb, mut a_uvb, mut a_cur) = (base, base + pitch as u64 * h as u64, self.cur_buf);
let (mut a_yp, mut a_uvp) = (pitch as i32, pitch as i32);
let (mut a_w, mut a_h) = (w as i32, h as i32);
let (mut a_cw, mut a_ch) = (cw.min(CURSOR_MAX) as i32, ch.min(CURSOR_MAX) as i32);
let (mut a_ox, mut a_oy) = (ox, oy);
let mut args: [*mut c_void; 11] = [
&mut a_yb as *mut _ as *mut c_void,
&mut a_yp as *mut _ as *mut c_void,
&mut a_uvb as *mut _ as *mut c_void,
&mut a_uvp as *mut _ as *mut c_void,
&mut a_w as *mut _ as *mut c_void,
&mut a_h as *mut _ as *mut c_void,
&mut a_cur as *mut _ as *mut c_void,
&mut a_cw as *mut _ as *mut c_void,
&mut a_ch as *mut _ as *mut c_void,
&mut a_ox as *mut _ as *mut c_void,
&mut a_oy as *mut _ as *mut c_void,
];
// One thread per 2x2 luma block → grid over ceil(cw/2) × ceil(ch/2).
self.launch(
self.f_nv12,
(a_cw as u32).div_ceil(2),
(a_ch as u32).div_ceil(2),
&mut args,
sync,
)
}
/// Launch `f` over a `work_w × work_h` grid (16×16 blocks) on the copy stream; `sync` waits
/// for it, `!sync` leaves completion to the stream (stream-ordered consumers only — the
/// kernel PARAMETERS are copied at launch time, so the arg locals need not outlive the call).
fn launch(
&self,
f: CUfunction,
work_w: u32,
work_h: u32,
args: &mut [*mut c_void],
sync: bool,
) -> Result<()> {
if work_w == 0 || work_h == 0 {
return Ok(());
}
const B: u32 = 16;
let stream = copy_stream();
// SAFETY: `f` is a resolved kernel from our loaded module; `args` holds pointers to live
// locals whose types match the kernel's C parameters (per the call site above) — CUDA
// copies the parameter values during `cuLaunchKernel` itself, so they need not outlive
// the call. Grid/block dims are non-zero. Launched on the copy stream (ordered after the
// input-surface copy issued on the same stream); `sync` waits, `!sync` leaves ordering to
// the stream (the NVENC IO-stream binding). Requires the context current.
unsafe {
ck(
cuLaunchKernel(
f,
work_w.div_ceil(B),
work_h.div_ceil(B),
1,
B,
B,
1,
0,
stream,
args.as_mut_ptr(),
std::ptr::null_mut(),
),
"cuLaunchKernel(cursor)",
)?;
if sync {
ck(cuStreamSynchronize(stream), "cuStreamSynchronize(cursor)")?;
}
Ok(())
}
}
}
impl Drop for CursorBlend {
fn drop(&mut self) {
// SAFETY: `cur_buf`/`module` are our own handles, freed exactly once here; the context is
// current on the encode thread that drops the encoder. Errors are ignored on teardown.
unsafe {
let _ = cuMemFree_v2(self.cur_buf);
let _ = cuModuleUnload(self.module);
}
}
}
/// Allocate one pitched device buffer for `width`x`height` 4-byte pixels; returns `(ptr, pitch)`. /// Allocate one pitched device buffer for `width`x`height` 4-byte pixels; returns `(ptr, pitch)`.
fn alloc_pitched(width: u32, height: u32) -> Result<(CUdeviceptr, usize)> { fn alloc_pitched(width: u32, height: u32) -> Result<(CUdeviceptr, usize)> {
let mut ptr: CUdeviceptr = 0; let mut ptr: CUdeviceptr = 0;
+1 -75
View File
@@ -9,7 +9,7 @@
#![deny(clippy::undocumented_unsafe_blocks)] #![deny(clippy::undocumented_unsafe_blocks)]
use anyhow::{bail, Result}; use anyhow::{bail, Result};
use std::os::raw::{c_char, c_int, c_uint, c_void}; use std::os::raw::{c_int, c_uint, c_void};
use std::sync::OnceLock; use std::sync::OnceLock;
pub type CUresult = c_uint; // CUDA_SUCCESS == 0 pub type CUresult = c_uint; // CUDA_SUCCESS == 0
@@ -20,8 +20,6 @@ pub type CUdeviceptr = u64;
pub type CUgraphicsResource = *mut c_void; pub type CUgraphicsResource = *mut c_void;
pub type CUarray = *mut c_void; pub type CUarray = *mut c_void;
pub type CUexternalMemory = *mut c_void; // opaque CUextMemory_st* pub type CUexternalMemory = *mut c_void; // opaque CUextMemory_st*
pub type CUmodule = *mut c_void; // opaque CUmod_st*
pub type CUfunction = *mut c_void; // opaque CUfunc_st*
/// `CUmemorytype` (cuda.h): HOST=1, DEVICE=2, ARRAY=3, UNIFIED=4. /// `CUmemorytype` (cuda.h): HOST=1, DEVICE=2, ARRAY=3, UNIFIED=4.
pub const CU_MEMORYTYPE_DEVICE: c_uint = 2; pub const CU_MEMORYTYPE_DEVICE: c_uint = 2;
@@ -144,26 +142,6 @@ pub(crate) struct CudaApi {
cuIpcGetMemHandle: unsafe extern "C" fn(*mut CUipcMemHandle, CUdeviceptr) -> CUresult, cuIpcGetMemHandle: unsafe extern "C" fn(*mut CUipcMemHandle, CUdeviceptr) -> CUresult,
cuIpcOpenMemHandle: unsafe extern "C" fn(*mut CUdeviceptr, CUipcMemHandle, c_uint) -> CUresult, cuIpcOpenMemHandle: unsafe extern "C" fn(*mut CUdeviceptr, CUipcMemHandle, c_uint) -> CUresult,
cuIpcCloseMemHandle: unsafe extern "C" fn(CUdeviceptr) -> CUresult, cuIpcCloseMemHandle: unsafe extern "C" fn(CUdeviceptr) -> CUresult,
// Cursor-overlay blend: a linear device alloc + a PTX module with the blend kernels launched
// over the cursor's small rectangle (see [`CursorBlend`]).
cuMemAlloc_v2: unsafe extern "C" fn(*mut CUdeviceptr, usize) -> CUresult,
cuModuleLoadData: unsafe extern "C" fn(*mut CUmodule, *const c_void) -> CUresult,
cuModuleUnload: unsafe extern "C" fn(CUmodule) -> CUresult,
cuModuleGetFunction: unsafe extern "C" fn(*mut CUfunction, CUmodule, *const c_char) -> CUresult,
#[allow(clippy::type_complexity)]
cuLaunchKernel: unsafe extern "C" fn(
CUfunction,
c_uint,
c_uint,
c_uint,
c_uint,
c_uint,
c_uint,
c_uint,
CUstream,
*mut *mut c_void,
*mut *mut c_void,
) -> CUresult,
} }
// SAFETY: every field is a bare `extern "C" fn` address into the leaked, process-lifetime // SAFETY: every field is a bare `extern "C" fn` address into the leaked, process-lifetime
// `libcuda` mapping (`cuda_api` `forget`s the `Library`, so it is never unloaded) — an immutable // `libcuda` mapping (`cuda_api` `forget`s the `Library`, so it is never unloaded) — an immutable
@@ -236,11 +214,6 @@ pub(crate) fn cuda_api() -> Option<&'static CudaApi> {
.or_else(|_| lib.get(b"cuIpcOpenMemHandle\0")) .or_else(|_| lib.get(b"cuIpcOpenMemHandle\0"))
.ok()?, .ok()?,
cuIpcCloseMemHandle: *lib.get(b"cuIpcCloseMemHandle\0").ok()?, cuIpcCloseMemHandle: *lib.get(b"cuIpcCloseMemHandle\0").ok()?,
cuMemAlloc_v2: *lib.get(b"cuMemAlloc_v2\0").ok()?,
cuModuleLoadData: *lib.get(b"cuModuleLoadData\0").ok()?,
cuModuleUnload: *lib.get(b"cuModuleUnload\0").ok()?,
cuModuleGetFunction: *lib.get(b"cuModuleGetFunction\0").ok()?,
cuLaunchKernel: *lib.get(b"cuLaunchKernel\0").ok()?,
}; };
std::mem::forget(lib); // keep libcuda mapped for the fn pointers' lifetime (process) std::mem::forget(lib); // keep libcuda mapped for the fn pointers' lifetime (process)
Some(api) Some(api)
@@ -304,53 +277,6 @@ pub(crate) unsafe fn cuMemFree_v2(dptr: CUdeviceptr) -> CUresult {
None => CU_ERROR_NOT_LOADED, None => CU_ERROR_NOT_LOADED,
} }
} }
pub(crate) unsafe fn cuMemAlloc_v2(dptr: *mut CUdeviceptr, size: usize) -> CUresult {
match cuda_api() {
Some(a) => (a.cuMemAlloc_v2)(dptr, size),
None => CU_ERROR_NOT_LOADED,
}
}
pub(crate) unsafe fn cuModuleLoadData(m: *mut CUmodule, image: *const c_void) -> CUresult {
match cuda_api() {
Some(a) => (a.cuModuleLoadData)(m, image),
None => CU_ERROR_NOT_LOADED,
}
}
pub(crate) unsafe fn cuModuleUnload(m: CUmodule) -> CUresult {
match cuda_api() {
Some(a) => (a.cuModuleUnload)(m),
None => CU_ERROR_NOT_LOADED,
}
}
pub(crate) unsafe fn cuModuleGetFunction(
f: *mut CUfunction,
m: CUmodule,
name: *const c_char,
) -> CUresult {
match cuda_api() {
Some(a) => (a.cuModuleGetFunction)(f, m, name),
None => CU_ERROR_NOT_LOADED,
}
}
#[allow(clippy::too_many_arguments)]
pub(crate) unsafe fn cuLaunchKernel(
f: CUfunction,
gx: c_uint,
gy: c_uint,
gz: c_uint,
bx: c_uint,
by: c_uint,
bz: c_uint,
shmem: c_uint,
stream: CUstream,
params: *mut *mut c_void,
extra: *mut *mut c_void,
) -> CUresult {
match cuda_api() {
Some(a) => (a.cuLaunchKernel)(f, gx, gy, gz, bx, by, bz, shmem, stream, params, extra),
None => CU_ERROR_NOT_LOADED,
}
}
pub(crate) unsafe fn cuMemcpy2DAsync_v2(copy: *const CUDA_MEMCPY2D, stream: CUstream) -> CUresult { pub(crate) unsafe fn cuMemcpy2DAsync_v2(copy: *const CUDA_MEMCPY2D, stream: CUstream) -> CUresult {
match cuda_api() { match cuda_api() {
Some(a) => (a.cuMemcpy2DAsync_v2)(copy, stream), Some(a) => (a.cuMemcpy2DAsync_v2)(copy, stream),
@@ -0,0 +1,170 @@
#version 450
// Cursor-overlay blend for the direct-SDK NVENC path (cursor-as-metadata), dispatched over the
// cursor's rectangle only — the Vulkan replacement for the retired cursor_blend.cu PTX kernels
// (PTX is JIT'd against the driver's ISA ceiling, so a vendored blob silently dies on older
// drivers: CUDA errors 222/218 on-glass; SPIR-V has no such coupling). The NVENC input surface is
// Vulkan-allocated, CUDA-imported external memory (see vkslot.rs), so this shader writes the very
// bytes NVENC encodes.
//
// MODE (spec constant): 0 = packed 4-byte ARGB (NVENC byte order B,G,R,A), 1 = NV12 (Y plane +
// interleaved half-res UV at row surfH), 2 = planar YUV444 (3 full-res planes stacked at
// pitch*surfH). BT.709 limited-range coefficients — identical to rgb2nv12_buf.comp and the
// retired .cu, so the cursor colour matches the frame regardless of backend.
//
// The surface SSBO is uint[] (no 8-bit storage dependency — maximum driver reach): every
// invocation exclusively owns the 32-bit words it read-modify-writes. ARGB: one invocation per
// cursor pixel = one word. NV12/YUV444: one invocation per WORD-ALIGNED 4-px luma span (per two
// rows for NV12, whose 2 chroma bytes-pairs land in one exclusive word). Spans are aligned to the
// SURFACE, not the cursor, so neighbouring invocations never share a word even at odd `ox`.
//
// Rebuild: glslc cursor_blend.comp -o cursor_blend.spv (vendored beside this file)
layout(local_size_x = 8, local_size_y = 8, local_size_z = 1) in;
layout(constant_id = 0) const uint MODE = 0;
layout(std430, binding = 0) buffer Surf { uint surf[]; };
layout(std430, binding = 1) readonly buffer Cur { uint cur[]; };
layout(push_constant) uniform Push {
uint pitch; // surface row stride, bytes (4-aligned by construction)
uint surfW; // content width, px
uint surfH; // luma rows (plane stride multiplier)
uint curW; // cursor bitmap width, px
uint curH; // cursor bitmap height, px
int ox; // cursor top-left on the surface, px (may be negative)
int oy;
} pc;
// Cursor texel (straight-alpha RGBA, tight rows) or (0,0,0,0) outside the bitmap.
uvec4 cursor_px(int cx, int cy) {
if (cx < 0 || cy < 0 || cx >= int(pc.curW) || cy >= int(pc.curH)) return uvec4(0);
uint w = cur[uint(cy) * pc.curW + uint(cx)];
return uvec4(w & 0xFFu, (w >> 8) & 0xFFu, (w >> 16) & 0xFFu, (w >> 24) & 0xFFu); // R,G,B,A
}
uint blend8(uint dst, uint src, uint a) {
return (src * a + dst * (255u - a)) / 255u;
}
// BT.709 limited RGB→Y/U/V (matches the retired .cu / rgb2nv12_buf.comp).
uint y_of(uvec4 s) {
return uint(clamp(16.0 + 0.1826 * float(s.r) + 0.6142 * float(s.g) + 0.0620 * float(s.b) + 0.5, 0.0, 255.0));
}
float u_of(uvec4 s) { return 128.0 - 0.1006 * float(s.r) - 0.3386 * float(s.g) + 0.4392 * float(s.b); }
float v_of(uvec4 s) { return 128.0 + 0.4392 * float(s.r) - 0.3989 * float(s.g) - 0.0403 * float(s.b); }
// Read-modify-write one byte lane of a word index.
void rmw_byte(uint word_idx, uint lane, uint val8, uint a) {
uint w = surf[word_idx];
uint shift = lane * 8u;
uint d = (w >> shift) & 0xFFu;
uint b = blend8(d, val8, a);
surf[word_idx] = (w & ~(0xFFu << shift)) | (b << shift);
}
void main() {
if (MODE == 0u) {
// ARGB: one invocation per cursor pixel; each surface pixel is one exclusive word.
int cx = int(gl_GlobalInvocationID.x);
int cy = int(gl_GlobalInvocationID.y);
if (cx >= int(pc.curW) || cy >= int(pc.curH)) return;
int px = pc.ox + cx, py = pc.oy + cy;
if (px < 0 || py < 0 || px >= int(pc.surfW) || py >= int(pc.surfH)) return;
uvec4 s = cursor_px(cx, cy);
if (s.a == 0u) return;
uint idx = (uint(py) * pc.pitch + uint(px) * 4u) / 4u;
uint w = surf[idx];
uint b = blend8(w & 0xFFu, s.b, s.a); // B lane
uint g = blend8((w >> 8) & 0xFFu, s.g, s.a); // G lane
uint r = blend8((w >> 16) & 0xFFu, s.r, s.a); // R lane
surf[idx] = (w & 0xFF000000u) | (r << 16) | (g << 8) | b;
return;
}
// NV12 / YUV444: one invocation per SURFACE-word-aligned 4-px luma span. Span origin:
// x0 = floor(ox/4)*4 + span*4 (surface px), rows walk the cursor rect.
int span = int(gl_GlobalInvocationID.x);
int row = int(gl_GlobalInvocationID.y);
int x0 = (pc.ox >> 2) << 2; // word-aligned start at/left-of ox (ox may be negative)
int px0 = x0 + span * 4;
if (MODE == 2u) {
// YUV444: rows walk cursor rows one at a time.
if (row >= int(pc.curH)) return;
int py = pc.oy + row;
if (py < 0 || py >= int(pc.surfH)) return;
uint plane = pc.pitch * pc.surfH;
for (int i = 0; i < 4; i++) {
int px = px0 + i;
int cx = px - pc.ox;
if (px < 0 || px >= int(pc.surfW)) continue;
uvec4 s = cursor_px(cx, row);
if (s.a == 0u) continue;
uint off = uint(py) * pc.pitch + uint(px);
uint U = uint(clamp(u_of(s) + 0.5, 0.0, 255.0));
uint V = uint(clamp(v_of(s) + 0.5, 0.0, 255.0));
rmw_byte(off / 4u, off % 4u, y_of(s), s.a);
rmw_byte((plane + off) / 4u, (plane + off) % 4u, U, s.a);
rmw_byte((2u * plane + off) / 4u, (2u * plane + off) % 4u, V, s.a);
}
return;
}
// NV12: rows walk 2-row luma blocks (row = block row). The span's 4 luma px × 2 rows are
// exclusive words; its 2 chroma samples (4 bytes) are one exclusive word.
int base_cy = row * 2;
if (base_cy >= int(pc.curH)) return;
// Luma: 4 px × 2 rows.
for (int j = 0; j < 2; j++) {
int cy = base_cy + j;
int py = pc.oy + cy;
if (cy >= int(pc.curH) || py < 0 || py >= int(pc.surfH)) continue;
for (int i = 0; i < 4; i++) {
int px = px0 + i;
int cx = px - pc.ox;
if (px < 0 || px >= int(pc.surfW)) continue;
uvec4 s = cursor_px(cx, cy);
if (s.a == 0u) continue;
uint off = uint(py) * pc.pitch + uint(px);
rmw_byte(off / 4u, off % 4u, y_of(s), s.a);
}
}
// Chroma: two UV samples covering the span's 2x2 blocks, alpha-weighted like the .cu kernel.
// The UV plane starts at row surfH; sample (uvx, uvy) lives at uv_base + uvy*pitch + uvx*2.
// Guard: only spans whose px0 is 4-aligned own their chroma word (px0 is by construction).
int py_top = pc.oy + base_cy;
int uvy = py_top >> 1;
if (py_top < 0 || uvy < 0 || uvy * 2 >= int(pc.surfH)) return;
uint uv_base = pc.pitch * pc.surfH;
for (int hf = 0; hf < 2; hf++) {
// Each hf = one 2x2 luma block = one UV sample (2 bytes).
int bx = px0 + hf * 2;
if (bx < 0 || bx >= int(pc.surfW)) continue;
int uvx = bx >> 1;
float ua = 0.0, va = 0.0, wa = 0.0;
int cnt = 0;
for (int j = 0; j < 2; j++) {
for (int i = 0; i < 2; i++) {
int px = bx + i;
int py = py_top + j;
int cx = px - pc.ox;
int cy = base_cy + j;
if (px < 0 || py < 0 || px >= int(pc.surfW) || py >= int(pc.surfH)) continue;
uvec4 s = cursor_px(cx, cy);
if (s.a == 0u) continue;
ua += u_of(s) * float(s.a);
va += v_of(s) * float(s.a);
wa += float(s.a);
cnt++;
}
}
if (wa <= 0.0 || cnt == 0) continue;
uint U = uint(clamp(ua / wa + 0.5, 0.0, 255.0));
uint V = uint(clamp(va / wa + 0.5, 0.0, 255.0));
uint amean = uint(clamp(wa / float(cnt) + 0.5, 0.0, 255.0));
uint off = uv_base + uint(uvy) * pc.pitch + uint(uvx) * 2u;
rmw_byte(off / 4u, off % 4u, U, amean);
rmw_byte((off + 1u) / 4u, (off + 1u) % 4u, V, amean);
}
}
Binary file not shown.
+1
View File
@@ -15,6 +15,7 @@ pub mod client;
pub mod cuda; pub mod cuda;
pub mod egl; pub mod egl;
pub mod proto; pub mod proto;
pub mod vkslot;
pub mod vulkan; pub mod vulkan;
pub mod worker; pub mod worker;

Some files were not shown because too many files have changed in this diff Show More