feat(vdisplay): compositor-embedded pointer for sessions without the cursor channel (Phase B)

Since the cursor-channel work, every Linux virtual output was created in
metadata pointer mode — making ALL sessions depend on host-side cursor
compositing, including the ones that can never use the channel (Moonlight/
GameStream, legacy clients, capture-mode starts). Those sessions paid the
blend bring-up per session and, whenever a visible cursor was composited,
the loss of NVENC's stream-ordered submit — for a strictly worse cursor
than the compositor's own.

Mirror the Windows no-regression gate: the already-wired per-session
set_hw_cursor(cursor_forward) now drives the Linux backends too. A
cursor-channel session gets metadata (shapes forwarded, composite flip
blends host-side — today's validated path, unchanged); every other session
gets the pointer compositor-EMBEDDED at creation (KWin zkde pointer=2,
Mutter cursor-mode=1, wlroots/hyprland portal CursorMode::Embedded — their
pre-channel default; the portal pair also gains the Metadata arm for
channel sessions, wired but untested on-glass). SessionPlan.cursor_blend
narrows to cursor-forward sessions and rides into the direct-SDK NVENC
open, which skips the Vulkan slot-blend bring-up entirely when off —
embedded sessions ring on plain CUDA surfaces and pay zero cursor cost,
per-session or per-frame.

The keep-alive registry's reuse key grows the created pointer mode (new
VirtualDisplay::hw_cursor getter): a kept embedded display has no cursor
metadata for a channel session to forward, and a kept metadata display
would leave a channel-less session with no pointer in its frames.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-22 23:47:53 +02:00
co-authored by Claude Fable 5
parent d2c46eaf3c
commit 925130d1f9
10 changed files with 205 additions and 46 deletions
+14 -8
View File
@@ -1000,12 +1000,16 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
ctx.bit_depth,
ctx.chroma,
ctx.codec,
// Blend CAPABILITY wherever the capture HAS a pointer (not gamescope) — including
// cursor-forward sessions: the client can flip to the capture mouse model mid-stream
// Blend CAPABILITY only for cursor-FORWARD sessions (Phase B, the Windows gate
// mirrored): their client can flip to the capture mouse model mid-stream
// (`CursorRenderMode`), and the composite side of that flip must not need an encoder
// rebuild. WHETHER a frame's pointer is drawn is per-tick: the encode loop strips
// `frame.cursor` while the client draws locally (see the forwarder tick).
ctx.compositor != pf_vdisplay::Compositor::Gamescope,
// rebuild WHETHER a frame's pointer is drawn stays per-tick (the encode loop strips
// `frame.cursor` while the client draws locally, see the forwarder tick). Every OTHER
// session's output is created with the pointer compositor-EMBEDDED
// (`vd.set_hw_cursor(false)` → no cursor metadata ever arrives, nothing to blend), so
// it keeps the zero-cost pre-channel path — and gamescope never has a pointer either
// way.
ctx.compositor != pf_vdisplay::Compositor::Gamescope && ctx.cursor_forward,
ctx.cursor_forward,
);
// PyroWave rides the datagram-aligned wire mode (§4.4): every encoder this session opens
@@ -2788,9 +2792,11 @@ pub(super) fn prepare_display(
bit_depth,
chroma,
codec,
// Blend capability regardless of cursor_forward — must MATCH virtual_stream's resolve
// (the mid-stream `CursorRenderMode` flip strips/keeps `frame.cursor` per tick).
compositor != pf_vdisplay::Compositor::Gamescope,
// Blend capability only for cursor-forward sessions — must MATCH virtual_stream's
// resolve (Phase B: non-channel sessions get the pointer compositor-EMBEDDED, nothing
// to blend; the mid-stream `CursorRenderMode` flip strips/keeps `frame.cursor` per
// tick for channel sessions).
compositor != pf_vdisplay::Compositor::Gamescope && cursor_forward,
cursor_forward,
);
if codec == crate::encode::Codec::PyroWave {