fix(host/encode): negotiate the cursor around what the encoder can blend
EncoderCaps::blends_cursor's contract said the HOST must fall back to capturer-side compositing when a cursor-as-metadata session lands on an encoder that can't composite — but that host half was never built: open_video warned and the session streamed WITHOUT a pointer (confirmed on the VAAPI dmabuf and libav-NVENC CUDA paths; latent on vulkan RGB-direct/native-NV12). The negotiation is now caps-aware, ahead of capture, on both planes: * pf-encode grows cursor_blend_capable() — the pre-open dispatch mirror (sibling of linux_native_nv12_ok) answering whether the resolved backend composites frame.cursor; its pure core is test-pinned arm by arm. * Native plane: handshake::cursor_forward grants the cursor channel only where the resolved backend can blend (the capture-mouse flip makes the host draw the pointer on demand); denied sessions keep the pre-channel path — the compositor EMBEDS the pointer, never cursorless, never doubled. The Welcome's HOST_CAP_CURSOR bit is computed once and read back at both session-wiring sites instead of recomputed. SessionPlan::output_format additionally keeps every cursor-blend session off producer-native NV12 (the arm with no CSC to fold a cursor into), and vulkan RGB-direct now yields to a cursor-blend session even when pinned (EFC cannot composite; the open logs the override). Windows plans cursor_blend=false via the new shared cursor_blend_for() rule — the IDD capturer composites the pointer itself, and asking the encoder anyway fired the blends-cursor warn spuriously on every cursor-channel session. * GameStream plane: the hardcoded cursor_blend=true is gone. The portal source asks for cursor-as-metadata only when the resolved backend blends, otherwise negotiates an Embedded pointer (choose_cursor_mode's new ladder); the capturer pool now also keys on that mode. The virtual-output source passes false — its capture embeds the pointer where it can. The per-arm warns in vulkan_video (RGB-direct, native-NV12) are now structurally unreachable and removed. open_video's post-open check stays as the single backstop for what planning cannot see: a Vulkan-open falling back to VAAPI mid-session, and the gamescope residual (no embedded mode exists there, so a never-blending backend — H.264-on-AMD VAAPI, software — still streams cursorless; fixing that needs a compositing stage, deliberately not built in this pass). Zero-copy is preserved throughout — every fallback is a capture-negotiation change, never a readback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1022,8 +1022,13 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
|
||||
// pointer compositor-EMBEDDED (`vd.set_hw_cursor(false)` → no cursor metadata, nothing to
|
||||
// blend), keeping the zero-cost pre-channel path. gamescope is the exception (Phase C):
|
||||
// it can't embed the pointer, so the host ALWAYS composites the XFixes-sourced cursor —
|
||||
// the blend must be built for every gamescope session.
|
||||
ctx.compositor == pf_vdisplay::Compositor::Gamescope || ctx.cursor_forward,
|
||||
// the blend must be built for every gamescope session. (`cursor_forward` is already
|
||||
// blend-gated: `handshake::cursor_forward` grants the channel only where
|
||||
// `encode::cursor_blend_capable` says the resolved backend composites.)
|
||||
crate::session_plan::cursor_blend_for(
|
||||
ctx.cursor_forward,
|
||||
ctx.compositor == pf_vdisplay::Compositor::Gamescope,
|
||||
),
|
||||
ctx.cursor_forward,
|
||||
);
|
||||
// gamescope: the XFixes cursor source feeds the always-on composite (Phase C). Set after
|
||||
@@ -2098,8 +2103,10 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
|
||||
// capture-mode channel); a switch AWAY restores the prior
|
||||
// gating. `plan` is `Copy` — this is the value the rebuild
|
||||
// (and its `build_pipeline` attach) reads.
|
||||
plan.cursor_blend = plan.cursor_forward
|
||||
|| c == crate::vdisplay::Compositor::Gamescope;
|
||||
plan.cursor_blend = crate::session_plan::cursor_blend_for(
|
||||
plan.cursor_forward,
|
||||
c == crate::vdisplay::Compositor::Gamescope,
|
||||
);
|
||||
plan.gamescope_cursor =
|
||||
c == crate::vdisplay::Compositor::Gamescope;
|
||||
gamescope_composite =
|
||||
@@ -2987,7 +2994,10 @@ pub(super) fn prepare_display(
|
||||
// non-gamescope sessions get the pointer compositor-EMBEDDED, nothing to blend; the
|
||||
// mid-stream `CursorRenderMode` flip strips/keeps `frame.cursor` per tick for channel
|
||||
// sessions). gamescope (Phase C) can't embed → always composites the XFixes cursor.
|
||||
compositor == pf_vdisplay::Compositor::Gamescope || cursor_forward,
|
||||
crate::session_plan::cursor_blend_for(
|
||||
cursor_forward,
|
||||
compositor == pf_vdisplay::Compositor::Gamescope,
|
||||
),
|
||||
cursor_forward,
|
||||
);
|
||||
plan.gamescope_cursor = compositor == pf_vdisplay::Compositor::Gamescope;
|
||||
|
||||
Reference in New Issue
Block a user