fix(host/encode): negotiate the cursor around what the encoder can blend

EncoderCaps::blends_cursor's contract said the HOST must fall back to
capturer-side compositing when a cursor-as-metadata session lands on an
encoder that can't composite — but that host half was never built: open_video
warned and the session streamed WITHOUT a pointer (confirmed on the VAAPI
dmabuf and libav-NVENC CUDA paths; latent on vulkan RGB-direct/native-NV12).
The negotiation is now caps-aware, ahead of capture, on both planes:

* pf-encode grows cursor_blend_capable() — the pre-open dispatch mirror
  (sibling of linux_native_nv12_ok) answering whether the resolved backend
  composites frame.cursor; its pure core is test-pinned arm by arm.
* Native plane: handshake::cursor_forward grants the cursor channel only
  where the resolved backend can blend (the capture-mouse flip makes the host
  draw the pointer on demand); denied sessions keep the pre-channel path —
  the compositor EMBEDS the pointer, never cursorless, never doubled. The
  Welcome's HOST_CAP_CURSOR bit is computed once and read back at both
  session-wiring sites instead of recomputed. SessionPlan::output_format
  additionally keeps every cursor-blend session off producer-native NV12 (the
  arm with no CSC to fold a cursor into), and vulkan RGB-direct now yields to
  a cursor-blend session even when pinned (EFC cannot composite; the open
  logs the override). Windows plans cursor_blend=false via the new shared
  cursor_blend_for() rule — the IDD capturer composites the pointer itself,
  and asking the encoder anyway fired the blends-cursor warn spuriously on
  every cursor-channel session.
* GameStream plane: the hardcoded cursor_blend=true is gone. The portal
  source asks for cursor-as-metadata only when the resolved backend blends,
  otherwise negotiates an Embedded pointer (choose_cursor_mode's new ladder);
  the capturer pool now also keys on that mode. The virtual-output source
  passes false — its capture embeds the pointer where it can.

The per-arm warns in vulkan_video (RGB-direct, native-NV12) are now
structurally unreachable and removed. open_video's post-open check stays as
the single backstop for what planning cannot see: a Vulkan-open falling back
to VAAPI mid-session, and the gamescope residual (no embedded mode exists
there, so a never-blending backend — H.264-on-AMD VAAPI, software — still
streams cursorless; fixing that needs a compositing stage, deliberately not
built in this pass). Zero-copy is preserved throughout — every fallback is a
capture-negotiation change, never a readback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-26 10:50:33 +02:00
co-authored by Claude Fable 5
parent f495b201e1
commit fc335b39e9
12 changed files with 405 additions and 109 deletions
+13 -3
View File
@@ -447,15 +447,25 @@ pub mod synthetic_nv12;
/// `want_hdr` offers the GNOME 50+ HDR formats (10-bit PQ/BT.2020 dmabufs) instead of the SDR
/// set — pass it only when the mirrored monitor is actually in HDR mode (the host probes
/// DisplayConfig) or the negotiation runs into its 10 s timeout and latches the SDR downgrade.
/// The [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts (the one-way edge).
/// `want_metadata_cursor` asks for cursor-as-metadata (`SPA_META_Cursor`) — pass it only when
/// the session's encode path composites `CapturedFrame::cursor` (the host consults
/// `pf-encode`'s `cursor_blend_capable`); otherwise the portal EMBEDS the pointer so it is
/// never silently lost. The [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts
/// (the one-way edge).
#[cfg(target_os = "linux")]
pub fn open_portal_monitor(
anchored: bool,
want_hdr: bool,
want_metadata_cursor: bool,
policy: ZeroCopyPolicy,
) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::open(anchored, want_hdr && !hdr_capture_failed(), policy)
.map(|c| Box::new(c) as Box<dyn Capturer>)
linux::PortalCapturer::open(
anchored,
want_hdr && !hdr_capture_failed(),
want_metadata_cursor,
policy,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
/// Open the Linux portal capturer bound to an already-created virtual output's PipeWire node. The