fix(host/encode): negotiate the cursor around what the encoder can blend

EncoderCaps::blends_cursor's contract said the HOST must fall back to
capturer-side compositing when a cursor-as-metadata session lands on an
encoder that can't composite — but that host half was never built: open_video
warned and the session streamed WITHOUT a pointer (confirmed on the VAAPI
dmabuf and libav-NVENC CUDA paths; latent on vulkan RGB-direct/native-NV12).
The negotiation is now caps-aware, ahead of capture, on both planes:

* pf-encode grows cursor_blend_capable() — the pre-open dispatch mirror
  (sibling of linux_native_nv12_ok) answering whether the resolved backend
  composites frame.cursor; its pure core is test-pinned arm by arm.
* Native plane: handshake::cursor_forward grants the cursor channel only
  where the resolved backend can blend (the capture-mouse flip makes the host
  draw the pointer on demand); denied sessions keep the pre-channel path —
  the compositor EMBEDS the pointer, never cursorless, never doubled. The
  Welcome's HOST_CAP_CURSOR bit is computed once and read back at both
  session-wiring sites instead of recomputed. SessionPlan::output_format
  additionally keeps every cursor-blend session off producer-native NV12 (the
  arm with no CSC to fold a cursor into), and vulkan RGB-direct now yields to
  a cursor-blend session even when pinned (EFC cannot composite; the open
  logs the override). Windows plans cursor_blend=false via the new shared
  cursor_blend_for() rule — the IDD capturer composites the pointer itself,
  and asking the encoder anyway fired the blends-cursor warn spuriously on
  every cursor-channel session.
* GameStream plane: the hardcoded cursor_blend=true is gone. The portal
  source asks for cursor-as-metadata only when the resolved backend blends,
  otherwise negotiates an Embedded pointer (choose_cursor_mode's new ladder);
  the capturer pool now also keys on that mode. The virtual-output source
  passes false — its capture embeds the pointer where it can.

The per-arm warns in vulkan_video (RGB-direct, native-NV12) are now
structurally unreachable and removed. open_video's post-open check stays as
the single backstop for what planning cannot see: a Vulkan-open falling back
to VAAPI mid-session, and the gamescope residual (no embedded mode exists
there, so a never-blending backend — H.264-on-AMD VAAPI, software — still
streams cursorless; fixing that needs a compositing stage, deliberately not
built in this pass). Zero-copy is preserved throughout — every fallback is a
capture-negotiation change, never a readback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

crates/punktfunk-host/src/native/handshake.rs

@@ -12,31 +12,46 @@ use super::*;
 /// the client asked ([`CLIENT_CAP_CURSOR`](punktfunk_core::quic::CLIENT_CAP_CURSOR)) AND the
 /// capture path can deliver cursor metadata separately from the frame — the Linux portal
 /// `SPA_META_Cursor` path (not gamescope, whose capture paints no cursor at all), or Windows
-/// with a proto-v5 pf-vdisplay driver (the IddCx hardware-cursor channel, M2c). THE single
-/// predicate: the Welcome's `HOST_CAP_CURSOR` bit and the session's forwarding/blend-off
-/// wiring both read it, so they can never disagree.
+/// with a proto-v5 pf-vdisplay driver (the IddCx hardware-cursor channel, M2c) — AND, on
+/// Linux, the encode backend this session resolves to can composite the pointer on demand
+/// (`encode::cursor_blend_capable`): the channel's capture-mouse flip (`CursorRenderMode`,
+/// `client_draws = false`) makes the HOST draw the pointer, and on Linux the encoder is that
+/// compositing stage — granting the channel over a backend that can't blend (libav
+/// VAAPI/NVENC, software) shipped a cursorless stream on every capture-mode flip. Denied, the
+/// session keeps the pre-channel path: the compositor EMBEDS the pointer and the client never
+/// draws — never cursorless, never doubled. THE single predicate: the Welcome's
+/// `HOST_CAP_CURSOR` bit is computed from it, and the session wiring reads that bit back.
 pub(super) fn cursor_forward(
     client_caps: u8,
     compositor: Option<crate::vdisplay::Compositor>,
+    codec: crate::encode::Codec,
+    bit_depth: u8,
 ) -> bool {
     if client_caps & punktfunk_core::quic::CLIENT_CAP_CURSOR == 0 {
         return false;
     }
     #[cfg(target_os = "linux")]
     {
+        // CUDA-payload prediction — the same one `SessionPlan` makes: the NVIDIA resolution
+        // plus the zero-copy master switch. It decides direct-SDK NVENC (blends) vs libav
+        // NVENC (doesn't) inside the capability mirror.
+        let cuda_planned = !crate::encode::linux_zero_copy_is_vaapi() && crate::zerocopy::enabled();
         compositor.is_some_and(|c| c != crate::vdisplay::Compositor::Gamescope)
+            && crate::encode::cursor_blend_capable(codec, cuda_planned, bit_depth == 10)
     }
     #[cfg(target_os = "windows")]
     {
         // Windows (M2c): the pf-vdisplay driver must speak the v5 hardware-cursor channel —
         // DWM composites the pointer into the IDD frame otherwise, and forwarding a second
-        // copy would double it. The probe latches by opening the control device once.
-        let _ = compositor;
+        // copy would double it. The probe latches by opening the control device once. The
+        // encoder is deliberately NOT consulted: the IDD capturer itself composites on the
+        // capture-mouse flip (`set_cursor_forward`), so no Windows encode backend blends.
+        let _ = (compositor, codec, bit_depth);
         crate::vdisplay::manager::hw_cursor_capable()
     }
     #[cfg(not(any(target_os = "linux", target_os = "windows")))]
     {
-        let _ = compositor;
+        let _ = (compositor, codec, bit_depth);
         false
     }
 }
@@ -488,9 +503,10 @@ pub(super) async fn negotiate(
                 0
             }
             // Cursor channel granted (client asked + this capture path can deliver cursor
-            // metadata out of the frame) — the client turns its local renderer on ONLY when
-            // it sees this bit, and serve_session wires forwarding from the same predicate.
-            | if cursor_forward(hello.client_caps, compositor) {
+            // metadata out of the frame + the resolved encoder can composite on the
+            // capture-mouse flip) — the client turns its local renderer on ONLY when it sees
+            // this bit, and serve_session wires forwarding by reading the bit back.
+            | if cursor_forward(hello.client_caps, compositor, codec, bit_depth) {
                 punktfunk_core::quic::HOST_CAP_CURSOR
             } else {
                 0
@@ -536,9 +552,9 @@ pub(super) async fn negotiate(
             let (ctx_tx, ctx_rx) = std::sync::mpsc::sync_channel::<SessionContext>(1);
             let client_identity = endpoint::peer_fingerprint(conn);
             let client_hdr = hello.display_hdr;
-            // Same predicate the Welcome's HOST_CAP_CURSOR bit used — the prepared display and
-            // the session wiring must agree with what we just advertised.
-            let cursor_fw = cursor_forward(hello.client_caps, Some(comp));
+            // The bit the Welcome just advertised — read back rather than recomputed, so the
+            // prepared display and the session wiring cannot disagree with it.
+            let cursor_fw = welcome.host_caps & punktfunk_core::quic::HOST_CAP_CURSOR != 0;
             let (mode, shard_payload) = (hello.mode, welcome.shard_payload);
             let trace = bringup.clone();
             std::thread::Builder::new()

crates/punktfunk-host/src/native/stream.rs

@@ -1022,8 +1022,13 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
         // pointer compositor-EMBEDDED (`vd.set_hw_cursor(false)` → no cursor metadata, nothing to
         // blend), keeping the zero-cost pre-channel path. gamescope is the exception (Phase C):
         // it can't embed the pointer, so the host ALWAYS composites the XFixes-sourced cursor —
-        // the blend must be built for every gamescope session.
-        ctx.compositor == pf_vdisplay::Compositor::Gamescope || ctx.cursor_forward,
+        // the blend must be built for every gamescope session. (`cursor_forward` is already
+        // blend-gated: `handshake::cursor_forward` grants the channel only where
+        // `encode::cursor_blend_capable` says the resolved backend composites.)
+        crate::session_plan::cursor_blend_for(
+            ctx.cursor_forward,
+            ctx.compositor == pf_vdisplay::Compositor::Gamescope,
+        ),
         ctx.cursor_forward,
     );
     // gamescope: the XFixes cursor source feeds the always-on composite (Phase C). Set after
@@ -2098,8 +2103,10 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
                                         // capture-mode channel); a switch AWAY restores the prior
                                         // gating. `plan` is `Copy` — this is the value the rebuild
                                         // (and its `build_pipeline` attach) reads.
-                                        plan.cursor_blend = plan.cursor_forward
-                                            || c == crate::vdisplay::Compositor::Gamescope;
+                                        plan.cursor_blend = crate::session_plan::cursor_blend_for(
+                                            plan.cursor_forward,
+                                            c == crate::vdisplay::Compositor::Gamescope,
+                                        );
                                         plan.gamescope_cursor =
                                             c == crate::vdisplay::Compositor::Gamescope;
                                         gamescope_composite =
@@ -2987,7 +2994,10 @@ pub(super) fn prepare_display(
         // non-gamescope sessions get the pointer compositor-EMBEDDED, nothing to blend; the
         // mid-stream `CursorRenderMode` flip strips/keeps `frame.cursor` per tick for channel
         // sessions). gamescope (Phase C) can't embed → always composites the XFixes cursor.
-        compositor == pf_vdisplay::Compositor::Gamescope || cursor_forward,
+        crate::session_plan::cursor_blend_for(
+            cursor_forward,
+            compositor == pf_vdisplay::Compositor::Gamescope,
+        ),
         cursor_forward,
     );
     plan.gamescope_cursor = compositor == pf_vdisplay::Compositor::Gamescope;