Merge origin/main into the pf-capture sweep
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m10s
ci / bench (push) Successful in 5m43s
windows-host / package (push) Successful in 10m48s
ci / rust-arm64 (push) Successful in 13m38s
decky / build-publish (push) Successful in 34s
deb / build-publish (push) Successful in 12m32s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 19s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 40s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 15s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 21s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 15s
android / android (push) Successful in 17m7s
deb / build-publish-host (push) Successful in 9m49s
arch / build-publish (push) Successful in 17m59s
ci / rust (push) Successful in 21m32s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m51s
docker / build-push-arm64cross (push) Successful in 9s
docker / deploy-docs (push) Successful in 26s
deb / build-publish-client-arm64 (push) Successful in 9m55s
apple / swift (push) Successful in 6m10s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 23m10s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 23m24s
apple / screenshots (push) Successful in 24m29s

Upstream landed `fc335b39` (negotiate the cursor around what the encoder can blend) on the same
files this sweep restructured. Merged rather than rebased: the sweep MOVED ~2,000 lines out of
`linux/mod.rs` into `pipewire.rs`/`portal.rs`/`pw_*.rs`, so a rebase would have re-fought the same
conflict in up to nine commits; merging resolves it once with both sides in view. The repo already
carries merge commits (`land/sweep-all`).

Two conflicts, both resolved by keeping BOTH changes:

* `linux/mod.rs` — `PortalCapturer::open` gains upstream's `want_metadata_cursor` AND keeps the
  sweep's `PortalSession` teardown, so the portal threads now take `(setup_tx, quit_rx,
  want_metadata_cursor)`. The second conflict was upstream editing inside the region Phase 5.1/5.3
  moved out; the split wins and upstream's change is ported to where the code now lives:
  `choose_cursor_mode`'s new `want_metadata` ladder (4 arms — prefer Embedded when the encoder can't
  blend, and warn-then-take Metadata when Embedded isn't advertised) is now in `portal.rs`, taken
  verbatim.
* `gamestream/stream.rs` — the pooled-capturer slot keeps upstream's third reuse key
  (`metadata_cursor`, beside HDR-ness) AND the sweep's `result.is_ok() && capturer.is_alive()`
  re-pool gate. Both guard the same slot against different failures: theirs against reusing a
  session negotiated for the wrong cursor mode, the sweep's (L2) against reusing a dead one.

Everything else auto-merged, including the `want_metadata_cursor` thread through
`host/capture.rs`'s facade and `native/stream.rs`'s `session_plan::cursor_blend_for`.

Verified after the merge: workspace `cargo test` green, `clippy --workspace --all-targets` clean on
Linux AND on x86_64-pc-windows-msvc, `cargo fmt --check --all` clean, pf-capture 38/38.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-26 11:44:22 +02:00
co-authored by Claude Opus 5
19 changed files with 1172 additions and 308 deletions
+13 -3
View File
@@ -515,15 +515,25 @@ pub mod synthetic_nv12;
/// `want_hdr` offers the GNOME 50+ HDR formats (10-bit PQ/BT.2020 dmabufs) instead of the SDR /// `want_hdr` offers the GNOME 50+ HDR formats (10-bit PQ/BT.2020 dmabufs) instead of the SDR
/// set — pass it only when the mirrored monitor is actually in HDR mode (the host probes /// set — pass it only when the mirrored monitor is actually in HDR mode (the host probes
/// DisplayConfig) or the negotiation runs into its 10 s timeout and latches the SDR downgrade. /// DisplayConfig) or the negotiation runs into its 10 s timeout and latches the SDR downgrade.
/// The [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts (the one-way edge). /// `want_metadata_cursor` asks for cursor-as-metadata (`SPA_META_Cursor`) — pass it only when
/// the session's encode path composites `CapturedFrame::cursor` (the host consults
/// `pf-encode`'s `cursor_blend_capable`); otherwise the portal EMBEDS the pointer so it is
/// never silently lost. The [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts
/// (the one-way edge).
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub fn open_portal_monitor( pub fn open_portal_monitor(
anchored: bool, anchored: bool,
want_hdr: bool, want_hdr: bool,
want_metadata_cursor: bool,
policy: ZeroCopyPolicy, policy: ZeroCopyPolicy,
) -> Result<Box<dyn Capturer>> { ) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::open(anchored, want_hdr && !hdr_capture_failed(), policy) linux::PortalCapturer::open(
.map(|c| Box::new(c) as Box<dyn Capturer>) anchored,
want_hdr && !hdr_capture_failed(),
want_metadata_cursor,
policy,
)
.map(|c| Box::new(c) as Box<dyn Capturer>)
} }
/// Open the Linux portal capturer bound to an already-created virtual output's PipeWire node. The /// Open the Linux portal capturer bound to an already-created virtual output's PipeWire node. The
+11 -3
View File
@@ -233,7 +233,15 @@ impl PortalCapturer {
/// RemoteDesktop grant and never raises a separate ScreenCast dialog; `false` uses a plain /// RemoteDesktop grant and never raises a separate ScreenCast dialog; `false` uses a plain
/// ScreenCast session (wlroots, which has no RemoteDesktop portal). `want_hdr` offers the /// ScreenCast session (wlroots, which has no RemoteDesktop portal). `want_hdr` offers the
/// GNOME 50+ HDR formats (10-bit PQ/BT.2020, dmabuf-only) instead of the SDR set. /// GNOME 50+ HDR formats (10-bit PQ/BT.2020, dmabuf-only) instead of the SDR set.
pub fn open(anchored: bool, want_hdr: bool, policy: ZeroCopyPolicy) -> Result<PortalCapturer> { /// `want_metadata_cursor` picks the cursor mode — `true` asks for `SPA_META_Cursor` (this
/// session's encode path composites it), `false` asks the compositor to EMBED the pointer; see
/// `portal::choose_cursor_mode`.
pub fn open(
anchored: bool,
want_hdr: bool,
want_metadata_cursor: bool,
policy: ZeroCopyPolicy,
) -> Result<PortalCapturer> {
// Portal handshake (async) on its own thread; hands back the PW fd + node id. // Portal handshake (async) on its own thread; hands back the PW fd + node id.
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>(); let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>();
// Teardown plumbing (see `PortalSession`): `quit_rx` parks the thread once the handshake is // Teardown plumbing (see `PortalSession`): `quit_rx` parks the thread once the handshake is
@@ -244,9 +252,9 @@ impl PortalCapturer {
.name("punktfunk-portal".into()) .name("punktfunk-portal".into())
.spawn(move || { .spawn(move || {
if anchored { if anchored {
portal_thread_remote_desktop(setup_tx, quit_rx) portal_thread_remote_desktop(setup_tx, quit_rx, want_metadata_cursor)
} else { } else {
portal_thread(setup_tx, quit_rx) portal_thread(setup_tx, quit_rx, want_metadata_cursor)
} }
// After the runtime has been dropped inside the fn above, so a successful // After the runtime has been dropped inside the fn above, so a successful
// `recv_timeout` in `Drop` means the zbus connection is really gone. Covers the // `recv_timeout` in `Drop` means the zbus connection is really gone. Covers the
+38 -14
View File
@@ -91,19 +91,23 @@ pub fn gnome_hdr_monitor_active() -> bool {
} }
} }
/// Pick the ScreenCast cursor mode from what the backend advertises (`AvailableCursorModes`), /// Pick the ScreenCast cursor mode from what the backend advertises (`AvailableCursorModes`).
/// preferring **cursor-as-metadata**: the compositor keeps its cheap hardware cursor plane and /// With `want_metadata` the ladder prefers **cursor-as-metadata**: the compositor keeps its cheap
/// ships the pointer as PipeWire `SPA_META_Cursor` metadata (position + an occasional bitmap), /// hardware cursor plane and ships the pointer as PipeWire `SPA_META_Cursor` metadata (position +
/// which the consumer composites itself. That avoids forcing the producer to burn the cursor into /// an occasional bitmap), which the consumer composites itself — avoiding the producer burning the
/// every frame — the `Embedded` mode — which on gamescope would defeat its HW cursor plane. Falls /// cursor into every frame (`Embedded`), which on gamescope would defeat its HW cursor plane.
/// back to `Embedded`, then `Hidden`, and (if the property query fails, e.g. an older portal) /// Without it — the session's encode path has no compositing stage for a metadata cursor
/// keeps the prior `Embedded` behavior so the cursor is never silently lost. /// (`pf-encode`'s `cursor_blend_capable` said the resolved backend can't blend) — the ladder
/// prefers `Embedded`, so the pointer is in the pixels instead of in metadata nothing would draw.
/// Both ladders fall through to the other mode, then `Hidden`; a failed property query (an older
/// portal) keeps the prior `Embedded` behavior so the cursor is never silently lost.
async fn choose_cursor_mode( async fn choose_cursor_mode(
proxy: &ashpd::desktop::screencast::Screencast, proxy: &ashpd::desktop::screencast::Screencast,
want_metadata: bool,
) -> ashpd::desktop::screencast::CursorMode { ) -> ashpd::desktop::screencast::CursorMode {
use ashpd::desktop::screencast::CursorMode; use ashpd::desktop::screencast::CursorMode;
match proxy.available_cursor_modes().await { match proxy.available_cursor_modes().await {
Ok(avail) if avail.contains(CursorMode::Metadata) => { Ok(avail) if want_metadata && avail.contains(CursorMode::Metadata) => {
tracing::info!( tracing::info!(
?avail, ?avail,
"ScreenCast: requesting cursor-as-metadata (SPA_META_Cursor)" "ScreenCast: requesting cursor-as-metadata (SPA_META_Cursor)"
@@ -111,12 +115,30 @@ async fn choose_cursor_mode(
CursorMode::Metadata CursorMode::Metadata
} }
Ok(avail) if avail.contains(CursorMode::Embedded) => { Ok(avail) if avail.contains(CursorMode::Embedded) => {
tracing::info!( if want_metadata {
?avail, tracing::info!(
"ScreenCast: cursor metadata unavailable — requesting Embedded cursor" ?avail,
); "ScreenCast: cursor metadata unavailable — requesting Embedded cursor"
);
} else {
tracing::info!(
?avail,
"ScreenCast: requesting Embedded cursor (this session's encoder does not \
composite a metadata cursor)"
);
}
CursorMode::Embedded CursorMode::Embedded
} }
Ok(avail) if avail.contains(CursorMode::Metadata) => {
// Embedded wanted but not offered. Metadata still beats Hidden: the CPU capture
// path composites `SPA_META_Cursor` inline, so part of the matrix keeps a pointer.
tracing::warn!(
?avail,
"ScreenCast: Embedded cursor not advertised — requesting cursor-as-metadata \
(only CPU-path frames will composite it)"
);
CursorMode::Metadata
}
Ok(avail) => { Ok(avail) => {
tracing::warn!( tracing::warn!(
?avail, ?avail,
@@ -140,6 +162,7 @@ async fn choose_cursor_mode(
pub(super) fn portal_thread( pub(super) fn portal_thread(
setup_tx: std::sync::mpsc::Sender<Result<(OwnedFd, u32), String>>, setup_tx: std::sync::mpsc::Sender<Result<(OwnedFd, u32), String>>,
quit_rx: tokio::sync::oneshot::Receiver<()>, quit_rx: tokio::sync::oneshot::Receiver<()>,
want_metadata_cursor: bool,
) { ) {
use ashpd::desktop::screencast::{Screencast, SelectSourcesOptions, SourceType}; use ashpd::desktop::screencast::{Screencast, SelectSourcesOptions, SourceType};
use ashpd::desktop::PersistMode; use ashpd::desktop::PersistMode;
@@ -170,7 +193,7 @@ pub(super) fn portal_thread(
.create_session(Default::default()) .create_session(Default::default())
.await .await
.context("create_session")?; .context("create_session")?;
let cursor_mode = choose_cursor_mode(&proxy).await; let cursor_mode = choose_cursor_mode(&proxy, want_metadata_cursor).await;
proxy proxy
.select_sources( .select_sources(
&session, &session,
@@ -236,6 +259,7 @@ pub(super) fn portal_thread(
pub(super) fn portal_thread_remote_desktop( pub(super) fn portal_thread_remote_desktop(
setup_tx: std::sync::mpsc::Sender<Result<(OwnedFd, u32), String>>, setup_tx: std::sync::mpsc::Sender<Result<(OwnedFd, u32), String>>,
quit_rx: tokio::sync::oneshot::Receiver<()>, quit_rx: tokio::sync::oneshot::Receiver<()>,
want_metadata_cursor: bool,
) { ) {
use ashpd::desktop::remote_desktop::{DeviceType, RemoteDesktop, SelectDevicesOptions}; use ashpd::desktop::remote_desktop::{DeviceType, RemoteDesktop, SelectDevicesOptions};
use ashpd::desktop::screencast::{Screencast, SelectSourcesOptions, SourceType}; use ashpd::desktop::screencast::{Screencast, SelectSourcesOptions, SourceType};
@@ -281,7 +305,7 @@ pub(super) fn portal_thread_remote_desktop(
.context("select_devices")? .context("select_devices")?
.response() .response()
.context("select_devices rejected")?; .context("select_devices rejected")?;
let cursor_mode = choose_cursor_mode(&screencast).await; let cursor_mode = choose_cursor_mode(&screencast, want_metadata_cursor).await;
screencast screencast
.select_sources( .select_sources(
&session, &session,
+25 -18
View File
@@ -249,9 +249,16 @@ impl Codec {
} }
} }
/// Static capabilities an [`Encoder`] declares so the session glue routes loss-recovery and HDR /// Static capabilities an [`Encoder`] declares so the session glue routes loss-recovery and
/// plumbing by *query* rather than relying on a method's no-op/`false` default. Cheap `Copy`; fixed /// cursor plumbing by *query* rather than relying on a method's no-op/`false` default. Cheap
/// for the session (an HDR toggle re-initialises the encoder — re-query if that matters). /// `Copy`; fixed for the session (an HDR toggle re-initialises the encoder — re-query if that
/// matters).
///
/// (There is deliberately NO `supports_hdr_metadata` cap: in-band HDR SEI/OBU embedding needs no
/// host-side routing — every first-party client reads the static grade exclusively out-of-band
/// (the native 0xCE datagram / the GameStream 0x010e control message), both planes send it
/// unconditionally, and the in-band grade is a decoder-side bonus for stock clients. A cap field
/// nothing reads is a contract nobody honors; it was deleted after shipping write-only.)
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct EncoderCaps { pub struct EncoderCaps {
/// The encoder can perform real reference-frame invalidation — i.e. /// The encoder can perform real reference-frame invalidation — i.e.
@@ -261,10 +268,6 @@ pub struct EncoderCaps {
/// AMF (user-LTR force-reference, when the driver accepted the LTR slots at open). The /// AMF (user-LTR force-reference, when the driver accepted the LTR slots at open). The
/// libavcodec paths (Linux NVENC, VAAPI, QSV) can't express it and always keyframe. /// libavcodec paths (Linux NVENC, VAAPI, QSV) can't express it and always keyframe.
pub supports_rfi: bool, pub supports_rfi: bool,
/// The encoder emits in-band HDR mastering/CLL SEI from [`set_hdr_meta`](Encoder::set_hdr_meta).
/// When `false`, `set_hdr_meta` is a no-op and no in-band grade reaches the client. Only the
/// Windows direct-NVENC path attaches it today.
pub supports_hdr_metadata: bool,
/// The opened encoder is actually producing a full-chroma 4:4:4 (`chroma_format_idc = 3`) stream. /// The opened encoder is actually producing a full-chroma 4:4:4 (`chroma_format_idc = 3`) stream.
/// `false` on every 4:2:0 session (the default) and on a backend that declined 4:4:4. Set by the /// `false` on every 4:2:0 session (the default) and on a backend that declined 4:4:4. Set by the
/// NVENC backends (Linux + Windows). The chroma is committed to the wire (`Welcome::chroma_format`) /// NVENC backends (Linux + Windows). The chroma is committed to the wire (`Welcome::chroma_format`)
@@ -304,8 +307,11 @@ pub struct EncoderCaps {
/// ///
/// This makes the answer queryable instead of assumed. It is deliberately a plain fact about the /// This makes the answer queryable instead of assumed. It is deliberately a plain fact about the
/// encoder, not a policy: what to DO when a session wants blending and the backend cannot is the /// encoder, not a policy: what to DO when a session wants blending and the backend cannot is the
/// host's call, since only the host can re-plan capture (fall back to capturer-side compositing). /// host's call, since only the host can re-plan capture. That call is wired now — the
/// `open_video` can only warn, which it does. /// negotiation consults the pre-open mirror ([`cursor_blend_capable`](crate::cursor_blend_capable))
/// to gate the cursor channel and to keep capture on embedded-cursor / CSC-capable shapes for
/// any backend that can't blend; `open_video`'s post-open check remains as the backstop for
/// open-time fallbacks the plan can't see.
pub blends_cursor: bool, pub blends_cursor: bool,
} }
@@ -333,11 +339,10 @@ pub trait Encoder: Send {
let _ = wire_index; let _ = wire_index;
self.submit(frame) self.submit(frame)
} }
/// This encoder's static [capabilities](EncoderCaps) (RFI, HDR SEI), so the session glue can /// This encoder's static [capabilities](EncoderCaps) (RFI, intra-refresh, chroma, cursor
/// route by query rather than rely on the no-op/`false` defaults of /// blending), so the session glue can route by query rather than rely on the no-op/`false`
/// [`invalidate_ref_frames`](Self::invalidate_ref_frames) / [`set_hdr_meta`](Self::set_hdr_meta). /// defaults of methods like [`invalidate_ref_frames`](Self::invalidate_ref_frames).
/// Default: no optional capabilities (the SDR / libavcodec backends) — only the direct-NVENC /// Default: no optional capabilities (the software / libavcodec backends).
/// path overrides it.
fn caps(&self) -> EncoderCaps { fn caps(&self) -> EncoderCaps {
EncoderCaps::default() EncoderCaps::default()
} }
@@ -345,10 +350,12 @@ pub trait Encoder: Send {
/// reference-frame-invalidation request). Default: no-op. /// reference-frame-invalidation request). Default: no-op.
fn request_keyframe(&mut self) {} fn request_keyframe(&mut self) {}
/// Set the source's static HDR mastering metadata (from the capturer). An HDR encoder emits it /// Set the source's static HDR mastering metadata (from the capturer). An HDR encoder emits it
/// as in-band SEI (`mastering_display_colour_volume` + `content_light_level_info`) on each /// in-band (HEVC/H.264 `mastering_display_colour_volume` + `content_light_level_info` SEI, or
/// keyframe so any decoder — including stock Moonlight — tone-maps from the source's real grade. /// AV1 metadata OBUs) on keyframes so a stock decoder — e.g. stock Moonlight — tone-maps from
/// Default: no-op (SDR encoders / libavcodec paths that don't attach it yet). Cheap to call /// the source's real grade. Default: no-op (SDR encoders / paths that don't attach it).
/// every frame; only the direct-NVENC path consumes it. /// Cheap to call every frame; consumed by Windows direct-NVENC, native AMF, and native QSV.
/// Every first-party client reads the grade out-of-band (the 0xCE datagram) regardless, so
/// this is a bonus for stock decoders, never the primary channel.
fn set_hdr_meta(&mut self, _meta: Option<punktfunk_core::quic::HdrMeta>) {} fn set_hdr_meta(&mut self, _meta: Option<punktfunk_core::quic::HdrMeta>) {}
/// Invalidate a contiguous range of previously-encoded reference frames (client frame numbers /// Invalidate a contiguous range of previously-encoded reference frames (client frame numbers
/// — WIRE frame indexes, the domain [`submit_indexed`](Self::submit_indexed) pins the encoder's /// — WIRE frame indexes, the domain [`submit_indexed`](Self::submit_indexed) pins the encoder's
@@ -1850,7 +1850,6 @@ impl Encoder for NvencCudaEncoder {
// Composites `frame.cursor` via the SPIR-V blend over the Vulkan-allocated input slot. // Composites `frame.cursor` via the SPIR-V blend over the Vulkan-allocated input slot.
blends_cursor: true, blends_cursor: true,
supports_rfi: self.rfi_supported, supports_rfi: self.rfi_supported,
supports_hdr_metadata: self.hdr,
chroma_444: self.chroma_444, chroma_444: self.chroma_444,
intra_refresh: false, intra_refresh: false,
intra_refresh_recovery: false, intra_refresh_recovery: false,
+412 -71
View File
@@ -94,23 +94,152 @@ type LpKey = (String, &'static str, bool);
/// The [`LP_MODE`] key for this device/codec/depth. `render_node()` is re-read rather than cached /// The [`LP_MODE`] key for this device/codec/depth. `render_node()` is re-read rather than cached
/// so a GPU-preference change is picked up on the next open. /// so a GPU-preference change is picked up on the next open.
fn lp_key(codec: Codec, ten_bit: bool) -> LpKey { fn lp_key(codec: Codec, ten_bit: bool) -> LpKey {
( lp_key_for(&render_node().to_string_lossy(), codec, ten_bit)
render_node().to_string_lossy().into_owned(), }
codec.label(),
ten_bit, /// [`lp_key`] with the render node explicit (device-free for the unit tests). Every part of the
) /// key is load-bearing — see [`LP_MODE`] for the two field-motivated bugs (node: cross-GPU
/// session-killer; depth: HDR under-advertisement).
fn lp_key_for(node: &str, codec: Codec, ten_bit: bool) -> LpKey {
(node.to_owned(), codec.label(), ten_bit)
} }
/// `PUNKTFUNK_VAAPI_LOW_POWER` pins the entrypoint mode (`1` = low-power only, `0` = full-feature /// `PUNKTFUNK_VAAPI_LOW_POWER` pins the entrypoint mode (`1` = low-power only, `0` = full-feature
/// only); unset → try full-feature first, fall back to low-power. /// only); unset → try full-feature first, fall back to low-power.
fn low_power_override() -> Option<bool> { fn low_power_override() -> Option<bool> {
match std::env::var("PUNKTFUNK_VAAPI_LOW_POWER").ok()?.trim() { parse_low_power(&std::env::var("PUNKTFUNK_VAAPI_LOW_POWER").ok()?)
}
/// [`low_power_override`]'s value grammar (device-free for the unit tests). Anything outside the
/// two literal sets means "no pin" — the `[full-feature, low-power]` ladder runs.
fn parse_low_power(raw: &str) -> Option<bool> {
match raw.trim() {
"1" | "true" | "yes" | "on" => Some(true), "1" | "true" | "yes" | "on" => Some(true),
"0" | "false" | "no" | "off" => Some(false), "0" | "false" | "no" | "off" => Some(false),
_ => None, _ => None,
} }
} }
/// The entrypoint modes to attempt, in order (`false` = full-feature `EncSlice`, `true` =
/// low-power VDEnc): an operator pin tries exactly that one; a cached resolution ([`LP_MODE`]:
/// 1 = full-feature, 2 = low-power) skips the known-failing attempt; anything else runs the full
/// ladder — full-feature first so AMD's first-try open stays byte-for-byte unchanged. Never
/// empty: [`open_vaapi_encoder`] relies on at least one attempt running.
fn entrypoint_ladder(pin: Option<bool>, cached: u8) -> &'static [bool] {
match pin {
Some(true) => &[true],
Some(false) => &[false],
None => match cached {
1 => &[false],
2 => &[true],
_ => &[false, true],
},
}
}
/// The [`LP_MODE`] value a successful open latches (1 = full-feature, 2 = low-power). Paired with
/// [`entrypoint_ladder`], which maps it back to the single-mode retry list.
fn latched_mode(low_power: bool) -> u8 {
if low_power {
2
} else {
1
}
}
/// The VUI colour metadata the encoder signals for the session's depth.
struct Vui {
colorspace: ffi::AVColorSpace,
range: ffi::AVColorRange,
primaries: ffi::AVColorPrimaries,
trc: ffi::AVColorTransferCharacteristic,
}
/// 10-bit HDR: BT.2020 primaries + SMPTE-2084 (PQ) transfer, limited range — matches the P010 the
/// CSC produces (swscale BT.2020 on the CPU path; `scale_vaapi` pinned to bt2020 on the zero-copy
/// path); the client decoder auto-detects PQ from the VUI. SDR: we hand the encoder BT.709
/// *limited* NV12 (swscale CSC on the CPU path; `scale_vaapi` pinned to
/// `out_color_matrix=bt709:out_range=limited` on the zero-copy path, with the full-range RGB input
/// tagged), so signal that VUI — else the client decoder washes the picture out.
fn vui_for(ten_bit: bool) -> Vui {
if ten_bit {
Vui {
colorspace: ffi::AVColorSpace::AVCOL_SPC_BT2020_NCL,
range: ffi::AVColorRange::AVCOL_RANGE_MPEG,
primaries: ffi::AVColorPrimaries::AVCOL_PRI_BT2020,
trc: ffi::AVColorTransferCharacteristic::AVCOL_TRC_SMPTE2084,
}
} else {
Vui {
colorspace: ffi::AVColorSpace::AVCOL_SPC_BT709,
range: ffi::AVColorRange::AVCOL_RANGE_MPEG,
primaries: ffi::AVColorPrimaries::AVCOL_PRI_BT709,
trc: ffi::AVColorTransferCharacteristic::AVCOL_TRC_BT709,
}
}
}
/// The explicit `profile` option for this codec/depth, or `None` to let the encoder derive it.
/// HEVC Main10 is pinned explicitly so the depth is never silently dropped (`hevc_vaapi` would
/// derive it from the P010 surfaces, but a derivation can regress quietly); 10-bit AV1 is
/// input-driven — no profile knob — and every 8-bit open keeps the encoder default.
fn explicit_profile(codec: Codec, ten_bit: bool) -> Option<&'static str> {
(ten_bit && codec == Codec::H265).then_some("main10")
}
/// `PUNKTFUNK_VAAPI_ASYNC_DEPTH` grammar: 1..=8 accepted verbatim; unset, junk, and out-of-range
/// values all resolve to depth 1 — the lowest-latency structure (see the `async_depth` note at the
/// call site for why 1 is the default and what depth ≥ 2 trades).
fn async_depth(raw: Option<&str>) -> u32 {
raw.and_then(|s| s.parse::<u32>().ok())
.filter(|d| (1..=8).contains(d))
.unwrap_or(1)
}
/// The `scale_vaapi` filter args for the session's depth. The VPP's output colour is pinned to
/// exactly what the encoder's VUI signals ([`vui_for`]) — without the explicit options the
/// conversion matrix is whatever the driver defaults to for an unspecified output (Mesa: BT.601),
/// a hue shift against the signaled VUI. (The PQ transfer is per-channel and rides through the
/// matrix untouched.)
fn scale_vaapi_args(ten_bit: bool) -> &'static CStr {
if ten_bit {
c"format=p010:out_color_matrix=bt2020:out_range=limited"
} else {
c"format=nv12:out_color_matrix=bt709:out_range=limited"
}
}
/// What a (captured format, negotiated bit depth) pair resolves to at open. 10-bit rides on the
/// captured PIXELS, not the negotiated depth — see [`crate::ten_bit_input`] for the failure the
/// reverse shape produces.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum DepthResolution {
/// PQ-graded 10-bit frames on a non-10-bit session: refuse the open, so PQ content is never
/// mislabeled BT.709.
RefuseMislabeledPq,
/// 10-bit negotiated but the capture stayed SDR: honestly encode 8-bit (with a warning).
SdrDowngrade,
/// Format and negotiated depth agree.
Agreed,
}
/// See [`DepthResolution`].
fn resolve_depth(format: PixelFormat, bit_depth: u8) -> DepthResolution {
if format.is_hdr_rgb10() && bit_depth != 10 {
DepthResolution::RefuseMislabeledPq
} else if bit_depth == 10 && !format.is_hdr_rgb10() {
DepthResolution::SdrDowngrade
} else {
DepthResolution::Agreed
}
}
/// Whether a codec is even eligible for the 10-bit probe: PyroWave answers 10-bit support on its
/// own path (no VAAPI involved), and a codec without a 10-bit profile can never pass.
fn ten_bit_probe_eligible(codec: Codec) -> bool {
codec.supports_10bit() && codec != Codec::PyroWave
}
/// Open the VAAPI encoder, resolving the entrypoint mode: try the full-feature entrypoint first /// Open the VAAPI encoder, resolving the entrypoint mode: try the full-feature entrypoint first
/// and, if the driver rejects it, retry with `low_power=1` — modern Intel (Gen12+/Arc) exposes /// and, if the driver rejects it, retry with `low_power=1` — modern Intel (Gen12+/Arc) exposes
/// ONLY the low-power VDEnc entrypoint (ffmpeg's `vaapi_encode` defaults `low_power=0` and errors /// ONLY the low-power VDEnc entrypoint (ffmpeg's `vaapi_encode` defaults `low_power=0` and errors
@@ -135,15 +264,7 @@ unsafe fn open_vaapi_encoder(
.lock() .lock()
.map(|m| m.get(&key).copied().unwrap_or(0)) .map(|m| m.get(&key).copied().unwrap_or(0))
.unwrap_or(0); .unwrap_or(0);
let modes: &[bool] = match low_power_override() { let modes: &[bool] = entrypoint_ladder(low_power_override(), cached);
Some(true) => &[true],
Some(false) => &[false],
None => match cached {
1 => &[false],
2 => &[true],
_ => &[false, true],
},
};
let mut first_err = None; let mut first_err = None;
for &lp in modes { for &lp in modes {
match open_vaapi_encoder_mode( match open_vaapi_encoder_mode(
@@ -159,7 +280,7 @@ unsafe fn open_vaapi_encoder(
) { ) {
Ok(enc) => { Ok(enc) => {
if let Ok(mut m) = LP_MODE.get_or_init(|| Mutex::new(HashMap::new())).lock() { if let Ok(mut m) = LP_MODE.get_or_init(|| Mutex::new(HashMap::new())).lock() {
m.insert(key.clone(), if lp { 2 } else { 1 }); m.insert(key.clone(), latched_mode(lp));
} }
if lp { if lp {
tracing::info!( tracing::info!(
@@ -216,32 +337,18 @@ unsafe fn open_vaapi_encoder_mode(
apply_low_latency_rc(&mut video, fps, bitrate_bps); apply_low_latency_rc(&mut video, fps, bitrate_bps);
let raw = video.as_mut_ptr(); let raw = video.as_mut_ptr();
(*raw).gop_size = i32::MAX; // no periodic IDR (forced-IDR via pict_type=I on RFI) (*raw).gop_size = i32::MAX; // no periodic IDR (forced-IDR via pict_type=I on RFI)
if ten_bit { let vui = vui_for(ten_bit);
// HDR10: BT.2020 primaries + SMPTE-2084 (PQ) transfer, limited range — matches the P010 (*raw).colorspace = vui.colorspace;
// the CSC produces (swscale BT.2020 on the CPU path; scale_vaapi pinned to bt2020 on the (*raw).color_range = vui.range;
// zero-copy path). The client decoder auto-detects PQ from the VUI. (*raw).color_primaries = vui.primaries;
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT2020_NCL; (*raw).color_trc = vui.trc;
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT2020;
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_SMPTE2084;
} else {
// We hand the encoder BT.709 *limited* NV12 (swscale CSC on the CPU path; scale_vaapi pinned
// to `out_color_matrix=bt709:out_range=limited` on the zero-copy path, with the full-range
// RGB input tagged), so signal that VUI — else the client decoder washes the picture out.
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT709;
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT709;
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_BT709;
}
(*raw).pix_fmt = ffi::AVPixelFormat::AV_PIX_FMT_VAAPI; (*raw).pix_fmt = ffi::AVPixelFormat::AV_PIX_FMT_VAAPI;
(*raw).hw_device_ctx = ffi::av_buffer_ref(device_ref); (*raw).hw_device_ctx = ffi::av_buffer_ref(device_ref);
(*raw).hw_frames_ctx = ffi::av_buffer_ref(frames_ref); (*raw).hw_frames_ctx = ffi::av_buffer_ref(frames_ref);
let mut opts = Dictionary::new(); let mut opts = Dictionary::new();
if ten_bit && codec == Codec::H265 { if let Some(profile) = explicit_profile(codec, ten_bit) {
// HEVC Main10. `hevc_vaapi` derives it from the P010 surfaces, but pin it explicitly so opts.set("profile", profile);
// the depth is never silently dropped. (10-bit AV1 is input-driven — no profile knob.)
opts.set("profile", "main10");
} }
// async_depth=1: `send_frame` blocks until THIS frame's ASIC encode completes — the lowest // async_depth=1: `send_frame` blocks until THIS frame's ASIC encode completes — the lowest
// latency structure libavcodec's vaapi_encode offers. Measured on the 780M at 1440p60: depth 1 // latency structure libavcodec's vaapi_encode offers. Measured on the 780M at 1440p60: depth 1
@@ -251,11 +358,7 @@ unsafe fn open_vaapi_encoder_mode(
// where depth 2 restores throughput at that one-frame cost. NOTE: the per-frame block tracks // where depth 2 restores throughput at that one-frame cost. NOTE: the per-frame block tracks
// GPU CLOCKS — a paced 60 fps trickle lets the VCN downclock (~8 ms/frame vs ~4.4 ms hot); // GPU CLOCKS — a paced 60 fps trickle lets the VCN downclock (~8 ms/frame vs ~4.4 ms hot);
// see `gpuclocks` for the session clock pin that removes the ramp tax. // see `gpuclocks` for the session clock pin that removes the ramp tax.
let depth = std::env::var("PUNKTFUNK_VAAPI_ASYNC_DEPTH") let depth = async_depth(std::env::var("PUNKTFUNK_VAAPI_ASYNC_DEPTH").ok().as_deref());
.ok()
.and_then(|s| s.parse::<u32>().ok())
.filter(|d| (1..=8).contains(d))
.unwrap_or(1);
opts.set("async_depth", &depth.to_string()); opts.set("async_depth", &depth.to_string());
if low_power { if low_power {
opts.set("low_power", "1"); // VDEnc — the only encode entrypoint on modern Intel opts.set("low_power", "1"); // VDEnc — the only encode entrypoint on modern Intel
@@ -309,7 +412,7 @@ pub fn probe_can_encode(codec: Codec) -> bool {
/// ([`crate::can_encode_10bit`]), so a non-Main10 GPU resolves every session to 8-bit SDR before /// ([`crate::can_encode_10bit`]), so a non-Main10 GPU resolves every session to 8-bit SDR before
/// the Welcome (honest downgrade). /// the Welcome (honest downgrade).
pub fn probe_can_encode_10bit(codec: Codec) -> bool { pub fn probe_can_encode_10bit(codec: Codec) -> bool {
if !codec.supports_10bit() || codec == Codec::PyroWave { if !ten_bit_probe_eligible(codec) {
return false; return false;
} }
if ffmpeg::init().is_err() { if ffmpeg::init().is_err() {
@@ -834,24 +937,7 @@ impl DmabufInner {
} }
init!(src, ptr::null(), "buffer"); init!(src, ptr::null(), "buffer");
init!(hwmap, c"mode=read".as_ptr(), "hwmap"); init!(hwmap, c"mode=read".as_ptr(), "hwmap");
// Pin the VPP's output colour to what the encoder's VUI signals (BT.709 limited SDR, init!(scale, scale_vaapi_args(ten_bit).as_ptr(), "scale_vaapi");
// or BT.2020 limited P010 for HDR — the PQ transfer is per-channel and rides through
// the matrix untouched). Without the explicit options the conversion matrix is
// whatever the driver defaults to for an unspecified output (Mesa: BT.601) — a hue
// shift against the signaled VUI.
if ten_bit {
init!(
scale,
c"format=p010:out_color_matrix=bt2020:out_range=limited".as_ptr(),
"scale_vaapi"
);
} else {
init!(
scale,
c"format=nv12:out_color_matrix=bt709:out_range=limited".as_ptr(),
"scale_vaapi"
);
}
init!(sink, ptr::null(), "buffersink"); init!(sink, ptr::null(), "buffersink");
let link = |a: *mut ffi::AVFilterContext, b: *mut ffi::AVFilterContext| -> c_int { let link = |a: *mut ffi::AVFilterContext, b: *mut ffi::AVFilterContext| -> c_int {
@@ -1143,21 +1229,18 @@ impl VaapiEncoder {
chroma: super::ChromaFormat, chroma: super::ChromaFormat,
) -> Result<Self> { ) -> Result<Self> {
// 10-bit rides on the captured format: an HDR capture (X2RGB10/X2BGR10) opens the P010 / // 10-bit rides on the captured format: an HDR capture (X2RGB10/X2BGR10) opens the P010 /
// Main10 / PQ-VUI variant of whichever inner path the first frame selects. A 10-bit // Main10 / PQ-VUI variant of whichever inner path the first frame selects.
// request whose capture stayed SDR honestly encodes 8-bit; the reverse (PQ frames on an match resolve_depth(format, bit_depth) {
// 8-bit session) is refused so PQ content is never mislabeled BT.709. DepthResolution::RefuseMislabeledPq => bail!(
if format.is_hdr_rgb10() && bit_depth != 10 {
bail!(
"captured 10-bit HDR frames ({format:?}) on an {bit_depth}-bit VAAPI session — \ "captured 10-bit HDR frames ({format:?}) on an {bit_depth}-bit VAAPI session — \
refusing to mislabel PQ content" refusing to mislabel PQ content"
); ),
} DepthResolution::SdrDowngrade => tracing::warn!(
if bit_depth == 10 && !format.is_hdr_rgb10() {
tracing::warn!(
bit_depth, bit_depth,
?format, ?format,
"10-bit requested but the capture stayed SDR — encoding 8-bit" "10-bit requested but the capture stayed SDR — encoding 8-bit"
); ),
DepthResolution::Agreed => {}
} }
// VAAPI 4:4:4 is deferred (see `probe_can_encode_444`): no validated AMD/Intel hardware in the // VAAPI 4:4:4 is deferred (see `probe_can_encode_444`): no validated AMD/Intel hardware in the
// lab exposes a HEVC 4:4:4 encode entrypoint, and the probe returns false so the host never // lab exposes a HEVC 4:4:4 encode entrypoint, and the probe returns false so the host never
@@ -1307,3 +1390,261 @@ impl Encoder for VaapiEncoder {
Ok(()) Ok(())
} }
} }
#[cfg(test)]
mod tests {
use super::*;
/// The operator pin tries exactly one mode; a cached resolution ([`LP_MODE`]) tries only the
/// mode that worked; anything else runs the full ladder, full-feature FIRST — AMD's first-try
/// open must stay byte-for-byte unchanged.
#[test]
fn entrypoint_ladder_orders_and_pins() {
assert_eq!(entrypoint_ladder(None, 0), &[false, true]);
assert_eq!(entrypoint_ladder(None, 1), &[false]);
assert_eq!(entrypoint_ladder(None, 2), &[true]);
// A corrupt/unknown cache value degrades to the full ladder, never to a wrong pin.
assert_eq!(entrypoint_ladder(None, 77), &[false, true]);
// The pin beats the cache in BOTH directions — what makes PUNKTFUNK_VAAPI_LOW_POWER a
// real escape hatch from a stale latch.
for cached in [0u8, 1, 2, 77] {
assert_eq!(entrypoint_ladder(Some(true), cached), &[true]);
assert_eq!(entrypoint_ladder(Some(false), cached), &[false]);
}
}
/// The latch round-trip: what a successful open stores makes the NEXT open attempt exactly
/// the mode that worked (the "skip the known-failing attempt and its libav error spew"
/// contract — and, per [`LP_MODE`], the shape that must never cross devices or depths).
#[test]
fn latch_round_trip_pins_the_resolved_mode() {
for lp in [false, true] {
assert_eq!(entrypoint_ladder(None, latched_mode(lp)), &[lp]);
}
}
/// `PUNKTFUNK_VAAPI_LOW_POWER` grammar: the two lowercase literal sets pin; anything else —
/// including uppercase — means "no pin" (the ladder runs). Case-sensitivity is pinned as
/// shipped behavior.
#[test]
fn low_power_grammar() {
for s in ["1", "true", "yes", "on", " on ", "yes\n"] {
assert_eq!(parse_low_power(s), Some(true), "{s:?}");
}
for s in ["0", "false", "no", "off", " off "] {
assert_eq!(parse_low_power(s), Some(false), "{s:?}");
}
for s in ["", "2", "TRUE", "On", "enabled", "low_power"] {
assert_eq!(parse_low_power(s), None, "{s:?}");
}
}
/// Every part of the LP_MODE key is load-bearing (see the [`LP_MODE`] field comments): the
/// node (a GPU-preference switch must re-resolve — the old codec-only key was a
/// session-killer), the codec, and the depth (an 8-bit answer must never pin the 10-bit open —
/// the HDR under-advertisement).
#[test]
fn lp_key_separates_node_codec_and_depth() {
let base = lp_key_for("/dev/dri/renderD128", Codec::H265, false);
assert_ne!(base, lp_key_for("/dev/dri/renderD129", Codec::H265, false));
assert_ne!(base, lp_key_for("/dev/dri/renderD128", Codec::H264, false));
assert_ne!(base, lp_key_for("/dev/dri/renderD128", Codec::H265, true));
}
/// `PUNKTFUNK_VAAPI_ASYNC_DEPTH` grammar: 1..=8 verbatim; unset, junk, zero, and past-the-cap
/// all resolve to the lowest-latency depth 1.
#[test]
fn async_depth_grammar() {
assert_eq!(async_depth(None), 1);
assert_eq!(async_depth(Some("1")), 1);
assert_eq!(async_depth(Some("2")), 2);
assert_eq!(async_depth(Some("8")), 8);
assert_eq!(async_depth(Some("0")), 1);
assert_eq!(async_depth(Some("9")), 1);
assert_eq!(async_depth(Some("-1")), 1);
assert_eq!(async_depth(Some("fast")), 1);
}
/// The swscale source map: packed RGB/BGR and the GNOME 50+ HDR 2:10:10:10 layouts convert;
/// planar/video formats are refused (the CPU path is a packed-RGB fallback, not a general
/// converter).
#[test]
fn sws_src_accepts_packed_rgb_only() {
assert_eq!(vaapi_sws_src(PixelFormat::Bgrx).unwrap(), Pixel::BGRZ);
assert_eq!(vaapi_sws_src(PixelFormat::Rgbx).unwrap(), Pixel::RGBZ);
assert_eq!(vaapi_sws_src(PixelFormat::Bgra).unwrap(), Pixel::BGRA);
assert_eq!(vaapi_sws_src(PixelFormat::Rgba).unwrap(), Pixel::RGBA);
assert_eq!(vaapi_sws_src(PixelFormat::Rgb).unwrap(), Pixel::RGB24);
assert_eq!(vaapi_sws_src(PixelFormat::Bgr).unwrap(), Pixel::BGR24);
assert_eq!(
vaapi_sws_src(PixelFormat::X2Rgb10).unwrap(),
Pixel::X2RGB10LE
);
assert_eq!(
vaapi_sws_src(PixelFormat::X2Bgr10).unwrap(),
Pixel::X2BGR10LE
);
for f in [
PixelFormat::Nv12,
PixelFormat::P010,
PixelFormat::Rgb10a2,
PixelFormat::Yuv444,
] {
assert!(vaapi_sws_src(f).is_err(), "{f:?} must be refused");
}
}
/// VUI ↔ CSC agreement: the signaled VUI and the zero-copy VPP's pinned output colour must
/// name the SAME matrix/range per depth — a mismatch is exactly the Mesa-BT.601 hue shift the
/// pin exists to prevent.
#[test]
fn vui_and_scale_args_agree_per_depth() {
let sdr = vui_for(false);
assert!(matches!(sdr.colorspace, ffi::AVColorSpace::AVCOL_SPC_BT709));
assert!(matches!(sdr.range, ffi::AVColorRange::AVCOL_RANGE_MPEG));
assert!(matches!(
sdr.primaries,
ffi::AVColorPrimaries::AVCOL_PRI_BT709
));
assert!(matches!(
sdr.trc,
ffi::AVColorTransferCharacteristic::AVCOL_TRC_BT709
));
let args = scale_vaapi_args(false).to_str().unwrap();
for needle in ["format=nv12", "out_color_matrix=bt709", "out_range=limited"] {
assert!(
args.contains(needle),
"SDR scale args miss {needle}: {args}"
);
}
let hdr = vui_for(true);
assert!(matches!(
hdr.colorspace,
ffi::AVColorSpace::AVCOL_SPC_BT2020_NCL
));
assert!(matches!(hdr.range, ffi::AVColorRange::AVCOL_RANGE_MPEG));
assert!(matches!(
hdr.primaries,
ffi::AVColorPrimaries::AVCOL_PRI_BT2020
));
assert!(matches!(
hdr.trc,
ffi::AVColorTransferCharacteristic::AVCOL_TRC_SMPTE2084
));
let args = scale_vaapi_args(true).to_str().unwrap();
for needle in [
"format=p010",
"out_color_matrix=bt2020",
"out_range=limited",
] {
assert!(
args.contains(needle),
"HDR scale args miss {needle}: {args}"
);
}
}
/// The honest-downgrade table at open: PQ pixels demand a 10-bit session (refused otherwise —
/// PQ content must never be mislabeled BT.709); a 10-bit session over SDR pixels downgrades
/// honestly to 8-bit; agreement passes both ways.
#[test]
fn depth_resolution_table() {
use DepthResolution::*;
assert_eq!(resolve_depth(PixelFormat::X2Rgb10, 8), RefuseMislabeledPq);
assert_eq!(resolve_depth(PixelFormat::X2Bgr10, 8), RefuseMislabeledPq);
assert_eq!(resolve_depth(PixelFormat::Bgrx, 10), SdrDowngrade);
assert_eq!(resolve_depth(PixelFormat::Bgrx, 8), Agreed);
assert_eq!(resolve_depth(PixelFormat::X2Rgb10, 10), Agreed);
assert_eq!(resolve_depth(PixelFormat::X2Bgr10, 10), Agreed);
}
/// Main10 is pinned explicitly for 10-bit HEVC ONLY: 10-bit AV1 is input-driven (no profile
/// knob), and every 8-bit open keeps the encoder's default profile.
#[test]
fn explicit_profile_is_hevc_main10_only() {
assert_eq!(explicit_profile(Codec::H265, true), Some("main10"));
assert_eq!(explicit_profile(Codec::H265, false), None);
assert_eq!(explicit_profile(Codec::Av1, true), None);
assert_eq!(explicit_profile(Codec::Av1, false), None);
assert_eq!(explicit_profile(Codec::H264, true), None);
assert_eq!(explicit_profile(Codec::H264, false), None);
}
/// The 10-bit probe gate: HEVC and AV1 are probe-eligible; H.264 (no 10-bit path here) and
/// PyroWave (answers 10-bit on its own path, no VAAPI involved) are refused before any device
/// is touched — this is what keeps the probe safe on GPU-less CI.
#[test]
fn ten_bit_probe_gate() {
assert!(ten_bit_probe_eligible(Codec::H265));
assert!(ten_bit_probe_eligible(Codec::Av1));
assert!(!ten_bit_probe_eligible(Codec::H264));
assert!(!ten_bit_probe_eligible(Codec::PyroWave));
}
/// Probe smoke on real silicon: H.264 VAAPI encode opens on every supported AMD/Intel GPU;
/// the narrower codecs are printed, not asserted (device-dependent). Build anywhere, run on a
/// VAAPI host:
/// cargo test -p pf-encode --no-run
/// <host> target/debug/deps/pf_encode-<hash> --ignored --nocapture vaapi_probe_smoke
#[test]
#[ignore = "needs a real VAAPI device (run on an AMD/Intel host, not the build box)"]
fn vaapi_probe_smoke() {
assert!(
probe_can_encode(Codec::H264),
"H.264 VAAPI encode should open on any supported AMD/Intel GPU"
);
for codec in [Codec::H265, Codec::Av1] {
eprintln!("probe_can_encode({codec:?}) = {}", probe_can_encode(codec));
eprintln!(
"probe_can_encode_10bit({codec:?}) = {}",
probe_can_encode_10bit(codec)
);
}
}
/// CPU-path encode round-trip on real silicon: open → BGRX frames → poll AUs → the first AU
/// is the IDR. Exercises the swscale CSC, the VA surface upload, and the entrypoint ladder
/// end-to-end (same recipe as [`vaapi_probe_smoke`]).
#[test]
#[ignore = "needs a real VAAPI device (run on an AMD/Intel host, not the build box)"]
fn vaapi_cpu_encode_smoke() {
let (w, h) = (256u32, 256u32);
let mut enc = VaapiEncoder::open(
Codec::H264,
PixelFormat::Bgrx,
w,
h,
30,
2_000_000,
8,
crate::ChromaFormat::Yuv420,
)
.expect("open");
let mut aus = Vec::new();
for i in 0..30u32 {
let mut buf = vec![0u8; (w * h * 4) as usize];
for px in buf.chunks_exact_mut(4) {
px.copy_from_slice(&[(i * 8) as u8, 0x40, 0xC0, 0xFF]);
}
let frame = CapturedFrame {
width: w,
height: h,
pts_ns: u64::from(i) * 33_333_333,
format: PixelFormat::Bgrx,
payload: FramePayload::Cpu(buf),
cursor: None,
};
enc.submit(&frame).expect("submit");
while let Some(au) = enc.poll().expect("poll") {
aus.push(au);
}
}
enc.flush().expect("flush");
while let Some(au) = enc.poll().expect("poll") {
aus.push(au);
}
assert!(!aus.is_empty(), "no AUs out of 30 submitted frames");
assert!(aus[0].keyframe, "the first AU must be the IDR");
}
}
+58 -60
View File
@@ -76,9 +76,11 @@ fn quality_request() -> u32 {
/// (design/vulkan-rgb-direct-encode.md): the captured RGB frame is the encode source and the /// (design/vulkan-rgb-direct-encode.md): the captured RGB frame is the encode source and the
/// VCN EFC front-end does the 709-narrow CSC inline — no compute CSC, no plane copies, one /// VCN EFC front-end does the 709-narrow CSC inline — no compute CSC, no plane copies, one
/// queue submit per frame (unaligned modes go through the padded-copy staging blit). B2 /// queue submit per frame (unaligned modes go through the padded-copy staging blit). B2
/// default: ON wherever the probe passes, EXCEPT sessions that may need the CSC's cursor /// default: ON wherever the probe passes, EXCEPT sessions that need the CSC's cursor blend
/// blend (see [`VulkanVideoEncoder::open`]). `=0` disables outright; `=1` forces it even on /// (see [`VulkanVideoEncoder::open`]). `=0` disables outright; `=1` forces it on non-cursor
/// cursor-blend sessions (the pointer will be missing from the stream); unset = the default. /// sessions; unset = the default. A cursor-blend session IGNORES `=1` — EFC cannot composite
/// the pointer, and the caps-aware negotiation promised the client a composited one, so the
/// pointer outranks the lab pin (the open logs the override).
/// ///
/// Parses like every sibling knob (`matches!(v.trim(), …)`): anything unrecognised — including /// Parses like every sibling knob (`matches!(v.trim(), …)`): anything unrecognised — including
/// an empty value and a value that is only whitespace — falls back to the default rather than /// an empty value and a value that is only whitespace — falls back to the default rather than
@@ -614,10 +616,6 @@ pub struct VulkanVideoEncoder {
/// GPU reset (those paths keep their aligned-size sources/staging). /// GPU reset (those paths keep their aligned-size sources/staging).
native_nv12: bool, native_nv12: bool,
/// One-shot warning latch: a cursor bitmap arrived on an RGB-direct or native-NV12 session
/// (neither has a compositing stage — the cursor will be missing from the stream until the
/// CSC path is used).
warned_cursor: bool,
/// A [`reconfigure_bitrate`](Encoder::reconfigure_bitrate) rate not yet installed in the video /// A [`reconfigure_bitrate`](Encoder::reconfigure_bitrate) rate not yet installed in the video
/// session. The next `record_submit` emits an `ENCODE_RATE_CONTROL` control command carrying it /// session. The next `record_submit` emits an `ENCODE_RATE_CONTROL` control command carrying it
/// (mid-stream) or folds it into the first frame's RESET+RC install, then promotes it into /// (mid-stream) or folds it into the first frame's RESET+RC install, then promotes it into
@@ -671,7 +669,16 @@ impl VulkanVideoEncoder {
cursor_blend: bool, cursor_blend: bool,
) -> Result<Self> { ) -> Result<Self> {
let native_nv12 = format == PixelFormat::Nv12; let native_nv12 = format == PixelFormat::Nv12;
let want_rgb = !native_nv12 && rgb_request().unwrap_or(!cursor_blend); // A cursor-blend session must keep the compute-CSC path — the only arm with the cursor
// blend — so it outranks an explicit RGB-direct pin (EFC cannot composite; the
// negotiation promised the client a composited pointer).
if cursor_blend && rgb_request() == Some(true) {
tracing::info!(
"PUNKTFUNK_VULKAN_RGB_DIRECT=1 ignored for this session — it composites the \
pointer, which the EFC front-end cannot; using the compute-CSC path"
);
}
let want_rgb = !native_nv12 && !cursor_blend && rgb_request().unwrap_or(true);
Self::open_opts_inner( Self::open_opts_inner(
codec, codec,
width, width,
@@ -1448,7 +1455,6 @@ impl VulkanVideoEncoder {
cpu_expand: Vec::new(), cpu_expand: Vec::new(),
rgb: rgb_cfg, rgb: rgb_cfg,
native_nv12, native_nv12,
warned_cursor: false,
pending_bitrate: None, pending_bitrate: None,
width: w, width: w,
height: h, height: h,
@@ -2621,17 +2627,10 @@ impl VulkanVideoEncoder {
d.modifier d.modifier
); );
} }
// No compositing stage exists here (like RGB-direct/EFC): gamescope embeds its pointer // No compositing stage exists here (like RGB-direct/EFC) — and none is needed: the
// in the produced pixels, but any other NV12 producer's metadata cursor would be lost — // session plan negotiates native NV12 only for a non-cursor-blend session
// say so once instead of silently. // (`SessionPlan::output_format` gates `nv12_native` on `!cursor_blend`), so no cursor
if frame.cursor.is_some() && !self.warned_cursor { // bitmap ever reaches this arm. Gamescope embeds its pointer in the produced pixels.
self.warned_cursor = true;
tracing::warn!(
"cursor bitmap on a native-NV12 session — nothing composites it; the cursor \
will be missing from the stream (unset PUNKTFUNK_PIPEWIRE_NV12 for \
metadata-cursor captures)"
);
}
let dev = self.device.clone(); let dev = self.device.clone();
let cmd = self.frames[slot].cmd; let cmd = self.frames[slot].cmd;
let fence = self.frames[slot].fence; let fence = self.frames[slot].fence;
@@ -2691,16 +2690,9 @@ impl VulkanVideoEncoder {
let query_pool = self.frames[slot].query_pool; let query_pool = self.frames[slot].query_pool;
let bs_buf = self.frames[slot].bs_buf; let bs_buf = self.frames[slot].bs_buf;
let ts_pool = self.frames[slot].ts_pool; let ts_pool = self.frames[slot].ts_pool;
// EFC cannot composite the cursor bitmap the metadata-cursor captures hand us — say so // EFC cannot composite a cursor bitmap — and never has to: `open` refuses the RGB-direct
// once instead of silently losing the pointer (gamescope, the flagship, embeds it). // shape for a cursor-blend session (the pin override above), so no cursor bitmap ever
if frame.cursor.is_some() && !self.warned_cursor { // reaches this arm. Gamescope, the flagship, embeds its pointer in the produced pixels.
self.warned_cursor = true;
tracing::warn!(
"cursor bitmap on an RGB-direct session — EFC cannot composite it; the cursor \
will be missing from the stream (unset PUNKTFUNK_VULKAN_RGB_DIRECT for \
metadata-cursor captures)"
);
}
let padded = self.rgb.as_ref().is_some_and(|r| r.padded); let padded = self.rgb.as_ref().is_some_and(|r| r.padded);
// Only the padded Dmabuf arm below records timestamps (via `record_pad_blit`); the // Only the padded Dmabuf arm below records timestamps (via `record_pad_blit`); the
// CPU-upload arm records its own command buffer and writes none. Default to "not written" // CPU-upload arm records its own command buffer and writes none. Default to "not written"
@@ -4486,44 +4478,50 @@ mod tests {
} }
} }
/// A CPU-capture source that CHANGES SIZE mid-session must not copy past the cached staging /// The CSC arm REFUSES a source that doesn't match the session mode (the e3354b6d guard —
/// image. `ensure_cpu_rgb` keys that image on (format, width, height); when it was keyed on /// the check every sibling arm always had; a clamped-texelFetch mismatch used to stream a
/// format alone the image kept the FIRST frame's size while `cmd_copy_buffer_to_image` used the /// silently cropped/edge-padded picture). This test previously DROVE mismatched sizes through
/// current frame's extent, so a same-format size increase wrote out of bounds — and `submit` /// the lenient arm to exercise the per-slot `cpu_img` staging across a size change (the
/// still returned `Ok`, so nothing upstream noticed. /// format-only-keyed cache wrote out of bounds while `submit` returned `Ok`); the guard makes
/// /// that scenario unrepresentable through `submit`, structurally retiring the hazard — the
/// The out-of-bounds copy is only *observable* through the Vulkan validation layers, so run this /// size-keyed staging from that fix stays as belt-and-braces. What's left to pin:
/// as: `VK_LOADER_LAYERS_ENABLE='*validation*' cargo test ... -- --ignored`. Confirmed on RADV /// refusal in BOTH directions, and that a refused submit does not WEDGE the session — the
/// PHOENIX 2026-07-25: 8 x VUID-vkCmdCopyBufferToImage-imageSubresource-07971 before the fix /// bail happens after step 1's frame-type bookkeeping, so "the next well-sized frame still
/// ("extent.width (512) exceeds imageSubresource width extent (128)"), zero after. /// encodes" is a real property, not a formality (the host routes the error to its
/// encoder-rebuild path and the session must be able to continue if that path retries).
#[test] #[test]
#[ignore = "needs a real VK_KHR_video_encode_h265 device; meaningful only under validation layers"] #[ignore = "needs a real VK_KHR_video_encode_h265 device; meaningful only under validation layers"]
fn vulkan_cpu_img_survives_a_source_size_change() { fn vulkan_csc_refuses_a_mismatched_source() {
// CSC mode (rgb=false) — that is the arm where the image is sized to the SOURCE frame. // CSC mode (rgb=false) — the arm the guard covers.
let mut enc = VulkanVideoEncoder::open_opts(Codec::H265, 512, 512, 60, 10_000_000, false) let mut enc = VulkanVideoEncoder::open_opts(Codec::H265, 512, 512, 60, 10_000_000, false)
.expect("open"); .expect("open");
// `cpu_img` is cached PER RING SLOT, so the small frames must first populate every slot; enc.submit_indexed(&cpu_frame(512, 512, 0, [40, 40, 200, 255]), 0)
// only when the ring wraps does a slot holding a 128x128 image get a 512x512 copy. .expect("well-sized baseline");
eprintln!("phase 1: 8x 128x128 — populates every ring slot with a 128x128 cpu_img"); while enc.poll().expect("poll").is_some() {}
for i in 0..8u64 { // Smaller AND larger both refuse — the guard is equality on the MODE (render size), not
// a ceiling; the render-vs-CODED padding tolerance lives in the direct arms, untouched.
let e = enc
.submit_indexed(&cpu_frame(128, 128, 16_666_667, [200, 40, 40, 255]), 1)
.expect_err("smaller source must refuse");
assert!(e.to_string().contains("mismatched"), "{e:#}");
let e = enc
.submit_indexed(&cpu_frame(640, 640, 33_333_334, [200, 40, 40, 255]), 2)
.expect_err("larger source must refuse");
assert!(e.to_string().contains("mismatched"), "{e:#}");
// The refusals must not wedge the session: a well-sized frame still encodes and an AU
// still comes out the other end.
let mut got_au = false;
for i in 3..11u64 {
enc.submit_indexed( enc.submit_indexed(
&cpu_frame(128, 128, i * 16_666_667, [40, 40, 200, 255]), &cpu_frame(512, 512, i * 16_666_667, [40, 200, 40, 255]),
i as u32, i as u32,
) )
.expect("submit small"); .expect("well-sized after refusal");
while enc.poll().expect("poll").is_some() {} while let Ok(Some(_)) = enc.poll() {
got_au = true;
}
} }
eprintln!("phase 2: 8x 512x512 — SAME format, so each slot REUSES its 128x128 image"); assert!(got_au, "no AU after the refused submits — session wedged");
for i in 8..16u64 {
let r = enc.submit_indexed(
&cpu_frame(512, 512, i * 16_666_667, [200, 40, 40, 255]),
i as u32,
);
r.expect("submit after the source grew");
while matches!(enc.poll(), Ok(Some(_))) {}
}
let _ = enc.flush();
while matches!(enc.poll(), Ok(Some(_))) {}
eprintln!("done — under validation layers this run must report ZERO VUID errors"); eprintln!("done — under validation layers this run must report ZERO VUID errors");
} }
-7
View File
@@ -2014,9 +2014,6 @@ impl Encoder for AmfEncoder {
// frame, force a later one to re-reference it). True only when the live driver accepted // frame, force a later one to re-reference it). True only when the live driver accepted
// the LTR slots at open — otherwise loss recovery falls back to a full IDR. // the LTR slots at open — otherwise loss recovery falls back to a full IDR.
supports_rfi: self.ltr_active, supports_rfi: self.ltr_active,
// In-band mastering/CLL via `*InHDRMetadata` (HEVC SEI / AV1 metadata OBU); AVC has
// no such property (and no HDR sessions negotiate H.264).
supports_hdr_metadata: self.ten_bit && self.props.hdr_metadata.is_some(),
// Permanent: VCN hardware does not encode 4:4:4. // Permanent: VCN hardware does not encode 4:4:4.
chroma_444: false, chroma_444: false,
// True only when `PUNKTFUNK_INTRA_REFRESH` asked for the wave AND the live driver // True only when `PUNKTFUNK_INTRA_REFRESH` asked for the wave AND the live driver
@@ -2715,10 +2712,6 @@ mod tests {
} }
}; };
enc.set_hdr_meta(Some(sample_hdr_meta())); enc.set_hdr_meta(Some(sample_hdr_meta()));
assert!(
enc.caps().supports_hdr_metadata,
"HEVC 10-bit reports HDR SEI capability"
);
let mut aus: Vec<EncodedFrame> = Vec::new(); let mut aus: Vec<EncodedFrame> = Vec::new();
for i in 0..6 { for i in 0..6 {
let frame = CapturedFrame { let frame = CapturedFrame {
+296 -65
View File
@@ -129,9 +129,13 @@ impl WinVendor {
/// open-failure fallback only catches *setup* errors; a derive that opens but maps wrong would /// open-failure fallback only catches *setup* errors; a derive that opens but maps wrong would
/// corrupt silently, so it stays opt-in per the probe-never-assume rule). /// corrupt silently, so it stays opt-in per the probe-never-assume rule).
fn zerocopy_enabled(vendor: WinVendor) -> bool { fn zerocopy_enabled(vendor: WinVendor) -> bool {
pf_host_config::config() zerocopy_active(pf_host_config::config().zerocopy, vendor)
.zerocopy }
.unwrap_or(matches!(vendor, WinVendor::Amf))
/// The pure half of [`zerocopy_enabled`]: an operator override wins; unset resolves to the
/// per-vendor default (AMF on, QSV off — see the validation status above).
fn zerocopy_active(override_: Option<bool>, vendor: WinVendor) -> bool {
override_.unwrap_or(matches!(vendor, WinVendor::Amf))
} }
/// Upper bound on `PUNKTFUNK_FFWIN_POLL_MS`. This knob spins the **encode thread** waiting for an /// Upper bound on `PUNKTFUNK_FFWIN_POLL_MS`. This knob spins the **encode thread** waiting for an
@@ -164,14 +168,19 @@ const MAX_POLL_SPIN_MS: u64 = 1_000;
fn poll_spin_cap_us() -> u64 { fn poll_spin_cap_us() -> u64 {
static CAP_US: std::sync::OnceLock<u64> = std::sync::OnceLock::new(); static CAP_US: std::sync::OnceLock<u64> = std::sync::OnceLock::new();
*CAP_US.get_or_init(|| { *CAP_US.get_or_init(|| {
std::env::var("PUNKTFUNK_FFWIN_POLL_MS") parse_poll_spin_cap_us(std::env::var("PUNKTFUNK_FFWIN_POLL_MS").ok().as_deref())
.ok()
.and_then(|s| s.trim().parse::<u64>().ok())
.map(|ms| ms.min(MAX_POLL_SPIN_MS) * 1000)
.unwrap_or(0) // default: no spin — the libavcodec AMF buffer can't be spun out
}) })
} }
/// The pure half of [`poll_spin_cap_us`]: parse, clamp to [`MAX_POLL_SPIN_MS`] BEFORE the µs
/// conversion (the ordering the doc above proves is load-bearing), and default to 0 — no spin,
/// the libavcodec AMF buffer can't be spun out.
fn parse_poll_spin_cap_us(raw: Option<&str>) -> u64 {
raw.and_then(|s| s.trim().parse::<u64>().ok())
.map(|ms| ms.min(MAX_POLL_SPIN_MS) * 1000)
.unwrap_or(0)
}
/// The swscale *source* pixel format for a captured packed-RGB/BGR layout (8-bit BGRA fallback only). /// The swscale *source* pixel format for a captured packed-RGB/BGR layout (8-bit BGRA fallback only).
fn sws_src(format: PixelFormat) -> Result<Pixel> { fn sws_src(format: PixelFormat) -> Result<Pixel> {
Ok(match format { Ok(match format {
@@ -206,6 +215,86 @@ fn is_10bit_format(format: PixelFormat) -> bool {
matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2) matches!(format, PixelFormat::P010 | PixelFormat::Rgb10a2)
} }
/// Which lane the system-memory path routes a captured D3D11 format through. Device-free — the
/// routing DECISION, split from the D3D11 copies so it is testable.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum ReadbackRoute {
/// Same-format `CopyResource` + plane-by-plane copy (NV12/P010 from the video processor).
Yuv,
/// BGRA staging + swscale BGRA→NV12 — the 8-bit fallback when the capturer's video
/// processor latched off.
Bgra,
/// R10G10B10A2 staging + swscale X2BGR10→P010 — the HDR twin of that fallback.
Rgb10,
}
/// Route a captured format, guarding the mid-stream depth change first: the predicate matches
/// what the encoder was built from (`ten_bit_input`), so the guard can only fire on a GENUINE
/// depth change under the encoder — never, as it used to, on every frame of a session that
/// merely negotiated 10-bit over an 8-bit capture (see [`is_10bit_format`]).
fn readback_route(format: PixelFormat, ten_bit: bool) -> Result<ReadbackRoute> {
anyhow::ensure!(
is_10bit_format(format) == ten_bit,
"captured format {format:?} bit-depth changed under the encoder (built {}-bit)",
if ten_bit { 10 } else { 8 }
);
Ok(match format {
PixelFormat::Nv12 | PixelFormat::P010 => ReadbackRoute::Yuv,
PixelFormat::Bgra | PixelFormat::Bgrx => ReadbackRoute::Bgra,
PixelFormat::Rgb10a2 => ReadbackRoute::Rgb10,
other => {
bail!("ffmpeg_win system path cannot read back captured D3D11 format {other:?}")
}
})
}
/// The vendor-specific low-latency option set for [`open_win_encoder`], pure so the latency
/// contract is pinned by tests. Unknown private options are ignored by `avcodec_open2` (left in
/// the dict), so vendor/codec-specific keys are safe to set unconditionally.
fn vendor_opts(vendor: WinVendor, amf_usage: &str) -> Vec<(&'static str, String)> {
match vendor {
WinVendor::Amf => vec![
// Field-tuning override (ultralowlatency | lowlatency | lowlatency_high_quality |
// transcoding): AMF usage presets bundle driver-side pipeline behavior that varies
// by VCN generation/driver — measured on-box rather than assumed.
("usage", amf_usage.to_owned()),
("rc", "cbr".into()),
// Streaming is latency-first: `speed` trims per-frame motion-estimation depth — the
// difference between ~encode-time and ~frame-budget on iGPU-class VCN (matches the
// low-latency preset choice on the NVENC path).
("quality", "speed".into()),
("preanalysis", "false".into()),
("enforce_hrd", "true".into()),
// AMF low-latency submission mode (FFmpeg ≥ 6.1; unknown-option-ignored on older).
("latency", "true".into()),
// Never B-frames: h264_amf defaults >0 on RDNA3+ HW that supports them, and each
// B-frame is a full frame period of added latency. (HEVC VCN has none; ignored there.)
("bf", "0".into()),
// VPS/SPS/PPS on each IDR (clean mid-stream join) — HEVC/AV1 only; ignored elsewhere.
("header_insertion_mode", "idr".into()),
],
WinVendor::Qsv => vec![
("preset", "veryfast".into()),
("async_depth", "1".into()), // bound in-flight frames — the big QSV latency lever
("low_power", "1".into()), // VDEnc fixed-function path (lower latency)
("look_ahead", "0".into()), // (h264_qsv only; ignored on hevc/av1)
("forced_idr", "1".into()), // a forced key frame becomes a real IDR
("scenario", "displayremoting".into()),
],
}
}
/// Bind flags on the FFmpeg-allocated zero-copy pool. AMF reads it as encoder input
/// (RENDER_TARGET + SHADER_RESOURCE, matching the video-processor output); QSV maps it as an mfx
/// surface (DECODER | VIDEO_ENCODER). The `CopySubresourceRegion` into the pool works with any
/// usable DEFAULT-usage texture regardless.
fn pool_bind_flags(vendor: WinVendor) -> u32 {
match vendor {
WinVendor::Amf => (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
WinVendor::Qsv => (D3D11_BIND_DECODER.0 | D3D11_BIND_VIDEO_ENCODER.0) as u32,
}
}
/// Build the FFmpeg encoder context shared by both inner paths: name, mode, low-latency RC, /// Build the FFmpeg encoder context shared by both inner paths: name, mode, low-latency RC,
/// infinite GOP, the BT.709-limited (SDR) or BT.2020-PQ (HDR) VUI, the given `pix_fmt`, and the /// infinite GOP, the BT.709-limited (SDR) or BT.2020-PQ (HDR) VUI, the given `pix_fmt`, and the
/// optional hw device/frames contexts (null for the system path). Returns the opened encoder. /// optional hw device/frames contexts (null for the system path). Returns the opened encoder.
@@ -266,40 +355,11 @@ unsafe fn open_win_encoder(
(*raw).hw_frames_ctx = ffi::av_buffer_ref(frames_ref); (*raw).hw_frames_ctx = ffi::av_buffer_ref(frames_ref);
} }
// Low-latency tuning. Unknown private options are ignored by avcodec_open2 (left in the dict), // Low-latency tuning — the per-vendor contract lives in `vendor_opts` (pure, test-pinned).
// so vendor-specific keys are safe to set unconditionally.
let mut opts = Dictionary::new(); let mut opts = Dictionary::new();
match vendor { let usage = std::env::var("PUNKTFUNK_AMF_USAGE").unwrap_or_else(|_| "ultralowlatency".into());
WinVendor::Amf => { for (k, v) in vendor_opts(vendor, &usage) {
// Field-tuning override (ultralowlatency | lowlatency | lowlatency_high_quality | opts.set(k, &v);
// transcoding): AMF usage presets bundle driver-side pipeline behavior that varies by
// VCN generation/driver — measured on-box rather than assumed.
let usage =
std::env::var("PUNKTFUNK_AMF_USAGE").unwrap_or_else(|_| "ultralowlatency".into());
opts.set("usage", &usage);
opts.set("rc", "cbr");
// Streaming is latency-first: `speed` trims per-frame motion-estimation depth — the
// difference between ~encode-time and ~frame-budget on iGPU-class VCN (matches the
// low-latency preset choice on the NVENC path).
opts.set("quality", "speed");
opts.set("preanalysis", "false");
opts.set("enforce_hrd", "true");
// AMF low-latency submission mode (FFmpeg ≥ 6.1; unknown-option-ignored on older).
opts.set("latency", "true");
// Never B-frames: h264_amf defaults >0 on RDNA3+ HW that supports them, and each
// B-frame is a full frame period of added latency. (HEVC VCN has none; ignored there.)
opts.set("bf", "0");
// VPS/SPS/PPS on each IDR (clean mid-stream join) — HEVC/AV1 only; ignored elsewhere.
opts.set("header_insertion_mode", "idr");
}
WinVendor::Qsv => {
opts.set("preset", "veryfast");
opts.set("async_depth", "1"); // bound in-flight frames — the big QSV latency lever
opts.set("low_power", "1"); // VDEnc fixed-function path (lower latency)
opts.set("look_ahead", "0"); // (h264_qsv only; ignored on hevc/av1)
opts.set("forced_idr", "1"); // a forced key frame becomes a real IDR
opts.set("scenario", "displayremoting");
}
} }
video video
.open_with(opts) .open_with(opts)
@@ -528,22 +588,10 @@ impl SystemInner {
pts: i64, pts: i64,
idr: bool, idr: bool,
) -> Result<()> { ) -> Result<()> {
// Same predicate the encoder was built from (`ten_bit_input`), so this can only fire on a match readback_route(format, self.ten_bit)? {
// genuine MID-STREAM depth change — never, as it used to, on every frame of a session that ReadbackRoute::Yuv => self.readback_yuv(frame, pts, idr),
// merely negotiated 10-bit over an 8-bit capture. ReadbackRoute::Bgra => self.readback_bgra(frame, pts, idr),
let fmt_10 = is_10bit_format(format); ReadbackRoute::Rgb10 => self.readback_rgb10(frame, pts, idr),
anyhow::ensure!(
fmt_10 == self.ten_bit,
"captured format {format:?} bit-depth changed under the encoder (built {}-bit)",
if self.ten_bit { 10 } else { 8 }
);
match format {
PixelFormat::Nv12 | PixelFormat::P010 => self.readback_yuv(frame, pts, idr),
PixelFormat::Bgra | PixelFormat::Bgrx => self.readback_bgra(frame, pts, idr),
PixelFormat::Rgb10a2 => self.readback_rgb10(frame, pts, idr),
other => {
bail!("ffmpeg_win system path cannot read back captured D3D11 format {other:?}")
}
} }
} }
@@ -921,14 +969,7 @@ impl ZeroCopyInner {
} else { } else {
PixelFormat::Nv12 PixelFormat::Nv12
}; };
// Bind flags on the FFmpeg-allocated pool. AMF reads it as encoder input (RENDER_TARGET + let bind_flags = pool_bind_flags(vendor);
// SHADER_RESOURCE, matching the video-processor output); QSV maps it as an mfx surface
// (DECODER | VIDEO_ENCODER). The CopySubresourceRegion into the pool works with any usable
// DEFAULT-usage texture regardless.
let bind_flags = match vendor {
WinVendor::Amf => (D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32,
WinVendor::Qsv => (D3D11_BIND_DECODER.0 | D3D11_BIND_VIDEO_ENCODER.0) as u32,
};
const POOL: c_int = 8; const POOL: c_int = 8;
// SAFETY: `D3d11Hw::new` wraps the capturer's `device` as a D3D11VA hwdevice (handing FFmpeg an // SAFETY: `D3d11Hw::new` wraps the capturer's `device` as a D3D11VA hwdevice (handing FFmpeg an
// owned AddRef of it, balanced by FFmpeg's teardown Release) and builds an owned // owned AddRef of it, balanced by FFmpeg's teardown Release) and builds an owned
@@ -1402,3 +1443,193 @@ impl Encoder for FfmpegWinEncoder {
Ok(()) Ok(())
} }
} }
#[cfg(test)]
mod tests {
use super::*;
/// Zero-copy default matrix: the operator override wins in both directions; unset resolves
/// AMF on (on-glass validated) and QSV off (opt-in until validated on Intel glass — the
/// probe-never-assume rule).
#[test]
fn zerocopy_default_is_per_vendor_and_override_wins() {
assert!(zerocopy_active(None, WinVendor::Amf));
assert!(!zerocopy_active(None, WinVendor::Qsv));
for vendor in [WinVendor::Amf, WinVendor::Qsv] {
assert!(zerocopy_active(Some(true), vendor));
assert!(!zerocopy_active(Some(false), vendor));
}
}
/// `PUNKTFUNK_FFWIN_POLL_MS` grammar: default 0 (no spin), verbatim ms → µs inside the clamp,
/// and the clamp applies BEFORE the µs conversion — the slipped-digit value that used to be a
/// 27.7-hour spin resolves to the 1 s cap, not a wedged encode thread.
#[test]
fn poll_spin_cap_clamps_before_the_us_conversion() {
assert_eq!(parse_poll_spin_cap_us(None), 0);
assert_eq!(parse_poll_spin_cap_us(Some("0")), 0);
assert_eq!(parse_poll_spin_cap_us(Some("5")), 5_000);
assert_eq!(parse_poll_spin_cap_us(Some(" 12 ")), 12_000);
assert_eq!(
parse_poll_spin_cap_us(Some("100000000")),
MAX_POLL_SPIN_MS * 1000
);
assert_eq!(
parse_poll_spin_cap_us(Some(&u64::MAX.to_string())),
MAX_POLL_SPIN_MS * 1000
);
assert_eq!(parse_poll_spin_cap_us(Some("junk")), 0);
assert_eq!(parse_poll_spin_cap_us(Some("-1")), 0);
}
/// The swscale source map: packed RGB/BGR converts; every YUV/10-bit layout is refused (the
/// swscale lane is the 8-bit BGRA fallback, not a general converter — the Linux HDR formats
/// are listed explicitly so a `PixelFormat` addition re-breaks the match on purpose).
#[test]
fn sws_src_accepts_packed_rgb_only() {
assert_eq!(sws_src(PixelFormat::Bgrx).unwrap(), Pixel::BGRZ);
assert_eq!(sws_src(PixelFormat::Rgbx).unwrap(), Pixel::RGBZ);
assert_eq!(sws_src(PixelFormat::Bgra).unwrap(), Pixel::BGRA);
assert_eq!(sws_src(PixelFormat::Rgba).unwrap(), Pixel::RGBA);
assert_eq!(sws_src(PixelFormat::Rgb).unwrap(), Pixel::RGB24);
assert_eq!(sws_src(PixelFormat::Bgr).unwrap(), Pixel::BGR24);
for f in [
PixelFormat::Nv12,
PixelFormat::P010,
PixelFormat::Rgb10a2,
PixelFormat::Yuv444,
PixelFormat::X2Rgb10,
PixelFormat::X2Bgr10,
] {
assert!(sws_src(f).is_err(), "{f:?} must be refused");
}
}
/// The readback routing table, and its depth guard: NV12/P010 take the plane copy, the
/// video-processor fallbacks take their swscale lanes, a depth CHANGE under the encoder is
/// refused (in both directions), and depth-consistent routing never trips the guard.
#[test]
fn readback_routing_and_depth_guard() {
assert_eq!(
readback_route(PixelFormat::Nv12, false).unwrap(),
ReadbackRoute::Yuv
);
assert_eq!(
readback_route(PixelFormat::P010, true).unwrap(),
ReadbackRoute::Yuv
);
assert_eq!(
readback_route(PixelFormat::Bgra, false).unwrap(),
ReadbackRoute::Bgra
);
assert_eq!(
readback_route(PixelFormat::Bgrx, false).unwrap(),
ReadbackRoute::Bgra
);
assert_eq!(
readback_route(PixelFormat::Rgb10a2, true).unwrap(),
ReadbackRoute::Rgb10
);
// Mid-stream depth changes — the genuine error the guard exists for.
assert!(readback_route(PixelFormat::P010, false).is_err());
assert!(readback_route(PixelFormat::Rgb10a2, false).is_err());
assert!(readback_route(PixelFormat::Nv12, true).is_err());
assert!(readback_route(PixelFormat::Bgra, true).is_err());
// A format neither lane can read back.
assert!(readback_route(PixelFormat::Yuv444, false).is_err());
}
/// The 10-bit predicate follows the PIXELS (P010/Rgb10a2), not the negotiated depth — see
/// `ten_bit_input` for the forever-failing-session shape the reverse produced here.
#[test]
fn ten_bit_follows_the_pixels() {
assert!(is_10bit_format(PixelFormat::P010));
assert!(is_10bit_format(PixelFormat::Rgb10a2));
assert!(!is_10bit_format(PixelFormat::Nv12));
assert!(!is_10bit_format(PixelFormat::Bgra));
assert!(!is_10bit_format(PixelFormat::Bgrx));
}
/// The QSV low-latency contract, pinned: these five knobs are the difference between
/// display-remoting latency and transcode behavior — a silent regression here changes every
/// Intel Windows session.
#[test]
fn qsv_opts_pin_the_latency_contract() {
let opts = vendor_opts(WinVendor::Qsv, "ignored");
let get = |k: &str| {
opts.iter()
.find(|(key, _)| *key == k)
.map(|(_, v)| v.as_str())
};
assert_eq!(get("async_depth"), Some("1"));
assert_eq!(get("low_power"), Some("1"));
assert_eq!(get("look_ahead"), Some("0"));
assert_eq!(get("forced_idr"), Some("1"));
assert_eq!(get("scenario"), Some("displayremoting"));
assert_eq!(get("preset"), Some("veryfast"));
assert_eq!(get("usage"), None, "AMF-only knob must not leak into QSV");
}
/// The AMF (benchmark-comparator) contract: usage passes through, B-frames are pinned OFF
/// (each one is a full frame period of latency on RDNA3+), and the low-latency submission
/// mode + IDR header insertion are requested.
#[test]
fn amf_opts_pin_no_bframes_and_the_usage_passthrough() {
let opts = vendor_opts(WinVendor::Amf, "lowlatency");
let get = |k: &str| {
opts.iter()
.find(|(key, _)| *key == k)
.map(|(_, v)| v.as_str())
};
assert_eq!(get("usage"), Some("lowlatency"));
assert_eq!(get("bf"), Some("0"));
assert_eq!(get("rc"), Some("cbr"));
assert_eq!(get("quality"), Some("speed"));
assert_eq!(get("latency"), Some("true"));
assert_eq!(get("header_insertion_mode"), Some("idr"));
assert_eq!(get("preanalysis"), Some("false"));
assert_eq!(get("enforce_hrd"), Some("true"));
}
/// The zero-copy pool's bind flags per vendor — AMF's encoder-input shape vs QSV's mfx
/// surface shape (a wrong flag set fails `av_hwframe_ctx_init`, or worse, opens and maps
/// wrong).
#[test]
fn pool_bind_flags_per_vendor() {
assert_eq!(
pool_bind_flags(WinVendor::Amf),
(D3D11_BIND_RENDER_TARGET.0 | D3D11_BIND_SHADER_RESOURCE.0) as u32
);
assert_eq!(
pool_bind_flags(WinVendor::Qsv),
(D3D11_BIND_DECODER.0 | D3D11_BIND_VIDEO_ENCODER.0) as u32
);
}
/// The libavcodec encoder-name dispatch (name-selected — the codec id would pick the
/// software encoder).
#[test]
fn encoder_names_dispatch_by_vendor() {
assert_eq!(WinVendor::Qsv.encoder_name(Codec::H264), "h264_qsv");
assert_eq!(WinVendor::Qsv.encoder_name(Codec::H265), "hevc_qsv");
assert_eq!(WinVendor::Qsv.encoder_name(Codec::Av1), "av1_qsv");
assert_eq!(WinVendor::Amf.encoder_name(Codec::H265), "hevc_amf");
}
/// Probe smoke: resolve the QSV probe on this machine without crashing — `false` on a box
/// without the Intel runtime is a valid outcome; the value is printed, not asserted. Only
/// compiled in the `amf-qsv`-without-`qsv` combo (the shipped combo answers via native VPL).
/// Run on the Windows CI runner:
/// cargo test -p pf-encode --no-default-features --features amf-qsv -- --ignored ffmpeg_win
#[cfg(not(feature = "qsv"))]
#[test]
#[ignore = "needs a real FFmpeg runtime probe (run on the Windows CI runner, not a dev box)"]
fn ffmpeg_win_probe_smoke() {
for codec in [Codec::H264, Codec::H265, Codec::Av1] {
eprintln!(
"probe_can_encode(Qsv, {codec:?}) = {}",
probe_can_encode(WinVendor::Qsv, codec)
);
}
}
}
+4 -7
View File
@@ -1690,17 +1690,14 @@ impl Encoder for NvencD3d11Encoder {
} }
fn caps(&self) -> EncoderCaps { fn caps(&self) -> EncoderCaps {
// RFI is probed once at open (`rfi_supported`); HDR SEI rides keyframes whenever the // RFI is probed once at open (`rfi_supported`) — the real capability the session glue
// session is in HDR mode. Both are the real capabilities the session glue routes on. // routes on. (In-band HDR SEI needs no cap: it rides keyframes on HEVC/H.264 HDR
// sessions — see `submit` — and every first-party client reads the grade out-of-band
// via the 0xCE datagram regardless.)
EncoderCaps { EncoderCaps {
// The Windows capture path composites the pointer; this backend never reads `frame.cursor`. // The Windows capture path composites the pointer; this backend never reads `frame.cursor`.
blends_cursor: false, blends_cursor: false,
supports_rfi: self.rfi_supported, supports_rfi: self.rfi_supported,
// In-band mastering/CLL is attached as keyframe SEI on HEVC/H.264 only — AV1 carries
// it in METADATA OBUs (`HDR_MDCV`/`HDR_CLL`), which this backend doesn't emit yet
// (see `submit`); the grade still reaches punktfunk clients out-of-band via the 0xCE
// datagram. Don't claim a capability the AV1 path doesn't have.
supports_hdr_metadata: self.hdr && self.codec != Codec::Av1,
// Reflects what the session actually configured (cleared in `query_caps` if the GPU lacks // Reflects what the session actually configured (cleared in `query_caps` if the GPU lacks
// YUV444 encode), so the glue can confirm 4:4:4 vs the negotiated request. // YUV444 encode), so the glue can confirm 4:4:4 vs the negotiated request.
chroma_444: self.chroma_444, chroma_444: self.chroma_444,
-3
View File
@@ -1471,9 +1471,6 @@ impl Encoder for QsvEncoder {
// As Windows NVENC: the capturer composites; this backend never reads `frame.cursor`. // As Windows NVENC: the capturer composites; this backend never reads `frame.cursor`.
blends_cursor: false, blends_cursor: false,
supports_rfi: self.ltr_active, supports_rfi: self.ltr_active,
// In-band mastering/CLL at IDR (HEVC prefix SEI / AV1 metadata OBU); AVC sessions
// are never HDR.
supports_hdr_metadata: self.ten_bit && self.codec != Codec::H264,
chroma_444: false, chroma_444: false,
intra_refresh: self.ir_active, intra_refresh: self.ir_active,
// Unvalidated on-glass — the host keeps the IDR recovery path until then. // Unvalidated on-glass — the host keeps the IDR recovery path until then.
+156 -8
View File
@@ -203,14 +203,15 @@ pub fn open_video(
} }
}; };
// The session asked for a composited pointer; say so loudly if the backend that actually opened // The session asked for a composited pointer; say so loudly if the backend that actually opened
// cannot deliver one. `cursor_blend` was a REQUEST with no answer for most of this crate's life // cannot deliver one. Since the negotiation became caps-aware ([`cursor_blend_capable`] gates
// (`let _ = cursor_blend;` below), and the result was a stream with no mouse cursor and nothing // the cursor channel, and the session plan keeps cursor sessions off the native-NV12/RGB-direct
// in the logs — confirmed on the VAAPI dmabuf path and the libav-NVENC CUDA path. // shapes), no PLANNED path reaches this: capture negotiates embedded-cursor mode wherever the
// // resolved backend can't blend. What remains reachable is the open-time divergence the plan
// A warning is deliberately all this does. `open_video` cannot re-plan capture, so refusing here // cannot see — a Vulkan Video open failing back to VAAPI mid-`open_amd_intel`, and the
// would only trade a missing pointer for a dead session; the host owns `plan.cursor_blend` and is // gamescope residual (gamescope has no embedded mode, so a never-blending backend there —
// the only layer that can fall back to capturer-side compositing. This makes the condition // H.264→VAAPI, software — still streams cursorless). This is the backstop that keeps those
// visible and queryable (`EncoderCaps::blends_cursor`) so that decision can be made upstream. // honest in the logs; `open_video` cannot re-plan capture, so a warning is deliberately all
// it does.
if cursor_blend && !inner.caps().blends_cursor { if cursor_blend && !inner.caps().blends_cursor {
tracing::warn!( tracing::warn!(
backend, backend,
@@ -989,6 +990,87 @@ pub fn linux_native_nv12_ok(codec: Codec) -> bool {
} }
} }
/// Whether the encode backend this session will resolve to composites [`CapturedFrame::cursor`]
/// ([`EncoderCaps::blends_cursor`]) — answered BEFORE capture opens, so the host plans cursor
/// delivery honestly instead of discovering a cursorless stream after the fact (the
/// `blends_cursor` audit finding): a blend-capable backend takes cursor-as-metadata capture
/// (pointer-free frames + host composite on demand — the cursor channel's contract); for
/// anything else the host must have the compositor EMBED the pointer. The sibling verdict of
/// [`linux_native_nv12_ok`], threaded into the same negotiation.
///
/// `cuda_planned` is the caller's prediction of a CUDA capture payload (NVIDIA + zero-copy — the
/// prediction `SessionPlan` already makes); `ten_bit` the negotiated depth. Both shift the
/// dispatch: a CPU payload keeps NVIDIA on libav NVENC (no blend), and a 10-bit HDR session
/// skips Vulkan Video for libav VAAPI's P010/Main10 wiring (no blend).
#[cfg(target_os = "linux")]
pub fn cursor_blend_capable(codec: Codec, cuda_planned: bool, ten_bit: bool) -> bool {
// A negotiated PyroWave session routes to that backend before the pref is consulted
// (`open_video_backend_linux`), and its wavelet CSC composites the metadata cursor.
if codec == Codec::PyroWave {
return true;
}
let direct_nvenc = {
#[cfg(feature = "nvenc")]
{
nvenc_direct_enabled()
}
#[cfg(not(feature = "nvenc"))]
{
false
}
};
let vulkan_csc = {
// The compute-CSC arm — the one that blends. Eligibility mirrors `open_amd_intel`;
// the device probe runs last (it opens a Vulkan instance, cached per GPU+codec).
#[cfg(feature = "vulkan-encode")]
{
matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
&& vulkan_encode_available(codec)
}
#[cfg(not(feature = "vulkan-encode"))]
{
false
}
};
let backend = resolve_linux_backend(
pf_host_config::config().encoder_pref.as_str(),
linux_auto_is_vaapi,
cuda_planned,
);
cursor_blend_capable_for(backend, cuda_planned, ten_bit, direct_nvenc, vulkan_csc)
}
/// The dispatch-mirroring core of [`cursor_blend_capable`], device-free for the unit tests.
/// `direct_nvenc` = the direct-SDK NVENC path is compiled in and enabled; `vulkan_csc` = the
/// Vulkan Video compute-CSC arm (the one that blends) is compiled in, enabled, and
/// device-supported for the session's codec.
#[cfg(target_os = "linux")]
fn cursor_blend_capable_for(
backend: Option<LinuxBackend>,
cuda_planned: bool,
ten_bit: bool,
direct_nvenc: bool,
vulkan_csc: bool,
) -> bool {
match backend {
// The wavelet CSC composites the metadata cursor (`linux/pyrowave.rs`).
Some(LinuxBackend::Pyrowave) => true,
// Only the direct-SDK arm blends (VkSlotBlend), and it only takes CUDA payloads —
// a CPU-payload session stays on libav NVENC, which cannot blend.
Some(LinuxBackend::Nvenc) => cuda_planned && direct_nvenc,
// The Vulkan Video compute-CSC path blends; a 10-bit HDR session skips it for libav
// VAAPI (no blend). The session plan keeps a cursor-blend session off the native-NV12
// and RGB-direct shapes (`SessionPlan::output_format` / `VulkanVideoEncoder::open`),
// so CSC eligibility IS the answer.
Some(LinuxBackend::AmdIntel) | Some(LinuxBackend::Vulkan) => !ten_bit && vulkan_csc,
// CPU frames: the capturer composites the metadata cursor inline before the encoder
// runs, but the ENCODER blends nothing — the cursor channel's on-demand composite
// contract can't be honored. Report the encoder's truth.
Some(LinuxBackend::Software) | None => false,
}
}
/// Can this GPU + driver actually open a Vulkan Video **encode** session for `codec`? Cached per /// Can this GPU + driver actually open a Vulkan Video **encode** session for `codec`? Cached per
/// (selected GPU, codec) — the [`can_encode_10bit`] idiom, with the probe run outside the lock. /// (selected GPU, codec) — the [`can_encode_10bit`] idiom, with the probe run outside the lock.
/// ///
@@ -1754,6 +1836,72 @@ mod tests {
assert_eq!(none.wire_mask(), None); assert_eq!(none.wire_mask(), None);
} }
/// The cursor-blend capability mirror, arm by arm — the table the caps-aware negotiation
/// (cursor channel grant, metadata-vs-embedded capture) stands on. Each row names the arm's
/// blending stage or the reason there is none.
#[cfg(target_os = "linux")]
#[test]
fn cursor_blend_capability_mirrors_the_dispatch() {
use LinuxBackend::*;
// PyroWave: the wavelet CSC composites, always.
assert!(cursor_blend_capable_for(
Some(Pyrowave),
false,
false,
false,
false
));
// NVIDIA: only the direct-SDK arm blends (VkSlotBlend), and only for CUDA payloads.
assert!(cursor_blend_capable_for(
Some(Nvenc),
true,
false,
true,
false
));
assert!(
!cursor_blend_capable_for(Some(Nvenc), false, false, true, false),
"a CPU payload stays on libav NVENC, which cannot blend"
);
assert!(
!cursor_blend_capable_for(Some(Nvenc), true, false, false, false),
"PUNKTFUNK_NVENC_DIRECT=0 (or a build without the feature) is the libav path"
);
// AMD/Intel: the Vulkan Video compute-CSC arm blends; VAAPI never does.
assert!(cursor_blend_capable_for(
Some(AmdIntel),
false,
false,
false,
true
));
assert!(
!cursor_blend_capable_for(Some(AmdIntel), false, false, false, false),
"no eligible Vulkan CSC arm (H.264, PUNKTFUNK_VULKAN_ENCODE=0, unsupported \
device) resolves to libav VAAPI, which cannot blend"
);
assert!(
!cursor_blend_capable_for(Some(AmdIntel), false, true, false, true),
"a 10-bit HDR session skips Vulkan Video for VAAPI's P010 wiring — no blend"
);
assert!(cursor_blend_capable_for(
Some(Vulkan),
false,
false,
false,
true
));
// Software / unknown pref: CPU frames; the encoder blends nothing.
assert!(!cursor_blend_capable_for(
Some(Software),
false,
false,
true,
true
));
assert!(!cursor_blend_capable_for(None, false, false, true, true));
}
/// WP7.7 guard (the cheap half): every `Encoder` trait method must be explicitly forwarded by /// WP7.7 guard (the cheap half): every `Encoder` trait method must be explicitly forwarded by
/// `TrackedEncoder`. A defaulted trait method that isn't forwarded silently no-ops through the /// `TrackedEncoder`. A defaulted trait method that isn't forwarded silently no-ops through the
/// wrapper — the trap has bitten three times (`set_wire_chunking`'s §4.4 chunking probe, /// wrapper — the trap has bitten three times (`set_wire_chunking`'s §4.4 chunking probe,
+17 -3
View File
@@ -66,8 +66,14 @@ fn zero_copy_policy(
/// Open a live capturer for a client-sized monitor via the xdg ScreenCast portal. `want_hdr` /// Open a live capturer for a client-sized monitor via the xdg ScreenCast portal. `want_hdr`
/// offers the GNOME 50+ 10-bit PQ/BT.2020 formats (pass it only when the session negotiated HDR /// offers the GNOME 50+ 10-bit PQ/BT.2020 formats (pass it only when the session negotiated HDR
/// AND the mirrored monitor is in HDR mode — see [`pf_capture::gnome_hdr_monitor_active`]). /// AND the mirrored monitor is in HDR mode — see [`pf_capture::gnome_hdr_monitor_active`]).
/// `want_metadata_cursor` asks for cursor-as-metadata — pass it only when the session's encode
/// backend composites `CapturedFrame::cursor` (`encode::cursor_blend_capable`); otherwise the
/// portal embeds the pointer, so no backend × cursor-mode combination streams cursorless.
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
pub fn open_portal_monitor(want_hdr: bool) -> Result<Box<dyn Capturer>> { pub fn open_portal_monitor(
want_hdr: bool,
want_metadata_cursor: bool,
) -> Result<Box<dyn Capturer>> {
// On RemoteDesktop-capable desktops (KWin/GNOME) anchor ScreenCast to a RemoteDesktop // On RemoteDesktop-capable desktops (KWin/GNOME) anchor ScreenCast to a RemoteDesktop
// session so it inherits that grant headlessly; wlroots/Sway has no RemoteDesktop portal, // session so it inherits that grant headlessly; wlroots/Sway has no RemoteDesktop portal,
// so use a plain ScreenCast session there. // so use a plain ScreenCast session there.
@@ -76,11 +82,19 @@ pub fn open_portal_monitor(want_hdr: bool) -> Result<Box<dyn Capturer>> {
// passthrough is virtual-output-only; the global encoder-pref lever still applies inside. // passthrough is virtual-output-only; the global encoder-pref lever still applies inside.
// Native NV12 stays off too: the mirror path doesn't resolve the codec here, and the desktop // Native NV12 stays off too: the mirror path doesn't resolve the codec here, and the desktop
// compositors it mirrors (GNOME/KWin) don't produce NV12 anyway. // compositors it mirrors (GNOME/KWin) don't produce NV12 anyway.
pf_capture::open_portal_monitor(anchored, want_hdr, zero_copy_policy(false, false)) pf_capture::open_portal_monitor(
anchored,
want_hdr,
want_metadata_cursor,
zero_copy_policy(false, false),
)
} }
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
pub fn open_portal_monitor(_want_hdr: bool) -> Result<Box<dyn Capturer>> { pub fn open_portal_monitor(
_want_hdr: bool,
_want_metadata_cursor: bool,
) -> Result<Box<dyn Capturer>> {
anyhow::bail!("portal capture requires Linux (xdg-desktop-portal + PipeWire)") anyhow::bail!("portal capture requires Linux (xdg-desktop-portal + PipeWire)")
} }
+57 -15
View File
@@ -33,10 +33,14 @@ pub struct StreamConfig {
pub hdr: bool, pub hdr: bool,
} }
/// A pooled capturer plus the two PipeWire-negotiation-time properties reuse must match on —
/// its HDR-ness and its metadata-cursor mode; a mismatch on either needs a fresh screencast
/// session (see `AppState::video_cap`).
pub type PooledCapturer = (Box<dyn Capturer>, bool, bool);
/// Slot for the persistent screen capturer, shared with the control plane and reused across /// Slot for the persistent screen capturer, shared with the control plane and reused across
/// streams so a reconnect doesn't open a second (conflicting) screencast session. The `bool` is /// streams so a reconnect doesn't open a second (conflicting) screencast session.
/// the pooled capturer's HDR-ness (see `AppState::video_cap`). pub type CapturerSlot = Arc<std::sync::Mutex<Option<PooledCapturer>>>;
pub type CapturerSlot = Arc<std::sync::Mutex<Option<(Box<dyn Capturer>, bool)>>>;
/// A pending client reference-frame-invalidation range (lost `firstFrame..=lastFrame`), set by the /// A pending client reference-frame-invalidation range (lost `firstFrame..=lastFrame`), set by the
/// control plane and drained by the video thread (see [`AppState::rfi_range`](super::AppState)). /// control plane and drained by the video thread (see [`AppState::rfi_range`](super::AppState)).
@@ -136,7 +140,7 @@ fn run(
running: &Arc<AtomicBool>, running: &Arc<AtomicBool>,
force_idr: &AtomicBool, force_idr: &AtomicBool,
rfi_range: &std::sync::Mutex<Option<(i64, i64)>>, rfi_range: &std::sync::Mutex<Option<(i64, i64)>>,
video_cap: &std::sync::Mutex<Option<(Box<dyn Capturer>, bool)>>, video_cap: &std::sync::Mutex<Option<PooledCapturer>>,
// Shared stats recorder for the web-console capture/graph. Threaded into `stream_body` (the // Shared stats recorder for the web-console capture/graph. Threaded into `stream_body` (the
// encode loop); per-frame sample emission is wired by a later pass. // encode loop); per-frame sample emission is wired by a later pass.
stats: &Arc<crate::stats_recorder::StatsRecorder>, stats: &Arc<crate::stats_recorder::StatsRecorder>,
@@ -250,6 +254,12 @@ fn run(
return stream_body( return stream_body(
&mut capturer, &mut capturer,
Some(&rebuild), Some(&rebuild),
// The virtual-output source never selects cursor-as-metadata (`set_hw_cursor` is
// never called → the compositor EMBEDS the pointer where it can), so the encoder
// is handed nothing to composite. gamescope remains the pointerless residual —
// its capture carries no cursor either way (the native plane's XFixes source is
// not wired on this plane).
false,
&sock, &sock,
cfg, cfg,
running, running,
@@ -266,13 +276,34 @@ fn run(
// pooled capturer's HDR-ness matching this stream's negotiated `cfg.hdr` — the depth is a // pooled capturer's HDR-ness matching this stream's negotiated `cfg.hdr` — the depth is a
// PipeWire-negotiation-time property of the screencast session, so an HDR↔SDR change needs a // PipeWire-negotiation-time property of the screencast session, so an HDR↔SDR change needs a
// fresh session (same pattern as the audio capturer's channel-count gate). // fresh session (same pattern as the audio capturer's channel-count gate).
// Cursor-as-metadata only where the encode backend this session resolves to composites
// `frame.cursor` (the caps-aware negotiation — mirror of the native plane's); otherwise ask
// the portal to EMBED the pointer so no backend × cursor-mode combination streams
// cursorless. Synthetic frames carry no pointer either way.
let metadata_cursor = {
#[cfg(target_os = "linux")]
{
// Same CUDA-payload prediction SessionPlan/`handshake::cursor_forward` make:
// the NVIDIA resolution plus the zero-copy master switch.
let cuda_planned =
!crate::encode::linux_zero_copy_is_vaapi() && crate::zerocopy::enabled();
crate::encode::cursor_blend_capable(cfg.codec, cuda_planned, cfg.hdr)
}
#[cfg(not(target_os = "linux"))]
false
};
let pooled = match video_cap.lock().unwrap().take() { let pooled = match video_cap.lock().unwrap().take() {
Some((c, was_hdr)) if was_hdr == cfg.hdr => Some(c), Some((c, was_hdr, was_meta)) if was_hdr == cfg.hdr && was_meta == metadata_cursor => {
Some((c, was_hdr)) => { Some(c)
}
Some((c, was_hdr, was_meta)) => {
tracing::info!( tracing::info!(
was_hdr, was_hdr,
want_hdr = cfg.hdr, want_hdr = cfg.hdr,
"video source: pooled capturer depth mismatch — opening a fresh screencast session" was_metadata_cursor = was_meta,
want_metadata_cursor = metadata_cursor,
"video source: pooled capturer depth/cursor-mode mismatch — opening a fresh \
screencast session"
); );
drop(c); drop(c);
None None
@@ -285,8 +316,13 @@ fn run(
c c
} }
None if pf_host_config::config().video_source.as_deref() == Some("portal") => { None if pf_host_config::config().video_source.as_deref() == Some("portal") => {
tracing::info!(hdr = cfg.hdr, "video source: portal desktop capture"); tracing::info!(
capture::open_portal_monitor(cfg.hdr).context("open portal capturer")? hdr = cfg.hdr,
metadata_cursor,
"video source: portal desktop capture"
);
capture::open_portal_monitor(cfg.hdr, metadata_cursor)
.context("open portal capturer")?
} }
None => { None => {
tracing::info!("video source: synthetic test pattern"); tracing::info!("video source: synthetic test pattern");
@@ -298,6 +334,7 @@ fn run(
let result = stream_body( let result = stream_body(
&mut capturer, &mut capturer,
None, None,
metadata_cursor,
&sock, &sock,
cfg, cfg,
running, running,
@@ -314,9 +351,10 @@ fn run(
// point — and this path has no rebuild closure (unlike the virtual-output path above), so a // point — and this path has no rebuild closure (unlike the virtual-output path above), so a
// re-admitted dead capturer wedged GameStream portal video permanently, at 10 s per reconnect // re-admitted dead capturer wedged GameStream portal video permanently, at 10 s per reconnect
// attempt. Dropping it instead costs one fresh screencast session on the next connect. Note // attempt. Dropping it instead costs one fresh screencast session on the next connect. Note
// `result` may already be `Err` here, which is itself that signal. // `result` may already be `Err` here, which is itself that signal. (`metadata_cursor` rides
// along as the second reuse key, beside HDR-ness — see `PooledCapturer`.)
if result.is_ok() && capturer.is_alive() { if result.is_ok() && capturer.is_alive() {
*video_cap.lock().unwrap() = Some((capturer, cfg.hdr)); *video_cap.lock().unwrap() = Some((capturer, cfg.hdr, metadata_cursor));
} else { } else {
tracing::info!( tracing::info!(
stream_failed = result.is_err(), stream_failed = result.is_err(),
@@ -662,6 +700,10 @@ fn stream_body(
// Re-open the video source on capture loss (virtual-display path → follow a Desktop<->Game switch); // Re-open the video source on capture loss (virtual-display path → follow a Desktop<->Game switch);
// `None` for the portal/synthetic source, which has nothing to re-detect (propagate the error). // `None` for the portal/synthetic source, which has nothing to re-detect (propagate the error).
rebuild: Option<&dyn Fn() -> Result<Box<dyn Capturer>>>, rebuild: Option<&dyn Fn() -> Result<Box<dyn Capturer>>>,
// The capture hands the encoder cursor bitmaps to composite (cursor-as-metadata negotiated
// because the resolved backend blends — see the callers). `false` = the pointer is embedded
// in the pixels (or absent), so the encoder is asked to composite nothing.
cursor_blend: bool,
sock: &UdpSocket, sock: &UdpSocket,
cfg: StreamConfig, cfg: StreamConfig,
running: &Arc<AtomicBool>, running: &Arc<AtomicBool>,
@@ -697,9 +739,9 @@ fn stream_body(
// GameStream/Moonlight stays 4:2:0 — stock Moonlight clients can't decode 4:4:4, and the // GameStream/Moonlight stays 4:2:0 — stock Moonlight clients can't decode 4:4:4, and the
// Windows IDD-push capturer can't yet deliver full-chroma frames. 4:4:4 is punktfunk/1-native only. // Windows IDD-push capturer can't yet deliver full-chroma frames. 4:4:4 is punktfunk/1-native only.
encode::ChromaFormat::Yuv420, encode::ChromaFormat::Yuv420,
// Desktop monitor capture negotiates cursor-as-metadata where available — the encoder // True only when THIS session's capture negotiated cursor-as-metadata — which the
// may be handed cursor bitmaps to composite. // callers grant only where the resolved backend composites (`cursor_blend_capable`).
true, cursor_blend,
) )
.context("open video encoder for stream")?; .context("open video encoder for stream")?;
// Tell the encoder how deep the capturer lets it pipeline. Without this an in-place backend // Tell the encoder how deep the capturer lets it pipeline. Without this an in-place backend
@@ -871,7 +913,7 @@ fn stream_body(
frame.is_cuda(), frame.is_cuda(),
gs_bit_depth(frame.format), gs_bit_depth(frame.format),
encode::ChromaFormat::Yuv420, // GameStream stays 4:2:0 encode::ChromaFormat::Yuv420, // GameStream stays 4:2:0
true, // metadata-cursor capture — see the first open cursor_blend, // same capture cursor mode — see the first open
) )
.context("reopen encoder after rebuild")?; .context("reopen encoder after rebuild")?;
// A rebuilt encoder starts unconfigured — same reason as the first open above. // A rebuilt encoder starts unconfigured — same reason as the first open above.
+4 -3
View File
@@ -1012,9 +1012,10 @@ async fn serve_session(
// just never fires then. // just never fires then.
let (cursor_shape_tx, cursor_shape_rx) = let (cursor_shape_tx, cursor_shape_rx) =
tokio::sync::mpsc::unbounded_channel::<punktfunk_core::quic::CursorShape>(); tokio::sync::mpsc::unbounded_channel::<punktfunk_core::quic::CursorShape>();
// Negotiated cursor forwarding: MUST match the HOST_CAP_CURSOR bit the Welcome advertised // Negotiated cursor forwarding: the HOST_CAP_CURSOR bit the Welcome advertised, read back
// (handshake::cursor_forward is the single predicate both read). // rather than recomputed (`handshake::cursor_forward` computed it once, with the encoder
let cursor_forward = handshake::cursor_forward(hello.client_caps, compositor); // blend-capability gate — re-running it here could drift, and would re-probe).
let cursor_forward = welcome.host_caps & punktfunk_core::quic::HOST_CAP_CURSOR != 0;
// Who renders the pointer RIGHT NOW (client `CursorRenderMode`, flipped live by the mouse- // Who renders the pointer RIGHT NOW (client `CursorRenderMode`, flipped live by the mouse-
// model chord): `true` = client draws (exclude + forward), `false` = host composites (the // model chord): `true` = client draws (exclude + forward), `false` = host composites (the
// capture model). Starts true — the pre-message behavior for cap sessions. Control task // capture model). Starts true — the pre-message behavior for cap sessions. Control task
+28 -12
View File
@@ -12,31 +12,46 @@ use super::*;
/// the client asked ([`CLIENT_CAP_CURSOR`](punktfunk_core::quic::CLIENT_CAP_CURSOR)) AND the /// the client asked ([`CLIENT_CAP_CURSOR`](punktfunk_core::quic::CLIENT_CAP_CURSOR)) AND the
/// capture path can deliver cursor metadata separately from the frame — the Linux portal /// capture path can deliver cursor metadata separately from the frame — the Linux portal
/// `SPA_META_Cursor` path (not gamescope, whose capture paints no cursor at all), or Windows /// `SPA_META_Cursor` path (not gamescope, whose capture paints no cursor at all), or Windows
/// with a proto-v5 pf-vdisplay driver (the IddCx hardware-cursor channel, M2c). THE single /// with a proto-v5 pf-vdisplay driver (the IddCx hardware-cursor channel, M2c) — AND, on
/// predicate: the Welcome's `HOST_CAP_CURSOR` bit and the session's forwarding/blend-off /// Linux, the encode backend this session resolves to can composite the pointer on demand
/// wiring both read it, so they can never disagree. /// (`encode::cursor_blend_capable`): the channel's capture-mouse flip (`CursorRenderMode`,
/// `client_draws = false`) makes the HOST draw the pointer, and on Linux the encoder is that
/// compositing stage — granting the channel over a backend that can't blend (libav
/// VAAPI/NVENC, software) shipped a cursorless stream on every capture-mode flip. Denied, the
/// session keeps the pre-channel path: the compositor EMBEDS the pointer and the client never
/// draws — never cursorless, never doubled. THE single predicate: the Welcome's
/// `HOST_CAP_CURSOR` bit is computed from it, and the session wiring reads that bit back.
pub(super) fn cursor_forward( pub(super) fn cursor_forward(
client_caps: u8, client_caps: u8,
compositor: Option<crate::vdisplay::Compositor>, compositor: Option<crate::vdisplay::Compositor>,
codec: crate::encode::Codec,
bit_depth: u8,
) -> bool { ) -> bool {
if client_caps & punktfunk_core::quic::CLIENT_CAP_CURSOR == 0 { if client_caps & punktfunk_core::quic::CLIENT_CAP_CURSOR == 0 {
return false; return false;
} }
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
{ {
// CUDA-payload prediction — the same one `SessionPlan` makes: the NVIDIA resolution
// plus the zero-copy master switch. It decides direct-SDK NVENC (blends) vs libav
// NVENC (doesn't) inside the capability mirror.
let cuda_planned = !crate::encode::linux_zero_copy_is_vaapi() && crate::zerocopy::enabled();
compositor.is_some_and(|c| c != crate::vdisplay::Compositor::Gamescope) compositor.is_some_and(|c| c != crate::vdisplay::Compositor::Gamescope)
&& crate::encode::cursor_blend_capable(codec, cuda_planned, bit_depth == 10)
} }
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
{ {
// Windows (M2c): the pf-vdisplay driver must speak the v5 hardware-cursor channel — // Windows (M2c): the pf-vdisplay driver must speak the v5 hardware-cursor channel —
// DWM composites the pointer into the IDD frame otherwise, and forwarding a second // DWM composites the pointer into the IDD frame otherwise, and forwarding a second
// copy would double it. The probe latches by opening the control device once. // copy would double it. The probe latches by opening the control device once. The
let _ = compositor; // encoder is deliberately NOT consulted: the IDD capturer itself composites on the
// capture-mouse flip (`set_cursor_forward`), so no Windows encode backend blends.
let _ = (compositor, codec, bit_depth);
crate::vdisplay::manager::hw_cursor_capable() crate::vdisplay::manager::hw_cursor_capable()
} }
#[cfg(not(any(target_os = "linux", target_os = "windows")))] #[cfg(not(any(target_os = "linux", target_os = "windows")))]
{ {
let _ = compositor; let _ = (compositor, codec, bit_depth);
false false
} }
} }
@@ -493,9 +508,10 @@ pub(super) async fn negotiate(
0 0
} }
// Cursor channel granted (client asked + this capture path can deliver cursor // Cursor channel granted (client asked + this capture path can deliver cursor
// metadata out of the frame) — the client turns its local renderer on ONLY when // metadata out of the frame + the resolved encoder can composite on the
// it sees this bit, and serve_session wires forwarding from the same predicate. // capture-mouse flip) — the client turns its local renderer on ONLY when it sees
| if cursor_forward(hello.client_caps, compositor) { // this bit, and serve_session wires forwarding by reading the bit back.
| if cursor_forward(hello.client_caps, compositor, codec, bit_depth) {
punktfunk_core::quic::HOST_CAP_CURSOR punktfunk_core::quic::HOST_CAP_CURSOR
} else { } else {
0 0
@@ -541,9 +557,9 @@ pub(super) async fn negotiate(
let (ctx_tx, ctx_rx) = std::sync::mpsc::sync_channel::<SessionContext>(1); let (ctx_tx, ctx_rx) = std::sync::mpsc::sync_channel::<SessionContext>(1);
let client_identity = endpoint::peer_fingerprint(conn); let client_identity = endpoint::peer_fingerprint(conn);
let client_hdr = hello.display_hdr; let client_hdr = hello.display_hdr;
// Same predicate the Welcome's HOST_CAP_CURSOR bit used — the prepared display and // The bit the Welcome just advertised — read back rather than recomputed, so the
// the session wiring must agree with what we just advertised. // prepared display and the session wiring cannot disagree with it.
let cursor_fw = cursor_forward(hello.client_caps, Some(comp)); let cursor_fw = welcome.host_caps & punktfunk_core::quic::HOST_CAP_CURSOR != 0;
let (mode, shard_payload) = (hello.mode, welcome.shard_payload); let (mode, shard_payload) = (hello.mode, welcome.shard_payload);
let trace = bringup.clone(); let trace = bringup.clone();
std::thread::Builder::new() std::thread::Builder::new()
+15 -5
View File
@@ -1022,8 +1022,13 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// pointer compositor-EMBEDDED (`vd.set_hw_cursor(false)` → no cursor metadata, nothing to // pointer compositor-EMBEDDED (`vd.set_hw_cursor(false)` → no cursor metadata, nothing to
// blend), keeping the zero-cost pre-channel path. gamescope is the exception (Phase C): // blend), keeping the zero-cost pre-channel path. gamescope is the exception (Phase C):
// it can't embed the pointer, so the host ALWAYS composites the XFixes-sourced cursor — // it can't embed the pointer, so the host ALWAYS composites the XFixes-sourced cursor —
// the blend must be built for every gamescope session. // the blend must be built for every gamescope session. (`cursor_forward` is already
ctx.compositor == pf_vdisplay::Compositor::Gamescope || ctx.cursor_forward, // blend-gated: `handshake::cursor_forward` grants the channel only where
// `encode::cursor_blend_capable` says the resolved backend composites.)
crate::session_plan::cursor_blend_for(
ctx.cursor_forward,
ctx.compositor == pf_vdisplay::Compositor::Gamescope,
),
ctx.cursor_forward, ctx.cursor_forward,
); );
// gamescope: the XFixes cursor source feeds the always-on composite (Phase C). Set after // gamescope: the XFixes cursor source feeds the always-on composite (Phase C). Set after
@@ -2098,8 +2103,10 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// capture-mode channel); a switch AWAY restores the prior // capture-mode channel); a switch AWAY restores the prior
// gating. `plan` is `Copy` — this is the value the rebuild // gating. `plan` is `Copy` — this is the value the rebuild
// (and its `build_pipeline` attach) reads. // (and its `build_pipeline` attach) reads.
plan.cursor_blend = plan.cursor_forward plan.cursor_blend = crate::session_plan::cursor_blend_for(
|| c == crate::vdisplay::Compositor::Gamescope; plan.cursor_forward,
c == crate::vdisplay::Compositor::Gamescope,
);
plan.gamescope_cursor = plan.gamescope_cursor =
c == crate::vdisplay::Compositor::Gamescope; c == crate::vdisplay::Compositor::Gamescope;
gamescope_composite = gamescope_composite =
@@ -2987,7 +2994,10 @@ pub(super) fn prepare_display(
// non-gamescope sessions get the pointer compositor-EMBEDDED, nothing to blend; the // non-gamescope sessions get the pointer compositor-EMBEDDED, nothing to blend; the
// mid-stream `CursorRenderMode` flip strips/keeps `frame.cursor` per tick for channel // mid-stream `CursorRenderMode` flip strips/keeps `frame.cursor` per tick for channel
// sessions). gamescope (Phase C) can't embed → always composites the XFixes cursor. // sessions). gamescope (Phase C) can't embed → always composites the XFixes cursor.
compositor == pf_vdisplay::Compositor::Gamescope || cursor_forward, crate::session_plan::cursor_blend_for(
cursor_forward,
compositor == pf_vdisplay::Compositor::Gamescope,
),
cursor_forward, cursor_forward,
); );
plan.gamescope_cursor = compositor == pf_vdisplay::Compositor::Gamescope; plan.gamescope_cursor = compositor == pf_vdisplay::Compositor::Gamescope;
+35 -9
View File
@@ -104,9 +104,13 @@ pub struct SessionPlan {
/// AUs stay shard-aligned across mode/bitrate/stall rebuilds. `None` for the H.26x codecs. /// AUs stay shard-aligned across mode/bitrate/stall rebuilds. `None` for the H.26x codecs.
pub wire_chunk: Option<usize>, pub wire_chunk: Option<usize>,
/// The session may hand the encoder cursor bitmaps to composite (cursor-as-metadata /// The session may hand the encoder cursor bitmaps to composite (cursor-as-metadata
/// captures — every non-gamescope compositor; gamescope embeds the pointer itself). /// captures). Set via [`cursor_blend_for`] — the single platform rule — so it is `true` only
/// Encoders whose fast path cannot blend (the Vulkan EFC RGB-direct source) stay on their /// where the ENCODER is the compositing stage (Linux cursor-forward and gamescope sessions);
/// blending path when this is set, so the pointer never silently vanishes from the stream. /// Windows is always `false` (the IDD capturer composites the pointer itself). Encoders
/// whose fast path cannot blend (the Vulkan EFC RGB-direct source, native NV12) stay off
/// those shapes when this is set — see [`Self::output_format`] and
/// `encode::cursor_blend_capable`, the pre-open mirror that gates the cursor channel — so
/// the pointer never silently vanishes from the stream.
pub cursor_blend: bool, pub cursor_blend: bool,
/// The session negotiated the cursor-forward channel (M2/M2c): the client draws the pointer /// The session negotiated the cursor-forward channel (M2/M2c): the client draws the pointer
/// locally, so `cursor_blend` is off AND (on Windows) the capturer sets the driver's /// locally, so `cursor_blend` is off AND (on Windows) the capturer sets the driver's
@@ -198,13 +202,15 @@ impl SessionPlan {
// Producer-native NV12 (gamescope) is consumable only by the Linux Vulkan Video // Producer-native NV12 (gamescope) is consumable only by the Linux Vulkan Video
// backend — resolved HERE from the plan's codec so the capturer never reaches back // backend — resolved HERE from the plan's codec so the capturer never reaches back
// into encode (the same one-way edge as `gpu` above). BUT the native-NV12 encode path // into encode (the same one-way edge as `gpu` above). BUT the native-NV12 encode path
// has no CSC stage to fold the cursor into (it assumes gamescope embeds its pointer, // has no CSC stage to fold the cursor into — so ANY cursor-compositing session
// which it does NOT into the PipeWire node) — so a gamescope-cursor session (Phase C) // (gamescope Phase C, whose XFixes pointer is absent from the PipeWire node, AND a
// must capture RGB instead, routing to the compute-CSC / VkSlotBlend blend that draws // cursor-forward session, whose capture-mouse flip needs the host composite on
// `frame.cursor`. Costs the RGB→NV12 CSC we'd otherwise skip; the native-NV12 cursor // demand) must capture RGB instead, routing to the compute-CSC / VkSlotBlend blend
// blend is the perf-preserving follow-up. // that draws `frame.cursor`. Costs the RGB→NV12 CSC we'd otherwise skip; the
// native-NV12 cursor blend is the perf-preserving follow-up. (`cursor_blend`
// subsumes `gamescope_cursor` — see [`cursor_blend_for`].)
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
nv12_native: crate::encode::linux_native_nv12_ok(self.codec) && !self.gamescope_cursor, nv12_native: crate::encode::linux_native_nv12_ok(self.codec) && !self.cursor_blend,
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
nv12_native: false, nv12_native: false,
} }
@@ -217,6 +223,26 @@ pub(crate) fn resolve_topology() -> SessionTopology {
SessionTopology::SingleProcess SessionTopology::SingleProcess
} }
/// THE rule for [`SessionPlan::cursor_blend`], shared by every resolve caller (initial plan and
/// the mid-stream compositor re-gate) so they can't drift:
/// * **Linux**: the encoder is the compositing stage — blend for a cursor-forward session (the
/// capture-mouse flip needs the host composite on demand) and for gamescope (its capture
/// carries no pointer at all; the XFixes-sourced cursor must be drawn into the video).
/// * **Windows**: never — the IDD capturer composites the pointer itself (`cursor_blend.rs` /
/// DWM), and no Windows encode backend reads `frame.cursor`. Asking the encoder anyway made
/// `open_video`'s blends-cursor backstop fire spuriously on every cursor-channel session.
pub(crate) fn cursor_blend_for(cursor_forward: bool, gamescope: bool) -> bool {
#[cfg(target_os = "windows")]
{
let _ = (cursor_forward, gamescope);
false
}
#[cfg(not(target_os = "windows"))]
{
cursor_forward || gamescope
}
}
#[cfg(target_os = "windows")] #[cfg(target_os = "windows")]
fn resolve_encoder() -> EncoderBackend { fn resolve_encoder() -> EncoderBackend {
match crate::encode::windows_resolved_backend() { match crate::encode::windows_resolved_backend() {
+3 -1
View File
@@ -94,7 +94,9 @@ pub fn run(opts: Options) -> Result<()> {
want_hdr, want_hdr,
"spike source: xdg ScreenCast portal (live monitor)" "spike source: xdg ScreenCast portal (live monitor)"
); );
capture::open_portal_monitor(want_hdr).context("open portal capturer")? // Embedded cursor: the spike passes `cursor_blend = false` to its encoder open, so
// a metadata pointer would be composited by nothing.
capture::open_portal_monitor(want_hdr, false).context("open portal capturer")?
} }
Source::KwinVirtual => { Source::KwinVirtual => {
let compositor = crate::vdisplay::detect().unwrap_or(crate::vdisplay::Compositor::Kwin); let compositor = crate::vdisplay::detect().unwrap_or(crate::vdisplay::Compositor::Kwin);