feat(hdr): GNOME 50 HDR screencast capture + Linux Main10 encode
ci / docs-site (push) Successful in 58s
ci / rust (push) Failing after 1m3s
ci / web (push) Successful in 1m4s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 19s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 46s
windows-host / package (push) Failing after 5m38s
ci / bench (push) Successful in 6m12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m38s
apple / screenshots (push) Successful in 6m25s
docker / deploy-docs (push) Successful in 29s
deb / build-publish-host (push) Failing after 7m30s
deb / build-publish (push) Successful in 11m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m59s
arch / build-publish (push) Successful in 11m57s
android / android (push) Successful in 15m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m46s

GNOME 50 (Mutter MR 4928, PipeWire >= 1.6) added HDR screen sharing for
monitor streams: 10-bit PQ formats (xRGB_210LE/xBGR_210LE) with MANDATORY
BT.2020 + SMPTE-2084 colorimetry props, advertised while the mirrored
monitor is in BT.2100 colour mode. Wire the Linux host into it end-to-end
on the GameStream desktop-mirror path (PUNKTFUNK_VIDEO_SOURCE=portal):

* pf-frame: PixelFormat::X2Rgb10/X2Bgr10 (DRM XR30/XB30; X2Bgr10 is the
  Windows Rgb10a2 layout) + fourccs.
* pf-capture: want_hdr portal offer — HDR-only LINEAR-dmabuf pods with
  MANDATORY PQ/BT.2020 props (SHM excluded: Mutter's SHM record path
  paints 8-bit ARGB32 regardless of format; tiled excluded: the EGL
  de-tile blit is 8-bit RGBA8), negotiated-colorimetry parse, generic
  HDR10 hdr_meta(), packed-10-bit CPU cursor blend, a process-wide SDR
  downgrade latch on negotiation timeout, and a DisplayConfig BT.2100
  colour-mode probe (gnome_hdr_monitor_active).
* pf-encode: libav NVENC X2RGB10->P010 swscale (BT.2020 limited) ->
  HEVC Main10 / 10-bit AV1 with PQ VUI; VAAPI 10-bit on both paths (CPU
  P010 upload + dmabuf XR30 scale_vaapi p010/bt2020); can_encode_10bit
  now probes for real on Linux; 10-bit sessions route around the
  8-bit-only Vulkan-video/direct-NVENC backends.
* GameStream: host_hdr_capable() Linux arm, live monitor-HDR check at
  RTSP honor time, capturer-pool reuse keyed on HDR-ness, gs_bit_depth
  covers the new formats. New `punktfunk-host hdr-probe` diagnostic and
  a PUNKTFUNK_SPIKE_HDR spike lever.
* Native plane stays honestly 8-bit via capturer_supports_hdr(): Mutter
  RecordVirtual streams are SDR-only upstream (GNOME 50 and 51-dev), so
  virtual-display sources cannot deliver HDR yet.

Validated on the RTX 5070 Ti (GNOME 50.3 / PipeWire 1.6.8): the Main10
probes pass and the ignored nvenc_hdr10_smoke GPU test emits an IDR that
ffprobe reads as Main 10 / yuv420p10le / bt2020nc / smpte2084 / limited.
Live HDR capture negotiation still needs an HDR monitor on glass; VAAPI
10-bit needs the AMD box.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-19 09:30:44 +02:00
parent 4f64125025
commit 0e977817f9
19 changed files with 995 additions and 175 deletions
+64 -4
View File
@@ -254,6 +254,54 @@ pub struct ZeroCopyPolicy {
pub fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
true
}
/// Whether the **native-plane** capturer (a compositor virtual output) can deliver an HDR (10-bit
/// PQ/BT.2020) source on this platform — the capture-side gate the punktfunk/1 handshake consults
/// before negotiating 10-bit (mirroring [`capturer_supports_444`]).
///
/// Linux: `false`. GNOME 50 added HDR **screen sharing** for *monitor* streams only — Mutter's
/// `RecordVirtual` virtual-monitor streams advertise 8-bit BGRx/BGRA exclusively (still true on
/// the GNOME 51 dev branch), and virtual outputs report no BT2020/PQ colour capabilities, so they
/// can't be flipped into HDR mode via DisplayConfig either. The Linux HDR path that DOES exist —
/// the GNOME 50+ portal **monitor mirror** (`open_portal_monitor` with `want_hdr`) — is gated
/// separately by the GameStream plane (`host_hdr_capable` + the live monitor colour-mode probe).
#[cfg(target_os = "linux")]
pub fn capturer_supports_hdr() -> bool {
false
}
/// Windows: the IDD-push capturer proactively enables advanced colour and delivers P010/Rgb10a2.
#[cfg(target_os = "windows")]
pub fn capturer_supports_hdr() -> bool {
true
}
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
pub fn capturer_supports_hdr() -> bool {
false
}
/// Process-wide latch: a `want_hdr` portal capture failed to negotiate the HDR (10-bit PQ) offer —
/// the compositor never accepted it (monitor left HDR mode between the probe and the negotiation,
/// NVIDIA EGL not listing LINEAR for XR30, a pre-50 Mutter…). Later sessions consult
/// [`hdr_capture_failed`] and fall back to the SDR offer instead of re-running the same doomed
/// 10-second negotiation timeout on every reconnect. Sticky until host restart (matching the
/// zero-copy downgrade latches); the log line at latch time says so.
#[cfg(target_os = "linux")]
static HDR_CAPTURE_FAILED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
#[cfg(target_os = "linux")]
pub fn hdr_capture_failed() -> bool {
HDR_CAPTURE_FAILED.load(std::sync::atomic::Ordering::Relaxed)
}
#[cfg(target_os = "linux")]
pub(crate) fn note_hdr_capture_failed() {
if !HDR_CAPTURE_FAILED.swap(true, std::sync::atomic::Ordering::Relaxed) {
tracing::warn!(
"HDR capture negotiation failed — this host will offer SDR capture for the rest of \
the process lifetime (restart the host after fixing the monitor's HDR mode to retry)"
);
}
}
#[cfg(target_os = "windows")]
pub fn capturer_supports_444(encoder_ingests_rgb_444: bool) -> bool {
// IDD-push delivers full-chroma BGRA for an SDR 4:4:4 session (skipping the NV12 VideoConverter),
@@ -316,16 +364,28 @@ pub use idd_push::verify_is_wudfhost;
#[cfg(target_os = "linux")]
#[path = "linux/mod.rs"]
mod linux;
// The GNOME BT.2100 colour-mode probe — the host's capture-side gate for offering HDR on the
// portal monitor path (see `open_portal_monitor`'s `want_hdr`).
#[cfg(target_os = "linux")]
pub use linux::gnome_hdr_monitor_active;
#[cfg(target_os = "windows")]
#[path = "windows/synthetic_nv12.rs"]
pub mod synthetic_nv12;
/// Open the Linux xdg-ScreenCast portal capturer for a client-sized monitor. `anchored` drives
/// ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits that grant headlessly. The
/// [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts (the one-way edge).
/// ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits that grant headlessly.
/// `want_hdr` offers the GNOME 50+ HDR formats (10-bit PQ/BT.2020 dmabufs) instead of the SDR
/// set — pass it only when the mirrored monitor is actually in HDR mode (the host probes
/// DisplayConfig) or the negotiation runs into its 10 s timeout and latches the SDR downgrade.
/// The [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts (the one-way edge).
#[cfg(target_os = "linux")]
pub fn open_portal_monitor(anchored: bool, policy: ZeroCopyPolicy) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::open(anchored, policy).map(|c| Box::new(c) as Box<dyn Capturer>)
pub fn open_portal_monitor(
anchored: bool,
want_hdr: bool,
policy: ZeroCopyPolicy,
) -> Result<Box<dyn Capturer>> {
linux::PortalCapturer::open(anchored, want_hdr && !hdr_capture_failed(), policy)
.map(|c| Box::new(c) as Box<dyn Capturer>)
}
/// Open the Linux portal capturer bound to an already-created virtual output's PipeWire node. The
+340 -20
View File
@@ -62,6 +62,13 @@ pub struct PortalCapturer {
/// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
/// rebuild retries on the CPU offer instead of failing identically forever.
vaapi_dmabuf: bool,
/// This capture ran the HDR (10-bit PQ/BT.2020 dmabuf) offer — see [`Self::open`]'s
/// `want_hdr`. Read by the negotiation-timeout diagnosis (a failed HDR offer latches the
/// process-wide SDR downgrade) and by [`hdr_meta`](Capturer::hdr_meta).
hdr_offer: bool,
/// Set once the stream negotiated one of the 10-bit PQ formats (`param_changed`), i.e. frames
/// really are PQ/BT.2020 — drives [`hdr_meta`](Capturer::hdr_meta).
hdr_negotiated: Arc<AtomicBool>,
/// The PipeWire node this capturer consumes — surfaced in error messages for diagnosis.
node_id: u32,
/// Stops the PipeWire loop on teardown (sent in `Drop`). Without it a dropped or failed
@@ -80,8 +87,9 @@ pub struct PortalCapturer {
impl PortalCapturer {
/// `anchored` drives ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits the
/// RemoteDesktop grant and never raises a separate ScreenCast dialog; `false` uses a plain
/// ScreenCast session (wlroots, which has no RemoteDesktop portal).
pub fn open(anchored: bool, policy: ZeroCopyPolicy) -> Result<PortalCapturer> {
/// ScreenCast session (wlroots, which has no RemoteDesktop portal). `want_hdr` offers the
/// GNOME 50+ HDR formats (10-bit PQ/BT.2020, dmabuf-only) instead of the SDR set.
pub fn open(anchored: bool, want_hdr: bool, policy: ZeroCopyPolicy) -> Result<PortalCapturer> {
// Portal handshake (async) on its own thread; hands back the PW fd + node id.
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>();
thread::Builder::new()
@@ -102,11 +110,12 @@ impl PortalCapturer {
};
tracing::info!(
node_id,
want_hdr,
"ScreenCast portal session started; connecting PipeWire"
);
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
Ok(
spawn_pipewire(Some(fd), node_id, None, true, false, policy)?
spawn_pipewire(Some(fd), node_id, None, true, false, want_hdr, policy)?
.into_capturer(node_id, None),
)
}
@@ -135,12 +144,15 @@ impl PortalCapturer {
want_444,
"connecting PipeWire to virtual output"
);
// Virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise 8-bit
// BGRx/BGRA exclusively, GNOME 50 and 51-dev alike) — never run the HDR offer here.
Ok(spawn_pipewire(
remote_fd,
node_id,
preferred_mode,
allow_zerocopy,
want_444,
false,
policy,
)?
.into_capturer(node_id, Some(keepalive)))
@@ -160,6 +172,10 @@ struct PwHandles {
/// This capture will offer LINEAR-dmabuf-only for the VAAPI passthrough (see
/// [`PortalCapturer::vaapi_dmabuf`]).
vaapi_dmabuf: bool,
/// This capture ran the HDR offer (see [`PortalCapturer::hdr_offer`]).
hdr_offer: bool,
/// See [`PortalCapturer::hdr_negotiated`].
hdr_negotiated: Arc<AtomicBool>,
quit: ::pipewire::channel::Sender<()>,
join: thread::JoinHandle<()>,
}
@@ -177,6 +193,8 @@ impl PwHandles {
broken: self.broken,
stall_since: None,
vaapi_dmabuf: self.vaapi_dmabuf,
hdr_offer: self.hdr_offer,
hdr_negotiated: self.hdr_negotiated,
node_id,
quit: Some(self.quit),
join: Some(self.join),
@@ -199,6 +217,10 @@ fn spawn_pipewire(
// 4:4:4 session: tiled dmabufs convert to planar YUV444 on the GPU (`ImportKind::Tiled444`)
// instead of NV12/RGB, so the session stays zero-copy at full chroma.
want_444: bool,
// HDR session (GNOME 50+ monitor mirror): offer ONLY the 10-bit PQ/BT.2020 formats as
// LINEAR dmabufs (SHM can't carry them — Mutter's SHM record path paints 8-bit ARGB32
// regardless of the negotiated format, and the tiled EGL de-tile blit is 8-bit).
want_hdr: bool,
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
// capture→encode edge (plan §W6).
policy: ZeroCopyPolicy,
@@ -213,17 +235,29 @@ fn spawn_pipewire(
let streaming_cb = streaming.clone();
let broken = Arc::new(AtomicBool::new(false));
let broken_cb = broken.clone();
let hdr_negotiated = Arc::new(AtomicBool::new(false));
let hdr_negotiated_cb = hdr_negotiated.clone();
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
// inner `mod pipewire` shadows the crate name at this scope.
let (quit_tx, quit_rx) = ::pipewire::channel::channel::<()>();
let zerocopy = allow_zerocopy && pf_zerocopy::enabled();
// HDR cannot ride the SHM path (see `want_hdr` above): under PUNKTFUNK_FORCE_SHM the HDR
// offer is dropped — SDR capture, loudly.
let force_shm = std::env::var("PUNKTFUNK_FORCE_SHM").as_deref() == Ok("1");
let want_hdr = if want_hdr && force_shm {
tracing::warn!(
"HDR capture requested but PUNKTFUNK_FORCE_SHM=1 — the SHM path is 8-bit only; \
offering SDR"
);
false
} else {
want_hdr
};
// Mirror of the thread's `vaapi_passthrough` decision (deterministic from here: on a VAAPI
// backend the EGL→CUDA importer is never built) — kept on the capturer so `next_frame`'s
// negotiation-timeout branch knows a failed negotiation was the LINEAR-dmabuf offer.
let vaapi_dmabuf = zerocopy
&& std::env::var("PUNKTFUNK_FORCE_SHM").as_deref() != Ok("1")
&& policy.backend_is_vaapi;
let vaapi_dmabuf = zerocopy && !force_shm && policy.backend_is_vaapi;
let join = thread::Builder::new()
.name("punktfunk-pipewire".into())
.spawn(move || {
@@ -235,8 +269,10 @@ fn spawn_pipewire(
negotiated_cb,
streaming_cb,
broken_cb,
hdr_negotiated_cb,
zerocopy,
want_444,
want_hdr,
preferred,
quit_rx,
policy,
@@ -252,6 +288,8 @@ fn spawn_pipewire(
streaming,
broken,
vaapi_dmabuf,
hdr_offer: want_hdr,
hdr_negotiated,
quit: quit_tx,
join,
})
@@ -354,6 +392,26 @@ impl Capturer for PortalCapturer {
fn set_active(&self, active: bool) {
self.active.store(active, Ordering::Relaxed);
}
/// Generic HDR10 mastering metadata once the stream negotiated a 10-bit PQ format. Mutter
/// exposes no per-monitor mastering volume through the screencast, so this is the standard
/// HDR10 default block (BT.2020 primaries, D65 white, 1000 / 0.005 cd/m², CLL unknown) — the
/// same fallback Windows uses when a display reports nothing. The native stream loop prefers
/// the client display's own volume when the client sent one (`Hello::display_hdr`).
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
if !self.hdr_negotiated.load(Ordering::Relaxed) {
return None;
}
Some(punktfunk_core::quic::HdrMeta {
// ST.2086 order G, B, R; (x, y) chromaticity in 1/50000 units.
display_primaries: [[8500, 39850], [6550, 2300], [35400, 14600]],
white_point: [15635, 16450], // D65
max_display_mastering_luminance: 10_000_000, // 1000 cd/m² (0.0001 units)
min_display_mastering_luminance: 50, // 0.005 cd/m²
max_cll: 0,
max_fall: 0,
})
}
}
impl PortalCapturer {
@@ -372,6 +430,20 @@ impl PortalCapturer {
or capture never started)",
self.node_id
))
} else if self.hdr_offer {
// The HDR (10-bit PQ dmabuf) offer was never accepted — the monitor left HDR
// mode between the probe and the negotiation, the compositor pre-dates the
// GNOME 50 HDR formats, or its allocator can't do LINEAR for XR30/XB30.
// Latch the process-wide SDR downgrade so the next session (Moonlight
// auto-reconnects) negotiates SDR instead of re-running this same timeout.
super::note_hdr_capture_failed();
Err(anyhow!(
"no PipeWire frame within 10s (node {}): the compositor never accepted \
the HDR (10-bit PQ/BT.2020 dmabuf) offer — is the mirrored monitor in \
HDR mode on GNOME 50+? Downgrading this host to SDR capture; reconnect \
to stream SDR",
self.node_id
))
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
// The LINEAR-dmabuf-only offer (VAAPI passthrough default) was never accepted.
// Latch the process-wide downgrade so the encode loop's pipeline rebuild
@@ -416,6 +488,85 @@ impl Drop for PortalCapturer {
}
}
/// Whether any monitor of the live GNOME session is currently in BT.2100 (HDR) colour mode — the
/// precondition for Mutter's monitor screencast advertising the 10-bit PQ formats (GNOME 50+;
/// Mutter only appends the HDR formats while the mirrored monitor's colour state is BT.2020+PQ).
/// Queried over the session bus: `DisplayConfig.GetCurrentState`, monitor property
/// `"color-mode" == 1` (`META_COLOR_MODE_BT2100`). `false` on any error — not GNOME, a pre-48
/// Mutter without colour modes, no monitors — so callers fall back to the honest SDR offer.
/// Blocking (one D-Bus round-trip on a fresh connection); call from control-plane threads only.
pub fn gnome_hdr_monitor_active() -> bool {
use ashpd::zbus;
// GetCurrentState reply: (serial, monitors, logical_monitors, properties); each monitor is
// (spec(ssss), modes a(siiddada{sv}), properties a{sv}) — "color-mode" lives in the monitor
// properties.
type Mode = (
String,
i32,
i32,
f64,
f64,
Vec<f64>,
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
);
type Monitor = (
(String, String, String, String),
Vec<Mode>,
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
);
type LogicalMonitor = (
i32,
i32,
f64,
u32,
bool,
Vec<(String, String, String, String)>,
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
);
type State = (
u32,
Vec<Monitor>,
Vec<LogicalMonitor>,
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
);
let probe = || -> Result<bool> {
// zbus is built async-only here (ashpd's tokio integration) — run the one round-trip on
// a throwaway current-thread runtime; this is a control-plane call, never per-frame.
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.context("build tokio runtime")?;
rt.block_on(async {
let conn = zbus::Connection::session().await.context("session bus")?;
let reply = conn
.call_method(
Some("org.gnome.Mutter.DisplayConfig"),
"/org/gnome/Mutter/DisplayConfig",
Some("org.gnome.Mutter.DisplayConfig"),
"GetCurrentState",
&(),
)
.await
.context("DisplayConfig.GetCurrentState")?;
let (_serial, monitors, _logical, _props): State =
reply.body().deserialize().context("parse GetCurrentState")?;
Ok(monitors.iter().any(|(_spec, _modes, props)| {
props
.get("color-mode")
.and_then(|v| u32::try_from(v).ok())
.is_some_and(|mode| mode == 1) // META_COLOR_MODE_BT2100
}))
})
};
match probe() {
Ok(hdr) => hdr,
Err(e) => {
tracing::debug!(error = %format!("{e:#}"), "GNOME HDR colour-mode probe failed — SDR");
false
}
}
}
/// Pick the ScreenCast cursor mode from what the backend advertises (`AvailableCursorModes`),
/// preferring **cursor-as-metadata**: the compositor keeps its cheap hardware cursor plane and
/// ships the pointer as PipeWire `SPA_META_Cursor` metadata (position + an occasional bitmap),
@@ -669,6 +820,10 @@ mod pipewire {
VideoFormat::RGBA => PixelFormat::Rgba,
VideoFormat::RGB => PixelFormat::Rgb,
VideoFormat::BGR => PixelFormat::Bgr,
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by
// the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020).
VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10,
VideoFormat::xBGR_210LE => PixelFormat::X2Bgr10,
_ => return None,
})
}
@@ -732,6 +887,9 @@ mod pipewire {
/// irrecoverably gone for this stream — the import worker died, or tiled imports failed
/// [`IMPORT_FAIL_POISON`] times in a row.
broken: Arc<AtomicBool>,
/// Set when the negotiated format is one of the 10-bit PQ formats (`param_changed`) —
/// read by [`PortalCapturer::hdr_meta`](super::PortalCapturer).
hdr_negotiated: Arc<AtomicBool>,
/// Consecutive tiled-import failures (reset on success); see [`IMPORT_FAIL_POISON`].
import_fail_streak: u32,
/// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer,
@@ -886,6 +1044,80 @@ mod pipewire {
serialize_pod(obj)
}
/// Build one GNOME 50+ HDR format pod: `format` (xRGB_210LE / xBGR_210LE) as a LINEAR-only
/// dmabuf with **MANDATORY** BT.2020 primaries + SMPTE ST.2084 (PQ) transfer-function props —
/// the exact colorimetry Mutter's monitor stream advertises while the mirrored monitor is in
/// HDR mode (its HDR pods carry the same props MANDATORY, so both sides must speak them for
/// the intersection to exist; an SDR or pre-50 producer can never match this pod).
///
/// LINEAR-only because every 10-bit consumer we have reads the buffer without a de-tile pass:
/// the CPU path mmaps it, and the VAAPI passthrough imports it into a VA surface. The tiled
/// EGL de-tile blit renders into an 8-bit `GL_RGBA8` texture — it would silently crush the
/// depth — so tiled modifiers are deliberately NOT advertised (a zero-copy 10-bit de-tile is
/// the follow-up). SHM is excluded entirely: Mutter's SHM record path paints 8-bit ARGB32
/// regardless of the negotiated format.
fn build_hdr_dmabuf_format(
format: VideoFormat,
preferred: Option<(u32, u32, u32)>,
) -> Result<Vec<u8>> {
let (dw, dh, dhz) = preferred.unwrap_or((1920, 1080, 60));
use pw::spa::param::format::{FormatProperties, MediaSubtype, MediaType};
let mut obj = pw::spa::pod::object!(
pw::spa::utils::SpaTypes::ObjectParamFormat,
pw::spa::param::ParamType::EnumFormat,
pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video),
pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw),
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, format),
pw::spa::pod::property!(
FormatProperties::VideoSize,
Choice,
Range,
Rectangle,
pw::spa::utils::Rectangle {
width: dw,
height: dh
},
pw::spa::utils::Rectangle {
width: 1,
height: 1
},
pw::spa::utils::Rectangle {
width: 8192,
height: 8192
}
),
pw::spa::pod::property!(
FormatProperties::VideoFramerate,
Choice,
Range,
Fraction,
pw::spa::utils::Fraction { num: dhz, denom: 1 },
pw::spa::utils::Fraction { num: 0, denom: 1 },
pw::spa::utils::Fraction { num: 240, denom: 1 }
),
);
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Long(0), // DRM_FORMAT_MOD_LINEAR
});
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_transferFunction,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_TRANSFER_SMPTE2084,
)),
});
obj.properties.push(pw::spa::pod::Property {
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorPrimaries,
flags: pw::spa::pod::PropertyFlags::MANDATORY,
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
pw::spa::sys::SPA_VIDEO_COLOR_PRIMARIES_BT2020,
)),
});
serialize_pod(obj)
}
/// The default (shm/CPU-path) format offer: raw video in any encoder-mappable layout, any
/// size, any framerate (0/1 = variable allowed — gamescope fixates exactly that).
fn build_default_format_obj(preferred: Option<(u32, u32, u32)>) -> pw::spa::pod::Object {
@@ -1157,6 +1389,54 @@ mod pipewire {
})
}
/// Alpha-blend the cached cursor bitmap into a packed 10-bit (`X2Rgb10`/`X2Bgr10`) CPU frame:
/// unpack each u32, blend the 8-bit cursor channels scaled to 10 bits (`v<<2 | v>>6`), repack.
/// The frame samples are PQ-encoded, so like the 8-bit gamma-space blend this is a display-
/// referred approximation — fine for a cursor. `r_shift` is the R channel's bit offset (20 for
/// x:R:G:B, 0 for x:B:G:R); G is always at 10 and B mirrors R.
fn composite_cursor_rgb10(
tight: &mut [u8],
w: usize,
h: usize,
r_shift: u32,
cursor: &CursorState,
) {
let b_shift = 20 - r_shift; // 0 or 20 — the opposite end from R
let (bw, bh) = (cursor.bw as i32, cursor.bh as i32);
for cy in 0..bh {
let dy = cursor.y + cy;
if dy < 0 || dy as usize >= h {
continue;
}
for cx in 0..bw {
let dx = cursor.x + cx;
if dx < 0 || dx as usize >= w {
continue;
}
let s = ((cy * bw + cx) as usize) * 4;
let a = cursor.rgba[s + 3] as u32;
if a == 0 {
continue;
}
// 8-bit cursor channel → 10-bit (replicate the top bits into the bottom).
let up10 = |v: u8| ((v as u32) << 2) | ((v as u32) >> 6);
let (sr, sg, sb) = (
up10(cursor.rgba[s]),
up10(cursor.rgba[s + 1]),
up10(cursor.rgba[s + 2]),
);
let di = (dy as usize * w + dx as usize) * 4;
let px = u32::from_le_bytes(tight[di..di + 4].try_into().unwrap());
let blend = |dst: u32, src: u32| (src * a + dst * (255 - a)) / 255;
let dr = blend((px >> r_shift) & 0x3ff, sr);
let dg = blend((px >> 10) & 0x3ff, sg);
let db = blend((px >> b_shift) & 0x3ff, sb);
let out = (px & 0xc000_0000) | (dr << r_shift) | (dg << 10) | (db << b_shift);
tight[di..di + 4].copy_from_slice(&out.to_le_bytes());
}
}
}
/// Alpha-blend the cached cursor bitmap into the tightly-packed CPU frame at its latched
/// position. Cheap: a straight-alpha blit over at most ~256×256 pixels, clipped to the frame —
/// the whole point of cursor-as-metadata (no forced full-frame composite on the producer).
@@ -1170,6 +1450,12 @@ mod pipewire {
if !cursor.visible || cursor.rgba.is_empty() {
return;
}
// The packed 10-bit HDR layouts blend via bit unpack/repack, not byte offsets.
match fmt {
PixelFormat::X2Rgb10 => return composite_cursor_rgb10(tight, w, h, 20, cursor),
PixelFormat::X2Bgr10 => return composite_cursor_rgb10(tight, w, h, 0, cursor),
_ => {}
}
let Some((ri, gi, bi, bpp)) = dst_offsets(fmt) else {
return;
};
@@ -1344,7 +1630,10 @@ mod pipewire {
// through to the shm de-pad copy below.
let mut gpu_import_broken = false;
if let (Some(importer), Some(fmt)) = (ud.importer.as_mut(), ud.format) {
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
// Defense-in-depth: the 10-bit PQ formats must never enter the EGL→CUDA import (its
// de-tile blit is 8-bit RGBA8 — silent depth loss). An HDR offer never builds the
// importer, so this gate only matters if those invariants ever drift apart.
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf && !fmt.is_hdr_rgb10() {
let plane = pf_zerocopy::DmabufPlane {
fd: datas[0].fd(),
offset: datas[0].chunk().offset(),
@@ -1604,9 +1893,13 @@ mod pipewire {
negotiated: Arc<AtomicBool>,
streaming: Arc<AtomicBool>,
broken: Arc<AtomicBool>,
hdr_negotiated: Arc<AtomicBool>,
zerocopy: bool,
// 4:4:4 session: tiled dmabufs take the worker's planar-YUV444 GPU convert.
want_444: bool,
// HDR session: offer ONLY the 10-bit PQ/BT.2020 formats as LINEAR dmabufs (see
// `build_hdr_dmabuf_format`); the SDR offers are not built at all.
want_hdr: bool,
preferred: Option<(u32, u32, u32)>,
quit_rx: pw::channel::Receiver<()>,
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
@@ -1645,7 +1938,10 @@ mod pipewire {
// succeed and produce CUDA payloads the VAAPI encoder must reject. Also skipped once
// repeated worker deaths latched the import off (a wedged GPU stack must not crash-loop).
let backend_is_vaapi = policy.backend_is_vaapi;
let mut importer = if zerocopy && !backend_is_vaapi {
// HDR never builds the EGL→CUDA importer: its de-tile blit renders into 8-bit RGBA8,
// which would silently crush the 10-bit depth. The HDR consumers are the CPU mmap path
// (LINEAR de-pad → X2Rgb10 CPU frames) and the VAAPI raw-dmabuf passthrough.
let mut importer = if zerocopy && !backend_is_vaapi && !want_hdr {
if pf_zerocopy::gpu_import_disabled() {
tracing::warn!(
"zero-copy GPU import disabled after repeated import-worker deaths — using CPU path"
@@ -1755,6 +2051,7 @@ mod pipewire {
negotiated,
streaming,
broken,
hdr_negotiated,
import_fail_streak: 0,
importer,
vaapi_passthrough,
@@ -1822,12 +2119,20 @@ mod pipewire {
let sz = ud.info.size();
ud.format = map_format(ud.info.format());
ud.modifier = ud.info.modifier();
// HDR: the 10-bit PQ formats are only ever offered with MANDATORY BT.2020/PQ
// colorimetry props, so a 10-bit negotiation IS an HDR negotiation — but log
// what the producer actually fixated for diagnosis.
let hdr = ud.format.is_some_and(|f| f.is_hdr_rgb10());
ud.hdr_negotiated.store(hdr, Ordering::Relaxed);
tracing::info!(
width = sz.width,
height = sz.height,
spa_format = ?ud.info.format(),
mapped = ?ud.format,
modifier = ud.modifier,
hdr,
transfer_function = ud.info.transfer_function(),
color_primaries = ud.info.color_primaries(),
"pipewire format negotiated"
);
if ud.format.is_none() {
@@ -2029,20 +2334,36 @@ mod pipewire {
// (offering shm too makes the compositor pick shm). The modifier list is advertised with
// DONT_FIXATE so the compositor's allocator chooses one; we re-emit the fixated format in
// `param_changed` (the two-step DMA-BUF handshake). Otherwise offer the multi-format shm
// pod and let MAP_BUFFERS map it.
let shm_values = serialize_pod(obj)?;
let (dmabuf_values, buffers_values) = if want_dmabuf {
(
Some(build_dmabuf_format(&modifiers, preferred)?),
Some(build_dmabuf_buffers()?),
)
// pod and let MAP_BUFFERS map it. An HDR session replaces ALL of this with the two 10-bit
// PQ pods (LINEAR dmabuf, MANDATORY colorimetry — see `build_hdr_dmabuf_format`): offering
// SDR alongside would make the producer pick its earlier-listed SDR format, and the
// negotiation-timeout path latches the process-wide SDR downgrade if nothing matches.
let format_pods: Vec<Vec<u8>> = if want_hdr {
tracing::info!(
"HDR capture: offering xRGB_210LE/xBGR_210LE LINEAR dmabufs with MANDATORY \
BT.2020 + SMPTE-2084 (PQ) colorimetry (GNOME 50+ monitor stream)"
);
vec![
build_hdr_dmabuf_format(VideoFormat::xRGB_210LE, preferred)?,
build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?,
]
} else if want_dmabuf {
vec![build_dmabuf_format(&modifiers, preferred)?]
} else {
vec![serialize_pod(obj)?]
};
let buffers_values = if want_hdr || want_dmabuf {
// Dmabuf-only. For HDR this is load-bearing beyond zero-copy: Mutter's SHM record
// path paints 8-bit ARGB32 regardless of the negotiated format, so a MemFd buffer
// under a 10-bit format would carry mislabeled bytes.
Some(build_dmabuf_buffers()?)
} else if force_shm {
// True SHM: exclude DmaBuf so Mutter MUST download (glReadPixels orders against render).
(None, Some(build_shm_only_buffers()?))
Some(build_shm_only_buffers()?)
} else {
// CPU path still accepts mappable dmabufs (gamescope offers only those once its
// modifier-bearing format pod wins the intersection).
(None, Some(build_mappable_buffers()?))
Some(build_mappable_buffers()?)
};
// Ask for cursor-as-metadata on every path (harmless if the producer can't supply it): the
@@ -2050,9 +2371,8 @@ mod pipewire {
// compositor keeps its cheap hardware cursor plane (see `choose_cursor_mode`).
let cursor_meta = build_cursor_meta_param()?;
let mut byte_slices: Vec<&[u8]> = Vec::new();
match &dmabuf_values {
Some(d) => byte_slices.push(d),
None => byte_slices.push(&shm_values),
for pod in &format_pods {
byte_slices.push(pod);
}
if let Some(b) = &buffers_values {
byte_slices.push(b);
+197 -38
View File
@@ -27,8 +27,8 @@ use super::libav::{
use ffmpeg::ffi; // = ffmpeg_sys_next
/// The swscale *source* pixel format for a captured packed RGB/BGR layout (the real byte order, not
/// the NVENC-padded `*0` form). Used by the 4:4:4 RGB→YUV444P conversion path. Mirrors the VAAPI
/// CPU-input mapping; YUV/10-bit inputs can't feed this path (the 4:4:4 session forces packed RGB).
/// the NVENC-padded `*0` form). Used by the CPU conversion paths: 4:4:4 RGB→YUV444P, and HDR
/// X2RGB10/X2BGR10→P010. Mirrors the VAAPI CPU-input mapping; YUV inputs can't feed this path.
fn sws_src_pixel(format: PixelFormat) -> Result<Pixel> {
Ok(match format {
PixelFormat::Bgrx => Pixel::BGRZ, // bgr0
@@ -37,8 +37,12 @@ fn sws_src_pixel(format: PixelFormat) -> Result<Pixel> {
PixelFormat::Rgba => Pixel::RGBA,
PixelFormat::Rgb => Pixel::RGB24,
PixelFormat::Bgr => Pixel::BGR24,
// The GNOME 50+ HDR capture formats (PQ/BT.2020 packed 2:10:10:10) — the HDR CPU path's
// swscale source for the X2RGB10→P010 conversion.
PixelFormat::X2Rgb10 => Pixel::X2RGB10LE,
PixelFormat::X2Bgr10 => Pixel::X2BGR10LE,
PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Yuv444 => {
bail!("NVENC 4:4:4 CPU-input path supports packed RGB/BGR only; got {format:?}")
bail!("NVENC CPU-input conversion supports packed RGB/BGR only; got {format:?}")
}
})
}
@@ -136,6 +140,9 @@ fn nvenc_input(format: PixelFormat) -> (Pixel, bool) {
// the Windows paths; the Linux capturer never emits them. Map to BGRA so the match is
// exhaustive — unreachable here.
PixelFormat::Rgb10a2 | PixelFormat::P010 => (Pixel::BGRA, false),
// The Linux HDR capture formats never take the RGB-passthrough input: `open` intercepts
// them onto the X2RGB10→P010 swscale path before consulting this mapping (like 4:4:4).
PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10 => (Pixel::BGRA, false),
}
}
@@ -164,11 +171,12 @@ pub struct NvencEncoder {
frame: Option<VideoFrame>,
/// Zero-copy path: CUDA hwdevice/hwframes contexts (the encoder takes `AV_PIX_FMT_CUDA`).
cuda: Option<CudaHw>,
/// 4:4:4 CPU path only: swscale context converting the captured packed RGB/BGR → planar
/// YUV444P into [`Self::frame`], because `hevc_nvenc` only emits 4:4:4 from a YUV444 *input*
/// (RGB-in is always 4:2:0). `None` on the 4:2:0 paths AND on the zero-copy 4:4:4 path (the
/// worker's GPU convert delivers YUV444 CUDA frames). Freed in `Drop`.
sws_444: Option<*mut ffi::SwsContext>,
/// CPU CSC paths only: swscale context converting the captured packed source into
/// [`Self::frame`] — RGB/BGR → planar YUV444P for a 4:4:4 session (`hevc_nvenc` only emits
/// 4:4:4 from a YUV444 *input*; RGB-in is always 4:2:0), or X2RGB10/X2BGR10 → P010 (BT.2020
/// limited) for an HDR session. `None` on the plain RGB paths AND on the zero-copy paths (the
/// worker's GPU convert delivers ready CUDA frames). Freed in `Drop`.
sws_csc: Option<*mut ffi::SwsContext>,
/// This session opened as full-chroma 4:4:4 (FREXT) — via either input path.
want_444: bool,
src_format: PixelFormat,
@@ -191,7 +199,7 @@ pub struct NvencEncoder {
args: OpenArgs,
}
// `CudaHw` holds raw `AVBufferRef`s and `sws_444` a raw `SwsContext`; the encoder lives on a single
// `CudaHw` holds raw `AVBufferRef`s and `sws_csc` a raw `SwsContext`; the encoder lives on a single
// thread. The CPU encoder is already `Send` via ffmpeg-next; assert it for the raw fields too.
// SAFETY: `NvencEncoder` owns an ffmpeg-next `Encoder`/`VideoFrame` (already `Send`) plus a `CudaHw`
// holding raw `AVBufferRef`s and an optional raw `SwsContext`, none of which are `Send` by default.
@@ -247,14 +255,27 @@ impl NvencEncoder {
bit_depth: u8,
chroma: ChromaFormat,
) -> Result<Self> {
// TODO(hdr): Linux 10-bit parity. Unlike the Windows raw-SDK path (which upconverts 8-bit
// ARGB → Main10 via pixelBitDepthMinus8), libavcodec hevc_nvenc needs a 10-bit input pixel
// format (p010) for Main10, so it's a bigger change; deferred until a Linux GPU box is
// available to validate. The Linux host stays 8-bit for now.
if bit_depth != 8 {
// HDR / 10-bit (GNOME 50+ HDR screencast): a 10-bit session whose capture negotiated a
// packed 2:10:10:10 PQ/BT.2020 format (`X2Rgb10`/`X2Bgr10`) encodes HEVC Main10 / 10-bit
// AV1 from a P010 input frame we produce by swscale (BT.2020 limited; the PQ transfer
// rides through per-channel — BT.2020 NCL Y'CbCr *is* derived from the PQ-encoded R'G'B').
// A 10-bit request whose capture stayed SDR (HDR offer downgraded) honestly encodes 8-bit.
let want_hdr10 = bit_depth == 10 && format.is_hdr_rgb10() && codec.supports_10bit();
if bit_depth == 10 && !want_hdr10 {
tracing::warn!(
bit_depth,
"Linux NVENC 10-bit not yet wired — encoding 8-bit"
?format,
codec = codec.nvenc_name(),
"10-bit requested but the capture format/codec has no 10-bit path — encoding 8-bit"
);
}
if format.is_hdr_rgb10() && !want_hdr10 {
// A 10-bit PQ capture on an 8-bit session would be encoded with a BT.709 VUI and
// garbage bit-packing — never silently; the session must renegotiate.
bail!(
"captured 10-bit HDR frames ({format:?}) on an 8-bit/{} session — refusing to \
mislabel PQ content",
codec.nvenc_name()
);
}
// Full-chroma 4:4:4 (HEVC Range Extensions). `hevc_nvenc` only emits 4:4:4 from a YUV444
@@ -263,6 +284,11 @@ impl NvencEncoder {
// (planar-YUV444 CUDA frames — `cuda` true), or the CPU path's swscale RGB→YUV444P. Both
// feed `profile=rext`; the range follows `PUNKTFUNK_444_FULLRANGE` in both.
let want_444 = chroma.is_444() && codec == Codec::H265;
if want_444 && want_hdr10 {
// The handshake resolves 4:4:4∧10-bit down to 8-bit on Linux, so this can't happen —
// fail loudly if it ever does rather than picking one silently.
bail!("4:4:4 + 10-bit HDR is not a supported Linux NVENC combination");
}
ffmpeg::init().context("ffmpeg init")?;
if std::env::var_os("PUNKTFUNK_FFMPEG_DEBUG").is_some() {
// SAFETY: `av_log_set_level` sets libav's global integer log level; `48` (= AV_LOG_DEBUG)
@@ -274,10 +300,13 @@ impl NvencEncoder {
let av_codec = encoder::find_by_name(name)
.ok_or_else(|| anyhow!("{name} not built into libavcodec"))?;
let (rgb_pixel, rgb_expand) = nvenc_input(format);
// 4:4:4 feeds NVENC a planar YUV444P frame we produce by swscale; the ordinary path feeds the
// captured RGB straight in and lets NVENC's internal CSC subsample to 4:2:0.
// 4:4:4 feeds NVENC a planar YUV444P frame we produce by swscale; HDR feeds it a P010
// frame likewise; the ordinary path feeds the captured RGB straight in and lets NVENC's
// internal CSC subsample to 4:2:0.
let (nvenc_pixel, expand) = if want_444 {
(Pixel::YUV444P, false)
} else if want_hdr10 {
(Pixel::P010LE, false)
} else {
(rgb_pixel, rgb_expand)
};
@@ -325,7 +354,21 @@ impl NvencEncoder {
// visible win. Linux-only: the Windows path's NVENC-internal CSC range is unmeasured.
let full_range_444 =
want_444 && std::env::var("PUNKTFUNK_444_FULLRANGE").is_ok_and(|v| v.trim() == "1");
if matches!(format, PixelFormat::Nv12) || want_444 {
if want_hdr10 {
// HDR10: BT.2020 primaries + SMPTE-2084 (PQ) transfer, limited range — matches the
// swscale BT.2020 CSC below and the Windows paths' signalling. The client decoder
// auto-detects PQ from the VUI; static mastering metadata rides out-of-band.
// SAFETY: `raw = video.as_mut_ptr()` is the non-null, properly-aligned, sole-owned,
// not-yet-opened `AVCodecContext`; we set its four VUI colour enum fields to valid
// variants before `open_with`. Sole owner → no aliasing; synchronous writes.
unsafe {
let raw = video.as_mut_ptr();
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT2020_NCL;
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT2020;
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_SMPTE2084;
}
} else if matches!(format, PixelFormat::Nv12) || want_444 {
// SAFETY: same `video` builder — `raw = video.as_mut_ptr()` is the non-null, properly-
// aligned, sole-owned, not-yet-opened `AVCodecContext`. We set its four VUI colour enum
// fields to valid `AVColorSpace`/`AVColorRange`/`AVColorPrimaries`/`AVColorTransfer-
@@ -370,17 +413,20 @@ impl NvencEncoder {
None
};
// 4:4:4 CPU path: build the RGB→YUV444P swscale (BT.709, range per the flag; no rescale).
// Mirrors the VAAPI CPU path's RGB→NV12 scaler, but the dst is full-chroma planar 4:4:4.
// Skipped on the zero-copy path (`cuda`): the worker's GPU convert already delivers
// planar YUV444 CUDA frames — no CPU pixels exist to scale.
let sws_444 = if want_444 && !cuda {
// CPU CSC paths: build the packed-RGB → planar swscale (no rescale) into the encoder's
// input frame. Two users: 4:4:4 (RGB→YUV444P, BT.709, range per the flag) and HDR
// (X2RGB10/X2BGR10→P010, BT.2020 limited — the PQ transfer is per-channel and rides
// through the matrix untouched). Skipped on the zero-copy path (`cuda`): the worker's GPU
// convert already delivers ready CUDA frames — no CPU pixels exist to scale.
let sws_csc = if (want_444 || want_hdr10) && !cuda {
let src_av = pixel_to_av(sws_src_pixel(format)?);
let dst_av = pixel_to_av(nvenc_pixel);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dims + pixel
// formats. Both dims are the encoder's positive `width`/`height` as `c_int`; `src_av` is a
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated, packed-RGB-only source), the
// dst is YUV444P. The trailing filter/param pointers are null = "use defaults" (documented
// as accepted). No Rust memory is borrowed; the returned pointer is null-checked below.
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated packed-RGB source), the dst is
// YUV444P (4:4:4) or P010LE (HDR). The trailing filter/param pointers are null = "use
// defaults" (documented as accepted). No Rust memory is borrowed; the returned pointer is
// null-checked below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
@@ -388,7 +434,7 @@ impl NvencEncoder {
src_av,
width as c_int,
height as c_int,
ffi::AVPixelFormat::AV_PIX_FMT_YUV444P,
dst_av,
SWS_POINT,
ptr::null_mut(),
ptr::null_mut(),
@@ -396,17 +442,22 @@ impl NvencEncoder {
)
};
if sws.is_null() {
bail!("sws_getContext(RGB→YUV444P) failed");
bail!("sws_getContext(RGB→{nvenc_pixel:?}) failed");
}
// SAFETY: `sws` is the non-null context from the call above (null-checked). The ITU-709
// coefficient table from `sws_getCoefficients` is a process-lifetime libswscale static,
// reused for src+dst matrices; `sws_setColorspaceDetails` only reads it and writes scalar
// CSC settings into `sws` (dstRange matches the VUI: 0 = limited, 1 = the
// PUNKTFUNK_444_FULLRANGE experiment). No Rust memory is passed.
// SAFETY: `sws` is the non-null context from the call above (null-checked). The
// coefficient tables from `sws_getCoefficients` (ITU-709 for 4:4:4, BT.2020 NCL for HDR
// — matching the VUI written above) are process-lifetime libswscale statics, reused for
// src+dst matrices; `sws_setColorspaceDetails` only reads them and writes scalar CSC
// settings into `sws` (dstRange matches the VUI: 0 = limited, 1 = the
// PUNKTFUNK_444_FULLRANGE experiment; HDR is always limited). No Rust memory is passed.
unsafe {
let cs709 = ffi::sws_getCoefficients(SWS_CS_ITU709);
let cs = ffi::sws_getCoefficients(if want_hdr10 {
super::libav::SWS_CS_BT2020
} else {
SWS_CS_ITU709
});
let dst_range = i32::from(full_range_444);
ffi::sws_setColorspaceDetails(sws, cs709, 1, cs709, dst_range, 0, 1 << 16, 1 << 16);
ffi::sws_setColorspaceDetails(sws, cs, 1, cs, dst_range, 0, 1 << 16, 1 << 16);
}
Some(sws)
} else {
@@ -432,6 +483,12 @@ impl NvencEncoder {
// dropped on a future libavcodec.
opts.set("profile", "rext");
}
if want_hdr10 && codec == Codec::H265 {
// HEVC Main10. `hevc_nvenc` auto-selects it from the P010 input, but pin it explicitly
// so the depth is never silently dropped on a future libavcodec. (10-bit AV1 needs no
// profile — AV1 Main carries 10-bit, driven by the input format.)
opts.set("profile", "main10");
}
// Split-frame encode across both NVENC engines (GB203 has 2) when the pixel rate exceeds
// a single engine's HEVC capacity (~1 Gpix/s); e.g. 5120x1440@240 = 1.77 Gpix/s needs it,
@@ -501,7 +558,7 @@ impl NvencEncoder {
enc,
frame,
cuda: cuda_hw,
sws_444,
sws_csc,
want_444,
src_format: format,
expand,
@@ -640,7 +697,7 @@ impl NvencEncoder {
);
// 4:4:4: swscale the packed RGB straight into the planar YUV444P input frame (BT.709 limited),
// then send it — no byte-expand. The 4:2:0 RGB path (below) feeds NVENC packed RGB directly.
if let Some(sws) = self.sws_444 {
if let Some(sws) = self.sws_csc {
let frame = self
.frame
.as_mut()
@@ -810,7 +867,7 @@ impl NvencEncoder {
impl Drop for NvencEncoder {
fn drop(&mut self) {
if let Some(sws) = self.sws_444.take() {
if let Some(sws) = self.sws_csc.take() {
// SAFETY: `sws` is the non-null `SwsContext` allocated by `sws_getContext` in `open` and
// owned exclusively by this encoder (taken out of the field so it can't be freed twice).
// `sws_freeContext` frees it; nothing else references it after this single-threaded drop.
@@ -855,3 +912,105 @@ pub fn probe_can_encode_444(codec: Codec) -> bool {
unsafe { ffi::av_log_set_level(prev) };
ok
}
#[cfg(test)]
mod hdr_tests {
use super::*;
/// The Linux HDR (GNOME 50 portal) encode path end-to-end on a real NVIDIA GPU: a synthetic
/// PQ-ish X2RGB10 CPU frame → swscale BT.2020 → P010 → `hevc_nvenc` Main10, drained to a real
/// AU. `#[ignore]`d (needs NVENC):
/// `cargo test -p pf-encode nvenc_hdr10_smoke -- --ignored --nocapture`
#[test]
#[ignore]
fn nvenc_hdr10_smoke() {
let (w, h) = (640u32, 480u32);
let mut enc = NvencEncoder::open(
Codec::H265,
PixelFormat::X2Rgb10,
w,
h,
30,
2_000_000,
false,
10,
ChromaFormat::Yuv420,
)
.expect("open hevc_nvenc Main10 (P010 input)");
// Packed x:R:G:B 2:10:10:10 gradient (values are treated as PQ-encoded — fine for a smoke).
let mut bytes = vec![0u8; (w * h * 4) as usize];
for y in 0..h {
for x in 0..w {
let r = (x * 1023 / w.max(1)) & 0x3ff;
let g = (y * 1023 / h.max(1)) & 0x3ff;
let b = ((x + y) * 1023 / (w + h)) & 0x3ff;
let px: u32 = (r << 20) | (g << 10) | b;
let i = ((y * w + x) * 4) as usize;
bytes[i..i + 4].copy_from_slice(&px.to_le_bytes());
}
}
let frame = CapturedFrame {
width: w,
height: h,
pts_ns: 0,
format: PixelFormat::X2Rgb10,
payload: FramePayload::Cpu(bytes),
cursor: None,
};
let mut au = None;
for _ in 0..30 {
enc.submit(&frame).expect("submit X2Rgb10 frame");
if let Some(a) = enc.poll().expect("poll") {
au = Some(a);
break;
}
}
let au = au.expect("no AU produced within 30 frames");
assert!(!au.data.is_empty(), "empty AU");
assert!(au.keyframe, "first AU should be the IDR");
println!("HDR10 smoke: first AU {} bytes (IDR)", au.data.len());
// PF_HDR_SMOKE_DUMP=/path.h265: write the Annex-B AU for external inspection —
// `ffprobe -show_streams` should report Main 10, bt2020nc/smpte2084/bt2020 colours.
if let Ok(path) = std::env::var("PF_HDR_SMOKE_DUMP") {
std::fs::write(&path, &au.data).expect("dump AU");
println!("HDR10 smoke: AU written to {path}");
}
}
}
/// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode 10-bit (HEVC Main10 /
/// 10-bit AV1) from a P010 input — the exact path [`NvencEncoder::open`] takes for a live HDR
/// stream (a tiny X2RGB10-sourced, P010-input open). The result is cached by the caller
/// ([`crate::can_encode_10bit`]); a GPU/driver/ffmpeg without the 10-bit encode fails the open
/// here, so the host resolves the session to 8-bit SDR before the Welcome (honest downgrade).
pub fn probe_can_encode_10bit(codec: Codec) -> bool {
if !codec.supports_10bit() {
return false;
}
if ffmpeg::init().is_err() {
return false;
}
// Quiet ffmpeg's open error on a GPU that lacks 10-bit — the probe failing is an expected outcome.
// SAFETY: libav initialized above; `av_log_{get,set}_level` only read/write the global int level
// (no pointer args) and are always sound post-init.
let prev = unsafe {
let p = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
p
};
let ok = NvencEncoder::open(
codec,
PixelFormat::X2Rgb10,
640,
480,
30,
2_000_000,
false, // CPU input (the HDR swscale path)
10,
ChromaFormat::Yuv420,
)
.is_ok();
// SAFETY: restore the saved global log level (scalar arg, no pointers).
unsafe { ffi::av_log_set_level(prev) };
ok
}
+6 -2
View File
@@ -474,9 +474,13 @@ impl NvencCudaEncoder {
// clear reason instead of an opaque session error on the first frame.
try_api().map_err(|e| anyhow!("NVENC (Linux direct) unavailable: {e}"))?;
if bit_depth >= 10 {
// An HDR (GNOME 50 portal) session never reaches this backend: its X2RGB10 frames ride
// the CPU/dmabuf paths (no CUDA import for the 10-bit formats yet), so the dispatcher
// opens the libav P010 path instead. Reaching here 10-bit means a CUDA capture payload
// on a 10-bit session — not wired; encode 8-bit rather than mislabel.
tracing::warn!(
"Linux direct-NVENC: 10-bit requested but no P010 capture path exists yet \
(Phase 5.1) — encoding 8-bit SDR"
"Linux direct-NVENC: 10-bit requested but the CUDA capture path has no 10-bit \
import yet (HDR rides the libav P010 path) — encoding 8-bit SDR"
);
}
Ok(Self {
+176 -49
View File
@@ -61,6 +61,10 @@ fn vaapi_sws_src(format: PixelFormat) -> Result<Pixel> {
PixelFormat::Rgba => Pixel::RGBA,
PixelFormat::Rgb => Pixel::RGB24,
PixelFormat::Bgr => Pixel::BGR24,
// The GNOME 50+ HDR capture formats (PQ/BT.2020 packed 2:10:10:10) — the HDR CPU path's
// swscale source for the X2RGB10→P010 conversion.
PixelFormat::X2Rgb10 => Pixel::X2RGB10LE,
PixelFormat::X2Bgr10 => Pixel::X2BGR10LE,
PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Yuv444 => {
bail!("VAAPI CPU-input path supports packed RGB/BGR only; got {format:?}")
}
@@ -101,6 +105,7 @@ fn low_power_override() -> Option<bool> {
/// default on those kernels). AMD keeps its first-try full-feature open byte-for-byte unchanged.
/// The resolved mode is cached per codec; `PUNKTFUNK_VAAPI_LOW_POWER` pins it.
/// Safety contract is [`open_vaapi_encoder_mode`]'s (borrowed `device_ref`/`frames_ref`).
#[allow(clippy::too_many_arguments)]
unsafe fn open_vaapi_encoder(
codec: Codec,
width: u32,
@@ -109,6 +114,7 @@ unsafe fn open_vaapi_encoder(
bitrate_bps: u64,
device_ref: *mut ffi::AVBufferRef,
frames_ref: *mut ffi::AVBufferRef,
ten_bit: bool,
) -> Result<encoder::video::Encoder> {
let idx = lp_idx(codec);
let modes: &[bool] = match low_power_override() {
@@ -130,6 +136,7 @@ unsafe fn open_vaapi_encoder(
bitrate_bps,
device_ref,
frames_ref,
ten_bit,
lp,
) {
Ok(enc) => {
@@ -158,8 +165,9 @@ unsafe fn open_vaapi_encoder(
}
/// Build the FFmpeg encoder context (shared by both inner paths): name, mode, low-latency RC,
/// infinite GOP, BT.709-limited VUI, `pix_fmt=VAAPI`, and the given hw device + frames contexts.
/// Returns the opened encoder. `device_ref`/`frames_ref` are borrowed (ref'd into the context).
/// infinite GOP, the VUI (BT.709 limited SDR, or BT.2020 PQ limited for `ten_bit` HDR),
/// `pix_fmt=VAAPI`, and the given hw device + frames contexts. Returns the opened encoder.
/// `device_ref`/`frames_ref` are borrowed (ref'd into the context).
#[allow(clippy::too_many_arguments)]
unsafe fn open_vaapi_encoder_mode(
codec: Codec,
@@ -169,6 +177,7 @@ unsafe fn open_vaapi_encoder_mode(
bitrate_bps: u64,
device_ref: *mut ffi::AVBufferRef,
frames_ref: *mut ffi::AVBufferRef,
ten_bit: bool,
low_power: bool,
) -> Result<encoder::video::Encoder> {
let name = codec.vaapi_name();
@@ -181,23 +190,39 @@ unsafe fn open_vaapi_encoder_mode(
.context("alloc video encoder")?;
video.set_width(width);
video.set_height(height);
video.set_format(Pixel::NV12); // sw view; pix_fmt overridden to VAAPI below
// Fixed rate, CBR, no B-frames, ~1-frame VBV — the shared low-latency RC contract.
// sw view (pix_fmt overridden to VAAPI below): NV12, or P010 for the 10-bit HDR session.
video.set_format(if ten_bit { Pixel::P010LE } else { Pixel::NV12 });
// Fixed rate, CBR, no B-frames, ~1-frame VBV — the shared low-latency RC contract.
apply_low_latency_rc(&mut video, fps, bitrate_bps);
let raw = video.as_mut_ptr();
(*raw).gop_size = i32::MAX; // no periodic IDR (forced-IDR via pict_type=I on RFI)
// We hand the encoder BT.709 *limited* NV12 (swscale CSC on the CPU path; scale_vaapi pinned
// to `out_color_matrix=bt709:out_range=limited` on the zero-copy path, with the full-range
// RGB input tagged), so signal that VUI — else the client decoder washes the picture out.
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT709;
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT709;
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_BT709;
if ten_bit {
// HDR10: BT.2020 primaries + SMPTE-2084 (PQ) transfer, limited range — matches the P010
// the CSC produces (swscale BT.2020 on the CPU path; scale_vaapi pinned to bt2020 on the
// zero-copy path). The client decoder auto-detects PQ from the VUI.
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT2020_NCL;
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT2020;
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_SMPTE2084;
} else {
// We hand the encoder BT.709 *limited* NV12 (swscale CSC on the CPU path; scale_vaapi pinned
// to `out_color_matrix=bt709:out_range=limited` on the zero-copy path, with the full-range
// RGB input tagged), so signal that VUI — else the client decoder washes the picture out.
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT709;
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT709;
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_BT709;
}
(*raw).pix_fmt = ffi::AVPixelFormat::AV_PIX_FMT_VAAPI;
(*raw).hw_device_ctx = ffi::av_buffer_ref(device_ref);
(*raw).hw_frames_ctx = ffi::av_buffer_ref(frames_ref);
let mut opts = Dictionary::new();
if ten_bit && codec == Codec::H265 {
// HEVC Main10. `hevc_vaapi` derives it from the P010 surfaces, but pin it explicitly so
// the depth is never silently dropped. (10-bit AV1 is input-driven — no profile knob.)
opts.set("profile", "main10");
}
// async_depth=1: `send_frame` blocks until THIS frame's ASIC encode completes — the lowest
// latency structure libavcodec's vaapi_encode offers. Measured on the 780M at 1440p60: depth 1
// = 8.3 ms end-to-end p50 vs depth 2 = 18 ms, because with depth ≥ 2 frame N's packet only
@@ -242,10 +267,59 @@ pub fn probe_can_encode(codec: Codec) -> bool {
let prev = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
let ok = match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, 640, 480, 2) {
Ok(hw) => {
open_vaapi_encoder(codec, 640, 480, 30, 2_000_000, hw.device_ref, hw.frames_ref)
.is_ok()
}
Ok(hw) => open_vaapi_encoder(
codec,
640,
480,
30,
2_000_000,
hw.device_ref,
hw.frames_ref,
false,
)
.is_ok(),
Err(_) => false,
};
ffi::av_log_set_level(prev);
ok
}
}
/// Probe whether the active VAAPI GPU can encode **10-bit** (HEVC Main10 / 10-bit AV1) from P010
/// surfaces — the exact shape a live HDR session opens (P010 pool + Main10 profile + PQ VUI). The
/// driver rejects what the video engine can't do; the result is cached by the caller
/// ([`crate::can_encode_10bit`]), so a non-Main10 GPU resolves every session to 8-bit SDR before
/// the Welcome (honest downgrade).
pub fn probe_can_encode_10bit(codec: Codec) -> bool {
if !codec.supports_10bit() || codec == Codec::PyroWave {
return false;
}
if ffmpeg::init().is_err() {
return false;
}
// SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `av_log_{get,set}_level`
// only read/write libav's global integer log level (no pointer args). `VaapiHw::new` (an
// `unsafe fn`) builds a VAAPI device + P010 frames pool from the literal args and hands back a
// RAII handle; `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` —
// the two non-null refs `VaapiHw::new` just created, live locals for the whole match arm — and
// `av_buffer_ref`s them into the probe encoder. Both `hw` and the encoder drop (RAII) at arm end.
unsafe {
// A missing VA device / no Main10 entrypoint is an expected probe outcome — quiet ffmpeg's
// error for the probe, then restore the level.
let prev = ffi::av_log_get_level();
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
let ok = match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_P010LE, 640, 480, 2) {
Ok(hw) => open_vaapi_encoder(
codec,
640,
480,
30,
2_000_000,
hw.device_ref,
hw.frames_ref,
true,
)
.is_ok(),
Err(_) => false,
};
ffi::av_log_set_level(prev);
@@ -348,12 +422,21 @@ impl CpuInner {
bitrate_bps: u64,
) -> Result<Self> {
let src_pixel = vaapi_sws_src(format)?;
// A 10-bit HDR capture (X2RGB10/X2BGR10, PQ/BT.2020) uploads P010 and encodes Main10; the
// 8-bit paths keep NV12/BT.709 byte-for-byte unchanged.
let ten_bit = format.is_hdr_rgb10();
let staging_av = if ten_bit {
ffi::AVPixelFormat::AV_PIX_FMT_P010LE
} else {
ffi::AVPixelFormat::AV_PIX_FMT_NV12
};
const POOL: c_int = 16;
// SAFETY: `VaapiHw::new` (an `unsafe fn`) requires libav initialized — guaranteed because the
// only path here is `VaapiEncoder::open` → `ensure_inner` → `CpuInner::open`, and `open` ran
// `ffmpeg::init()`. The args are valid: NV12 sw_format, the validated positive `width`/`height`,
// pool=16. It returns a RAII `VaapiHw` that unrefs its two `AVBufferRef`s on drop.
let hw = unsafe { VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, width, height, POOL)? };
// `ffmpeg::init()`. The args are valid: an NV12/P010 sw_format, the validated positive
// `width`/`height`, pool=16. It returns a RAII `VaapiHw` that unrefs its two `AVBufferRef`s
// on drop.
let hw = unsafe { VaapiHw::new(staging_av, width, height, POOL)? };
// SAFETY: `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` — both
// non-null (`VaapiHw::new` guarantees it) and from the `hw` just built above, which is a live
// local that outlives this synchronous call. The fn `av_buffer_ref`s them into the encoder, so
@@ -368,16 +451,19 @@ impl CpuInner {
bitrate_bps,
hw.device_ref,
hw.frames_ref,
ten_bit,
)?
};
// swscale RGB→NV12, BT.709 limited (matches the VUI), no rescale.
// swscale RGB→NV12 (BT.709 limited) or X2RGB10→P010 (BT.2020 limited, HDR) — matches the
// VUI; no rescale.
let src_av = pixel_to_av(src_pixel);
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dimensions and
// pixel formats. All four dims are the encoder's positive `width`/`height` cast to `c_int`;
// `src_av` is a valid `AVPixelFormat` (from `pixel_to_av` of the `vaapi_sws_src`-validated
// `src_pixel`), the dst is NV12. The three trailing pointers (srcFilter, dstFilter, param) are
// explicitly null = "use defaults", which the API documents as accepted. No Rust memory is
// borrowed — only by-value ints/enums — and the returned pointer is null-checked just below.
// `src_pixel`), the dst is NV12/P010. The three trailing pointers (srcFilter, dstFilter,
// param) are explicitly null = "use defaults", which the API documents as accepted. No Rust
// memory is borrowed — only by-value ints/enums — and the returned pointer is null-checked
// just below.
let sws = unsafe {
ffi::sws_getContext(
width as c_int,
@@ -385,7 +471,7 @@ impl CpuInner {
src_av,
width as c_int,
height as c_int,
ffi::AVPixelFormat::AV_PIX_FMT_NV12,
staging_av,
SWS_POINT,
ptr::null_mut(),
ptr::null_mut(),
@@ -393,45 +479,51 @@ impl CpuInner {
)
};
if sws.is_null() {
bail!("sws_getContext(RGB→NV12) failed");
bail!("sws_getContext(RGB→{})", if ten_bit { "P010" } else { "NV12" });
}
// SAFETY: `sws` is the non-null `SwsContext` from `sws_getContext` above (the `is_null()`
// check immediately preceding returned false). `sws_getCoefficients(SWS_CS_ITU709)` returns a
// pointer into a libswscale static const coefficient table valid for the whole process, reused
// here for both the inverse (src) and forward (dst) matrices. `sws_setColorspaceDetails` only
// reads those tables and writes scalar CSC settings into `sws`; the table pointer outlives the
// synchronous call and no Rust memory is passed.
// check immediately preceding returned false). The coefficient table from
// `sws_getCoefficients` (ITU-709, or BT.2020 NCL for the HDR path — matching the VUI) is a
// libswscale static const valid for the whole process, reused here for both the inverse
// (src) and forward (dst) matrices. `sws_setColorspaceDetails` only reads those tables and
// writes scalar CSC settings into `sws`; the table pointer outlives the synchronous call and
// no Rust memory is passed.
unsafe {
let cs709 = ffi::sws_getCoefficients(SWS_CS_ITU709);
ffi::sws_setColorspaceDetails(sws, cs709, 1, cs709, 0, 0, 1 << 16, 1 << 16);
let cs = ffi::sws_getCoefficients(if ten_bit {
super::libav::SWS_CS_BT2020
} else {
SWS_CS_ITU709
});
ffi::sws_setColorspaceDetails(sws, cs, 1, cs, 0, 0, 1 << 16, 1 << 16);
}
// SAFETY: `av_frame_alloc` returns a fresh, uniquely-owned heap `AVFrame` (null-checked — on
// null we free the already-built `sws` and bail). We then write the plain `format`/`width`/
// `height` fields through the non-null, properly-aligned `f` (sole owner, not yet shared).
// `av_frame_get_buffer(f, 0)` allocates backing storage for those dims/format; on failure we
// free `f` and `sws` (unwinding the half-built state) and bail. On success `f` is a fully-owned
// NV12 frame stored in `CpuInner.nv12` and freed once in `CpuInner::drop`. `f` is a unique
// fresh pointer, so none of these writes alias anything.
// NV12/P010 frame stored in `CpuInner.nv12` and freed once in `CpuInner::drop`. `f` is a
// unique fresh pointer, so none of these writes alias anything.
let nv12 = unsafe {
let f = ffi::av_frame_alloc();
if f.is_null() {
ffi::sws_freeContext(sws);
bail!("av_frame_alloc(NV12) failed");
bail!("av_frame_alloc(staging) failed");
}
(*f).format = ffi::AVPixelFormat::AV_PIX_FMT_NV12 as c_int;
(*f).format = staging_av as c_int;
(*f).width = width as c_int;
(*f).height = height as c_int;
if ffi::av_frame_get_buffer(f, 0) < 0 {
let mut f = f;
ffi::av_frame_free(&mut f);
ffi::sws_freeContext(sws);
bail!("av_frame_get_buffer(NV12) failed");
bail!("av_frame_get_buffer(staging) failed");
}
f
};
tracing::info!(
encoder = codec.vaapi_name(),
"VAAPI encode active ({width}x{height}@{fps}, CPU→NV12 upload path)"
"VAAPI encode active ({width}x{height}@{fps}, CPU→{} upload path)",
if ten_bit { "P010 (HDR10)" } else { "NV12" }
);
Ok(CpuInner {
enc,
@@ -563,6 +655,15 @@ impl DmabufInner {
) -> Result<Self> {
let drm_fourcc = pf_frame::drm_fourcc(format)
.ok_or_else(|| anyhow!("no DRM fourcc for {format:?} (VAAPI zero-copy)"))?;
// A 10-bit HDR capture (X2RGB10/X2BGR10 dmabufs, PQ/BT.2020) maps + CSCs to P010 and
// encodes Main10; the 8-bit paths keep the NV12/BT.709 graph byte-for-byte unchanged.
let ten_bit = format.is_hdr_rgb10();
let sw_format = match format {
PixelFormat::X2Rgb10 => ffi::AVPixelFormat::AV_PIX_FMT_X2RGB10LE,
PixelFormat::X2Bgr10 => ffi::AVPixelFormat::AV_PIX_FMT_X2BGR10LE,
// The 8-bit capture formats are all XR24-shaped packed RGB (the historical BGR0 view).
_ => ffi::AVPixelFormat::AV_PIX_FMT_BGR0,
};
let node = render_node();
// SAFETY: libav is initialized (`VaapiEncoder::open` ran `ffmpeg::init()` before
// `ensure_inner` → `DmabufInner::open`). Every raw pointer dereferenced below is either freshly
@@ -628,7 +729,7 @@ impl DmabufInner {
}
let fc = (*drm_frames).data as *mut ffi::AVHWFramesContext;
(*fc).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME;
(*fc).sw_format = ffi::AVPixelFormat::AV_PIX_FMT_BGR0; // packed XR24 RGB plane
(*fc).sw_format = sw_format; // packed XR24 RGB plane, or XR30/XB30 for HDR
(*fc).width = width as c_int;
(*fc).height = height as c_int;
if ffi::av_hwframe_ctx_init(drm_frames) < 0 {
@@ -715,14 +816,24 @@ impl DmabufInner {
}
init!(src, ptr::null(), "buffer");
init!(hwmap, c"mode=read".as_ptr(), "hwmap");
// Pin the VPP's output colour to what the encoder's VUI signals (BT.709 limited).
// Without the explicit options the conversion matrix is whatever the driver defaults
// to for an unspecified output (Mesa: BT.601) — a hue shift against the signaled VUI.
init!(
scale,
c"format=nv12:out_color_matrix=bt709:out_range=limited".as_ptr(),
"scale_vaapi"
);
// Pin the VPP's output colour to what the encoder's VUI signals (BT.709 limited SDR,
// or BT.2020 limited P010 for HDR — the PQ transfer is per-channel and rides through
// the matrix untouched). Without the explicit options the conversion matrix is
// whatever the driver defaults to for an unspecified output (Mesa: BT.601) — a hue
// shift against the signaled VUI.
if ten_bit {
init!(
scale,
c"format=p010:out_color_matrix=bt2020:out_range=limited".as_ptr(),
"scale_vaapi"
);
} else {
init!(
scale,
c"format=nv12:out_color_matrix=bt709:out_range=limited".as_ptr(),
"scale_vaapi"
);
}
init!(sink, ptr::null(), "buffersink");
let link = |a: *mut ffi::AVFilterContext, b: *mut ffi::AVFilterContext| -> c_int {
@@ -766,6 +877,7 @@ impl DmabufInner {
bitrate_bps,
vaapi_device,
nv12_ctx,
ten_bit,
) {
Ok(enc) => enc,
Err(e) => {
@@ -779,7 +891,8 @@ impl DmabufInner {
tracing::info!(
encoder = codec.vaapi_name(),
"VAAPI encode active ({width}x{height}@{fps}, zero-copy dmabuf → GPU NV12)"
"VAAPI encode active ({width}x{height}@{fps}, zero-copy dmabuf → GPU {})",
if ten_bit { "P010 (HDR10)" } else { "NV12" }
);
Ok(DmabufInner {
enc,
@@ -987,8 +1100,22 @@ impl VaapiEncoder {
bit_depth: u8,
chroma: super::ChromaFormat,
) -> Result<Self> {
if bit_depth != 8 {
tracing::warn!(bit_depth, "VAAPI 10-bit not yet wired — encoding 8-bit");
// 10-bit rides on the captured format: an HDR capture (X2RGB10/X2BGR10) opens the P010 /
// Main10 / PQ-VUI variant of whichever inner path the first frame selects. A 10-bit
// request whose capture stayed SDR honestly encodes 8-bit; the reverse (PQ frames on an
// 8-bit session) is refused so PQ content is never mislabeled BT.709.
if format.is_hdr_rgb10() && bit_depth != 10 {
bail!(
"captured 10-bit HDR frames ({format:?}) on an {bit_depth}-bit VAAPI session — \
refusing to mislabel PQ content"
);
}
if bit_depth == 10 && !format.is_hdr_rgb10() {
tracing::warn!(
bit_depth,
?format,
"10-bit requested but the capture stayed SDR — encoding 8-bit"
);
}
// VAAPI 4:4:4 is deferred (see `probe_can_encode_444`): no validated AMD/Intel hardware in the
// lab exposes a HEVC 4:4:4 encode entrypoint, and the probe returns false so the host never
+4 -4
View File
@@ -181,10 +181,10 @@ impl Encoder for OpenH264Encoder {
PixelFormat::Bgr => (3, 2, 1, 0),
PixelFormat::Rgba | PixelFormat::Rgbx => (4, 0, 1, 2),
PixelFormat::Bgra | PixelFormat::Bgrx => (4, 2, 1, 0),
// 10-bit HDR comes only from the GPU NVENC path; the software 8-bit H.264 encoder
// can't represent it (and never receives it — the capturer pairs Rgb10a2 with NVENC).
PixelFormat::Rgb10a2 => {
anyhow::bail!("software H.264 encoder cannot encode 10-bit HDR (Rgb10a2)")
// 10-bit HDR comes only from the GPU paths; the software 8-bit H.264 encoder can't
// represent it (and never receives it — HDR is never negotiated on a software host).
PixelFormat::Rgb10a2 | PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10 => {
anyhow::bail!("software H.264 encoder cannot encode 10-bit HDR ({:?})", self.src_format)
}
// NV12/P010 are GPU-resident video-processor outputs for the NVENC path; the software
// encoder never receives them (it only gets CPU RGB frames).
+23 -8
View File
@@ -288,8 +288,14 @@ fn open_video_backend(
// stream never dies over the new path. `format`/`bit_depth`/`chroma` only matter to VAAPI —
// the Vulkan backend imports the dmabuf and does its own 8-bit 4:2:0 CSC.
let open_amd_intel = || -> Result<(Box<dyn Encoder>, &'static str)> {
// An HDR session (10-bit + a PQ/BT.2020 capture format) must skip the Vulkan Video
// backend — it hardcodes an 8-bit 4:2:0 BT.709 CSC — and take the libav VAAPI path,
// which has the P010/Main10/PQ wiring. SDR sessions keep the Vulkan default.
#[cfg(feature = "vulkan-encode")]
if matches!(codec, Codec::H265 | Codec::Av1) && vulkan_encode_enabled() {
if matches!(codec, Codec::H265 | Codec::Av1)
&& vulkan_encode_enabled()
&& !(bit_depth == 10 && format.is_hdr_rgb10())
{
match vulkan_video::VulkanVideoEncoder::open(codec, width, height, fps, bitrate_bps)
{
Ok(e) => {
@@ -956,11 +962,10 @@ pub fn can_encode_444(_codec: Codec) -> bool {
/// Backend truth: Windows **NVENC** queries the per-codec `NV_ENC_CAPS_SUPPORT_10BIT_ENCODE` cap;
/// native **AMF** `Init`s a tiny P010 encoder with the 10-bit profile props (the driver rejects
/// what the VCN can't do). **QSV** stays `false` until validated on Intel glass — the libavcodec
/// Main10 incantation can silently encode 8-bit, the same stance as its 4:4:4 probe. Every
/// **Linux** backend is `false` today: direct-NVENC/CUDA pins 8-bit until a P010 capture path
/// exists (Phase 5.1), libav `hevc_nvenc` needs a 10-bit input format the capturer never feeds,
/// VAAPI 10-bit isn't wired, and Vulkan-video hardcodes 8-bit — so Linux hosts honestly negotiate
/// 8-bit SDR.
/// Main10 incantation can silently encode 8-bit, the same stance as its 4:4:4 probe. **Linux**
/// probes a tiny real Main10 open on the auto-resolved backend — libav NVENC (the HDR X2RGB10→
/// P010 swscale path) or VAAPI (P010 pool + Main10) — for the GNOME 50+ HDR portal capture;
/// the direct-SDK CUDA path and Vulkan-video stay 8-bit and a 10-bit session routes around them.
#[cfg(any(target_os = "linux", target_os = "windows"))]
pub fn can_encode_10bit(codec: Codec) -> bool {
use std::collections::HashMap;
@@ -985,8 +990,18 @@ pub fn can_encode_10bit(codec: Codec) -> bool {
let supported = {
#[cfg(target_os = "linux")]
{
// No Linux backend encodes 10-bit yet (see the fn doc) — never negotiate it.
false
// NVENC (libav, the HDR P010 swscale path) or VAAPI (P010 upload / dmabuf graph),
// probed by opening a tiny real Main10 encoder — the same honesty contract as
// `can_encode_444`. Vulkan-video and the direct-SDK CUDA path stay 8-bit; a 10-bit
// session routes around them (see `open_video_backend`). NOTE: encode capability is
// only half the Linux gate — the capture side (GNOME 50+ portal monitor in HDR mode)
// is resolved separately by the host (`capturer_supports_hdr` / the GameStream RTSP
// honor), since this probe can't know what the compositor will negotiate.
if linux_auto_is_vaapi() {
vaapi::probe_can_encode_10bit(codec)
} else {
linux::probe_can_encode_10bit(codec)
}
}
#[cfg(target_os = "windows")]
{
+22
View File
@@ -56,6 +56,19 @@ pub enum PixelFormat {
/// `DeviceBuffer::yuv444` — three full-res planes stacked in one allocation); NVENC encodes
/// it natively under the Range-Extensions profile. Never a CPU payload.
Yuv444,
/// 10-bit RGB packed `x:R:G:B 2:10:10:10` little-endian (SPA `xRGB_210LE`, DRM `XRGB2101010` /
/// `XR30`, ffmpeg `x2rgb10le`, NVENC `ARGB10`) — as an LE u32: B in bits 0-9, G 10-19, R 20-29.
/// The Linux GNOME 50+ HDR screencast source format: Mutter advertises it (with BT.2020
/// primaries + SMPTE ST.2084 PQ transfer) for a monitor in HDR mode, so the samples are
/// PQ-encoded BT.2020 RGB. Linux-only; the Windows HDR path stays `Rgb10a2`/`P010`.
X2Rgb10,
/// 10-bit RGB packed `x:B:G:R 2:10:10:10` little-endian (SPA `xBGR_210LE`, DRM `XBGR2101010` /
/// `XB30`, ffmpeg `x2bgr10le`, NVENC `ABGR10`) — as an LE u32: R in bits 0-9, G 10-19, B 20-29;
/// the same memory layout as the Windows [`Rgb10a2`](Self::Rgb10a2) (DXGI `R10G10B10A2`). The
/// second GNOME 50+ HDR screencast format (same PQ/BT.2020 colorimetry as
/// [`X2Rgb10`](Self::X2Rgb10)); kept separate from `Rgb10a2` so the Linux and Windows HDR
/// paths stay independently greppable.
X2Bgr10,
}
impl PixelFormat {
@@ -67,6 +80,12 @@ impl PixelFormat {
_ => 4,
}
}
/// True for the packed 10-bit RGB layouts a Linux HDR (BT.2020 PQ) capture negotiates —
/// the formats that make a session's encode bit depth 10 (HEVC Main10 / 10-bit AV1).
pub fn is_hdr_rgb10(self) -> bool {
matches!(self, PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10)
}
}
/// DRM FourCC for a packed 32-bit format name (little-endian, e.g. `b"XR24"`).
@@ -86,6 +105,9 @@ pub fn drm_fourcc(format: PixelFormat) -> Option<u32> {
Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888
Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888
Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020).
X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010
X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010
// 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path.
// Rgb10a2/Nv12/P010 are the Windows HDR / video-processor formats — never produced on
// Linux; Yuv444 is OUR convert's OUTPUT, never a capture source format.
+17 -10
View File
@@ -12,7 +12,10 @@ use anyhow::Result;
// `crate::capture::*` (the capture mechanics that used the rest moved into pf-capture).
pub use pf_frame::{CapturedFrame, OutputFormat, PixelFormat};
// The capturer types + trait + synthetics live in `pf-capture`; re-export them at the old paths.
pub use pf_capture::{capturer_supports_444, Capturer, FastSyntheticCapturer, SyntheticCapturer};
pub use pf_capture::{
capturer_supports_444, capturer_supports_hdr, Capturer, FastSyntheticCapturer,
SyntheticCapturer,
};
// `crate::capture::dxgi::{install_gpu_pref_hook, hdr_p010_selftest}` (main.rs subcommands) and
// `crate::capture::synthetic_nv12` resolve through pf-capture's Windows modules.
#[cfg(target_os = "windows")]
@@ -45,18 +48,20 @@ fn zero_copy_policy() -> pf_capture::ZeroCopyPolicy {
}
}
/// Open a live capturer for a client-sized monitor via the xdg ScreenCast portal.
/// Open a live capturer for a client-sized monitor via the xdg ScreenCast portal. `want_hdr`
/// offers the GNOME 50+ 10-bit PQ/BT.2020 formats (pass it only when the session negotiated HDR
/// AND the mirrored monitor is in HDR mode — see [`pf_capture::gnome_hdr_monitor_active`]).
#[cfg(target_os = "linux")]
pub fn open_portal_monitor() -> Result<Box<dyn Capturer>> {
pub fn open_portal_monitor(want_hdr: bool) -> Result<Box<dyn Capturer>> {
// On RemoteDesktop-capable desktops (KWin/GNOME) anchor ScreenCast to a RemoteDesktop
// session so it inherits that grant headlessly; wlroots/Sway has no RemoteDesktop portal,
// so use a plain ScreenCast session there.
let anchored = crate::inject::default_backend() == crate::inject::Backend::Libei;
pf_capture::open_portal_monitor(anchored, zero_copy_policy())
pf_capture::open_portal_monitor(anchored, want_hdr, zero_copy_policy())
}
#[cfg(not(target_os = "linux"))]
pub fn open_portal_monitor() -> Result<Box<dyn Capturer>> {
pub fn open_portal_monitor(_want_hdr: bool) -> Result<Box<dyn Capturer>> {
anyhow::bail!("portal capture requires Linux (xdg-desktop-portal + PipeWire)")
}
@@ -69,11 +74,13 @@ pub fn capture_virtual_output(
want: OutputFormat,
_capture: crate::session_plan::CaptureBackend,
) -> Result<Box<dyn Capturer>> {
// The Linux host stays 8-bit (HDR is blocked upstream) and the portal negotiates its own pixel
// format, so `want.gpu` gates GPU zero-copy capture (the capture backend is always the portal
// the `CaptureBackend` arg is a Windows-only dispatch) and `want.chroma_444` selects the
// worker's planar-YUV444 GPU convert. `gpu = false` (4:4:4 without zero-copy) forces the CPU
// mmap path so the encoder gets CPU-resident RGB to swscale into YUV444P.
// The Linux NATIVE plane stays 8-bit (Mutter's virtual-monitor streams are SDR-only upstream;
// the GNOME 50+ HDR path is monitor-mirror only — `open_portal_monitor`) and the portal
// negotiates its own pixel format, so `want.gpu` gates GPU zero-copy capture (the capture
// backend is always the portal — the `CaptureBackend` arg is a Windows-only dispatch) and
// `want.chroma_444` selects the worker's planar-YUV444 GPU convert. `gpu = false` (4:4:4
// without zero-copy) forces the CPU mmap path so the encoder gets CPU-resident RGB to swscale
// into YUV444P.
pf_capture::open_virtual_output(
vout.remote_fd,
vout.node_id,
+38 -11
View File
@@ -50,21 +50,45 @@ pub const SCM_AV1_MAIN10: u32 = 0x0002_0000;
/// The **SDR baseline** codec mask: H.264, HEVC Main, AV1 Main 8-bit (= 65793). HEVC Main10 (HDR) is
/// layered on top of this at runtime by `serverinfo::codec_mode_support` when — and only when — the
/// host can actually deliver it ([`host_hdr_capable`]); it is never a static claim, because a non-HDR
/// host (Linux, or a Windows host without the `PUNKTFUNK_10BIT` opt-in) must not invite a client into
/// an HDR mode it can't produce. (The previous placeholder 3843 = 0xF03 wrongly claimed HEVC Main10 +
/// host (a host without the `PUNKTFUNK_10BIT` opt-in, or a Linux host whose video source / encoder
/// can't do Main10) must not invite a client into an HDR mode it can't produce. (The previous placeholder 3843 = 0xF03 wrongly claimed HEVC Main10 +
/// 4:4:4 and *no* AV1.) 4:4:4 stays off entirely on GameStream: stock Moonlight is 4:2:0 —
/// full-chroma is a punktfunk/1-native negotiation only (`crate::capture::capturer_supports_444`).
pub const SERVER_CODEC_MODE_SUPPORT: u32 = SCM_H264 | SCM_HEVC | SCM_AV1_MAIN8;
/// Whether this host can deliver an **HDR** (HEVC Main10 / BT.2020 PQ) GameStream — the single gate
/// for advertising [`SCM_HEVC_MAIN10`] in serverinfo and `IsHdrSupported` per app, and for honoring a
/// client's `dynamicRangeMode` request. HDR capture+encode is **Windows-only** (the Linux host is
/// 8-bit, blocked upstream) and behind the operator's `PUNKTFUNK_10BIT` opt-in — the same policy gate
/// the native punktfunk/1 plane honors. When this is true the IDD-push capturer streams HEVC Main10 PQ
/// whenever the desktop is HDR, and a client HDR request makes the GameStream video path proactively
/// enable advanced color on the per-session virtual display so PQ flows even from an SDR desktop.
/// for advertising [`SCM_HEVC_MAIN10`] in serverinfo and `IsHdrSupported` per app, and (together
/// with the live capture-side check at RTSP time) for honoring a client's `dynamicRangeMode`
/// request. Behind the operator's `PUNKTFUNK_10BIT` opt-in — the same policy gate the native
/// punktfunk/1 plane honors — on both OSes.
///
/// **Windows**: the IDD-push capturer streams HEVC Main10 PQ whenever the desktop is HDR, and a
/// client HDR request proactively enables advanced color on the per-session virtual display so PQ
/// flows even from an SDR desktop.
///
/// **Linux**: the GNOME 50+ portal **monitor mirror** (`video_source=portal`) can negotiate the
/// 10-bit PQ formats while the mirrored monitor is in HDR mode, and the NVENC/VAAPI encoders have
/// a probed Main10 path ([`crate::encode::can_encode_10bit`]). The virtual-output source stays SDR
/// (Mutter's RecordVirtual streams are 8-bit-only upstream), so this is `false` for it. Whether
/// the monitor is ACTUALLY in HDR mode right now is checked live at RTSP honor time
/// ([`pf_capture::gnome_hdr_monitor_active`]) — this fn is the static serverinfo capability.
pub fn host_hdr_capable() -> bool {
cfg!(target_os = "windows") && pf_host_config::config().ten_bit
if !pf_host_config::config().ten_bit {
return false;
}
#[cfg(target_os = "windows")]
{
true
}
#[cfg(target_os = "linux")]
{
pf_host_config::config().video_source.as_deref() == Some("portal")
&& crate::encode::can_encode_10bit(crate::encode::Codec::H265)
}
#[cfg(not(any(target_os = "windows", target_os = "linux")))]
{
false
}
}
/// Stable host identity + advertised capabilities, shared across control-plane handlers.
@@ -141,8 +165,11 @@ pub struct AppState {
pub rfi_range: std::sync::Arc<std::sync::Mutex<Option<(i64, i64)>>>,
/// Persistent screen capturer, reused across streams so reconnects don't spawn a second
/// (conflicting) screencast session. The video thread borrows it for the stream's duration
/// and returns it; `set_active` gates its cost while idle.
pub video_cap: std::sync::Arc<std::sync::Mutex<Option<Box<dyn crate::capture::Capturer>>>>,
/// and returns it; `set_active` gates its cost while idle. The slot's `bool` records whether
/// it was opened with the HDR (10-bit PQ) offer — a stream whose negotiated `hdr` differs
/// drops the pooled capturer and opens a fresh screencast session at the right depth
/// (mirroring the audio capturer's channel-count reuse gate).
pub video_cap: stream::CapturerSlot,
/// Persistent audio capturer, reused across streams when the channel count still matches
/// (avoids a PipeWire stream setup per reconnect); drained on reuse so no stale audio is
/// sent, dropped + reopened when a session negotiates a different channel count.
+18 -6
View File
@@ -396,18 +396,30 @@ fn stream_config(map: &HashMap<String, String>) -> Option<StreamConfig> {
_ => Codec::H264,
};
// 10-bit/HDR request (Moonlight sets `dynamicRangeMode != 0` only when it both saw our Main10 SCM
// bit AND the user enabled HDR). Honor it only when the host can actually deliver Main10 (Windows +
// PUNKTFUNK_10BIT, `host_hdr_capable`); when honored, the video path proactively enables advanced
// color on the virtual display so a PQ stream flows even from an SDR desktop. A request we can't
// honor degrades to 8-bit SDR (and a desktop that is ALREADY HDR still streams PQ regardless, since
// the IDD-push capturer follows the display).
// bit AND the user enabled HDR). Honor it only when the host can actually deliver Main10
// (`host_hdr_capable` — Windows IDD-push, or the Linux GNOME 50+ portal mirror). On Windows,
// when honored, the video path proactively enables advanced color on the virtual display so a
// PQ stream flows even from an SDR desktop. On Linux the portal can only deliver PQ while the
// MIRRORED monitor is in HDR mode, so additionally probe the live colour mode here (one D-Bus
// round-trip, sync RTSP thread) — an SDR desktop honestly degrades to 8-bit SDR up front
// instead of running the capture negotiation into its timeout. A request we can't honor
// degrades to 8-bit SDR (and a Windows desktop that is ALREADY HDR still streams PQ
// regardless, since the IDD-push capturer follows the display).
let hdr_requested = parse_u("x-nv-video[0].dynamicRangeMode").unwrap_or(0) != 0;
let hdr = hdr_requested && crate::gamestream::host_hdr_capable();
let mut hdr = hdr_requested && crate::gamestream::host_hdr_capable();
if hdr_requested && !hdr {
tracing::warn!(
"client requested HDR (dynamicRangeMode != 0) but host is not HDR-capable — streaming 8-bit SDR"
);
}
#[cfg(target_os = "linux")]
if hdr && !pf_capture::gnome_hdr_monitor_active() {
tracing::warn!(
"client requested HDR but no monitor is in BT.2100 (HDR) colour mode — enable HDR in \
GNOME Settings Displays (GNOME 50+) to stream it; streaming 8-bit SDR"
);
hdr = false;
}
// The client's requested CSC (moonlight-common-c SdpGenerator.c: `encoderCscMode =
// (colorspace << 1) | fullRange` — colorspace 0=Rec601, 1=Rec709, 2=Rec2020). Moonlight
// renderers configure their YUV→RGB from this REQUESTED value (not the bitstream VUI), so a
+33 -10
View File
@@ -34,8 +34,9 @@ pub struct StreamConfig {
}
/// Slot for the persistent screen capturer, shared with the control plane and reused across
/// streams so a reconnect doesn't open a second (conflicting) screencast session.
pub type CapturerSlot = Arc<std::sync::Mutex<Option<Box<dyn Capturer>>>>;
/// streams so a reconnect doesn't open a second (conflicting) screencast session. The `bool` is
/// the pooled capturer's HDR-ness (see `AppState::video_cap`).
pub type CapturerSlot = Arc<std::sync::Mutex<Option<(Box<dyn Capturer>, bool)>>>;
/// A pending client reference-frame-invalidation range (lost `firstFrame..=lastFrame`), set by the
/// control plane and drained by the video thread (see [`AppState::rfi_range`](super::AppState)).
@@ -120,7 +121,7 @@ fn run(
running: &Arc<AtomicBool>,
force_idr: &AtomicBool,
rfi_range: &std::sync::Mutex<Option<(i64, i64)>>,
video_cap: &std::sync::Mutex<Option<Box<dyn Capturer>>>,
video_cap: &std::sync::Mutex<Option<(Box<dyn Capturer>, bool)>>,
// Shared stats recorder for the web-console capture/graph. Threaded into `stream_body` (the
// encode loop); per-frame sample emission is wired by a later pass.
stats: &Arc<crate::stats_recorder::StatsRecorder>,
@@ -243,15 +244,31 @@ fn run(
}
// Reuse the persistent capturer (one screencast session → clean reconnect); create it on
// the first stream. Borrow it for this stream and return it on exit.
let mut capturer: Box<dyn Capturer> = match video_cap.lock().unwrap().take() {
// the first stream. Borrow it for this stream and return it on exit. Reuse is gated on the
// pooled capturer's HDR-ness matching this stream's negotiated `cfg.hdr` — the depth is a
// PipeWire-negotiation-time property of the screencast session, so an HDR↔SDR change needs a
// fresh session (same pattern as the audio capturer's channel-count gate).
let pooled = match video_cap.lock().unwrap().take() {
Some((c, was_hdr)) if was_hdr == cfg.hdr => Some(c),
Some((c, was_hdr)) => {
tracing::info!(
was_hdr,
want_hdr = cfg.hdr,
"video source: pooled capturer depth mismatch — opening a fresh screencast session"
);
drop(c);
None
}
None => None,
};
let mut capturer: Box<dyn Capturer> = match pooled {
Some(c) => {
tracing::info!("video source: reusing capturer");
c
}
None if pf_host_config::config().video_source.as_deref() == Some("portal") => {
tracing::info!("video source: portal desktop capture");
capture::open_portal_monitor().context("open portal capturer")?
tracing::info!(hdr = cfg.hdr, "video source: portal desktop capture");
capture::open_portal_monitor(cfg.hdr).context("open portal capturer")?
}
None => {
tracing::info!("video source: synthetic test pattern");
@@ -272,7 +289,7 @@ fn run(
&client_label,
);
capturer.set_active(false);
*video_cap.lock().unwrap() = Some(capturer);
*video_cap.lock().unwrap() = Some((capturer, cfg.hdr));
result
}
@@ -380,7 +397,9 @@ fn open_gs_virtual_source(
// HDR: pass the negotiated `cfg.hdr` (client asked for HDR AND the host can deliver it). On the
// Windows IDD-push path this proactively enables advanced color on the virtual display so a Main10
// PQ stream flows even from an SDR desktop; an already-HDR desktop streams PQ regardless (the
// capturer follows the display). No-op on Linux (8-bit, and `cfg.hdr` is always false there).
// capturer follows the display). No-op on Linux: virtual-output capture is SDR-only upstream
// (Mutter RecordVirtual), and `host_hdr_capable` therefore keeps `cfg.hdr` false for this
// source — the Linux HDR path is the portal monitor mirror (`video_source=portal`).
let capturer = capture::capture_virtual_output(
vout,
capture::OutputFormat::resolve(cfg.hdr, crate::encode::resolved_backend_is_gpu()),
@@ -399,7 +418,11 @@ fn open_gs_virtual_source(
fn gs_bit_depth(format: crate::capture::PixelFormat) -> u8 {
use crate::capture::PixelFormat;
match format {
PixelFormat::P010 | PixelFormat::Rgb10a2 => 10,
// Windows IDD-push HDR formats, and the Linux GNOME 50+ portal HDR formats.
PixelFormat::P010
| PixelFormat::Rgb10a2
| PixelFormat::X2Rgb10
| PixelFormat::X2Bgr10 => 10,
_ => 8,
}
}
+18
View File
@@ -283,6 +283,24 @@ fn real_main() -> Result<()> {
// PASS/FAIL + max Y/Cb/Cr error.
#[cfg(target_os = "windows")]
Some("hdr-p010-selftest") => crate::capture::dxgi::hdr_p010_selftest(),
// Linux HDR readiness probe (GNOME 50+ portal path): prints whether a monitor is currently
// in BT.2100 (HDR) colour mode, whether the NVENC/VAAPI backend probes Main10 for
// HEVC/AV1, and the GameStream HDR capability the two combine into — the "why isn't my
// stream HDR?" diagnostic (no display/session needed for the encoder half).
#[cfg(target_os = "linux")]
Some("hdr-probe") => {
let monitor_hdr = pf_capture::gnome_hdr_monitor_active();
let hevc10 = encode::can_encode_10bit(encode::Codec::H265);
let av110 = encode::can_encode_10bit(encode::Codec::Av1);
println!("monitor in BT.2100 (HDR) colour mode: {monitor_hdr}");
println!("encoder Main10 (HEVC): {hevc10}");
println!("encoder 10-bit (AV1): {av110}");
println!(
"GameStream HDR capable (PUNKTFUNK_10BIT + video_source=portal + encoder): {}",
gamestream::host_hdr_capable()
);
Ok(())
}
// Compositor readiness probe: exit 0 iff the (detected or PUNKTFUNK_COMPOSITOR-forced)
// compositor is up and able to create a virtual output *now*. A session-bringup
// script polls this to gate on real readiness instead of a blind `sleep`.
+14 -1
View File
@@ -212,10 +212,22 @@ pub(super) async fn negotiate(
// label that matches the stream.
let host_wants_10bit = pf_host_config::config().ten_bit;
let client_supports_10bit = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_10BIT != 0;
// The capture side must be able to deliver a 10-bit HDR source for the NATIVE plane's
// virtual-output capture — the honest-downgrade gate, mirroring `capturer_supports_444`.
// Windows IDD-push can (it proactively enables advanced colour); Linux cannot: Mutter's
// RecordVirtual virtual-monitor streams are 8-bit-only upstream (GNOME 50 added HDR for
// *monitor* streams only — the GameStream portal-mirror path uses that; see
// `gamestream::host_hdr_capable`), so a Linux native session honestly stays 8-bit SDR even
// though `can_encode_10bit` now probes true on a Main10-capable GPU.
let capture_supports_hdr = crate::capture::capturer_supports_hdr();
// The GPU probe may open a tiny encoder on first use, so run it off the reactor like the
// 4:4:4 probe below (blocking probes → spawn_blocking), short-circuited behind the cheap
// gates. The result is cached process-wide per (GPU, codec).
let gpu_can_10bit = if host_wants_10bit && client_supports_10bit && codec.supports_10bit() {
let gpu_can_10bit = if host_wants_10bit
&& client_supports_10bit
&& codec.supports_10bit()
&& capture_supports_hdr
{
tokio::task::spawn_blocking(move || crate::encode::can_encode_10bit(codec))
.await
.context("10-bit capability probe task")?
@@ -227,6 +239,7 @@ pub(super) async fn negotiate(
bit_depth,
host_wants_10bit,
client_supports_10bit,
capture_supports_hdr,
codec = ?codec,
gpu_can_10bit,
client_video_caps = hello.video_caps,
+3 -1
View File
@@ -89,7 +89,9 @@ pub struct SessionPlan {
/// Handshake-negotiated encode bit depth (8, or 10 = HEVC Main10).
pub bit_depth: u8,
/// The IDD-push HDR hint (`bit_depth >= 10`) — the want-HDR flag handed to the capturer so it
/// proactively enables advanced color on the virtual display. Linux is 8-bit (HDR blocked upstream).
/// proactively enables advanced color on the virtual display. The Linux NATIVE plane is 8-bit
/// (Mutter's virtual-monitor streams are SDR-only upstream — GNOME 50 HDR is monitor-mirror
/// only, which the GameStream portal path uses; see `capture::capturer_supports_hdr`).
pub hdr: bool,
/// Handshake-negotiated chroma subsampling (4:2:0, or full-chroma 4:4:4 when the client + host +
/// GPU all support it). Resolved before the Welcome; `Yuv420` on every backend that declined it.
+5 -2
View File
@@ -87,8 +87,11 @@ pub fn run(opts: Options) -> Result<()> {
}
}
Source::Portal => {
tracing::info!("spike source: xdg ScreenCast portal (live monitor)");
capture::open_portal_monitor().context("open portal capturer")?
// PUNKTFUNK_SPIKE_HDR=1: run the GNOME 50+ HDR offer (10-bit PQ dmabufs) — the dev
// validation lever for the Linux HDR capture path without a full GameStream client.
let want_hdr = std::env::var("PUNKTFUNK_SPIKE_HDR").as_deref() == Ok("1");
tracing::info!(want_hdr, "spike source: xdg ScreenCast portal (live monitor)");
capture::open_portal_monitor(want_hdr).context("open portal capturer")?
}
Source::KwinVirtual => {
let compositor = crate::vdisplay::detect().unwrap_or(crate::vdisplay::Compositor::Kwin);
+1 -1
View File
@@ -88,7 +88,7 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t
| Setting | Values | Meaning |
|---|---|---|
| `PUNKTFUNK_FEC_PCT` | `N` (percent) | Forward-error-correction redundancy for lossy links (the default is sensible for a normal LAN). Higher = more loss-resilient, more bandwidth. |
| `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. **Windows host only** (the Linux host stays 8-bit). |
| `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. Windows host, plus the Linux **GNOME 50+ GameStream desktop mirror** (`PUNKTFUNK_VIDEO_SOURCE=portal`, mirrored monitor in HDR mode — check with `punktfunk-host hdr-probe`). Linux **virtual displays** (native protocol, GameStream default) stay 8-bit: Mutter's virtual-monitor screencast is SDR-only upstream. |
| `PUNKTFUNK_444` | `1` · `0` *(default on)* | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **On by default** on the host; the client's own 4:4:4 setting (default off) is the real switch. Set `0` to force 4:2:0. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. |
| `PUNKTFUNK_PYROWAVE_MAX_MBPS` | `N` (Mbps) | Cap the [PyroWave](/docs/pyrowave) Automatic bitrate pin, for a host on a link that the open-loop pin can outrun (e.g. 4:4:4 + HDR at 5120×1440@240 pins ~5.3 Gbps, over a 5GbE link). Unset = no cap. Only affects Automatic (bitrate `0`) PyroWave sessions; an explicit client bitrate bypasses it. |
| `PUNKTFUNK_DSCP` | `1` | Opt-in DSCP / `SO_PRIORITY` QoS tagging on the media sockets. No-op on the wire on Windows without a qWAVE policy. |
+9 -5
View File
@@ -24,7 +24,7 @@ see [Status & Progress](/docs/status).
| Web console + pairing | ✅ |
| Concurrent sessions (shared desktop) | ✅ |
| Network speed test + bitrate | ✅ |
| HDR / 10-bit streaming | ✅ Windows host · Linux host |
| HDR / 10-bit streaming | ✅ Windows host · 🚧 Linux host (GNOME 50+ desktop mirror; virtual displays blocked upstream) |
| Surround audio (5.1 / 7.1) | ✅ |
| Sub-frame pipelining (latency) | 🔭 |
@@ -100,10 +100,14 @@ see [Status & Progress](/docs/status).
## ⛔ Parked / blocked
- **HDR / 10-bit on the *Linux* host.** HDR streaming already works from a
[Windows host](/docs/windows-host) to an HDR-capable client (Windows, Android). On Linux it's
blocked upstream — no shipping compositor emits a 10-bit/HDR capture stream yet — and ready the
moment one does.
- **HDR / 10-bit on the *Linux* host — virtual displays.** GNOME 50 added HDR screencasting for
**monitor** streams, and the host now uses it: the GameStream desktop mirror
(`PUNKTFUNK_VIDEO_SOURCE=portal`) negotiates the 10-bit PQ formats and encodes HEVC Main10
BT.2020 PQ (`punktfunk-host hdr-probe` reports readiness; on-glass validation pending). What
stays blocked upstream is HDR on **virtual monitors** — Mutter's `RecordVirtual` streams are
still SDR-only (through the GNOME 51 dev branch), so the native protocol and GameStream's
default virtual-display source stream 8-bit until that lands; the host is ready the moment it
does.
- **Advanced DualSense voice-coil haptics.** Scoped and shelved (it rides the controller's USB audio
interface, with near-zero game support on Linux). Adaptive triggers, rumble, and the lightbar
already ship.
+7 -3
View File
@@ -35,9 +35,13 @@ host is newer than the Linux host.)
- **Zero-copy GPU pipeline.** Captured frames stay on the GPU — dmabuf → CUDA → NVENC on NVIDIA, and
VAAPI or Vulkan Video on AMD/Intel — with automatic split-encode at very high resolutions. Stable
240 fps at 5120×1440 has been measured. A GPU-less software H.264 encoder exists as an explicit fallback.
- **HDR (10-bit), on the Windows host.** An HDR Windows desktop is captured and encoded as HEVC
Main10 (BT.2020 PQ) to HDR-capable clients (Windows, Android). Linux hosts stream 8-bit for now —
HDR there is blocked upstream at the compositor.
- **HDR (10-bit).** An HDR Windows desktop is captured and encoded as HEVC Main10 (BT.2020 PQ) to
HDR-capable clients (Windows, Android). On Linux, a **GNOME 50+** host can mirror an HDR monitor
over the GameStream desktop-capture source (`PUNKTFUNK_VIDEO_SOURCE=portal`): the portal
negotiates the 10-bit PQ screencast formats GNOME 50 added and encodes Main10 PQ (run
`punktfunk-host hdr-probe` to check readiness; pending on-glass validation). Linux **virtual
displays** — the native protocol and GameStream's default source — still stream 8-bit: Mutter's
virtual-monitor screencast is SDR-only upstream.
- **Secure by default.** A **SPAKE2 PIN pairing** ceremony establishes trust (the host
shows a 4-digit PIN; an attacker gets a single online guess, no offline dictionary
attack). Trust-on-first-use (TOFU) remains an explicit opt-in for fully trusted LANs.