feat(hdr): GNOME 50 HDR screencast capture + Linux Main10 encode
ci / docs-site (push) Successful in 58s
ci / rust (push) Failing after 1m3s
ci / web (push) Successful in 1m4s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 19s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 46s
windows-host / package (push) Failing after 5m38s
ci / bench (push) Successful in 6m12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m38s
apple / screenshots (push) Successful in 6m25s
docker / deploy-docs (push) Successful in 29s
deb / build-publish-host (push) Failing after 7m30s
deb / build-publish (push) Successful in 11m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m59s
arch / build-publish (push) Successful in 11m57s
android / android (push) Successful in 15m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m46s
ci / docs-site (push) Successful in 58s
ci / rust (push) Failing after 1m3s
ci / web (push) Successful in 1m4s
apple / swift (push) Successful in 1m22s
decky / build-publish (push) Successful in 19s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 10s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 46s
windows-host / package (push) Failing after 5m38s
ci / bench (push) Successful in 6m12s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 6m38s
apple / screenshots (push) Successful in 6m25s
docker / deploy-docs (push) Successful in 29s
deb / build-publish-host (push) Failing after 7m30s
deb / build-publish (push) Successful in 11m22s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 8m59s
arch / build-publish (push) Successful in 11m57s
android / android (push) Successful in 15m26s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m46s
GNOME 50 (Mutter MR 4928, PipeWire >= 1.6) added HDR screen sharing for monitor streams: 10-bit PQ formats (xRGB_210LE/xBGR_210LE) with MANDATORY BT.2020 + SMPTE-2084 colorimetry props, advertised while the mirrored monitor is in BT.2100 colour mode. Wire the Linux host into it end-to-end on the GameStream desktop-mirror path (PUNKTFUNK_VIDEO_SOURCE=portal): * pf-frame: PixelFormat::X2Rgb10/X2Bgr10 (DRM XR30/XB30; X2Bgr10 is the Windows Rgb10a2 layout) + fourccs. * pf-capture: want_hdr portal offer — HDR-only LINEAR-dmabuf pods with MANDATORY PQ/BT.2020 props (SHM excluded: Mutter's SHM record path paints 8-bit ARGB32 regardless of format; tiled excluded: the EGL de-tile blit is 8-bit RGBA8), negotiated-colorimetry parse, generic HDR10 hdr_meta(), packed-10-bit CPU cursor blend, a process-wide SDR downgrade latch on negotiation timeout, and a DisplayConfig BT.2100 colour-mode probe (gnome_hdr_monitor_active). * pf-encode: libav NVENC X2RGB10->P010 swscale (BT.2020 limited) -> HEVC Main10 / 10-bit AV1 with PQ VUI; VAAPI 10-bit on both paths (CPU P010 upload + dmabuf XR30 scale_vaapi p010/bt2020); can_encode_10bit now probes for real on Linux; 10-bit sessions route around the 8-bit-only Vulkan-video/direct-NVENC backends. * GameStream: host_hdr_capable() Linux arm, live monitor-HDR check at RTSP honor time, capturer-pool reuse keyed on HDR-ness, gs_bit_depth covers the new formats. New `punktfunk-host hdr-probe` diagnostic and a PUNKTFUNK_SPIKE_HDR spike lever. * Native plane stays honestly 8-bit via capturer_supports_hdr(): Mutter RecordVirtual streams are SDR-only upstream (GNOME 50 and 51-dev), so virtual-display sources cannot deliver HDR yet. Validated on the RTX 5070 Ti (GNOME 50.3 / PipeWire 1.6.8): the Main10 probes pass and the ignored nvenc_hdr10_smoke GPU test emits an IDR that ffprobe reads as Main 10 / yuv420p10le / bt2020nc / smpte2084 / limited. Live HDR capture negotiation still needs an HDR monitor on glass; VAAPI 10-bit needs the AMD box. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -254,6 +254,54 @@ pub struct ZeroCopyPolicy {
|
||||
pub fn capturer_supports_444(_encoder_ingests_rgb_444: bool) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
/// Whether the **native-plane** capturer (a compositor virtual output) can deliver an HDR (10-bit
|
||||
/// PQ/BT.2020) source on this platform — the capture-side gate the punktfunk/1 handshake consults
|
||||
/// before negotiating 10-bit (mirroring [`capturer_supports_444`]).
|
||||
///
|
||||
/// Linux: `false`. GNOME 50 added HDR **screen sharing** for *monitor* streams only — Mutter's
|
||||
/// `RecordVirtual` virtual-monitor streams advertise 8-bit BGRx/BGRA exclusively (still true on
|
||||
/// the GNOME 51 dev branch), and virtual outputs report no BT2020/PQ colour capabilities, so they
|
||||
/// can't be flipped into HDR mode via DisplayConfig either. The Linux HDR path that DOES exist —
|
||||
/// the GNOME 50+ portal **monitor mirror** (`open_portal_monitor` with `want_hdr`) — is gated
|
||||
/// separately by the GameStream plane (`host_hdr_capable` + the live monitor colour-mode probe).
|
||||
#[cfg(target_os = "linux")]
|
||||
pub fn capturer_supports_hdr() -> bool {
|
||||
false
|
||||
}
|
||||
/// Windows: the IDD-push capturer proactively enables advanced colour and delivers P010/Rgb10a2.
|
||||
#[cfg(target_os = "windows")]
|
||||
pub fn capturer_supports_hdr() -> bool {
|
||||
true
|
||||
}
|
||||
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
|
||||
pub fn capturer_supports_hdr() -> bool {
|
||||
false
|
||||
}
|
||||
|
||||
/// Process-wide latch: a `want_hdr` portal capture failed to negotiate the HDR (10-bit PQ) offer —
|
||||
/// the compositor never accepted it (monitor left HDR mode between the probe and the negotiation,
|
||||
/// NVIDIA EGL not listing LINEAR for XR30, a pre-50 Mutter…). Later sessions consult
|
||||
/// [`hdr_capture_failed`] and fall back to the SDR offer instead of re-running the same doomed
|
||||
/// 10-second negotiation timeout on every reconnect. Sticky until host restart (matching the
|
||||
/// zero-copy downgrade latches); the log line at latch time says so.
|
||||
#[cfg(target_os = "linux")]
|
||||
static HDR_CAPTURE_FAILED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
pub fn hdr_capture_failed() -> bool {
|
||||
HDR_CAPTURE_FAILED.load(std::sync::atomic::Ordering::Relaxed)
|
||||
}
|
||||
|
||||
#[cfg(target_os = "linux")]
|
||||
pub(crate) fn note_hdr_capture_failed() {
|
||||
if !HDR_CAPTURE_FAILED.swap(true, std::sync::atomic::Ordering::Relaxed) {
|
||||
tracing::warn!(
|
||||
"HDR capture negotiation failed — this host will offer SDR capture for the rest of \
|
||||
the process lifetime (restart the host after fixing the monitor's HDR mode to retry)"
|
||||
);
|
||||
}
|
||||
}
|
||||
#[cfg(target_os = "windows")]
|
||||
pub fn capturer_supports_444(encoder_ingests_rgb_444: bool) -> bool {
|
||||
// IDD-push delivers full-chroma BGRA for an SDR 4:4:4 session (skipping the NV12 VideoConverter),
|
||||
@@ -316,16 +364,28 @@ pub use idd_push::verify_is_wudfhost;
|
||||
#[cfg(target_os = "linux")]
|
||||
#[path = "linux/mod.rs"]
|
||||
mod linux;
|
||||
// The GNOME BT.2100 colour-mode probe — the host's capture-side gate for offering HDR on the
|
||||
// portal monitor path (see `open_portal_monitor`'s `want_hdr`).
|
||||
#[cfg(target_os = "linux")]
|
||||
pub use linux::gnome_hdr_monitor_active;
|
||||
#[cfg(target_os = "windows")]
|
||||
#[path = "windows/synthetic_nv12.rs"]
|
||||
pub mod synthetic_nv12;
|
||||
|
||||
/// Open the Linux xdg-ScreenCast portal capturer for a client-sized monitor. `anchored` drives
|
||||
/// ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits that grant headlessly. The
|
||||
/// [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts (the one-way edge).
|
||||
/// ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits that grant headlessly.
|
||||
/// `want_hdr` offers the GNOME 50+ HDR formats (10-bit PQ/BT.2020 dmabufs) instead of the SDR
|
||||
/// set — pass it only when the mirrored monitor is actually in HDR mode (the host probes
|
||||
/// DisplayConfig) or the negotiation runs into its 10 s timeout and latches the SDR downgrade.
|
||||
/// The [`ZeroCopyPolicy`] carries the pre-resolved encode-backend facts (the one-way edge).
|
||||
#[cfg(target_os = "linux")]
|
||||
pub fn open_portal_monitor(anchored: bool, policy: ZeroCopyPolicy) -> Result<Box<dyn Capturer>> {
|
||||
linux::PortalCapturer::open(anchored, policy).map(|c| Box::new(c) as Box<dyn Capturer>)
|
||||
pub fn open_portal_monitor(
|
||||
anchored: bool,
|
||||
want_hdr: bool,
|
||||
policy: ZeroCopyPolicy,
|
||||
) -> Result<Box<dyn Capturer>> {
|
||||
linux::PortalCapturer::open(anchored, want_hdr && !hdr_capture_failed(), policy)
|
||||
.map(|c| Box::new(c) as Box<dyn Capturer>)
|
||||
}
|
||||
|
||||
/// Open the Linux portal capturer bound to an already-created virtual output's PipeWire node. The
|
||||
|
||||
@@ -62,6 +62,13 @@ pub struct PortalCapturer {
|
||||
/// the process-wide downgrade ([`pf_zerocopy::note_vaapi_dmabuf_failed`]) so the pipeline
|
||||
/// rebuild retries on the CPU offer instead of failing identically forever.
|
||||
vaapi_dmabuf: bool,
|
||||
/// This capture ran the HDR (10-bit PQ/BT.2020 dmabuf) offer — see [`Self::open`]'s
|
||||
/// `want_hdr`. Read by the negotiation-timeout diagnosis (a failed HDR offer latches the
|
||||
/// process-wide SDR downgrade) and by [`hdr_meta`](Capturer::hdr_meta).
|
||||
hdr_offer: bool,
|
||||
/// Set once the stream negotiated one of the 10-bit PQ formats (`param_changed`), i.e. frames
|
||||
/// really are PQ/BT.2020 — drives [`hdr_meta`](Capturer::hdr_meta).
|
||||
hdr_negotiated: Arc<AtomicBool>,
|
||||
/// The PipeWire node this capturer consumes — surfaced in error messages for diagnosis.
|
||||
node_id: u32,
|
||||
/// Stops the PipeWire loop on teardown (sent in `Drop`). Without it a dropped or failed
|
||||
@@ -80,8 +87,9 @@ pub struct PortalCapturer {
|
||||
impl PortalCapturer {
|
||||
/// `anchored` drives ScreenCast off a RemoteDesktop session (KWin/GNOME) so it inherits the
|
||||
/// RemoteDesktop grant and never raises a separate ScreenCast dialog; `false` uses a plain
|
||||
/// ScreenCast session (wlroots, which has no RemoteDesktop portal).
|
||||
pub fn open(anchored: bool, policy: ZeroCopyPolicy) -> Result<PortalCapturer> {
|
||||
/// ScreenCast session (wlroots, which has no RemoteDesktop portal). `want_hdr` offers the
|
||||
/// GNOME 50+ HDR formats (10-bit PQ/BT.2020, dmabuf-only) instead of the SDR set.
|
||||
pub fn open(anchored: bool, want_hdr: bool, policy: ZeroCopyPolicy) -> Result<PortalCapturer> {
|
||||
// Portal handshake (async) on its own thread; hands back the PW fd + node id.
|
||||
let (setup_tx, setup_rx) = std::sync::mpsc::channel::<Result<(OwnedFd, u32), String>>();
|
||||
thread::Builder::new()
|
||||
@@ -102,11 +110,12 @@ impl PortalCapturer {
|
||||
};
|
||||
tracing::info!(
|
||||
node_id,
|
||||
want_hdr,
|
||||
"ScreenCast portal session started; connecting PipeWire"
|
||||
);
|
||||
// This portal path (GameStream / monitor capture) is always 4:2:0, so allow zero-copy as before.
|
||||
Ok(
|
||||
spawn_pipewire(Some(fd), node_id, None, true, false, policy)?
|
||||
spawn_pipewire(Some(fd), node_id, None, true, false, want_hdr, policy)?
|
||||
.into_capturer(node_id, None),
|
||||
)
|
||||
}
|
||||
@@ -135,12 +144,15 @@ impl PortalCapturer {
|
||||
want_444,
|
||||
"connecting PipeWire to virtual output"
|
||||
);
|
||||
// Virtual outputs are SDR-only upstream (Mutter's RecordVirtual streams advertise 8-bit
|
||||
// BGRx/BGRA exclusively, GNOME 50 and 51-dev alike) — never run the HDR offer here.
|
||||
Ok(spawn_pipewire(
|
||||
remote_fd,
|
||||
node_id,
|
||||
preferred_mode,
|
||||
allow_zerocopy,
|
||||
want_444,
|
||||
false,
|
||||
policy,
|
||||
)?
|
||||
.into_capturer(node_id, Some(keepalive)))
|
||||
@@ -160,6 +172,10 @@ struct PwHandles {
|
||||
/// This capture will offer LINEAR-dmabuf-only for the VAAPI passthrough (see
|
||||
/// [`PortalCapturer::vaapi_dmabuf`]).
|
||||
vaapi_dmabuf: bool,
|
||||
/// This capture ran the HDR offer (see [`PortalCapturer::hdr_offer`]).
|
||||
hdr_offer: bool,
|
||||
/// See [`PortalCapturer::hdr_negotiated`].
|
||||
hdr_negotiated: Arc<AtomicBool>,
|
||||
quit: ::pipewire::channel::Sender<()>,
|
||||
join: thread::JoinHandle<()>,
|
||||
}
|
||||
@@ -177,6 +193,8 @@ impl PwHandles {
|
||||
broken: self.broken,
|
||||
stall_since: None,
|
||||
vaapi_dmabuf: self.vaapi_dmabuf,
|
||||
hdr_offer: self.hdr_offer,
|
||||
hdr_negotiated: self.hdr_negotiated,
|
||||
node_id,
|
||||
quit: Some(self.quit),
|
||||
join: Some(self.join),
|
||||
@@ -199,6 +217,10 @@ fn spawn_pipewire(
|
||||
// 4:4:4 session: tiled dmabufs convert to planar YUV444 on the GPU (`ImportKind::Tiled444`)
|
||||
// instead of NV12/RGB, so the session stays zero-copy at full chroma.
|
||||
want_444: bool,
|
||||
// HDR session (GNOME 50+ monitor mirror): offer ONLY the 10-bit PQ/BT.2020 formats as
|
||||
// LINEAR dmabufs (SHM can't carry them — Mutter's SHM record path paints 8-bit ARGB32
|
||||
// regardless of the negotiated format, and the tiled EGL de-tile blit is 8-bit).
|
||||
want_hdr: bool,
|
||||
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
|
||||
// capture→encode edge (plan §W6).
|
||||
policy: ZeroCopyPolicy,
|
||||
@@ -213,17 +235,29 @@ fn spawn_pipewire(
|
||||
let streaming_cb = streaming.clone();
|
||||
let broken = Arc::new(AtomicBool::new(false));
|
||||
let broken_cb = broken.clone();
|
||||
let hdr_negotiated = Arc::new(AtomicBool::new(false));
|
||||
let hdr_negotiated_cb = hdr_negotiated.clone();
|
||||
// pipewire's own cross-thread channel: the receiver attaches to the loop and quits it; the
|
||||
// sender lives on the capturer and fires in its `Drop`. Absolute `::pipewire` path — the
|
||||
// inner `mod pipewire` shadows the crate name at this scope.
|
||||
let (quit_tx, quit_rx) = ::pipewire::channel::channel::<()>();
|
||||
let zerocopy = allow_zerocopy && pf_zerocopy::enabled();
|
||||
// HDR cannot ride the SHM path (see `want_hdr` above): under PUNKTFUNK_FORCE_SHM the HDR
|
||||
// offer is dropped — SDR capture, loudly.
|
||||
let force_shm = std::env::var("PUNKTFUNK_FORCE_SHM").as_deref() == Ok("1");
|
||||
let want_hdr = if want_hdr && force_shm {
|
||||
tracing::warn!(
|
||||
"HDR capture requested but PUNKTFUNK_FORCE_SHM=1 — the SHM path is 8-bit only; \
|
||||
offering SDR"
|
||||
);
|
||||
false
|
||||
} else {
|
||||
want_hdr
|
||||
};
|
||||
// Mirror of the thread's `vaapi_passthrough` decision (deterministic from here: on a VAAPI
|
||||
// backend the EGL→CUDA importer is never built) — kept on the capturer so `next_frame`'s
|
||||
// negotiation-timeout branch knows a failed negotiation was the LINEAR-dmabuf offer.
|
||||
let vaapi_dmabuf = zerocopy
|
||||
&& std::env::var("PUNKTFUNK_FORCE_SHM").as_deref() != Ok("1")
|
||||
&& policy.backend_is_vaapi;
|
||||
let vaapi_dmabuf = zerocopy && !force_shm && policy.backend_is_vaapi;
|
||||
let join = thread::Builder::new()
|
||||
.name("punktfunk-pipewire".into())
|
||||
.spawn(move || {
|
||||
@@ -235,8 +269,10 @@ fn spawn_pipewire(
|
||||
negotiated_cb,
|
||||
streaming_cb,
|
||||
broken_cb,
|
||||
hdr_negotiated_cb,
|
||||
zerocopy,
|
||||
want_444,
|
||||
want_hdr,
|
||||
preferred,
|
||||
quit_rx,
|
||||
policy,
|
||||
@@ -252,6 +288,8 @@ fn spawn_pipewire(
|
||||
streaming,
|
||||
broken,
|
||||
vaapi_dmabuf,
|
||||
hdr_offer: want_hdr,
|
||||
hdr_negotiated,
|
||||
quit: quit_tx,
|
||||
join,
|
||||
})
|
||||
@@ -354,6 +392,26 @@ impl Capturer for PortalCapturer {
|
||||
fn set_active(&self, active: bool) {
|
||||
self.active.store(active, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// Generic HDR10 mastering metadata once the stream negotiated a 10-bit PQ format. Mutter
|
||||
/// exposes no per-monitor mastering volume through the screencast, so this is the standard
|
||||
/// HDR10 default block (BT.2020 primaries, D65 white, 1000 / 0.005 cd/m², CLL unknown) — the
|
||||
/// same fallback Windows uses when a display reports nothing. The native stream loop prefers
|
||||
/// the client display's own volume when the client sent one (`Hello::display_hdr`).
|
||||
fn hdr_meta(&self) -> Option<punktfunk_core::quic::HdrMeta> {
|
||||
if !self.hdr_negotiated.load(Ordering::Relaxed) {
|
||||
return None;
|
||||
}
|
||||
Some(punktfunk_core::quic::HdrMeta {
|
||||
// ST.2086 order G, B, R; (x, y) chromaticity in 1/50000 units.
|
||||
display_primaries: [[8500, 39850], [6550, 2300], [35400, 14600]],
|
||||
white_point: [15635, 16450], // D65
|
||||
max_display_mastering_luminance: 10_000_000, // 1000 cd/m² (0.0001 units)
|
||||
min_display_mastering_luminance: 50, // 0.005 cd/m²
|
||||
max_cll: 0,
|
||||
max_fall: 0,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl PortalCapturer {
|
||||
@@ -372,6 +430,20 @@ impl PortalCapturer {
|
||||
or capture never started)",
|
||||
self.node_id
|
||||
))
|
||||
} else if self.hdr_offer {
|
||||
// The HDR (10-bit PQ dmabuf) offer was never accepted — the monitor left HDR
|
||||
// mode between the probe and the negotiation, the compositor pre-dates the
|
||||
// GNOME 50 HDR formats, or its allocator can't do LINEAR for XR30/XB30.
|
||||
// Latch the process-wide SDR downgrade so the next session (Moonlight
|
||||
// auto-reconnects) negotiates SDR instead of re-running this same timeout.
|
||||
super::note_hdr_capture_failed();
|
||||
Err(anyhow!(
|
||||
"no PipeWire frame within 10s (node {}): the compositor never accepted \
|
||||
the HDR (10-bit PQ/BT.2020 dmabuf) offer — is the mirrored monitor in \
|
||||
HDR mode on GNOME 50+? Downgrading this host to SDR capture; reconnect \
|
||||
to stream SDR",
|
||||
self.node_id
|
||||
))
|
||||
} else if self.vaapi_dmabuf && !pf_zerocopy::vaapi_dmabuf_forced() {
|
||||
// The LINEAR-dmabuf-only offer (VAAPI passthrough default) was never accepted.
|
||||
// Latch the process-wide downgrade so the encode loop's pipeline rebuild
|
||||
@@ -416,6 +488,85 @@ impl Drop for PortalCapturer {
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether any monitor of the live GNOME session is currently in BT.2100 (HDR) colour mode — the
|
||||
/// precondition for Mutter's monitor screencast advertising the 10-bit PQ formats (GNOME 50+;
|
||||
/// Mutter only appends the HDR formats while the mirrored monitor's colour state is BT.2020+PQ).
|
||||
/// Queried over the session bus: `DisplayConfig.GetCurrentState`, monitor property
|
||||
/// `"color-mode" == 1` (`META_COLOR_MODE_BT2100`). `false` on any error — not GNOME, a pre-48
|
||||
/// Mutter without colour modes, no monitors — so callers fall back to the honest SDR offer.
|
||||
/// Blocking (one D-Bus round-trip on a fresh connection); call from control-plane threads only.
|
||||
pub fn gnome_hdr_monitor_active() -> bool {
|
||||
use ashpd::zbus;
|
||||
// GetCurrentState reply: (serial, monitors, logical_monitors, properties); each monitor is
|
||||
// (spec(ssss), modes a(siiddada{sv}), properties a{sv}) — "color-mode" lives in the monitor
|
||||
// properties.
|
||||
type Mode = (
|
||||
String,
|
||||
i32,
|
||||
i32,
|
||||
f64,
|
||||
f64,
|
||||
Vec<f64>,
|
||||
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
|
||||
);
|
||||
type Monitor = (
|
||||
(String, String, String, String),
|
||||
Vec<Mode>,
|
||||
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
|
||||
);
|
||||
type LogicalMonitor = (
|
||||
i32,
|
||||
i32,
|
||||
f64,
|
||||
u32,
|
||||
bool,
|
||||
Vec<(String, String, String, String)>,
|
||||
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
|
||||
);
|
||||
type State = (
|
||||
u32,
|
||||
Vec<Monitor>,
|
||||
Vec<LogicalMonitor>,
|
||||
std::collections::HashMap<String, zbus::zvariant::OwnedValue>,
|
||||
);
|
||||
let probe = || -> Result<bool> {
|
||||
// zbus is built async-only here (ashpd's tokio integration) — run the one round-trip on
|
||||
// a throwaway current-thread runtime; this is a control-plane call, never per-frame.
|
||||
let rt = tokio::runtime::Builder::new_current_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
.context("build tokio runtime")?;
|
||||
rt.block_on(async {
|
||||
let conn = zbus::Connection::session().await.context("session bus")?;
|
||||
let reply = conn
|
||||
.call_method(
|
||||
Some("org.gnome.Mutter.DisplayConfig"),
|
||||
"/org/gnome/Mutter/DisplayConfig",
|
||||
Some("org.gnome.Mutter.DisplayConfig"),
|
||||
"GetCurrentState",
|
||||
&(),
|
||||
)
|
||||
.await
|
||||
.context("DisplayConfig.GetCurrentState")?;
|
||||
let (_serial, monitors, _logical, _props): State =
|
||||
reply.body().deserialize().context("parse GetCurrentState")?;
|
||||
Ok(monitors.iter().any(|(_spec, _modes, props)| {
|
||||
props
|
||||
.get("color-mode")
|
||||
.and_then(|v| u32::try_from(v).ok())
|
||||
.is_some_and(|mode| mode == 1) // META_COLOR_MODE_BT2100
|
||||
}))
|
||||
})
|
||||
};
|
||||
match probe() {
|
||||
Ok(hdr) => hdr,
|
||||
Err(e) => {
|
||||
tracing::debug!(error = %format!("{e:#}"), "GNOME HDR colour-mode probe failed — SDR");
|
||||
false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Pick the ScreenCast cursor mode from what the backend advertises (`AvailableCursorModes`),
|
||||
/// preferring **cursor-as-metadata**: the compositor keeps its cheap hardware cursor plane and
|
||||
/// ships the pointer as PipeWire `SPA_META_Cursor` metadata (position + an occasional bitmap),
|
||||
@@ -669,6 +820,10 @@ mod pipewire {
|
||||
VideoFormat::RGBA => PixelFormat::Rgba,
|
||||
VideoFormat::RGB => PixelFormat::Rgb,
|
||||
VideoFormat::BGR => PixelFormat::Bgr,
|
||||
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10; only ever negotiated by
|
||||
// the `want_hdr` offer, whose MANDATORY colorimetry props pin them to PQ/BT.2020).
|
||||
VideoFormat::xRGB_210LE => PixelFormat::X2Rgb10,
|
||||
VideoFormat::xBGR_210LE => PixelFormat::X2Bgr10,
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
@@ -732,6 +887,9 @@ mod pipewire {
|
||||
/// irrecoverably gone for this stream — the import worker died, or tiled imports failed
|
||||
/// [`IMPORT_FAIL_POISON`] times in a row.
|
||||
broken: Arc<AtomicBool>,
|
||||
/// Set when the negotiated format is one of the 10-bit PQ formats (`param_changed`) —
|
||||
/// read by [`PortalCapturer::hdr_meta`](super::PortalCapturer).
|
||||
hdr_negotiated: Arc<AtomicBool>,
|
||||
/// Consecutive tiled-import failures (reset on success); see [`IMPORT_FAIL_POISON`].
|
||||
import_fail_streak: u32,
|
||||
/// Present when zero-copy is enabled on NVIDIA: imports a dmabuf → CUDA device buffer,
|
||||
@@ -886,6 +1044,80 @@ mod pipewire {
|
||||
serialize_pod(obj)
|
||||
}
|
||||
|
||||
/// Build one GNOME 50+ HDR format pod: `format` (xRGB_210LE / xBGR_210LE) as a LINEAR-only
|
||||
/// dmabuf with **MANDATORY** BT.2020 primaries + SMPTE ST.2084 (PQ) transfer-function props —
|
||||
/// the exact colorimetry Mutter's monitor stream advertises while the mirrored monitor is in
|
||||
/// HDR mode (its HDR pods carry the same props MANDATORY, so both sides must speak them for
|
||||
/// the intersection to exist; an SDR or pre-50 producer can never match this pod).
|
||||
///
|
||||
/// LINEAR-only because every 10-bit consumer we have reads the buffer without a de-tile pass:
|
||||
/// the CPU path mmaps it, and the VAAPI passthrough imports it into a VA surface. The tiled
|
||||
/// EGL de-tile blit renders into an 8-bit `GL_RGBA8` texture — it would silently crush the
|
||||
/// depth — so tiled modifiers are deliberately NOT advertised (a zero-copy 10-bit de-tile is
|
||||
/// the follow-up). SHM is excluded entirely: Mutter's SHM record path paints 8-bit ARGB32
|
||||
/// regardless of the negotiated format.
|
||||
fn build_hdr_dmabuf_format(
|
||||
format: VideoFormat,
|
||||
preferred: Option<(u32, u32, u32)>,
|
||||
) -> Result<Vec<u8>> {
|
||||
let (dw, dh, dhz) = preferred.unwrap_or((1920, 1080, 60));
|
||||
use pw::spa::param::format::{FormatProperties, MediaSubtype, MediaType};
|
||||
let mut obj = pw::spa::pod::object!(
|
||||
pw::spa::utils::SpaTypes::ObjectParamFormat,
|
||||
pw::spa::param::ParamType::EnumFormat,
|
||||
pw::spa::pod::property!(FormatProperties::MediaType, Id, MediaType::Video),
|
||||
pw::spa::pod::property!(FormatProperties::MediaSubtype, Id, MediaSubtype::Raw),
|
||||
pw::spa::pod::property!(FormatProperties::VideoFormat, Id, format),
|
||||
pw::spa::pod::property!(
|
||||
FormatProperties::VideoSize,
|
||||
Choice,
|
||||
Range,
|
||||
Rectangle,
|
||||
pw::spa::utils::Rectangle {
|
||||
width: dw,
|
||||
height: dh
|
||||
},
|
||||
pw::spa::utils::Rectangle {
|
||||
width: 1,
|
||||
height: 1
|
||||
},
|
||||
pw::spa::utils::Rectangle {
|
||||
width: 8192,
|
||||
height: 8192
|
||||
}
|
||||
),
|
||||
pw::spa::pod::property!(
|
||||
FormatProperties::VideoFramerate,
|
||||
Choice,
|
||||
Range,
|
||||
Fraction,
|
||||
pw::spa::utils::Fraction { num: dhz, denom: 1 },
|
||||
pw::spa::utils::Fraction { num: 0, denom: 1 },
|
||||
pw::spa::utils::Fraction { num: 240, denom: 1 }
|
||||
),
|
||||
);
|
||||
obj.properties.push(pw::spa::pod::Property {
|
||||
key: pw::spa::sys::SPA_FORMAT_VIDEO_modifier,
|
||||
flags: pw::spa::pod::PropertyFlags::MANDATORY,
|
||||
value: pw::spa::pod::Value::Long(0), // DRM_FORMAT_MOD_LINEAR
|
||||
});
|
||||
obj.properties.push(pw::spa::pod::Property {
|
||||
key: pw::spa::sys::SPA_FORMAT_VIDEO_transferFunction,
|
||||
flags: pw::spa::pod::PropertyFlags::MANDATORY,
|
||||
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
|
||||
pw::spa::sys::SPA_VIDEO_TRANSFER_SMPTE2084,
|
||||
)),
|
||||
});
|
||||
obj.properties.push(pw::spa::pod::Property {
|
||||
key: pw::spa::sys::SPA_FORMAT_VIDEO_colorPrimaries,
|
||||
flags: pw::spa::pod::PropertyFlags::MANDATORY,
|
||||
value: pw::spa::pod::Value::Id(pw::spa::utils::Id(
|
||||
pw::spa::sys::SPA_VIDEO_COLOR_PRIMARIES_BT2020,
|
||||
)),
|
||||
});
|
||||
serialize_pod(obj)
|
||||
}
|
||||
|
||||
/// The default (shm/CPU-path) format offer: raw video in any encoder-mappable layout, any
|
||||
/// size, any framerate (0/1 = variable allowed — gamescope fixates exactly that).
|
||||
fn build_default_format_obj(preferred: Option<(u32, u32, u32)>) -> pw::spa::pod::Object {
|
||||
@@ -1157,6 +1389,54 @@ mod pipewire {
|
||||
})
|
||||
}
|
||||
|
||||
/// Alpha-blend the cached cursor bitmap into a packed 10-bit (`X2Rgb10`/`X2Bgr10`) CPU frame:
|
||||
/// unpack each u32, blend the 8-bit cursor channels scaled to 10 bits (`v<<2 | v>>6`), repack.
|
||||
/// The frame samples are PQ-encoded, so like the 8-bit gamma-space blend this is a display-
|
||||
/// referred approximation — fine for a cursor. `r_shift` is the R channel's bit offset (20 for
|
||||
/// x:R:G:B, 0 for x:B:G:R); G is always at 10 and B mirrors R.
|
||||
fn composite_cursor_rgb10(
|
||||
tight: &mut [u8],
|
||||
w: usize,
|
||||
h: usize,
|
||||
r_shift: u32,
|
||||
cursor: &CursorState,
|
||||
) {
|
||||
let b_shift = 20 - r_shift; // 0 or 20 — the opposite end from R
|
||||
let (bw, bh) = (cursor.bw as i32, cursor.bh as i32);
|
||||
for cy in 0..bh {
|
||||
let dy = cursor.y + cy;
|
||||
if dy < 0 || dy as usize >= h {
|
||||
continue;
|
||||
}
|
||||
for cx in 0..bw {
|
||||
let dx = cursor.x + cx;
|
||||
if dx < 0 || dx as usize >= w {
|
||||
continue;
|
||||
}
|
||||
let s = ((cy * bw + cx) as usize) * 4;
|
||||
let a = cursor.rgba[s + 3] as u32;
|
||||
if a == 0 {
|
||||
continue;
|
||||
}
|
||||
// 8-bit cursor channel → 10-bit (replicate the top bits into the bottom).
|
||||
let up10 = |v: u8| ((v as u32) << 2) | ((v as u32) >> 6);
|
||||
let (sr, sg, sb) = (
|
||||
up10(cursor.rgba[s]),
|
||||
up10(cursor.rgba[s + 1]),
|
||||
up10(cursor.rgba[s + 2]),
|
||||
);
|
||||
let di = (dy as usize * w + dx as usize) * 4;
|
||||
let px = u32::from_le_bytes(tight[di..di + 4].try_into().unwrap());
|
||||
let blend = |dst: u32, src: u32| (src * a + dst * (255 - a)) / 255;
|
||||
let dr = blend((px >> r_shift) & 0x3ff, sr);
|
||||
let dg = blend((px >> 10) & 0x3ff, sg);
|
||||
let db = blend((px >> b_shift) & 0x3ff, sb);
|
||||
let out = (px & 0xc000_0000) | (dr << r_shift) | (dg << 10) | (db << b_shift);
|
||||
tight[di..di + 4].copy_from_slice(&out.to_le_bytes());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Alpha-blend the cached cursor bitmap into the tightly-packed CPU frame at its latched
|
||||
/// position. Cheap: a straight-alpha blit over at most ~256×256 pixels, clipped to the frame —
|
||||
/// the whole point of cursor-as-metadata (no forced full-frame composite on the producer).
|
||||
@@ -1170,6 +1450,12 @@ mod pipewire {
|
||||
if !cursor.visible || cursor.rgba.is_empty() {
|
||||
return;
|
||||
}
|
||||
// The packed 10-bit HDR layouts blend via bit unpack/repack, not byte offsets.
|
||||
match fmt {
|
||||
PixelFormat::X2Rgb10 => return composite_cursor_rgb10(tight, w, h, 20, cursor),
|
||||
PixelFormat::X2Bgr10 => return composite_cursor_rgb10(tight, w, h, 0, cursor),
|
||||
_ => {}
|
||||
}
|
||||
let Some((ri, gi, bi, bpp)) = dst_offsets(fmt) else {
|
||||
return;
|
||||
};
|
||||
@@ -1344,7 +1630,10 @@ mod pipewire {
|
||||
// through to the shm de-pad copy below.
|
||||
let mut gpu_import_broken = false;
|
||||
if let (Some(importer), Some(fmt)) = (ud.importer.as_mut(), ud.format) {
|
||||
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf {
|
||||
// Defense-in-depth: the 10-bit PQ formats must never enter the EGL→CUDA import (its
|
||||
// de-tile blit is 8-bit RGBA8 — silent depth loss). An HDR offer never builds the
|
||||
// importer, so this gate only matters if those invariants ever drift apart.
|
||||
if datas[0].type_() == pw::spa::buffer::DataType::DmaBuf && !fmt.is_hdr_rgb10() {
|
||||
let plane = pf_zerocopy::DmabufPlane {
|
||||
fd: datas[0].fd(),
|
||||
offset: datas[0].chunk().offset(),
|
||||
@@ -1604,9 +1893,13 @@ mod pipewire {
|
||||
negotiated: Arc<AtomicBool>,
|
||||
streaming: Arc<AtomicBool>,
|
||||
broken: Arc<AtomicBool>,
|
||||
hdr_negotiated: Arc<AtomicBool>,
|
||||
zerocopy: bool,
|
||||
// 4:4:4 session: tiled dmabufs take the worker's planar-YUV444 GPU convert.
|
||||
want_444: bool,
|
||||
// HDR session: offer ONLY the 10-bit PQ/BT.2020 formats as LINEAR dmabufs (see
|
||||
// `build_hdr_dmabuf_format`); the SDR offers are not built at all.
|
||||
want_hdr: bool,
|
||||
preferred: Option<(u32, u32, u32)>,
|
||||
quit_rx: pw::channel::Receiver<()>,
|
||||
// Encode-backend facts resolved by the facade (never re-derived here) — the one-way
|
||||
@@ -1645,7 +1938,10 @@ mod pipewire {
|
||||
// succeed and produce CUDA payloads the VAAPI encoder must reject. Also skipped once
|
||||
// repeated worker deaths latched the import off (a wedged GPU stack must not crash-loop).
|
||||
let backend_is_vaapi = policy.backend_is_vaapi;
|
||||
let mut importer = if zerocopy && !backend_is_vaapi {
|
||||
// HDR never builds the EGL→CUDA importer: its de-tile blit renders into 8-bit RGBA8,
|
||||
// which would silently crush the 10-bit depth. The HDR consumers are the CPU mmap path
|
||||
// (LINEAR de-pad → X2Rgb10 CPU frames) and the VAAPI raw-dmabuf passthrough.
|
||||
let mut importer = if zerocopy && !backend_is_vaapi && !want_hdr {
|
||||
if pf_zerocopy::gpu_import_disabled() {
|
||||
tracing::warn!(
|
||||
"zero-copy GPU import disabled after repeated import-worker deaths — using CPU path"
|
||||
@@ -1755,6 +2051,7 @@ mod pipewire {
|
||||
negotiated,
|
||||
streaming,
|
||||
broken,
|
||||
hdr_negotiated,
|
||||
import_fail_streak: 0,
|
||||
importer,
|
||||
vaapi_passthrough,
|
||||
@@ -1822,12 +2119,20 @@ mod pipewire {
|
||||
let sz = ud.info.size();
|
||||
ud.format = map_format(ud.info.format());
|
||||
ud.modifier = ud.info.modifier();
|
||||
// HDR: the 10-bit PQ formats are only ever offered with MANDATORY BT.2020/PQ
|
||||
// colorimetry props, so a 10-bit negotiation IS an HDR negotiation — but log
|
||||
// what the producer actually fixated for diagnosis.
|
||||
let hdr = ud.format.is_some_and(|f| f.is_hdr_rgb10());
|
||||
ud.hdr_negotiated.store(hdr, Ordering::Relaxed);
|
||||
tracing::info!(
|
||||
width = sz.width,
|
||||
height = sz.height,
|
||||
spa_format = ?ud.info.format(),
|
||||
mapped = ?ud.format,
|
||||
modifier = ud.modifier,
|
||||
hdr,
|
||||
transfer_function = ud.info.transfer_function(),
|
||||
color_primaries = ud.info.color_primaries(),
|
||||
"pipewire format negotiated"
|
||||
);
|
||||
if ud.format.is_none() {
|
||||
@@ -2029,20 +2334,36 @@ mod pipewire {
|
||||
// (offering shm too makes the compositor pick shm). The modifier list is advertised with
|
||||
// DONT_FIXATE so the compositor's allocator chooses one; we re-emit the fixated format in
|
||||
// `param_changed` (the two-step DMA-BUF handshake). Otherwise offer the multi-format shm
|
||||
// pod and let MAP_BUFFERS map it.
|
||||
let shm_values = serialize_pod(obj)?;
|
||||
let (dmabuf_values, buffers_values) = if want_dmabuf {
|
||||
(
|
||||
Some(build_dmabuf_format(&modifiers, preferred)?),
|
||||
Some(build_dmabuf_buffers()?),
|
||||
)
|
||||
// pod and let MAP_BUFFERS map it. An HDR session replaces ALL of this with the two 10-bit
|
||||
// PQ pods (LINEAR dmabuf, MANDATORY colorimetry — see `build_hdr_dmabuf_format`): offering
|
||||
// SDR alongside would make the producer pick its earlier-listed SDR format, and the
|
||||
// negotiation-timeout path latches the process-wide SDR downgrade if nothing matches.
|
||||
let format_pods: Vec<Vec<u8>> = if want_hdr {
|
||||
tracing::info!(
|
||||
"HDR capture: offering xRGB_210LE/xBGR_210LE LINEAR dmabufs with MANDATORY \
|
||||
BT.2020 + SMPTE-2084 (PQ) colorimetry (GNOME 50+ monitor stream)"
|
||||
);
|
||||
vec![
|
||||
build_hdr_dmabuf_format(VideoFormat::xRGB_210LE, preferred)?,
|
||||
build_hdr_dmabuf_format(VideoFormat::xBGR_210LE, preferred)?,
|
||||
]
|
||||
} else if want_dmabuf {
|
||||
vec![build_dmabuf_format(&modifiers, preferred)?]
|
||||
} else {
|
||||
vec![serialize_pod(obj)?]
|
||||
};
|
||||
let buffers_values = if want_hdr || want_dmabuf {
|
||||
// Dmabuf-only. For HDR this is load-bearing beyond zero-copy: Mutter's SHM record
|
||||
// path paints 8-bit ARGB32 regardless of the negotiated format, so a MemFd buffer
|
||||
// under a 10-bit format would carry mislabeled bytes.
|
||||
Some(build_dmabuf_buffers()?)
|
||||
} else if force_shm {
|
||||
// True SHM: exclude DmaBuf so Mutter MUST download (glReadPixels orders against render).
|
||||
(None, Some(build_shm_only_buffers()?))
|
||||
Some(build_shm_only_buffers()?)
|
||||
} else {
|
||||
// CPU path still accepts mappable dmabufs (gamescope offers only those once its
|
||||
// modifier-bearing format pod wins the intersection).
|
||||
(None, Some(build_mappable_buffers()?))
|
||||
Some(build_mappable_buffers()?)
|
||||
};
|
||||
|
||||
// Ask for cursor-as-metadata on every path (harmless if the producer can't supply it): the
|
||||
@@ -2050,9 +2371,8 @@ mod pipewire {
|
||||
// compositor keeps its cheap hardware cursor plane (see `choose_cursor_mode`).
|
||||
let cursor_meta = build_cursor_meta_param()?;
|
||||
let mut byte_slices: Vec<&[u8]> = Vec::new();
|
||||
match &dmabuf_values {
|
||||
Some(d) => byte_slices.push(d),
|
||||
None => byte_slices.push(&shm_values),
|
||||
for pod in &format_pods {
|
||||
byte_slices.push(pod);
|
||||
}
|
||||
if let Some(b) = &buffers_values {
|
||||
byte_slices.push(b);
|
||||
|
||||
@@ -27,8 +27,8 @@ use super::libav::{
|
||||
use ffmpeg::ffi; // = ffmpeg_sys_next
|
||||
|
||||
/// The swscale *source* pixel format for a captured packed RGB/BGR layout (the real byte order, not
|
||||
/// the NVENC-padded `*0` form). Used by the 4:4:4 RGB→YUV444P conversion path. Mirrors the VAAPI
|
||||
/// CPU-input mapping; YUV/10-bit inputs can't feed this path (the 4:4:4 session forces packed RGB).
|
||||
/// the NVENC-padded `*0` form). Used by the CPU conversion paths: 4:4:4 RGB→YUV444P, and HDR
|
||||
/// X2RGB10/X2BGR10→P010. Mirrors the VAAPI CPU-input mapping; YUV inputs can't feed this path.
|
||||
fn sws_src_pixel(format: PixelFormat) -> Result<Pixel> {
|
||||
Ok(match format {
|
||||
PixelFormat::Bgrx => Pixel::BGRZ, // bgr0
|
||||
@@ -37,8 +37,12 @@ fn sws_src_pixel(format: PixelFormat) -> Result<Pixel> {
|
||||
PixelFormat::Rgba => Pixel::RGBA,
|
||||
PixelFormat::Rgb => Pixel::RGB24,
|
||||
PixelFormat::Bgr => Pixel::BGR24,
|
||||
// The GNOME 50+ HDR capture formats (PQ/BT.2020 packed 2:10:10:10) — the HDR CPU path's
|
||||
// swscale source for the X2RGB10→P010 conversion.
|
||||
PixelFormat::X2Rgb10 => Pixel::X2RGB10LE,
|
||||
PixelFormat::X2Bgr10 => Pixel::X2BGR10LE,
|
||||
PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Yuv444 => {
|
||||
bail!("NVENC 4:4:4 CPU-input path supports packed RGB/BGR only; got {format:?}")
|
||||
bail!("NVENC CPU-input conversion supports packed RGB/BGR only; got {format:?}")
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -136,6 +140,9 @@ fn nvenc_input(format: PixelFormat) -> (Pixel, bool) {
|
||||
// the Windows paths; the Linux capturer never emits them. Map to BGRA so the match is
|
||||
// exhaustive — unreachable here.
|
||||
PixelFormat::Rgb10a2 | PixelFormat::P010 => (Pixel::BGRA, false),
|
||||
// The Linux HDR capture formats never take the RGB-passthrough input: `open` intercepts
|
||||
// them onto the X2RGB10→P010 swscale path before consulting this mapping (like 4:4:4).
|
||||
PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10 => (Pixel::BGRA, false),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -164,11 +171,12 @@ pub struct NvencEncoder {
|
||||
frame: Option<VideoFrame>,
|
||||
/// Zero-copy path: CUDA hwdevice/hwframes contexts (the encoder takes `AV_PIX_FMT_CUDA`).
|
||||
cuda: Option<CudaHw>,
|
||||
/// 4:4:4 CPU path only: swscale context converting the captured packed RGB/BGR → planar
|
||||
/// YUV444P into [`Self::frame`], because `hevc_nvenc` only emits 4:4:4 from a YUV444 *input*
|
||||
/// (RGB-in is always 4:2:0). `None` on the 4:2:0 paths AND on the zero-copy 4:4:4 path (the
|
||||
/// worker's GPU convert delivers YUV444 CUDA frames). Freed in `Drop`.
|
||||
sws_444: Option<*mut ffi::SwsContext>,
|
||||
/// CPU CSC paths only: swscale context converting the captured packed source into
|
||||
/// [`Self::frame`] — RGB/BGR → planar YUV444P for a 4:4:4 session (`hevc_nvenc` only emits
|
||||
/// 4:4:4 from a YUV444 *input*; RGB-in is always 4:2:0), or X2RGB10/X2BGR10 → P010 (BT.2020
|
||||
/// limited) for an HDR session. `None` on the plain RGB paths AND on the zero-copy paths (the
|
||||
/// worker's GPU convert delivers ready CUDA frames). Freed in `Drop`.
|
||||
sws_csc: Option<*mut ffi::SwsContext>,
|
||||
/// This session opened as full-chroma 4:4:4 (FREXT) — via either input path.
|
||||
want_444: bool,
|
||||
src_format: PixelFormat,
|
||||
@@ -191,7 +199,7 @@ pub struct NvencEncoder {
|
||||
args: OpenArgs,
|
||||
}
|
||||
|
||||
// `CudaHw` holds raw `AVBufferRef`s and `sws_444` a raw `SwsContext`; the encoder lives on a single
|
||||
// `CudaHw` holds raw `AVBufferRef`s and `sws_csc` a raw `SwsContext`; the encoder lives on a single
|
||||
// thread. The CPU encoder is already `Send` via ffmpeg-next; assert it for the raw fields too.
|
||||
// SAFETY: `NvencEncoder` owns an ffmpeg-next `Encoder`/`VideoFrame` (already `Send`) plus a `CudaHw`
|
||||
// holding raw `AVBufferRef`s and an optional raw `SwsContext`, none of which are `Send` by default.
|
||||
@@ -247,14 +255,27 @@ impl NvencEncoder {
|
||||
bit_depth: u8,
|
||||
chroma: ChromaFormat,
|
||||
) -> Result<Self> {
|
||||
// TODO(hdr): Linux 10-bit parity. Unlike the Windows raw-SDK path (which upconverts 8-bit
|
||||
// ARGB → Main10 via pixelBitDepthMinus8), libavcodec hevc_nvenc needs a 10-bit input pixel
|
||||
// format (p010) for Main10, so it's a bigger change; deferred until a Linux GPU box is
|
||||
// available to validate. The Linux host stays 8-bit for now.
|
||||
if bit_depth != 8 {
|
||||
// HDR / 10-bit (GNOME 50+ HDR screencast): a 10-bit session whose capture negotiated a
|
||||
// packed 2:10:10:10 PQ/BT.2020 format (`X2Rgb10`/`X2Bgr10`) encodes HEVC Main10 / 10-bit
|
||||
// AV1 from a P010 input frame we produce by swscale (BT.2020 limited; the PQ transfer
|
||||
// rides through per-channel — BT.2020 NCL Y'CbCr *is* derived from the PQ-encoded R'G'B').
|
||||
// A 10-bit request whose capture stayed SDR (HDR offer downgraded) honestly encodes 8-bit.
|
||||
let want_hdr10 = bit_depth == 10 && format.is_hdr_rgb10() && codec.supports_10bit();
|
||||
if bit_depth == 10 && !want_hdr10 {
|
||||
tracing::warn!(
|
||||
bit_depth,
|
||||
"Linux NVENC 10-bit not yet wired — encoding 8-bit"
|
||||
?format,
|
||||
codec = codec.nvenc_name(),
|
||||
"10-bit requested but the capture format/codec has no 10-bit path — encoding 8-bit"
|
||||
);
|
||||
}
|
||||
if format.is_hdr_rgb10() && !want_hdr10 {
|
||||
// A 10-bit PQ capture on an 8-bit session would be encoded with a BT.709 VUI and
|
||||
// garbage bit-packing — never silently; the session must renegotiate.
|
||||
bail!(
|
||||
"captured 10-bit HDR frames ({format:?}) on an 8-bit/{} session — refusing to \
|
||||
mislabel PQ content",
|
||||
codec.nvenc_name()
|
||||
);
|
||||
}
|
||||
// Full-chroma 4:4:4 (HEVC Range Extensions). `hevc_nvenc` only emits 4:4:4 from a YUV444
|
||||
@@ -263,6 +284,11 @@ impl NvencEncoder {
|
||||
// (planar-YUV444 CUDA frames — `cuda` true), or the CPU path's swscale RGB→YUV444P. Both
|
||||
// feed `profile=rext`; the range follows `PUNKTFUNK_444_FULLRANGE` in both.
|
||||
let want_444 = chroma.is_444() && codec == Codec::H265;
|
||||
if want_444 && want_hdr10 {
|
||||
// The handshake resolves 4:4:4∧10-bit down to 8-bit on Linux, so this can't happen —
|
||||
// fail loudly if it ever does rather than picking one silently.
|
||||
bail!("4:4:4 + 10-bit HDR is not a supported Linux NVENC combination");
|
||||
}
|
||||
ffmpeg::init().context("ffmpeg init")?;
|
||||
if std::env::var_os("PUNKTFUNK_FFMPEG_DEBUG").is_some() {
|
||||
// SAFETY: `av_log_set_level` sets libav's global integer log level; `48` (= AV_LOG_DEBUG)
|
||||
@@ -274,10 +300,13 @@ impl NvencEncoder {
|
||||
let av_codec = encoder::find_by_name(name)
|
||||
.ok_or_else(|| anyhow!("{name} not built into libavcodec"))?;
|
||||
let (rgb_pixel, rgb_expand) = nvenc_input(format);
|
||||
// 4:4:4 feeds NVENC a planar YUV444P frame we produce by swscale; the ordinary path feeds the
|
||||
// captured RGB straight in and lets NVENC's internal CSC subsample to 4:2:0.
|
||||
// 4:4:4 feeds NVENC a planar YUV444P frame we produce by swscale; HDR feeds it a P010
|
||||
// frame likewise; the ordinary path feeds the captured RGB straight in and lets NVENC's
|
||||
// internal CSC subsample to 4:2:0.
|
||||
let (nvenc_pixel, expand) = if want_444 {
|
||||
(Pixel::YUV444P, false)
|
||||
} else if want_hdr10 {
|
||||
(Pixel::P010LE, false)
|
||||
} else {
|
||||
(rgb_pixel, rgb_expand)
|
||||
};
|
||||
@@ -325,7 +354,21 @@ impl NvencEncoder {
|
||||
// visible win. Linux-only: the Windows path's NVENC-internal CSC range is unmeasured.
|
||||
let full_range_444 =
|
||||
want_444 && std::env::var("PUNKTFUNK_444_FULLRANGE").is_ok_and(|v| v.trim() == "1");
|
||||
if matches!(format, PixelFormat::Nv12) || want_444 {
|
||||
if want_hdr10 {
|
||||
// HDR10: BT.2020 primaries + SMPTE-2084 (PQ) transfer, limited range — matches the
|
||||
// swscale BT.2020 CSC below and the Windows paths' signalling. The client decoder
|
||||
// auto-detects PQ from the VUI; static mastering metadata rides out-of-band.
|
||||
// SAFETY: `raw = video.as_mut_ptr()` is the non-null, properly-aligned, sole-owned,
|
||||
// not-yet-opened `AVCodecContext`; we set its four VUI colour enum fields to valid
|
||||
// variants before `open_with`. Sole owner → no aliasing; synchronous writes.
|
||||
unsafe {
|
||||
let raw = video.as_mut_ptr();
|
||||
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT2020_NCL;
|
||||
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
|
||||
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT2020;
|
||||
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_SMPTE2084;
|
||||
}
|
||||
} else if matches!(format, PixelFormat::Nv12) || want_444 {
|
||||
// SAFETY: same `video` builder — `raw = video.as_mut_ptr()` is the non-null, properly-
|
||||
// aligned, sole-owned, not-yet-opened `AVCodecContext`. We set its four VUI colour enum
|
||||
// fields to valid `AVColorSpace`/`AVColorRange`/`AVColorPrimaries`/`AVColorTransfer-
|
||||
@@ -370,17 +413,20 @@ impl NvencEncoder {
|
||||
None
|
||||
};
|
||||
|
||||
// 4:4:4 CPU path: build the RGB→YUV444P swscale (BT.709, range per the flag; no rescale).
|
||||
// Mirrors the VAAPI CPU path's RGB→NV12 scaler, but the dst is full-chroma planar 4:4:4.
|
||||
// Skipped on the zero-copy path (`cuda`): the worker's GPU convert already delivers
|
||||
// planar YUV444 CUDA frames — no CPU pixels exist to scale.
|
||||
let sws_444 = if want_444 && !cuda {
|
||||
// CPU CSC paths: build the packed-RGB → planar swscale (no rescale) into the encoder's
|
||||
// input frame. Two users: 4:4:4 (RGB→YUV444P, BT.709, range per the flag) and HDR
|
||||
// (X2RGB10/X2BGR10→P010, BT.2020 limited — the PQ transfer is per-channel and rides
|
||||
// through the matrix untouched). Skipped on the zero-copy path (`cuda`): the worker's GPU
|
||||
// convert already delivers ready CUDA frames — no CPU pixels exist to scale.
|
||||
let sws_csc = if (want_444 || want_hdr10) && !cuda {
|
||||
let src_av = pixel_to_av(sws_src_pixel(format)?);
|
||||
let dst_av = pixel_to_av(nvenc_pixel);
|
||||
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dims + pixel
|
||||
// formats. Both dims are the encoder's positive `width`/`height` as `c_int`; `src_av` is a
|
||||
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated, packed-RGB-only source), the
|
||||
// dst is YUV444P. The trailing filter/param pointers are null = "use defaults" (documented
|
||||
// as accepted). No Rust memory is borrowed; the returned pointer is null-checked below.
|
||||
// valid `AVPixelFormat` (from the `sws_src_pixel`-validated packed-RGB source), the dst is
|
||||
// YUV444P (4:4:4) or P010LE (HDR). The trailing filter/param pointers are null = "use
|
||||
// defaults" (documented as accepted). No Rust memory is borrowed; the returned pointer is
|
||||
// null-checked below.
|
||||
let sws = unsafe {
|
||||
ffi::sws_getContext(
|
||||
width as c_int,
|
||||
@@ -388,7 +434,7 @@ impl NvencEncoder {
|
||||
src_av,
|
||||
width as c_int,
|
||||
height as c_int,
|
||||
ffi::AVPixelFormat::AV_PIX_FMT_YUV444P,
|
||||
dst_av,
|
||||
SWS_POINT,
|
||||
ptr::null_mut(),
|
||||
ptr::null_mut(),
|
||||
@@ -396,17 +442,22 @@ impl NvencEncoder {
|
||||
)
|
||||
};
|
||||
if sws.is_null() {
|
||||
bail!("sws_getContext(RGB→YUV444P) failed");
|
||||
bail!("sws_getContext(RGB→{nvenc_pixel:?}) failed");
|
||||
}
|
||||
// SAFETY: `sws` is the non-null context from the call above (null-checked). The ITU-709
|
||||
// coefficient table from `sws_getCoefficients` is a process-lifetime libswscale static,
|
||||
// reused for src+dst matrices; `sws_setColorspaceDetails` only reads it and writes scalar
|
||||
// CSC settings into `sws` (dstRange matches the VUI: 0 = limited, 1 = the
|
||||
// PUNKTFUNK_444_FULLRANGE experiment). No Rust memory is passed.
|
||||
// SAFETY: `sws` is the non-null context from the call above (null-checked). The
|
||||
// coefficient tables from `sws_getCoefficients` (ITU-709 for 4:4:4, BT.2020 NCL for HDR
|
||||
// — matching the VUI written above) are process-lifetime libswscale statics, reused for
|
||||
// src+dst matrices; `sws_setColorspaceDetails` only reads them and writes scalar CSC
|
||||
// settings into `sws` (dstRange matches the VUI: 0 = limited, 1 = the
|
||||
// PUNKTFUNK_444_FULLRANGE experiment; HDR is always limited). No Rust memory is passed.
|
||||
unsafe {
|
||||
let cs709 = ffi::sws_getCoefficients(SWS_CS_ITU709);
|
||||
let cs = ffi::sws_getCoefficients(if want_hdr10 {
|
||||
super::libav::SWS_CS_BT2020
|
||||
} else {
|
||||
SWS_CS_ITU709
|
||||
});
|
||||
let dst_range = i32::from(full_range_444);
|
||||
ffi::sws_setColorspaceDetails(sws, cs709, 1, cs709, dst_range, 0, 1 << 16, 1 << 16);
|
||||
ffi::sws_setColorspaceDetails(sws, cs, 1, cs, dst_range, 0, 1 << 16, 1 << 16);
|
||||
}
|
||||
Some(sws)
|
||||
} else {
|
||||
@@ -432,6 +483,12 @@ impl NvencEncoder {
|
||||
// dropped on a future libavcodec.
|
||||
opts.set("profile", "rext");
|
||||
}
|
||||
if want_hdr10 && codec == Codec::H265 {
|
||||
// HEVC Main10. `hevc_nvenc` auto-selects it from the P010 input, but pin it explicitly
|
||||
// so the depth is never silently dropped on a future libavcodec. (10-bit AV1 needs no
|
||||
// profile — AV1 Main carries 10-bit, driven by the input format.)
|
||||
opts.set("profile", "main10");
|
||||
}
|
||||
|
||||
// Split-frame encode across both NVENC engines (GB203 has 2) when the pixel rate exceeds
|
||||
// a single engine's HEVC capacity (~1 Gpix/s); e.g. 5120x1440@240 = 1.77 Gpix/s needs it,
|
||||
@@ -501,7 +558,7 @@ impl NvencEncoder {
|
||||
enc,
|
||||
frame,
|
||||
cuda: cuda_hw,
|
||||
sws_444,
|
||||
sws_csc,
|
||||
want_444,
|
||||
src_format: format,
|
||||
expand,
|
||||
@@ -640,7 +697,7 @@ impl NvencEncoder {
|
||||
);
|
||||
// 4:4:4: swscale the packed RGB straight into the planar YUV444P input frame (BT.709 limited),
|
||||
// then send it — no byte-expand. The 4:2:0 RGB path (below) feeds NVENC packed RGB directly.
|
||||
if let Some(sws) = self.sws_444 {
|
||||
if let Some(sws) = self.sws_csc {
|
||||
let frame = self
|
||||
.frame
|
||||
.as_mut()
|
||||
@@ -810,7 +867,7 @@ impl NvencEncoder {
|
||||
|
||||
impl Drop for NvencEncoder {
|
||||
fn drop(&mut self) {
|
||||
if let Some(sws) = self.sws_444.take() {
|
||||
if let Some(sws) = self.sws_csc.take() {
|
||||
// SAFETY: `sws` is the non-null `SwsContext` allocated by `sws_getContext` in `open` and
|
||||
// owned exclusively by this encoder (taken out of the field so it can't be freed twice).
|
||||
// `sws_freeContext` frees it; nothing else references it after this single-threaded drop.
|
||||
@@ -855,3 +912,105 @@ pub fn probe_can_encode_444(codec: Codec) -> bool {
|
||||
unsafe { ffi::av_log_set_level(prev) };
|
||||
ok
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod hdr_tests {
|
||||
use super::*;
|
||||
|
||||
/// The Linux HDR (GNOME 50 portal) encode path end-to-end on a real NVIDIA GPU: a synthetic
|
||||
/// PQ-ish X2RGB10 CPU frame → swscale BT.2020 → P010 → `hevc_nvenc` Main10, drained to a real
|
||||
/// AU. `#[ignore]`d (needs NVENC):
|
||||
/// `cargo test -p pf-encode nvenc_hdr10_smoke -- --ignored --nocapture`
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn nvenc_hdr10_smoke() {
|
||||
let (w, h) = (640u32, 480u32);
|
||||
let mut enc = NvencEncoder::open(
|
||||
Codec::H265,
|
||||
PixelFormat::X2Rgb10,
|
||||
w,
|
||||
h,
|
||||
30,
|
||||
2_000_000,
|
||||
false,
|
||||
10,
|
||||
ChromaFormat::Yuv420,
|
||||
)
|
||||
.expect("open hevc_nvenc Main10 (P010 input)");
|
||||
// Packed x:R:G:B 2:10:10:10 gradient (values are treated as PQ-encoded — fine for a smoke).
|
||||
let mut bytes = vec![0u8; (w * h * 4) as usize];
|
||||
for y in 0..h {
|
||||
for x in 0..w {
|
||||
let r = (x * 1023 / w.max(1)) & 0x3ff;
|
||||
let g = (y * 1023 / h.max(1)) & 0x3ff;
|
||||
let b = ((x + y) * 1023 / (w + h)) & 0x3ff;
|
||||
let px: u32 = (r << 20) | (g << 10) | b;
|
||||
let i = ((y * w + x) * 4) as usize;
|
||||
bytes[i..i + 4].copy_from_slice(&px.to_le_bytes());
|
||||
}
|
||||
}
|
||||
let frame = CapturedFrame {
|
||||
width: w,
|
||||
height: h,
|
||||
pts_ns: 0,
|
||||
format: PixelFormat::X2Rgb10,
|
||||
payload: FramePayload::Cpu(bytes),
|
||||
cursor: None,
|
||||
};
|
||||
let mut au = None;
|
||||
for _ in 0..30 {
|
||||
enc.submit(&frame).expect("submit X2Rgb10 frame");
|
||||
if let Some(a) = enc.poll().expect("poll") {
|
||||
au = Some(a);
|
||||
break;
|
||||
}
|
||||
}
|
||||
let au = au.expect("no AU produced within 30 frames");
|
||||
assert!(!au.data.is_empty(), "empty AU");
|
||||
assert!(au.keyframe, "first AU should be the IDR");
|
||||
println!("HDR10 smoke: first AU {} bytes (IDR)", au.data.len());
|
||||
// PF_HDR_SMOKE_DUMP=/path.h265: write the Annex-B AU for external inspection —
|
||||
// `ffprobe -show_streams` should report Main 10, bt2020nc/smpte2084/bt2020 colours.
|
||||
if let Ok(path) = std::env::var("PF_HDR_SMOKE_DUMP") {
|
||||
std::fs::write(&path, &au.data).expect("dump AU");
|
||||
println!("HDR10 smoke: AU written to {path}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Probe whether this NVIDIA GPU + driver + libavcodec can actually encode 10-bit (HEVC Main10 /
|
||||
/// 10-bit AV1) from a P010 input — the exact path [`NvencEncoder::open`] takes for a live HDR
|
||||
/// stream (a tiny X2RGB10-sourced, P010-input open). The result is cached by the caller
|
||||
/// ([`crate::can_encode_10bit`]); a GPU/driver/ffmpeg without the 10-bit encode fails the open
|
||||
/// here, so the host resolves the session to 8-bit SDR before the Welcome (honest downgrade).
|
||||
pub fn probe_can_encode_10bit(codec: Codec) -> bool {
|
||||
if !codec.supports_10bit() {
|
||||
return false;
|
||||
}
|
||||
if ffmpeg::init().is_err() {
|
||||
return false;
|
||||
}
|
||||
// Quiet ffmpeg's open error on a GPU that lacks 10-bit — the probe failing is an expected outcome.
|
||||
// SAFETY: libav initialized above; `av_log_{get,set}_level` only read/write the global int level
|
||||
// (no pointer args) and are always sound post-init.
|
||||
let prev = unsafe {
|
||||
let p = ffi::av_log_get_level();
|
||||
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
|
||||
p
|
||||
};
|
||||
let ok = NvencEncoder::open(
|
||||
codec,
|
||||
PixelFormat::X2Rgb10,
|
||||
640,
|
||||
480,
|
||||
30,
|
||||
2_000_000,
|
||||
false, // CPU input (the HDR swscale path)
|
||||
10,
|
||||
ChromaFormat::Yuv420,
|
||||
)
|
||||
.is_ok();
|
||||
// SAFETY: restore the saved global log level (scalar arg, no pointers).
|
||||
unsafe { ffi::av_log_set_level(prev) };
|
||||
ok
|
||||
}
|
||||
|
||||
@@ -474,9 +474,13 @@ impl NvencCudaEncoder {
|
||||
// clear reason instead of an opaque session error on the first frame.
|
||||
try_api().map_err(|e| anyhow!("NVENC (Linux direct) unavailable: {e}"))?;
|
||||
if bit_depth >= 10 {
|
||||
// An HDR (GNOME 50 portal) session never reaches this backend: its X2RGB10 frames ride
|
||||
// the CPU/dmabuf paths (no CUDA import for the 10-bit formats yet), so the dispatcher
|
||||
// opens the libav P010 path instead. Reaching here 10-bit means a CUDA capture payload
|
||||
// on a 10-bit session — not wired; encode 8-bit rather than mislabel.
|
||||
tracing::warn!(
|
||||
"Linux direct-NVENC: 10-bit requested but no P010 capture path exists yet \
|
||||
(Phase 5.1) — encoding 8-bit SDR"
|
||||
"Linux direct-NVENC: 10-bit requested but the CUDA capture path has no 10-bit \
|
||||
import yet (HDR rides the libav P010 path) — encoding 8-bit SDR"
|
||||
);
|
||||
}
|
||||
Ok(Self {
|
||||
|
||||
@@ -61,6 +61,10 @@ fn vaapi_sws_src(format: PixelFormat) -> Result<Pixel> {
|
||||
PixelFormat::Rgba => Pixel::RGBA,
|
||||
PixelFormat::Rgb => Pixel::RGB24,
|
||||
PixelFormat::Bgr => Pixel::BGR24,
|
||||
// The GNOME 50+ HDR capture formats (PQ/BT.2020 packed 2:10:10:10) — the HDR CPU path's
|
||||
// swscale source for the X2RGB10→P010 conversion.
|
||||
PixelFormat::X2Rgb10 => Pixel::X2RGB10LE,
|
||||
PixelFormat::X2Bgr10 => Pixel::X2BGR10LE,
|
||||
PixelFormat::Nv12 | PixelFormat::P010 | PixelFormat::Rgb10a2 | PixelFormat::Yuv444 => {
|
||||
bail!("VAAPI CPU-input path supports packed RGB/BGR only; got {format:?}")
|
||||
}
|
||||
@@ -101,6 +105,7 @@ fn low_power_override() -> Option<bool> {
|
||||
/// default on those kernels). AMD keeps its first-try full-feature open byte-for-byte unchanged.
|
||||
/// The resolved mode is cached per codec; `PUNKTFUNK_VAAPI_LOW_POWER` pins it.
|
||||
/// Safety contract is [`open_vaapi_encoder_mode`]'s (borrowed `device_ref`/`frames_ref`).
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
unsafe fn open_vaapi_encoder(
|
||||
codec: Codec,
|
||||
width: u32,
|
||||
@@ -109,6 +114,7 @@ unsafe fn open_vaapi_encoder(
|
||||
bitrate_bps: u64,
|
||||
device_ref: *mut ffi::AVBufferRef,
|
||||
frames_ref: *mut ffi::AVBufferRef,
|
||||
ten_bit: bool,
|
||||
) -> Result<encoder::video::Encoder> {
|
||||
let idx = lp_idx(codec);
|
||||
let modes: &[bool] = match low_power_override() {
|
||||
@@ -130,6 +136,7 @@ unsafe fn open_vaapi_encoder(
|
||||
bitrate_bps,
|
||||
device_ref,
|
||||
frames_ref,
|
||||
ten_bit,
|
||||
lp,
|
||||
) {
|
||||
Ok(enc) => {
|
||||
@@ -158,8 +165,9 @@ unsafe fn open_vaapi_encoder(
|
||||
}
|
||||
|
||||
/// Build the FFmpeg encoder context (shared by both inner paths): name, mode, low-latency RC,
|
||||
/// infinite GOP, BT.709-limited VUI, `pix_fmt=VAAPI`, and the given hw device + frames contexts.
|
||||
/// Returns the opened encoder. `device_ref`/`frames_ref` are borrowed (ref'd into the context).
|
||||
/// infinite GOP, the VUI (BT.709 limited SDR, or BT.2020 PQ limited for `ten_bit` HDR),
|
||||
/// `pix_fmt=VAAPI`, and the given hw device + frames contexts. Returns the opened encoder.
|
||||
/// `device_ref`/`frames_ref` are borrowed (ref'd into the context).
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
unsafe fn open_vaapi_encoder_mode(
|
||||
codec: Codec,
|
||||
@@ -169,6 +177,7 @@ unsafe fn open_vaapi_encoder_mode(
|
||||
bitrate_bps: u64,
|
||||
device_ref: *mut ffi::AVBufferRef,
|
||||
frames_ref: *mut ffi::AVBufferRef,
|
||||
ten_bit: bool,
|
||||
low_power: bool,
|
||||
) -> Result<encoder::video::Encoder> {
|
||||
let name = codec.vaapi_name();
|
||||
@@ -181,23 +190,39 @@ unsafe fn open_vaapi_encoder_mode(
|
||||
.context("alloc video encoder")?;
|
||||
video.set_width(width);
|
||||
video.set_height(height);
|
||||
video.set_format(Pixel::NV12); // sw view; pix_fmt overridden to VAAPI below
|
||||
// Fixed rate, CBR, no B-frames, ~1-frame VBV — the shared low-latency RC contract.
|
||||
// sw view (pix_fmt overridden to VAAPI below): NV12, or P010 for the 10-bit HDR session.
|
||||
video.set_format(if ten_bit { Pixel::P010LE } else { Pixel::NV12 });
|
||||
// Fixed rate, CBR, no B-frames, ~1-frame VBV — the shared low-latency RC contract.
|
||||
apply_low_latency_rc(&mut video, fps, bitrate_bps);
|
||||
let raw = video.as_mut_ptr();
|
||||
(*raw).gop_size = i32::MAX; // no periodic IDR (forced-IDR via pict_type=I on RFI)
|
||||
// We hand the encoder BT.709 *limited* NV12 (swscale CSC on the CPU path; scale_vaapi pinned
|
||||
// to `out_color_matrix=bt709:out_range=limited` on the zero-copy path, with the full-range
|
||||
// RGB input tagged), so signal that VUI — else the client decoder washes the picture out.
|
||||
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT709;
|
||||
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
|
||||
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT709;
|
||||
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_BT709;
|
||||
if ten_bit {
|
||||
// HDR10: BT.2020 primaries + SMPTE-2084 (PQ) transfer, limited range — matches the P010
|
||||
// the CSC produces (swscale BT.2020 on the CPU path; scale_vaapi pinned to bt2020 on the
|
||||
// zero-copy path). The client decoder auto-detects PQ from the VUI.
|
||||
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT2020_NCL;
|
||||
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
|
||||
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT2020;
|
||||
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_SMPTE2084;
|
||||
} else {
|
||||
// We hand the encoder BT.709 *limited* NV12 (swscale CSC on the CPU path; scale_vaapi pinned
|
||||
// to `out_color_matrix=bt709:out_range=limited` on the zero-copy path, with the full-range
|
||||
// RGB input tagged), so signal that VUI — else the client decoder washes the picture out.
|
||||
(*raw).colorspace = ffi::AVColorSpace::AVCOL_SPC_BT709;
|
||||
(*raw).color_range = ffi::AVColorRange::AVCOL_RANGE_MPEG;
|
||||
(*raw).color_primaries = ffi::AVColorPrimaries::AVCOL_PRI_BT709;
|
||||
(*raw).color_trc = ffi::AVColorTransferCharacteristic::AVCOL_TRC_BT709;
|
||||
}
|
||||
(*raw).pix_fmt = ffi::AVPixelFormat::AV_PIX_FMT_VAAPI;
|
||||
(*raw).hw_device_ctx = ffi::av_buffer_ref(device_ref);
|
||||
(*raw).hw_frames_ctx = ffi::av_buffer_ref(frames_ref);
|
||||
|
||||
let mut opts = Dictionary::new();
|
||||
if ten_bit && codec == Codec::H265 {
|
||||
// HEVC Main10. `hevc_vaapi` derives it from the P010 surfaces, but pin it explicitly so
|
||||
// the depth is never silently dropped. (10-bit AV1 is input-driven — no profile knob.)
|
||||
opts.set("profile", "main10");
|
||||
}
|
||||
// async_depth=1: `send_frame` blocks until THIS frame's ASIC encode completes — the lowest
|
||||
// latency structure libavcodec's vaapi_encode offers. Measured on the 780M at 1440p60: depth 1
|
||||
// = 8.3 ms end-to-end p50 vs depth 2 = 18 ms, because with depth ≥ 2 frame N's packet only
|
||||
@@ -242,10 +267,59 @@ pub fn probe_can_encode(codec: Codec) -> bool {
|
||||
let prev = ffi::av_log_get_level();
|
||||
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
|
||||
let ok = match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, 640, 480, 2) {
|
||||
Ok(hw) => {
|
||||
open_vaapi_encoder(codec, 640, 480, 30, 2_000_000, hw.device_ref, hw.frames_ref)
|
||||
.is_ok()
|
||||
}
|
||||
Ok(hw) => open_vaapi_encoder(
|
||||
codec,
|
||||
640,
|
||||
480,
|
||||
30,
|
||||
2_000_000,
|
||||
hw.device_ref,
|
||||
hw.frames_ref,
|
||||
false,
|
||||
)
|
||||
.is_ok(),
|
||||
Err(_) => false,
|
||||
};
|
||||
ffi::av_log_set_level(prev);
|
||||
ok
|
||||
}
|
||||
}
|
||||
|
||||
/// Probe whether the active VAAPI GPU can encode **10-bit** (HEVC Main10 / 10-bit AV1) from P010
|
||||
/// surfaces — the exact shape a live HDR session opens (P010 pool + Main10 profile + PQ VUI). The
|
||||
/// driver rejects what the video engine can't do; the result is cached by the caller
|
||||
/// ([`crate::can_encode_10bit`]), so a non-Main10 GPU resolves every session to 8-bit SDR before
|
||||
/// the Welcome (honest downgrade).
|
||||
pub fn probe_can_encode_10bit(codec: Codec) -> bool {
|
||||
if !codec.supports_10bit() || codec == Codec::PyroWave {
|
||||
return false;
|
||||
}
|
||||
if ffmpeg::init().is_err() {
|
||||
return false;
|
||||
}
|
||||
// SAFETY: `ffmpeg::init()` returned Ok above, so libav is initialized. `av_log_{get,set}_level`
|
||||
// only read/write libav's global integer log level (no pointer args). `VaapiHw::new` (an
|
||||
// `unsafe fn`) builds a VAAPI device + P010 frames pool from the literal args and hands back a
|
||||
// RAII handle; `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` —
|
||||
// the two non-null refs `VaapiHw::new` just created, live locals for the whole match arm — and
|
||||
// `av_buffer_ref`s them into the probe encoder. Both `hw` and the encoder drop (RAII) at arm end.
|
||||
unsafe {
|
||||
// A missing VA device / no Main10 entrypoint is an expected probe outcome — quiet ffmpeg's
|
||||
// error for the probe, then restore the level.
|
||||
let prev = ffi::av_log_get_level();
|
||||
ffi::av_log_set_level(ffi::AV_LOG_FATAL);
|
||||
let ok = match VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_P010LE, 640, 480, 2) {
|
||||
Ok(hw) => open_vaapi_encoder(
|
||||
codec,
|
||||
640,
|
||||
480,
|
||||
30,
|
||||
2_000_000,
|
||||
hw.device_ref,
|
||||
hw.frames_ref,
|
||||
true,
|
||||
)
|
||||
.is_ok(),
|
||||
Err(_) => false,
|
||||
};
|
||||
ffi::av_log_set_level(prev);
|
||||
@@ -348,12 +422,21 @@ impl CpuInner {
|
||||
bitrate_bps: u64,
|
||||
) -> Result<Self> {
|
||||
let src_pixel = vaapi_sws_src(format)?;
|
||||
// A 10-bit HDR capture (X2RGB10/X2BGR10, PQ/BT.2020) uploads P010 and encodes Main10; the
|
||||
// 8-bit paths keep NV12/BT.709 byte-for-byte unchanged.
|
||||
let ten_bit = format.is_hdr_rgb10();
|
||||
let staging_av = if ten_bit {
|
||||
ffi::AVPixelFormat::AV_PIX_FMT_P010LE
|
||||
} else {
|
||||
ffi::AVPixelFormat::AV_PIX_FMT_NV12
|
||||
};
|
||||
const POOL: c_int = 16;
|
||||
// SAFETY: `VaapiHw::new` (an `unsafe fn`) requires libav initialized — guaranteed because the
|
||||
// only path here is `VaapiEncoder::open` → `ensure_inner` → `CpuInner::open`, and `open` ran
|
||||
// `ffmpeg::init()`. The args are valid: NV12 sw_format, the validated positive `width`/`height`,
|
||||
// pool=16. It returns a RAII `VaapiHw` that unrefs its two `AVBufferRef`s on drop.
|
||||
let hw = unsafe { VaapiHw::new(ffi::AVPixelFormat::AV_PIX_FMT_NV12, width, height, POOL)? };
|
||||
// `ffmpeg::init()`. The args are valid: an NV12/P010 sw_format, the validated positive
|
||||
// `width`/`height`, pool=16. It returns a RAII `VaapiHw` that unrefs its two `AVBufferRef`s
|
||||
// on drop.
|
||||
let hw = unsafe { VaapiHw::new(staging_av, width, height, POOL)? };
|
||||
// SAFETY: `open_vaapi_encoder` (an `unsafe fn`) borrows `hw.device_ref`/`hw.frames_ref` — both
|
||||
// non-null (`VaapiHw::new` guarantees it) and from the `hw` just built above, which is a live
|
||||
// local that outlives this synchronous call. The fn `av_buffer_ref`s them into the encoder, so
|
||||
@@ -368,16 +451,19 @@ impl CpuInner {
|
||||
bitrate_bps,
|
||||
hw.device_ref,
|
||||
hw.frames_ref,
|
||||
ten_bit,
|
||||
)?
|
||||
};
|
||||
// swscale RGB→NV12, BT.709 limited (matches the VUI), no rescale.
|
||||
// swscale RGB→NV12 (BT.709 limited) or X2RGB10→P010 (BT.2020 limited, HDR) — matches the
|
||||
// VUI; no rescale.
|
||||
let src_av = pixel_to_av(src_pixel);
|
||||
// SAFETY: `sws_getContext` allocates a swscale context for the given src/dst dimensions and
|
||||
// pixel formats. All four dims are the encoder's positive `width`/`height` cast to `c_int`;
|
||||
// `src_av` is a valid `AVPixelFormat` (from `pixel_to_av` of the `vaapi_sws_src`-validated
|
||||
// `src_pixel`), the dst is NV12. The three trailing pointers (srcFilter, dstFilter, param) are
|
||||
// explicitly null = "use defaults", which the API documents as accepted. No Rust memory is
|
||||
// borrowed — only by-value ints/enums — and the returned pointer is null-checked just below.
|
||||
// `src_pixel`), the dst is NV12/P010. The three trailing pointers (srcFilter, dstFilter,
|
||||
// param) are explicitly null = "use defaults", which the API documents as accepted. No Rust
|
||||
// memory is borrowed — only by-value ints/enums — and the returned pointer is null-checked
|
||||
// just below.
|
||||
let sws = unsafe {
|
||||
ffi::sws_getContext(
|
||||
width as c_int,
|
||||
@@ -385,7 +471,7 @@ impl CpuInner {
|
||||
src_av,
|
||||
width as c_int,
|
||||
height as c_int,
|
||||
ffi::AVPixelFormat::AV_PIX_FMT_NV12,
|
||||
staging_av,
|
||||
SWS_POINT,
|
||||
ptr::null_mut(),
|
||||
ptr::null_mut(),
|
||||
@@ -393,45 +479,51 @@ impl CpuInner {
|
||||
)
|
||||
};
|
||||
if sws.is_null() {
|
||||
bail!("sws_getContext(RGB→NV12) failed");
|
||||
bail!("sws_getContext(RGB→{})", if ten_bit { "P010" } else { "NV12" });
|
||||
}
|
||||
// SAFETY: `sws` is the non-null `SwsContext` from `sws_getContext` above (the `is_null()`
|
||||
// check immediately preceding returned false). `sws_getCoefficients(SWS_CS_ITU709)` returns a
|
||||
// pointer into a libswscale static const coefficient table valid for the whole process, reused
|
||||
// here for both the inverse (src) and forward (dst) matrices. `sws_setColorspaceDetails` only
|
||||
// reads those tables and writes scalar CSC settings into `sws`; the table pointer outlives the
|
||||
// synchronous call and no Rust memory is passed.
|
||||
// check immediately preceding returned false). The coefficient table from
|
||||
// `sws_getCoefficients` (ITU-709, or BT.2020 NCL for the HDR path — matching the VUI) is a
|
||||
// libswscale static const valid for the whole process, reused here for both the inverse
|
||||
// (src) and forward (dst) matrices. `sws_setColorspaceDetails` only reads those tables and
|
||||
// writes scalar CSC settings into `sws`; the table pointer outlives the synchronous call and
|
||||
// no Rust memory is passed.
|
||||
unsafe {
|
||||
let cs709 = ffi::sws_getCoefficients(SWS_CS_ITU709);
|
||||
ffi::sws_setColorspaceDetails(sws, cs709, 1, cs709, 0, 0, 1 << 16, 1 << 16);
|
||||
let cs = ffi::sws_getCoefficients(if ten_bit {
|
||||
super::libav::SWS_CS_BT2020
|
||||
} else {
|
||||
SWS_CS_ITU709
|
||||
});
|
||||
ffi::sws_setColorspaceDetails(sws, cs, 1, cs, 0, 0, 1 << 16, 1 << 16);
|
||||
}
|
||||
// SAFETY: `av_frame_alloc` returns a fresh, uniquely-owned heap `AVFrame` (null-checked — on
|
||||
// null we free the already-built `sws` and bail). We then write the plain `format`/`width`/
|
||||
// `height` fields through the non-null, properly-aligned `f` (sole owner, not yet shared).
|
||||
// `av_frame_get_buffer(f, 0)` allocates backing storage for those dims/format; on failure we
|
||||
// free `f` and `sws` (unwinding the half-built state) and bail. On success `f` is a fully-owned
|
||||
// NV12 frame stored in `CpuInner.nv12` and freed once in `CpuInner::drop`. `f` is a unique
|
||||
// fresh pointer, so none of these writes alias anything.
|
||||
// NV12/P010 frame stored in `CpuInner.nv12` and freed once in `CpuInner::drop`. `f` is a
|
||||
// unique fresh pointer, so none of these writes alias anything.
|
||||
let nv12 = unsafe {
|
||||
let f = ffi::av_frame_alloc();
|
||||
if f.is_null() {
|
||||
ffi::sws_freeContext(sws);
|
||||
bail!("av_frame_alloc(NV12) failed");
|
||||
bail!("av_frame_alloc(staging) failed");
|
||||
}
|
||||
(*f).format = ffi::AVPixelFormat::AV_PIX_FMT_NV12 as c_int;
|
||||
(*f).format = staging_av as c_int;
|
||||
(*f).width = width as c_int;
|
||||
(*f).height = height as c_int;
|
||||
if ffi::av_frame_get_buffer(f, 0) < 0 {
|
||||
let mut f = f;
|
||||
ffi::av_frame_free(&mut f);
|
||||
ffi::sws_freeContext(sws);
|
||||
bail!("av_frame_get_buffer(NV12) failed");
|
||||
bail!("av_frame_get_buffer(staging) failed");
|
||||
}
|
||||
f
|
||||
};
|
||||
tracing::info!(
|
||||
encoder = codec.vaapi_name(),
|
||||
"VAAPI encode active ({width}x{height}@{fps}, CPU→NV12 upload path)"
|
||||
"VAAPI encode active ({width}x{height}@{fps}, CPU→{} upload path)",
|
||||
if ten_bit { "P010 (HDR10)" } else { "NV12" }
|
||||
);
|
||||
Ok(CpuInner {
|
||||
enc,
|
||||
@@ -563,6 +655,15 @@ impl DmabufInner {
|
||||
) -> Result<Self> {
|
||||
let drm_fourcc = pf_frame::drm_fourcc(format)
|
||||
.ok_or_else(|| anyhow!("no DRM fourcc for {format:?} (VAAPI zero-copy)"))?;
|
||||
// A 10-bit HDR capture (X2RGB10/X2BGR10 dmabufs, PQ/BT.2020) maps + CSCs to P010 and
|
||||
// encodes Main10; the 8-bit paths keep the NV12/BT.709 graph byte-for-byte unchanged.
|
||||
let ten_bit = format.is_hdr_rgb10();
|
||||
let sw_format = match format {
|
||||
PixelFormat::X2Rgb10 => ffi::AVPixelFormat::AV_PIX_FMT_X2RGB10LE,
|
||||
PixelFormat::X2Bgr10 => ffi::AVPixelFormat::AV_PIX_FMT_X2BGR10LE,
|
||||
// The 8-bit capture formats are all XR24-shaped packed RGB (the historical BGR0 view).
|
||||
_ => ffi::AVPixelFormat::AV_PIX_FMT_BGR0,
|
||||
};
|
||||
let node = render_node();
|
||||
// SAFETY: libav is initialized (`VaapiEncoder::open` ran `ffmpeg::init()` before
|
||||
// `ensure_inner` → `DmabufInner::open`). Every raw pointer dereferenced below is either freshly
|
||||
@@ -628,7 +729,7 @@ impl DmabufInner {
|
||||
}
|
||||
let fc = (*drm_frames).data as *mut ffi::AVHWFramesContext;
|
||||
(*fc).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME;
|
||||
(*fc).sw_format = ffi::AVPixelFormat::AV_PIX_FMT_BGR0; // packed XR24 RGB plane
|
||||
(*fc).sw_format = sw_format; // packed XR24 RGB plane, or XR30/XB30 for HDR
|
||||
(*fc).width = width as c_int;
|
||||
(*fc).height = height as c_int;
|
||||
if ffi::av_hwframe_ctx_init(drm_frames) < 0 {
|
||||
@@ -715,14 +816,24 @@ impl DmabufInner {
|
||||
}
|
||||
init!(src, ptr::null(), "buffer");
|
||||
init!(hwmap, c"mode=read".as_ptr(), "hwmap");
|
||||
// Pin the VPP's output colour to what the encoder's VUI signals (BT.709 limited).
|
||||
// Without the explicit options the conversion matrix is whatever the driver defaults
|
||||
// to for an unspecified output (Mesa: BT.601) — a hue shift against the signaled VUI.
|
||||
init!(
|
||||
scale,
|
||||
c"format=nv12:out_color_matrix=bt709:out_range=limited".as_ptr(),
|
||||
"scale_vaapi"
|
||||
);
|
||||
// Pin the VPP's output colour to what the encoder's VUI signals (BT.709 limited SDR,
|
||||
// or BT.2020 limited P010 for HDR — the PQ transfer is per-channel and rides through
|
||||
// the matrix untouched). Without the explicit options the conversion matrix is
|
||||
// whatever the driver defaults to for an unspecified output (Mesa: BT.601) — a hue
|
||||
// shift against the signaled VUI.
|
||||
if ten_bit {
|
||||
init!(
|
||||
scale,
|
||||
c"format=p010:out_color_matrix=bt2020:out_range=limited".as_ptr(),
|
||||
"scale_vaapi"
|
||||
);
|
||||
} else {
|
||||
init!(
|
||||
scale,
|
||||
c"format=nv12:out_color_matrix=bt709:out_range=limited".as_ptr(),
|
||||
"scale_vaapi"
|
||||
);
|
||||
}
|
||||
init!(sink, ptr::null(), "buffersink");
|
||||
|
||||
let link = |a: *mut ffi::AVFilterContext, b: *mut ffi::AVFilterContext| -> c_int {
|
||||
@@ -766,6 +877,7 @@ impl DmabufInner {
|
||||
bitrate_bps,
|
||||
vaapi_device,
|
||||
nv12_ctx,
|
||||
ten_bit,
|
||||
) {
|
||||
Ok(enc) => enc,
|
||||
Err(e) => {
|
||||
@@ -779,7 +891,8 @@ impl DmabufInner {
|
||||
|
||||
tracing::info!(
|
||||
encoder = codec.vaapi_name(),
|
||||
"VAAPI encode active ({width}x{height}@{fps}, zero-copy dmabuf → GPU NV12)"
|
||||
"VAAPI encode active ({width}x{height}@{fps}, zero-copy dmabuf → GPU {})",
|
||||
if ten_bit { "P010 (HDR10)" } else { "NV12" }
|
||||
);
|
||||
Ok(DmabufInner {
|
||||
enc,
|
||||
@@ -987,8 +1100,22 @@ impl VaapiEncoder {
|
||||
bit_depth: u8,
|
||||
chroma: super::ChromaFormat,
|
||||
) -> Result<Self> {
|
||||
if bit_depth != 8 {
|
||||
tracing::warn!(bit_depth, "VAAPI 10-bit not yet wired — encoding 8-bit");
|
||||
// 10-bit rides on the captured format: an HDR capture (X2RGB10/X2BGR10) opens the P010 /
|
||||
// Main10 / PQ-VUI variant of whichever inner path the first frame selects. A 10-bit
|
||||
// request whose capture stayed SDR honestly encodes 8-bit; the reverse (PQ frames on an
|
||||
// 8-bit session) is refused so PQ content is never mislabeled BT.709.
|
||||
if format.is_hdr_rgb10() && bit_depth != 10 {
|
||||
bail!(
|
||||
"captured 10-bit HDR frames ({format:?}) on an {bit_depth}-bit VAAPI session — \
|
||||
refusing to mislabel PQ content"
|
||||
);
|
||||
}
|
||||
if bit_depth == 10 && !format.is_hdr_rgb10() {
|
||||
tracing::warn!(
|
||||
bit_depth,
|
||||
?format,
|
||||
"10-bit requested but the capture stayed SDR — encoding 8-bit"
|
||||
);
|
||||
}
|
||||
// VAAPI 4:4:4 is deferred (see `probe_can_encode_444`): no validated AMD/Intel hardware in the
|
||||
// lab exposes a HEVC 4:4:4 encode entrypoint, and the probe returns false so the host never
|
||||
|
||||
@@ -181,10 +181,10 @@ impl Encoder for OpenH264Encoder {
|
||||
PixelFormat::Bgr => (3, 2, 1, 0),
|
||||
PixelFormat::Rgba | PixelFormat::Rgbx => (4, 0, 1, 2),
|
||||
PixelFormat::Bgra | PixelFormat::Bgrx => (4, 2, 1, 0),
|
||||
// 10-bit HDR comes only from the GPU NVENC path; the software 8-bit H.264 encoder
|
||||
// can't represent it (and never receives it — the capturer pairs Rgb10a2 with NVENC).
|
||||
PixelFormat::Rgb10a2 => {
|
||||
anyhow::bail!("software H.264 encoder cannot encode 10-bit HDR (Rgb10a2)")
|
||||
// 10-bit HDR comes only from the GPU paths; the software 8-bit H.264 encoder can't
|
||||
// represent it (and never receives it — HDR is never negotiated on a software host).
|
||||
PixelFormat::Rgb10a2 | PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10 => {
|
||||
anyhow::bail!("software H.264 encoder cannot encode 10-bit HDR ({:?})", self.src_format)
|
||||
}
|
||||
// NV12/P010 are GPU-resident video-processor outputs for the NVENC path; the software
|
||||
// encoder never receives them (it only gets CPU RGB frames).
|
||||
|
||||
@@ -288,8 +288,14 @@ fn open_video_backend(
|
||||
// stream never dies over the new path. `format`/`bit_depth`/`chroma` only matter to VAAPI —
|
||||
// the Vulkan backend imports the dmabuf and does its own 8-bit 4:2:0 CSC.
|
||||
let open_amd_intel = || -> Result<(Box<dyn Encoder>, &'static str)> {
|
||||
// An HDR session (10-bit + a PQ/BT.2020 capture format) must skip the Vulkan Video
|
||||
// backend — it hardcodes an 8-bit 4:2:0 BT.709 CSC — and take the libav VAAPI path,
|
||||
// which has the P010/Main10/PQ wiring. SDR sessions keep the Vulkan default.
|
||||
#[cfg(feature = "vulkan-encode")]
|
||||
if matches!(codec, Codec::H265 | Codec::Av1) && vulkan_encode_enabled() {
|
||||
if matches!(codec, Codec::H265 | Codec::Av1)
|
||||
&& vulkan_encode_enabled()
|
||||
&& !(bit_depth == 10 && format.is_hdr_rgb10())
|
||||
{
|
||||
match vulkan_video::VulkanVideoEncoder::open(codec, width, height, fps, bitrate_bps)
|
||||
{
|
||||
Ok(e) => {
|
||||
@@ -956,11 +962,10 @@ pub fn can_encode_444(_codec: Codec) -> bool {
|
||||
/// Backend truth: Windows **NVENC** queries the per-codec `NV_ENC_CAPS_SUPPORT_10BIT_ENCODE` cap;
|
||||
/// native **AMF** `Init`s a tiny P010 encoder with the 10-bit profile props (the driver rejects
|
||||
/// what the VCN can't do). **QSV** stays `false` until validated on Intel glass — the libavcodec
|
||||
/// Main10 incantation can silently encode 8-bit, the same stance as its 4:4:4 probe. Every
|
||||
/// **Linux** backend is `false` today: direct-NVENC/CUDA pins 8-bit until a P010 capture path
|
||||
/// exists (Phase 5.1), libav `hevc_nvenc` needs a 10-bit input format the capturer never feeds,
|
||||
/// VAAPI 10-bit isn't wired, and Vulkan-video hardcodes 8-bit — so Linux hosts honestly negotiate
|
||||
/// 8-bit SDR.
|
||||
/// Main10 incantation can silently encode 8-bit, the same stance as its 4:4:4 probe. **Linux**
|
||||
/// probes a tiny real Main10 open on the auto-resolved backend — libav NVENC (the HDR X2RGB10→
|
||||
/// P010 swscale path) or VAAPI (P010 pool + Main10) — for the GNOME 50+ HDR portal capture;
|
||||
/// the direct-SDK CUDA path and Vulkan-video stay 8-bit and a 10-bit session routes around them.
|
||||
#[cfg(any(target_os = "linux", target_os = "windows"))]
|
||||
pub fn can_encode_10bit(codec: Codec) -> bool {
|
||||
use std::collections::HashMap;
|
||||
@@ -985,8 +990,18 @@ pub fn can_encode_10bit(codec: Codec) -> bool {
|
||||
let supported = {
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
// No Linux backend encodes 10-bit yet (see the fn doc) — never negotiate it.
|
||||
false
|
||||
// NVENC (libav, the HDR P010 swscale path) or VAAPI (P010 upload / dmabuf graph),
|
||||
// probed by opening a tiny real Main10 encoder — the same honesty contract as
|
||||
// `can_encode_444`. Vulkan-video and the direct-SDK CUDA path stay 8-bit; a 10-bit
|
||||
// session routes around them (see `open_video_backend`). NOTE: encode capability is
|
||||
// only half the Linux gate — the capture side (GNOME 50+ portal monitor in HDR mode)
|
||||
// is resolved separately by the host (`capturer_supports_hdr` / the GameStream RTSP
|
||||
// honor), since this probe can't know what the compositor will negotiate.
|
||||
if linux_auto_is_vaapi() {
|
||||
vaapi::probe_can_encode_10bit(codec)
|
||||
} else {
|
||||
linux::probe_can_encode_10bit(codec)
|
||||
}
|
||||
}
|
||||
#[cfg(target_os = "windows")]
|
||||
{
|
||||
|
||||
@@ -56,6 +56,19 @@ pub enum PixelFormat {
|
||||
/// `DeviceBuffer::yuv444` — three full-res planes stacked in one allocation); NVENC encodes
|
||||
/// it natively under the Range-Extensions profile. Never a CPU payload.
|
||||
Yuv444,
|
||||
/// 10-bit RGB packed `x:R:G:B 2:10:10:10` little-endian (SPA `xRGB_210LE`, DRM `XRGB2101010` /
|
||||
/// `XR30`, ffmpeg `x2rgb10le`, NVENC `ARGB10`) — as an LE u32: B in bits 0-9, G 10-19, R 20-29.
|
||||
/// The Linux GNOME 50+ HDR screencast source format: Mutter advertises it (with BT.2020
|
||||
/// primaries + SMPTE ST.2084 PQ transfer) for a monitor in HDR mode, so the samples are
|
||||
/// PQ-encoded BT.2020 RGB. Linux-only; the Windows HDR path stays `Rgb10a2`/`P010`.
|
||||
X2Rgb10,
|
||||
/// 10-bit RGB packed `x:B:G:R 2:10:10:10` little-endian (SPA `xBGR_210LE`, DRM `XBGR2101010` /
|
||||
/// `XB30`, ffmpeg `x2bgr10le`, NVENC `ABGR10`) — as an LE u32: R in bits 0-9, G 10-19, B 20-29;
|
||||
/// the same memory layout as the Windows [`Rgb10a2`](Self::Rgb10a2) (DXGI `R10G10B10A2`). The
|
||||
/// second GNOME 50+ HDR screencast format (same PQ/BT.2020 colorimetry as
|
||||
/// [`X2Rgb10`](Self::X2Rgb10)); kept separate from `Rgb10a2` so the Linux and Windows HDR
|
||||
/// paths stay independently greppable.
|
||||
X2Bgr10,
|
||||
}
|
||||
|
||||
impl PixelFormat {
|
||||
@@ -67,6 +80,12 @@ impl PixelFormat {
|
||||
_ => 4,
|
||||
}
|
||||
}
|
||||
|
||||
/// True for the packed 10-bit RGB layouts a Linux HDR (BT.2020 PQ) capture negotiates —
|
||||
/// the formats that make a session's encode bit depth 10 (HEVC Main10 / 10-bit AV1).
|
||||
pub fn is_hdr_rgb10(self) -> bool {
|
||||
matches!(self, PixelFormat::X2Rgb10 | PixelFormat::X2Bgr10)
|
||||
}
|
||||
}
|
||||
|
||||
/// DRM FourCC for a packed 32-bit format name (little-endian, e.g. `b"XR24"`).
|
||||
@@ -86,6 +105,9 @@ pub fn drm_fourcc(format: PixelFormat) -> Option<u32> {
|
||||
Bgra => drm_fourcc_code(b"AR24"), // DRM_FORMAT_ARGB8888
|
||||
Rgbx => drm_fourcc_code(b"XB24"), // DRM_FORMAT_XBGR8888
|
||||
Rgba => drm_fourcc_code(b"AB24"), // DRM_FORMAT_ABGR8888
|
||||
// The GNOME 50+ HDR screencast formats (packed 2:10:10:10, PQ/BT.2020).
|
||||
X2Rgb10 => drm_fourcc_code(b"XR30"), // DRM_FORMAT_XRGB2101010
|
||||
X2Bgr10 => drm_fourcc_code(b"XB30"), // DRM_FORMAT_XBGR2101010
|
||||
// 24-bit packed RGB/BGR have no straightforward dmabuf import here; use the CPU path.
|
||||
// Rgb10a2/Nv12/P010 are the Windows HDR / video-processor formats — never produced on
|
||||
// Linux; Yuv444 is OUR convert's OUTPUT, never a capture source format.
|
||||
|
||||
@@ -12,7 +12,10 @@ use anyhow::Result;
|
||||
// `crate::capture::*` (the capture mechanics that used the rest moved into pf-capture).
|
||||
pub use pf_frame::{CapturedFrame, OutputFormat, PixelFormat};
|
||||
// The capturer types + trait + synthetics live in `pf-capture`; re-export them at the old paths.
|
||||
pub use pf_capture::{capturer_supports_444, Capturer, FastSyntheticCapturer, SyntheticCapturer};
|
||||
pub use pf_capture::{
|
||||
capturer_supports_444, capturer_supports_hdr, Capturer, FastSyntheticCapturer,
|
||||
SyntheticCapturer,
|
||||
};
|
||||
// `crate::capture::dxgi::{install_gpu_pref_hook, hdr_p010_selftest}` (main.rs subcommands) and
|
||||
// `crate::capture::synthetic_nv12` resolve through pf-capture's Windows modules.
|
||||
#[cfg(target_os = "windows")]
|
||||
@@ -45,18 +48,20 @@ fn zero_copy_policy() -> pf_capture::ZeroCopyPolicy {
|
||||
}
|
||||
}
|
||||
|
||||
/// Open a live capturer for a client-sized monitor via the xdg ScreenCast portal.
|
||||
/// Open a live capturer for a client-sized monitor via the xdg ScreenCast portal. `want_hdr`
|
||||
/// offers the GNOME 50+ 10-bit PQ/BT.2020 formats (pass it only when the session negotiated HDR
|
||||
/// AND the mirrored monitor is in HDR mode — see [`pf_capture::gnome_hdr_monitor_active`]).
|
||||
#[cfg(target_os = "linux")]
|
||||
pub fn open_portal_monitor() -> Result<Box<dyn Capturer>> {
|
||||
pub fn open_portal_monitor(want_hdr: bool) -> Result<Box<dyn Capturer>> {
|
||||
// On RemoteDesktop-capable desktops (KWin/GNOME) anchor ScreenCast to a RemoteDesktop
|
||||
// session so it inherits that grant headlessly; wlroots/Sway has no RemoteDesktop portal,
|
||||
// so use a plain ScreenCast session there.
|
||||
let anchored = crate::inject::default_backend() == crate::inject::Backend::Libei;
|
||||
pf_capture::open_portal_monitor(anchored, zero_copy_policy())
|
||||
pf_capture::open_portal_monitor(anchored, want_hdr, zero_copy_policy())
|
||||
}
|
||||
|
||||
#[cfg(not(target_os = "linux"))]
|
||||
pub fn open_portal_monitor() -> Result<Box<dyn Capturer>> {
|
||||
pub fn open_portal_monitor(_want_hdr: bool) -> Result<Box<dyn Capturer>> {
|
||||
anyhow::bail!("portal capture requires Linux (xdg-desktop-portal + PipeWire)")
|
||||
}
|
||||
|
||||
@@ -69,11 +74,13 @@ pub fn capture_virtual_output(
|
||||
want: OutputFormat,
|
||||
_capture: crate::session_plan::CaptureBackend,
|
||||
) -> Result<Box<dyn Capturer>> {
|
||||
// The Linux host stays 8-bit (HDR is blocked upstream) and the portal negotiates its own pixel
|
||||
// format, so `want.gpu` gates GPU zero-copy capture (the capture backend is always the portal —
|
||||
// the `CaptureBackend` arg is a Windows-only dispatch) and `want.chroma_444` selects the
|
||||
// worker's planar-YUV444 GPU convert. `gpu = false` (4:4:4 without zero-copy) forces the CPU
|
||||
// mmap path so the encoder gets CPU-resident RGB to swscale into YUV444P.
|
||||
// The Linux NATIVE plane stays 8-bit (Mutter's virtual-monitor streams are SDR-only upstream;
|
||||
// the GNOME 50+ HDR path is monitor-mirror only — `open_portal_monitor`) and the portal
|
||||
// negotiates its own pixel format, so `want.gpu` gates GPU zero-copy capture (the capture
|
||||
// backend is always the portal — the `CaptureBackend` arg is a Windows-only dispatch) and
|
||||
// `want.chroma_444` selects the worker's planar-YUV444 GPU convert. `gpu = false` (4:4:4
|
||||
// without zero-copy) forces the CPU mmap path so the encoder gets CPU-resident RGB to swscale
|
||||
// into YUV444P.
|
||||
pf_capture::open_virtual_output(
|
||||
vout.remote_fd,
|
||||
vout.node_id,
|
||||
|
||||
@@ -50,21 +50,45 @@ pub const SCM_AV1_MAIN10: u32 = 0x0002_0000;
|
||||
/// The **SDR baseline** codec mask: H.264, HEVC Main, AV1 Main 8-bit (= 65793). HEVC Main10 (HDR) is
|
||||
/// layered on top of this at runtime by `serverinfo::codec_mode_support` when — and only when — the
|
||||
/// host can actually deliver it ([`host_hdr_capable`]); it is never a static claim, because a non-HDR
|
||||
/// host (Linux, or a Windows host without the `PUNKTFUNK_10BIT` opt-in) must not invite a client into
|
||||
/// an HDR mode it can't produce. (The previous placeholder 3843 = 0xF03 wrongly claimed HEVC Main10 +
|
||||
/// host (a host without the `PUNKTFUNK_10BIT` opt-in, or a Linux host whose video source / encoder
|
||||
/// can't do Main10) must not invite a client into an HDR mode it can't produce. (The previous placeholder 3843 = 0xF03 wrongly claimed HEVC Main10 +
|
||||
/// 4:4:4 and *no* AV1.) 4:4:4 stays off entirely on GameStream: stock Moonlight is 4:2:0 —
|
||||
/// full-chroma is a punktfunk/1-native negotiation only (`crate::capture::capturer_supports_444`).
|
||||
pub const SERVER_CODEC_MODE_SUPPORT: u32 = SCM_H264 | SCM_HEVC | SCM_AV1_MAIN8;
|
||||
|
||||
/// Whether this host can deliver an **HDR** (HEVC Main10 / BT.2020 PQ) GameStream — the single gate
|
||||
/// for advertising [`SCM_HEVC_MAIN10`] in serverinfo and `IsHdrSupported` per app, and for honoring a
|
||||
/// client's `dynamicRangeMode` request. HDR capture+encode is **Windows-only** (the Linux host is
|
||||
/// 8-bit, blocked upstream) and behind the operator's `PUNKTFUNK_10BIT` opt-in — the same policy gate
|
||||
/// the native punktfunk/1 plane honors. When this is true the IDD-push capturer streams HEVC Main10 PQ
|
||||
/// whenever the desktop is HDR, and a client HDR request makes the GameStream video path proactively
|
||||
/// enable advanced color on the per-session virtual display so PQ flows even from an SDR desktop.
|
||||
/// for advertising [`SCM_HEVC_MAIN10`] in serverinfo and `IsHdrSupported` per app, and (together
|
||||
/// with the live capture-side check at RTSP time) for honoring a client's `dynamicRangeMode`
|
||||
/// request. Behind the operator's `PUNKTFUNK_10BIT` opt-in — the same policy gate the native
|
||||
/// punktfunk/1 plane honors — on both OSes.
|
||||
///
|
||||
/// **Windows**: the IDD-push capturer streams HEVC Main10 PQ whenever the desktop is HDR, and a
|
||||
/// client HDR request proactively enables advanced color on the per-session virtual display so PQ
|
||||
/// flows even from an SDR desktop.
|
||||
///
|
||||
/// **Linux**: the GNOME 50+ portal **monitor mirror** (`video_source=portal`) can negotiate the
|
||||
/// 10-bit PQ formats while the mirrored monitor is in HDR mode, and the NVENC/VAAPI encoders have
|
||||
/// a probed Main10 path ([`crate::encode::can_encode_10bit`]). The virtual-output source stays SDR
|
||||
/// (Mutter's RecordVirtual streams are 8-bit-only upstream), so this is `false` for it. Whether
|
||||
/// the monitor is ACTUALLY in HDR mode right now is checked live at RTSP honor time
|
||||
/// ([`pf_capture::gnome_hdr_monitor_active`]) — this fn is the static serverinfo capability.
|
||||
pub fn host_hdr_capable() -> bool {
|
||||
cfg!(target_os = "windows") && pf_host_config::config().ten_bit
|
||||
if !pf_host_config::config().ten_bit {
|
||||
return false;
|
||||
}
|
||||
#[cfg(target_os = "windows")]
|
||||
{
|
||||
true
|
||||
}
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
pf_host_config::config().video_source.as_deref() == Some("portal")
|
||||
&& crate::encode::can_encode_10bit(crate::encode::Codec::H265)
|
||||
}
|
||||
#[cfg(not(any(target_os = "windows", target_os = "linux")))]
|
||||
{
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
/// Stable host identity + advertised capabilities, shared across control-plane handlers.
|
||||
@@ -141,8 +165,11 @@ pub struct AppState {
|
||||
pub rfi_range: std::sync::Arc<std::sync::Mutex<Option<(i64, i64)>>>,
|
||||
/// Persistent screen capturer, reused across streams so reconnects don't spawn a second
|
||||
/// (conflicting) screencast session. The video thread borrows it for the stream's duration
|
||||
/// and returns it; `set_active` gates its cost while idle.
|
||||
pub video_cap: std::sync::Arc<std::sync::Mutex<Option<Box<dyn crate::capture::Capturer>>>>,
|
||||
/// and returns it; `set_active` gates its cost while idle. The slot's `bool` records whether
|
||||
/// it was opened with the HDR (10-bit PQ) offer — a stream whose negotiated `hdr` differs
|
||||
/// drops the pooled capturer and opens a fresh screencast session at the right depth
|
||||
/// (mirroring the audio capturer's channel-count reuse gate).
|
||||
pub video_cap: stream::CapturerSlot,
|
||||
/// Persistent audio capturer, reused across streams when the channel count still matches
|
||||
/// (avoids a PipeWire stream setup per reconnect); drained on reuse so no stale audio is
|
||||
/// sent, dropped + reopened when a session negotiates a different channel count.
|
||||
|
||||
@@ -396,18 +396,30 @@ fn stream_config(map: &HashMap<String, String>) -> Option<StreamConfig> {
|
||||
_ => Codec::H264,
|
||||
};
|
||||
// 10-bit/HDR request (Moonlight sets `dynamicRangeMode != 0` only when it both saw our Main10 SCM
|
||||
// bit AND the user enabled HDR). Honor it only when the host can actually deliver Main10 (Windows +
|
||||
// PUNKTFUNK_10BIT, `host_hdr_capable`); when honored, the video path proactively enables advanced
|
||||
// color on the virtual display so a PQ stream flows even from an SDR desktop. A request we can't
|
||||
// honor degrades to 8-bit SDR (and a desktop that is ALREADY HDR still streams PQ regardless, since
|
||||
// the IDD-push capturer follows the display).
|
||||
// bit AND the user enabled HDR). Honor it only when the host can actually deliver Main10
|
||||
// (`host_hdr_capable` — Windows IDD-push, or the Linux GNOME 50+ portal mirror). On Windows,
|
||||
// when honored, the video path proactively enables advanced color on the virtual display so a
|
||||
// PQ stream flows even from an SDR desktop. On Linux the portal can only deliver PQ while the
|
||||
// MIRRORED monitor is in HDR mode, so additionally probe the live colour mode here (one D-Bus
|
||||
// round-trip, sync RTSP thread) — an SDR desktop honestly degrades to 8-bit SDR up front
|
||||
// instead of running the capture negotiation into its timeout. A request we can't honor
|
||||
// degrades to 8-bit SDR (and a Windows desktop that is ALREADY HDR still streams PQ
|
||||
// regardless, since the IDD-push capturer follows the display).
|
||||
let hdr_requested = parse_u("x-nv-video[0].dynamicRangeMode").unwrap_or(0) != 0;
|
||||
let hdr = hdr_requested && crate::gamestream::host_hdr_capable();
|
||||
let mut hdr = hdr_requested && crate::gamestream::host_hdr_capable();
|
||||
if hdr_requested && !hdr {
|
||||
tracing::warn!(
|
||||
"client requested HDR (dynamicRangeMode != 0) but host is not HDR-capable — streaming 8-bit SDR"
|
||||
);
|
||||
}
|
||||
#[cfg(target_os = "linux")]
|
||||
if hdr && !pf_capture::gnome_hdr_monitor_active() {
|
||||
tracing::warn!(
|
||||
"client requested HDR but no monitor is in BT.2100 (HDR) colour mode — enable HDR in \
|
||||
GNOME Settings → Displays (GNOME 50+) to stream it; streaming 8-bit SDR"
|
||||
);
|
||||
hdr = false;
|
||||
}
|
||||
// The client's requested CSC (moonlight-common-c SdpGenerator.c: `encoderCscMode =
|
||||
// (colorspace << 1) | fullRange` — colorspace 0=Rec601, 1=Rec709, 2=Rec2020). Moonlight
|
||||
// renderers configure their YUV→RGB from this REQUESTED value (not the bitstream VUI), so a
|
||||
|
||||
@@ -34,8 +34,9 @@ pub struct StreamConfig {
|
||||
}
|
||||
|
||||
/// Slot for the persistent screen capturer, shared with the control plane and reused across
|
||||
/// streams so a reconnect doesn't open a second (conflicting) screencast session.
|
||||
pub type CapturerSlot = Arc<std::sync::Mutex<Option<Box<dyn Capturer>>>>;
|
||||
/// streams so a reconnect doesn't open a second (conflicting) screencast session. The `bool` is
|
||||
/// the pooled capturer's HDR-ness (see `AppState::video_cap`).
|
||||
pub type CapturerSlot = Arc<std::sync::Mutex<Option<(Box<dyn Capturer>, bool)>>>;
|
||||
|
||||
/// A pending client reference-frame-invalidation range (lost `firstFrame..=lastFrame`), set by the
|
||||
/// control plane and drained by the video thread (see [`AppState::rfi_range`](super::AppState)).
|
||||
@@ -120,7 +121,7 @@ fn run(
|
||||
running: &Arc<AtomicBool>,
|
||||
force_idr: &AtomicBool,
|
||||
rfi_range: &std::sync::Mutex<Option<(i64, i64)>>,
|
||||
video_cap: &std::sync::Mutex<Option<Box<dyn Capturer>>>,
|
||||
video_cap: &std::sync::Mutex<Option<(Box<dyn Capturer>, bool)>>,
|
||||
// Shared stats recorder for the web-console capture/graph. Threaded into `stream_body` (the
|
||||
// encode loop); per-frame sample emission is wired by a later pass.
|
||||
stats: &Arc<crate::stats_recorder::StatsRecorder>,
|
||||
@@ -243,15 +244,31 @@ fn run(
|
||||
}
|
||||
|
||||
// Reuse the persistent capturer (one screencast session → clean reconnect); create it on
|
||||
// the first stream. Borrow it for this stream and return it on exit.
|
||||
let mut capturer: Box<dyn Capturer> = match video_cap.lock().unwrap().take() {
|
||||
// the first stream. Borrow it for this stream and return it on exit. Reuse is gated on the
|
||||
// pooled capturer's HDR-ness matching this stream's negotiated `cfg.hdr` — the depth is a
|
||||
// PipeWire-negotiation-time property of the screencast session, so an HDR↔SDR change needs a
|
||||
// fresh session (same pattern as the audio capturer's channel-count gate).
|
||||
let pooled = match video_cap.lock().unwrap().take() {
|
||||
Some((c, was_hdr)) if was_hdr == cfg.hdr => Some(c),
|
||||
Some((c, was_hdr)) => {
|
||||
tracing::info!(
|
||||
was_hdr,
|
||||
want_hdr = cfg.hdr,
|
||||
"video source: pooled capturer depth mismatch — opening a fresh screencast session"
|
||||
);
|
||||
drop(c);
|
||||
None
|
||||
}
|
||||
None => None,
|
||||
};
|
||||
let mut capturer: Box<dyn Capturer> = match pooled {
|
||||
Some(c) => {
|
||||
tracing::info!("video source: reusing capturer");
|
||||
c
|
||||
}
|
||||
None if pf_host_config::config().video_source.as_deref() == Some("portal") => {
|
||||
tracing::info!("video source: portal desktop capture");
|
||||
capture::open_portal_monitor().context("open portal capturer")?
|
||||
tracing::info!(hdr = cfg.hdr, "video source: portal desktop capture");
|
||||
capture::open_portal_monitor(cfg.hdr).context("open portal capturer")?
|
||||
}
|
||||
None => {
|
||||
tracing::info!("video source: synthetic test pattern");
|
||||
@@ -272,7 +289,7 @@ fn run(
|
||||
&client_label,
|
||||
);
|
||||
capturer.set_active(false);
|
||||
*video_cap.lock().unwrap() = Some(capturer);
|
||||
*video_cap.lock().unwrap() = Some((capturer, cfg.hdr));
|
||||
result
|
||||
}
|
||||
|
||||
@@ -380,7 +397,9 @@ fn open_gs_virtual_source(
|
||||
// HDR: pass the negotiated `cfg.hdr` (client asked for HDR AND the host can deliver it). On the
|
||||
// Windows IDD-push path this proactively enables advanced color on the virtual display so a Main10
|
||||
// PQ stream flows even from an SDR desktop; an already-HDR desktop streams PQ regardless (the
|
||||
// capturer follows the display). No-op on Linux (8-bit, and `cfg.hdr` is always false there).
|
||||
// capturer follows the display). No-op on Linux: virtual-output capture is SDR-only upstream
|
||||
// (Mutter RecordVirtual), and `host_hdr_capable` therefore keeps `cfg.hdr` false for this
|
||||
// source — the Linux HDR path is the portal monitor mirror (`video_source=portal`).
|
||||
let capturer = capture::capture_virtual_output(
|
||||
vout,
|
||||
capture::OutputFormat::resolve(cfg.hdr, crate::encode::resolved_backend_is_gpu()),
|
||||
@@ -399,7 +418,11 @@ fn open_gs_virtual_source(
|
||||
fn gs_bit_depth(format: crate::capture::PixelFormat) -> u8 {
|
||||
use crate::capture::PixelFormat;
|
||||
match format {
|
||||
PixelFormat::P010 | PixelFormat::Rgb10a2 => 10,
|
||||
// Windows IDD-push HDR formats, and the Linux GNOME 50+ portal HDR formats.
|
||||
PixelFormat::P010
|
||||
| PixelFormat::Rgb10a2
|
||||
| PixelFormat::X2Rgb10
|
||||
| PixelFormat::X2Bgr10 => 10,
|
||||
_ => 8,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -283,6 +283,24 @@ fn real_main() -> Result<()> {
|
||||
// PASS/FAIL + max Y/Cb/Cr error.
|
||||
#[cfg(target_os = "windows")]
|
||||
Some("hdr-p010-selftest") => crate::capture::dxgi::hdr_p010_selftest(),
|
||||
// Linux HDR readiness probe (GNOME 50+ portal path): prints whether a monitor is currently
|
||||
// in BT.2100 (HDR) colour mode, whether the NVENC/VAAPI backend probes Main10 for
|
||||
// HEVC/AV1, and the GameStream HDR capability the two combine into — the "why isn't my
|
||||
// stream HDR?" diagnostic (no display/session needed for the encoder half).
|
||||
#[cfg(target_os = "linux")]
|
||||
Some("hdr-probe") => {
|
||||
let monitor_hdr = pf_capture::gnome_hdr_monitor_active();
|
||||
let hevc10 = encode::can_encode_10bit(encode::Codec::H265);
|
||||
let av110 = encode::can_encode_10bit(encode::Codec::Av1);
|
||||
println!("monitor in BT.2100 (HDR) colour mode: {monitor_hdr}");
|
||||
println!("encoder Main10 (HEVC): {hevc10}");
|
||||
println!("encoder 10-bit (AV1): {av110}");
|
||||
println!(
|
||||
"GameStream HDR capable (PUNKTFUNK_10BIT + video_source=portal + encoder): {}",
|
||||
gamestream::host_hdr_capable()
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
// Compositor readiness probe: exit 0 iff the (detected or PUNKTFUNK_COMPOSITOR-forced)
|
||||
// compositor is up and able to create a virtual output *now*. A session-bringup
|
||||
// script polls this to gate on real readiness instead of a blind `sleep`.
|
||||
|
||||
@@ -212,10 +212,22 @@ pub(super) async fn negotiate(
|
||||
// label that matches the stream.
|
||||
let host_wants_10bit = pf_host_config::config().ten_bit;
|
||||
let client_supports_10bit = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_10BIT != 0;
|
||||
// The capture side must be able to deliver a 10-bit HDR source for the NATIVE plane's
|
||||
// virtual-output capture — the honest-downgrade gate, mirroring `capturer_supports_444`.
|
||||
// Windows IDD-push can (it proactively enables advanced colour); Linux cannot: Mutter's
|
||||
// RecordVirtual virtual-monitor streams are 8-bit-only upstream (GNOME 50 added HDR for
|
||||
// *monitor* streams only — the GameStream portal-mirror path uses that; see
|
||||
// `gamestream::host_hdr_capable`), so a Linux native session honestly stays 8-bit SDR even
|
||||
// though `can_encode_10bit` now probes true on a Main10-capable GPU.
|
||||
let capture_supports_hdr = crate::capture::capturer_supports_hdr();
|
||||
// The GPU probe may open a tiny encoder on first use, so run it off the reactor like the
|
||||
// 4:4:4 probe below (blocking probes → spawn_blocking), short-circuited behind the cheap
|
||||
// gates. The result is cached process-wide per (GPU, codec).
|
||||
let gpu_can_10bit = if host_wants_10bit && client_supports_10bit && codec.supports_10bit() {
|
||||
let gpu_can_10bit = if host_wants_10bit
|
||||
&& client_supports_10bit
|
||||
&& codec.supports_10bit()
|
||||
&& capture_supports_hdr
|
||||
{
|
||||
tokio::task::spawn_blocking(move || crate::encode::can_encode_10bit(codec))
|
||||
.await
|
||||
.context("10-bit capability probe task")?
|
||||
@@ -227,6 +239,7 @@ pub(super) async fn negotiate(
|
||||
bit_depth,
|
||||
host_wants_10bit,
|
||||
client_supports_10bit,
|
||||
capture_supports_hdr,
|
||||
codec = ?codec,
|
||||
gpu_can_10bit,
|
||||
client_video_caps = hello.video_caps,
|
||||
|
||||
@@ -89,7 +89,9 @@ pub struct SessionPlan {
|
||||
/// Handshake-negotiated encode bit depth (8, or 10 = HEVC Main10).
|
||||
pub bit_depth: u8,
|
||||
/// The IDD-push HDR hint (`bit_depth >= 10`) — the want-HDR flag handed to the capturer so it
|
||||
/// proactively enables advanced color on the virtual display. Linux is 8-bit (HDR blocked upstream).
|
||||
/// proactively enables advanced color on the virtual display. The Linux NATIVE plane is 8-bit
|
||||
/// (Mutter's virtual-monitor streams are SDR-only upstream — GNOME 50 HDR is monitor-mirror
|
||||
/// only, which the GameStream portal path uses; see `capture::capturer_supports_hdr`).
|
||||
pub hdr: bool,
|
||||
/// Handshake-negotiated chroma subsampling (4:2:0, or full-chroma 4:4:4 when the client + host +
|
||||
/// GPU all support it). Resolved before the Welcome; `Yuv420` on every backend that declined it.
|
||||
|
||||
@@ -87,8 +87,11 @@ pub fn run(opts: Options) -> Result<()> {
|
||||
}
|
||||
}
|
||||
Source::Portal => {
|
||||
tracing::info!("spike source: xdg ScreenCast portal (live monitor)");
|
||||
capture::open_portal_monitor().context("open portal capturer")?
|
||||
// PUNKTFUNK_SPIKE_HDR=1: run the GNOME 50+ HDR offer (10-bit PQ dmabufs) — the dev
|
||||
// validation lever for the Linux HDR capture path without a full GameStream client.
|
||||
let want_hdr = std::env::var("PUNKTFUNK_SPIKE_HDR").as_deref() == Ok("1");
|
||||
tracing::info!(want_hdr, "spike source: xdg ScreenCast portal (live monitor)");
|
||||
capture::open_portal_monitor(want_hdr).context("open portal capturer")?
|
||||
}
|
||||
Source::KwinVirtual => {
|
||||
let compositor = crate::vdisplay::detect().unwrap_or(crate::vdisplay::Compositor::Kwin);
|
||||
|
||||
@@ -88,7 +88,7 @@ See your desktop page ([KDE](/docs/kde), [GNOME](/docs/gnome)) for when to set t
|
||||
| Setting | Values | Meaning |
|
||||
|---|---|---|
|
||||
| `PUNKTFUNK_FEC_PCT` | `N` (percent) | Forward-error-correction redundancy for lossy links (the default is sensible for a normal LAN). Higher = more loss-resilient, more bandwidth. |
|
||||
| `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. **Windows host only** (the Linux host stays 8-bit). |
|
||||
| `PUNKTFUNK_10BIT` | `1` · `0` *(default on)* | HEVC Main10 / HDR. **On by default** — the host permits 10-bit; a session goes 10-bit only when the client advertises it (behind the client's HDR setting). Set `0` to force 8-bit. Windows host, plus the Linux **GNOME 50+ GameStream desktop mirror** (`PUNKTFUNK_VIDEO_SOURCE=portal`, mirrored monitor in HDR mode — check with `punktfunk-host hdr-probe`). Linux **virtual displays** (native protocol, GameStream default) stay 8-bit: Mutter's virtual-monitor screencast is SDR-only upstream. |
|
||||
| `PUNKTFUNK_444` | `1` · `0` *(default on)* | Full-chroma HEVC 4:4:4 (Range Extensions) — sharper text/desktop, no chroma loss. **On by default** on the host; the client's own 4:4:4 setting (default off) is the real switch. Set `0` to force 4:2:0. **punktfunk/1 native only** (Moonlight stays 4:2:0), HEVC-only, honored only when the client advertises 4:4:4 **and** the GPU supports it (probed; NVENC is the validated path — VAAPI/AMF/QSV decline). Independent of 10-bit. |
|
||||
| `PUNKTFUNK_PYROWAVE_MAX_MBPS` | `N` (Mbps) | Cap the [PyroWave](/docs/pyrowave) Automatic bitrate pin, for a host on a link that the open-loop pin can outrun (e.g. 4:4:4 + HDR at 5120×1440@240 pins ~5.3 Gbps, over a 5GbE link). Unset = no cap. Only affects Automatic (bitrate `0`) PyroWave sessions; an explicit client bitrate bypasses it. |
|
||||
| `PUNKTFUNK_DSCP` | `1` | Opt-in DSCP / `SO_PRIORITY` QoS tagging on the media sockets. No-op on the wire on Windows without a qWAVE policy. |
|
||||
|
||||
@@ -24,7 +24,7 @@ see [Status & Progress](/docs/status).
|
||||
| Web console + pairing | ✅ |
|
||||
| Concurrent sessions (shared desktop) | ✅ |
|
||||
| Network speed test + bitrate | ✅ |
|
||||
| HDR / 10-bit streaming | ✅ Windows host · ⛔ Linux host |
|
||||
| HDR / 10-bit streaming | ✅ Windows host · 🚧 Linux host (GNOME 50+ desktop mirror; virtual displays blocked upstream) |
|
||||
| Surround audio (5.1 / 7.1) | ✅ |
|
||||
| Sub-frame pipelining (latency) | 🔭 |
|
||||
|
||||
@@ -100,10 +100,14 @@ see [Status & Progress](/docs/status).
|
||||
|
||||
## ⛔ Parked / blocked
|
||||
|
||||
- **HDR / 10-bit on the *Linux* host.** HDR streaming already works from a
|
||||
[Windows host](/docs/windows-host) to an HDR-capable client (Windows, Android). On Linux it's
|
||||
blocked upstream — no shipping compositor emits a 10-bit/HDR capture stream yet — and ready the
|
||||
moment one does.
|
||||
- **HDR / 10-bit on the *Linux* host — virtual displays.** GNOME 50 added HDR screencasting for
|
||||
**monitor** streams, and the host now uses it: the GameStream desktop mirror
|
||||
(`PUNKTFUNK_VIDEO_SOURCE=portal`) negotiates the 10-bit PQ formats and encodes HEVC Main10
|
||||
BT.2020 PQ (`punktfunk-host hdr-probe` reports readiness; on-glass validation pending). What
|
||||
stays blocked upstream is HDR on **virtual monitors** — Mutter's `RecordVirtual` streams are
|
||||
still SDR-only (through the GNOME 51 dev branch), so the native protocol and GameStream's
|
||||
default virtual-display source stream 8-bit until that lands; the host is ready the moment it
|
||||
does.
|
||||
- **Advanced DualSense voice-coil haptics.** Scoped and shelved (it rides the controller's USB audio
|
||||
interface, with near-zero game support on Linux). Adaptive triggers, rumble, and the lightbar
|
||||
already ship.
|
||||
|
||||
@@ -35,9 +35,13 @@ host is newer than the Linux host.)
|
||||
- **Zero-copy GPU pipeline.** Captured frames stay on the GPU — dmabuf → CUDA → NVENC on NVIDIA, and
|
||||
VAAPI or Vulkan Video on AMD/Intel — with automatic split-encode at very high resolutions. Stable
|
||||
240 fps at 5120×1440 has been measured. A GPU-less software H.264 encoder exists as an explicit fallback.
|
||||
- **HDR (10-bit), on the Windows host.** An HDR Windows desktop is captured and encoded as HEVC
|
||||
Main10 (BT.2020 PQ) to HDR-capable clients (Windows, Android). Linux hosts stream 8-bit for now —
|
||||
HDR there is blocked upstream at the compositor.
|
||||
- **HDR (10-bit).** An HDR Windows desktop is captured and encoded as HEVC Main10 (BT.2020 PQ) to
|
||||
HDR-capable clients (Windows, Android). On Linux, a **GNOME 50+** host can mirror an HDR monitor
|
||||
over the GameStream desktop-capture source (`PUNKTFUNK_VIDEO_SOURCE=portal`): the portal
|
||||
negotiates the 10-bit PQ screencast formats GNOME 50 added and encodes Main10 PQ (run
|
||||
`punktfunk-host hdr-probe` to check readiness; pending on-glass validation). Linux **virtual
|
||||
displays** — the native protocol and GameStream's default source — still stream 8-bit: Mutter's
|
||||
virtual-monitor screencast is SDR-only upstream.
|
||||
- **Secure by default.** A **SPAKE2 PIN pairing** ceremony establishes trust (the host
|
||||
shows a 4-digit PIN; an attacker gets a single online guess, no offline dictionary
|
||||
attack). Trust-on-first-use (TOFU) remains an explicit opt-in for fully trusted LANs.
|
||||
|
||||
Reference in New Issue
Block a user