feat(host): HDR and 4:4:4 stop being mutually exclusive on Windows
windows-host / package (push) Failing after 22s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
ci / web (push) Successful in 1m7s
ci / rust-arm64 (push) Successful in 1m30s
ci / docs-site (push) Successful in 1m31s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 6s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 9s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 19s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 13s
deb / build-publish-client-arm64 (push) Successful in 2m21s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m6s
deb / build-publish (push) Successful in 4m43s
docker / builders-arm64cross (push) Successful in 11s
docker / deploy-docs (push) Successful in 33s
deb / build-publish-host (push) Successful in 5m4s
android / android (push) Successful in 6m40s
ci / rust (push) Successful in 7m10s
arch / build-publish (push) Successful in 8m11s
apple / swift (push) Successful in 5m8s
apple / screenshots (push) Canceled after 10m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 16m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 15m50s
windows-host / package (push) Failing after 22s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
ci / web (push) Successful in 1m7s
ci / rust-arm64 (push) Successful in 1m30s
ci / docs-site (push) Successful in 1m31s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 8s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 6s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 9s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 11s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 19s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 15s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 13s
deb / build-publish-client-arm64 (push) Successful in 2m21s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m6s
deb / build-publish (push) Successful in 4m43s
docker / builders-arm64cross (push) Successful in 11s
docker / deploy-docs (push) Successful in 33s
deb / build-publish-host (push) Successful in 5m4s
android / android (push) Successful in 6m40s
ci / rust (push) Successful in 7m10s
arch / build-publish (push) Successful in 8m11s
apple / swift (push) Successful in 5m8s
apple / screenshots (push) Canceled after 10m53s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Canceled after 16m14s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Canceled after 15m50s
An HDR display cost you full chroma: the IDD-push capturer's only 10-bit
output was P010, so a session that negotiated 4:4:4 on an HDR desktop was
converted to 4:2:0 at capture time — *after* the Welcome had already told
the client 4:4:4. The client believed it (nothing on the wire contradicts
a Welcome), and the new chroma tag in the stats overlay is what finally
made the discrepancy visible.
Everything except the source was already in place, which is why this is
small: the NVENC config layer has stamped `FREXT` + `chromaFormatIDC=3` +
`pixelBitDepthMinus8=2` — HEVC Main 4:4:4 10 — with a unit test since the
4:4:4 work landed, `PixelFormat::Rgb10a2` already maps to `ABGR10` and
already counts as a full-chroma input, and the desktop client learned the
10-bit 4:4:4 Vulkan pool format in 74863c96. The one missing piece was a
capture format that keeps 10 bits AND full chroma.
`HdrRgb10Converter` is that piece: one full-res pass from the FP16 scRGB
desktop to packed `R10G10B10A2` in BT.2020 PQ, reusing the P010 shader's
`scrgb_to_pq2020` verbatim so both HDR outputs share bit-identical colour
math — it simply stops before the RGB→YUV matrix, the studio-range
squeeze and the chroma decimation. NVENC then does the CSC to YUV 4:4:4
itself under FREXT, exactly as the SDR BGRA passthrough has always done
at 8 bits.
No swizzle is involved and that is worth stating, because it looks like
it should be: NVENC names packed formats from the MSB down, so its
`ABGR10` (A2B10G10R10) puts R in the low 10 bits — bit-identical to DXGI
`R10G10B10A2_UNORM`. It is the same relationship the proven SDR pair
relies on between DXGI `B8G8R8A8` and NVENC's `ARGB`.
The honesty gap closes as a consequence: `capturer_supports_444` no
longer has a depth it cannot serve, so the chroma resolved before the
Welcome is the chroma the wire carries. AV1 is deliberately untouched —
Range Extensions are HEVC-only and no consumer encoder does AV1 4:4:4.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -488,12 +488,15 @@ pub(crate) fn note_hdr_capture_failed(source: HdrSource) {
|
||||
}
|
||||
#[cfg(target_os = "windows")]
|
||||
pub fn capturer_supports_444(encoder_ingests_rgb_444: bool) -> bool {
|
||||
// IDD-push delivers full-chroma BGRA for an SDR 4:4:4 session (skipping the NV12 VideoConverter),
|
||||
// but only a backend that ingests RGB and CSCs it to 4:4:4 itself can use it — today just
|
||||
// direct-NVENC (AMF can't 4:4:4 at all; the QSV/ffmpeg path has no RGB-input 4:4:4 wiring). An HDR
|
||||
// display can't be known here (the virtual display's mode settles after the Welcome); that
|
||||
// combination downgrades at capture time — the capturer emits P010 and the encoder's caps
|
||||
// cross-check reports the 4:2:0 truth (the in-band SPS keeps the client correct either way).
|
||||
// IDD-push delivers full-chroma RGB for a 4:4:4 session — BGRA on an SDR display, packed 10-bit
|
||||
// BT.2020 PQ (`Rgb10a2`) on an HDR one — skipping the subsampling converters entirely. Only a
|
||||
// backend that ingests RGB and CSCs it to 4:4:4 itself can use that: today just direct-NVENC
|
||||
// (AMF can't 4:4:4 at all; the QSV/ffmpeg path has no RGB-input 4:4:4 wiring).
|
||||
//
|
||||
// The display's HDR state is deliberately NOT part of this answer, and no longer needs to be:
|
||||
// both depths have a full-chroma source now, so the chroma resolved here — before the Welcome —
|
||||
// is the chroma the stream really carries. (It used to be a lie whenever the display was HDR:
|
||||
// this returned true, the Welcome promised 4:4:4, and the capturer then quietly emitted P010.)
|
||||
encoder_ingests_rgb_444
|
||||
}
|
||||
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
|
||||
|
||||
@@ -44,8 +44,8 @@ use windows::Win32::Graphics::Direct3D11::{
|
||||
D3D11_USAGE_IMMUTABLE, D3D11_USAGE_STAGING, D3D11_VIEWPORT,
|
||||
};
|
||||
use windows::Win32::Graphics::Dxgi::Common::{
|
||||
DXGI_FORMAT, DXGI_FORMAT_P010, DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM,
|
||||
DXGI_FORMAT_R16_UNORM, DXGI_SAMPLE_DESC,
|
||||
DXGI_FORMAT, DXGI_FORMAT_P010, DXGI_FORMAT_R10G10B10A2_UNORM, DXGI_FORMAT_R16G16B16A16_FLOAT,
|
||||
DXGI_FORMAT_R16G16_UNORM, DXGI_FORMAT_R16_UNORM, DXGI_SAMPLE_DESC,
|
||||
};
|
||||
|
||||
/// How many times DXGI has actually called our hooked `NtGdiDdDDIGetCachedHybridQueryValue`.
|
||||
@@ -362,11 +362,145 @@ float2 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
|
||||
}
|
||||
";
|
||||
|
||||
/// scRGB FP16 → **R10G10B10A2** (BT.2020 PQ, FULL-range RGB) — one full-res pass, the HDR twin of
|
||||
/// the SDR 4:4:4 BGRA passthrough. Keeps full chroma all the way to the encoder: NVENC ingests the
|
||||
/// packed 10-bit RGB (`NV_ENC_BUFFER_FORMAT_ABGR10`) and CSCs it to YUV **4:4:4** itself under
|
||||
/// FREXT, per the BT.2020/PQ VUI the encoder writes — HEVC Main 4:4:4 10. Without this the HDR
|
||||
/// path had only [`HdrP010Converter`], whose chroma pass subsamples, so a session that negotiated
|
||||
/// 4:4:4 on an HDR display silently fell back to 4:2:0.
|
||||
///
|
||||
/// The colour math is [`HDR_P010_COMMON`]'s `scrgb_to_pq2020` verbatim — the SAME pixels the P010
|
||||
/// luma pass starts from — so the two HDR outputs agree bit-for-bit before quantization. Only the
|
||||
/// destination differs: no RGB→YUV, no studio-range squeeze and no chroma decimation here, just the
|
||||
/// hardware's UNORM quantization of the PQ values into 10 bits per channel.
|
||||
///
|
||||
/// Channel order: DXGI `R10G10B10A2_UNORM` stores R in the low 10 bits, which is exactly what
|
||||
/// NVENC calls `ABGR10` (it names A2B10G10R10 from the MSB down) — the same relationship the SDR
|
||||
/// path relies on between DXGI `B8G8R8A8` and NVENC's `ARGB`. So the shader writes natural RGB
|
||||
/// order and no swizzle is needed.
|
||||
pub(crate) struct HdrRgb10Converter {
|
||||
vs: ID3D11VertexShader,
|
||||
ps: ID3D11PixelShader,
|
||||
sampler: ID3D11SamplerState,
|
||||
}
|
||||
|
||||
/// R10G10B10A2 pass PS — full-res, writes PQ-encoded BT.2020 RGB straight to the packed 10-bit
|
||||
/// target. `saturate` is implicit in the UNORM render target; `scrgb_to_pq2020` already clamps.
|
||||
const HDR_RGB10_PS: &str = r"
|
||||
#include_common
|
||||
float4 main(float4 pos : SV_POSITION, float2 uv : TEXCOORD0) : SV_TARGET {
|
||||
return float4(scrgb_to_pq2020(uv), 1.0);
|
||||
}
|
||||
";
|
||||
|
||||
impl HdrRgb10Converter {
|
||||
pub(crate) fn new(device: &ID3D11Device) -> Result<Self> {
|
||||
// SAFETY: every call is a `?`-checked D3D11 method on the live `device` borrow, over
|
||||
// fully-initialized stack descriptors and live `Option` out-params; `compile_shader`
|
||||
// receives `s!()` literals (its contract). Each created COM interface owns its own
|
||||
// reference, and no raw pointer outlives the call that produced it.
|
||||
unsafe {
|
||||
let src = HDR_RGB10_PS.replace("#include_common", HDR_P010_COMMON);
|
||||
let vsb = compile_shader(HDR_VS, s!("main"), s!("vs_5_0"))?;
|
||||
let psb = compile_shader(&src, s!("main"), s!("ps_5_0"))?;
|
||||
let mut vs = None;
|
||||
device.CreateVertexShader(&vsb, None, Some(&mut vs))?;
|
||||
let mut ps = None;
|
||||
device.CreatePixelShader(&psb, None, Some(&mut ps))?;
|
||||
// POINT, like the P010 luma pass: this is a 1:1 full-res resample, so every RT pixel
|
||||
// maps to exactly one source texel centre and filtering would only blur it.
|
||||
let sd = D3D11_SAMPLER_DESC {
|
||||
Filter: D3D11_FILTER_MIN_MAG_MIP_POINT,
|
||||
AddressU: D3D11_TEXTURE_ADDRESS_CLAMP,
|
||||
AddressV: D3D11_TEXTURE_ADDRESS_CLAMP,
|
||||
AddressW: D3D11_TEXTURE_ADDRESS_CLAMP,
|
||||
ComparisonFunc: D3D11_COMPARISON_NEVER,
|
||||
MaxLOD: f32::MAX,
|
||||
..Default::default()
|
||||
};
|
||||
let mut sampler = None;
|
||||
device.CreateSamplerState(&sd, Some(&mut sampler))?;
|
||||
Ok(Self {
|
||||
vs: vs.context("rgb10 vs")?,
|
||||
ps: ps.context("rgb10 ps")?,
|
||||
sampler: sampler.context("rgb10 sampler")?,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// A plain (non-planar) RTV of the packed 10-bit output texture. Built once per out-ring slot,
|
||||
/// like the P010 plane views — never per frame.
|
||||
pub(crate) fn rtv(
|
||||
device: &ID3D11Device,
|
||||
dst: &ID3D11Texture2D,
|
||||
) -> Result<ID3D11RenderTargetView> {
|
||||
// SAFETY: one `?`-checked `CreateRenderTargetView` on the live `device` borrow, with a
|
||||
// fully-initialized descriptor local whose address is taken only for the synchronous call,
|
||||
// plus a live `Option` out-param.
|
||||
unsafe {
|
||||
let desc = D3D11_RENDER_TARGET_VIEW_DESC {
|
||||
Format: DXGI_FORMAT_R10G10B10A2_UNORM,
|
||||
ViewDimension: D3D11_RTV_DIMENSION_TEXTURE2D,
|
||||
Anonymous: D3D11_RENDER_TARGET_VIEW_DESC_0 {
|
||||
Texture2D: D3D11_TEX2D_RTV { MipSlice: 0 },
|
||||
},
|
||||
};
|
||||
let mut rtv: Option<ID3D11RenderTargetView> = None;
|
||||
device
|
||||
.CreateRenderTargetView(
|
||||
dst,
|
||||
Some(&desc as *const D3D11_RENDER_TARGET_VIEW_DESC),
|
||||
Some(&mut rtv),
|
||||
)
|
||||
.context("CreateRenderTargetView(R10G10B10A2 out slot)")?;
|
||||
rtv.context("rgb10 rtv null")
|
||||
}
|
||||
}
|
||||
|
||||
/// Convert `src_srv` (FP16 scRGB, WxH) into the `R10G10B10A2` texture behind `rtv`.
|
||||
pub(crate) fn convert(
|
||||
&self,
|
||||
ctx: &ID3D11DeviceContext,
|
||||
src_srv: &ID3D11ShaderResourceView,
|
||||
rtv: &ID3D11RenderTargetView,
|
||||
w: u32,
|
||||
h: u32,
|
||||
) -> Result<()> {
|
||||
// SAFETY: all D3D11 work runs on the caller's live `ctx` borrow (the owning capture
|
||||
// thread's immediate context) over borrowed slices of fully-initialized locals and clones
|
||||
// of the caller's live SRV/RTV. No raw pointers and no mapping on this path.
|
||||
unsafe {
|
||||
ctx.OMSetBlendState(None, None, 0xffff_ffff); // opaque overwrite
|
||||
ctx.VSSetShader(&self.vs, None);
|
||||
ctx.PSSetShaderResources(0, Some(&[Some(src_srv.clone())]));
|
||||
ctx.PSSetSamplers(0, Some(&[Some(self.sampler.clone())]));
|
||||
ctx.IASetInputLayout(None);
|
||||
ctx.IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
|
||||
let vp = D3D11_VIEWPORT {
|
||||
TopLeftX: 0.0,
|
||||
TopLeftY: 0.0,
|
||||
Width: w as f32,
|
||||
Height: h as f32,
|
||||
MinDepth: 0.0,
|
||||
MaxDepth: 1.0,
|
||||
};
|
||||
ctx.RSSetViewports(Some(&[vp]));
|
||||
ctx.OMSetRenderTargets(Some(&[Some(rtv.clone())]), None);
|
||||
ctx.PSSetShader(&self.ps, None);
|
||||
ctx.Draw(3, 0);
|
||||
// Unbind for the next frame's re-RTV / NVENC read.
|
||||
ctx.OMSetRenderTargets(Some(&[None]), None);
|
||||
ctx.PSSetShaderResources(0, Some(&[None]));
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// scRGB FP16 → **P010** (BT.2020 PQ, 10-bit limited/studio range) conversion, in OUR OWN shader (two
|
||||
/// passes: full-res luma + half-res chroma). NVIDIA's D3D11 VideoProcessor cannot do RGB→P010 (renders
|
||||
/// green), so we quantize to studio-range 10-bit YUV directly and feed NVENC native P010 — skipping
|
||||
/// NVENC's internal RGB→YUV CSC (which runs on the contended SM). One per capture device (rebuilt on
|
||||
/// device recreate).
|
||||
/// device recreate). The 4:4:4 twin is [`HdrRgb10Converter`].
|
||||
///
|
||||
/// Plane writes use per-plane render-target views of the single P010 texture: an `R16_UNORM` RTV
|
||||
/// selects plane 0 (luma, full WxH), an `R16G16_UNORM` RTV selects plane 1 (chroma, W/2 x H/2). This
|
||||
|
||||
@@ -20,8 +20,8 @@
|
||||
#![deny(clippy::undocumented_unsafe_blocks)]
|
||||
|
||||
use super::dxgi::{
|
||||
make_device, BgraToYuvPlanes, D3d11Frame, HdrP010Converter, PyroFrameShare, VideoConverter,
|
||||
WinCaptureTarget,
|
||||
make_device, BgraToYuvPlanes, D3d11Frame, HdrP010Converter, HdrRgb10Converter, PyroFrameShare,
|
||||
VideoConverter, WinCaptureTarget,
|
||||
};
|
||||
use super::{CapturedFrame, Capturer, FramePayload, PixelFormat};
|
||||
use anyhow::{bail, Context, Result};
|
||||
@@ -44,8 +44,8 @@ use windows::Win32::Graphics::Direct3D11::{
|
||||
};
|
||||
use windows::Win32::Graphics::Dxgi::Common::{
|
||||
DXGI_FORMAT, DXGI_FORMAT_B8G8R8A8_UNORM, DXGI_FORMAT_NV12, DXGI_FORMAT_P010,
|
||||
DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM, DXGI_FORMAT_R16_UNORM,
|
||||
DXGI_FORMAT_R8G8_UNORM, DXGI_FORMAT_R8_UNORM, DXGI_SAMPLE_DESC,
|
||||
DXGI_FORMAT_R10G10B10A2_UNORM, DXGI_FORMAT_R16G16B16A16_FLOAT, DXGI_FORMAT_R16G16_UNORM,
|
||||
DXGI_FORMAT_R16_UNORM, DXGI_FORMAT_R8G8_UNORM, DXGI_FORMAT_R8_UNORM, DXGI_SAMPLE_DESC,
|
||||
};
|
||||
use windows::Win32::Graphics::Dxgi::{
|
||||
CreateDXGIFactory1, IDXGIAdapter1, IDXGIFactory4, IDXGIKeyedMutex, IDXGIResource1,
|
||||
@@ -178,6 +178,9 @@ struct OutSlot {
|
||||
/// `(luma R16_UNORM, chroma R16G16_UNORM)` plane views. `None` for NV12/BGRA outputs, which the
|
||||
/// video processor or a plain `CopyResource` writes without an RTV of ours.
|
||||
p010: Option<(ID3D11RenderTargetView, ID3D11RenderTargetView)>,
|
||||
/// Plain RTV of the packed 10-bit slot, for the HDR + 4:4:4 output ([`HdrRgb10Converter`]).
|
||||
/// `None` for every other format.
|
||||
rgb10: Option<ID3D11RenderTargetView>,
|
||||
}
|
||||
|
||||
/// One PyroWave output-ring slot: the two SEPARATE shareable plane textures the wavelet encoder
|
||||
@@ -438,8 +441,9 @@ pub struct IddPushCapturer {
|
||||
/// THROUGH (a plain copy into the out ring, no NV12 VideoConverter) so NVENC gets full-chroma
|
||||
/// RGB and CSCs to 4:4:4 itself — measured on-glass: `chromaFormatIDC=3` + ARGB input yields
|
||||
/// TRUE 4:4:4 and the conversion follows the VUI matrix (BT.709 limited, always written).
|
||||
/// While the display is HDR this is overridden to the P010 path (no 10-bit 4:4:4 source):
|
||||
/// the stream honestly downgrades to 4:2:0 — the encoder's caps cross-check reports it.
|
||||
/// While the display is HDR the same idea runs at 10 bits: [`HdrRgb10Converter`] writes packed
|
||||
/// BT.2020 PQ RGB and NVENC CSCs that to YUV 4:4:4 (Main 4:4:4 10). Either way the chroma the
|
||||
/// Welcome promised is the chroma the wire carries.
|
||||
want_444: bool,
|
||||
/// A PyroWave (wavelet) session (design/pyrowave-windows-host-zerocopy.md +
|
||||
/// design/pyrowave-444-hdr.md). When set, frames come from the separate-plane `pyro_ring`
|
||||
@@ -521,10 +525,15 @@ pub struct IddPushCapturer {
|
||||
/// SDR — keeps the colour-convert OFF the contended 3D/compute engine. Built lazily; rebuilt on a
|
||||
/// size/HDR flip.
|
||||
video_conv: Option<VideoConverter>,
|
||||
/// FP16 scRGB slot → P010 (BT.2020 PQ limited) via two shader passes, used while the display is HDR
|
||||
/// FP16 scRGB slot → P010 (BT.2020 PQ limited) via two shader passes, used while the display is
|
||||
/// HDR and the session did NOT negotiate 4:4:4 (that case takes [`Self::hdr_rgb10_conv`])
|
||||
/// (NVIDIA's VideoProcessor can't do RGB→P010). The passes run on the 3D engine, but it still skips
|
||||
/// NVENC's internal SM-side CSC. Built lazily.
|
||||
hdr_p010_conv: Option<HdrP010Converter>,
|
||||
/// FP16 scRGB slot → packed 10-bit BT.2020 PQ RGB, used while the display is HDR **and** the
|
||||
/// session negotiated 4:4:4 — the full-chroma twin of [`Self::hdr_p010_conv`]. Rebuilt with the
|
||||
/// ring on a mode/HDR flip.
|
||||
hdr_rgb10_conv: Option<HdrRgb10Converter>,
|
||||
last_seq: u64,
|
||||
last_present: Option<(ID3D11Texture2D, PixelFormat)>,
|
||||
status_logged: bool,
|
||||
@@ -649,8 +658,10 @@ impl IddPushCapturer {
|
||||
/// SM-side CSC, because the video processor can only produce subsampled output). We do NOT
|
||||
/// gate HDR on the client's advertised `VIDEO_CAP_10BIT` — clients under-report it (e.g. the
|
||||
/// Mac advertises 10-bit only when its OWN display is HDR), yet all decode Main10 +
|
||||
/// auto-switch, exactly as on the WGC path. HDR wins over 4:4:4 (there is no 10-bit
|
||||
/// full-chroma source): the stream downgrades to 4:2:0 with a warning.
|
||||
/// auto-switch, exactly as on the WGC path. HDR and 4:4:4 now COMPOSE: an HDR display that
|
||||
/// negotiated full chroma emits packed 10-bit BT.2020 PQ RGB (`Rgb10a2`) for NVENC to CSC to
|
||||
/// YUV 4:4:4 — HEVC Main 4:4:4 10. (Before, HDR won and the stream silently downgraded to
|
||||
/// 4:2:0 *after* the Welcome had already promised 4:4:4.)
|
||||
fn out_format(&self) -> (DXGI_FORMAT, PixelFormat) {
|
||||
// PyroWave never uses this out-ring (it has its own separate-plane `pyro_ring`); the
|
||||
// format here only labels the frame. SDR sessions label NV12 (BT.709 limited), HDR
|
||||
@@ -664,7 +675,10 @@ impl IddPushCapturer {
|
||||
}
|
||||
if self.display_hdr {
|
||||
if self.want_444 {
|
||||
warn_444_hdr_downgrade_once();
|
||||
// HDR + full chroma: packed 10-bit RGB (BT.2020 PQ), which NVENC CSCs to YUV
|
||||
// 4:4:4 itself — the HDR twin of the SDR BGRA passthrough below. No subsampling
|
||||
// anywhere on this path (see `HdrRgb10Converter`).
|
||||
return (DXGI_FORMAT_R10G10B10A2_UNORM, PixelFormat::Rgb10a2);
|
||||
}
|
||||
(DXGI_FORMAT_P010, PixelFormat::P010)
|
||||
} else if self.want_444 {
|
||||
@@ -798,6 +812,7 @@ impl IddPushCapturer {
|
||||
self.out_ring.clear(); // the output format changed → rebuild lazily at the new format
|
||||
self.video_conv = None; // converters are sized + HDR-specific → rebuild at the new mode
|
||||
self.hdr_p010_conv = None;
|
||||
self.hdr_rgb10_conv = None;
|
||||
// The PyroWave CSC is mode-baked too (BgraToYuvPlanes picks different SDR vs HDR shaders
|
||||
// and R8/R8G8 vs R16/R16G16 outputs). Without this, a display_hdr flip (Downgrade point D:
|
||||
// client_10bit=true but HDR couldn't enable at open) reused the stale SDR converter against
|
||||
@@ -981,7 +996,12 @@ impl IddPushCapturer {
|
||||
} else {
|
||||
None
|
||||
};
|
||||
self.out_ring.push(OutSlot { tex, p010 });
|
||||
let rgb10 = if format == DXGI_FORMAT_R10G10B10A2_UNORM {
|
||||
Some(HdrRgb10Converter::rtv(&self.device, &tex)?)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
self.out_ring.push(OutSlot { tex, p010, rgb10 });
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
@@ -1076,7 +1096,13 @@ impl IddPushCapturer {
|
||||
/// SDR display, or the FP16→P010 shader on an HDR display. Both keep NVENC's RGB→YUV CSC off the SM.
|
||||
/// An SDR 4:4:4 session needs NO converter — the BGRA slot passes through (see `out_format`).
|
||||
fn ensure_converter(&mut self) -> Result<()> {
|
||||
if self.display_hdr {
|
||||
if self.display_hdr && self.want_444 {
|
||||
// HDR + full chroma: one full-res pass to packed 10-bit BT.2020 PQ RGB; NVENC does
|
||||
// the RGB→YUV444 CSC (there is nothing to subsample, so no second pass).
|
||||
if self.hdr_rgb10_conv.is_none() {
|
||||
self.hdr_rgb10_conv = Some(HdrRgb10Converter::new(&self.device)?);
|
||||
}
|
||||
} else if self.display_hdr {
|
||||
if self.hdr_p010_conv.is_none() {
|
||||
self.hdr_p010_conv = Some(HdrP010Converter::new(
|
||||
&self.device,
|
||||
@@ -1537,7 +1563,7 @@ impl IddPushCapturer {
|
||||
self.ensure_out_ring()?;
|
||||
self.ensure_converter()?;
|
||||
let s = &self.out_ring[i];
|
||||
(Some((s.tex.clone(), s.p010.clone())), None)
|
||||
(Some((s.tex.clone(), s.p010.clone(), s.rgb10.clone())), None)
|
||||
};
|
||||
let (_, pf) = self.out_format();
|
||||
let ring_len = if self.pyrowave {
|
||||
@@ -1584,11 +1610,20 @@ impl IddPushCapturer {
|
||||
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
|
||||
conv.convert(&self.context, src, y_rtv, cbcr_rtv, self.width, self.height)?;
|
||||
}
|
||||
} else if self.display_hdr && self.want_444 {
|
||||
// HDR 4:4:4: FP16 slot SRV → packed 10-bit BT.2020 PQ RGB; NVENC ingests it as
|
||||
// ABGR10 and CSCs to YUV 4:4:4 under FREXT (HEVC Main 4:4:4 10).
|
||||
if let Some(conv) = self.hdr_rgb10_conv.as_ref() {
|
||||
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
|
||||
let (_, _, rtv) = out.as_ref().expect("out ring");
|
||||
let rtv = rtv.as_ref().expect("Rgb10a2 out slot has an RTV");
|
||||
conv.convert(&self.context, src, rtv, self.width, self.height)?;
|
||||
}
|
||||
} else if self.display_hdr {
|
||||
// HDR: FP16 slot SRV → P010 (BT.2020 PQ) via the shader; NVENC takes native P010.
|
||||
if let Some(conv) = self.hdr_p010_conv.as_ref() {
|
||||
let src = blended.as_ref().map(|(_, srv)| srv).unwrap_or(&slot_srv);
|
||||
let (_, rtvs) = out.as_ref().expect("out ring");
|
||||
let (_, rtvs, _) = out.as_ref().expect("out ring");
|
||||
// The slot's P010 plane views, built once in `ensure_out_ring`.
|
||||
let (y_rtv, uv_rtv) = rtvs.as_ref().expect("P010 out slot has plane RTVs");
|
||||
conv.convert(&self.context, src, y_rtv, uv_rtv, self.width, self.height)?;
|
||||
@@ -2008,21 +2043,6 @@ impl Capturer for IddPushCapturer {
|
||||
}
|
||||
}
|
||||
|
||||
/// A 4:4:4 session while the display is HDR: there is no 10-bit full-chroma source (the FP16
|
||||
/// desktop needs the PQ tone curve, which the P010 shader provides at 4:2:0), so the stream
|
||||
/// honestly downgrades — the encoder's `chroma_444` caps cross-check reports it and the in-band
|
||||
/// SPS keeps the client decoding correctly. Once per process: the state can flap mid-session.
|
||||
fn warn_444_hdr_downgrade_once() {
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
static ONCE: AtomicBool = AtomicBool::new(true);
|
||||
if ONCE.swap(false, Ordering::Relaxed) {
|
||||
tracing::warn!(
|
||||
"4:4:4 negotiated but the display is HDR — no 10-bit full-chroma source exists; \
|
||||
encoding HDR 4:2:0 (P010) instead (disable HDR on the virtual display for 4:4:4)"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for IddPushCapturer {
|
||||
fn drop(&mut self) {
|
||||
// A channel session ending while the secure-desktop guard is engaged must not leave the
|
||||
|
||||
@@ -644,6 +644,7 @@ impl IddPushCapturer {
|
||||
out_idx: 0,
|
||||
video_conv: None,
|
||||
hdr_p010_conv: None,
|
||||
hdr_rgb10_conv: None,
|
||||
last_seq: 0,
|
||||
last_present: None,
|
||||
status_logged: false,
|
||||
|
||||
@@ -326,14 +326,15 @@ pub(super) async fn negotiate(
|
||||
let client_supports_444 = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_444 != 0;
|
||||
// The active capturer must be able to deliver a full-chroma (RGB) source — the honest-downgrade
|
||||
// gate. Linux's portal capturer always can (`capturer_supports_444` returns `true`
|
||||
// unconditionally). On WINDOWS the IDD-push path CAN too — for an SDR 4:4:4 session it passes
|
||||
// the BGRA ring slot straight through, skipping the NV12 VideoConverter — but only a backend
|
||||
// that ingests RGB and CSCs it to 4:4:4 itself can consume that, so the Windows arm forwards
|
||||
// unconditionally). On WINDOWS the IDD-push path CAN too, at either depth: an SDR session
|
||||
// passes the BGRA ring slot straight through and an HDR one converts the FP16 desktop to
|
||||
// packed 10-bit BT.2020 PQ RGB — both skip the subsampling converters. Only a backend that
|
||||
// ingests RGB and CSCs it to 4:4:4 itself can consume that, so the Windows arm forwards
|
||||
// `resolved_backend_ingests_rgb_444()` (today: direct-NVENC only; AMF can't 4:4:4 at all and
|
||||
// the QSV/ffmpeg path has no RGB-input 4:4:4 wiring). An HDR display still downgrades to 4:2:0
|
||||
// at capture time — there is no 10-bit full-chroma source — and the encoder's caps cross-check
|
||||
// reports that truth. (Replaces the old `single_process` gate — single-process is now the only
|
||||
// topology, and 4:4:4 routed to DDA, which was removed.)
|
||||
// the QSV/ffmpeg path has no RGB-input 4:4:4 wiring). HDR no longer costs the chroma: 10-bit
|
||||
// 4:4:4 is HEVC Main 4:4:4 10, which is what this resolves to. (Replaces the old
|
||||
// `single_process` gate — single-process is now the only topology, and 4:4:4 routed to DDA,
|
||||
// which was removed.)
|
||||
// PyroWave does its own RGB→YCbCr CSC and its capture mode always delivers a full-chroma
|
||||
// (RGB/BGRA) source on both OSes — the capturer gate is inherently satisfied; the real
|
||||
// gate is `can_encode_444` (the full-res-chroma CSC variant existing on this OS).
|
||||
|
||||
Reference in New Issue
Block a user