feat(client): 4:4:4 can stay on hardware, and the overlay says what you actually got
Two halves of the same honesty problem (client-gaps handoff items 1+2,
done together as the handoff asked).
The presenter learns full chroma: `vkframe_plane_views` accepts the
2-plane 4:4:4 pool formats (8-bit, and the 10-bit 3PACK16 sibling) —
what NVIDIA's Vulkan Video reports for HEVC RExt decode — with the
accepted set extracted into `vkframe_plane_formats` and pinned by a
decision-table test. The CSC shader needed nothing: its 4:2:0 siting
correction already self-disables when the plane widths match. The VAAPI
leg gets the same treatment (NV24 in `drm_fourcc_for` and the dmabuf
import, full-size chroma plane), and the Vulkan decoder's sw-format
gate admits NV24/P410. 3-plane 4:4:4 stays rejected — it needs a third
CSC binding — and demotes cleanly like every other unsupported format.
Design call (the handoff's fork, argued here as requested): (B)+(C),
not (A). No capability probe gates VIDEO_CAP_444 — software decode is
the guaranteed display floor on both OSes (swscale → RGBA), the decoder
ladder demotes on its own, and a probe-gated bit would turn the switch
inert on exactly the boxes that rely on the fallback. What (A) wanted
from a prediction, the overlay now delivers as ground truth:
The Detailed tier prints the encoder's target next to the measured
rate — `19.4 Mb/s · target 20 Mb/s (auto)` — and the resolved chroma,
`4:4:4→4:2:0` when the host declined the ask (mirroring `HDR→SDR`).
The target is live: `NativeClient::current_bitrate_kbps()` mirrors
every BitrateChanged ack, so an Automatic session's ABR re-targets are
visible as they move. This is the figure whose absence let the
settings-drop bug (9c5af8d7) ship four releases — measured goodput
alone cannot distinguish "the encoder is capped at 20" from "my
200 Mb/s grant met a cheap scene".
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -248,9 +248,14 @@ mod session_main {
|
||||
// 4:4:4 is opt-in and off by default (Settings "Full chroma"): the bit only says
|
||||
// "upgrade me if you can" — the host still gates on its own policy, its capturer,
|
||||
// HEVC, and a real GPU 4:4:4 encode probe, and answers the resolved chroma in the
|
||||
// Welcome BEFORE we build a decoder. Advertised unconditionally when the user asks
|
||||
// for it because every decode path here can produce it: the hardware ones where the
|
||||
// driver decodes RExt, and swscale for the rest (the decoder demotes on its own).
|
||||
// Welcome BEFORE we build a decoder. Advertised whenever the user asks because
|
||||
// every path can DISPLAY it: the Vulkan presenter samples the 2-plane 4:4:4 pool
|
||||
// formats (hardware RExt decode where the driver offers it — NVIDIA today) and
|
||||
// swscale converts anything else for the software rung, with the decoder ladder
|
||||
// demoting on its own. No capability probe gates the bit — software decode is the
|
||||
// guaranteed floor — but the cost is VISIBLE, not silent: the Detailed stats
|
||||
// overlay prints the resolved chroma ("4:4:4→4:2:0" when the host declined) and
|
||||
// the decode path frames actually took.
|
||||
video_caps: punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE
|
||||
| if settings.hdr_enabled {
|
||||
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
|
||||
|
||||
@@ -124,6 +124,21 @@ pub struct Stats {
|
||||
/// The decode path frames actually took this window (`"vaapi"`/`"software"`, empty
|
||||
/// until the first frame) — the OSD's trailing tag; tracks a mid-session fallback.
|
||||
pub decoder: &'static str,
|
||||
/// The encoder's CURRENT target bitrate (kbps): the Welcome resolve, then live per
|
||||
/// `BitrateChanged` ack. What `mbps` (measured goodput) is judged AGAINST — a user
|
||||
/// staring at "19 Mb/s" can't otherwise tell "the encoder is capped at 20" from "my
|
||||
/// 200 Mb/s ask was honoured and this scene is cheap" (the gap that let the
|
||||
/// settings-drop bug ship four releases). `0` = an old host that never reported one.
|
||||
pub target_kbps: u32,
|
||||
/// Automatic bitrate is armed (ABR moves `target_kbps` on its own) — the OSD tags the
|
||||
/// target `(auto)` so a moving figure reads as policy, not a broken setting.
|
||||
pub auto_rate: bool,
|
||||
/// The host resolved full-chroma 4:4:4 for this session (`Welcome::chroma_format`).
|
||||
pub chroma_444: bool,
|
||||
/// This session ADVERTISED `VIDEO_CAP_444` (the Settings "Full chroma" opt-in): with
|
||||
/// `chroma_444` false, the host declined — the OSD says so instead of leaving the
|
||||
/// switch's effect unobservable.
|
||||
pub asked_444: bool,
|
||||
}
|
||||
|
||||
/// Frames the pump keeps waiting for their 0xCF host timing (pts → capture→received µs).
|
||||
@@ -361,6 +376,12 @@ fn pump(
|
||||
}
|
||||
};
|
||||
let force_software = params.force_software.clone();
|
||||
// Session-constant stats facts (design/stats-unification.md): what the target figure is
|
||||
// judged against and whether the 4:4:4 opt-in was honoured. `target_kbps` itself is read
|
||||
// live per window — an Automatic session's ABR moves it.
|
||||
let auto_rate = connector.wants_decode_latency();
|
||||
let chroma_444 = connector.chroma_format == punktfunk_core::quic::CHROMA_IDC_444;
|
||||
let asked_444 = params.video_caps & punktfunk_core::quic::VIDEO_CAP_444 != 0;
|
||||
// Audio is best-effort: a session without it still streams. Gamepads are the
|
||||
// app-lifetime service's job (the UI attaches it on Connected). Audio runs on its own
|
||||
// thread (one puller per plane), blocking on the audio queue like the Apple client.
|
||||
@@ -823,6 +844,10 @@ fn pump(
|
||||
0.0
|
||||
},
|
||||
decoder: dec_path,
|
||||
target_kbps: connector.current_bitrate_kbps(),
|
||||
auto_rate,
|
||||
chroma_444,
|
||||
asked_444,
|
||||
}));
|
||||
window_start = Instant::now();
|
||||
frames_n = 0;
|
||||
|
||||
@@ -950,6 +950,9 @@ pub(crate) fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32>
|
||||
Some(match sw {
|
||||
AV_PIX_FMT_NV12 => fourcc(b'N', b'V', b'1', b'2'),
|
||||
AV_PIX_FMT_P010LE => fourcc(b'P', b'0', b'1', b'0'),
|
||||
// Full-chroma 4:4:4 semi-planar (HEVC RExt decode on drivers that export it as
|
||||
// two planes) — the presenter imports the full-size chroma plane like any other.
|
||||
AV_PIX_FMT_NV24 => fourcc(b'N', b'V', b'2', b'4'),
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
@@ -1024,6 +1027,10 @@ mod tests {
|
||||
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NV12),
|
||||
Some(0x3231_564e)
|
||||
);
|
||||
assert_eq!(
|
||||
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NV24),
|
||||
Some(0x3432_564e)
|
||||
);
|
||||
assert_eq!(
|
||||
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_RGBA),
|
||||
None
|
||||
|
||||
@@ -347,10 +347,16 @@ impl VulkanDecoder {
|
||||
}
|
||||
let fc = (*hwfc_ref).data as *mut ffi::AVHWFramesContext;
|
||||
let sw = (*fc).sw_format;
|
||||
// The 2-plane layouts the presenter's CSC can sample: 4:2:0 (NV12/P010) and
|
||||
// full-chroma 4:4:4 (NV24/P410 — HEVC RExt decode, semi-planar like all
|
||||
// NVDEC output). The presenter's `vkframe_plane_formats` table is the final
|
||||
// authority; anything else bails here so the session demotes cleanly.
|
||||
if sw != ffi::AVPixelFormat::AV_PIX_FMT_NV12
|
||||
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_P010LE
|
||||
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_NV24
|
||||
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_P410LE
|
||||
{
|
||||
bail!("Vulkan decode output {sw:?} unsupported (NV12/P010 only)");
|
||||
bail!("Vulkan decode output {sw:?} unsupported (NV12/P010/NV24/P410 only)");
|
||||
}
|
||||
let vkfc = (*fc).hwctx as *const pf_ffvk::AVVulkanFramesContext;
|
||||
let vk_format = (*vkfc).format[0] as i32;
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
//! VAAPI dmabuf → Vulkan import: per-plane `VkImage`s (R8/GR88 for NV12, R16/GR1616
|
||||
//! for 10-bit P010) with the
|
||||
//! VAAPI dmabuf → Vulkan import: per-plane `VkImage`s (R8/GR88 for NV12 and full-chroma
|
||||
//! NV24, R16/GR1616 for 10-bit P010) with the
|
||||
//! surface's explicit DRM format modifier — the same layer-wise import the EGL presenter
|
||||
//! (`video_gl.rs`) proved on this hardware, minus the toolkit. Same-Mesa export/import
|
||||
//! is the contract; anything a driver rejects surfaces as a clean error and the caller
|
||||
@@ -14,6 +14,13 @@ use std::os::fd::{BorrowedFd, IntoRawFd as _};
|
||||
const DRM_FORMAT_NV12: u32 = 0x3231_564e;
|
||||
/// `fourcc('P','0','1','0')` — 10-bit 4:2:0, 10 bits MSB-aligned in 16 (the HDR path).
|
||||
const DRM_FORMAT_P010: u32 = 0x3031_3050;
|
||||
/// `fourcc('N','V','2','4')` — 8-bit 4:4:4 semi-planar (full-size interleaved chroma
|
||||
/// plane): the 2-plane full-chroma export a VAAPI HEVC RExt decode can hand over. Same
|
||||
/// R8 + R8G8 views as NV12; the CSC shader keys chroma siting off the plane widths, so
|
||||
/// the full-size plane needs nothing else. (Intel's iHD prefers PACKED 4:4:4 exports —
|
||||
/// AYUV/Y410 — which are single-plane and would need their own CSC arm; those still
|
||||
/// demote to software decode. 10-bit 4:4:4 has no settled dmabuf fourcc at all.)
|
||||
const DRM_FORMAT_NV24: u32 = 0x3432_564e;
|
||||
const DRM_FORMAT_MOD_INVALID: u64 = 0x00ff_ffff_ffff_ffff;
|
||||
/// `DRM_FORMAT_MOD_LINEAR` — the fallback when the export carried no explicit modifier.
|
||||
const DRM_FORMAT_MOD_LINEAR: u64 = 0;
|
||||
@@ -92,13 +99,14 @@ pub fn import(
|
||||
if std::env::var_os("PUNKTFUNK_HW_FAULT").is_some_and(|v| v == "import") {
|
||||
bail!("injected import failure (PUNKTFUNK_HW_FAULT=import)");
|
||||
}
|
||||
let (luma_fmt, chroma_fmt) = match frame.fourcc {
|
||||
DRM_FORMAT_NV12 => (vk::Format::R8_UNORM, vk::Format::R8G8_UNORM),
|
||||
DRM_FORMAT_P010 => (vk::Format::R16_UNORM, vk::Format::R16G16_UNORM),
|
||||
other => bail!("hw presenter handles NV12/P010 only (got {other:#x})"),
|
||||
let (luma_fmt, chroma_fmt, chroma_full_res) = match frame.fourcc {
|
||||
DRM_FORMAT_NV12 => (vk::Format::R8_UNORM, vk::Format::R8G8_UNORM, false),
|
||||
DRM_FORMAT_P010 => (vk::Format::R16_UNORM, vk::Format::R16G16_UNORM, false),
|
||||
DRM_FORMAT_NV24 => (vk::Format::R8_UNORM, vk::Format::R8G8_UNORM, true),
|
||||
other => bail!("hw presenter handles NV12/P010/NV24 only (got {other:#x})"),
|
||||
};
|
||||
if frame.planes.len() < 2 {
|
||||
bail!("2-plane 4:2:0 needs 2 planes (got {})", frame.planes.len());
|
||||
bail!("2-plane YCbCr needs 2 planes (got {})", frame.planes.len());
|
||||
}
|
||||
// EGL could leave an INVALID modifier to the driver's implied choice; explicit-
|
||||
// modifier images can't — LINEAR is the only honest guess (debug-visible if wrong).
|
||||
@@ -123,16 +131,14 @@ pub fn import(
|
||||
modifier,
|
||||
)
|
||||
.context("luma plane")?;
|
||||
// 4:2:0 subsamples the chroma plane both ways; 4:4:4 (NV24) keeps it full-size.
|
||||
let (cw, ch) = if chroma_full_res {
|
||||
(frame.width, frame.height)
|
||||
} else {
|
||||
(frame.width.div_ceil(2), frame.height.div_ceil(2))
|
||||
};
|
||||
let (chroma_img, chroma_mem) = match plane_image(
|
||||
device,
|
||||
ext_mem_fd,
|
||||
frame.width.div_ceil(2),
|
||||
frame.height.div_ceil(2),
|
||||
chroma_fmt,
|
||||
c.fd,
|
||||
c.offset,
|
||||
c.stride,
|
||||
modifier,
|
||||
device, ext_mem_fd, cw, ch, chroma_fmt, c.fd, c.offset, c.stride, modifier,
|
||||
)
|
||||
.context("chroma plane")
|
||||
{
|
||||
|
||||
@@ -1976,8 +1976,27 @@ fn stats_text(
|
||||
}
|
||||
let detailed = verbosity == StatsVerbosity::Detailed;
|
||||
let mut text = if detailed {
|
||||
// The encoder target next to the measured rate is the figure whose absence let the
|
||||
// settings-drop bug ship four releases: "19 Mb/s" alone can't distinguish "the
|
||||
// encoder is capped at 20" from "my 200 Mb/s grant met a cheap scene". `(auto)`
|
||||
// marks an Automatic session — the ABR moves the target by design, so a shifting
|
||||
// number reads as policy, not a broken setting. Omitted against an old host that
|
||||
// never reported a rate.
|
||||
let target = match (s.target_kbps, s.auto_rate) {
|
||||
(0, _) => String::new(),
|
||||
(t, true) => format!(" · target {:.0} Mb/s (auto)", f64::from(t) / 1000.0),
|
||||
(t, false) => format!(" · target {:.0} Mb/s", f64::from(t) / 1000.0),
|
||||
};
|
||||
// The chroma tag mirrors the HDR tag's honesty: `4:4:4→4:2:0` = the session asked
|
||||
// for full chroma and the host resolved 4:2:0 (its policy/capturer/encoder gates
|
||||
// said no) — otherwise the Settings switch's effect is unobservable.
|
||||
let chroma = match (s.asked_444, s.chroma_444) {
|
||||
(_, true) => " · 4:4:4",
|
||||
(true, false) => " · 4:4:4→4:2:0",
|
||||
_ => "",
|
||||
};
|
||||
format!(
|
||||
"{mode_line} · {:.0} fps · {:.1} Mb/s · {}{}",
|
||||
"{mode_line} · {:.0} fps · {:.1} Mb/s{target} · {}{}{chroma}",
|
||||
s.fps,
|
||||
s.mbps,
|
||||
if s.decoder.is_empty() { "-" } else { s.decoder },
|
||||
@@ -2263,6 +2282,12 @@ mod tests {
|
||||
lost: 3,
|
||||
lost_pct: 0.4,
|
||||
decoder: "vulkan",
|
||||
// Old-host baseline (no reported target, 4:2:0 never asked): the tier
|
||||
// texts stay exactly what they were before the target/chroma elements.
|
||||
target_kbps: 0,
|
||||
auto_rate: false,
|
||||
chroma_444: false,
|
||||
asked_444: false,
|
||||
},
|
||||
PresentedWindow {
|
||||
e2e_p50_ms: 6.4,
|
||||
@@ -2303,6 +2328,42 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// Detailed shows the negotiated encoder target next to the measured rate — the
|
||||
/// figure whose absence let the settings-drop bug ship four releases — tagged
|
||||
/// `(auto)` when the ABR owns it, plus the honest chroma tag when 4:4:4 was asked.
|
||||
#[test]
|
||||
fn detailed_shows_target_and_chroma_resolution() {
|
||||
let (mut s, p) = sample();
|
||||
let line1 = |s: &Stats, v| {
|
||||
stats_text(v, "m", s, &p, false, false, None)
|
||||
.lines()
|
||||
.next()
|
||||
.unwrap()
|
||||
.to_string()
|
||||
};
|
||||
// Explicit 200 Mb/s honoured, cheap scene: measured AND target both show — the
|
||||
// exact pair a user needs to tell a capped encoder from an idle one.
|
||||
s.target_kbps = 200_000;
|
||||
assert!(line1(&s, StatsVerbosity::Detailed).contains("24.3 Mb/s · target 200 Mb/s · "));
|
||||
// An Automatic session's moving target reads as policy, not a broken setting.
|
||||
(s.target_kbps, s.auto_rate) = (20_000, true);
|
||||
assert!(line1(&s, StatsVerbosity::Detailed).contains("target 20 Mb/s (auto)"));
|
||||
// Normal keeps its old line — the target is a Detailed element.
|
||||
assert!(!line1(&s, StatsVerbosity::Normal).contains("target"));
|
||||
// An old host that never reported a rate shows no target element at all.
|
||||
s.target_kbps = 0;
|
||||
assert!(!line1(&s, StatsVerbosity::Detailed).contains("target"));
|
||||
// 4:4:4 asked and granted…
|
||||
(s.asked_444, s.chroma_444) = (true, true);
|
||||
assert!(line1(&s, StatsVerbosity::Detailed).ends_with("· 4:4:4"));
|
||||
// …vs asked and declined: the downgrade is said out loud, mirroring `HDR→SDR`.
|
||||
s.chroma_444 = false;
|
||||
assert!(line1(&s, StatsVerbosity::Detailed).ends_with("· 4:4:4→4:2:0"));
|
||||
// Unasked stays untagged (4:2:0 is the default — not noise worth a tag).
|
||||
s.asked_444 = false;
|
||||
assert!(!line1(&s, StatsVerbosity::Detailed).contains("4:4:4"));
|
||||
}
|
||||
|
||||
/// Compact omits the latency term until the presenter's first e2e window lands.
|
||||
#[test]
|
||||
fn compact_waits_for_e2e() {
|
||||
|
||||
@@ -815,19 +815,12 @@ impl Presenter {
|
||||
|
||||
/// Per-plane views over a Vulkan-Video frame's multiplanar image — the CSC pass's
|
||||
/// exact sampling contract (the frames pool was created MUTABLE_FORMAT for this).
|
||||
/// 8-bit NV12 (R8 + R8G8) and 10-bit P010/X6 (R10X6 + R10X6G10X6).
|
||||
/// See [`vkframe_plane_formats`] for the accepted pool formats.
|
||||
fn vkframe_plane_views(&self, f: &VkVideoFrame) -> Result<[vk::ImageView; 2]> {
|
||||
let (luma_fmt, chroma_fmt) = if f.vk_format == vk::Format::G8_B8R8_2PLANE_420_UNORM.as_raw()
|
||||
{
|
||||
(vk::Format::R8_UNORM, vk::Format::R8G8_UNORM)
|
||||
} else if f.vk_format == vk::Format::G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16.as_raw() {
|
||||
(
|
||||
vk::Format::R10X6_UNORM_PACK16,
|
||||
vk::Format::R10X6G10X6_UNORM_2PACK16,
|
||||
)
|
||||
} else {
|
||||
let Some((luma_fmt, chroma_fmt)) = vkframe_plane_formats(f.vk_format) else {
|
||||
bail!(
|
||||
"Vulkan-Video pool format {} unsupported (expected 2-plane 4:2:0, 8/10-bit)",
|
||||
"Vulkan-Video pool format {} unsupported (expected 2-plane 4:2:0 or 4:4:4, \
|
||||
8/10-bit — 3-plane layouts need a third CSC binding)",
|
||||
f.vk_format
|
||||
);
|
||||
};
|
||||
@@ -875,6 +868,36 @@ impl Presenter {
|
||||
}
|
||||
}
|
||||
|
||||
/// The (luma, chroma) per-plane view formats for a Vulkan-Video pool format, or `None`
|
||||
/// when this presenter can't sample it (the caller bails; the decoder demotes to
|
||||
/// software — never a black screen).
|
||||
///
|
||||
/// The decision table IS the desktop 4:4:4 display contract, so it's a pure function
|
||||
/// with a test pinning it:
|
||||
/// - 2-plane 4:2:0, 8-bit (NV12-layout) and 10-bit (P010/X6) — the classic pair.
|
||||
/// - 2-plane 4:4:4, 8- and 10-bit — what NVIDIA's Vulkan Video reports for HEVC RExt
|
||||
/// full-chroma decode (semi-planar, like all NVDEC output). The CSC shader already
|
||||
/// handles the full-size chroma plane (its 4:2:0 siting correction self-disables when
|
||||
/// the plane widths match), so accepting the format here is all hardware 4:4:4 needs.
|
||||
/// - 3-plane 4:4:4 stays rejected: the CSC pass samples exactly two planes (luma +
|
||||
/// interleaved chroma); a triplanar pool needs a third binding + shader variant. No
|
||||
/// supported driver reports it for HEVC decode today — revisit when one does.
|
||||
fn vkframe_plane_formats(raw: i32) -> Option<(vk::Format, vk::Format)> {
|
||||
let eight = (vk::Format::R8_UNORM, vk::Format::R8G8_UNORM);
|
||||
let ten = (
|
||||
vk::Format::R10X6_UNORM_PACK16,
|
||||
vk::Format::R10X6G10X6_UNORM_2PACK16,
|
||||
);
|
||||
[
|
||||
(vk::Format::G8_B8R8_2PLANE_420_UNORM, eight),
|
||||
(vk::Format::G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16, ten),
|
||||
(vk::Format::G8_B8R8_2PLANE_444_UNORM, eight),
|
||||
(vk::Format::G10X6_B10X6R10X6_2PLANE_444_UNORM_3PACK16, ten),
|
||||
]
|
||||
.into_iter()
|
||||
.find_map(|(f, planes)| (f.as_raw() == raw).then_some(planes))
|
||||
}
|
||||
|
||||
/// Flatten the 3×vec4 rows for the push-constant block.
|
||||
fn bytemuck_rows(rows: &[[f32; 4]; 3]) -> &[f32] {
|
||||
// SAFETY: [[f32;4];3] is 12 contiguous f32s.
|
||||
@@ -934,3 +957,42 @@ fn unlock_vkframe(f: &VkVideoFrame, sync: &VkFrameSync, submitted: bool, graphic
|
||||
unlock(f.frames_ctx as *mut pf_ffvk::AVHWFramesContext, vkf);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The pool-format decision table (what this presenter can sample → what stays on the
|
||||
/// hardware path, everything else demotes to software decode). Pinned so a format
|
||||
/// added or dropped here is a deliberate act, not a drive-by.
|
||||
#[test]
|
||||
fn vkframe_pool_format_decision_table() {
|
||||
let eight = Some((vk::Format::R8_UNORM, vk::Format::R8G8_UNORM));
|
||||
let ten = Some((
|
||||
vk::Format::R10X6_UNORM_PACK16,
|
||||
vk::Format::R10X6G10X6_UNORM_2PACK16,
|
||||
));
|
||||
// 2-plane 4:2:0, both depths — the classic pair.
|
||||
let f = |fmt: vk::Format| vkframe_plane_formats(fmt.as_raw());
|
||||
assert_eq!(f(vk::Format::G8_B8R8_2PLANE_420_UNORM), eight);
|
||||
assert_eq!(
|
||||
f(vk::Format::G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16),
|
||||
ten
|
||||
);
|
||||
// 2-plane 4:4:4, both depths — hardware full-chroma (NVIDIA RExt decode). Same
|
||||
// per-plane view formats; the full-size chroma plane is the shader's business.
|
||||
assert_eq!(f(vk::Format::G8_B8R8_2PLANE_444_UNORM), eight);
|
||||
assert_eq!(
|
||||
f(vk::Format::G10X6_B10X6R10X6_2PLANE_444_UNORM_3PACK16),
|
||||
ten
|
||||
);
|
||||
// 3-plane 4:4:4 and 2-plane 4:2:2: real formats a driver could report, NOT
|
||||
// sampleable by the two-binding CSC — they must demote, not corrupt.
|
||||
assert_eq!(f(vk::Format::G8_B8_R8_3PLANE_444_UNORM), None);
|
||||
assert_eq!(f(vk::Format::G8_B8R8_2PLANE_422_UNORM), None);
|
||||
assert_eq!(f(vk::Format::G16_B16R16_2PLANE_444_UNORM), None);
|
||||
// Garbage never maps.
|
||||
assert_eq!(vkframe_plane_formats(0), None);
|
||||
assert_eq!(vkframe_plane_formats(-1), None);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -166,6 +166,12 @@ pub struct NativeClient {
|
||||
/// drained per window by the data-plane pump to feed the adaptive-bitrate controller's decode
|
||||
/// signal. Shared with the pump; see [`DecodeLatAcc`].
|
||||
decode_lat: Arc<Mutex<DecodeLatAcc>>,
|
||||
/// The encoder's CURRENT target bitrate (kbps): seeded with the Welcome resolve, then updated
|
||||
/// by every host `BitrateChanged` ack (the ABR's re-targets, host-side clamps). Where
|
||||
/// [`resolved_bitrate_kbps`](Self::resolved_bitrate_kbps) is the session-start negotiation
|
||||
/// frozen for the ABI, this one moves — it's what a stats HUD should print as "target".
|
||||
/// `0` = an old host that never reported a rate.
|
||||
live_bitrate_kbps: Arc<AtomicU32>,
|
||||
/// Whether the adaptive-bitrate controller is armed for this session (Automatic bitrate and not
|
||||
/// a rate-pinned PyroWave stream) — exposed via [`wants_decode_latency`](Self::wants_decode_latency)
|
||||
/// so an embedder skips the per-frame decode measurement when the controller wouldn't use it.
|
||||
@@ -399,6 +405,8 @@ impl NativeClient {
|
||||
let hot_tids = Arc::new(Mutex::new(Vec::new()));
|
||||
let clock_offset = Arc::new(AtomicI64::new(0));
|
||||
let decode_lat = Arc::new(Mutex::new(DecodeLatAcc::default()));
|
||||
// Seeded by the pump from the Welcome (before ready_tx), then follows every ack.
|
||||
let live_bitrate = Arc::new(AtomicU32::new(0));
|
||||
|
||||
let host = host.to_string();
|
||||
let frame_chan_w = frame_chan.clone();
|
||||
@@ -411,6 +419,7 @@ impl NativeClient {
|
||||
let hot_tids_w = hot_tids.clone();
|
||||
let clock_offset_w = clock_offset.clone();
|
||||
let decode_lat_w = decode_lat.clone();
|
||||
let live_bitrate_w = live_bitrate.clone();
|
||||
let ctrl_tx_pump = ctrl_tx.clone(); // the data-plane pump sends adaptive-FEC LossReports
|
||||
let worker = std::thread::Builder::new()
|
||||
.name("punktfunk-client".into())
|
||||
@@ -475,6 +484,7 @@ impl NativeClient {
|
||||
hot_tids: hot_tids_w,
|
||||
clock_offset: clock_offset_w,
|
||||
decode_lat: decode_lat_w,
|
||||
live_bitrate: live_bitrate_w,
|
||||
}));
|
||||
})
|
||||
.map_err(PunktfunkError::Io)?;
|
||||
@@ -523,6 +533,7 @@ impl NativeClient {
|
||||
hot_tids,
|
||||
clock_offset,
|
||||
decode_lat,
|
||||
live_bitrate_kbps: live_bitrate,
|
||||
// The controller arms exactly when the pump does — all three terms, not two: Automatic
|
||||
// (the user asked for bitrate 0), not a rate-pinned PyroWave stream, AND the host
|
||||
// echoed the rate it actually configured. Dropping the last term made this
|
||||
@@ -780,6 +791,15 @@ impl NativeClient {
|
||||
self.wants_decode
|
||||
}
|
||||
|
||||
/// The encoder's CURRENT target bitrate (kbps): the Welcome-resolved rate, live-updated by
|
||||
/// every host `BitrateChanged` ack — an Automatic session's ABR re-targets and the host's
|
||||
/// own clamps included. This is the figure a stats HUD should show as "target" next to
|
||||
/// measured throughput (the [`resolved_bitrate_kbps`](Self::resolved_bitrate_kbps) field
|
||||
/// stays the frozen session-start value). `0` = an old host that never reported one.
|
||||
pub fn current_bitrate_kbps(&self) -> u32 {
|
||||
self.live_bitrate_kbps.load(Ordering::Relaxed)
|
||||
}
|
||||
|
||||
/// Report this client's display-latch grid so the host can phase-lock its capture tick
|
||||
/// (design/phase-locked-capture.md; the vsync-aware presenters call this ~1 Hz).
|
||||
/// `next_latch_host_ns` must already be HOST clock — convert with
|
||||
|
||||
@@ -71,6 +71,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
|
||||
hot_tids,
|
||||
clock_offset,
|
||||
decode_lat,
|
||||
live_bitrate,
|
||||
..
|
||||
} = args;
|
||||
// Copies the pump needs after `negotiated` is handed over to `connect`.
|
||||
@@ -80,6 +81,9 @@ pub(super) async fn run_pump(args: WorkerArgs) {
|
||||
// Seed the live offset with the connect-time estimate BEFORE the embedder can observe the
|
||||
// client (ready_tx): clock_offset_now_ns() never reads a pre-handshake 0 on a skewed pair.
|
||||
clock_offset.store(negotiated.clock_offset_ns, Ordering::Relaxed);
|
||||
// Same discipline for the live encoder target: the Welcome resolve is the starting truth
|
||||
// (0 against an old host that reports none); every BitrateChanged ack moves it from there.
|
||||
live_bitrate.store(negotiated.bitrate_kbps, Ordering::Relaxed);
|
||||
// Bumped by the control task each time a re-sync batch is APPLIED; the pump watches it to
|
||||
// reset its staleness counters and re-arm the clock-based jump-to-live detector.
|
||||
let clock_gen = Arc::new(AtomicU32::new(0));
|
||||
@@ -134,6 +138,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
|
||||
mode_slot,
|
||||
probe: probe.clone(),
|
||||
bitrate_ack: bitrate_ack.clone(),
|
||||
live_bitrate,
|
||||
recovery_kf: recovery_kf.clone(),
|
||||
clock_offset: clock_offset.clone(),
|
||||
clock_gen: clock_gen.clone(),
|
||||
|
||||
@@ -16,6 +16,9 @@ pub(super) struct ControlTask {
|
||||
pub(super) probe: Arc<Mutex<ProbeState>>,
|
||||
/// The latest host `BitrateChanged` ack, drained by the pump's ABR on its report tick.
|
||||
pub(super) bitrate_ack: Arc<Mutex<Option<u32>>>,
|
||||
/// The live encoder-target mirror ([`NativeClient::current_bitrate_kbps`]): unlike the
|
||||
/// drain-once ack slot above, this one always holds the latest acked rate for stats HUDs.
|
||||
pub(super) live_bitrate: Arc<AtomicU32>,
|
||||
/// Outbound decode-recovery KEYFRAME asks, counted here because this is the one choke point
|
||||
/// every emitter funnels through (embedder, `note_frame_index`, the pump's own asks) — the
|
||||
/// pump drains the count per report window as the ABR's recovery signal.
|
||||
@@ -44,6 +47,7 @@ impl ControlTask {
|
||||
mode_slot,
|
||||
probe,
|
||||
bitrate_ack,
|
||||
live_bitrate,
|
||||
recovery_kf,
|
||||
clock_offset,
|
||||
clock_gen,
|
||||
@@ -157,6 +161,11 @@ impl ControlTask {
|
||||
kbps = ack.bitrate_kbps,
|
||||
"host re-targeted encoder bitrate"
|
||||
);
|
||||
// 0 would be a nonsense ack (the controller ignores it too) — don't
|
||||
// let it wipe the HUD's live target.
|
||||
if ack.bitrate_kbps > 0 {
|
||||
live_bitrate.store(ack.bitrate_kbps, Ordering::Relaxed);
|
||||
}
|
||||
*bitrate_ack.lock().unwrap() = Some(ack.bitrate_kbps);
|
||||
} else if let Ok(echo) = ClockEcho::decode(&msg) {
|
||||
match resync.on_echo(&echo, wall_clock_ns()) {
|
||||
|
||||
@@ -6,7 +6,7 @@ use crate::config::{CompositorPref, GamepadPref, Mode};
|
||||
use crate::error::Result;
|
||||
use crate::input::InputEvent;
|
||||
use crate::quic::{HdrMeta, HidOutput};
|
||||
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU64};
|
||||
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU32, AtomicU64};
|
||||
use std::sync::mpsc::SyncSender;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
@@ -73,6 +73,9 @@ pub(crate) struct WorkerArgs {
|
||||
/// Decode-stage latency samples from the embedder (see [`NativeClient::decode_lat`]): the pump
|
||||
/// drains a window mean into the adaptive-bitrate controller's decode signal.
|
||||
pub(crate) decode_lat: Arc<Mutex<DecodeLatAcc>>,
|
||||
/// The live encoder-target mirror (see [`NativeClient::live_bitrate_kbps`]): the worker seeds
|
||||
/// it from the Welcome; the control task updates it on every `BitrateChanged` ack.
|
||||
pub(crate) live_bitrate: Arc<AtomicU32>,
|
||||
}
|
||||
|
||||
/// The worker: QUIC handshake, then the input/datagram/control tasks + the blocking
|
||||
|
||||
@@ -45,8 +45,9 @@ captured input, switch mouse mode, disconnect — are in
|
||||
|
||||
**Compact** is a one-line pill (fps · end-to-end ms · Mb/s, plus a loss flag when frames are being
|
||||
lost). **Normal** adds the stream line and the p50/p95 headline. **Detailed** adds the per-stage
|
||||
breakdown everywhere; on Linux/Windows it also adds the decode path and an HDR tag, on Android the
|
||||
decoder plus the full codec/bit-depth/colour line, and on iOS/tvOS the excluded OS present floor.
|
||||
breakdown everywhere; on Linux/Windows it also adds the encoder's target bitrate, the decode path,
|
||||
an HDR tag and a chroma tag, on Android the decoder plus the full codec/bit-depth/colour line, and
|
||||
on iOS/tvOS the excluded OS present floor.
|
||||
You can also set the level a stream starts at in each client's
|
||||
[Settings](/docs/client-settings#overlay). The examples below are the **Detailed** view.
|
||||
|
||||
@@ -60,7 +61,7 @@ Every client reports the same measurements, but each family lays them out a litt
|
||||
differently. Linux · Windows · Steam Deck:
|
||||
|
||||
```
|
||||
1920×1080@120 · 120 fps · 24.3 Mb/s · vulkan · HDR
|
||||
1920×1080@120 · 120 fps · 24.3 Mb/s · target 30 Mb/s (auto) · vulkan · HDR
|
||||
e2e 14.2/19.8 ms (p50/p95) · host 3.1 · net 6.7 · decode 2.1 · display 2.3 ms
|
||||
host: queue 0.6 · encode 1.8 · xfer 0.2 · pace 0.5 ms
|
||||
lost 3 (2.4%)
|
||||
@@ -90,10 +91,18 @@ lost 3 (2.4%)
|
||||
```
|
||||
|
||||
- **Line 1 — the stream.** Resolution@refresh, frames received per second, and the
|
||||
received video bitrate (goodput — FEC overhead not counted). Linux/Windows append the
|
||||
decode path and an [HDR](/docs/hdr) tag (`HDR`, or `HDR→SDR` when a PQ stream is tone-mapped onto
|
||||
an SDR screen); Android puts its decoder and the negotiated codec, bit depth, colour and
|
||||
chroma on rows of their own underneath; the Apple clients don't report a codec at all.
|
||||
received video bitrate (goodput — FEC overhead not counted). Linux/Windows follow the
|
||||
measured rate with `target N Mb/s` — what the host's encoder is currently *allowed* to
|
||||
produce — so a quiet desktop under a large grant (measured far below target) reads
|
||||
differently from an encoder pinned at its cap (measured hugging the target). `(auto)`
|
||||
means the [Automatic bitrate](/docs/client-settings#bitrate) controller owns the target
|
||||
and moves it with network conditions; no target at all means an older host that doesn't
|
||||
report one. Then the decode path, an [HDR](/docs/hdr) tag (`HDR`, or `HDR→SDR` when a PQ
|
||||
stream is tone-mapped onto an SDR screen), and — when you asked for
|
||||
[full chroma](/docs/client-settings) — the resolved chroma: `4:4:4` when the host
|
||||
granted it, `4:4:4→4:2:0` when it couldn't. Android puts its decoder and the negotiated
|
||||
codec, bit depth, colour and chroma on rows of their own underneath; the Apple clients
|
||||
don't report a codec at all.
|
||||
If the session resolved to a [settings profile](/docs/profiles-and-links), its name closes this
|
||||
line. On **Android** a `⚠ panel NN Hz` warning joins it whenever the device's panel is refreshing
|
||||
*below* the stream's rate — the tell for a phone or TV governor that ignored the requested mode,
|
||||
|
||||
Reference in New Issue
Block a user