feat(client): 4:4:4 can stay on hardware, and the overlay says what you actually got

Two halves of the same honesty problem (client-gaps handoff items 1+2,
done together as the handoff asked).

The presenter learns full chroma: `vkframe_plane_views` accepts the
2-plane 4:4:4 pool formats (8-bit, and the 10-bit 3PACK16 sibling) —
what NVIDIA's Vulkan Video reports for HEVC RExt decode — with the
accepted set extracted into `vkframe_plane_formats` and pinned by a
decision-table test. The CSC shader needed nothing: its 4:2:0 siting
correction already self-disables when the plane widths match. The VAAPI
leg gets the same treatment (NV24 in `drm_fourcc_for` and the dmabuf
import, full-size chroma plane), and the Vulkan decoder's sw-format
gate admits NV24/P410. 3-plane 4:4:4 stays rejected — it needs a third
CSC binding — and demotes cleanly like every other unsupported format.

Design call (the handoff's fork, argued here as requested): (B)+(C),
not (A). No capability probe gates VIDEO_CAP_444 — software decode is
the guaranteed display floor on both OSes (swscale → RGBA), the decoder
ladder demotes on its own, and a probe-gated bit would turn the switch
inert on exactly the boxes that rely on the fallback. What (A) wanted
from a prediction, the overlay now delivers as ground truth:

The Detailed tier prints the encoder's target next to the measured
rate — `19.4 Mb/s · target 20 Mb/s (auto)` — and the resolved chroma,
`4:4:4→4:2:0` when the host declined the ask (mirroring `HDR→SDR`).
The target is live: `NativeClient::current_bitrate_kbps()` mirrors
every BitrateChanged ack, so an Automatic session's ABR re-targets are
visible as they move. This is the figure whose absence let the
settings-drop bug (9c5af8d7) ship four releases — measured goodput
alone cannot distinguish "the encoder is capped at 20" from "my
200 Mb/s grant met a cheap scene".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-31 21:21:10 +02:00
co-authored by Claude Fable 5
parent 5926306a4c
commit 74863c96b3
12 changed files with 258 additions and 40 deletions
+8 -3
View File
@@ -248,9 +248,14 @@ mod session_main {
// 4:4:4 is opt-in and off by default (Settings "Full chroma"): the bit only says
// "upgrade me if you can" — the host still gates on its own policy, its capturer,
// HEVC, and a real GPU 4:4:4 encode probe, and answers the resolved chroma in the
// Welcome BEFORE we build a decoder. Advertised unconditionally when the user asks
// for it because every decode path here can produce it: the hardware ones where the
// driver decodes RExt, and swscale for the rest (the decoder demotes on its own).
// Welcome BEFORE we build a decoder. Advertised whenever the user asks because
// every path can DISPLAY it: the Vulkan presenter samples the 2-plane 4:4:4 pool
// formats (hardware RExt decode where the driver offers it — NVIDIA today) and
// swscale converts anything else for the software rung, with the decoder ladder
// demoting on its own. No capability probe gates the bit — software decode is the
// guaranteed floor — but the cost is VISIBLE, not silent: the Detailed stats
// overlay prints the resolved chroma ("4:4:4→4:2:0" when the host declined) and
// the decode path frames actually took.
video_caps: punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE
| if settings.hdr_enabled {
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
+25
View File
@@ -124,6 +124,21 @@ pub struct Stats {
/// The decode path frames actually took this window (`"vaapi"`/`"software"`, empty
/// until the first frame) — the OSD's trailing tag; tracks a mid-session fallback.
pub decoder: &'static str,
/// The encoder's CURRENT target bitrate (kbps): the Welcome resolve, then live per
/// `BitrateChanged` ack. What `mbps` (measured goodput) is judged AGAINST — a user
/// staring at "19 Mb/s" can't otherwise tell "the encoder is capped at 20" from "my
/// 200 Mb/s ask was honoured and this scene is cheap" (the gap that let the
/// settings-drop bug ship four releases). `0` = an old host that never reported one.
pub target_kbps: u32,
/// Automatic bitrate is armed (ABR moves `target_kbps` on its own) — the OSD tags the
/// target `(auto)` so a moving figure reads as policy, not a broken setting.
pub auto_rate: bool,
/// The host resolved full-chroma 4:4:4 for this session (`Welcome::chroma_format`).
pub chroma_444: bool,
/// This session ADVERTISED `VIDEO_CAP_444` (the Settings "Full chroma" opt-in): with
/// `chroma_444` false, the host declined — the OSD says so instead of leaving the
/// switch's effect unobservable.
pub asked_444: bool,
}
/// Frames the pump keeps waiting for their 0xCF host timing (pts → capture→received µs).
@@ -361,6 +376,12 @@ fn pump(
}
};
let force_software = params.force_software.clone();
// Session-constant stats facts (design/stats-unification.md): what the target figure is
// judged against and whether the 4:4:4 opt-in was honoured. `target_kbps` itself is read
// live per window — an Automatic session's ABR moves it.
let auto_rate = connector.wants_decode_latency();
let chroma_444 = connector.chroma_format == punktfunk_core::quic::CHROMA_IDC_444;
let asked_444 = params.video_caps & punktfunk_core::quic::VIDEO_CAP_444 != 0;
// Audio is best-effort: a session without it still streams. Gamepads are the
// app-lifetime service's job (the UI attaches it on Connected). Audio runs on its own
// thread (one puller per plane), blocking on the audio queue like the Apple client.
@@ -823,6 +844,10 @@ fn pump(
0.0
},
decoder: dec_path,
target_kbps: connector.current_bitrate_kbps(),
auto_rate,
chroma_444,
asked_444,
}));
window_start = Instant::now();
frames_n = 0;
+7
View File
@@ -950,6 +950,9 @@ pub(crate) fn drm_fourcc_for(sw: ffmpeg_next::ffi::AVPixelFormat) -> Option<u32>
Some(match sw {
AV_PIX_FMT_NV12 => fourcc(b'N', b'V', b'1', b'2'),
AV_PIX_FMT_P010LE => fourcc(b'P', b'0', b'1', b'0'),
// Full-chroma 4:4:4 semi-planar (HEVC RExt decode on drivers that export it as
// two planes) — the presenter imports the full-size chroma plane like any other.
AV_PIX_FMT_NV24 => fourcc(b'N', b'V', b'2', b'4'),
_ => return None,
})
}
@@ -1024,6 +1027,10 @@ mod tests {
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NV12),
Some(0x3231_564e)
);
assert_eq!(
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_NV24),
Some(0x3432_564e)
);
assert_eq!(
drm_fourcc_for(ffmpeg::ffi::AVPixelFormat::AV_PIX_FMT_RGBA),
None
+7 -1
View File
@@ -347,10 +347,16 @@ impl VulkanDecoder {
}
let fc = (*hwfc_ref).data as *mut ffi::AVHWFramesContext;
let sw = (*fc).sw_format;
// The 2-plane layouts the presenter's CSC can sample: 4:2:0 (NV12/P010) and
// full-chroma 4:4:4 (NV24/P410 — HEVC RExt decode, semi-planar like all
// NVDEC output). The presenter's `vkframe_plane_formats` table is the final
// authority; anything else bails here so the session demotes cleanly.
if sw != ffi::AVPixelFormat::AV_PIX_FMT_NV12
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_P010LE
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_NV24
&& sw != ffi::AVPixelFormat::AV_PIX_FMT_P410LE
{
bail!("Vulkan decode output {sw:?} unsupported (NV12/P010 only)");
bail!("Vulkan decode output {sw:?} unsupported (NV12/P010/NV24/P410 only)");
}
let vkfc = (*fc).hwctx as *const pf_ffvk::AVVulkanFramesContext;
let vk_format = (*vkfc).format[0] as i32;
+22 -16
View File
@@ -1,5 +1,5 @@
//! VAAPI dmabuf → Vulkan import: per-plane `VkImage`s (R8/GR88 for NV12, R16/GR1616
//! for 10-bit P010) with the
//! VAAPI dmabuf → Vulkan import: per-plane `VkImage`s (R8/GR88 for NV12 and full-chroma
//! NV24, R16/GR1616 for 10-bit P010) with the
//! surface's explicit DRM format modifier — the same layer-wise import the EGL presenter
//! (`video_gl.rs`) proved on this hardware, minus the toolkit. Same-Mesa export/import
//! is the contract; anything a driver rejects surfaces as a clean error and the caller
@@ -14,6 +14,13 @@ use std::os::fd::{BorrowedFd, IntoRawFd as _};
const DRM_FORMAT_NV12: u32 = 0x3231_564e;
/// `fourcc('P','0','1','0')` — 10-bit 4:2:0, 10 bits MSB-aligned in 16 (the HDR path).
const DRM_FORMAT_P010: u32 = 0x3031_3050;
/// `fourcc('N','V','2','4')` — 8-bit 4:4:4 semi-planar (full-size interleaved chroma
/// plane): the 2-plane full-chroma export a VAAPI HEVC RExt decode can hand over. Same
/// R8 + R8G8 views as NV12; the CSC shader keys chroma siting off the plane widths, so
/// the full-size plane needs nothing else. (Intel's iHD prefers PACKED 4:4:4 exports —
/// AYUV/Y410 — which are single-plane and would need their own CSC arm; those still
/// demote to software decode. 10-bit 4:4:4 has no settled dmabuf fourcc at all.)
const DRM_FORMAT_NV24: u32 = 0x3432_564e;
const DRM_FORMAT_MOD_INVALID: u64 = 0x00ff_ffff_ffff_ffff;
/// `DRM_FORMAT_MOD_LINEAR` — the fallback when the export carried no explicit modifier.
const DRM_FORMAT_MOD_LINEAR: u64 = 0;
@@ -92,13 +99,14 @@ pub fn import(
if std::env::var_os("PUNKTFUNK_HW_FAULT").is_some_and(|v| v == "import") {
bail!("injected import failure (PUNKTFUNK_HW_FAULT=import)");
}
let (luma_fmt, chroma_fmt) = match frame.fourcc {
DRM_FORMAT_NV12 => (vk::Format::R8_UNORM, vk::Format::R8G8_UNORM),
DRM_FORMAT_P010 => (vk::Format::R16_UNORM, vk::Format::R16G16_UNORM),
other => bail!("hw presenter handles NV12/P010 only (got {other:#x})"),
let (luma_fmt, chroma_fmt, chroma_full_res) = match frame.fourcc {
DRM_FORMAT_NV12 => (vk::Format::R8_UNORM, vk::Format::R8G8_UNORM, false),
DRM_FORMAT_P010 => (vk::Format::R16_UNORM, vk::Format::R16G16_UNORM, false),
DRM_FORMAT_NV24 => (vk::Format::R8_UNORM, vk::Format::R8G8_UNORM, true),
other => bail!("hw presenter handles NV12/P010/NV24 only (got {other:#x})"),
};
if frame.planes.len() < 2 {
bail!("2-plane 4:2:0 needs 2 planes (got {})", frame.planes.len());
bail!("2-plane YCbCr needs 2 planes (got {})", frame.planes.len());
}
// EGL could leave an INVALID modifier to the driver's implied choice; explicit-
// modifier images can't — LINEAR is the only honest guess (debug-visible if wrong).
@@ -123,16 +131,14 @@ pub fn import(
modifier,
)
.context("luma plane")?;
// 4:2:0 subsamples the chroma plane both ways; 4:4:4 (NV24) keeps it full-size.
let (cw, ch) = if chroma_full_res {
(frame.width, frame.height)
} else {
(frame.width.div_ceil(2), frame.height.div_ceil(2))
};
let (chroma_img, chroma_mem) = match plane_image(
device,
ext_mem_fd,
frame.width.div_ceil(2),
frame.height.div_ceil(2),
chroma_fmt,
c.fd,
c.offset,
c.stride,
modifier,
device, ext_mem_fd, cw, ch, chroma_fmt, c.fd, c.offset, c.stride, modifier,
)
.context("chroma plane")
{
+62 -1
View File
@@ -1976,8 +1976,27 @@ fn stats_text(
}
let detailed = verbosity == StatsVerbosity::Detailed;
let mut text = if detailed {
// The encoder target next to the measured rate is the figure whose absence let the
// settings-drop bug ship four releases: "19 Mb/s" alone can't distinguish "the
// encoder is capped at 20" from "my 200 Mb/s grant met a cheap scene". `(auto)`
// marks an Automatic session — the ABR moves the target by design, so a shifting
// number reads as policy, not a broken setting. Omitted against an old host that
// never reported a rate.
let target = match (s.target_kbps, s.auto_rate) {
(0, _) => String::new(),
(t, true) => format!(" · target {:.0} Mb/s (auto)", f64::from(t) / 1000.0),
(t, false) => format!(" · target {:.0} Mb/s", f64::from(t) / 1000.0),
};
// The chroma tag mirrors the HDR tag's honesty: `4:4:4→4:2:0` = the session asked
// for full chroma and the host resolved 4:2:0 (its policy/capturer/encoder gates
// said no) — otherwise the Settings switch's effect is unobservable.
let chroma = match (s.asked_444, s.chroma_444) {
(_, true) => " · 4:4:4",
(true, false) => " · 4:4:4→4:2:0",
_ => "",
};
format!(
"{mode_line} · {:.0} fps · {:.1} Mb/s · {}{}",
"{mode_line} · {:.0} fps · {:.1} Mb/s{target} · {}{}{chroma}",
s.fps,
s.mbps,
if s.decoder.is_empty() { "-" } else { s.decoder },
@@ -2263,6 +2282,12 @@ mod tests {
lost: 3,
lost_pct: 0.4,
decoder: "vulkan",
// Old-host baseline (no reported target, 4:2:0 never asked): the tier
// texts stay exactly what they were before the target/chroma elements.
target_kbps: 0,
auto_rate: false,
chroma_444: false,
asked_444: false,
},
PresentedWindow {
e2e_p50_ms: 6.4,
@@ -2303,6 +2328,42 @@ mod tests {
);
}
/// Detailed shows the negotiated encoder target next to the measured rate — the
/// figure whose absence let the settings-drop bug ship four releases — tagged
/// `(auto)` when the ABR owns it, plus the honest chroma tag when 4:4:4 was asked.
#[test]
fn detailed_shows_target_and_chroma_resolution() {
let (mut s, p) = sample();
let line1 = |s: &Stats, v| {
stats_text(v, "m", s, &p, false, false, None)
.lines()
.next()
.unwrap()
.to_string()
};
// Explicit 200 Mb/s honoured, cheap scene: measured AND target both show — the
// exact pair a user needs to tell a capped encoder from an idle one.
s.target_kbps = 200_000;
assert!(line1(&s, StatsVerbosity::Detailed).contains("24.3 Mb/s · target 200 Mb/s · "));
// An Automatic session's moving target reads as policy, not a broken setting.
(s.target_kbps, s.auto_rate) = (20_000, true);
assert!(line1(&s, StatsVerbosity::Detailed).contains("target 20 Mb/s (auto)"));
// Normal keeps its old line — the target is a Detailed element.
assert!(!line1(&s, StatsVerbosity::Normal).contains("target"));
// An old host that never reported a rate shows no target element at all.
s.target_kbps = 0;
assert!(!line1(&s, StatsVerbosity::Detailed).contains("target"));
// 4:4:4 asked and granted…
(s.asked_444, s.chroma_444) = (true, true);
assert!(line1(&s, StatsVerbosity::Detailed).ends_with("· 4:4:4"));
// …vs asked and declined: the downgrade is said out loud, mirroring `HDR→SDR`.
s.chroma_444 = false;
assert!(line1(&s, StatsVerbosity::Detailed).ends_with("· 4:4:4→4:2:0"));
// Unasked stays untagged (4:2:0 is the default — not noise worth a tag).
s.asked_444 = false;
assert!(!line1(&s, StatsVerbosity::Detailed).contains("4:4:4"));
}
/// Compact omits the latency term until the presenter's first e2e window lands.
#[test]
fn compact_waits_for_e2e() {
+73 -11
View File
@@ -815,19 +815,12 @@ impl Presenter {
/// Per-plane views over a Vulkan-Video frame's multiplanar image — the CSC pass's
/// exact sampling contract (the frames pool was created MUTABLE_FORMAT for this).
/// 8-bit NV12 (R8 + R8G8) and 10-bit P010/X6 (R10X6 + R10X6G10X6).
/// See [`vkframe_plane_formats`] for the accepted pool formats.
fn vkframe_plane_views(&self, f: &VkVideoFrame) -> Result<[vk::ImageView; 2]> {
let (luma_fmt, chroma_fmt) = if f.vk_format == vk::Format::G8_B8R8_2PLANE_420_UNORM.as_raw()
{
(vk::Format::R8_UNORM, vk::Format::R8G8_UNORM)
} else if f.vk_format == vk::Format::G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16.as_raw() {
(
vk::Format::R10X6_UNORM_PACK16,
vk::Format::R10X6G10X6_UNORM_2PACK16,
)
} else {
let Some((luma_fmt, chroma_fmt)) = vkframe_plane_formats(f.vk_format) else {
bail!(
"Vulkan-Video pool format {} unsupported (expected 2-plane 4:2:0, 8/10-bit)",
"Vulkan-Video pool format {} unsupported (expected 2-plane 4:2:0 or 4:4:4, \
8/10-bit 3-plane layouts need a third CSC binding)",
f.vk_format
);
};
@@ -875,6 +868,36 @@ impl Presenter {
}
}
/// The (luma, chroma) per-plane view formats for a Vulkan-Video pool format, or `None`
/// when this presenter can't sample it (the caller bails; the decoder demotes to
/// software — never a black screen).
///
/// The decision table IS the desktop 4:4:4 display contract, so it's a pure function
/// with a test pinning it:
/// - 2-plane 4:2:0, 8-bit (NV12-layout) and 10-bit (P010/X6) — the classic pair.
/// - 2-plane 4:4:4, 8- and 10-bit — what NVIDIA's Vulkan Video reports for HEVC RExt
/// full-chroma decode (semi-planar, like all NVDEC output). The CSC shader already
/// handles the full-size chroma plane (its 4:2:0 siting correction self-disables when
/// the plane widths match), so accepting the format here is all hardware 4:4:4 needs.
/// - 3-plane 4:4:4 stays rejected: the CSC pass samples exactly two planes (luma +
/// interleaved chroma); a triplanar pool needs a third binding + shader variant. No
/// supported driver reports it for HEVC decode today — revisit when one does.
fn vkframe_plane_formats(raw: i32) -> Option<(vk::Format, vk::Format)> {
let eight = (vk::Format::R8_UNORM, vk::Format::R8G8_UNORM);
let ten = (
vk::Format::R10X6_UNORM_PACK16,
vk::Format::R10X6G10X6_UNORM_2PACK16,
);
[
(vk::Format::G8_B8R8_2PLANE_420_UNORM, eight),
(vk::Format::G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16, ten),
(vk::Format::G8_B8R8_2PLANE_444_UNORM, eight),
(vk::Format::G10X6_B10X6R10X6_2PLANE_444_UNORM_3PACK16, ten),
]
.into_iter()
.find_map(|(f, planes)| (f.as_raw() == raw).then_some(planes))
}
/// Flatten the 3×vec4 rows for the push-constant block.
fn bytemuck_rows(rows: &[[f32; 4]; 3]) -> &[f32] {
// SAFETY: [[f32;4];3] is 12 contiguous f32s.
@@ -934,3 +957,42 @@ fn unlock_vkframe(f: &VkVideoFrame, sync: &VkFrameSync, submitted: bool, graphic
unlock(f.frames_ctx as *mut pf_ffvk::AVHWFramesContext, vkf);
}
}
#[cfg(test)]
mod tests {
use super::*;
/// The pool-format decision table (what this presenter can sample → what stays on the
/// hardware path, everything else demotes to software decode). Pinned so a format
/// added or dropped here is a deliberate act, not a drive-by.
#[test]
fn vkframe_pool_format_decision_table() {
let eight = Some((vk::Format::R8_UNORM, vk::Format::R8G8_UNORM));
let ten = Some((
vk::Format::R10X6_UNORM_PACK16,
vk::Format::R10X6G10X6_UNORM_2PACK16,
));
// 2-plane 4:2:0, both depths — the classic pair.
let f = |fmt: vk::Format| vkframe_plane_formats(fmt.as_raw());
assert_eq!(f(vk::Format::G8_B8R8_2PLANE_420_UNORM), eight);
assert_eq!(
f(vk::Format::G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16),
ten
);
// 2-plane 4:4:4, both depths — hardware full-chroma (NVIDIA RExt decode). Same
// per-plane view formats; the full-size chroma plane is the shader's business.
assert_eq!(f(vk::Format::G8_B8R8_2PLANE_444_UNORM), eight);
assert_eq!(
f(vk::Format::G10X6_B10X6R10X6_2PLANE_444_UNORM_3PACK16),
ten
);
// 3-plane 4:4:4 and 2-plane 4:2:2: real formats a driver could report, NOT
// sampleable by the two-binding CSC — they must demote, not corrupt.
assert_eq!(f(vk::Format::G8_B8_R8_3PLANE_444_UNORM), None);
assert_eq!(f(vk::Format::G8_B8R8_2PLANE_422_UNORM), None);
assert_eq!(f(vk::Format::G16_B16R16_2PLANE_444_UNORM), None);
// Garbage never maps.
assert_eq!(vkframe_plane_formats(0), None);
assert_eq!(vkframe_plane_formats(-1), None);
}
}
+20
View File
@@ -166,6 +166,12 @@ pub struct NativeClient {
/// drained per window by the data-plane pump to feed the adaptive-bitrate controller's decode
/// signal. Shared with the pump; see [`DecodeLatAcc`].
decode_lat: Arc<Mutex<DecodeLatAcc>>,
/// The encoder's CURRENT target bitrate (kbps): seeded with the Welcome resolve, then updated
/// by every host `BitrateChanged` ack (the ABR's re-targets, host-side clamps). Where
/// [`resolved_bitrate_kbps`](Self::resolved_bitrate_kbps) is the session-start negotiation
/// frozen for the ABI, this one moves — it's what a stats HUD should print as "target".
/// `0` = an old host that never reported a rate.
live_bitrate_kbps: Arc<AtomicU32>,
/// Whether the adaptive-bitrate controller is armed for this session (Automatic bitrate and not
/// a rate-pinned PyroWave stream) — exposed via [`wants_decode_latency`](Self::wants_decode_latency)
/// so an embedder skips the per-frame decode measurement when the controller wouldn't use it.
@@ -399,6 +405,8 @@ impl NativeClient {
let hot_tids = Arc::new(Mutex::new(Vec::new()));
let clock_offset = Arc::new(AtomicI64::new(0));
let decode_lat = Arc::new(Mutex::new(DecodeLatAcc::default()));
// Seeded by the pump from the Welcome (before ready_tx), then follows every ack.
let live_bitrate = Arc::new(AtomicU32::new(0));
let host = host.to_string();
let frame_chan_w = frame_chan.clone();
@@ -411,6 +419,7 @@ impl NativeClient {
let hot_tids_w = hot_tids.clone();
let clock_offset_w = clock_offset.clone();
let decode_lat_w = decode_lat.clone();
let live_bitrate_w = live_bitrate.clone();
let ctrl_tx_pump = ctrl_tx.clone(); // the data-plane pump sends adaptive-FEC LossReports
let worker = std::thread::Builder::new()
.name("punktfunk-client".into())
@@ -475,6 +484,7 @@ impl NativeClient {
hot_tids: hot_tids_w,
clock_offset: clock_offset_w,
decode_lat: decode_lat_w,
live_bitrate: live_bitrate_w,
}));
})
.map_err(PunktfunkError::Io)?;
@@ -523,6 +533,7 @@ impl NativeClient {
hot_tids,
clock_offset,
decode_lat,
live_bitrate_kbps: live_bitrate,
// The controller arms exactly when the pump does — all three terms, not two: Automatic
// (the user asked for bitrate 0), not a rate-pinned PyroWave stream, AND the host
// echoed the rate it actually configured. Dropping the last term made this
@@ -780,6 +791,15 @@ impl NativeClient {
self.wants_decode
}
/// The encoder's CURRENT target bitrate (kbps): the Welcome-resolved rate, live-updated by
/// every host `BitrateChanged` ack — an Automatic session's ABR re-targets and the host's
/// own clamps included. This is the figure a stats HUD should show as "target" next to
/// measured throughput (the [`resolved_bitrate_kbps`](Self::resolved_bitrate_kbps) field
/// stays the frozen session-start value). `0` = an old host that never reported one.
pub fn current_bitrate_kbps(&self) -> u32 {
self.live_bitrate_kbps.load(Ordering::Relaxed)
}
/// Report this client's display-latch grid so the host can phase-lock its capture tick
/// (design/phase-locked-capture.md; the vsync-aware presenters call this ~1 Hz).
/// `next_latch_host_ns` must already be HOST clock — convert with
+5
View File
@@ -71,6 +71,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
hot_tids,
clock_offset,
decode_lat,
live_bitrate,
..
} = args;
// Copies the pump needs after `negotiated` is handed over to `connect`.
@@ -80,6 +81,9 @@ pub(super) async fn run_pump(args: WorkerArgs) {
// Seed the live offset with the connect-time estimate BEFORE the embedder can observe the
// client (ready_tx): clock_offset_now_ns() never reads a pre-handshake 0 on a skewed pair.
clock_offset.store(negotiated.clock_offset_ns, Ordering::Relaxed);
// Same discipline for the live encoder target: the Welcome resolve is the starting truth
// (0 against an old host that reports none); every BitrateChanged ack moves it from there.
live_bitrate.store(negotiated.bitrate_kbps, Ordering::Relaxed);
// Bumped by the control task each time a re-sync batch is APPLIED; the pump watches it to
// reset its staleness counters and re-arm the clock-based jump-to-live detector.
let clock_gen = Arc::new(AtomicU32::new(0));
@@ -134,6 +138,7 @@ pub(super) async fn run_pump(args: WorkerArgs) {
mode_slot,
probe: probe.clone(),
bitrate_ack: bitrate_ack.clone(),
live_bitrate,
recovery_kf: recovery_kf.clone(),
clock_offset: clock_offset.clone(),
clock_gen: clock_gen.clone(),
@@ -16,6 +16,9 @@ pub(super) struct ControlTask {
pub(super) probe: Arc<Mutex<ProbeState>>,
/// The latest host `BitrateChanged` ack, drained by the pump's ABR on its report tick.
pub(super) bitrate_ack: Arc<Mutex<Option<u32>>>,
/// The live encoder-target mirror ([`NativeClient::current_bitrate_kbps`]): unlike the
/// drain-once ack slot above, this one always holds the latest acked rate for stats HUDs.
pub(super) live_bitrate: Arc<AtomicU32>,
/// Outbound decode-recovery KEYFRAME asks, counted here because this is the one choke point
/// every emitter funnels through (embedder, `note_frame_index`, the pump's own asks) — the
/// pump drains the count per report window as the ABR's recovery signal.
@@ -44,6 +47,7 @@ impl ControlTask {
mode_slot,
probe,
bitrate_ack,
live_bitrate,
recovery_kf,
clock_offset,
clock_gen,
@@ -157,6 +161,11 @@ impl ControlTask {
kbps = ack.bitrate_kbps,
"host re-targeted encoder bitrate"
);
// 0 would be a nonsense ack (the controller ignores it too) — don't
// let it wipe the HUD's live target.
if ack.bitrate_kbps > 0 {
live_bitrate.store(ack.bitrate_kbps, Ordering::Relaxed);
}
*bitrate_ack.lock().unwrap() = Some(ack.bitrate_kbps);
} else if let Ok(echo) = ClockEcho::decode(&msg) {
match resync.on_echo(&echo, wall_clock_ns()) {
+4 -1
View File
@@ -6,7 +6,7 @@ use crate::config::{CompositorPref, GamepadPref, Mode};
use crate::error::Result;
use crate::input::InputEvent;
use crate::quic::{HdrMeta, HidOutput};
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU64};
use std::sync::atomic::{AtomicBool, AtomicI64, AtomicU32, AtomicU64};
use std::sync::mpsc::SyncSender;
use std::sync::{Arc, Mutex};
@@ -73,6 +73,9 @@ pub(crate) struct WorkerArgs {
/// Decode-stage latency samples from the embedder (see [`NativeClient::decode_lat`]): the pump
/// drains a window mean into the adaptive-bitrate controller's decode signal.
pub(crate) decode_lat: Arc<Mutex<DecodeLatAcc>>,
/// The live encoder-target mirror (see [`NativeClient::live_bitrate_kbps`]): the worker seeds
/// it from the Welcome; the control task updates it on every `BitrateChanged` ack.
pub(crate) live_bitrate: Arc<AtomicU32>,
}
/// The worker: QUIC handshake, then the input/datagram/control tasks + the blocking
+16 -7
View File
@@ -45,8 +45,9 @@ captured input, switch mouse mode, disconnect — are in
**Compact** is a one-line pill (fps · end-to-end ms · Mb/s, plus a loss flag when frames are being
lost). **Normal** adds the stream line and the p50/p95 headline. **Detailed** adds the per-stage
breakdown everywhere; on Linux/Windows it also adds the decode path and an HDR tag, on Android the
decoder plus the full codec/bit-depth/colour line, and on iOS/tvOS the excluded OS present floor.
breakdown everywhere; on Linux/Windows it also adds the encoder's target bitrate, the decode path,
an HDR tag and a chroma tag, on Android the decoder plus the full codec/bit-depth/colour line, and
on iOS/tvOS the excluded OS present floor.
You can also set the level a stream starts at in each client's
[Settings](/docs/client-settings#overlay). The examples below are the **Detailed** view.
@@ -60,7 +61,7 @@ Every client reports the same measurements, but each family lays them out a litt
differently. Linux · Windows · Steam Deck:
```
1920×1080@120 · 120 fps · 24.3 Mb/s · vulkan · HDR
1920×1080@120 · 120 fps · 24.3 Mb/s · target 30 Mb/s (auto) · vulkan · HDR
e2e 14.2/19.8 ms (p50/p95) · host 3.1 · net 6.7 · decode 2.1 · display 2.3 ms
host: queue 0.6 · encode 1.8 · xfer 0.2 · pace 0.5 ms
lost 3 (2.4%)
@@ -90,10 +91,18 @@ lost 3 (2.4%)
```
- **Line 1 — the stream.** Resolution@refresh, frames received per second, and the
received video bitrate (goodput — FEC overhead not counted). Linux/Windows append the
decode path and an [HDR](/docs/hdr) tag (`HDR`, or `HDR→SDR` when a PQ stream is tone-mapped onto
an SDR screen); Android puts its decoder and the negotiated codec, bit depth, colour and
chroma on rows of their own underneath; the Apple clients don't report a codec at all.
received video bitrate (goodput — FEC overhead not counted). Linux/Windows follow the
measured rate with `target N Mb/s` — what the host's encoder is currently *allowed* to
produce — so a quiet desktop under a large grant (measured far below target) reads
differently from an encoder pinned at its cap (measured hugging the target). `(auto)`
means the [Automatic bitrate](/docs/client-settings#bitrate) controller owns the target
and moves it with network conditions; no target at all means an older host that doesn't
report one. Then the decode path, an [HDR](/docs/hdr) tag (`HDR`, or `HDR→SDR` when a PQ
stream is tone-mapped onto an SDR screen), and — when you asked for
[full chroma](/docs/client-settings) — the resolved chroma: `4:4:4` when the host
granted it, `4:4:4→4:2:0` when it couldn't. Android puts its decoder and the negotiated
codec, bit depth, colour and chroma on rows of their own underneath; the Apple clients
don't report a codec at all.
If the session resolved to a [settings profile](/docs/profiles-and-links), its name closes this
line. On **Android** a `⚠ panel NN Hz` warning joins it whenever the device's panel is refreshing
*below* the stream's rate — the tell for a phone or TV governor that ignored the requested mode,