fix(host): IDD capture immune to display-config stalls/flaps + metronomic-recovery self-diagnosis
ci / docs-site (push) Successful in 1m7s
ci / web (push) Successful in 1m8s
decky / build-publish (push) Successful in 32s
android / android (push) Failing after 2m38s
windows-host / package (push) Failing after 11m13s
ci / rust (push) Successful in 22m2s
ci / bench (push) Successful in 5m36s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10m9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12m15s
deb / build-publish (push) Successful in 15m40s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 12m41s
apple / swift (push) Successful in 1m12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 21s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
arch / build-publish (push) Successful in 14m26s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 12m11s
apple / screenshots (push) Successful in 5m30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12m52s
docker / deploy-docs (push) Failing after 6m14s
ci / docs-site (push) Successful in 1m7s
ci / web (push) Successful in 1m8s
decky / build-publish (push) Successful in 32s
android / android (push) Failing after 2m38s
windows-host / package (push) Failing after 11m13s
ci / rust (push) Successful in 22m2s
ci / bench (push) Successful in 5m36s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 10m9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 12m15s
deb / build-publish (push) Successful in 15m40s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 12m41s
apple / swift (push) Successful in 1m12s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 21s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 12s
arch / build-publish (push) Successful in 14m26s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 12m11s
apple / screenshots (push) Successful in 5m30s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 12m52s
docker / deploy-docs (push) Failing after 6m14s
Field report (0.8.2→0.8.4): metronomic stutter every ~3.7-4 s, each event a double-jolt, on ALL clients — and the same box stuttered on Apollo (virtual display) but not Sunshine (physical). The disturbance class is host-side display-topology events / display-poller software (SteelSeries-GG class, Apollo #480/#358/#368) hitting the virtual-display path. Three changes make the IDD path resilient to it and self-diagnosing: - DescriptorPoller: the display-descriptor CCD queries (QueryDisplayConfig twice per sample) serialize on the session-global display-config lock, which topology churn and display-poller software can hold for tens-to- hundreds of ms. Polled inline, that stall landed ON the capture/encode thread 4x/s — a periodic frame hitch, invisible in any log. A dedicated thread now samples every 250 ms and publishes a snapshot; the capture loop pays one uncontended mutex read, and a sample slower than 50 ms logs a rate-limited warn — the stall is measured instead of streamed. - Flap immunity: advanced_color_enabled returns Option (None on a failed query or the target briefly missing from the active paths during a re-probe) and the poller keeps last-known-good per field — a transient no longer reads as "HDR off", which used to cost an HDR session TWO spurious ring recreates (false, then true again a poll later). On top, a two-strikes debounce: only two consecutive samples agreeing on the same new descriptor trigger the recreate; a real "Use HDR" flip or game mode-set still lands in ~1/2 s. - RecoveryCadence: served recovery IDRs within 1.5 s coalesce into one event (a double-jolt's paired IDRs count once); four consecutive events evenly spaced (±20%) warn "client keyframe recoveries are METRONOMIC — a periodic host/display disturbance, not random network loss" with the measured period (re-warn ≤ every 30 s) — so an affected host self-diagnoses in its own log / web console instead of reporting "nothing in the logs". Unit tests include the field report's exact double-jolt-pair schedule. Verified: host tests + clippy clean on Linux and on the RTX box (211/211, --features nvenc). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -3163,6 +3163,82 @@ struct SessionContext {
|
||||
launch: Option<String>,
|
||||
}
|
||||
|
||||
/// Detector for METRONOMIC client keyframe-recovery cycles — the "periodic double-jolt" symptom
|
||||
/// class field reports keep describing: a host/display-side disturbance repeating every few
|
||||
/// seconds (display-topology churn, display-poller software, virtual-display timing), where each
|
||||
/// cycle ends in a client keyframe request the host serves. Random network loss is bursty and
|
||||
/// irregular; a stable period is a machine, and saying so in the host log turns a "nothing in the
|
||||
/// logs :/" report into a self-diagnosis.
|
||||
///
|
||||
/// Served forced IDRs within [`Self::COALESCE`] count as ONE event (a double-jolt's paired IDRs —
|
||||
/// the cooldown re-issue of a lost keyframe — are one user-visible disturbance). When the gaps
|
||||
/// between the last [`Self::STREAK`] events are all within ±[`Self::TOLERANCE`] of their mean,
|
||||
/// [`Self::note`] returns the mean period for the caller to warn with, then stays quiet for
|
||||
/// [`Self::REWARN`] while the cycle persists. Pure logic — unit-tested below.
|
||||
struct RecoveryCadence {
|
||||
events: std::collections::VecDeque<std::time::Instant>,
|
||||
last_warn: Option<std::time::Instant>,
|
||||
}
|
||||
|
||||
impl RecoveryCadence {
|
||||
/// Serves closer together than this are the same user-visible disturbance.
|
||||
const COALESCE: std::time::Duration = std::time::Duration::from_millis(1500);
|
||||
/// Consecutive evenly-spaced events before the cycle counts as metronomic.
|
||||
const STREAK: usize = 4;
|
||||
/// "Evenly spaced" = every gap within this fraction of the mean gap.
|
||||
const TOLERANCE: f64 = 0.2;
|
||||
/// Once warned, re-warn at most this often while the cycle persists.
|
||||
const REWARN: std::time::Duration = std::time::Duration::from_secs(30);
|
||||
|
||||
fn new() -> Self {
|
||||
Self {
|
||||
events: std::collections::VecDeque::new(),
|
||||
last_warn: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Record a served client-recovery IDR at `now`; `Some(mean period)` exactly when the
|
||||
/// metronomic-cycle warning should fire.
|
||||
fn note(&mut self, now: std::time::Instant) -> Option<std::time::Duration> {
|
||||
if self
|
||||
.events
|
||||
.back()
|
||||
.is_some_and(|last| now.duration_since(*last) < Self::COALESCE)
|
||||
{
|
||||
return None;
|
||||
}
|
||||
self.events.push_back(now);
|
||||
if self.events.len() > Self::STREAK {
|
||||
self.events.pop_front();
|
||||
}
|
||||
if self.events.len() < Self::STREAK {
|
||||
return None;
|
||||
}
|
||||
let gaps: Vec<f64> = self
|
||||
.events
|
||||
.iter()
|
||||
.zip(self.events.iter().skip(1))
|
||||
.map(|(a, b)| b.duration_since(*a).as_secs_f64())
|
||||
.collect();
|
||||
let mean = gaps.iter().sum::<f64>() / gaps.len() as f64;
|
||||
if mean <= 0.0
|
||||
|| gaps
|
||||
.iter()
|
||||
.any(|g| (g - mean).abs() > mean * Self::TOLERANCE)
|
||||
{
|
||||
return None;
|
||||
}
|
||||
if self
|
||||
.last_warn
|
||||
.is_some_and(|t| now.duration_since(t) < Self::REWARN)
|
||||
{
|
||||
return None;
|
||||
}
|
||||
self.last_warn = Some(now);
|
||||
Some(std::time::Duration::from_secs_f64(mean))
|
||||
}
|
||||
}
|
||||
|
||||
fn virtual_stream(ctx: SessionContext) -> Result<()> {
|
||||
// This thread runs the capture+encode loop (single-process — the only topology: Linux portal /
|
||||
// synthetic, Windows in-process IDD-push). Elevate it so a CPU-heavy game can't deschedule our GPU
|
||||
@@ -3377,6 +3453,9 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
|
||||
// clock now — that coalesces the keyframe storm a client fires while its decoder wedges on the cold
|
||||
// opening GOP, instead of answering it with a redundant second IDR.
|
||||
let mut last_forced_idr: Option<std::time::Instant> = Some(std::time::Instant::now());
|
||||
// Self-diagnosis for the periodic-stutter class: warns when the served recovery IDRs settle
|
||||
// into a stable multi-second rhythm (see [`RecoveryCadence`]).
|
||||
let mut recovery_cadence = RecoveryCadence::new();
|
||||
// Per-stage latency breakdown (PUNKTFUNK_PERF): per-call µs for the GPU-bound stages so we see
|
||||
// exactly where the capture→encoded latency goes — cap=try_latest (ring read + colour convert),
|
||||
// submit=encode_picture launch, wait=lock_bitstream (the scheduling wait + ASIC encode, the one
|
||||
@@ -3578,7 +3657,18 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
|
||||
} else {
|
||||
tracing::debug!("forcing keyframe (client decode recovery)");
|
||||
enc.request_keyframe();
|
||||
last_forced_idr = Some(std::time::Instant::now());
|
||||
let now = std::time::Instant::now();
|
||||
last_forced_idr = Some(now);
|
||||
if let Some(period) = recovery_cadence.note(now) {
|
||||
tracing::warn!(
|
||||
period_s = format!("{:.1}", period.as_secs_f64()),
|
||||
"client keyframe recoveries are METRONOMIC — a periodic host/display \
|
||||
disturbance (display-topology churn, display-poller software, \
|
||||
virtual-display timing) is the likely cause, not random network loss; \
|
||||
correlate with 'slow display-descriptor poll' / 'display descriptor \
|
||||
changed' lines"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Measure the per-stage split when `PUNKTFUNK_PERF` is set OR a web-console stats capture is
|
||||
@@ -4209,6 +4299,69 @@ fn build_pipeline(
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// Feed [`RecoveryCadence`] a schedule of event offsets (ms from a common origin) and return
|
||||
/// what each `note` produced.
|
||||
fn cadence_run(offsets_ms: &[u64]) -> Vec<Option<std::time::Duration>> {
|
||||
let base = std::time::Instant::now();
|
||||
let mut c = RecoveryCadence::new();
|
||||
offsets_ms
|
||||
.iter()
|
||||
.map(|ms| c.note(base + std::time::Duration::from_millis(*ms)))
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cadence_detects_metronomic_recoveries() {
|
||||
// Four IDR serves ~4 s apart (±5%) → the fourth trips the detector at ~4 s.
|
||||
let out = cadence_run(&[0, 4_000, 8_100, 11_950]);
|
||||
assert_eq!(out[..3], [None, None, None]);
|
||||
let period = out[3].expect("metronomic series must be detected");
|
||||
assert!(
|
||||
(period.as_secs_f64() - 3.98).abs() < 0.2,
|
||||
"period={period:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cadence_coalesces_double_jolt_pairs() {
|
||||
// The field signature: a jolt pair (second IDR ~0.7 s after the first, the cooldown
|
||||
// re-issue) every ~4 s. Each pair is ONE event; detection still lands on the ~4 s cycle.
|
||||
let out = cadence_run(&[
|
||||
0, 700, // pair 1
|
||||
4_000, 4_700, // pair 2
|
||||
8_000, 8_650, // pair 3
|
||||
12_000, // pair 4 (first serve trips it)
|
||||
]);
|
||||
assert!(out[..6].iter().all(Option::is_none));
|
||||
let period = out[6].expect("coalesced pairs must still read as a 4 s cycle");
|
||||
assert!(
|
||||
(period.as_secs_f64() - 4.0).abs() < 0.2,
|
||||
"period={period:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cadence_ignores_irregular_bursts() {
|
||||
// Genuine Wi-Fi-style loss: irregular gaps → never flagged.
|
||||
assert!(cadence_run(&[0, 2_000, 9_000, 12_500, 21_000])
|
||||
.iter()
|
||||
.all(Option::is_none));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cadence_rewarns_at_most_every_30s() {
|
||||
// A persisting 4 s cycle: warn on the 4th event (t=12 s), then stay quiet until ≥30 s
|
||||
// past the warn — the t=44 s event (index 11) is the first at or beyond t=42 s.
|
||||
let offsets: Vec<u64> = (0..12).map(|i| i * 4_000).collect();
|
||||
let out = cadence_run(&offsets);
|
||||
let warned: Vec<usize> = out
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter_map(|(i, o)| o.map(|_| i))
|
||||
.collect();
|
||||
assert_eq!(warned, vec![3, 11], "warn indices");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn adapt_fec_maps_loss_to_recovery_band() {
|
||||
// A perfectly clean window (0 loss) lands on the floor.
|
||||
|
||||
Reference in New Issue
Block a user