feat(core/host/android): phase-locked capture — frames arrive on the client's latch schedule
ci / web (push) Successful in 1m59s
ci / docs-site (push) Successful in 2m13s
ci / rust-arm64 (push) Successful in 5m17s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 5s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 4s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 4s
android / android (push) Successful in 5m52s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
deb / build-publish (push) Successful in 5m38s
ci / rust (push) Failing after 5m52s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 18s
arch / build-publish (push) Failing after 6m7s
deb / build-publish-client-arm64 (push) Successful in 3m55s
docker / builders-arm64cross (push) Successful in 19s
deb / build-publish-host (push) Failing after 4m24s
apple / swift (push) Successful in 4m35s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m13s
docker / deploy-docs (push) Successful in 31s
windows-host / package (push) Failing after 6m22s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m35s
flatpak / build-publish (push) Successful in 6m33s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m22s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 10m8s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 2m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 11m43s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 4m1s
release / apple (push) Successful in 25m5s
apple / screenshots (push) Successful in 20m50s
ci / web (push) Successful in 1m59s
ci / docs-site (push) Successful in 2m13s
ci / rust-arm64 (push) Successful in 5m17s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 5s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 4s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 4s
android / android (push) Successful in 5m52s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
deb / build-publish (push) Successful in 5m38s
ci / rust (push) Failing after 5m52s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 18s
arch / build-publish (push) Failing after 6m7s
deb / build-publish-client-arm64 (push) Successful in 3m55s
docker / builders-arm64cross (push) Successful in 19s
deb / build-publish-host (push) Failing after 4m24s
apple / swift (push) Successful in 4m35s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 1m13s
docker / deploy-docs (push) Successful in 31s
windows-host / package (push) Failing after 6m22s
windows-host / canary-manifest (push) Skipped
windows-host / winget-source (push) Skipped
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 2m35s
flatpak / build-publish (push) Successful in 6m33s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m22s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 10m8s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 2m56s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 11m43s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 4m1s
release / apple (push) Successful in 25m5s
apple / screenshots (push) Successful in 20m50s
The host's capture tick and a client's panel vsync are independent ~120 Hz oscillators; their drifting phase sweeps every frame's wait-for-latch across a full refresh period (measured on-glass: latch p50 oscillating 5.4-8.9 ms with a fixed margin) — the beat is the residual judder and the fat p95, and no client can fix it alone. Design: punktfunk-planning design/phase-locked-capture.md. Protocol (punktfunk-core): - PhaseReport (control 0x32, next to the clock family): the client's next display latch ALREADY CONVERTED to host clock (the skew offset lives only client-side), panel period, uncertainty, and the measured median arrival-lead — the controller's error signal. ~1 Hz, latest-wins. CLIENT_CAP_PHASE_LOCK advertises it; CtrlRequest::Phase + report_phase() + the C ABI mirror carry it. - The 0xCF host-timing tail grows a phase ACK (applied_phase_ns, 29-byte form) under the same strict-prefix append discipline — old readers parse the shorter forms; degradation pinned by tests. Host engine (arrival-slaved loop — no backend can move the source vsync, per the tick-ownership audit in the design doc): - PhaseCtl bridges control task → encode loop (the fec_target pattern, multi-field). PhaseController walks a per-frame HOLD before submit toward the client's reported lead hitting target = max(2.5 ms, uncertainty+1ms): 1 Hz adjust, 2 ms max step, 300 µs deadband, period-wrapping (the newest-wins capture slot makes a wrapped hold sample fresher content, not staler). A loop local, so every mid-stream rebuild keeps the lock; a new session re-acquires. PUNKTFUNK_PHASE_LOCK=0 disarms. Android reporter: the presenter's 1 Hz pf.present flush returns the window's measured latch p50; the async loop converts the vsync clock's next timeline monotonic→realtime→host and reports. Inert toward old hosts. Gates: docker amd64 clippy --all-targets -D warnings (host+core, nvenc) clean; core suite incl. the new wire tests; cargo ndk arm64 check/clippy clean. On-glass A/B vs the .173 host owed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -380,7 +380,29 @@ pub(super) fn run_async(
|
||||
if p.pump(&codec, clock, &tracker, &stats, now_monotonic_ns()) {
|
||||
rendered += 1;
|
||||
}
|
||||
p.flush_log(&meter, clock);
|
||||
// The 1 Hz window flush doubles as the phase-lock report tick: the measured latch
|
||||
// p50 is the arrival-lead error signal the host's capture controller drives toward
|
||||
// its target (design/phase-locked-capture.md §6). Timestamps convert
|
||||
// monotonic→realtime→host here — the skew offset lives only client-side.
|
||||
if let (Some(latch_p50_ns), Some(c)) = (p.flush_log(&meter, clock), clock) {
|
||||
let period = c.panel_period_ns().max(c.period_ns());
|
||||
if period > 0 {
|
||||
if let Some(t) = c.next_target(now_monotonic_ns(), 0) {
|
||||
let mono_now = now_monotonic_ns();
|
||||
let real_now = now_realtime_ns();
|
||||
let latch_real_ns = real_now + (t.expected_present_ns - mono_now) as i128;
|
||||
let latch_host_ns = (latch_real_ns
|
||||
+ clock_offset.load(Ordering::Relaxed) as i128)
|
||||
.max(0) as u64;
|
||||
client.report_phase(
|
||||
latch_host_ns,
|
||||
period.clamp(0, u32::MAX as i64) as u32,
|
||||
1_000_000, // skew residual + latch jitter — conservative 1 ms
|
||||
latch_p50_ns.min(u32::MAX as u64) as u32,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
let presented_now = rendered > rendered_before;
|
||||
// Start the vsync clock LAZILY on the first decoded output (eager, it ticks the panel
|
||||
|
||||
@@ -339,14 +339,21 @@ impl Presenter {
|
||||
/// `noBudget` (waits on the closed budget) / `forced` (stale force-opens — 0 when healthy) /
|
||||
/// `qDry` (FIFO underflows) / `pace` (decoded→release) / `latch` (release→displayed) /
|
||||
/// `vsync` (the measured panel period).
|
||||
pub(super) fn flush_log(&mut self, meter: &PresentMeter, clock: Option<&VsyncShared>) {
|
||||
///
|
||||
/// Returns this window's measured latch p50 (ns) when a window actually flushed — the
|
||||
/// phase-lock reporter's `arrival_lead` error signal (design/phase-locked-capture.md §6).
|
||||
pub(super) fn flush_log(
|
||||
&mut self,
|
||||
meter: &PresentMeter,
|
||||
clock: Option<&VsyncShared>,
|
||||
) -> Option<u64> {
|
||||
if self.last_flush.elapsed() < std::time::Duration::from_secs(1) {
|
||||
return;
|
||||
return None;
|
||||
}
|
||||
self.last_flush = Instant::now();
|
||||
let (latch, displays) = meter.drain();
|
||||
if self.released == 0 && displays == 0 {
|
||||
return; // idle stream — nothing worth a line
|
||||
return None; // idle stream — nothing worth a line
|
||||
}
|
||||
let (pace_p50, pace_max) = p50_max_ms(std::mem::take(&mut self.pace_us));
|
||||
let (latch_p50, latch_max) = p50_max_ms(latch);
|
||||
@@ -376,6 +383,7 @@ impl Presenter {
|
||||
self.no_budget = 0;
|
||||
self.forced = 0;
|
||||
self.dry = 0;
|
||||
(latch_p50 > 0.0).then(|| (latch_p50 * 1e6) as u64)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -4013,6 +4013,39 @@ pub unsafe extern "C" fn punktfunk_connection_report_decode_us(
|
||||
})
|
||||
}
|
||||
|
||||
/// Report this client's display-latch grid so the host can phase-lock its capture tick
|
||||
/// (design/phase-locked-capture.md). `next_latch_host_ns` must already be host clock — convert
|
||||
/// with the connection's clock offset (`T_host = T_client + offset`). Fire-and-forget; call ~1 Hz
|
||||
/// from a vsync-aware presenter. No-op toward a host that never negotiated the capability.
|
||||
///
|
||||
/// # Safety
|
||||
/// `c` is an opaque handle from a `*_new`/`*_pair` the caller has not yet freed, or null (an
|
||||
/// error, not UB).
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn punktfunk_connection_report_phase(
|
||||
c: *const PunktfunkConnection,
|
||||
next_latch_host_ns: u64,
|
||||
latch_period_ns: u32,
|
||||
uncertainty_ns: u32,
|
||||
arrival_lead_ns: u32,
|
||||
) -> PunktfunkStatus {
|
||||
guard(|| {
|
||||
// SAFETY: per the ABI contract - an opaque handle from a `*_new`/`*_pair` that the caller
|
||||
// has not yet freed, or null, which `as_ref` reports as `None` and the `match` handles.
|
||||
let c = match unsafe { c.as_ref() } {
|
||||
Some(c) => c,
|
||||
None => return PunktfunkStatus::NullPointer,
|
||||
};
|
||||
c.inner.report_phase(
|
||||
next_latch_host_ns,
|
||||
latch_period_ns,
|
||||
uncertainty_ns,
|
||||
arrival_lead_ns,
|
||||
);
|
||||
PunktfunkStatus::Ok
|
||||
})
|
||||
}
|
||||
|
||||
/// Whether [`punktfunk_connection_report_decode_us`] is worth calling this session: writes 1 to
|
||||
/// `out` only when the adaptive-bitrate controller is armed (Automatic bitrate, non-PyroWave), so a
|
||||
/// client can skip the per-frame decode-latency measurement entirely for explicit-bitrate and
|
||||
|
||||
@@ -32,6 +32,10 @@ pub(crate) enum CtrlRequest {
|
||||
/// desktop mouse model — host excludes + forwards), `false` = host composites into the
|
||||
/// video (the capture model). Sent on every mouse-model flip; idempotent, latest-wins.
|
||||
CursorRender(crate::quic::CursorRenderMode),
|
||||
/// The client's display-latch grid, ~1 Hz, host-clock timestamps — feeds the host's
|
||||
/// phase-locked capture (design/phase-locked-capture.md). Latest-wins; an old host
|
||||
/// ignores the message.
|
||||
Phase(crate::quic::PhaseReport),
|
||||
}
|
||||
|
||||
/// What the worker reports to [`NativeClient::connect`] once the handshake lands: the
|
||||
|
||||
@@ -774,6 +774,30 @@ impl NativeClient {
|
||||
self.wants_decode
|
||||
}
|
||||
|
||||
/// Report this client's display-latch grid so the host can phase-lock its capture tick
|
||||
/// (design/phase-locked-capture.md; the vsync-aware presenters call this ~1 Hz).
|
||||
/// `next_latch_host_ns` must already be HOST clock — convert with
|
||||
/// [`clock_offset_now_ns`](Self::clock_offset_now_ns) (`T_host = T_client + offset`) before
|
||||
/// calling; the offset lives only on this side. Fire-and-forget: dropped silently if the
|
||||
/// control task's queue is momentarily full (the next report supersedes) or toward a host
|
||||
/// that never negotiated the capability.
|
||||
pub fn report_phase(
|
||||
&self,
|
||||
next_latch_host_ns: u64,
|
||||
latch_period_ns: u32,
|
||||
uncertainty_ns: u32,
|
||||
arrival_lead_ns: u32,
|
||||
) {
|
||||
let _ = self
|
||||
.ctrl_tx
|
||||
.try_send(CtrlRequest::Phase(crate::quic::PhaseReport {
|
||||
next_latch_host_ns,
|
||||
latch_period_ns,
|
||||
uncertainty_ns,
|
||||
arrival_lead_ns,
|
||||
}));
|
||||
}
|
||||
|
||||
/// Start a bandwidth speed test: ask the host to burst filler over the data plane at
|
||||
/// `target_kbps` of goodput for `duration_ms`, *briefly pausing video*. Non-blocking — the
|
||||
/// measurement accumulates in the background; poll [`NativeClient::probe_result`] until its
|
||||
|
||||
@@ -94,6 +94,7 @@ impl ControlTask {
|
||||
CtrlRequest::ClipControl(c) => c.encode(),
|
||||
CtrlRequest::ClipOffer(o) => o.encode(),
|
||||
CtrlRequest::CursorRender(m) => m.encode(),
|
||||
CtrlRequest::Phase(p) => p.encode(),
|
||||
};
|
||||
if io::write_msg(&mut ctrl_send, &bytes).await.is_err() {
|
||||
break;
|
||||
|
||||
@@ -104,6 +104,13 @@ pub const HOST_CAP_TEXT_INPUT: u8 = 0x04;
|
||||
/// an older or incapable host nothing changes.
|
||||
pub const CLIENT_CAP_CURSOR: u8 = 0x01;
|
||||
|
||||
/// `Hello.client_caps` bit: this client runs a vsync-aware presenter and will send
|
||||
/// [`PhaseReport`](super::control::PhaseReport)s (~1 Hz) so the host can phase-lock its
|
||||
/// capture/send tick to the client's display latch (design/phase-locked-capture.md). Without
|
||||
/// the bit the host never arms the phase controller; toward an older host the reports are
|
||||
/// simply ignored — no behavior change in either direction.
|
||||
pub const CLIENT_CAP_PHASE_LOCK: u8 = 0x02;
|
||||
|
||||
/// [`Welcome::host_caps`] bit: the host CAN forward the cursor out-of-band (it captures cursor
|
||||
/// metadata separately from the frame — the Linux portal `SPA_META_Cursor` path; NOT gamescope,
|
||||
/// whose capture carries no cursor, and NOT Windows yet, where DWM composites into the IDD
|
||||
|
||||
@@ -156,6 +156,30 @@ pub struct ClockEcho {
|
||||
pub t3_ns: u64,
|
||||
}
|
||||
|
||||
/// `client → host`, ~1 Hz: the client's display-latch grid, so the host can PHASE-LOCK its
|
||||
/// capture/send tick and land frames a constant, small margin before the client's vsync latch
|
||||
/// (design/phase-locked-capture.md). Sent only by clients with a vsync-aware presenter, gated on
|
||||
/// [`CLIENT_CAP_PHASE_LOCK`](crate::quic::CLIENT_CAP_PHASE_LOCK); an old host ignores it.
|
||||
///
|
||||
/// Timestamps are HOST clock (`CLOCK_REALTIME`): the client converts before sending
|
||||
/// (`T_host = T_client + offset` — the skew offset from the clock handshake lives only
|
||||
/// client-side, the host deliberately stores none).
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct PhaseReport {
|
||||
/// The client's next display latch, host clock. The host extrapolates forward by
|
||||
/// `latch_period_ns` — the report describes a grid, not one instant.
|
||||
pub next_latch_host_ns: u64,
|
||||
/// The client panel's refresh period (its true latch grid — not the app's possibly
|
||||
/// down-rated callback rate).
|
||||
pub latch_period_ns: u32,
|
||||
/// The client's own error bound on this grid (skew residual + latch jitter p95) — the host
|
||||
/// widens its arrival margin by this, never narrows below its floor.
|
||||
pub uncertainty_ns: u32,
|
||||
/// Measured median lead of frame ARRIVAL before latch over the last window (ns; clamped
|
||||
/// ≥ 0). The controller drives this toward its target lead — the error signal.
|
||||
pub arrival_lead_ns: u32,
|
||||
}
|
||||
|
||||
/// Type byte of [`Reconfigure`] (first byte after the magic).
|
||||
pub const MSG_RECONFIGURE: u8 = 0x01;
|
||||
/// Type byte of [`Reconfigured`].
|
||||
@@ -178,6 +202,8 @@ pub const MSG_PROBE_RESULT: u8 = 0x21;
|
||||
pub const MSG_CLOCK_PROBE: u8 = 0x30;
|
||||
/// Type byte of [`ClockEcho`].
|
||||
pub const MSG_CLOCK_ECHO: u8 = 0x31;
|
||||
/// Type byte of [`PhaseReport`].
|
||||
pub const MSG_PHASE_REPORT: u8 = 0x32;
|
||||
|
||||
impl Reconfigure {
|
||||
pub fn encode(&self) -> Vec<u8> {
|
||||
@@ -461,6 +487,33 @@ impl ClockEcho {
|
||||
}
|
||||
}
|
||||
|
||||
impl PhaseReport {
|
||||
pub fn encode(&self) -> Vec<u8> {
|
||||
// magic[0..4] type[4] latch[5..13] period[13..17] uncertainty[17..21] lead[21..25]
|
||||
let mut b = Vec::with_capacity(25);
|
||||
b.extend_from_slice(CTL_MAGIC);
|
||||
b.push(MSG_PHASE_REPORT);
|
||||
b.extend_from_slice(&self.next_latch_host_ns.to_le_bytes());
|
||||
b.extend_from_slice(&self.latch_period_ns.to_le_bytes());
|
||||
b.extend_from_slice(&self.uncertainty_ns.to_le_bytes());
|
||||
b.extend_from_slice(&self.arrival_lead_ns.to_le_bytes());
|
||||
b
|
||||
}
|
||||
|
||||
pub fn decode(b: &[u8]) -> Result<PhaseReport> {
|
||||
if b.len() != 25 || &b[0..4] != CTL_MAGIC || b[4] != MSG_PHASE_REPORT {
|
||||
return Err(PunktfunkError::InvalidArg("bad PhaseReport"));
|
||||
}
|
||||
let u32at = |o: usize| u32::from_le_bytes([b[o], b[o + 1], b[o + 2], b[o + 3]]);
|
||||
Ok(PhaseReport {
|
||||
next_latch_host_ns: u64::from_le_bytes(b[5..13].try_into().unwrap()),
|
||||
latch_period_ns: u32at(13),
|
||||
uncertainty_ns: u32at(17),
|
||||
arrival_lead_ns: u32at(21),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Frame a message for the control stream: `u16 LE length || payload`.
|
||||
pub fn frame(payload: &[u8]) -> Vec<u8> {
|
||||
let mut b = Vec::with_capacity(2 + payload.len());
|
||||
@@ -923,6 +976,21 @@ mod tests {
|
||||
.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn phase_report_roundtrip() {
|
||||
let pr = PhaseReport {
|
||||
next_latch_host_ns: 1_753_900_000_123_456_789,
|
||||
latch_period_ns: 8_333_333,
|
||||
uncertainty_ns: 900_000,
|
||||
arrival_lead_ns: 4_100_000,
|
||||
};
|
||||
assert_eq!(PhaseReport::decode(&pr.encode()).unwrap(), pr);
|
||||
// Wrong type byte (a ClockProbe) must not decode as a PhaseReport.
|
||||
assert!(PhaseReport::decode(&ClockProbe { t1_ns: 7 }.encode()).is_err());
|
||||
// Truncation must not decode.
|
||||
assert!(PhaseReport::decode(&pr.encode()[..24]).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reconfigure_roundtrip() {
|
||||
let rq = Reconfigure {
|
||||
|
||||
@@ -557,6 +557,11 @@ pub struct HostTiming {
|
||||
/// and takes the stage tail only when present), so no capability bit is needed in either
|
||||
/// direction: old client + new host reads the prefix, new client + old host gets `None`.
|
||||
pub stages: Option<HostStages>,
|
||||
/// Phase-lock ACK (design/phase-locked-capture.md): the capture-tick hold the host is
|
||||
/// currently applying, ns. Rides the same append-extensible tail (after the stages block):
|
||||
/// `None` from a pre-phase-lock host or one sending the shorter forms. The client's
|
||||
/// presenter compares it against its own requested correction to see the loop close.
|
||||
pub applied_phase_ns: Option<i32>,
|
||||
}
|
||||
|
||||
/// The extended 0xCF's per-stage split of [`HostTiming::host_us`], all µs against the same
|
||||
@@ -578,11 +583,15 @@ pub struct HostStages {
|
||||
const HOST_TIMING_LEN: usize = 1 + 8 + 4;
|
||||
/// Wire length with the [`HostStages`] tail appended: + 3 × u32 = 25 bytes.
|
||||
const HOST_TIMING_STAGES_LEN: usize = HOST_TIMING_LEN + 12;
|
||||
/// Wire length with the phase-lock ACK appended after the stages: + i32 = 29 bytes. The tail
|
||||
/// discipline holds: each form is a strict prefix of the next, so every reader takes what it
|
||||
/// knows and ignores the rest.
|
||||
const HOST_TIMING_PHASE_LEN: usize = HOST_TIMING_STAGES_LEN + 4;
|
||||
|
||||
/// Encode a [`HostTiming`] into a [`HOST_TIMING_MAGIC`] datagram (extended form when `stages`
|
||||
/// is set — an older client parses the prefix and ignores the tail).
|
||||
pub fn encode_host_timing_datagram(t: &HostTiming) -> Vec<u8> {
|
||||
let mut b = Vec::with_capacity(HOST_TIMING_STAGES_LEN);
|
||||
let mut b = Vec::with_capacity(HOST_TIMING_PHASE_LEN);
|
||||
b.push(HOST_TIMING_MAGIC);
|
||||
b.extend_from_slice(&t.pts_ns.to_le_bytes());
|
||||
b.extend_from_slice(&t.host_us.to_le_bytes());
|
||||
@@ -590,6 +599,11 @@ pub fn encode_host_timing_datagram(t: &HostTiming) -> Vec<u8> {
|
||||
b.extend_from_slice(&s.queue_us.to_le_bytes());
|
||||
b.extend_from_slice(&s.encode_us.to_le_bytes());
|
||||
b.extend_from_slice(&s.pace_us.to_le_bytes());
|
||||
// The phase ACK only ever rides AFTER a stages tail — a prefix-discipline wire can't
|
||||
// express "phase but no stages", and every host new enough to phase-lock sends stages.
|
||||
if let Some(p) = t.applied_phase_ns {
|
||||
b.extend_from_slice(&p.to_le_bytes());
|
||||
}
|
||||
}
|
||||
b
|
||||
}
|
||||
@@ -606,10 +620,13 @@ pub fn decode_host_timing_datagram(b: &[u8]) -> Option<HostTiming> {
|
||||
encode_us: u32::from_le_bytes(b[17..21].try_into().unwrap()),
|
||||
pace_us: u32::from_le_bytes(b[21..25].try_into().unwrap()),
|
||||
});
|
||||
let applied_phase_ns = (b.len() >= HOST_TIMING_PHASE_LEN)
|
||||
.then(|| i32::from_le_bytes(b[25..29].try_into().unwrap()));
|
||||
Some(HostTiming {
|
||||
pts_ns: u64::from_le_bytes(b[1..9].try_into().unwrap()),
|
||||
host_us: u32::from_le_bytes(b[9..13].try_into().unwrap()),
|
||||
stages,
|
||||
applied_phase_ns,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -715,6 +732,7 @@ mod tests {
|
||||
pts_ns: 1_751_500_000_123_456_789, // a realistic 2026 CLOCK_REALTIME capture stamp
|
||||
host_us: 4_321,
|
||||
stages: None,
|
||||
applied_phase_ns: None,
|
||||
};
|
||||
let d = encode_host_timing_datagram(&t);
|
||||
assert_eq!(d[0], HOST_TIMING_MAGIC);
|
||||
@@ -754,6 +772,35 @@ mod tests {
|
||||
"partial stage tail ({n} B) must degrade to the legacy decode"
|
||||
);
|
||||
}
|
||||
|
||||
// Phase-ACK form (design/phase-locked-capture.md): strict-prefix discipline holds a
|
||||
// third time — 29 B roundtrips, 25..28 degrade to the stages form, the prefix is
|
||||
// byte-identical, and a phase without stages is unencodable by construction.
|
||||
let tp = HostTiming {
|
||||
applied_phase_ns: Some(-2_750_000),
|
||||
..ts
|
||||
};
|
||||
let dp = encode_host_timing_datagram(&tp);
|
||||
assert_eq!(dp.len(), 29);
|
||||
assert_eq!(&dp[..25], &ds[..25], "stages form is a strict prefix");
|
||||
assert_eq!(decode_host_timing_datagram(&dp), Some(tp));
|
||||
for n in 25..dp.len() {
|
||||
assert_eq!(
|
||||
decode_host_timing_datagram(&dp[..n]),
|
||||
Some(ts),
|
||||
"partial phase tail ({n} B) must degrade to the stages decode"
|
||||
);
|
||||
}
|
||||
let no_stages = HostTiming {
|
||||
stages: None,
|
||||
applied_phase_ns: Some(1),
|
||||
..t
|
||||
};
|
||||
assert_eq!(
|
||||
encode_host_timing_datagram(&no_stages).len(),
|
||||
13,
|
||||
"phase without stages must encode as the legacy form (prefix discipline)"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -1095,6 +1095,11 @@ async fn serve_session(
|
||||
let adaptive_fec = fec_static_override().is_none();
|
||||
let fec_target = Arc::new(AtomicU8::new(welcome.fec.fec_percent));
|
||||
let fec_target_ctl = fec_target.clone();
|
||||
// Phase-locked capture bridge (design/phase-locked-capture.md): the control task stores the
|
||||
// client's PhaseReports here; the encode loop's controller drains them. Inert until a
|
||||
// vsync-aware client actually reports.
|
||||
let phase_ctl = Arc::new(stream::PhaseCtl::new());
|
||||
let phase_ctl_control = phase_ctl.clone();
|
||||
// The session's negotiated rate — the pin PyroWave retarget-refusals ack (§4.6).
|
||||
let session_bitrate_kbps = welcome.bitrate_kbps;
|
||||
// Shared-clipboard enable state (client `ClipControl` → host). The coordinator reads it to
|
||||
@@ -1120,6 +1125,7 @@ async fn serve_session(
|
||||
encoder_ceiling_kbps.clone(),
|
||||
cadence_degraded.clone(),
|
||||
fec_target_ctl,
|
||||
phase_ctl_control,
|
||||
reconfig_tx,
|
||||
keyframe_tx,
|
||||
rfi_tx,
|
||||
@@ -1568,6 +1574,7 @@ async fn serve_session(
|
||||
probe_result_tx,
|
||||
reconfig_result_tx,
|
||||
fec_target: fec_target_dp,
|
||||
phase: phase_ctl,
|
||||
conn: conn_stream,
|
||||
timing_conn,
|
||||
cursor_forward,
|
||||
|
||||
@@ -30,6 +30,9 @@ pub(super) async fn run(
|
||||
encoder_ceiling_kbps: Arc<AtomicU32>,
|
||||
cadence_degraded: Arc<AtomicBool>,
|
||||
fec_target_ctl: Arc<AtomicU8>,
|
||||
// Phase-locked capture bridge: client PhaseReports land here latest-wins; the encode loop's
|
||||
// controller drains at its own ~1 Hz cadence (design/phase-locked-capture.md).
|
||||
phase_ctl: Arc<super::stream::PhaseCtl>,
|
||||
reconfig_tx: std::sync::mpsc::Sender<punktfunk_core::Mode>,
|
||||
keyframe_tx: std::sync::mpsc::Sender<()>,
|
||||
rfi_tx: std::sync::mpsc::Sender<(u32, u32)>,
|
||||
@@ -230,6 +233,12 @@ pub(super) async fn run(
|
||||
if io::write_msg(&mut ctrl_send, &echo.encode()).await.is_err() {
|
||||
break;
|
||||
}
|
||||
} else if let Ok(pr) = punktfunk_core::quic::PhaseReport::decode(&msg) {
|
||||
// Phase-locked capture: latest-wins into the bridge — no data-plane hop, the
|
||||
// encode loop polls on its own cadence. Only vsync-aware presenters send
|
||||
// these (CLIENT_CAP_PHASE_LOCK), and PUNKTFUNK_PHASE_LOCK=0 host-side leaves
|
||||
// the stored report undrained/inert.
|
||||
phase_ctl.store(pr);
|
||||
} else if let Ok(m) = punktfunk_core::quic::CursorRenderMode::decode(&msg) {
|
||||
// Who renders the pointer (design/remote-desktop-sweep.md §8): the client's
|
||||
// mouse-model flip. Latest-wins into the shared flag; the data-plane loop
|
||||
|
||||
@@ -61,6 +61,7 @@ pub(super) fn synthetic_stream(
|
||||
pts_ns,
|
||||
host_us: (now_ns().saturating_sub(pts_ns) / 1000).min(u32::MAX as u64) as u32,
|
||||
stages: None, // synthetic loop: no capture/encode stages to split
|
||||
applied_phase_ns: None,
|
||||
};
|
||||
let _ = tc.send_datagram(punktfunk_core::quic::encode_host_timing_datagram(&t).into());
|
||||
}
|
||||
@@ -201,6 +202,112 @@ fn frame_driven_enabled() -> bool {
|
||||
*ON.get_or_init(|| std::env::var("PUNKTFUNK_FRAME_DRIVEN").as_deref() != Ok("0"))
|
||||
}
|
||||
|
||||
/// Phase-locked capture (design/phase-locked-capture.md): `PUNKTFUNK_PHASE_LOCK=0` disarms the
|
||||
/// controller — the rebuild-free A/B lever. Armed alone it does nothing until a client actually
|
||||
/// sends [`PhaseReport`](punktfunk_core::quic::PhaseReport)s.
|
||||
fn phase_lock_enabled() -> bool {
|
||||
static ON: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
|
||||
*ON.get_or_init(|| std::env::var("PUNKTFUNK_PHASE_LOCK").as_deref() != Ok("0"))
|
||||
}
|
||||
|
||||
/// Control-task → encode-loop bridge for phase-locked capture (the multi-field sibling of the
|
||||
/// `fec_target` atomic): the control task stores the client's latest
|
||||
/// [`PhaseReport`](punktfunk_core::quic::PhaseReport) (latest-wins), the encode loop drains it on
|
||||
/// its ~1 Hz adjust tick, and publishes the hold it is applying for the 0xCF ACK + diagnostics.
|
||||
pub(crate) struct PhaseCtl {
|
||||
report: std::sync::Mutex<Option<punktfunk_core::quic::PhaseReport>>,
|
||||
applied_ns: std::sync::atomic::AtomicI64,
|
||||
}
|
||||
|
||||
impl PhaseCtl {
|
||||
pub(crate) fn new() -> PhaseCtl {
|
||||
PhaseCtl {
|
||||
report: std::sync::Mutex::new(None),
|
||||
applied_ns: std::sync::atomic::AtomicI64::new(0),
|
||||
}
|
||||
}
|
||||
|
||||
/// Latest-wins store (control task).
|
||||
pub(crate) fn store(&self, r: punktfunk_core::quic::PhaseReport) {
|
||||
*self
|
||||
.report
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner) = Some(r);
|
||||
}
|
||||
|
||||
/// Drain the pending report, if any (encode loop, ~1 Hz).
|
||||
fn take(&self) -> Option<punktfunk_core::quic::PhaseReport> {
|
||||
self.report
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.take()
|
||||
}
|
||||
|
||||
fn set_applied(&self, ns: i64) {
|
||||
self.applied_ns.store(ns, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// The hold currently applied (0 = idle) — the send thread's 0xCF ACK readout.
|
||||
pub(crate) fn applied_ns(&self) -> i64 {
|
||||
self.applied_ns.load(Ordering::Relaxed)
|
||||
}
|
||||
}
|
||||
|
||||
/// The encode loop's phase controller state (design/phase-locked-capture.md §3): a per-frame
|
||||
/// HOLD before submit, walked toward the client's reported arrival lead hitting the target lead.
|
||||
/// Plain data — lives as a loop local so it survives every in-loop rebuild path; a new session
|
||||
/// (new loop call) starts unlocked, which is correct (new client, new grid).
|
||||
struct PhaseController {
|
||||
/// Applied per-frame hold before submit, ns ∈ [0, period).
|
||||
hold_ns: i64,
|
||||
/// Last adjust instant (~1 Hz cadence).
|
||||
last_adjust: std::time::Instant,
|
||||
}
|
||||
|
||||
impl PhaseController {
|
||||
/// Per-adjustment walk bound: 2 ms per second of reports keeps the wire cadence visually
|
||||
/// undisturbed while converging a worst-case half-period error in ~2-3 s.
|
||||
const MAX_STEP_NS: i64 = 2_000_000;
|
||||
/// Ignore errors under this — a locked loop does nothing (jitter would otherwise dither the
|
||||
/// hold every second).
|
||||
const DEADBAND_NS: i64 = 300_000;
|
||||
/// The lead floor the controller drives toward: SurfaceFlinger-class compositors need the
|
||||
/// frame in the queue ~2.5 ms before latch; the client's own `uncertainty_ns` widens this.
|
||||
const TARGET_LEAD_FLOOR_NS: i64 = 2_500_000;
|
||||
|
||||
fn new() -> PhaseController {
|
||||
PhaseController {
|
||||
hold_ns: 0,
|
||||
last_adjust: std::time::Instant::now(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Fold the client's latest report into the hold. `period_ns` is the wire interval (the
|
||||
/// session's frame period). Sign convention: `arrival_lead` is how long before its latch the
|
||||
/// median frame arrives — a LARGE lead means frames sit waiting at the client, so the host
|
||||
/// should send LATER (grow the hold); a small/zero lead risks missing the latch, so send
|
||||
/// EARLIER (shrink the hold, wrapping through the period when it hits zero — the newest-wins
|
||||
/// capture slot makes a wrapped hold sample a fresher frame, not a staler one).
|
||||
fn adjust(&mut self, r: &punktfunk_core::quic::PhaseReport, period_ns: i64) {
|
||||
if period_ns <= 0 {
|
||||
return;
|
||||
}
|
||||
self.last_adjust = std::time::Instant::now();
|
||||
let target = Self::TARGET_LEAD_FLOOR_NS.max(r.uncertainty_ns as i64 + 1_000_000);
|
||||
let error = r.arrival_lead_ns as i64 - target;
|
||||
if error.abs() < Self::DEADBAND_NS {
|
||||
return;
|
||||
}
|
||||
let step = error.clamp(-Self::MAX_STEP_NS, Self::MAX_STEP_NS);
|
||||
self.hold_ns = (self.hold_ns + step).rem_euclid(period_ns);
|
||||
}
|
||||
|
||||
/// Whether the ~1 Hz adjust window has elapsed.
|
||||
fn due(&self) -> bool {
|
||||
self.last_adjust.elapsed() >= std::time::Duration::from_secs(1)
|
||||
}
|
||||
}
|
||||
|
||||
/// Adaptive pipeline depth (latency plan, from the 2026-07-17 on-glass finding on a `.173` RTX
|
||||
/// 4090): the capturer's pipeline depth of 2 measured **~13 ms of glass-to-glass latency** over
|
||||
/// depth 1 at 60 fps (17 ms → 4 ms) — the AU is ready in µs but depth-2 holds it a whole frame
|
||||
@@ -526,6 +633,8 @@ fn send_loop(
|
||||
// `Some` = the client advertised VIDEO_CAP_HOST_TIMING: emit one 0xCF datagram per AU right
|
||||
// after its last packet left the socket (capture→sent, the whole host pipeline incl. pacing).
|
||||
timing_conn: Option<quinn::Connection>,
|
||||
// Phase-lock ACK source: the hold the encode loop currently applies rides the 0xCF tail.
|
||||
phase: Arc<PhaseCtl>,
|
||||
// The client advertised VIDEO_CAP_PROBE_SEQ — mid-session speed-test bursts may run in the
|
||||
// probe index space (else they're declined; see `run_probe_burst`).
|
||||
probe_seq: bool,
|
||||
@@ -631,6 +740,13 @@ fn send_loop(
|
||||
encode_us: msg.encode_us,
|
||||
pace_us: stat.spread_us,
|
||||
}),
|
||||
// Phase-lock ACK: the hold the capture tick is applying
|
||||
// right now (0 = controller idle/unarmed) — the client's
|
||||
// closed-loop readout.
|
||||
applied_phase_ns: Some(
|
||||
phase.applied_ns().clamp(i32::MIN as i64, i32::MAX as i64)
|
||||
as i32,
|
||||
),
|
||||
};
|
||||
let _ = tc.send_datagram(
|
||||
punktfunk_core::quic::encode_host_timing_datagram(&t).into(),
|
||||
@@ -968,6 +1084,9 @@ pub(super) struct SessionContext {
|
||||
/// thread emits one 0xCF datagram per AU (capture→sent µs) on it, so the client can split its
|
||||
/// `host+network` latency stage. `None` = older client, no emission.
|
||||
pub(super) timing_conn: Option<quinn::Connection>,
|
||||
/// Phase-locked capture bridge (design/phase-locked-capture.md): the control task stores
|
||||
/// client [`PhaseReport`]s here; the encode loop's controller drains them.
|
||||
pub(super) phase: Arc<PhaseCtl>,
|
||||
/// The session negotiated the cursor channel (design/remote-desktop-sweep.md M2 —
|
||||
/// `handshake::cursor_forward`): the encode loop forwards shape (via `cursor_shape_tx`)
|
||||
/// + per-tick `0xD0` state while the client draws the pointer locally.
|
||||
@@ -1143,6 +1262,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
|
||||
fec_target,
|
||||
conn,
|
||||
timing_conn,
|
||||
phase,
|
||||
cursor_forward,
|
||||
cursor_shape_tx,
|
||||
cursor_client_draws,
|
||||
@@ -1484,6 +1604,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
|
||||
.name("punktfunk-send".into())
|
||||
.spawn({
|
||||
let stop = stop.clone();
|
||||
let phase_send = phase.clone();
|
||||
move || {
|
||||
send_loop(
|
||||
session,
|
||||
@@ -1496,6 +1617,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
|
||||
fec_target,
|
||||
send_stats,
|
||||
timing_conn,
|
||||
phase_send,
|
||||
probe_seq,
|
||||
)
|
||||
}
|
||||
@@ -1541,6 +1663,11 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
|
||||
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(seconds as u64);
|
||||
let mut next = std::time::Instant::now();
|
||||
let mut sent: u64 = 0;
|
||||
// Phase-locked capture (design/phase-locked-capture.md): the per-frame hold this loop applies
|
||||
// after a fresh capture, walked ~1 Hz toward the client's reported arrival lead. A loop local
|
||||
// on purpose — it survives every in-loop rebuild path (session switch, mode/stall rebuilds,
|
||||
// encoder backoff), so a mid-stream rebuild keeps the acquired lock.
|
||||
let mut phase_ctl = PhaseController::new();
|
||||
// The session's video frame numbering, owned HERE (the wire `frame_index` of the next AU this
|
||||
// loop hands to the send thread; the packetizer seals with exactly this via `seal_frame_at`).
|
||||
// A submission's future index is predicted as `au_seq + inflight.len()` — exact because AUs
|
||||
@@ -2256,6 +2383,26 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
|
||||
Ok(Some(f)) => {
|
||||
frame = f;
|
||||
diag_new += 1;
|
||||
// Phase-locked capture: hold the fresh frame so its ARRIVAL at the client lands a
|
||||
// constant small lead before the client's display latch (§3 hold-then-submit; the
|
||||
// capture slot is newest-wins, so a long hold samples fresher content next tick,
|
||||
// never staler). Adjusted ~1 Hz from the client's PhaseReports; 0 until a report
|
||||
// arrives or when PUNKTFUNK_PHASE_LOCK=0.
|
||||
if phase_lock_enabled() {
|
||||
if phase_ctl.due() {
|
||||
if let Some(r) = phase.take() {
|
||||
phase_ctl.adjust(&r, interval.as_nanos() as i64);
|
||||
} else {
|
||||
phase_ctl.last_adjust = std::time::Instant::now();
|
||||
}
|
||||
phase.set_applied(phase_ctl.hold_ns);
|
||||
}
|
||||
if phase_ctl.hold_ns > 0 {
|
||||
std::thread::sleep(std::time::Duration::from_nanos(
|
||||
phase_ctl.hold_ns as u64,
|
||||
));
|
||||
}
|
||||
}
|
||||
capture_rebuilds = 0; // a delivered frame clears the consecutive-loss counter
|
||||
// Re-arm the park schedule for a (re)built display: pin the seat pointer to
|
||||
// the streamed surface (see `park_pointer` and the schedule state above).
|
||||
|
||||
@@ -587,6 +587,15 @@
|
||||
#define CLIENT_CAP_CURSOR 1
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// `Hello.client_caps` bit: this client runs a vsync-aware presenter and will send
|
||||
// [`PhaseReport`](super::control::PhaseReport)s (~1 Hz) so the host can phase-lock its
|
||||
// capture/send tick to the client's display latch (design/phase-locked-capture.md). Without
|
||||
// the bit the host never arms the phase controller; toward an older host the reports are
|
||||
// simply ignored — no behavior change in either direction.
|
||||
#define CLIENT_CAP_PHASE_LOCK 2
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// [`Welcome::host_caps`] bit: the host CAN forward the cursor out-of-band (it captures cursor
|
||||
// metadata separately from the frame — the Linux portal `SPA_META_Cursor` path; NOT gamescope,
|
||||
@@ -756,6 +765,11 @@
|
||||
#define MSG_CLOCK_ECHO 49
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// Type byte of [`PhaseReport`].
|
||||
#define MSG_PHASE_REPORT 50
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// Type byte of [`ClipControl`] (client → host): enable/disable the shared clipboard for this
|
||||
// session. Idempotent; opt-in is enforced here, not just in UI.
|
||||
@@ -2769,6 +2783,20 @@ PunktfunkStatus punktfunk_connection_report_decode_us(const PunktfunkConnection
|
||||
uint32_t us);
|
||||
#endif
|
||||
|
||||
// Report this client's display-latch grid so the host can phase-lock its capture tick
|
||||
// (design/phase-locked-capture.md). `next_latch_host_ns` must already be host clock — convert
|
||||
// with the connection's clock offset (`T_host = T_client + offset`). Fire-and-forget; call ~1 Hz
|
||||
// from a vsync-aware presenter. No-op toward a host that never negotiated the capability.
|
||||
//
|
||||
// # Safety
|
||||
// `c` is an opaque handle from a `*_new`/`*_pair` the caller has not yet freed, or null (an
|
||||
// error, not UB).
|
||||
PunktfunkStatus punktfunk_connection_report_phase(const PunktfunkConnection *c,
|
||||
uint64_t next_latch_host_ns,
|
||||
uint32_t latch_period_ns,
|
||||
uint32_t uncertainty_ns,
|
||||
uint32_t arrival_lead_ns);
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// Whether [`punktfunk_connection_report_decode_us`] is worth calling this session: writes 1 to
|
||||
// `out` only when the adaptive-bitrate controller is armed (Automatic bitrate, non-PyroWave), so a
|
||||
|
||||
Reference in New Issue
Block a user