Files
punktfunk/clients/apple/Sources/PunktfunkClient/Session/StreamHUDView.swift
T
enricobuehler a784682d4c
ci / docs-site (push) Successful in 56s
apple / swift (push) Successful in 1m14s
ci / web (push) Successful in 2m30s
decky / build-publish (push) Successful in 19s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 7s
docker / build-push (ci, ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 9s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 9s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 9s
ci / bench (push) Successful in 5m57s
release / apple (push) Successful in 9m8s
deb / build-publish (push) Successful in 9m33s
arch / build-publish (push) Successful in 13m14s
docker / deploy-docs (push) Successful in 23s
flatpak / build-publish (push) Failing after 8m1s
deb / build-publish-host (push) Successful in 10m24s
android / android (push) Successful in 14m56s
apple / screenshots (push) Successful in 6m43s
ci / rust (push) Successful in 19m29s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m35s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 15m29s
windows-host / package (push) Successful in 15m13s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 5m48s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 6m38s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 7m45s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 8m51s
fix(client/net): split the receipt stamp from the pull + bleed standing latency
The two-pair investigation (wired Mac clients stuck at a rock-steady
~18-19 ms "network" that survived the load ending and cleared only on
reconnect) exposed two structural gaps, one of measurement and one of
recovery:

- Receipt was stamped at the hand-off PULL (Swift nextAU, pf-client-core,
  Android decode loops), not at reassembly completion — so any client-side
  standing state between the reassembler and the pull read as NETWORK
  latency, undiagnosable from the HUD. ABI v9: `PunktfunkFrame`/`Frame`
  grow `received_ns`, stamped by `Session::poll_frame` as the AU crosses
  the session boundary. Every embedder now uses the core stamp; the Apple
  client keeps the pull instant as `AccessUnit.pulledNs` and shows the
  receipt→pull wait as its own "client queue" term (detailed HUD tier from
  2 ms + a `queue_p50` stats-log field). Decode stages keep their pull
  anchor on all platforms, so no historical stage shifts meaning.

- The jump-to-live detectors deliberately ignore anything under 6 queued
  frames / 400 ms behind — so a small, constant, loss-free elevation (a
  sub-frame standing backlog, or a stale clock offset after a wall-clock
  step/slew) is carried for the rest of the session. New third detector
  (`StandingLatency`, unit-tested ladder): window-MIN one-way delay
  ≥ 10 ms above the session floor with zero loss for ~4.5 s escalates
  gently — a free clock re-sync first (an applied re-sync re-bases the
  floor), then at most 3 flush+keyframe bleeds sharing the jump-to-live
  cooldown, then a loud disarm naming what it means. Loss windows reset
  the run: congestion belongs to FEC/ABR, not this detector.

Also: mid-stream re-sync apply/discard logs debug→info — they are the
forensic trail for the stale-offset case and were invisible in the field.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 23:38:11 +02:00

256 lines
14 KiB
Swift
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// The streaming overlay HUD, tiered by StatsVerbosity (the Android client's 3-tier semantics):
// * compact — one glass-pill line: fps · end-to-end p50 · throughput (+ loss when lossy);
// * normal — mode + fps/throughput, the unified latency HEADLINE (design/stats-unification.md
// — end-to-end under stage-2, capture→received under the stage-1 fallback), the loss
// counter, a capture hint (shown until input is captured), and disconnect;
// * detailed — everything normal has plus the stage equation line(s) under the headline.
// `.off` never reaches this view (ContentView gates the overlay on the tier).
import PunktfunkKit
import SwiftUI
#if canImport(UIKit)
import UIKit
#endif
struct StreamHUDView: View {
@ObservedObject var model: SessionModel
let connection: PunktfunkConnection
var placement: HUDPlacement = .topTrailing
let verbosity: StatsVerbosity
var body: some View {
// .off is gated upstream (ContentView only mounts the HUD when the tier is on) —
// render nothing if it ever slips through.
if verbosity != .off {
// ONE shared glass card wraps the tier-dependent content, so a verbosity change MORPHS
// this card — its frame (and, on iOS, its clamped corner) animate to the new size — rather
// than cross-fading a whole new card in. Only the inner content switches per tier.
tierContent
.padding(cardPadding)
.glassBackground(cardShape)
.padding(edgeInset)
}
}
/// The tier-dependent content, unwrapped (the shared card in `body` supplies the padding +
/// glass background). Compact is a one-line pill; normal/detailed the full stack.
@ViewBuilder private var tierContent: some View {
if verbosity == .compact {
compactContent
} else {
fullContent
}
}
// MARK: - Compact tier
/// One line: `{fps} fps · {e2e p50} ms · {mbps} Mb/s`. The ms segment is the best available
/// latency headline (stage-2 end-to-end, else the stage-1 capture→received) and is omitted until
/// either is valid. Loss appends in the same quiet styling the full HUD's lost line uses.
private var compactContent: some View {
HStack(spacing: 6) {
Circle()
.fill(Color.accentColor)
.frame(width: 7, height: 7)
Text(compactLine)
.font(.system(.caption, design: .monospaced))
if model.lostFrames > 0 {
Text("· lost \(model.lostFrames)")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
}
}
private var compactLine: String {
var parts = ["\(model.fps) fps"]
if model.endToEndValid {
// Floor-shaved (design/apple-presentation-rebuild.md): the OS present pipeline's
// fixed depth is excluded, so the headline describes Punktfunk's own latency.
parts.append(String(format: "%.1f ms", model.endToEndAdjP50Ms))
} else if model.hostNetworkValid {
parts.append(String(format: "%.1f ms", model.hostNetworkP50Ms))
}
parts.append(String(format: "%.1f Mb/s", model.mbps))
return parts.joined(separator: " · ")
}
// MARK: - Normal / detailed tiers
private var fullContent: some View {
VStack(alignment: placement.isTrailing ? .trailing : .leading, spacing: 4) {
HStack(spacing: 6) {
Circle()
.fill(Color.accentColor)
.frame(width: 7, height: 7)
Text("\(connection.width)×\(connection.height)@\(connection.refreshHz) \(model.fps) fps \(model.mbps, specifier: "%.1f") Mb/s")
.font(.system(.caption, design: .monospaced))
}
if model.endToEndValid {
// Stage-2: the end-to-end headline (capture→on-glass, measured directly, skew-
// corrected) — "(same-host clock)" when the host didn't answer the skew
// handshake. FLOOR-SHAVED (design/apple-presentation-rebuild.md): the OS present
// pipeline's fixed depth is excluded so the number describes Punktfunk's own
// latency; the detailed tier shows the excluded floor as its own line, and the
// stats log keeps the raw values.
Text("end-to-end \(model.endToEndAdjP50Ms, specifier: "%.1f") ms p50 · \(model.endToEndAdjP95Ms, specifier: "%.1f") p95 · capture→on-glass\(model.endToEndSkewCorrected ? "" : " (same-host clock)")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
// The equation (detailed tier only): the stages tiling the headline interval
// (per-window p50s — they only approximately sum to the directly-measured
// total). With a host that reports per-AU timings (0xCF) the first term splits
// into host + network (phase 2); an old host keeps the combined term. The
// display term is floor-shaved like the headline, so the equation still sums.
if verbosity == .detailed && model.hostNetworkValid && model.decodeValid && model.displayValid {
if model.splitValid {
Text("= host \(model.hostP50Ms, specifier: "%.1f") + network \(model.networkP50Ms, specifier: "%.1f") + decode \(model.decodeP50Ms, specifier: "%.1f") + display \(model.displayAdjP50Ms, specifier: "%.1f")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
} else {
Text("= host+network \(model.hostNetworkP50Ms, specifier: "%.1f") + decode \(model.decodeP50Ms, specifier: "%.1f") + display \(model.displayAdjP50Ms, specifier: "%.1f")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
if model.osFloorValid {
// The excluded OS term, kept visible for honesty: display-pipeline
// minimum no client can pace under (~2 refresh intervals composited).
Text("os present +\(model.osFloorP50Ms, specifier: "%.1f") excluded (display pipeline minimum)")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.tertiary)
}
// Client-queue wait (reassembly receipt → decode pull, ABI v9 split): ~0 on
// a healthy stream and hidden as noise; shown from 2 ms — a persistent value
// is a client-side standing backlog that pre-split builds displayed as
// "network" (the 2026-07 two-pair plateau). The core's standing-latency
// bleed logs alongside when it acts on the same state.
if model.clientQueueValid && model.clientQueueP50Ms >= 2 {
Text("client queue +\(model.clientQueueP50Ms, specifier: "%.1f") (receive backlog — standing if it persists)")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.tertiary)
}
}
} else if model.hostNetworkValid {
// Stage-1 fallback presenter: the layer decodes + presents internally with no
// per-frame stamp, so the honest headline ends at receipt. The host/network
// split still applies there (receipt is presenter-independent) — it becomes the
// only equation line (detailed tier); without it, host+network IS the whole
// measured interval.
Text("capture→received \(model.hostNetworkP50Ms, specifier: "%.1f") ms p50 · \(model.hostNetworkP95Ms, specifier: "%.1f") p95\(model.hostNetworkSkewCorrected ? "" : " (same-host clock)")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
if verbosity == .detailed && model.splitValid {
Text("= host \(model.hostP50Ms, specifier: "%.1f") + network \(model.networkP50Ms, specifier: "%.1f")")
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
}
if model.lostFrames > 0 {
// Unrecoverable network drops this window; hidden while the link is clean.
// String(format:) rather than specifier interpolation: the literal % would
// otherwise land in the LocalizedStringKey's format string as a bogus conversion.
Text(String(format: "lost %d (%.1f%%)", model.lostFrames, model.lostPct))
.font(.system(.caption2, design: .monospaced))
.foregroundStyle(.secondary)
}
// Capture hint, shown only until input is captured — how to grab it. The RELEASE
// shortcut is intentionally not surfaced in the overlay (it lives on the Stream menu
// and, on macOS, the start-of-stream banner), keeping the HUD uncluttered while playing.
#if os(macOS)
if !model.mouseCaptured {
Text("Click the stream to capture input")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
}
#elseif os(iOS)
// Touch always plays directly; ⌘⎋ (hardware keyboard) captures kb/mouse.
if !model.mouseCaptured {
Text("⌘⎋ captures keyboard & mouse")
.font(.geist(11, relativeTo: .caption2))
.foregroundStyle(.secondary)
}
#endif
// ⌃⌥⇧D lives on the app's Stream menu (so it still works when the HUD is hidden)
// and in InputCapture's monitor while captured; this button is the in-overlay,
// click-to-disconnect affordance. tvOS deliberately gets NEITHER a button (a
// focusable control would steal the controller's A press from the host) NOR a hint
// line: the exits are the hold gestures the start-of-stream banner teaches (hold
// the remote's Back; hold L1+R1+Start+Select on a pad).
#if os(macOS)
Button("Disconnect (⌃⌥⇧D)") { model.disconnect() }
.font(.geist(12, relativeTo: .caption))
#elseif os(iOS)
Button("Disconnect") { model.disconnect() }
.font(.geist(12, relativeTo: .caption))
#endif
}
}
// MARK: - Card metrics
/// The card's inner content padding. Roomier on tvOS — the stat text auto-scales for the
/// couch (relative system styles), so the card's chrome must keep pace or it reads cramped.
private var cardPadding: CGFloat {
#if os(tvOS)
return 16
#else
return 10
#endif
}
/// The OUTER gap between the card and the screen edge. On iOS the card hugs a physically
/// rounded display corner, so it sits a little further in and pairs with a concentric corner
/// radius (below); tvOS floats it well clear of the TV's overscan-ish edge; macOS windows
/// keep the classic 10.
private var edgeInset: CGFloat {
#if os(iOS)
return 14
#elseif os(tvOS)
return 24
#else
return 10
#endif
}
/// The card's corner radius. On iOS it's concentric with the physical display corner —
/// `displayCornerRadius edgeInset`, so the gap to the screen edge stays uniform right around the
/// corner instead of a small-radius card cutting into the very rounded glass. Clamped so a
/// flat-cornered device (or a hidden radius) still gets a sensibly rounded card.
private var cardCornerRadius: CGFloat {
#if os(iOS)
return max(12, DeviceMetrics.displayCornerRadius - edgeInset)
#elseif os(tvOS)
return 16 // scales with the roomier padding
#else
return 10
#endif
}
/// The card background shape — a continuous (squircle) rounded rectangle, matching the curve
/// Apple's hardware display corners use so the concentric inset actually reads as parallel.
private var cardShape: RoundedRectangle {
RoundedRectangle(cornerRadius: cardCornerRadius, style: .continuous)
}
}
#if os(iOS)
/// Device display geometry the overlay needs but UIKit doesn't expose publicly.
enum DeviceMetrics {
/// The physical display's corner radius. There's no public API for it, so read the private
/// `_displayCornerRadius` via KVC on the active window scene's screen, guarded by a fallback that
/// approximates a modern rounded device — a future OS that hides the key just yields a slightly
/// less-perfect inset, never a crash. The key is assembled from parts so it isn't a plain literal
/// in the binary; note the App Store private-API consideration regardless.
static var displayCornerRadius: CGFloat {
let key = ["_display", "Corner", "Radius"].joined()
guard
let screen = UIApplication.shared.connectedScenes
.compactMap({ $0 as? UIWindowScene })
.first?.screen,
let radius = screen.value(forKey: key) as? NSNumber,
radius.doubleValue > 0
else { return 44 }
return CGFloat(radius.doubleValue)
}
}
#endif