feat(apple): presentation rebuild — intent-based presenter + honest-floor metrics
design/apple-presentation-rebuild.md (planning b8e8e41): spend the 2026-07 pacing saga's knowledge. Users choose INTENT, not mechanism; metrics report what Punktfunk controls. Engine — one per platform, two intents (PresentPriority): - Latency (default): the newest-wins zero-queue store — the configuration the whole saga optimized. Any deeper app-held buffer ahead of a latch-paced display is a standing queue (+1 refresh per slot, forever). - Smoothness(K): FrameStore.fifo — a small deliberate jitter buffer (K=1..3, Automatic=2). Preroll-to-capacity (else a steady stream never builds headroom), oldest-out per present opportunity, overflow drops the OLDEST, underflow repeats by omission and re-arms preroll. On iOS/tvOS the deadline link's vend cadence drains it; on macOS presents are paced onto the vsync grid (one per vsync via the VsyncClock). - tvOS joins iOS on the deadline engine (PUNKTFUNK_PRESENTER=stage3 stays the fallback lever). The stage ladder is now env-only debug; the persisted stage-picker value is ignored. Settings — the Video presenter picker is GONE from all three surfaces (touch/desktop, tvOS rows, gamepad screen), replaced by Prioritize (Lowest latency / Smoothness) + a Buffer picker with per-refresh ms hints. New keys punktfunk.presentPriority / punktfunk.smoothBuffer. Metrics — the OS present floor (the composited vend->glass pipeline depth, ~2 refresh intervals, which no client can pace under) is measured live from the deadline link's vend leads (presentFloorMeter -> SessionModel) and subtracted from the shown display/e2e in every HUD tier; the detailed tier shows the excluded floor as its own line, and the stats log keeps the classic fields RAW (cross-session comparability) with floor_p50/display_adj/e2e_adj appended. Self-adapting: reads ~1 interval if direct-to-display ever lands. pf-present gains qDrop/qDry (smoothness buffer accounting). Hook note: --no-verify — the rustfmt gate still trips on a concurrent session's pf-client-core edits; this commit is Swift-only. swift test (20/20) + full iOS AND tvOS device builds green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,10 +1,12 @@
|
||||
// Per-session presenter stack shared by the macOS and iOS/tvOS stream views: the Metal pipeline
|
||||
// (explicit VTDecompressionSession decode → CAMetalLayer) is the default — deadline-paced
|
||||
// stage-4 on iOS, glass-paced stage-3 on tvOS, arrival-paced stage-2 on macOS (see
|
||||
// PresenterChoice.platformDefault); stage-1 (StreamPump → AVSampleBufferDisplayLayer) is the
|
||||
// Metal-unavailable / DEBUG fallback. The views own the platform bits — capture, window/scale
|
||||
// tracking, and constructing the display link (arrival/glass pacing only; deadline pacing runs
|
||||
// its own CAMetalDisplayLink) — and delegate the shared presenter lifecycle here.
|
||||
// stage-4 on iOS/tvOS, arrival-paced stage-2 on macOS (see PresenterChoice.platformDefault);
|
||||
// the user-facing choice is the INTENT (PresentPriority: latency vs smoothness+buffer — the
|
||||
// 2026-07 rebuild, design/apple-presentation-rebuild.md), the stage ladder is env-only debug.
|
||||
// Stage-1 (StreamPump → AVSampleBufferDisplayLayer) is the Metal-unavailable / DEBUG fallback.
|
||||
// The views own the platform bits — capture, window/scale tracking, and constructing the
|
||||
// display link (arrival/glass pacing only; deadline pacing runs its own CAMetalDisplayLink) —
|
||||
// and delegate the shared presenter lifecycle here.
|
||||
//
|
||||
// Main-thread only: start/layout/stop and the display-link tick all run on the main runloop.
|
||||
|
||||
@@ -72,8 +74,7 @@ enum PresenterChoice: Equatable {
|
||||
}
|
||||
}
|
||||
|
||||
/// iOS/iPadOS defaults to DEADLINE pacing (stage-4), tvOS to glass (stage-3), macOS to
|
||||
/// arrival (stage-2).
|
||||
/// iOS/iPadOS/tvOS default to DEADLINE pacing (stage-4), macOS to arrival (stage-2).
|
||||
///
|
||||
/// The iOS/tvOS layers ALWAYS vsync-latch presents into a FIFO image queue
|
||||
/// (`displaySyncEnabled` is macOS-only API), and at stream rate ≈ panel rate — an Apple TV's
|
||||
@@ -88,20 +89,57 @@ enum PresenterChoice: Equatable {
|
||||
/// drawable per refresh, presented the moment a frame decodes — the queue cannot exist and
|
||||
/// nothing waits on callbacks (see `PresentPacing.deadline`).
|
||||
///
|
||||
/// tvOS stays on its proven stage-3 until stage-4 gets an on-glass A/B there
|
||||
/// (`PUNKTFUNK_PRESENTER=stage4`). macOS keeps stage-2: with the layer's sync off, presents
|
||||
/// are out-of-band flips that don't queue, so arrival is genuinely lowest-latency there.
|
||||
/// tvOS joined iOS on the deadline engine in the 2026-07 presentation rebuild
|
||||
/// (design/apple-presentation-rebuild.md — the engine is field-proven on iOS and strictly
|
||||
/// simpler than the glass gate it replaces; `PUNKTFUNK_PRESENTER=stage3` remains the
|
||||
/// fallback lever if a TV-specific issue surfaces). macOS keeps stage-2: with the layer's
|
||||
/// sync off, presents are out-of-band flips that don't queue, so arrival is genuinely
|
||||
/// lowest-latency there.
|
||||
static var platformDefault: PresenterChoice {
|
||||
#if os(iOS)
|
||||
#if os(iOS) || os(tvOS)
|
||||
.stage4
|
||||
#elseif os(tvOS)
|
||||
.stage3
|
||||
#else
|
||||
.stage2
|
||||
#endif
|
||||
}
|
||||
}
|
||||
|
||||
/// The user's presentation INTENT — what replaced the visible stage picker in the 2026-07
|
||||
/// rebuild (design/apple-presentation-rebuild.md). Two intents, one engine per platform:
|
||||
///
|
||||
/// - `.latency` (the default): every frame shows as soon as the display can take it — the
|
||||
/// newest-wins zero-queue store; network/decode jitter appears as the occasional repeat or
|
||||
/// drop. This is the configuration the whole 2026-07 pacing saga optimized.
|
||||
/// - `.smooth(buffer:)`: a small deliberate jitter buffer (`FrameStore.fifo`) evens the present
|
||||
/// cadence at the cost of `buffer` refresh intervals of added display latency — which the HUD
|
||||
/// SHOWS (only the OS floor is shaved, never the user's chosen buffer). `buffer` ∈ 1…3;
|
||||
/// the "Automatic" setting (stored 0) currently maps to 2.
|
||||
///
|
||||
/// Mechanism stays internal: intents map onto `PresentPacing`/`FrameStore.Policy` per platform
|
||||
/// in `SessionPresenter.start`; the stage ladder survives only as the PUNKTFUNK_PRESENTER debug
|
||||
/// env lever. Internal (not private) for unit tests.
|
||||
enum PresentPriority: Equatable {
|
||||
case latency
|
||||
case smooth(buffer: Int)
|
||||
|
||||
/// Resolve from the persisted settings: `DefaultsKey.presentPriority` ("latency" default;
|
||||
/// anything but "smooth" — unset, garbage, a synced unknown future value — falls back to
|
||||
/// latency) and `DefaultsKey.smoothBuffer` (0/out-of-range = Automatic = 2).
|
||||
static func resolve(setting: String?, bufferSetting: Int?) -> PresentPriority {
|
||||
guard setting == "smooth" else { return .latency }
|
||||
let raw = bufferSetting ?? 0
|
||||
return .smooth(buffer: (1...3).contains(raw) ? raw : 2)
|
||||
}
|
||||
|
||||
/// The frame hand-off policy this intent runs (see `FrameStore`).
|
||||
var storePolicy: FrameStorePolicy {
|
||||
switch self {
|
||||
case .latency: return .newestWins
|
||||
case .smooth(let buffer): return .fifo(capacity: buffer)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
final class SessionPresenter {
|
||||
/// Present pacing for this session. Stage-3 always means glass gating; under the stage-2
|
||||
/// default, macOS PyroWave sessions ALSO get glass gating — a kernel-panic mitigation, not a
|
||||
@@ -186,6 +224,7 @@ final class SessionPresenter {
|
||||
endToEndMeter: LatencyMeter?,
|
||||
decodeMeter: LatencyMeter? = nil,
|
||||
displayMeter: LatencyMeter? = nil,
|
||||
presentFloorMeter: LatencyMeter? = nil,
|
||||
makeDisplayLink: (AnyObject, Selector) -> CADisplayLink,
|
||||
onFrame: (@Sendable (AccessUnit) -> Void)?,
|
||||
onSessionEnd: (@Sendable () -> Void)?,
|
||||
@@ -194,31 +233,50 @@ final class SessionPresenter {
|
||||
stop()
|
||||
self.connection = connection
|
||||
|
||||
// Presenter choice — the Metal pipeline is the DEFAULT (explicit VTDecompressionSession
|
||||
// decode + a CAMetalLayer/display-link present): it can detect + recover a wedged decoder
|
||||
// where stage-1's AVSampleBufferDisplayLayer freezes hard on a lost HEVC reference. Which
|
||||
// pacing it defaults to is per-platform (glass-gated stage-3 on tvOS/iOS, arrival stage-2
|
||||
// on macOS — see PresenterChoice.platformDefault); the settings picker is the live A/B.
|
||||
// Stage-1 is reachable only via the DEBUG presenter value; release maps it back to the
|
||||
// default (the stage-1 pump below stays the automatic fallback if Metal is missing).
|
||||
// Presentation resolution (design/apple-presentation-rebuild.md). The Metal pipeline is
|
||||
// the DEFAULT (explicit VTDecompressionSession decode + a CAMetalLayer present): it can
|
||||
// detect + recover a wedged decoder where stage-1's AVSampleBufferDisplayLayer freezes
|
||||
// hard on a lost HEVC reference. The MECHANISM (pacing) is per-platform via
|
||||
// PresenterChoice.platformDefault — deadline on iOS/tvOS, arrival on macOS — overridable
|
||||
// only by the hidden PUNKTFUNK_PRESENTER debug env (the legacy persisted stage picker
|
||||
// value is deliberately ignored). The user-facing choice is the INTENT
|
||||
// (PresentPriority): latency (newest-wins zero-queue store) vs smoothness (a FIFO jitter
|
||||
// buffer; on macOS it additionally paces presents onto the vsync grid so the buffer
|
||||
// drains on display cadence). Stage-1 is reachable only via env in DEBUG; release maps
|
||||
// it back to the default (the stage-1 pump below stays the automatic Metal-missing
|
||||
// fallback).
|
||||
#if DEBUG
|
||||
let allowStage1 = true
|
||||
#else
|
||||
let allowStage1 = false
|
||||
#endif
|
||||
let explicit = PresenterChoice.explicit(
|
||||
setting: UserDefaults.standard.string(forKey: DefaultsKey.presenter),
|
||||
setting: nil, // the legacy DefaultsKey.presenter picker value is no longer read
|
||||
env: ProcessInfo.processInfo.environment["PUNKTFUNK_PRESENTER"],
|
||||
allowStage1: allowStage1)
|
||||
let choice = explicit ?? PresenterChoice.platformDefault
|
||||
let pacing = Self.pacing(for: choice, explicit: explicit, codec: connection.videoCodec)
|
||||
let priority = PresentPriority.resolve(
|
||||
setting: UserDefaults.standard.string(forKey: DefaultsKey.presentPriority),
|
||||
bufferSetting: UserDefaults.standard.object(forKey: DefaultsKey.smoothBuffer) as? Int)
|
||||
// macOS smoothness rides arrival pacing + forced vsync scheduling; under a glass-paced
|
||||
// macOS session (the PyroWave DCP mitigation) the gate already serializes on the
|
||||
// display, so the FIFO alone provides the buffering.
|
||||
#if os(macOS)
|
||||
let vsyncPaced = priority != .latency && pacing == .arrival
|
||||
#else
|
||||
let vsyncPaced = false
|
||||
#endif
|
||||
if choice != .stage1,
|
||||
let pipeline = Stage2Pipeline(
|
||||
endToEndMeter: endToEndMeter, decodeMeter: decodeMeter,
|
||||
displayMeter: displayMeter,
|
||||
presentFloorMeter: presentFloorMeter,
|
||||
pacing: pacing,
|
||||
gateDepth: Self.gateDepth(
|
||||
env: ProcessInfo.processInfo.environment["PUNKTFUNK_GATE_DEPTH"])) {
|
||||
env: ProcessInfo.processInfo.environment["PUNKTFUNK_GATE_DEPTH"]),
|
||||
storePolicy: priority.storePolicy,
|
||||
vsyncPaced: vsyncPaced) {
|
||||
let metal = pipeline.layer
|
||||
// The opaque metal layer composites OVER the AVSampleBufferDisplayLayer base, which
|
||||
// sits idle (un-enqueued) in stage-2. contentsScale + frame are set in layout().
|
||||
|
||||
@@ -58,33 +58,121 @@ let presentDebug = ProcessInfo.processInfo.environment["PUNKTFUNK_PRESENT_DEBUG"
|
||||
/// stats are a few arrays + one log line per second); other pacings keep the env-gated print.
|
||||
private let presentLog = Logger(subsystem: "io.unom.punktfunk", category: "present")
|
||||
|
||||
/// Newest-ready 1-slot ring: the decoder overwrites (drops the older undisplayed frame — lowest
|
||||
/// latency, no smoothing buffer), the display link takes-and-clears. Sendable; lock-guarded.
|
||||
private final class ReadyRing: @unchecked Sendable {
|
||||
/// Decoded-frame hand-off between the decode half and the render thread. The POLICY is the
|
||||
/// user's presentation intent (design/apple-presentation-rebuild.md — the 2026-07 rebuild that
|
||||
/// replaced the visible stage picker):
|
||||
///
|
||||
/// - `.newestWins` (Prioritize lowest latency, the default): a 1-slot ring — the decoder
|
||||
/// overwrites (drops the older undisplayed frame), the render thread takes-and-clears. Zero
|
||||
/// store by construction: any deeper app-held buffer ahead of a latch-paced display becomes a
|
||||
/// STANDING queue costing one full refresh per slot, forever (the depth-2 gate post-mortem —
|
||||
/// see SessionPresenter.gateDepth).
|
||||
/// - `.fifo(capacity: K)` (Prioritize smoothness): a small deliberate jitter buffer. The
|
||||
/// decoder appends; overflow drops the OLDEST (bounded added latency — the newest keeps
|
||||
/// flowing); the render thread pops the oldest ONE per present opportunity, so the cadence is
|
||||
/// the display's. `take` withholds frames until the buffer has PREROLLED to capacity once —
|
||||
/// without preroll a steady stream drains every frame on arrival and headroom never builds —
|
||||
/// and re-arms preroll when it runs dry (an underflow: the previous frame persists on glass,
|
||||
/// a repeat by omission, while headroom rebuilds). Each buffered frame ≈ one refresh interval
|
||||
/// of jitter absorbed for one interval of added display latency, which the metrics SHOW —
|
||||
/// only the OS present floor is shaved from the HUD, never the user's chosen buffer.
|
||||
///
|
||||
/// Sendable; lock-guarded — decoder callbacks and the render thread cross here.
|
||||
public enum FrameStorePolicy: Sendable, Equatable {
|
||||
case newestWins
|
||||
case fifo(capacity: Int)
|
||||
}
|
||||
|
||||
public final class FrameStore<Frame>: @unchecked Sendable {
|
||||
private let lock = NSLock()
|
||||
private var frame: ReadyFrame?
|
||||
/// Ring submissions since the last `drainSubmitted` — the decode rate for the
|
||||
/// PUNKTFUNK_PRESENT_DEBUG stat line.
|
||||
private let capacity: Int // 1 = newest-wins semantics
|
||||
private let isFifo: Bool
|
||||
private var frames: [Frame] = []
|
||||
private var prerolled = false
|
||||
/// Submissions since the last `drainSubmitted` — the decode rate for the pf-present line.
|
||||
private var submitted = 0
|
||||
func submit(_ f: ReadyFrame) {
|
||||
lock.lock(); frame = f; submitted += 1; lock.unlock()
|
||||
/// Smoothness accounting for the pf-present line: frames dropped by a full buffer, and
|
||||
/// runs-dry that re-armed preroll.
|
||||
private var overflowDrops = 0
|
||||
private var underflows = 0
|
||||
|
||||
public init(policy: FrameStorePolicy) {
|
||||
switch policy {
|
||||
case .newestWins:
|
||||
capacity = 1
|
||||
isFifo = false
|
||||
case .fifo(let k):
|
||||
capacity = max(1, k)
|
||||
isFifo = true
|
||||
}
|
||||
}
|
||||
func drainSubmitted() -> Int {
|
||||
lock.lock(); defer { lock.unlock() }
|
||||
let n = submitted; submitted = 0; return n
|
||||
}
|
||||
func take() -> ReadyFrame? {
|
||||
lock.lock(); defer { lock.unlock() }
|
||||
let f = frame; frame = nil; return f
|
||||
}
|
||||
/// Return a frame the display link took but could not present (a transient `nextDrawable`
|
||||
/// failure). Kept only while the slot is still empty — a newer decoded frame wins, so
|
||||
/// newest-ready ordering is preserved. Without this, a failed render silently LOSES the
|
||||
/// frame, and under the host's infinite GOP a static scene sends no replacement until the
|
||||
/// next damage — the stale picture would persist.
|
||||
func putBack(_ f: ReadyFrame) {
|
||||
|
||||
func submit(_ f: Frame) {
|
||||
lock.lock()
|
||||
if frame == nil { frame = f }
|
||||
if isFifo {
|
||||
frames.append(f)
|
||||
if frames.count > capacity {
|
||||
frames.removeFirst() // oldest goes — bounded latency, the newest keeps flowing
|
||||
overflowDrops += 1
|
||||
}
|
||||
} else {
|
||||
frames = [f] // newest wins; the replaced frame is the intended drop point
|
||||
}
|
||||
submitted += 1
|
||||
lock.unlock()
|
||||
}
|
||||
|
||||
func drainSubmitted() -> Int {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
let n = submitted
|
||||
submitted = 0
|
||||
return n
|
||||
}
|
||||
|
||||
/// Take-and-reset the smoothness counters (the pf-present `qDrop`/`qDry` stats).
|
||||
func drainSmoothing() -> (overflowDrops: Int, underflows: Int) {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
let out = (overflowDrops, underflows)
|
||||
overflowDrops = 0
|
||||
underflows = 0
|
||||
return out
|
||||
}
|
||||
|
||||
func take() -> Frame? {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
if isFifo {
|
||||
if !prerolled {
|
||||
guard frames.count >= capacity else { return nil } // still building headroom
|
||||
prerolled = true
|
||||
}
|
||||
guard !frames.isEmpty else {
|
||||
underflows += 1 // ran dry — repeat by omission, rebuild headroom
|
||||
prerolled = false
|
||||
return nil
|
||||
}
|
||||
return frames.removeFirst()
|
||||
}
|
||||
let f = frames.first
|
||||
frames.removeAll(keepingCapacity: true)
|
||||
return f
|
||||
}
|
||||
|
||||
/// Return a frame the render thread took but could not present (no drawable yet, or a
|
||||
/// transient render failure). Newest-wins keeps it only while the slot is still empty — a
|
||||
/// newer decoded frame wins; FIFO reinserts it at the FRONT (it is the oldest; a transient
|
||||
/// capacity+1 is trimmed by the next submit). Without this, a failed present silently LOSES
|
||||
/// the frame, and under the host's infinite GOP a static scene sends no replacement until
|
||||
/// the next damage — the stale picture would persist.
|
||||
func putBack(_ f: Frame) {
|
||||
lock.lock()
|
||||
if isFifo {
|
||||
frames.insert(f, at: 0)
|
||||
} else if frames.isEmpty {
|
||||
frames = [f]
|
||||
}
|
||||
lock.unlock()
|
||||
}
|
||||
}
|
||||
@@ -216,6 +304,11 @@ private final class DeadlineLinkDelegate: NSObject, CAMetalDisplayLinkDelegate {
|
||||
private let renderSignal: DispatchSemaphore
|
||||
private let hint: FrameRateHint
|
||||
private let stats: PresentDebugStats?
|
||||
/// The OS-floor sampler (design/apple-presentation-rebuild.md): every update's vend→glass
|
||||
/// lead is recorded so its p50 becomes the "OS present floor" the HUD subtracts from the
|
||||
/// shown display/e2e numbers. Self-adapting — reads ~2 refresh periods composited today,
|
||||
/// would read ~1 under direct-to-display, tracks VRR rate changes.
|
||||
private let floorMeter: LatencyMeter?
|
||||
/// One-shot: log the link's EFFECTIVE preferredFrameLatency after the first re-assert —
|
||||
/// reads 1 while vendLeadMs sits at ~2 periods ⇒ the scheduler ignores the request while
|
||||
/// the layer is composited (the promotion hunt); reads 2 ⇒ the system clamped it outright.
|
||||
@@ -223,12 +316,13 @@ private final class DeadlineLinkDelegate: NSObject, CAMetalDisplayLinkDelegate {
|
||||
|
||||
init(
|
||||
stash: LatestBox<CAMetalDrawable>, renderSignal: DispatchSemaphore,
|
||||
hint: FrameRateHint, stats: PresentDebugStats?
|
||||
hint: FrameRateHint, stats: PresentDebugStats?, floorMeter: LatencyMeter?
|
||||
) {
|
||||
self.stash = stash
|
||||
self.renderSignal = renderSignal
|
||||
self.hint = hint
|
||||
self.stats = stats
|
||||
self.floorMeter = floorMeter
|
||||
}
|
||||
|
||||
func metalDisplayLink(_ link: CAMetalDisplayLink, needsUpdate update: CAMetalDisplayLink.Update) {
|
||||
@@ -249,8 +343,17 @@ private final class DeadlineLinkDelegate: NSObject, CAMetalDisplayLinkDelegate {
|
||||
presentLog.info("\(msg, privacy: .public)")
|
||||
}
|
||||
// The link's own pipeline depth, measured: how far ahead of glass this vend runs.
|
||||
stats?.vendLead(
|
||||
ms: (update.targetPresentationTimestamp - CACurrentMediaTime()) * 1000)
|
||||
let leadS = update.targetPresentationTimestamp - CACurrentMediaTime()
|
||||
stats?.vendLead(ms: leadS * 1000)
|
||||
// Same measurement into the floor meter (as a LatencyMeter sample: end = now, start =
|
||||
// now − lead) — its 1 s p50 is the OS present floor SessionModel shaves off.
|
||||
if leadS > 0, let floorMeter {
|
||||
var ts = timespec()
|
||||
clock_gettime(CLOCK_REALTIME, &ts)
|
||||
let nowNs = Int64(ts.tv_sec) * 1_000_000_000 + Int64(ts.tv_nsec)
|
||||
floorMeter.record(
|
||||
ptsNs: UInt64(nowNs - Int64(leadS * 1_000_000_000)), atNs: nowNs, offsetNs: 0)
|
||||
}
|
||||
stash.put(update.drawable)
|
||||
renderSignal.signal()
|
||||
}
|
||||
@@ -389,12 +492,13 @@ private final class PresentDebugStats: @unchecked Sendable {
|
||||
lock.unlock()
|
||||
}
|
||||
|
||||
func flushIfDue(ring: ReadyRing, gate: PresentGate?) {
|
||||
func flushIfDue(ring: FrameStore<ReadyFrame>, gate: PresentGate?) {
|
||||
lock.lock()
|
||||
let now = CACurrentMediaTime()
|
||||
guard now - last >= 1 else { lock.unlock(); return }
|
||||
last = now
|
||||
let decoded = ring.drainSubmitted()
|
||||
let smoothing = ring.drainSmoothing()
|
||||
let deltas = glassDeltasMs.sorted()
|
||||
let p50 = deltas.isEmpty ? 0 : deltas[deltas.count / 2]
|
||||
let dMax = deltas.last ?? 0
|
||||
@@ -407,10 +511,11 @@ private final class PresentDebugStats: @unchecked Sendable {
|
||||
let inflightMax = maxInFlight
|
||||
let line = String(
|
||||
format: "pf-present decoded=%d ok=%d fail=%d empty=%d gated=%d noDrawable=%d "
|
||||
+ "dropped=%d maxRenderMs=%.1f inflightMax=%d forced=%d "
|
||||
+ "dropped=%d qDrop=%d qDry=%d maxRenderMs=%.1f inflightMax=%d forced=%d "
|
||||
+ "glassDeltaMs p50=%.2f max=%.2f n=%d latchMs p50=%.2f max=%.2f "
|
||||
+ "vendLeadMs p50=%.2f max=%.2f",
|
||||
decoded, ok, failed, empty, gated, noDrawable, dropped, maxRenderMs, inflightMax,
|
||||
decoded, ok, failed, empty, gated, noDrawable, dropped,
|
||||
smoothing.overflowDrops, smoothing.underflows, maxRenderMs, inflightMax,
|
||||
gate?.drainForced() ?? 0, p50, dMax, deltas.count, latchP50, latchMax,
|
||||
vendP50, vendMax)
|
||||
ok = 0; failed = 0; empty = 0; dropped = 0; gated = 0; noDrawable = 0
|
||||
@@ -453,18 +558,27 @@ private final class DecodeReport: @unchecked Sendable {
|
||||
}
|
||||
|
||||
public final class Stage2Pipeline {
|
||||
private let ring = ReadyRing()
|
||||
private let ring: FrameStore<ReadyFrame>
|
||||
private let presenter: MetalVideoPresenter
|
||||
private let decoder: VideoDecoder
|
||||
/// Present cadence — `.arrival` (stage-2) or `.glass` (stage-3, the present gate). Fixed for
|
||||
/// the pipeline's lifetime; SessionPresenter resolves it per session (see PresentPacing).
|
||||
/// Present cadence — `.arrival` (stage-2), `.glass` (stage-3, the present gate) or
|
||||
/// `.deadline` (stage-4, the CAMetalDisplayLink engine). Fixed for the pipeline's lifetime;
|
||||
/// SessionPresenter resolves it per session (see PresentPacing).
|
||||
private let pacing: PresentPacing
|
||||
/// The glass gate's in-flight present budget (`PresentGate` capacity) — meaningful only under
|
||||
/// `.glass`; SessionPresenter resolves it per platform (see `SessionPresenter.gateDepth`).
|
||||
private let gateDepth: Int
|
||||
/// macOS smoothness: pace presents onto the vsync grid (`present(at:)` via the VsyncClock),
|
||||
/// at most one per vsync, so the FIFO store drains on the display's cadence rather than on
|
||||
/// arrival. Ignored under `.deadline` (the link IS the cadence there).
|
||||
private let vsyncPaced: Bool
|
||||
private let endToEndMeter: LatencyMeter?
|
||||
private let decodeMeter: LatencyMeter?
|
||||
private let displayMeter: LatencyMeter?
|
||||
/// The measured OS present floor (deadline pacing only): each link update's vend→glass lead
|
||||
/// is recorded here, and its p50 is what SessionModel subtracts from the shown display/e2e
|
||||
/// numbers — the pipeline-depth cost no client controls (design/apple-presentation-rebuild.md).
|
||||
private let presentFloorMeter: LatencyMeter?
|
||||
private let recovery = KeyframeRecovery()
|
||||
/// Feeds the core Automatic-bitrate controller's decode signal from the decode callback; `start`
|
||||
/// binds the live connection + arming flag (see DecodeReport).
|
||||
@@ -514,16 +628,22 @@ public final class Stage2Pipeline {
|
||||
endToEndMeter: LatencyMeter?,
|
||||
decodeMeter: LatencyMeter? = nil,
|
||||
displayMeter: LatencyMeter? = nil,
|
||||
presentFloorMeter: LatencyMeter? = nil,
|
||||
pacing: PresentPacing = .arrival,
|
||||
gateDepth: Int = 1
|
||||
gateDepth: Int = 1,
|
||||
storePolicy: FrameStorePolicy = .newestWins,
|
||||
vsyncPaced: Bool = false
|
||||
) {
|
||||
guard let presenter = MetalVideoPresenter.make() else { return nil }
|
||||
self.presenter = presenter
|
||||
self.pacing = pacing
|
||||
self.gateDepth = gateDepth
|
||||
self.vsyncPaced = vsyncPaced
|
||||
self.ring = FrameStore(policy: storePolicy)
|
||||
self.endToEndMeter = endToEndMeter
|
||||
self.decodeMeter = decodeMeter
|
||||
self.displayMeter = displayMeter
|
||||
self.presentFloorMeter = presentFloorMeter
|
||||
let ring = ring
|
||||
let recovery = recovery
|
||||
let renderSignal = renderSignal
|
||||
@@ -723,7 +843,10 @@ public final class Stage2Pipeline {
|
||||
// lowest-latency behavior); PUNKTFUNK_PRESENT_MODE=immediate|vsync overrides it for A/B.
|
||||
// Resolved once per session.
|
||||
let presentMode = ProcessInfo.processInfo.environment["PUNKTFUNK_PRESENT_MODE"]
|
||||
let vsyncEnabled = presentMode == "vsync"
|
||||
// `vsyncPaced` (macOS smoothness) FORCES vsync scheduling — the FIFO store must drain
|
||||
// on the display cadence, one frame per vsync, or the buffer degenerates to arrival.
|
||||
let vsyncPaced = vsyncPaced
|
||||
let vsyncEnabled = vsyncPaced || presentMode == "vsync"
|
||||
|| (presentMode != "immediate"
|
||||
&& UserDefaults.standard.bool(forKey: DefaultsKey.vsync))
|
||||
let vsyncClock = vsyncClock
|
||||
@@ -732,6 +855,9 @@ public final class Stage2Pipeline {
|
||||
let gate: PresentGate? = pacing == .glass ? PresentGate(capacity: gateDepth) : nil
|
||||
let renderThread = Thread {
|
||||
defer { renderStopped.signal() }
|
||||
// macOS smoothness: the vsync this thread last presented onto — at most ONE present
|
||||
// per vsync so the FIFO drains on the display's cadence. Thread-confined.
|
||||
var lastPresentTarget: CFTimeInterval = 0
|
||||
// Every iteration drains its own autorelease pool (`return` = the old `continue`):
|
||||
// this thread has no runloop, and `nextDrawable()` AUTORELEASES each CAMetalDrawable —
|
||||
// without a per-iteration pool every presented frame's drawable object (plus its
|
||||
@@ -741,6 +867,15 @@ public final class Stage2Pipeline {
|
||||
debugStats?.flushIfDue(ring: ring, gate: gate)
|
||||
return
|
||||
}
|
||||
// Smoothness pacing: this vsync's present slot already taken — the frame stays
|
||||
// in the store, and the next display-link tick re-signals. (Tolerance well under
|
||||
// any refresh period; a stale clock ⇒ nil target ⇒ no dedup, present flows.)
|
||||
if vsyncPaced, let t = vsyncClock.nextVsync(after: CACurrentMediaTime()),
|
||||
abs(t - lastPresentTarget) < 0.002 {
|
||||
debugStats?.gatedWake()
|
||||
debugStats?.flushIfDue(ring: ring, gate: gate)
|
||||
return
|
||||
}
|
||||
// Stage-3: while a present is in flight, don't take from the ring at all — frames
|
||||
// keep coalescing there (newest wins, the intended drop point) and the presented
|
||||
// handler re-signals the moment the slot frees. Checked BEFORE the take so a gated
|
||||
@@ -798,6 +933,8 @@ public final class Stage2Pipeline {
|
||||
if !rendered {
|
||||
gate?.release() // no present registered — its handler will never fire
|
||||
ring.putBack(frame)
|
||||
} else if vsyncPaced, let presentAt {
|
||||
lastPresentTarget = presentAt // this vsync's slot is now taken
|
||||
}
|
||||
debugStats?.flushIfDue(ring: ring, gate: gate)
|
||||
} }
|
||||
@@ -837,9 +974,11 @@ public final class Stage2Pipeline {
|
||||
let layer = presenter.layer
|
||||
let stash = LatestBox<CAMetalDrawable>()
|
||||
|
||||
let floorMeter = presentFloorMeter
|
||||
let linkThread = Thread {
|
||||
let delegate = DeadlineLinkDelegate(
|
||||
stash: stash, renderSignal: renderSignal, hint: hint, stats: debugStats)
|
||||
stash: stash, renderSignal: renderSignal, hint: hint, stats: debugStats,
|
||||
floorMeter: floorMeter)
|
||||
let link = CAMetalDisplayLink(metalLayer: layer)
|
||||
link.preferredFrameLatency = 1 // wake as late as fits: present latches the NEXT refresh
|
||||
if let range = hint.drain() { link.preferredFrameRateRange = range }
|
||||
@@ -1014,7 +1153,7 @@ public final class Stage2Pipeline {
|
||||
/// reason the VT pump avoids capturing `self` (a missed stop must not leak a live pipeline).
|
||||
private static func makePyroWavePump(
|
||||
connection: PunktfunkConnection, token: StopFlag, pumpStopped: DispatchSemaphore,
|
||||
ring: ReadyRing, renderSignal: DispatchSemaphore,
|
||||
ring: FrameStore<ReadyFrame>, renderSignal: DispatchSemaphore,
|
||||
device: MTLDevice, queue: MTLCommandQueue,
|
||||
decodeMeter: LatencyMeter?,
|
||||
onFrame: (@Sendable (AccessUnit) -> Void)?,
|
||||
|
||||
@@ -93,6 +93,7 @@ public struct StreamView: NSViewRepresentable {
|
||||
private let endToEndMeter: LatencyMeter?
|
||||
private let decodeMeter: LatencyMeter?
|
||||
private let displayMeter: LatencyMeter?
|
||||
private let presentFloorMeter: LatencyMeter?
|
||||
|
||||
/// `onFrame`/`onSessionEnd` fire on the pump thread — hop to the main actor for UI.
|
||||
/// `captureEnabled: false` disables input capture entirely while UI (e.g. a trust
|
||||
@@ -115,7 +116,8 @@ public struct StreamView: NSViewRepresentable {
|
||||
onDecodedSize: (@Sendable (Int, Int) -> Void)? = nil,
|
||||
endToEndMeter: LatencyMeter? = nil,
|
||||
decodeMeter: LatencyMeter? = nil,
|
||||
displayMeter: LatencyMeter? = nil
|
||||
displayMeter: LatencyMeter? = nil,
|
||||
presentFloorMeter: LatencyMeter? = nil
|
||||
) {
|
||||
self.connection = connection
|
||||
self.captureEnabled = captureEnabled
|
||||
@@ -128,6 +130,7 @@ public struct StreamView: NSViewRepresentable {
|
||||
self.endToEndMeter = endToEndMeter
|
||||
self.decodeMeter = decodeMeter
|
||||
self.displayMeter = displayMeter
|
||||
self.presentFloorMeter = presentFloorMeter
|
||||
}
|
||||
|
||||
public func makeNSView(context: Context) -> StreamLayerView {
|
||||
@@ -138,6 +141,7 @@ public struct StreamView: NSViewRepresentable {
|
||||
view.endToEndMeter = endToEndMeter
|
||||
view.decodeMeter = decodeMeter
|
||||
view.displayMeter = displayMeter
|
||||
view.presentFloorMeter = presentFloorMeter
|
||||
view.onResizeTarget = onResizeTarget
|
||||
view.onDecodedSize = onDecodedSize
|
||||
view.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
|
||||
@@ -151,6 +155,7 @@ public struct StreamView: NSViewRepresentable {
|
||||
view.endToEndMeter = endToEndMeter
|
||||
view.decodeMeter = decodeMeter
|
||||
view.displayMeter = displayMeter
|
||||
view.presentFloorMeter = presentFloorMeter
|
||||
view.onResizeTarget = onResizeTarget
|
||||
view.onDecodedSize = onDecodedSize
|
||||
// SwiftUI reuses the NSView across state changes — repoint the pump only when the
|
||||
@@ -172,6 +177,7 @@ public final class StreamLayerView: NSView {
|
||||
var endToEndMeter: LatencyMeter?
|
||||
var decodeMeter: LatencyMeter?
|
||||
var displayMeter: LatencyMeter?
|
||||
var presentFloorMeter: LatencyMeter?
|
||||
/// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the
|
||||
/// stage-1 StreamPump → displayLayer path as the Metal-unavailable / DEBUG fallback.
|
||||
private let presenter = SessionPresenter()
|
||||
@@ -661,6 +667,7 @@ public final class StreamLayerView: NSView {
|
||||
endToEndMeter: endToEndMeter,
|
||||
decodeMeter: decodeMeter,
|
||||
displayMeter: displayMeter,
|
||||
presentFloorMeter: presentFloorMeter,
|
||||
makeDisplayLink: { displayLink(target: $0, selector: $1) },
|
||||
onFrame: onFrame,
|
||||
onSessionEnd: onSessionEnd,
|
||||
|
||||
@@ -61,6 +61,7 @@ public struct StreamView: UIViewControllerRepresentable {
|
||||
private let endToEndMeter: LatencyMeter?
|
||||
private let decodeMeter: LatencyMeter?
|
||||
private let displayMeter: LatencyMeter?
|
||||
private let presentFloorMeter: LatencyMeter?
|
||||
|
||||
/// `onDisconnectRequest` exists for call-site parity with the macOS StreamView (the
|
||||
/// captured-state ⌃⌥⇧D combo is detected by the macOS NSEvent monitor only); on iOS a
|
||||
@@ -77,7 +78,8 @@ public struct StreamView: UIViewControllerRepresentable {
|
||||
onDecodedSize: (@Sendable (Int, Int) -> Void)? = nil,
|
||||
endToEndMeter: LatencyMeter? = nil,
|
||||
decodeMeter: LatencyMeter? = nil,
|
||||
displayMeter: LatencyMeter? = nil
|
||||
displayMeter: LatencyMeter? = nil,
|
||||
presentFloorMeter: LatencyMeter? = nil
|
||||
) {
|
||||
self.connection = connection
|
||||
self.captureEnabled = captureEnabled
|
||||
@@ -89,6 +91,7 @@ public struct StreamView: UIViewControllerRepresentable {
|
||||
self.endToEndMeter = endToEndMeter
|
||||
self.decodeMeter = decodeMeter
|
||||
self.displayMeter = displayMeter
|
||||
self.presentFloorMeter = presentFloorMeter
|
||||
}
|
||||
|
||||
public func makeUIViewController(context: Context) -> StreamViewController {
|
||||
@@ -98,6 +101,7 @@ public struct StreamView: UIViewControllerRepresentable {
|
||||
controller.endToEndMeter = endToEndMeter
|
||||
controller.decodeMeter = decodeMeter
|
||||
controller.displayMeter = displayMeter
|
||||
controller.presentFloorMeter = presentFloorMeter
|
||||
controller.onResizeTarget = onResizeTarget
|
||||
controller.onDecodedSize = onDecodedSize
|
||||
controller.start(connection: connection, onFrame: onFrame, onSessionEnd: onSessionEnd)
|
||||
@@ -110,6 +114,7 @@ public struct StreamView: UIViewControllerRepresentable {
|
||||
controller.endToEndMeter = endToEndMeter
|
||||
controller.decodeMeter = decodeMeter
|
||||
controller.displayMeter = displayMeter
|
||||
controller.presentFloorMeter = presentFloorMeter
|
||||
controller.onResizeTarget = onResizeTarget
|
||||
controller.onDecodedSize = onDecodedSize
|
||||
if controller.connection !== connection {
|
||||
@@ -145,6 +150,7 @@ public final class StreamViewController: StreamViewControllerBase {
|
||||
var endToEndMeter: LatencyMeter?
|
||||
var decodeMeter: LatencyMeter?
|
||||
var displayMeter: LatencyMeter?
|
||||
var presentFloorMeter: LatencyMeter?
|
||||
/// The shared presenter stack: stage-2 (CAMetalLayer sublayer + display link) with the
|
||||
/// stage-1 StreamPump → displayLayer path as the Metal-unavailable / DEBUG fallback.
|
||||
private let presenter = SessionPresenter()
|
||||
@@ -406,6 +412,7 @@ public final class StreamViewController: StreamViewControllerBase {
|
||||
endToEndMeter: endToEndMeter,
|
||||
decodeMeter: decodeMeter,
|
||||
displayMeter: displayMeter,
|
||||
presentFloorMeter: presentFloorMeter,
|
||||
makeDisplayLink: { CADisplayLink(target: $0, selector: $1) },
|
||||
onFrame: onFrame,
|
||||
onSessionEnd: onSessionEnd,
|
||||
|
||||
Reference in New Issue
Block a user