The experimental stage-2 presenter (CAMetalLayer + display link) stuttered badly
in fullscreen but ran fine windowed. render() runs on the display-link / MAIN
thread and calls layer.nextDrawable(), which blocks that thread until a drawable
frees. With the layer's own displaySyncEnabled left on (default), present also
waits for the hardware vsync, so the block serializes the main thread to the
display — windowed, the WindowServer's looser compositing hides it; fullscreen's
tighter, more-direct path exposes it as judder. (Apple dev-forum guidance:
displaySync off measurably reduces nextDrawable() blocking.)
- displaySyncEnabled = false (macOS-only): the display link is already the per-
vsync pacing source, so the layer's redundant vsync wait only adds the stall.
- maximumDrawableCount = 3 (explicit): more in-flight headroom before
nextDrawable() has to block on the main thread.
Swift-only (no core/ABI change → no xcframework rebuild). Validated: swift build;
swift test (39 passed, 0 failures).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Opt-in (Settings -> Presenter; `punktfunk.presenter`, default stage-1). Stage-1's
AVSampleBufferDisplayLayer decodes AND presents internally with no per-frame
callback, so neither decode nor present can be stamped or hand-paced. Stage-2
takes explicit control:
- VideoDecoder: VTDecompressionSession, async output callback stamps
decode-completion, session rebuilt on every IDR / format change. Unit-tested
(testVideoDecoderAsyncCallbackDeliversPixels).
- MetalVideoPresenter: CAMetalLayer + CVMetalTextureCache + a runtime-compiled
BT.709 limited-range NV12->RGB shader, present at the next vsync. The
CVMetalTextures + pixel buffer are held until the GPU completes.
- Stage2Pipeline: pump thread -> decoder -> newest-ready 1-slot ring; the hosting
view's display link drains it once per vsync and stamps capture->present
(the display-link target time projected into CLOCK_REALTIME).
- LatencyMeter gains record(ptsNs:atNs:offsetNs:); the HUD shows a capture->present
(glass-to-glass, modulo host render->capture) line, skew-corrected via
clockOffsetNs. Measured live ~11 ms p50 vs ~2.2 ms capture->client.
- StreamView / StreamViewIOS host the CAMetalLayer as a sublayer + a CADisplayLink
(NSView.displayLink on macOS) when stage-2; input capture + HUD unchanged. The
session-active gates switch from `pump != nil` to `connection != nil` so capture
engages without a StreamPump.
Validated: builds macOS/iOS/tvOS; the decode half is unit-tested; the Metal
present is live-validated on glass (correct image + the capture->present number).
Colorspace is BT.709 SDR for now; 10-bit/HDR + a pacing policy are later.
Plan: docs-site/content/docs/apple-stage2-presenter.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler