feat(apple): stage-2 presenter — explicit decode + Metal present + glass-to-glass
ci / web (push) Failing after 38s
ci / rust (push) Successful in 53s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 16s
ci / docs-site (push) Failing after 39s
docker / deploy-docs (push) Successful in 16s
apple / swift (push) Successful in 1m17s
ci / web (push) Failing after 38s
ci / rust (push) Successful in 53s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 16s
ci / docs-site (push) Failing after 39s
docker / deploy-docs (push) Successful in 16s
apple / swift (push) Successful in 1m17s
Opt-in (Settings -> Presenter; `punktfunk.presenter`, default stage-1). Stage-1's AVSampleBufferDisplayLayer decodes AND presents internally with no per-frame callback, so neither decode nor present can be stamped or hand-paced. Stage-2 takes explicit control: - VideoDecoder: VTDecompressionSession, async output callback stamps decode-completion, session rebuilt on every IDR / format change. Unit-tested (testVideoDecoderAsyncCallbackDeliversPixels). - MetalVideoPresenter: CAMetalLayer + CVMetalTextureCache + a runtime-compiled BT.709 limited-range NV12->RGB shader, present at the next vsync. The CVMetalTextures + pixel buffer are held until the GPU completes. - Stage2Pipeline: pump thread -> decoder -> newest-ready 1-slot ring; the hosting view's display link drains it once per vsync and stamps capture->present (the display-link target time projected into CLOCK_REALTIME). - LatencyMeter gains record(ptsNs:atNs:offsetNs:); the HUD shows a capture->present (glass-to-glass, modulo host render->capture) line, skew-corrected via clockOffsetNs. Measured live ~11 ms p50 vs ~2.2 ms capture->client. - StreamView / StreamViewIOS host the CAMetalLayer as a sublayer + a CADisplayLink (NSView.displayLink on macOS) when stage-2; input capture + HUD unchanged. The session-active gates switch from `pump != nil` to `connection != nil` so capture engages without a StreamPump. Validated: builds macOS/iOS/tvOS; the decode half is unit-tested; the Metal present is live-validated on glass (correct image + the capture->present number). Colorspace is BT.709 SDR for now; 10-bit/HDR + a pacing policy are later. Plan: docs-site/content/docs/apple-stage2-presenter.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -17,6 +17,7 @@ struct SettingsView: View {
|
||||
@AppStorage("punktfunk.compositor") private var compositor = 0
|
||||
@AppStorage("punktfunk.gamepadType") private var gamepadType = 0
|
||||
@AppStorage("punktfunk.bitrateKbps") private var bitrateKbps = 0
|
||||
@AppStorage("punktfunk.presenter") private var presenter = "stage1"
|
||||
@AppStorage("punktfunk.micEnabled") private var micEnabled = true
|
||||
@ObservedObject private var gamepads = GamepadManager.shared
|
||||
#if os(macOS)
|
||||
@@ -88,6 +89,10 @@ struct SettingsView: View {
|
||||
}
|
||||
TVSelectionRow(
|
||||
title: "Compositor", options: compositors, selection: $compositor)
|
||||
TVSelectionRow(
|
||||
title: "Presenter",
|
||||
options: [("Stage 1 (default)", "stage1"), ("Stage 2 (experimental)", "stage2")],
|
||||
selection: $presenter)
|
||||
Text("The host creates a virtual output at exactly this mode — native "
|
||||
+ "resolution, no scaling. \(Self.bitrateFooter) A specific compositor "
|
||||
+ "is honored only if available on the host.")
|
||||
@@ -366,6 +371,21 @@ struct SettingsView: View {
|
||||
.font(.caption)
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
Section {
|
||||
Picker("Presenter", selection: $presenter) {
|
||||
Text("Stage 1 (default)").tag("stage1")
|
||||
Text("Stage 2 (experimental)").tag("stage2")
|
||||
}
|
||||
} header: {
|
||||
Text("Video presenter")
|
||||
} footer: {
|
||||
Text("Stage 1 feeds compressed video to the system display layer (known-good). "
|
||||
+ "Stage 2 decodes explicitly and presents through Metal with a display "
|
||||
+ "link — it adds a capture→present (glass-to-glass) latency line in the HUD "
|
||||
+ "and shortens the present tail. Applies from the next session.")
|
||||
.font(.caption)
|
||||
.foregroundStyle(.secondary)
|
||||
}
|
||||
Section {
|
||||
if gamepads.controllers.isEmpty {
|
||||
Text("No controllers detected")
|
||||
|
||||
Reference in New Issue
Block a user