Field report (iPad Pro 13" M4, 2752x2064@120): display 23.1 ms on the default
arrival pacing vs 14 ms glass-gated, dominating an otherwise ~14 ms pipeline.
On iOS the layer ALWAYS vsync-latches (displaySyncEnabled is macOS-only API)
and default-on VRR steers the panel to the stream rate, so arrival pacing's
sticky-FIFO saturation (~2-3 refreshes of queue) is the common case, not the
corner — the exact regime that made tvOS default to glass.
- PresenterChoice.platformDefault -> stage3 on iOS/iPadOS (joins tvOS);
explicit stage-2 stays the honest arrival A/B; macOS unchanged.
- PresentGate generalized to a capacity: depth 1 is bit-identical to before;
iOS runs depth 2 (one flip scanning out + one queued for the next latch),
so a decoded frame presents immediately and latches the very next vsync
instead of serializing on the previous flip's on-glass callback — expected
~5-8 ms at 120 Hz. tvOS keeps depth 1 (proven; A/B first), macOS is pinned
to 1 (glass there is the DCP swapID-panic mitigation — serialization is
its point). PUNKTFUNK_GATE_DEPTH (1-3) is the on-device A/B lever.
- Settings picker derives its "(default)" marker from presenterDefault
instead of hardcoding stage-2 as default / stage-3 as experimental.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Real-world PyroWave streaming maxed ~2.5 Gbps with sagging fps while raw
transport does 4.8. Root-caused to serial per-frame paths at BOTH ends
(the transport was never the limit); this fixes the two dominant ones.
Host (vendored shim, patch 0004): pyrowave_encoder_encode_gpu_synchronous
allocated four Vulkan buffers (meta + bitstream, Device + CachedHost) on
EVERY frame. At 240 fps with MB-scale bitstreams that per-frame allocator
churn stalled the encode itself. Pool them on the encoder and reuse across
frames (recreate only on a size grow); the sizes are session-fixed, so it
is pure reuse after frame 1. On an RTX 4090 the 5120x1440 submit+fence-wait
drops ~15 ms -> ~1 ms, i.e. the host serial ceiling goes 64 -> 1025 fps
(444+HDR 44 -> 614). Safe under the synchronous encode model; re-validated
by pyrowave_win_smoke (Windows) and pyrowave_smoke/_444 (Linux). Applies to
both host encoder paths (they share the shim).
Client (Apple Metal decoder): WaveletBitstream.parse reserved the payload
buffer per packet (reserveCapacity(count + words), an exact realloc each of
~3000 packets/frame => O(n²)) and copied word-by-word. Reserve once up
front and memcpy each packet's coefficients in one shot (all Apple
platforms are little-endian, so the wire's LE u32s land verbatim; memcpy is
alignment-free). 5.44 ms -> 0.055 ms per 1.44 MB frame (25x); byte-identical
(parser unit tests + golden-frame PSNR unchanged).
Also:
- native.rs: PUNKTFUNK_PYROWAVE_MAX_MBPS caps PyroWave's open-loop Automatic
bitrate pin for hosts on a constrained link (unset => no cap; an explicit
client rate bypasses it). The pin is all-intra + ABR-off, so at a high
pixel rate it can outrun the fabric (4:4:4+HDR 5120x1440@240 pins ~5.3
Gbps, over a 5 GbE link) and the overshoot just becomes loss.
- pf-encode caps(): report the real opened chroma instead of a hardcoded
4:2:0 default, so a genuine 4:4:4 session no longer trips the spurious
"encoder chroma disagrees with the negotiated Welcome" warn. Also fix a
latent Windows reset() that rebuilt at 4:2:0 for a 4:4:4 session.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phases 4+5 of design/pyrowave-444-hdr.md. The Metal decoder needs NO new ABI:
every frame's sequence header carries chroma (444) and, since the Phase-3
stamps, the PQ/BT.2020 bits — so the decoder self-configures per session.
Decoder: WaveletLayout grows the 4:4:4 block space (chroma runs the full
pyramid like luma — no level-0 skip, no early half-res emit; the Metal
kernels were already chroma-agnostic, only the dispatch structure changes);
the parser accepts chroma_resolution=444, reads the PQ transfer bit, and
lifts the even-dims rule for 444; the plane ring allocates full-res chroma
and r16Unorm for PQ streams; CSC rows switch to depth-10 MSB-packed.
Presenter: planar HDR passthrough reuses pf_frag_planar on an rgba16Float
drawable (itur_2100_PQ + EDR metadata interpret the samples — same split as
pf_frag/pf_frag_hdr), plus a new pf_frag_planar_tm PQ->SDR tone-map (shared
pqToSdr tail refactored out of pf_frag_hdr_tv) for tvOS-without-headroom AND
macOS WINDOWED sessions, whose IOSurface present path (the DCP-panic
mitigation) is BGRA8-only. SessionModel stops stripping the HDR/10-bit/444
caps on the PyroWave opt-in.
New golden: au-dense444 + upstream's own 4:4:4 reference planes (regenerated
via the extended pyrowave_dump_golden); Metal decode matches at 64-67 dB
(420 fixtures re-verify 77-88 dB). Full Apple suite 157 tests green on a
real M-series GPU. Docs updated: the 8-bit-SDR-only wording is gone, the
Windows host is no longer 'on the roadmap', bpp scaling documented.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The DCP "mismatched swapID's" kernel panic reproduced on a 240 Hz Mac
Studio with stage-3 glass pacing active: a fully serialized,
one-in-flight present stream still races WindowServer's own swap
submissions. So the mitigation has to change the MECHANISM, not the
rate — the CAMetalLayer image queue itself is the racing path in a
composited (windowed) session.
Windowed PyroWave now presents the way video players do: the planar
CSC renders into a pooled IOSurface (4 × BGRA8, in-use-aware LRU
reuse) and the render thread hands it to a plain CALayer's `contents`
on main inside an ordinary CATransaction. WindowServer treats that as
normal layer damage on its own composite cadence — no out-of-band
image-queue swaps to race. Fullscreen keeps the CAMetalLayer path
(direct scanout, no compositing, no panic reports); the hosting view
pushes the window's composited state on every layout, and flipping
modes just covers/uncovers the metal layer (no black flash).
VT codecs keep the metal path everywhere: no panic reports there, and
their HDR/EDR presentation has no surface-contents equivalent wired.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Windowed PyroWave sessions were kernel-panicking Macs ("mismatched
swapID's" @UnifiedPipeline.cpp, WindowServer → AppleMobileDisp*-DCP).
The panic is an Apple DCP bug (SketchUp/Unity/240Hz-monitor reports hit
the same signature), but our stage-2 arrival pacing feeds the trigger
pattern: displaySyncEnabled=false out-of-band presents arriving faster
than the compositor latches them in a composited (windowed) session.
PyroWave makes that pattern routine — near-instant Metal wavelet decode
turns network clumps into same-millisecond present bursts, and it is
the codec that sustains stream rates above panel refresh.
Mitigation: PyroWave sessions on macOS now default to the stage-3
PresentGate (one presented-but-undisplayed swap in flight, serialized
on the on-glass callback, 100 ms stale backstop) — the racing pattern
cannot occur. Explicit stage2/stage3 picks (setting or
PUNKTFUNK_PRESENTER) still win, so the arrival-pacing A/B stays honest.
VideoToolbox codecs keep arrival pacing (decode latency spaces their
presents; no panic reports there).
PresenterChoice gains an `explicit` resolver (nil = no user selection)
so the codec-conditional default only applies when the user hasn't
picked a stage; pacing selection is a testable helper carrying the
rationale.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The W7/W8 client refactor (extracting FullscreenController/ApprovalRequest out of
ContentView, splitting Settings) landed the DefaultsKeys + scaffolding but dropped
the feature WIRING, which was uncommitted WIP shelved in a stash during the
16:28 clipboard/v4 reconcile. Recovered from that stash (+ its untracked-files
parent for the whole new ModifierLayout.swift) and re-ported onto the refactored
tree:
- invertScroll: applied in InputCapture.sendScroll (the one scroll sink) + Settings toggle
- modifierLayout (Cmd/Option switch): restored ModifierLayout.swift enum + KeyMaps
.applyModifierLayout + InputCapture.emitKey wire-boundary + Settings picker
- fullscreen shortcut (Ctrl+Cmd+F): InputCapture ⌃⌘F interception + onToggleFullscreen
+ StreamView wiring + Stream-menu item (the .punktfunkToggleFullscreen sink +
FullscreenController observer already survived on main)
- render scale: Settings picker + MatchWindowFollower renderScale/maxDimension
application + StreamView/StreamViewIOS plumbing (RenderScale.swift already on main)
StreamCommands re-ported by hand (the stash hunk deleted the since-landed clipboard
item — kept it, added the fullscreen item alongside). Recovered ModifierLayout +
RenderScale tests. swift build green; 15 tests pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Backgrounding a live session no longer freezes it when the user opts in: audio
keeps playing (UIBackgroundModes audio), the QUIC connection + pump stay live,
video decode is DROPPED, and a bounded timer auto-disconnects. Off by default.
- PunktfunkConnection.setVideoDropped/isVideoDropped: a tiny lock-guarded flag
both pumps read every iteration. StreamPump (stage-1), Stage2Pipeline (VT +
PyroWave) drain nextAU() for flow control but DISCARD the AU before any
VideoToolbox/Metal work — the crash/jetsam-safe seam (no GPU off-screen).
- SessionModel.enterBackground(timeoutMinutes:) / exitBackground(): set the drop
flag, mute the mic (privacy — SessionAudio.setMicMuted pauses the capture
engine), arm a DispatchSourceTimer that disconnect(deliberate:false)s on fire
(keeps host linger → fast late reconnect). exitBackground clears the flag and
requestKeyframe()s; the pump's freeze gate auto-arms on the resumed
frame-index gap so concealed frames are withheld until the IDR re-anchors.
disconnect() cancels the timer + clears isBackgrounded.
- ContentView scenePhase driver (iOS): .background+streaming+setting →
enterBackground; .active → exitBackground. scenePhase (not willResignActive)
so Control-Center/app-switcher peeks don't start the timer.
- Settings → General (iOS-only keepAliveSection): toggle + 1/5/10/30 timeout;
new keys backgroundKeepAlive (def off) / backgroundTimeoutMinutes (def 10).
- Info.plist: UIBackgroundModes [audio] + NSSupportsLiveActivities (for M3).
macOS swift build + swift test green (142 tests). The iOS-gated scenePhase
handler + settings section are not exercised by the macOS CI target (known §9
gap) — need on-glass verification (audio never gaps, video re-anchors <1s LAN,
timeout ends the session, phone-call audio-steal degrades gracefully).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Foundation milestone for Live Activities & Widgets (design/apple-live-
activities-and-widgets.md). No user-visible change beyond the URL scheme.
- New dependency-free PunktfunkShared SwiftPM target (+ library product) so a
future widget extension can link it WITHOUT PunktfunkKit (Rust staticlib +
presentation layer). Moves StoredHost (model + JSON codec), DefaultsKeys, and
punktfunkDefaultMgmtPort there; adds AppGroup.suiteName and the punktfunk://
DeepLink builder/parser. PunktfunkKit @_exported-imports it (no call-site
churn for consumers; intra-Kit files import it explicitly since imports are
file-scoped).
- HostStore reads/writes the shared App-Group suite (group.io.unom.punktfunk)
with a one-time migration from UserDefaults.standard (old value left in place
for staged rollout); reloads the "PunktfunkHosts" widget timeline on change.
- App Group entitlement on iOS/tvOS + macOS.
- CFBundleURLTypes scheme `punktfunk`; ContentView.onOpenURL routes
connect/<uuid>[?launch=<GameEntry.id>] into the existing connect() path
(unknown host / already-streaming guards; never tears down a live session).
- Round-trip tests: StoredHost JSON codec (+ legacy missing-optional decode),
DeepLink grammar. `swift build` + `swift test` green (142 tests, 0 failures).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends 56f9c8c4 (the Automatic-bitrate decode signal, core + Android) to the remaining
clients, so every platform caps Automatic at its real decoder limit instead of the network
link ceiling — the fix for a fast LAN feeding a slower hardware decoder.
- core/abi: punktfunk_connection_report_decode_us + _wants_decode_latency expose the
NativeClient methods to the C-ABI embedders (regenerated punktfunk_core.h, additive only).
- apple: PunktfunkConnection wrappers + Stage2Pipeline reports received→decoded from the
VideoToolbox decode-completion callback — every decoded frame, before the newest-wins ring
can drop the backlog. Stage-1 (AVSampleBufferDisplayLayer, no per-frame decode callback)
stays network-only; stage-2 is the metered path.
- windows/linux: the shared punktfunk-session client (pf-client-core) links core directly, so
it calls the NativeClient methods — report received→decoded from the pump, gated on
wants_decode_latency. Exact for the synchronous D3D11VA/software decode; received→submit
(still the decoder-input backpressure signal) for the async Vulkan-Video path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Apple client now decodes PyroWave natively on the presenter's own MTLDevice —
no MoltenVK, no upstream C++ in the app. Completes and wires up the decoder whose
early working-tree snapshot rode along in 9127c346:
- MetalWaveletShaders.swift: wavelet_dequant + idwt hand-ported from the vendored
GLSL (STORAGE_MODE 0 only; subgroup scans → 32-wide simdgroups; DCShift spec
constant → function constant; precision-1 split: fp16 levels 0-1 / fp32 2-4).
- MetalWaveletDecoder.swift: Swift reimplementation of push_packet/decode_packet
incl. the Phase-4 chunk-aligned window walk (FRAG chains, zeroed missing shards,
the >half-blocks partial rule), init_block_meta's block-index space, and the
42-dequant + 13-idwt dispatch structure with encoder-boundary barriers. SOF-dims
changes rebuild the size-dependent resources, which is also the mid-stream
resize path. Ring of 4 output plane sets on the presenter's queue.
- Presenter: pf_frag_planar (3xR8, the planar_csc.frag twin) + renderPlanar with
a shared present tail; ReadyFrame carries an image enum (.video | .planar).
- Stage2Pipeline: a dedicated PyroWave pump — no VideoToolbox machinery, no
keyframe/re-anchor recovery (all-intra; partials render as localized blur by
design), newest-frame-index staleness guard for late partials.
- Opt-in: "PyroWave (wired LAN)" codec entry (probe-gated, ≈A13 floor via a real
kernel-compile probe), selecting it advertises + prefers the codec and forces
the session SDR (HDR/10-bit/4:4:4 caps dropped, plan contract).
- Core ABI: punktfunk_connection_shard_payload() — the Welcome's negotiated shard
payload, needed by native decoders to walk chunk-aligned AUs.
- Validation: golden fixtures generated by the host encoder + upstream's own
decoder (pyrowave_dump_golden, RTX 5070 Ti); the Metal decode PSNR-matches at
77-88 dB across all planes for dense AND chunk-aligned AUs, and a hole-punched
partial still decodes. Parser unit tests cover the window walk, FRAG chains,
broken chains, the half-blocks gate, and the block-index layout.
Tests: apple 134 green (mac; iOS/tvOS build), host 312 w/ pyrowave on .21,
core 148 w/ quic; clippy/fmt clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- clients/apple: native Metal wavelet decoder + compute shaders (Phase 5),
decoding PyroWave without embedding MoltenVK.
- pf-client-core: plumb user_flags/completeness through Decoder::decode_frame
so the PyroWave backend parses chunk-aligned + partial AUs; gate the param's
unused-warning to exactly the non-pyrowave builds (fixes -D warnings on the
featureless Linux client build).
- punktfunk-host: on a mid-stream mode switch, re-resolve the "Automatic"
PyroWave bitrate for the new mode's ~1.6 bpp operating point (explicit rates
and H.26x ABR stay put); reject sub-128px PyroWave modes before the encoder
rebuild instead of after the ack.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- InputCapture / StreamViewIOS: iPad ⌃⌥⇧Q un-capture chord, recognized from the
GCKeyboard HID stream (no NSEvent monitor on iOS) for cross-client parity with
the macOS/Windows/Linux combo; and a click into the video re-engages capture —
the iPad analogue of macOS mouseDown → engageCapture(fromClick:), with the
engaging click suppressed toward the host.
- SessionPresenter: snap the aspect-fit sublayer frame to the backing pixel grid.
AVMakeRect centers the fit rect at fractional points, so the compositor
resampled the layer — a uniform "everything soft" blur even when the drawable
was pixel-exact 1:1. Rounding origin + size to device pixels makes the composite
a true 1:1 blit; idempotent when already aligned.
- MetalVideoPresenter: PUNKTFUNK_BILINEAR_LUMA=1 A/B lever — compiles the shader
with Catmull-Rom luma off (plain bilinear) to isolate bicubic overshoot from
upstream fringing.
- SettingsView / StreamView / StreamViewIOS: match-window reverted to opt-in
(default OFF) — the explicit mode is used and never auto-resized unless enabled.
After unrecoverable loss the host keeps sending delta frames that reference a
picture the client never received; hardware decoders conceal these as gray/
garbage with a success status. Linux already withheld them and held the last
good frame until a proven clean re-anchor — this brings that behavior to the
Android and Apple clients.
Extract the Linux pump's freeze state machine into a shared `ReanchorGate` in
punktfunk-core (reanchor.rs, 18 tests) exposed over the C ABI (ABI v6, additive —
no wire change) for the Swift clients. Migrate the Linux/Deck pump
(pf-client-core) onto it as the parity proof (no-op refactor). Then wire:
- Android (decode.rs, both sync + async loops): arm on the frame-index gap, a
pts-keyed flag map carries the wire flags to the output-buffer release, fold
the gate per drained output, gate.poll replaces the dropped-climb block.
- Apple Stage2Pipeline (default): arm on a gap (new noteFrameIndexGap), withhold
at the ring-submit seam (CAMetalLayer holds its last drawable), poll
framesDropped, fold VT decode errors through the no-output streak.
- Apple StreamPump (stage-1): fold at enqueue, withhold via
kCMSampleAttachmentKey_DoNotDisplay so the layer keeps decoding (reference
chain intact) but holds the last displayed frame.
- Apple VideoDecoder: thread the AU's wire flags to the async decode callback via
a retained FrameContext refcon (replaces the receivedNs bit-pattern scalar).
Lifts only on a proven re-anchor (IDR / RFI anchor / 2nd recovery mark) with a
500 ms backstop so a lost re-anchor can never freeze forever. Apple: swift build
clean, 123/123 tests pass (incl. VideoToolboxRoundTripTests). On-glass
loss-injection validation still owed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three client-UX changes that share the connect/present path (and settings files):
- Connect/wake overlay is now mode-aware: the console/gamepad UI keeps the
full-screen aurora takeover, while the default (touch/desktop) UI shows a
Liquid Glass modal over the host grid — the takeover looked out of place there.
- Add an auto-wake toggle (DefaultsKey.autoWake, default on) across macOS/iOS/tvOS
Settings + the gamepad settings view; gate startSession/prepareWake and the
gamepad "Wake & Connect" label on it. MAC-address learning stays always-on.
- Windowed sessions now stream at the window's native pixels (Match-window
default-on) so the picture is 1:1 pixel-exact instead of the presenter
resampling a fixed-mode frame; fullscreen reports full-display px, also 1:1.
Also lands the mid-resize aspect-fit tracking (decoded contentSize) that keeps
the picture undistorted after a resize.
swift build + swift test (121 tests) green; screenshot scenes verified.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Centralize the client-side loss-range detector in punktfunk-core so every
embedder shares one implementation instead of re-deriving the wrapping
frame-index arithmetic:
- NativeClient::note_frame_index(frame_index) folds each received AU (in
receive order) through RfiRecovery::observe, firing a throttled RFI request
for the exact lost span [first_missing, frame_index-1] on a forward gap. A
host that can RFI (AMD LTR / NVENC) re-references a known-good frame instead
of paying a 20-40x IDR spike; the frames_dropped-driven keyframe path stays
the backstop for when the recovery frame itself is lost.
- Export request_rfi + note_frame_index over the C ABI (Apple client).
- Call it from the Android (hw+sw pumps), Apple (StreamPump + Stage2Pipeline
via PunktfunkConnection.noteFrameIndex), and Windows in-process pumps.
Linux/Deck inherit it through pf-client-core's session pump.
- Split the decision into a pure RfiRecovery::observe(frame_index, now) and add
8 unit tests: arming, contiguous runs, exact lost-range, single-frame drop,
the 100ms throttle (burst-suppress then re-open), reorder stragglers, and
u32 wraparound (contiguous + gap-range).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make a Match-window resize deliberate instead of a stutter: blur the live
stream and show a spinner while the host rebuilds its virtual display +
encoder and VideoToolbox re-inits on the new-mode IDR. No new protocol —
driven entirely by existing client signals.
- ResizeIndicator (pure core, unit-tested): START = follower steering,
END = a decoded frame at the target size, TIMEOUT = 2.5s safety net for a
rejected/capped switch that never yields a new-size frame; re-arms only on
a CHANGED target, not a repeated same-size drag.
- MatchWindowFollower.onResizeTarget fires the instant the window differs
from the live mode (deduped via lastSteered); a new onDecodedSize callback
threads each new-mode IDR's coded dims through StreamPump/Stage2Pipeline →
SessionPresenter → both stream views.
- SessionModel gains @Published resizing (+ resizeTargeted/resizeDecoded, a
tick on the 1 Hz stats timer, reset on disconnect); ContentView blurs the
stream 16px and overlays ResizeIndicatorView while resizing (the 32px
trust-prompt blur is unchanged and takes precedence).
tvOS declares the props but never fires the follower (it drives modes via
AVDisplayManager), so the overlay stays dormant there. Pure core verified on
the Linux toolchain; full AppKit/UIKit build pending on a Mac.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
design/midstream-resolution-resize.md Phase 2, Apple client. The stream mode
follows the session window/scene: a windowed macOS window resize or an iPad
Stage Manager / Split View scene change renegotiates the host's virtual
display + encoder via the existing PunktfunkConnection.requestMode, so a
windowed session streams native-resolution pixels instead of scaling.
Decode/present need nothing — VideoToolbox recreates its session on the
keyframe-derived format-description change (§1 table).
- MatchWindowFollower (new): the shared D2 trigger discipline — physical
pixels even-floored + clamped ≥320×200, debounce to resize-end, ≥1 s
between requests, skip a size equal to the live mode, request each distinct
size at most once (stops re-asking a rejected size / looping on a host
rollback). Pure normalize/request core is unit-tested (MatchWindowTests).
- macOS StreamLayerView: fed from layoutPresenter() (bounds → convertToBacking),
guarded to once-in-a-window.
- iOS StreamViewController: fed from viewDidLayoutSubviews (bounds × render
scale); iOS-only (iPhone fullscreen no-ops, tvOS uses AVDisplayManager).
- Settings: "Match window" toggle in the Stream mode section (iOS + macOS),
DefaultsKey.matchWindow, read per session by the follower.
Verified on a Linux Swift 6.1.2 toolchain (the app target needs AppKit/UIKit,
unavailable here): the real MatchWindowFollower.swift type-checks in Swift 5
mode against a connection stub, and the pure discipline + the follower's
decision path pass a standalone harness (drag-settle + grow → exactly two
switches, refresh preserved, no re-request loop). A full build + on-device run
(macOS window, iPad Stage Manager) remains for a Mac.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The console UI now runs on tvOS through the NATIVE focus engine: carousel
cards and settings rows are focusable Buttons (Siri Remote and pads both
navigate; imperative scrollTo replaces the drop-prone scrollPosition binding),
while iOS/macOS keep the 60 Hz poll untouched - on tvOS it carries only what
focus has no concept of: X/Y screen actions and left/right value adjust with
the poll's dominant-axis feel (onMoveCommand proved input-source-dependent:
keyboard intercepted, pad dpad not -> double steps). Text entry uses the
system fullscreen keyboard (TVTextEntry); pairing + library present as covers
under the launcher; the game library defaults ON; settings values slide a
quiet 14 pt in the step's direction.
Session controls: controller/remote input routes EXCLUSIVELY through
GameController during a stream (GCEventViewController, interaction disabled) -
a pad's B no longer doubles as a UIKit menu press that ended sessions
mid-game. Deliberate exits only: the cross-client escape chord (hold
L1+R1+Start+Select 1.5 s - pf-client-core's contract, now implemented on all
Apple platforms) and holding the remote's Back >= 1 s; the start-of-stream
banner (now also on tvOS) teaches both. The Siri Remote's touch surface
drives the host pointer - press = left click, Play/Pause = right click,
release-tail jumps gated so motion stays truly relative.
tvOS 26 regressions fixed at the root: the app-wide brand tint rendered every
unfocused control as a blank pill (tint dropped on tvOS) and the 17 pt root
font shrank the whole platform (29 pt there), plus 10-foot sizing across host
cards, the gamepad screens, and the stats HUD (whose misleading "Press Menu"
hint is gone). Acknowledgements scrolls by focus-sized chunks and Menu pops
instead of suspending; full-width focusSections make the home actions
reachable from any column. The presenter defaults to stage-3 glass pacing on
tvOS (a 60 Hz panel fed a 60 fps stream is the sticky-FIFO worst case behind
the 50 ms display stage) and is pickable from the gamepad settings; HDR
capability advertises from AVPlayer.eligibleForHDRPlayback instead of the
current mode's EDR headroom, so an SDR home screen no longer hides an HDR TV.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Clients derive Y'CbCr->RGB from the stream's SIGNALED matrix x range x depth
via shared csc rows (Rust csc_rows + Swift CscRows) instead of hardcoded
709/2020 - a BT.601-signaled stream (a Linux host's RGB-input NVENC) no longer
renders with a constant hue error. Host-side signaling made honest across
NVENC/VAAPI/openh264/GameStream and the session plan's chroma/bit-depth.
Decoded color-bar fixtures (601/709 x limited/full) pin the math in tests on
both cores.
Same presenter, tvOS HDR: tvOS has no Metal EDR API and a bare PQ colorspace
tag composites UNTONE-MAPPED (the "overblown" Apple TV report), so HDR now
splits on the display's live EDR headroom - PQ passthrough when the
per-session AVDisplayManager mode switch landed (a real HDR10 output
tone-maps itself), else an in-shader PQ->SDR tone-map (203-nit reference
white, extended-Reinhard 1000-nit knee, 2020->709) into the proven SDR layer
config. The 10-bit stream keeps its full decode depth either way.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stage-2's present-on-arrival saturates CAMetalLayer's FIFO image queue
whenever the stream rate runs at the panel's refresh (iOS always
vsync-latches; the macOS 26 compositor latch-paces our out-of-band
presents the same way): one early burst fills the queue to
maximumDrawableCount and — arrivals then matching latches one-for-one —
it never drains. That sticky depth is the measured 29-30 ms display
stage on the 120 Hz ProMotion devices, and the full-queue regime is
where host<->panel clock drift turns into the reported fixed-interval
repeats/drops. The 240 Hz Studio never saturates, which is why it never
showed either symptom.
Stage-3 is the same pipeline with a PresentGate: at most ONE
presented-but-undisplayed drawable in flight; the drawable's presented
handler reopens the gate and re-signals the render thread, so the next
present always takes the freshest newest-wins ring frame — the hidden
queue latency becomes explicit, correct frame drops. A 100 ms stale
fallback force-opens a gate whose handler never fires (the "presents
aren't damage" hazard class) so a pathological system degrades visibly
instead of freezing; the PUNKTFUNK_PRESENT_DEBUG `forced` counter
exposes it (0 on healthy systems).
Selection: the Settings > Display presenter picker now ships in release
builds (stage 2 default / stage 3 experimental; the freeze-prone
stage-1 diagnostic stays DEBUG-only), resolved per session with a
PUNKTFUNK_PRESENTER=stage1|stage2|stage3 env override for CLI A/B. The
pf-present debug line gains gated/forced/inflightMax — inflightMax is
the direct image-queue-depth measurement for the A/B.
Live-verified both ways against a real host at 1080p120: stage-3 holds
120/120 fps with inflightMax=1, forced=0, glass deltas p50 8.33 ms;
stage-2 is behaviorally unchanged (120/120 fps, inflightMax=2 even on
the wired 240 Hz setup — the saturation signal in miniature). Unit
tests cover the gate (one-in-flight, stale force-open, idempotent
release) and the presenter resolution (env override, release stage-1
gating); macOS tests green, iOS/tvOS xcodebuild clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Apple client was HEVC/H.264-only: the receive path spoke Annex-B NALs
exclusively, so AV1 was never advertised and the codec picker hid it. Add
the OBU flavor of the same plumbing (AV1.swift, sibling of AnnexB.swift):
a zero-copy OBU walker, a full spec-5.5.1 sequence-header parser, an av1C
CMVideoFormatDescription with colorimetry extensions (so isHDRFormat and
the presenter stay codec-agnostic), and an ISOBMFF 'av01' sample repack
(temporal delimiter stripped, everything size-fielded, one copy per AU).
VideoCodec gains .av1 (wire 0x04); both pumps and VideoDecoder route
through dispatching formatDescription(fromKeyframe:)/sampleBuffer(au:) —
keyframe gating keys on the in-band sequence header exactly as the NAL
codecs key on in-band parameter sets, so loss recovery and mid-session
reconfigure work unchanged. AV1 sessions require a hardware decoder
(VideoToolbox has no software AV1; same fail-fast policy as 4:4:4), and
both the Hello advertisement and the Settings picker are gated on
VTIsHardwareDecodeSupported — AV1 only appears on devices that can
actually decode it (M3-class Macs, A17 Pro-class iPhones; no Apple TV).
Tests: real SVT-AV1 blobs (generation recipe in the file) cover the walk,
the parse against an independent reference, av1C bytes, delta-TU gating,
repack byte-exactness, and — on AV1 hardware — a real
VTDecompressionSession decode through VideoDecoder. Host precedence stays
HEVC > AV1 > H.264, so AV1 engages only when explicitly picked.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an "Allow VRR" toggle (Settings → Display; default on, non-tvOS) that
hands the presenter's display link a wide frame-rate RANGE — preferred =
the stream rate — so a ProMotion / adaptive-sync display can vary its
physical refresh to match the stream. `SessionPresenter.syncFrameRate`
now runs on macOS too: on it requests min 24 / max display / preferred
stream Hz; off restores the prior behavior (macOS free-runs at the native
rate, iOS keeps its 30 Hz floor). A no-op on fixed-refresh displays.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Align the macOS/iPad Stream menu on the cross-client Ctrl+Alt+Shift set
the Windows and Linux clients reserve — Release Mouse (⌃⌥⇧Q), Disconnect
(⌃⌥⇧D), HUD toggle (⌃⌥⇧S) — with ⌘⎋ kept as the macOS/iPad capture
toggle, and surface them on a 6-second banner at stream start.
Add an opt-in V-Sync present mode (punktfunk.vsync, default OFF =
lowest-latency immediate present; PUNKTFUNK_PRESENT_MODE overrides for
A/B), with the presenter reworked to a frame-arrival-triggered render
thread across Stage2Pipeline / MetalVideoPresenter / SessionPresenter,
plus the windowed title-bar safe-area handling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The clients' codec pickers sent a preference the host threw away:
host_wire_caps() hardcoded HEVC for every GPU backend, so the native
path never emitted H.264/AV1 regardless of what the user chose.
Host: advertise what the backend actually encodes, mirroring the
GameStream serverinfo logic — software openh264 emits H.264; probed
backends (Linux VAAPI, Windows AMF/QSV) advertise their per-GPU
CodecSupport via the new wire_mask(); NVENC keeps the
Moonlight-validated H.264|HEVC|AV1 static superset; an empty probe
falls back to the superset so auto clients still resolve HEVC. Gate
10-bit to HEVC (like the 4:4:4 gate) now that a client can steer the
codec — Main10 is the only 10-bit encode path. Fix the web-console
stats label hardcoded to "hevc" (new Codec::label(), shared with the
GameStream register_session mapping).
Android: replace the hardcoded H264|HEVC Hello advertisement with a
videoCodecs param fed by VideoDecoders.decodableCodecBits() (AV1 bit
only when a real hardware video/av01 decoder exists — the decode loop
was already mime-driven); offer AV1 in the codec picker on capable
devices.
Decky: add the missing "Video codec" dropdown (auto/hevc/h264/av1) to
the plugin settings, round-tripping the same codec key the flatpak
client reads.
Apple: unchanged by design (AnnexB.swift is NAL-only, AV1 is never
advertised); refresh the stale "hosts don't emit AV1 on the native
path yet" comments here and host-side.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Consumes the 0xCF host-timing plane (99cb8dd) on all four GUI clients: each
keeps a bounded pending ring of receipt samples keyed by pts, matches the
host's per-AU capture→sent reports against it, and the HUD equation becomes
= host 3.1 + network 6.7 + decode 2.1 + display 2.3
falling back to the combined `= host+network …` term whenever no timing
matched the window (old host / datagram loss) — same total, one split
fewer, never a misleading zero. Apple additionally gains the split as the
only equation line under the stage-1 fallback presenter (receipt is
presenter-independent), a `nextHostTiming` wrapper with its own plane lock,
and a unit-tested `HostNetworkSplitter`; Android extends the JNI stats
array 16→18 doubles (0–15 unchanged); Windows/Linux thread the split
through `Stats` into the HUD and the headless/debug logs.
Docs updated: design/stats-unification.md Phase 2 → implemented (wire
format, fallback semantics), and the docs-site matrix's Sunshine "Host
processing latency" row is now a direct match (ours includes the paced
send; avg vs p50).
Verified here: linux client clippy -D warnings green on the live tree,
windows stub check + hand-verified diff, android cargo-ndk arm64 check
green, apple loopback test extended (needs the rebuilt xcframework + swift
test on the mac). On-glass: pending on all platforms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One stat model everywhere (design/stats-unification.md): four measurement
points (capture/received/decoded/displayed), three stages that tile the
interval exactly, and a HUD that shows the addition explicitly —
end-to-end 14.2 ms p50 · 19.8 p95 · capture→on-glass
= host+network 9.8 + decode 2.1 + display 2.3
replacing each client's ad-hoc mix of overlapping absolutes (the Apple HUD's
three arrow lines that looked sequential but weren't), mean-vs-median decode
times (Windows/Linux), missing same-host-clock flags (Windows/Linux), and
three different names for the same capture→received measurement (probe's
"reassembled", Apple/Android's "client", Windows/Linux's post-decode "lat").
Per client: Apple threads receivedNs through the VT decode via the frame
refcon bit pattern so the decode stage exists at all (stage-1 fallback
honestly degrades to a capture→received headline); Windows carries
FrameTimes through the existing frame channel to the render thread and adds
e2e p50/p95 post-Present; Linux stamps received at AU pop and rides
decoded_ns on DecodedFrame to the paintable-set site; Android pairs receipt
stamps with MediaCodec output buffers via the codec's pts round-trip (JNI
stats array 14→16 doubles, indexes 0-13 unchanged). fps now uniformly counts
received AUs; lost/(received+lost) per window, hidden at zero.
docs-site gains "Understanding the Stats Overlay": what each line means, why
the equation only approximately sums (percentiles), and a line-by-line
Moonlight/Sunshine matrix — including that Moonlight has no end-to-end
number and its "network latency" is an ENet control RTT, so punktfunk's
headline must not be compared against any single Moonlight line.
Verified here: linux client + probe + core check/clippy/fmt green, android
native cargo-ndk arm64 check green. Pending: Windows CI + on-glass, swift
test on the mac, on-device Android.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sources reorganized (client: Home/Session/Settings/Stores/Support/Trust; kit:
Audio/Connection/Gamepad/Input/Support/Video/Views) with the big files split
along the same seams.
The gamepad mode is couch-complete, and now on macOS too (the living-room
Mac case), not just iOS/iPadOS:
- GamepadSettingsView: a console-style, fully controller-navigable settings
screen (X from the launcher) — up/down moves focus, left/right steps values
(clamped, boundary thud), A cycles/toggles, B closes; the focused row shows a
one-line description. Backed by GamepadMenuList, the vertical sibling of
GamepadCarousel, and SettingsOptions — the option lists hoisted out of
SettingsView statics and shared by the touch, tvOS and gamepad settings.
- GamepadAddHostView + GamepadKeyboard: register a host end to end with a pad
— field rows open an on-screen controller keyboard (dpad grid, A types,
X backspaces, B done); the launcher carousel ends in an Add Host tile, so
the dead-end "add one with touch first" empty state is gone.
- Launcher polish: contextual hint bar with the pad's real button glyphs,
controller name + battery chip, one shared console chrome.
- GamepadScreenBackground: an animated aurora (TimelineView-driven drifting
blobs in the brand's violet family, breathing radii, slow hue shift,
legibility scrim; freezes under Reduce Motion). Pure SwiftUI on purpose — a
.metal library only bundles reliably in one of the two build systems (SPM vs
the xcodeproj's synced folders) these sources compile under.
- macOS port: settings/add-host/library present as sized sheets (a macOS sheet
takes its content's IDEAL size, and the GeometryReader-driven screens
collapsed to nothing), NSScreen-based mode lists, scroll indicators .never
(the "always show scroll bars" setting overrides .hidden), tray scrims so
scrolled rows dim under the pinned title/hints, extra title clearance, and a
PUNKTFUNK_FORCE_GAMEPAD_UI=1 dev hook — launcher/settings/add-host/keyboard/
library render-verified live on a real Mac + LAN hosts.
- GamepadMenuInput: X button support, and (re)start now snapshots held buttons
so a controller handoff press never fires twice (the B that closed the
keyboard no longer also cancels the screen underneath).
- Cleanups: one "Connection failed" alert in ContentView instead of one per
home screen; HostDiscovery.advertises/unsaved shared by both home screens.
- host: can_encode_444 stub for the non-Linux/Windows host build (the macOS
synthetic-source loopback used by the Swift tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
