Everything the macOS app does that stage 1 lacked, before any new
feature work (user directive):
- Input capture is now a deliberate, reversible STATE (Moonlight-
style): engaged on stream start and click-into-video (the engaging
click is suppressed), released by Ctrl+Alt+Shift+Q (toggles) or
focus loss; held keys/buttons are flushed host-side on release;
cursor hiding + shortcut inhibition follow the state; HUD hint when
released. Per-session window handlers disconnect with the page.
- Gamepads: app-lifetime SDL service (GamepadManager parity) — pad
list + "Forwarded controller" pin in Settings (auto = most recent),
"Automatic" pad TYPE resolves from the physical pad at connect;
DualSense touchpad contacts + ~250 Hz motion samples on the 0xCC
plane (Swift GamepadWire scale constants); feedback grows adaptive-
trigger replay and player LEDs via raw DS5 effects packets (the
wire's 11-byte blocks drop into SDL_SendGamepadEffect verbatim);
held pad state zeroed on pad switch/detach. sdl3 "hidapi" feature.
- Microphone uplink: PipeWire capture -> Opus 20 ms -> 0xCB datagrams
(validated live: host received 711 mic packets), Settings toggle.
- Speed test per saved host (Swift's "Test Network Speed…"): 2 s
probe burst, goodput/loss + recommended ~70 % bitrate, one-tap apply.
- Settings: host compositor preference (sent in the Hello), native-
display resolution/refresh resolved from the window's monitor at
connect (new default), bitrate ceiling to 3 Gbit/s.
- Hosts page: saved/trusted hosts section for direct pinned reconnect
(mDNS not required), rebuilt on every page return.
Deliberately not ported: audio device pickers (PipeWire routing owns
this on Linux), resize-to-request_mode (not wired in Swift either),
pointer-lock relative mouse (stage-2 presenter, needs raw Wayland).
DualSense fidelity needs a physical pad to live-verify.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
NVENC ran CBR (bit_rate == max_bit_rate, rc=cbr) but never set rc_buffer_size,
so it used a loose default VBV. A high-motion P-frame was then allowed to spike
to many times the average frame size; the extra packets overflow the depth-2
send queue (newest frame dropped) and the kernel UDP buffer (WouldBlock drops),
which the client sees as framedrops/jitter — and on the infinite-GOP GameStream
path as old/stale frames flashing until the next RFI.
Set a tight ~1-frame VBV (rc_buffer_size = bitrate/fps) so the encoder holds
frame size roughly constant and absorbs motion as a momentary QP/quality dip
instead — the Sunshine/Moonlight low-latency model. Tunable via
PUNKTFUNK_VBV_FRAMES (default 1.0); larger trades burst tolerance for motion
quality. Fixes both the punktfunk/1 and GameStream paths (shared encoder).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MouseMoveAbs wire contract packs the client coordinate-space size
as (width << 16) | height in `flags` (same as touch); injectors
normalize against it and drop the event when it is zero. The GTK
client sent flags=0, so KWin's libei path refused every motion
(`emitted=false`) — found via the first real user test from
home-worker-3.
- ui_stream: send_abs() packs the negotiated mode into flags for
motion + click-position events.
- core input.rs: document the contract on MouseMoveAbs itself (it was
only implied by TouchDown's doc).
- client-rs --input-test: add a MouseMoveAbs sweep so the absolute
path stays covered — Moonlight and the Mac client only send relative
motion, which is why this gap survived every prior live test.
Validated live against serve --native: kind=MouseMoveAbs emitted=true.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Baked into the rust-ci image, plus an idempotent apt step in the rust
job itself — ci.yml runs against the previous push's image (docker.yml
bootstrap note), so the image change alone would leave this push and
the next one red.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New crate crates/punktfunk-client-linux (binary punktfunk-client), the
native Linux client on the Option A architecture (2026-06-12 research):
- GTK4/libadwaita shell linking punktfunk-core directly (no C ABI):
mDNS host list, TOFU fingerprint prompt, SPAKE2 PIN pairing dialog,
preferences (mode/bitrate/gamepad/shortcut capture), stats overlay,
--connect host[:port] for scripting.
- Video: FFmpeg software HEVC decode (LOW_DELAY, slice threads) ->
RGBA -> GdkMemoryTexture inside GtkGraphicsOffload (the dmabuf
subsurface path lights up when VAAPI lands; black-background keeps
fullscreen scanout-eligible).
- Audio: Opus -> PipeWire playback stream, the host virtual-mic's
adaptive jitter ring inverted.
- Input: keyboard as the exact inverse of the host VK table (evdev
keycodes, layout-independent; unit-tested), absolute mouse through
the Contain-fit transform, WHEEL_DELTA(120) scroll, compositor
shortcut inhibition while streaming, Ctrl+Alt+Shift+Q release chord,
F11 fullscreen. SDL3 gamepad capture (single pad-0 model) + rumble
and DualSense lightbar feedback on the same thread.
- Session pump owns video+audio pulls; the gamepad thread owns
rumble+hidout — possible because NativeClient's plane receivers are
now mutexed, making it Sync (Arc-shared, compiler-verified per-plane
contract instead of the ABI's manual assertion).
- Linux-gated deps + a stub main keep cargo build --workspace green on
macOS.
Validated live against serve --native on this box: 1920x1080@60,
locked 60 fps, capture->decoded p50 ~6.4 ms (software decode, debug
build). Teardown keys off AdwNavigationPage::hidden — NavigationView
push fires a transient unmap/map cycle that must not end the session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An identified-but-unpaired device that knocks on a pairing-required host is now
held as a pending request the operator approves from the web console — pairing it
with no PIN fetched out of band — instead of a flat reject.
- core: Hello gains an optional trailing device name (len u8 || UTF-8, ≤64,
same trailing-back-compat pattern as compositor/gamepad/bitrate). client-rs
--name sends it; the connector sends None (fingerprint-derived label).
- native_pairing: in-memory pending queue (note_pending dedups by fingerprint,
evicts the least-recently-active past a 32 cap, 10-min TTL); approve_pending
pins the fingerprint, deny drops it. Names are sanitized (strip control/ANSI/
bidi — untrusted wire input); add()/remove() roll back in-memory on a persist
failure; pairing clears any stale pending knock.
- m3: the require_pairing gate records the knock (sanitized label) before
rejecting; anonymous (certless) clients record nothing.
- mgmt: GET /native/pending, POST /native/pending/{id}/approve (optional {name})
and /deny; OpenAPI + tests; docs/api/openapi.json regenerated.
- web: a "Waiting for approval" section on the Pairing page (live-poll, Approve/
Deny, error-surfaced via QueryState); en+de strings.
- Also completes an in-progress NativeClient Sync refactor (receivers behind
per-plane mutexes) that was left half-applied in the tree.
Adversarially reviewed (4 lenses + 3-vote verify); the confirmed findings are
fixed here. Validated live on the GNOME box: knock (with a wire name, and a
malicious ANSI/bidi name that got neutralized) → pending → approve → the same
identity streams real video. Full workspace tests + clippy + fmt green; web tsc
clean. Roadmap §8b-1 marked done; §8b-2 (peer-push approval) is the client
follow-up. See docs-site pairing page.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
exportArchive's signing lookup consults the default keychain; search
list membership alone leaves the (valid) identity invisible to it.
Restored to login.keychain in cleanup.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With -allowProvisioningUpdates, exportArchive prefers cloud-managed
Developer ID signing; the App-Manager API key can't ("Cloud signing
permission error") and the valid local identity is never tried.
signingStyle=manual + explicit signingCertificate, cloud flags off
this step (archive keeps them for profile fetch).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fresh boxes lack the Developer ID / WWDR intermediates; without the
issuing chain the imported identity is invalid and xcodebuild says
"No signing certificate Developer ID Application found".
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The app shell's only navigation was the desktop sidebar (`hidden … sm:flex`), so
on phones (< sm) it was hidden with no replacement — you couldn't navigate at all.
Add a responsive mobile layout shown only below `sm`: a top bar (brand + language
switcher) and a fixed bottom tab bar with the five nav items (icon + label). The
desktop sidebar is unchanged. Page content gets bottom padding so the fixed bar
doesn't cover it, and the bar respects the iOS `safe-area-inset-bottom`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
release.yml (v* tags / dispatch, macos-arm64 runner): universal mac +
iOS xcframework -> xcodebuild archive -> Developer ID export ->
notarytool + staple -> dmg on the Gitea release; iOS archive uploads
to TestFlight (app-store-connect/upload). Per-run throwaway keychain;
ASC API key authenticates notarization, upload, and automatic-signing
profile fetch. macOS App Store lane deferred (needs App Sandbox);
tvOS deferred (tier-3 Rust targets).
All app targets now share bundle ID io.unom.punktfunk — ONE App Store
listing with universal purchase (decided pre-submission; effectively
unchangeable after). ITSAppUsesNonExemptEncryption=false declared
(standard-algorithm AES-GCM, exempt).
build-xcframework.sh resolves Apple toolchains itself: cargo's HOST
artifacts (proc-macros, build scripts) are loaded by the running OS,
and a newer-than-OS beta Xcode ld emits LINKEDIT layouts dyld rejects
("mis-aligned LINKEDIT string pool" -> misleading E0463) — so prefer
a non-beta Xcode for everything, fall back to CLT for mac-only slices
(env untouched: an explicit DEVELOPER_DIR=<CLT> trips xcrun's license
check), refuse iOS/tvOS without a real Xcode (CLT has no iOS SDK).
The runner plist no longer injects DEVELOPER_DIR for the same reason.
punktfunk_Logo.icon: dropped the Xcode-27-beta-only Icon Composer
features (refractivity, specular-location) — 26.5's actool crashes on
them, and store builds must use release Xcode. Visual delta is the
refraction/specular nuance only; re-author when 27 ships.
Validated on home-mac-mini-1 with Xcode 26.5: mac+iOS xcframework
slices, unified bundle IDs, signing-free app build.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Split the monolithic ContentView into focused view files — a pure structural refactor
with no behavior change (verified: builds macOS/iOS/tvOS, the test suite is green, and a
fidelity review against the original found no discrepancies):
- ContentView (272): the coordinator — owns the session model / host store / discovery,
switches home<->session, holds the connect logic (it reads @AppStorage) + the dev
hooks, and the stream builder (whose stable identity across awaiting-trust->streaming
must NOT move — it stays here).
- HomeView (251): the hosts grid + navigation + toolbar + sheets + "On this network"
discovery section + empty state.
- HostCards (158): HostCardView + DiscoveredCardView, sharing a CardMetrics struct
(dedupes the platform-tuned sizing the two cards had copy-pasted).
- TrustCardView (80): the TOFU prompt + fingerprint formatting.
- StreamHUDView (67): the streaming overlay HUD.
State flows idiomatically: @StateObject (ContentView) -> @ObservedObject in subviews,
@State -> @Binding; the connect logic is passed as closures. Sheet placement is
preserved — the pairing/speed-test sheets stay on the outer body so they survive the
trust->home transition.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A 6-agent adversarial audit of the client (11 confirmed of 39 findings, the rest
filtered) drove these:
- fix: SessionAudio ring buffer — guard a write larger than the ring (would push
readIdx past writeIdx and corrupt the buffer; never happens, but guard not corrupt).
- fix: CADisplayLink retain cycle (stage-2 presenter) — a weak-target DisplayLinkProxy
so the view can deallocate (the link retains its target); stage-2 teardown added to
both StreamView/StreamViewController deinits as a safety net.
- fix: GamepadFeedback deinit { flag.stop() } — the drain thread holds the connection
strongly and self weakly, so an abrupt teardown without stop() would leak it.
- refactor: centralize the 12 UserDefaults/@AppStorage key literals (scattered across
8 files) into one DefaultsKey enum — a typo silently splits a setting's reader from
its writer.
- docs: RumbleRenderer @unchecked Sendable invariant; the HID digit-row table; the
stage-2 layer compositing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The high-refresh teardown SIGSEGV was caused by ApplyMonitorsConfig disabling the
still-actively-captured high-refresh virtual output. Reorder teardown: Stop the screencast
FIRST (Mutter removes the virtual + auto-reverts the temporary config), then re-assert the
physical layout once the virtual is gone. Never reconfigure a live virtual CRTC.
With this, PUNKTFUNK_MUTTER_VIRTUAL_REFRESH=1 is stable: validated at 5120x1440@240 on
Mutter 50 + NVIDIA — virtual output Meta-0@240, real 240fps, gnome-shell survives back-to-back
sessions + teardowns, physical (HDMI-1) restored each time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the dev/agent-log pages with a proper user-facing doc set:
- Getting Started: Introduction (rewritten), How It Works, Quick Start.
- Host Setup: Requirements, then clean per-platform guides — Ubuntu GNOME,
Ubuntu KDE, Fedora KDE (new), Bazzite (rewritten) — plus Running as a Service
(desktop / headless GNOME / headless KDE).
- Connecting: Clients overview, Moonlight, Pairing & Trust.
- Configuration: host.env reference, Host CLI, Troubleshooting.
- The dev/design notes (architecture, roadmap, the deferred design specs, CI)
move to a clearly-separated "Project & Internals" nav section.
Removes the superseded box-specific pages (gnome-box, headless-box, linux-setup,
overview). status.md (the internal progress tracker, with box IPs) is kept as a
file but dropped from the public nav. Site builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pinning the virtual output to a high client refresh via RecordVirtual "modes" works
mid-stream, but a high-refresh virtual CRTC SIGSEGVs gnome-shell on session TEARDOWN
(observed at 5120x1440@240) — taking down the whole GNOME session, so subsequent connects
fail with RemoteDesktop ServiceUnknown.
Gate it behind PUNKTFUNK_MUTTER_VIRTUAL_REFRESH, default OFF — Mutter then derives the
virtual monitor's refresh from the PipeWire framerate (60Hz, stable). The >60Hz path stays
in-tree for investigation; re-enable once the teardown crash is understood.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RecordVirtual without a "modes" property makes Mutter derive the virtual monitor's refresh
from the PipeWire stream framerate and default to 60 Hz — so a 240 Hz client mode rendered
at 60 (the encoder just padded to 240 with duplicate frames). Pass an explicit "modes" entry
(size + refresh-rate + is-preferred) so Mutter creates the virtual monitor at the client's
exact WxH@Hz. Mutter >= 47; older Mutter ignores the unknown key (60 Hz fallback, no regression).
Confirmed first via raw D-Bus on the box, then validated end-to-end: the virtual output
Meta-0 reports 1920x1080@240.00 and the host encodes 480 *immediate* (real, not paced)
frames per 2 s.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the gamescope multi-user (independent-desktops) research and defer it:
the current shared host-lifetime input/audio/mic vs the per-session plumbing it
would need — per-instance EIS sockets + a per-session injector + per-session
null-sink audio routing + per-session mic — and why it's not worth it now (a large
multi-file refactor for the niche multi-user-on-one-box case, while the common
multi-device scenario is already covered by the shared-desktop multi-view
concurrency that landed). New gamescope-multiuser.md + roadmap section 14
(concurrent sessions: multi-view done, multi-user deferred).
Also park render->capture in section 12: pipewire-rs 0.9.2 exposes no
buffer-meta / raw-pointer / stream-timing API, so reading SPA_META_Header.pts
would need raw spa_sys FFI into the working capture hot path — disproportionate
for the smallest glass-to-glass term; g2g is effectively complete as
capture->present (the stage-2 presenter measures it).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Opt-in (Settings -> Presenter; `punktfunk.presenter`, default stage-1). Stage-1's
AVSampleBufferDisplayLayer decodes AND presents internally with no per-frame
callback, so neither decode nor present can be stamped or hand-paced. Stage-2
takes explicit control:
- VideoDecoder: VTDecompressionSession, async output callback stamps
decode-completion, session rebuilt on every IDR / format change. Unit-tested
(testVideoDecoderAsyncCallbackDeliversPixels).
- MetalVideoPresenter: CAMetalLayer + CVMetalTextureCache + a runtime-compiled
BT.709 limited-range NV12->RGB shader, present at the next vsync. The
CVMetalTextures + pixel buffer are held until the GPU completes.
- Stage2Pipeline: pump thread -> decoder -> newest-ready 1-slot ring; the hosting
view's display link drains it once per vsync and stamps capture->present
(the display-link target time projected into CLOCK_REALTIME).
- LatencyMeter gains record(ptsNs:atNs:offsetNs:); the HUD shows a capture->present
(glass-to-glass, modulo host render->capture) line, skew-corrected via
clockOffsetNs. Measured live ~11 ms p50 vs ~2.2 ms capture->client.
- StreamView / StreamViewIOS host the CAMetalLayer as a sublayer + a CADisplayLink
(NSView.displayLink on macOS) when stage-2; input capture + HUD unchanged. The
session-active gates switch from `pump != nil` to `connection != nil` so capture
engages without a StreamPump.
Validated: builds macOS/iOS/tvOS; the decode half is unit-tested; the Metal
present is live-validated on glass (correct image + the capture->present number).
Colorspace is BT.709 SDR for now; 10-bit/HDR + a pacing policy are later.
Plan: docs-site/content/docs/apple-stage2-presenter.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
website/cms deploy to the unom-1 DMZ VM (192.168.50.50) — the
website README's home-main-2 mention is stale. Caddy upstream fixed
in unom/reverse-proxy 6ae79b8, firewall port in unom/infra 9670aa8.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Keeping the physical monitor enabled as a secondary let the cursor, windows, and keyboard
focus land on it — relative pointer motion wandered off the streamed surface, so on the
client the cursor "disappeared" and clicks/keys went nowhere visible. Omit the physical
outputs from ApplyMonitorsConfig so Mutter disables them for the session; everything is
confined to the streamed virtual output. Restored on teardown.
Validated on-box: mid-session DisplayConfig shows only the virtual output (Meta-0) as the
sole primary; the physical (HDMI-1) is restored after the session ends.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
docker.yml gains a deploy-docs job after the image pushes: scp
compose.production.yml to ~/punktfunk-docs on home-main-2, then
docker compose pull + up over SSH — the unom/website / unom/cms
deploy pattern, same DEPLOY_* secret set (unom-ci-deploy key). Docs
bind host port 3220; the docs.punktfunk.unom.io vhost lives in
unom/reverse-proxy (306d9c0).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On a headless GNOME host the xdg-desktop-portal RemoteDesktop Start() blocks on an
interactive "Allow remote control?" approval nobody can click, so libei input timed out
("EIS setup timed out") and neither mouse nor keyboard worked — even though video worked
(it uses Mutter's direct RemoteDesktop API).
Add EiSource::MutterEis: obtain the EIS fd from
org.gnome.Mutter.RemoteDesktop.Session.ConnectToEIS (CreateSession → Start → ConnectToEIS),
no portal and no approval. Selected for GNOME/Mutter; KWin keeps the RemoteDesktop portal,
gamescope keeps its own EIS socket.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The accept loop no longer awaits each session inline — it spawns each onto a
JoinSet, bounded by a semaphore (--max-concurrent, default 4: a NVENC session
bound; overflow clients wait in QUIC's accept backlog until a slot frees). The
QUIC handshake stays in the accept loop so a failed handshake (e.g. a pin
mismatch where the client aborts) doesn't consume a session slot or block
accepting the next client; the slow part (control handshake, pairing, the
capture/encode pipeline) runs in the spawned task.
Each session already had its own virtual output + NVENC encoder; the
host-lifetime input/audio/mic services stay shared — the natural "multiple
devices viewing/controlling the same desktop" semantic on kwin/mutter/wlroots.
gamescope's independent-desktops (per-session input/audio) isolation is a
follow-up. New M3Options.max_concurrent + the `--max-concurrent` CLI flag.
Validated live (GNOME box): two clients connected at once -> two independent
Mutter virtual outputs (720p60 + 1080p60) streaming simultaneously (39 MB +
48 MB). All 61 host tests green (the c_abi/pairing tests exercise the new loop +
the failed-handshake-doesn't-count semantics).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
runs-on: ubuntu-24.04 (the label the existing Linux runner actually
advertises — ubuntu-latest queued forever). Mac runner: strip the
docker:// default labels generate-config seeds (they override the
host-mode registration labels and make the daemon demand a Docker
engine), and ship the service as a root LaunchDaemon — macOS Local
Network privacy silently blocks LAN dials from unbundled CLI binaries
in gui/user launchd domains ("no route to host"), system daemons are
exempt. Without sudo the script leaves an interim nohup daemon. CI
surface documented in CLAUDE.md + docs-site ci.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three workflows: ci.yml (Rust workspace inside the punktfunk-rust-ci
builder image + web/docs-site build+typecheck), docker.yml (build+push
punktfunk-web, punktfunk-docs, punktfunk-rust-ci to git.unom.io — host
and native clients stay un-dockerized by design), apple.yml (host-mode
macos-arm64 runner: Rust core -> PunktfunkCore.xcframework ->
swift build + swift test).
ci/rust-ci.Dockerfile: Ubuntu 26.04 with the workspace's link deps
(FFmpeg 8, PipeWire, Opus, GL/EGL/GBM, xkbcommon, libcuda via the
580-server userspace as a link stub) + pinned rustup + node for the JS
actions. Verified end to end in-container: build, 141/141 tests, C ABI
harness; all three images seeded to the registry manually.
scripts/ci/setup-macos-runner.sh provisions the Mac (rustup + darwin
targets, Node tarball, gitea-runner 1.0.8 host mode, LaunchAgent with
DEVELOPER_DIR auto-detect for sudo-free Xcode selection). Docs in
docs-site/content/docs/ci.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment reflow only — the pinned "stable" channel moved and CI checks
formatting with the current toolchain.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stage-2 was a one-line "next" in the README. Add a full, actionable spec
(docs-site apple-stage2-presenter.md) a Mac agent can execute: VTDecompressionSession
decode (with decode-completion stamping) -> CAMetalLayer + display-link present, the
exact integration points against the existing StreamPump/StreamView/AnnexB/LatencyMeter,
the three-stage measurement wiring (capture->decoded / decode->present / capture->present
= glass-to-glass, using the already-wired PunktfunkConnection.clockOffsetNs), a cheaper
decode-only intermediate, validation, and gotchas. Link it from the Apple README's
Stage 2 item. (meta.json nav entry left in the working tree to land with the CI docs WIP.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PUNKTFUNK_MUTTER_VIRTUAL_PRIMARY=1: after RecordVirtual, promote the per-session
virtual output to the primary monitor (physical kept on, secondary) via
org.gnome.Mutter.DisplayConfig.ApplyMonitorsConfig, restoring on teardown.
Without it, a GNOME host that also has a physical monitor attached keeps the physical
primary, so the virtual output is an empty extended desktop — the client streams only
the wallpaper. (The backend was validated on headless GNOME, where the virtual output
is the only display.)
Best-effort + opt-in: default behavior is unchanged; any DisplayConfig failure just
logs and streaming continues. method=temporary, so nothing is written to monitors.xml
and Mutter auto-reverts the layout when the virtual output is torn down.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The capture->client latency line concatenated a String onto a LocalizedStringKey
(Text("...\(x, specifier:)..." + (cond ? "" : "...")), which doesn't type-check:
the specifier: interpolation makes the literal a LocalizedStringKey, which has no
'+'. Fold the conditional suffix into the interpolation instead — the Apple
client didn't build on the latency-HUD commit (ac4f247).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Input handling, building on macOS/iOS/tvOS:
- macOS recapture after navigating out: engageCapture no longer latches
captured=true when the cursor grab is refused mid app-activation (which left
a free cursor that no later click could re-grab); cursorCapture.capture() now
reports success. + canBecomeKeyView.
- iOS/iPadOS recapture: restore the prior capture on didBecomeActive (nothing
re-grabbed mouse/keyboard on return before).
- iPad indirect pointer (no lock) is forwarded as an absolute MOUSE (move +
buttons + scroll via hover / UITouch.indirectPointer), not as touch, with the
local cursor visible; GCMouse owns the locked regime, gated so the two never
double-send. Adds the MouseMoveAbs wire helper.
- Trackpad scroll on iOS (was entirely missing): GCMouse scroll dpad when
locked + a scroll-only UIPanGestureRecognizer otherwise.
- tvOS: no focusable control during play (a focusable Disconnect button ate the
controller's A in the focus engine); Siri Remote Menu disconnects.
- Don't leak touch to the host under the TOFU trust prompt (gate on
captureEnabled).
LAN discovery: HostDiscovery (NWBrowser over _punktfunk._udp, the host's
crate::discovery advert) resolves each service to IP:port and parses the TXT
(fp advisory, pair, id); an "On this network" section in the grid (tap to save
+ connect, or pair if required). iOS/tvOS get NSBonjourServices via a merged
Config/Info.plist. Integration-tested end to end against a fake NWListener advert.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Apple speed test asked for only 400 Mbps, capping the measured throughput
there and hiding the link's real headroom. Request the host's full
MAX_PROBE_KBPS (3 Gbps) instead, and raise the recommended-bitrate clamp from
500 Mbps to the host's 2 Gbps session ceiling so a fast measurement yields a
usable recommendation.
Also fix the stale caps left when the host clamps were raised (0cd02e7): the
resolved-bitrate range and the probe doc comments (abi.rs, client.rs,
regenerated header), plus the section 9 roadmap copy, now read 3 Gbps probe /
2 Gbps session.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The clock_offset test's assert_eq! carried an inline message that newer rustfmt
wants to wrap while the repo's committed style keeps such asserts on one line.
Move the message to a comment and use bare assert_eq! so it formats identically
under any rustfmt version — no new fmt-check ambiguity from this addition.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Apple client now consumes the connector's clock offset. PunktfunkConnection
reads punktfunk_connection_clock_offset_ns into clockOffsetNs at connect; a new
LatencyMeter (PunktfunkKit, NSLock + percentiles, mirrors FrameMeter) records each
AU's capture->client-receipt latency = now(CLOCK_REALTIME) + offset - pts_ns, and
SessionModel drains p50/p95 into the macOS HUD ("capture->client N/N ms p50/p95",
"(same-host)" when the host didn't answer the skew handshake). Wired at the
existing onFrame hook in ContentView — additive, no change to the decode/present
path. Unit test for the meter (percentiles, skew flag, absurd-value guard).
This is the first cross-machine latency the real Apple client reports. SCOPE:
stage-1 AVSampleBufferDisplayLayer decodes+presents compressed samples internally
with no per-frame callback, so this excludes decode+present; true decode->present
needs the stage-2 presenter (VTDecompressionSession + CAMetalLayer). Rebuild
PunktfunkCore.xcframework (for the new C getter) before swift build/test on a Mac.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Factor the client-side skew handshake into a shared core helper (quic::clock_sync
-> ClockSkew) so both the reference client and the embeddable connector use one
implementation. NativeClient now runs the handshake at connect (right after Start,
before the control task takes the stream) and stores the host-client offset; it's
read over the C ABI via punktfunk_connection_clock_offset_ns (i64 ns, host minus
client; 0 = no correction / old host).
This is the substrate the Apple client needs for the decode->present (glass-to-
glass) term: stamp present time, add the offset to express it in the host's
capture clock, subtract the AU pts_ns. client-rs drops its local clock_sync copy
and uses the shared helper (behavior unchanged; validated locally).
Regenerates include/punktfunk_core.h. Roadmap section 12 + status updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per the new docs workflow (docs-site = KB layer; repo docs/ keeps design notes):
- Add a canonical Status & Progress tracker (status.md): milestones, per-box live
state, and a dated progress log — the go-forward place to track progress.
- Add setup guides: GNOME/Mutter host (gnome-box — Secure Boot MOK enroll, the
libnvidia-gl EGL fix, autologin, screen-lock disable, appliance unit), headless
KDE box, and Bazzite host (ujust input group, gamescope session, gotchas).
- Roadmap is now canonical in docs-site (synced the skew-handshake section 12
update); removed the repo docs/roadmap.md copy and repointed README to docs-site.
- Nav (meta.json) + landing cards updated; site builds (bun run build).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ClockProbe/ClockEcho on the QUIC control stream — 8 NTP-style rounds right after
Start; the min-RTT sample gives the host-client clock offset (clock_offset_ns
estimator in punktfunk-core). The client adds the offset to its receive instant
before differencing against the AU pts_ns, so the capture->reassembled latency
percentiles are valid across machines (skew_corrected=true), not just same-host.
Back-compat: an old host that doesn't answer the probe times out and the client
falls back to a shared-clock assumption (skew_corrected=false).
Host adds one ClockProbe dispatch arm in the control task; the client runs
clock_sync after Start, before the --remode/--speed-test tasks take the stream.
Validated cross-LAN (GNOME box -> dev box): offset ~ -1.57 ms (reproducible),
rtt ~140 us, p50 1.30 ms skew-corrected capture->reassembled — the offset is
exactly the systematic error the handshake removes. Unit tests for the message
codecs and the min-RTT offset estimator.
Roadmap §12: skew handshake done; remaining for true glass-to-glass is the Apple
client present-stamp (decode->present) plus the host render->capture term.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both the unified host (serve --native) and standalone m3-host now advertise the
native punktfunk/1 service over mDNS (_punktfunk._udp) — the analogue of the
GameStream _nvstream._tcp advert. TXT records carry proto, the host cert
fingerprint (fp, the value clients pin), the pairing requirement
(pair=required|optional), and the host id. New crate::discovery module, wired
into m3::serve so both host entry points get it; best-effort, never blocks
streaming (--connect always works).
Client gains `punktfunk-client-rs --discover [SECS]`: browses the LAN and prints
each host (name, addr:port, pairing, fingerprint), then exits. Apple clients
browse the same service natively via NWBrowser (service type + TXT keys are the
contract).
Validated cross-LAN: the dev box discovered the GNOME-box appliance
(pair=required) and a standalone synthetic host (pair=optional); fingerprint and
pairing state correct in both.
Also refresh the now-stale sendmmsg caveat in the bitrate doc (batched/paced send
landed + validated to 1 Gbps) and mark the encode|send thread split done in §12.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prep for a third (Ubuntu) test host: document the Mutter backend env — wayland-0
(not wayland-kde), XDG_CURRENT_DESKTOP=GNOME, PUNKTFUNK_COMPOSITOR=mutter, virtual
source via RecordVirtual, libei input via the RemoteDesktop portal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bigger-bet #1 from the latency plan. virtual_stream ran capture+encode+seal+
paced-send on ONE thread, so frame N+1's capture/encode couldn't start until
frame N's entire paced tail had left the wire — the pacing budget (~0.9×interval)
was serialized in front of the next encode. Port GameStream's spawn_sender model
to the native path:
- A dedicated send thread (`send_loop`) owns the WHOLE Session (so no socket
clone or shared/Arc stats needed — `seal_frame` mutates the nonce, `send_sealed`
+ the probe bursts all live there) and does FEC+seal + microburst-paced send.
- The encode thread captures+encodes + handles reconfig and hands each AU over a
bounded sync_channel(3) as a FrameMsg (data, capture_ns, flags, deadline,
encode_us). It BLOCKS on backpressure if the send falls behind — frames slow
down rather than a dropped frame freezing the infinite-GOP stream (we don't
drop). Clean shutdown: drop the channel → send thread drains/exits → join.
- Probes (run_probe_burst) move to the send thread since they need the Session; a
burst naturally pauses video (the encode thread blocks on the full channel).
- Per-frame encode_us/pace_us histogram moved to the send thread (carries
encode_us in the FrameMsg) and now reflects the overlap.
Removes the encode↔paced-tail serialization (~2-8 ms @60-120 fps), independent of
the pacing policy, no quality cost. Substrate for the future NVENC slice wrapper.
Verified live on this box (appliance restarted onto it): a client streamed the
KWin desktop (1.49 MB H.265, clean, no panic) and a 200 Mbps speed-test probe
completed through the send thread (0 drops). Build + clippy + fmt green.
Real-NIC sustained soak (reconfig under load, line-rate, mode switches) pending
the Ubuntu third host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
From a bug-hunt + unsafe-audit pass (4 reviewers + adversarial verify). It
confirmed ZERO real bugs in the recent batched/paced data-plane work — these are
the surfaced cleanups + one genuine soundness fix:
- SOUNDNESS (reduce unsafe): inject/gamepad.rs::pump_ff did `ptr::read` of an
InputEventRaw (align 8, holds a timeval) out of a 1-aligned [u8; N] buffer — UB
per the reference (x86_64 tolerates it, but it can miscompile under LTO). Use
ptr::read_unaligned + a SAFETY note. Zero behavior change.
- recv parity: recv_batch (recvmmsg) didn't drop an oversized/truncated datagram
the way scalar recv does — poll_frame now skips a message whose len fills the
buffer (> MAX_DATAGRAM_BYTES), matching recv's `n >= RECV_BUF` drop. (AEAD
already rejected these on encrypted sessions; this restores the documented
invariant on the batched path.)
- dedup unsafe FFI: factor the identical mmsghdr-from-iovec construction out of
send_batch + recv_batch into one `mmsghdrs()` helper — the raw-pointer
scaffolding + its lifetime SAFETY note now live in one place.
- docs: TARGET_SOCKBUF no longer calls paced sending future work (it landed,
m3.rs::paced_submit); gamescope.rs input is no longer "(TODO)" (wired +
live-validated); the PUNKTFUNK_PERF `wire_mbps` field is renamed `tx_mbps` and
noted as attempted/sealed bytes (send_dropped shows what didn't reach the wire).
Full suite (35 + loopback round-trip + 6) + clippy + fmt green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
From the latency investigation: the freeze-fix pacing (paced_submit) was the
single biggest software-controllable latency term — it unconditionally spread
EVERY multi-chunk frame over ~90% of the frame interval, adding up to ~7.5 ms
@120 / ~15 ms @60 to a frame's last packet even when the frame was small or the
link idle. Recover that on the common case while keeping the freeze fix:
- Microburst-cap pacing: a frame whose sealed size is <= a cap (default 128 KB,
PUNKTFUNK_PACE_BURST_KB) goes out in ONE immediate burst — no pacing latency.
Only the OVERFLOW of a bigger frame (IDR / sustained high bitrate, the bursts
that actually overran the tx buffer and froze) is spread. 128 KB is well under
the ~150 Mbps@60 frame size where drops began, so the default is safe; raise it
after confirming send_dropped stays 0 on a given link. Still never slower than
unpaced (budget collapses to 0 with no slack). seal-once/in-order nonce
preserved — chunks are split, never reordered or re-sealed.
- Per-frame instrumentation (PUNKTFUNK_PERF, zero-cost off): encode_us +
pace_us (the pacing tail) p50/p99/max histograms + immediate-vs-paced frame
counts in the periodic perf line, so the pacing tail is finally visible and the
cap is tunable against real numbers.
Host builds + clippy + fmt green. NOT yet deployed to the running hosts (still on
the safe full-pacing A+B build) — needs the user's LAN soak to validate the cap
doesn't reintroduce send_dropped before raising it. Deferred bigger bets (need
real-NIC/GPU/Mac validation): encode|send thread split on the native path,
CUDA stream+event (one redundant sync), NVENC slice wrapper, stage-2 Apple
presenter, glass-to-glass probe — see docs/roadmap.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Final increment of the 1 Gbps data-plane rework — the recv counterpart of the
sendmmsg work. The client recv path did one recvfrom + one Vec allocation per
packet (and the pump's 300µs idle sleep could let packets pile up at line rate).
- Transport gains recv_batch(&mut [Vec<u8>], &mut [usize]) -> count; default is
a single scalar recv into out[0] (loopback + non-Linux).
- UdpTransport overrides it on Linux with recvmmsg (MSG_DONTWAIT) draining up to
N datagrams per syscall into the caller's reused buffers — no per-packet alloc.
- Session::poll_frame owns a lazily-allocated recv ring (RECV_BATCH=32) and
consumes it one packet at a time across calls, refilling with one recvmmsg when
drained. Encapsulated: the punktfunk-client-rs + NativeClient pumps are
unchanged, and draining a batch per syscall means the 300µs sleep no longer
underdrains. Added UdpTransport::local_addr (used by the test, generally handy).
~125k → ~4k recv syscalls/sec at line rate, zero per-packet recv allocation.
Verified: new recv_batch_drains_over_loopback test (50 datagrams drained intact
via recvmmsg) + the existing loopback round-trip now runs through the batched
poll_frame; full suite (35 + round-trip + 6) + clippy + fmt green.
Decode-in-place (kill the per-packet open_from_wire alloc) is a separate later
optimization. With A (sendmmsg) + B (paced send) + C (recvmmsg), the native data
plane is batched + paced end to end.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Increment B of the send-path rework — the actual fix for "freezes get more
common over ~150 Mbps, no image at all at 400 Mbps" on the native path. Cause:
the encoder emits a frame and submit_frame blasted ALL its packets at once into
the NIC; a real link drops the line-rate burst (host send buffer EAGAINs), and
under infinite GOP one dropped frame freezes the decode until the next keyframe.
(The speed-test probe showed 0 drops at 400 Mbps because the probe is self-paced;
real video wasn't.)
Adaptive pacing, no extra thread, no regression:
- Session splits into seal_frame (FEC + packetize + seal → wire packets, no
send) and send_sealed (one batched sendmmsg of a chunk, counts drops);
submit_frame is now their composition (synthetic + probe paths unchanged).
- virtual_stream's paced_submit seals a frame then sends it in 16-packet chunks
spread over ~90% of the time until the next frame is due. At 60 fps desktop
(fast encode → lots of slack) the frame spreads across the interval → no NIC
burst → no freeze. At 240 fps@5K (encode ≈ interval → ~0 slack) the budget
collapses and every chunk goes out immediately → never slower than before.
Core suite (34 + loopback round-trip + 6) + clippy + fmt green. The seal/send
split is covered by the existing loopback tests; the pacing is host timing,
verified by review (live-test needs a real NIC — your Mac at a raised bitrate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler