The previous attempt (97ee75d) dropped zero-copy on Mutter+NVIDIA for a sticky
CPU/SHM fallback that (a) still listed SPA_DATA_DmaBuf in its buffer types, so
Mutter kept handing dmabufs that got mmap-read UNsynced — making the flashing
worse, not better — and (b) hinged on producer explicit sync, which Mutter+NVIDIA
cannot do (`error alloc buffers` / no cogl sync_fd, confirmed in worker-3 logs).
Revert the capture restructure to the original zero-copy dmabuf path, and fix the
NVIDIA stale-frame race the RIGHT way for a producer that can't do explicit sync:
the consumer snapshots the dmabuf's implicit fence (DMA_BUF_IOCTL_EXPORT_SYNC_FILE)
and waits the producer's render before sampling (new dmabuf_fence module, ioctl
number unit-tested). Covers the GPU import and the CPU mmap read. Logs once whether
a render was actually in flight (waited=true → the driver fences and the race is
closed; false → no implicit fence, so we learn zero-copy still needs SHM here).
drm_sync (the explicit-sync primitive) is kept and verified but marked unused —
no targeted compositor produces a usable sync_fd today; ready to wire in when one
does. The Bug-2 input fix (held-key release on disconnect) from 97ee75d is kept.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deep dive into the two GNOME-only host bugs (KWin/gamescope clean):
1. Stale-frame flashes (windows at old positions, typed text reverting):
Mutter renders its virtual monitors DIRECTLY into the PipeWire buffer
pool, and NVIDIA has no implicit dmabuf fencing — our zero-copy
import raced the render and encoded each pool buffer's PREVIOUS
contents. Fix, in order of preference:
- Consumer-side PipeWire explicit sync (SPA_META_SyncTimeline): new
drm_sync module (DRM timeline-syncobj wait/signal via raw ioctls,
unit-tested incl. a live signal->wait round trip); announced
post-format via update_params (the OBS pattern — at connect time
the meta makes producers fail allocation, observed on KWin), with
a blocks=3 Buffers filter so the producer's sync pod wins; acquire
point awaited before any read (GPU import or CPU mmap), release
point signaled on every path.
- Where the producer can't do explicit sync (Mutter on NVIDIA today:
no cogl sync_fd, "error alloc buffers"), a sticky fallback flips
the capture to the synchronous CPU/shm path — Mutter's glReadPixels
download orders against its render, so frames are correct by
construction. First session pays one ~10 s probe+retry; later
sessions go straight there. Validated live on home-worker-3
(GNOME 50 + RTX 4090): clean fallback, 30 MB HEVC streamed.
- Sync is only announced on Mutter sessions (new VirtualOutput.mutter
tag): KWin+NVIDIA fails allocation when merely asked, and doesn't
need it (verified unchanged: zero-copy CUDA import + 1.1 MB/10 s).
PUNKTFUNK_EXPLICIT_SYNC=0 disables the probe outright.
2. Clicks wedged in the focused app after disconnect+reconnect: a client
vanishing mid-press left keys/buttons latched in the compositor —
Mutter keeps the destroyed EIS device's implicit grab and the focused
app stops taking clicks until restarted. EiState now tracks held
keys/buttons/touches (wire codes) and synthesizes releases through
the normal inject path before the EIS connection goes away.
GNOME hosts on NVIDIA temporarily lose zero-copy (correctness over
throughput); the moment Mutter+driver gain working explicit sync, the
sync path engages automatically and zero-copy returns.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
macOS/iOS have no recvmmsg(2), so the Mac client did one recv() syscall per
packet (non-allocating after the earlier fix, but still a syscall each — a
single-core wall at line rate that Moonlight avoids). Add the Darwin recvmsg_x(2)
batched-receive path (the recv counterpart of Linux recvmmsg): one syscall drains
up to RECV_BATCH datagrams into the reused ring. struct msghdr_x + the extern
aren't in the libc crate, so declared here (cfg target_vendor=apple).
Opt-in via PUNKTFUNK_RECVMSG_X (it's FFI we can't exercise off-Apple) with
auto-fallback to the tested scalar recv-loop on any unexpected error. Linux
recvmmsg + the non-Apple scalar loop are unchanged; apple.yml compiles the path.
Re GRO: Linux recv already batches via recvmmsg (32/syscall), so UDP GRO is only a
marginal add there and needs a recv-path redesign to split coalesced buffers —
deferred as low-ROI vs the Mac, which had no batching at all.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The host sealed every packet with ~3 heap allocations: aes-gcm's convenience
encrypt() allocates the ciphertext Vec, seal_for_wire allocates the seq||ct||tag
wire Vec, and seal_frame allocated a fresh Vec<Vec<u8>> per frame. At line rate
(~250k–500k pkt/s for 2.5–5 Gbps) that's the single-core allocator wall.
- SessionCrypto::seal_in_place uses AeadInPlace::encrypt_in_place_detached to
encrypt into the caller's buffer and write the detached tag at the end —
byte-identical to seal's ciphertext||tag, no allocation (unit-tested for byte
equality + decrypt).
- Session keeps a wire_pool the caller returns via reclaim_wires; seal_frame
seals each packet in place into the reused buffers (clear() keeps capacity), so
after warmup there's no per-packet ciphertext/wire allocation. paced_submit and
submit_frame reclaim the pool after sending.
End-to-end encrypted/lossless multi-frame tests stay green (validates the pool
reuse doesn't corrupt across frames). Next: write packetize directly into a
contiguous send buffer (kills the remaining shard allocs + GSO's coalescing copy).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sendmmsg already batches syscalls but still builds one sk_buff per datagram —
the kernel-side wall above ~1 Gbps. UDP Generic Segmentation Offload hands the
kernel one big buffer it splits into gso_size datagrams, building ~1 GSO skb per
≤64 segments. Research (LWN/Cloudflare/Tailscale) measures ~2.4x throughput at
equal CPU and 17-44x fewer syscalls, and that sendmmsg batching alone is
insufficient — you need true segmentation offload.
Adds Transport::send_gso (default = send_batch) + a UdpTransport Linux override:
coalesces a frame's equal-size wire packets (shards are zero-padded to a constant
size, so a whole frame is one gso_size) into ≤64-segment sendmsg(UDP_SEGMENT)
calls. seal/send routes through it. Opt-in via PUNKTFUNK_GSO (new unsafe hot-path
code) with automatic fallback to sendmmsg on any GSO error (unsupported kernel/
path), latched per process. Loopback unit test validates the cmsg segmentation;
full session over loopback streams clean (0% loss). Linux-only; loopback/non-Linux
keep sendmmsg/scalar.
Next levers: in-place AES-GCM seal (kill per-packet allocs), UDP GRO on recv,
drop the sleep-pacing in favor of the kernel qdisc, jumbo MTU.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stage 1.5: on Intel/AMD clients libavcodec's VAAPI hwaccel decodes on
the GPU; frames map to DRM-PRIME dmabufs (av_hwframe_map, zero copy)
and reach GTK as GdkDmabufTexture (BT.709 limited CICP color state —
GDK's dmabuf default is BT.601). Inside GtkGraphicsOffload that is the
decoder-to-subsurface path, direct-scanout eligible when fullscreen.
Fallback ladder, live-verified on the NVIDIA dev box: no VAAPI device
-> software decode at session start (logged reason); a mid-session
VAAPI error (e.g. broken nvidia-vaapi-driver) demotes to software and
the host's IDR/RFI recovery resynchronizes; a rejected dmabuf import
logs and the stream continues. PUNKTFUNK_DECODER=software|vaapi
overrides; the first-frame log now names the active path.
The hwaccel path is raw ffmpeg-sys FFI (ffmpeg-next wraps none of it):
hw device ctx + get_format pinned to AV_PIX_FMT_VAAPI (NONE on
mismatch so cpu-fallback never silently engages inside libavcodec),
thread_count=1, LOW_DELAY. Surface lifetime rides DrmFrameGuard into
the texture's release func — GDK runs it on both success and failure.
Needs an Intel/AMD client box (Steam Deck/Bazzite) to live-verify the
hardware path; the software path is unchanged and revalidated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Groundwork for multi-Gbps (2.5G link here, 5G to the Mac Studio). The encoder is
pixel-rate bound, not bitrate bound, so these unblock the transport:
- MAX_BITRATE_KBPS 2G -> 8G, MAX_PROBE_KBPS 3G -> 10G (the cap was policy, not a
hardware limit — NVENC emits multi-Gbps trivially with the 2-way split).
- Welcome shard_payload 1200 -> 1452: fills a 1500 MTU, ~17% fewer packets for
free (even size, FEC-safe; negotiated so the client follows).
- PUNKTFUNK_FEC_PCT env overrides the 20% FEC default — a clean wired LAN can drop
it (every recovery shard is wire bytes+packets); 0 disables FEC.
Next: UDP GSO (the dominant lever — research shows ~2.4x throughput / ~40x fewer
syscalls; sendmmsg batching alone is insufficient) + in-place AES-GCM seal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ac80bff added a libc-based batched recv_batch for the Apple/BSD targets
(cfg(all(unix, not(target_os = "linux")))) but left libc declared only under
cfg(target_os = "linux"). The macOS host build pulls libc in transitively so it
compiled, but the iOS/tvOS cross-compiles (no transitive libc, dev-deps off) failed
with E0433 "cannot find crate libc", breaking the full xcframework build. Widen the
gate to cfg(unix): libc is now used by sendmmsg/recvmmsg on Linux AND recv() on the
other unix (Apple/BSD) targets.
Verified: cargo build --release -p punktfunk-core --features quic for
aarch64-apple-ios, x86_64-apple-ios, and aarch64-apple-tvos (-Z build-std) all link.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The batched `recvmmsg` recv path was Linux-only; macOS fell back to the trait
default, which calls the scalar `recv` — a fresh `vec![0u8; 2049]` allocation
(plus zeroing and a copy) PER PACKET on the single receive thread. At line rate
that alloc/free churn, not the syscall, was the single-core wall: measured the
real Mac client topping out ~315 Mbps and dropping the session at 800, while a
Linux client (recvmmsg) held a clean 1 Gbps against the same host, and Moonlight
(batched recv) does 900 on the same Mac.
Add a `cfg(all(unix, not(linux)))` `recv_batch` that drains up to RECV_BATCH
datagrams per call with `libc::recv(MSG_DONTWAIT)` straight into the caller's
reused ring buffers — no per-packet allocation or copy. Still one syscall per
datagram (a future `recvmsg_x` batch would cut that too), but it removes the
dominant cost. Linux recvmmsg path and the Windows/loopback default unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes the intermittent first-connect freeze. The host streams infinite GOP — one
opening IDR, then P-frames only (recovery keyframes just on loss) — so when the
client's decoder wedges on the cold first session (a lost/corrupt opening IDR, a
bad early P-frame) the picture stays frozen until the far-off next keyframe. The
client had no way to ask for one; now it does.
Add a RequestKeyframe control message (client -> host, reliable control stream),
mirroring Reconfigure:
- core: quic.rs RequestKeyframe (type 0x03) + roundtrip test; client.rs
CtrlRequest::Keyframe + NativeClient::request_keyframe; abi.rs
punktfunk_connection_request_keyframe (header regenerated).
- host: m3.rs decodes it in the control loop and signals the encode loop, which
coalesces a burst and calls enc.request_keyframe() — wiring the existing
NvencEncoder hook (force_kf -> next frame pict_type=I), the same recovery the
GameStream path already had via force_idr.
- apple: PunktfunkConnection.requestKeyframe(); StreamPump (stage-1) requests on
layer.status==.failed; Stage2Pipeline (stage-2) on a sync submit failure and on
the async decode-error callback via a thread-safe KeyframeRecovery. All
throttled to <=1/250ms (the decode stays wedged for several frames until the IDR
lands, so per-frame requests would flood the control stream).
Self-healing: a lost recovery IDR is re-requested after the throttle; the host
coalesces bursts into a single IDR.
Validated: cargo fmt + clippy clean; core + host test suites green (incl. new
request_keyframe_roundtrip); swift build + test (39 passed); xcframework rebuilt
(all 5 slices), header regenerated with no unrelated drift.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Everything the macOS app does that stage 1 lacked, before any new
feature work (user directive):
- Input capture is now a deliberate, reversible STATE (Moonlight-
style): engaged on stream start and click-into-video (the engaging
click is suppressed), released by Ctrl+Alt+Shift+Q (toggles) or
focus loss; held keys/buttons are flushed host-side on release;
cursor hiding + shortcut inhibition follow the state; HUD hint when
released. Per-session window handlers disconnect with the page.
- Gamepads: app-lifetime SDL service (GamepadManager parity) — pad
list + "Forwarded controller" pin in Settings (auto = most recent),
"Automatic" pad TYPE resolves from the physical pad at connect;
DualSense touchpad contacts + ~250 Hz motion samples on the 0xCC
plane (Swift GamepadWire scale constants); feedback grows adaptive-
trigger replay and player LEDs via raw DS5 effects packets (the
wire's 11-byte blocks drop into SDL_SendGamepadEffect verbatim);
held pad state zeroed on pad switch/detach. sdl3 "hidapi" feature.
- Microphone uplink: PipeWire capture -> Opus 20 ms -> 0xCB datagrams
(validated live: host received 711 mic packets), Settings toggle.
- Speed test per saved host (Swift's "Test Network Speed…"): 2 s
probe burst, goodput/loss + recommended ~70 % bitrate, one-tap apply.
- Settings: host compositor preference (sent in the Hello), native-
display resolution/refresh resolved from the window's monitor at
connect (new default), bitrate ceiling to 3 Gbit/s.
- Hosts page: saved/trusted hosts section for direct pinned reconnect
(mDNS not required), rebuilt on every page return.
Deliberately not ported: audio device pickers (PipeWire routing owns
this on Linux), resize-to-request_mode (not wired in Swift either),
pointer-lock relative mouse (stage-2 presenter, needs raw Wayland).
DualSense fidelity needs a physical pad to live-verify.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three native-client bugs isolated against a stock Moonlight client (which
stays connected / keeps input working under the same actions):
- Connection drops mid-stream: the quinn endpoints (host + client) ran with
default transport config, so keep_alive_interval was OFF. Any quiet stretch
(no input, audio muted/stalled, a capture hiccup, a mode change) let the
idle timer expire and quinn closed the session -> next_au=Closed -> "Session
ended". Moonlight's ENet sends keepalive pings; we sent nothing. Add a shared
TransportConfig (keep-alive 4s under an explicit 20s idle timeout) to both
endpoint::server_from_der and endpoint::client_pinned_with_identity.
- Reconnect input dead (macOS): the session-start auto-capture one-shot was
consumed even when engageCapture(fromClick:false) was refused (window not key
yet at the instant of reconnect), with no retry -> capture stayed off and
input never forwarded. Clear the one-shot only on a successful engage, and
retry on NSWindow.didBecomeKey. Stays scoped to session start, so it does not
resurrect the rejected auto-grab-on-activation behavior.
- Reconnect input dead (iOS): wasCapturedOnResign leaked stale state across
sessions and the foreground-restore could fire before this session's
InputCapture was wired (setForwarding no-ops on nil). Reset it per session in
start() and guard the didBecomeActive restore on inputCapture != nil.
Validated: cargo build -p punktfunk-core --features quic; swift build;
swift test (39 passed, 0 failures); xcframework rebuilt (all 5 slices), no
ABI/header drift.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
NVENC ran CBR (bit_rate == max_bit_rate, rc=cbr) but never set rc_buffer_size,
so it used a loose default VBV. A high-motion P-frame was then allowed to spike
to many times the average frame size; the extra packets overflow the depth-2
send queue (newest frame dropped) and the kernel UDP buffer (WouldBlock drops),
which the client sees as framedrops/jitter — and on the infinite-GOP GameStream
path as old/stale frames flashing until the next RFI.
Set a tight ~1-frame VBV (rc_buffer_size = bitrate/fps) so the encoder holds
frame size roughly constant and absorbs motion as a momentary QP/quality dip
instead — the Sunshine/Moonlight low-latency model. Tunable via
PUNKTFUNK_VBV_FRAMES (default 1.0); larger trades burst tolerance for motion
quality. Fixes both the punktfunk/1 and GameStream paths (shared encoder).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MouseMoveAbs wire contract packs the client coordinate-space size
as (width << 16) | height in `flags` (same as touch); injectors
normalize against it and drop the event when it is zero. The GTK
client sent flags=0, so KWin's libei path refused every motion
(`emitted=false`) — found via the first real user test from
home-worker-3.
- ui_stream: send_abs() packs the negotiated mode into flags for
motion + click-position events.
- core input.rs: document the contract on MouseMoveAbs itself (it was
only implied by TouchDown's doc).
- client-rs --input-test: add a MouseMoveAbs sweep so the absolute
path stays covered — Moonlight and the Mac client only send relative
motion, which is why this gap survived every prior live test.
Validated live against serve --native: kind=MouseMoveAbs emitted=true.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New crate crates/punktfunk-client-linux (binary punktfunk-client), the
native Linux client on the Option A architecture (2026-06-12 research):
- GTK4/libadwaita shell linking punktfunk-core directly (no C ABI):
mDNS host list, TOFU fingerprint prompt, SPAKE2 PIN pairing dialog,
preferences (mode/bitrate/gamepad/shortcut capture), stats overlay,
--connect host[:port] for scripting.
- Video: FFmpeg software HEVC decode (LOW_DELAY, slice threads) ->
RGBA -> GdkMemoryTexture inside GtkGraphicsOffload (the dmabuf
subsurface path lights up when VAAPI lands; black-background keeps
fullscreen scanout-eligible).
- Audio: Opus -> PipeWire playback stream, the host virtual-mic's
adaptive jitter ring inverted.
- Input: keyboard as the exact inverse of the host VK table (evdev
keycodes, layout-independent; unit-tested), absolute mouse through
the Contain-fit transform, WHEEL_DELTA(120) scroll, compositor
shortcut inhibition while streaming, Ctrl+Alt+Shift+Q release chord,
F11 fullscreen. SDL3 gamepad capture (single pad-0 model) + rumble
and DualSense lightbar feedback on the same thread.
- Session pump owns video+audio pulls; the gamepad thread owns
rumble+hidout — possible because NativeClient's plane receivers are
now mutexed, making it Sync (Arc-shared, compiler-verified per-plane
contract instead of the ABI's manual assertion).
- Linux-gated deps + a stub main keep cargo build --workspace green on
macOS.
Validated live against serve --native on this box: 1920x1080@60,
locked 60 fps, capture->decoded p50 ~6.4 ms (software decode, debug
build). Teardown keys off AdwNavigationPage::hidden — NavigationView
push fires a transient unmap/map cycle that must not end the session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An identified-but-unpaired device that knocks on a pairing-required host is now
held as a pending request the operator approves from the web console — pairing it
with no PIN fetched out of band — instead of a flat reject.
- core: Hello gains an optional trailing device name (len u8 || UTF-8, ≤64,
same trailing-back-compat pattern as compositor/gamepad/bitrate). client-rs
--name sends it; the connector sends None (fingerprint-derived label).
- native_pairing: in-memory pending queue (note_pending dedups by fingerprint,
evicts the least-recently-active past a 32 cap, 10-min TTL); approve_pending
pins the fingerprint, deny drops it. Names are sanitized (strip control/ANSI/
bidi — untrusted wire input); add()/remove() roll back in-memory on a persist
failure; pairing clears any stale pending knock.
- m3: the require_pairing gate records the knock (sanitized label) before
rejecting; anonymous (certless) clients record nothing.
- mgmt: GET /native/pending, POST /native/pending/{id}/approve (optional {name})
and /deny; OpenAPI + tests; docs/api/openapi.json regenerated.
- web: a "Waiting for approval" section on the Pairing page (live-poll, Approve/
Deny, error-surfaced via QueryState); en+de strings.
- Also completes an in-progress NativeClient Sync refactor (receivers behind
per-plane mutexes) that was left half-applied in the tree.
Adversarially reviewed (4 lenses + 3-vote verify); the confirmed findings are
fixed here. Validated live on the GNOME box: knock (with a wire name, and a
malicious ANSI/bidi name that got neutralized) → pending → approve → the same
identity streams real video. Full workspace tests + clippy + fmt green; web tsc
clean. Roadmap §8b-1 marked done; §8b-2 (peer-push approval) is the client
follow-up. See docs-site pairing page.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The high-refresh teardown SIGSEGV was caused by ApplyMonitorsConfig disabling the
still-actively-captured high-refresh virtual output. Reorder teardown: Stop the screencast
FIRST (Mutter removes the virtual + auto-reverts the temporary config), then re-assert the
physical layout once the virtual is gone. Never reconfigure a live virtual CRTC.
With this, PUNKTFUNK_MUTTER_VIRTUAL_REFRESH=1 is stable: validated at 5120x1440@240 on
Mutter 50 + NVIDIA — virtual output Meta-0@240, real 240fps, gnome-shell survives back-to-back
sessions + teardowns, physical (HDMI-1) restored each time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pinning the virtual output to a high client refresh via RecordVirtual "modes" works
mid-stream, but a high-refresh virtual CRTC SIGSEGVs gnome-shell on session TEARDOWN
(observed at 5120x1440@240) — taking down the whole GNOME session, so subsequent connects
fail with RemoteDesktop ServiceUnknown.
Gate it behind PUNKTFUNK_MUTTER_VIRTUAL_REFRESH, default OFF — Mutter then derives the
virtual monitor's refresh from the PipeWire framerate (60Hz, stable). The >60Hz path stays
in-tree for investigation; re-enable once the teardown crash is understood.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RecordVirtual without a "modes" property makes Mutter derive the virtual monitor's refresh
from the PipeWire stream framerate and default to 60 Hz — so a 240 Hz client mode rendered
at 60 (the encoder just padded to 240 with duplicate frames). Pass an explicit "modes" entry
(size + refresh-rate + is-preferred) so Mutter creates the virtual monitor at the client's
exact WxH@Hz. Mutter >= 47; older Mutter ignores the unknown key (60 Hz fallback, no regression).
Confirmed first via raw D-Bus on the box, then validated end-to-end: the virtual output
Meta-0 reports 1920x1080@240.00 and the host encodes 480 *immediate* (real, not paced)
frames per 2 s.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Keeping the physical monitor enabled as a secondary let the cursor, windows, and keyboard
focus land on it — relative pointer motion wandered off the streamed surface, so on the
client the cursor "disappeared" and clicks/keys went nowhere visible. Omit the physical
outputs from ApplyMonitorsConfig so Mutter disables them for the session; everything is
confined to the streamed virtual output. Restored on teardown.
Validated on-box: mid-session DisplayConfig shows only the virtual output (Meta-0) as the
sole primary; the physical (HDMI-1) is restored after the session ends.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a headless GNOME host the xdg-desktop-portal RemoteDesktop Start() blocks on an
interactive "Allow remote control?" approval nobody can click, so libei input timed out
("EIS setup timed out") and neither mouse nor keyboard worked — even though video worked
(it uses Mutter's direct RemoteDesktop API).
Add EiSource::MutterEis: obtain the EIS fd from
org.gnome.Mutter.RemoteDesktop.Session.ConnectToEIS (CreateSession → Start → ConnectToEIS),
no portal and no approval. Selected for GNOME/Mutter; KWin keeps the RemoteDesktop portal,
gamescope keeps its own EIS socket.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The accept loop no longer awaits each session inline — it spawns each onto a
JoinSet, bounded by a semaphore (--max-concurrent, default 4: a NVENC session
bound; overflow clients wait in QUIC's accept backlog until a slot frees). The
QUIC handshake stays in the accept loop so a failed handshake (e.g. a pin
mismatch where the client aborts) doesn't consume a session slot or block
accepting the next client; the slow part (control handshake, pairing, the
capture/encode pipeline) runs in the spawned task.
Each session already had its own virtual output + NVENC encoder; the
host-lifetime input/audio/mic services stay shared — the natural "multiple
devices viewing/controlling the same desktop" semantic on kwin/mutter/wlroots.
gamescope's independent-desktops (per-session input/audio) isolation is a
follow-up. New M3Options.max_concurrent + the `--max-concurrent` CLI flag.
Validated live (GNOME box): two clients connected at once -> two independent
Mutter virtual outputs (720p60 + 1080p60) streaming simultaneously (39 MB +
48 MB). All 61 host tests green (the c_abi/pairing tests exercise the new loop +
the failed-handshake-doesn't-count semantics).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment reflow only — the pinned "stable" channel moved and CI checks
formatting with the current toolchain.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PUNKTFUNK_MUTTER_VIRTUAL_PRIMARY=1: after RecordVirtual, promote the per-session
virtual output to the primary monitor (physical kept on, secondary) via
org.gnome.Mutter.DisplayConfig.ApplyMonitorsConfig, restoring on teardown.
Without it, a GNOME host that also has a physical monitor attached keeps the physical
primary, so the virtual output is an empty extended desktop — the client streams only
the wallpaper. (The backend was validated on headless GNOME, where the virtual output
is the only display.)
Best-effort + opt-in: default behavior is unchanged; any DisplayConfig failure just
logs and streaming continues. method=temporary, so nothing is written to monitors.xml
and Mutter auto-reverts the layout when the virtual output is torn down.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Apple speed test asked for only 400 Mbps, capping the measured throughput
there and hiding the link's real headroom. Request the host's full
MAX_PROBE_KBPS (3 Gbps) instead, and raise the recommended-bitrate clamp from
500 Mbps to the host's 2 Gbps session ceiling so a fast measurement yields a
usable recommendation.
Also fix the stale caps left when the host clamps were raised (0cd02e7): the
resolved-bitrate range and the probe doc comments (abi.rs, client.rs,
regenerated header), plus the section 9 roadmap copy, now read 3 Gbps probe /
2 Gbps session.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The clock_offset test's assert_eq! carried an inline message that newer rustfmt
wants to wrap while the repo's committed style keeps such asserts on one line.
Move the message to a comment and use bare assert_eq! so it formats identically
under any rustfmt version — no new fmt-check ambiguity from this addition.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Factor the client-side skew handshake into a shared core helper (quic::clock_sync
-> ClockSkew) so both the reference client and the embeddable connector use one
implementation. NativeClient now runs the handshake at connect (right after Start,
before the control task takes the stream) and stores the host-client offset; it's
read over the C ABI via punktfunk_connection_clock_offset_ns (i64 ns, host minus
client; 0 = no correction / old host).
This is the substrate the Apple client needs for the decode->present (glass-to-
glass) term: stamp present time, add the offset to express it in the host's
capture clock, subtract the AU pts_ns. client-rs drops its local clock_sync copy
and uses the shared helper (behavior unchanged; validated locally).
Regenerates include/punktfunk_core.h. Roadmap section 12 + status updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ClockProbe/ClockEcho on the QUIC control stream — 8 NTP-style rounds right after
Start; the min-RTT sample gives the host-client clock offset (clock_offset_ns
estimator in punktfunk-core). The client adds the offset to its receive instant
before differencing against the AU pts_ns, so the capture->reassembled latency
percentiles are valid across machines (skew_corrected=true), not just same-host.
Back-compat: an old host that doesn't answer the probe times out and the client
falls back to a shared-clock assumption (skew_corrected=false).
Host adds one ClockProbe dispatch arm in the control task; the client runs
clock_sync after Start, before the --remode/--speed-test tasks take the stream.
Validated cross-LAN (GNOME box -> dev box): offset ~ -1.57 ms (reproducible),
rtt ~140 us, p50 1.30 ms skew-corrected capture->reassembled — the offset is
exactly the systematic error the handshake removes. Unit tests for the message
codecs and the min-RTT offset estimator.
Roadmap §12: skew handshake done; remaining for true glass-to-glass is the Apple
client present-stamp (decode->present) plus the host render->capture term.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both the unified host (serve --native) and standalone m3-host now advertise the
native punktfunk/1 service over mDNS (_punktfunk._udp) — the analogue of the
GameStream _nvstream._tcp advert. TXT records carry proto, the host cert
fingerprint (fp, the value clients pin), the pairing requirement
(pair=required|optional), and the host id. New crate::discovery module, wired
into m3::serve so both host entry points get it; best-effort, never blocks
streaming (--connect always works).
Client gains `punktfunk-client-rs --discover [SECS]`: browses the LAN and prints
each host (name, addr:port, pairing, fingerprint), then exits. Apple clients
browse the same service natively via NWBrowser (service type + TXT keys are the
contract).
Validated cross-LAN: the dev box discovered the GNOME-box appliance
(pair=required) and a standalone synthetic host (pair=optional); fingerprint and
pairing state correct in both.
Also refresh the now-stale sendmmsg caveat in the bitrate doc (batched/paced send
landed + validated to 1 Gbps) and mark the encode|send thread split done in §12.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bigger-bet #1 from the latency plan. virtual_stream ran capture+encode+seal+
paced-send on ONE thread, so frame N+1's capture/encode couldn't start until
frame N's entire paced tail had left the wire — the pacing budget (~0.9×interval)
was serialized in front of the next encode. Port GameStream's spawn_sender model
to the native path:
- A dedicated send thread (`send_loop`) owns the WHOLE Session (so no socket
clone or shared/Arc stats needed — `seal_frame` mutates the nonce, `send_sealed`
+ the probe bursts all live there) and does FEC+seal + microburst-paced send.
- The encode thread captures+encodes + handles reconfig and hands each AU over a
bounded sync_channel(3) as a FrameMsg (data, capture_ns, flags, deadline,
encode_us). It BLOCKS on backpressure if the send falls behind — frames slow
down rather than a dropped frame freezing the infinite-GOP stream (we don't
drop). Clean shutdown: drop the channel → send thread drains/exits → join.
- Probes (run_probe_burst) move to the send thread since they need the Session; a
burst naturally pauses video (the encode thread blocks on the full channel).
- Per-frame encode_us/pace_us histogram moved to the send thread (carries
encode_us in the FrameMsg) and now reflects the overlap.
Removes the encode↔paced-tail serialization (~2-8 ms @60-120 fps), independent of
the pacing policy, no quality cost. Substrate for the future NVENC slice wrapper.
Verified live on this box (appliance restarted onto it): a client streamed the
KWin desktop (1.49 MB H.265, clean, no panic) and a 200 Mbps speed-test probe
completed through the send thread (0 drops). Build + clippy + fmt green.
Real-NIC sustained soak (reconfig under load, line-rate, mode switches) pending
the Ubuntu third host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
From a bug-hunt + unsafe-audit pass (4 reviewers + adversarial verify). It
confirmed ZERO real bugs in the recent batched/paced data-plane work — these are
the surfaced cleanups + one genuine soundness fix:
- SOUNDNESS (reduce unsafe): inject/gamepad.rs::pump_ff did `ptr::read` of an
InputEventRaw (align 8, holds a timeval) out of a 1-aligned [u8; N] buffer — UB
per the reference (x86_64 tolerates it, but it can miscompile under LTO). Use
ptr::read_unaligned + a SAFETY note. Zero behavior change.
- recv parity: recv_batch (recvmmsg) didn't drop an oversized/truncated datagram
the way scalar recv does — poll_frame now skips a message whose len fills the
buffer (> MAX_DATAGRAM_BYTES), matching recv's `n >= RECV_BUF` drop. (AEAD
already rejected these on encrypted sessions; this restores the documented
invariant on the batched path.)
- dedup unsafe FFI: factor the identical mmsghdr-from-iovec construction out of
send_batch + recv_batch into one `mmsghdrs()` helper — the raw-pointer
scaffolding + its lifetime SAFETY note now live in one place.
- docs: TARGET_SOCKBUF no longer calls paced sending future work (it landed,
m3.rs::paced_submit); gamescope.rs input is no longer "(TODO)" (wired +
live-validated); the PUNKTFUNK_PERF `wire_mbps` field is renamed `tx_mbps` and
noted as attempted/sealed bytes (send_dropped shows what didn't reach the wire).
Full suite (35 + loopback round-trip + 6) + clippy + fmt green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
From the latency investigation: the freeze-fix pacing (paced_submit) was the
single biggest software-controllable latency term — it unconditionally spread
EVERY multi-chunk frame over ~90% of the frame interval, adding up to ~7.5 ms
@120 / ~15 ms @60 to a frame's last packet even when the frame was small or the
link idle. Recover that on the common case while keeping the freeze fix:
- Microburst-cap pacing: a frame whose sealed size is <= a cap (default 128 KB,
PUNKTFUNK_PACE_BURST_KB) goes out in ONE immediate burst — no pacing latency.
Only the OVERFLOW of a bigger frame (IDR / sustained high bitrate, the bursts
that actually overran the tx buffer and froze) is spread. 128 KB is well under
the ~150 Mbps@60 frame size where drops began, so the default is safe; raise it
after confirming send_dropped stays 0 on a given link. Still never slower than
unpaced (budget collapses to 0 with no slack). seal-once/in-order nonce
preserved — chunks are split, never reordered or re-sealed.
- Per-frame instrumentation (PUNKTFUNK_PERF, zero-cost off): encode_us +
pace_us (the pacing tail) p50/p99/max histograms + immediate-vs-paced frame
counts in the periodic perf line, so the pacing tail is finally visible and the
cap is tunable against real numbers.
Host builds + clippy + fmt green. NOT yet deployed to the running hosts (still on
the safe full-pacing A+B build) — needs the user's LAN soak to validate the cap
doesn't reintroduce send_dropped before raising it. Deferred bigger bets (need
real-NIC/GPU/Mac validation): encode|send thread split on the native path,
CUDA stream+event (one redundant sync), NVENC slice wrapper, stage-2 Apple
presenter, glass-to-glass probe — see docs/roadmap.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Final increment of the 1 Gbps data-plane rework — the recv counterpart of the
sendmmsg work. The client recv path did one recvfrom + one Vec allocation per
packet (and the pump's 300µs idle sleep could let packets pile up at line rate).
- Transport gains recv_batch(&mut [Vec<u8>], &mut [usize]) -> count; default is
a single scalar recv into out[0] (loopback + non-Linux).
- UdpTransport overrides it on Linux with recvmmsg (MSG_DONTWAIT) draining up to
N datagrams per syscall into the caller's reused buffers — no per-packet alloc.
- Session::poll_frame owns a lazily-allocated recv ring (RECV_BATCH=32) and
consumes it one packet at a time across calls, refilling with one recvmmsg when
drained. Encapsulated: the punktfunk-client-rs + NativeClient pumps are
unchanged, and draining a batch per syscall means the 300µs sleep no longer
underdrains. Added UdpTransport::local_addr (used by the test, generally handy).
~125k → ~4k recv syscalls/sec at line rate, zero per-packet recv allocation.
Verified: new recv_batch_drains_over_loopback test (50 datagrams drained intact
via recvmmsg) + the existing loopback round-trip now runs through the batched
poll_frame; full suite (35 + round-trip + 6) + clippy + fmt green.
Decode-in-place (kill the per-packet open_from_wire alloc) is a separate later
optimization. With A (sendmmsg) + B (paced send) + C (recvmmsg), the native data
plane is batched + paced end to end.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Increment B of the send-path rework — the actual fix for "freezes get more
common over ~150 Mbps, no image at all at 400 Mbps" on the native path. Cause:
the encoder emits a frame and submit_frame blasted ALL its packets at once into
the NIC; a real link drops the line-rate burst (host send buffer EAGAINs), and
under infinite GOP one dropped frame freezes the decode until the next keyframe.
(The speed-test probe showed 0 drops at 400 Mbps because the probe is self-paced;
real video wasn't.)
Adaptive pacing, no extra thread, no regression:
- Session splits into seal_frame (FEC + packetize + seal → wire packets, no
send) and send_sealed (one batched sendmmsg of a chunk, counts drops);
submit_frame is now their composition (synthetic + probe paths unchanged).
- virtual_stream's paced_submit seals a frame then sends it in 16-packet chunks
spread over ~90% of the time until the next frame is due. At 60 fps desktop
(fast encode → lots of slack) the frame spreads across the interval → no NIC
burst → no freeze. At 240 fps@5K (encode ≈ interval → ~0 slack) the budget
collapses and every chunk goes out immediately → never slower than before.
Core suite (34 + loopback round-trip + 6) + clippy + fmt green. The seal/send
split is covered by the existing loopback tests; the pacing is host timing,
verified by review (live-test needs a real NIC — your Mac at a raised bitrate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First increment of the 1 Gbps send-path rework (the measured bottleneck): the
native data plane did one send() syscall per packet — at ~125k pkt/s (1 Gbps
wire) that burns a core on syscalls. Port the proven GameStream sendmmsg path
into the core Transport seam.
- Transport gains `send_batch(&[&[u8]]) -> usize` (count handed to the kernel;
caller counts the rest as send-buffer drops). Default = the scalar send loop
(loopback transport + non-Linux).
- UdpTransport overrides it on Linux with `sendmmsg` (64 datagrams/syscall);
the connected socket needs no per-message address. Non-blocking-aware: a full
send buffer yields a short count / EAGAIN, and we stop + report what went out
rather than block or retry (same lossy, FEC-protected contract as send()).
- Session::submit_frame seals every shard then hands the whole frame to
send_batch in ONE call instead of looping send() — ~64x fewer syscalls per
frame on the native + GameStream-over-core paths; send_dropped accounting
preserved (total - sent).
~125k → ~2k syscalls/sec at 1 Gbps line rate. Verified: new loopback-UDP test
send_batch_delivers_over_loopback (100 batched packets arrive intact, datagram
boundaries preserved); full core suite + clippy + fmt green.
Next increments: a paced send thread (microburst shaping so a real NIC doesn't
drop line-rate bursts) and recvmmsg on the client.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First step of 1 Gbps+ readiness (the whole point of the GF(2^16) Leopard FEC):
make 1 Gbps configurable and its dominant failure mode observable, before the
real transport work (sendmmsg + paced encode|send split) lands.
Investigation (6-way) verdict: we're ~halfway, and it's mostly clamps plus one
real piece of work. The integer/type path, FEC (a 1 Gbps frame is only a few
hundred shards in one GF(2^16) block, far under the 65535 ceiling), AES-GCM
(AES-NI, ~10-25x headroom), and the M1 reassembler bounds (fully derived from
the negotiated FecConfig) are ALL already 1 Gbps-ready and untouched.
This commit (the configurable + observable foundation):
- m3.rs: MAX_BITRATE_KBPS 500_000 -> 2_000_000 (2 Gbps headroom over the 1 Gbps+
target); MAX_PROBE_KBPS 1_000_000 -> 3_000_000 (probe can demonstrate headroom
ABOVE the session cap so a client can confidently pick a 1 Gbps+ bitrate).
- transport/udp.rs: TARGET_SOCKBUF 8 MB -> 32 MB (a multi-MB IDR keyframe burst
no longer fills the buffer); scripts/99-punktfunk-net.conf bumped to match.
- Observability: Transport::send now returns Ok(true|false) (false = WouldBlock
send-buffer drop, previously a silent Ok(())). Session counts these as a new
`packets_send_dropped` stat (distinct from recv-side packets_dropped) — in
Stats, the C ABI PunktfunkStats (header regenerated), a PUNKTFUNK_PERF periodic
wire-Mbps + drop dump in virtual_stream, and the speed-test probe completion
log. This is the dominant 1 Gbps+ loss mode and was invisible.
Loopback-verified: a probe now runs at 1.2 Gbps target (no longer truncated to
1 Gbps) with the drop counter live. NOT yet a sustained-1-Gbps proof — the
single-send()-per-packet native path is the next, real piece of work (port the
proven GameStream sendmmsg + paced send thread into the core Transport).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The uinput X-Box 360 backend logs "virtual gamepad created" on success, but
the UHID DualSense backend logged only on failure — so a working DualSense
session was silent and indistinguishable in the logs from one where no pad
was ever created. Add the matching success log.
This makes a DualSense-not-working report self-diagnosing: the host now logs
either "virtual DualSense created (UHID hid-playstation)" or the existing
"virtual DualSense creation failed — controller input disabled" (which fires
when /dev/uhid isn't writable — i.e. the 60-punktfunk.rules uhid rule isn't
installed or the user isn't in the 'input' group).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two related additions to the native protocol, host-side (the client side of
each is exposed over the C ABI so the platform clients can wire it up).
Bitrate negotiation
- Hello/Welcome carry `bitrate_kbps` (appended trailing-byte field, back-compat:
old peers decode 0 = host default). The client requests a rate; the host
clamps it to [500 kbps, 500 Mbps] (or its 20 Mbps default when 0) and echoes
the resolved value in Welcome. Replaces the hardcoded 20 Mbps NVENC bitrate in
m3.rs — threaded through virtual_stream → build_pipeline → open_video, applied
on the initial mode and every reconfigure rebuild.
- C ABI: punktfunk_connect_ex3(..., bitrate_kbps, ...) (ex2 delegates with 0);
punktfunk_connection_bitrate() reads the resolved value.
Speed test (bandwidth probe)
- New typed control messages ProbeRequest{target_kbps,duration_ms} (0x20) /
ProbeResult{bytes_sent,packets_sent,duration_ms} (0x21), plus a FLAG_PROBE
packet flag. The client asks the host to burst zero-filled, FLAG_PROBE-tagged
access units over the data plane at a target goodput for a duration (clamped
≤ 1 Gbps / ≤ 5 s), pacing by a bytes-allowed budget; video pauses for the
burst. The host reports what it actually sent; the client measures received
bytes + window → goodput and loss. Probe filler is never fed to the decoder
(diverted in the connector pump and the reference client's poll loop).
- The host control task now multiplexes Reconfigure + ProbeRequest (inbound)
and ProbeResult (outbound) over select!; a probe channel reaches the
data-plane thread (both virtual and synthetic sources).
- Connector: NativeClient::request_probe()/probe_result() with an internal
accumulator; C ABI punktfunk_connection_speed_test() +
punktfunk_connection_probe_result() → PunktfunkProbeResult.
- punktfunk-client-rs gains `--bitrate KBPS` and `--speed-test KBPS:MS` (its own
loop measures + logs goodput/loss) for loopback verification.
Validated on loopback (synthetic source): a 20 Mbps / 2 s probe measured
20050 kbps at 0% loss, bitrate negotiated (0→20000 and 50000→50000), and the
interleaved probe AUs were correctly excluded from frame verification
(mismatched=0). Wire codecs + trailing-byte back-compat have unit tests. C
header regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Nested games on the Bazzite host saw the wrong display: refresh capped at 60 Hz,
the box's connected TV's EDID modes leaking in (DOOM landed on 2560×1440@60), and
the resolution fixed at whatever the always-on session was launched at — the
client's requested mode never reached the game. Root causes: the session-plus
gamescope command has no --nested-refresh (Xwayland advertises 59.96 Hz for every
mode), --prefer-output HDMI-A-1 makes gamescope read the TV EDID, and the ATTACH
model launches one fixed-resolution session.
New vdisplay path: PUNKTFUNK_GAMESCOPE_SESSION=<client> — the host LAUNCHES
gamescope-session-plus headless AT THE CLIENT'S mode and relaunches it when the
mode changes. Injected via a host-written GAMESCOPE_BIN wrapper (--nested-refresh
$PF_HZ, the flag session-plus doesn't expose) + DRM_MODE=cvt (gamescope generates
clean CVT modes at that refresh instead of the TV's EDID). The session runs as a
transient `systemd-run --user` unit (clean cgroup teardown of the Steam tree);
state lives in a host-lifetime static (MANAGED_SESSION), NOT in GamescopeDisplay
(which is per-client-session) — so a same-mode reconnect REUSES the running
session instantly (no Steam restart) while a different mode RELAUNCHES it (games
can't change output mode live; a game/Steam restart on a mode change is
unavoidable and acceptable). Reuses the existing node + EIS auto-discovery
(find_gamescope_node / find_gamescope_eis_socket, factored into
point_injector_at_eis) and the existing mid-stream Reconfigure → vd.create(mode)
machinery — no protocol or m3 control-flow change.
Validated live on bazzite (RTX 4090): games' Xwayland now advertises 5120×1440 @
239.90 Hz as the preferred mode (was 59.96), the TV's 3840×2160/4096×2160@60 modes
are gone, frames stream; reconnect at 1920×1080@120 relaunches and games see that;
same-mode reconnect reuses with no restart and frames flow instantly.
scripts: host.env.example documents PUNKTFUNK_GAMESCOPE_SESSION (mutually exclusive
with the legacy NODE=auto attach); punktfunk-steam-session.service marked
deprecated (superseded — must not run alongside the host-managed path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Apple client grows full gamepad support and punktfunk/1 learns to negotiate
the virtual pad type:
- Protocol: Hello carries a GamepadPref byte (offset 21, the same trailing-byte
back-compat pattern as the compositor; echoed resolved in Welcome at 54).
Host precedence: explicit client choice > PUNKTFUNK_GAMEPAD env > Xbox 360,
DualSense (UHID) only where available. ABI: punktfunk_connect_ex2 +
punktfunk_connection_gamepad (connect_ex delegates; ABI_VERSION stays 2 — the
trailing byte IS the compat mechanism). punktfunk-client-rs gets --gamepad.
- Swift client: GamepadManager (app-lifetime discovery + selection — Settings
lists every controller with capabilities/battery/"In use"; exactly ONE pad
forwards as pad 0, auto = most recently connected, or pinned), GamepadCapture
(snapshot-diff button/axis events, DualSense touchpad + ~250 Hz motion on the
rich-input plane, held state released on switch/deactivate/stop),
GamepadFeedback (rumble → CoreHaptics per-handle engines; lightbar →
GCDeviceLight; player LEDs → playerIndex; adaptive-trigger blocks → the
table-driven DualSenseTriggerEffect parser → GCDualSenseAdaptiveTrigger,
exact for the 10-zone positional modes). The pad type auto-resolves from the
physical controller at connect time, user-overridable in Settings.
- Host DualSense fixes surfaced by adversarial review against hid-playstation /
SDL / Nielk1 ground truth: input-report sensor/touch offsets were off by one
(the kernel read garbage motion + phantom touches), the L2/R2 trigger blocks
were swapped (the report is right-trigger-first), feedback now gates on the
report's valid-flags (a plain rumble write no longer blanks lightbar/
triggers), and the touchpad rescale clamps to the advertised ABS_MT extents.
- Tests: Hello/Welcome trailing-byte back-compat, pick_gamepad precedence,
byte-exact input-report layout, valid-flag gating, per-mode trigger-parser
table (incl. packed 3-bit zones), wire conversions, and a scripted loopback
feedback burst (PUNKTFUNK_TEST_FEEDBACK=1) asserted through the xcframework
on the rumble + HID-output planes.
Validated: cargo test/clippy/fmt green on macOS + Linux (61 host tests), swift
build/test green, test-loopback.sh green, tvOS/iOS targets compile. DualSense
motion sign/scale is derived from the calibration blob, not yet live-verified
(constants isolated in GamepadWire).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The data-plane UDP sockets used the OS default buffer (~208 KB on Linux, similar
on macOS), which is smaller than a single high-resolution frame burst: a
5120×1440 keyframe is ~130 packets the encode|send thread hands to sendmmsg at
once. The burst overflows the buffer — EAGAIN on the host send (now dropped, was
fatal) or a silent drop on the client recv — and because the data plane runs
infinite-GOP, one lost frame breaks every subsequent reference and the decode
freezes on the last good frame until an RFI refresh that may never catch up.
Symptom: connect at 5120×1440, see ONE frame, then a frozen image (audio + input
keep working — those ride QUIC, not this socket).
Set SO_SNDBUF/SO_RCVBUF to 8 MB (clamped by the OS to net.core.{w,r}mem_max on
Linux / kern.ipc.maxsockbuf on macOS); warn if the grant lands far below target so
an undersized host is diagnosable. The client side matters most — the SAME
UdpTransport backs the Apple client's data plane via the C ABI, and macOS grants
multi-MB buffers without any sysctl, so a rebuilt client stops losing frames.
Validated live, bazzite→client at 5120×1440: was 1319/1500 frames (12% loss →
freeze), now 1500/1500 @60 and 5279/5279 @240 (split-encode active), zero
mismatches, p50 1.9–3.4 ms. Host send buffer was still capped at 416 KB and lost
nothing — the loss was purely the client recv buffer.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bazzite (and SteamOS-like hosts) run Steam Big Picture inside their OWN
gamescope-session-plus session. Nesting a second gamescope+Steam can't work — the
second Steam sees the first and exits, taking the nested gamescope down with it
(crash in its exit handlers), killing both video and input. The robust model is to
let punktfunk OWN that session: run gamescope-session-plus headless at the client's
resolution (full Steam Deck UI polish: MangoApp, VRR, controller config) and have
the host ATTACH to it rather than spawn its own.
The video half already existed (PUNKTFUNK_GAMESCOPE_NODE=<id> attaches to a
PipeWire node). This finishes it:
- PUNKTFUNK_GAMESCOPE_NODE=auto discovers the gamescope Video/Source node, so the
(dynamic) node id needn't be hand-wired.
- The attach path now also points the libei injector at the running session's EIS
socket: find_gamescope_eis_socket() scans XDG_RUNTIME_DIR for gamescope-<N>-ei,
connect()-probes each (stale dead-session sockets refuse), and writes the newest
live one to the relay file the injector reads. So input reaches the attached
session with zero manual config.
scripts/punktfunk-steam-session.service: a systemd --user unit that runs
gamescope-session-plus headless at a configured resolution, with the one-time
headless-appliance setup (linger + multi-user.target) documented inline.
Validated live on bazzite (RTX 4090): the full Steam Big Picture session streams
(1499 frames, p50 ~1ms) with mouse/keyboard injected into it (device resumed, all
caps, emitted=true), node + EIS socket both auto-detected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
UdpTransport sockets are non-blocking, so a momentarily-full kernel send buffer
makes socket.send return WouldBlock (EAGAIN). submit_frame propagated that as a
fatal error, tearing the whole punktfunk/1 session down — observed when attaching
to an already-running source (a headless Steam session) that emits frames at full
rate the instant capture connects: the first burst saturates the tx queue and the
session dies before a single frame reaches the client.
The data plane is lossy + Leopard-FEC-protected and runs infinite-GOP with RFI
keyframes, so the real-time-correct response to a full tx queue is to DROP the
packet (the next frame / FEC recovers) — exactly what the recv path already does
for WouldBlock. Blocking would queue stale frames and add latency. Loopback/M1
paths are unaffected (LoopbackTransport never blocks; M1 tests stay green).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The host-lifetime libei injector could connect to a gamescope EIS socket whose
listen socket exists but whose server never drives the EI handshake — a stale
socket left by a SIGKILLed prior session, or one created early in a new
gamescope's startup before its libei server is ready. `UnixStream::connect` to a
socket *file* succeeds the moment the path exists, so the worker sailed past the
connect and then hung forever in `handshake_tokio` (or sat connected with no
device ever resumed). Because `LibeiInjector::inject` only enqueues onto a
channel (the !Send worker owns the connection), the send never errors, so
InjectorService never noticed the dead worker and never reopened — every input
event for the whole session was silently swallowed. The 30s setup timeout didn't
help: a typical session ends first, so input just died with no error logged.
Reconnecting made it worse (more stale sockets to land on).
Two self-heal bounds, both paths (gamescope socket + KWin/GNOME portal):
- Bound the EI handshake at 8s — a non-responding EIS server now errors instead
of hanging, so the worker exits and the next inject() reopens.
- Watchdog: if no input device resumes within 5s of connecting, treat the
connection as dead-on-arrival and exit (same reopen path). Healthy servers
add+resume a device within a beat of the handshake.
Verified on-box: clean gamescope + KWin paths connect/resume/emit unchanged; a
stale listener that accepts-but-never-handshakes now errors in 8s; two
back-to-back gamescope sessions both inject (session 2 reopens against the fresh
socket). Independently confirmed end-to-end delivery on KWin — a focused wev got
the injected motions/keys/buttons — i.e. injection itself was never broken, only
its recovery from a bad connection.
Also adds permanent low-volume diagnostics so the next "input dead" report is
instantly triageable: log each EIS device's capabilities on resume, the first of
each InputKind a client sends + whether it emitted, and no-resumed-device drops.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of "input doesn't work" on the unified host: a single fresh session
injects fine (EIS connects, "Gamescope Virtual Input" device added), but the
host-lifetime injector reused a STALE per-session EIS socket across sessions →
"connect EIS socket …: Connection refused". (Headless gamescope is EIS-only — it
ignores uinput — so libei/EIS is the one input path for both gamescope and KWin;
no second path needed.)
- connect_socket_file: re-READ the relay file and RETRY the connect on
refused/missing (the live gamescope's EIS appears shortly), bounded at 15s,
instead of connecting once and bubbling ECONNREFUSED.
- GamescopeProc::drop: clear the relayed EIS socket name on teardown so a dead
session can't hand a stale path to the next reconnect.
Validated: two sessions back-to-back each reconnect (EIS connected + device added).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An open punktfunk/1 host any LAN device can trust-on-first-use and stream from is
insecure. The unified host now gates native sessions on pairing by DEFAULT: a client
must complete the SPAKE2 PIN ceremony (armed from the web console) before it's
admitted; paired devices persist. `serve --open` keeps the old TOFU behavior for
trusted single-user setups.
native_serve_opts now takes a NativeServe { port, require_pairing }; parse_serve
builds it with require_pairing = !--open. GameStream pairing (separate) is unchanged.
The require_pairing gate + ceremony are already covered by m3::pairing_ceremony_and_gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`serve --native` now runs the GameStream host AND the native punktfunk/1 (QUIC)
host in ONE process, sharing a single NativePairing handle with the management API
— so native pairing is operable from the web console instead of journalctl.
- gamestream::serve gains a native_port: spawns crate::m3::serve in the same
runtime and passes the shared NativePairing to mgmt::run. Validated live: one
process binds both RTSP 48010 and QUIC 9777.
- mgmt API: new `native` endpoints — GET /native/pair (status), POST
/native/pair/arm (mint a fresh, time-limited PIN to DISPLAY), DELETE /native/pair
(disarm), GET/DELETE /native/clients (list/unpair). GameStream-only hosts report
enabled:false. OpenAPI regenerated (checked-in doc + drift test).
- main.rs: serve --native / --native-port flags.
The native host arms pairing on demand (the operator reads the PIN from the
console; the SPAKE2 ceremony is host-shows-PIN). New mgmt + native_pairing tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Groundwork for web-UI-driven native (punktfunk/1) pairing. Replaces m3's fixed
startup PIN + local paired store with a shared `NativePairing` (new module):
arm-on-demand with a fresh, time-limited PIN (`arm(ttl)`), `current_pin()` read
per ceremony so a lapsed window stops pairing, plus the trust store (list/add/
remove/is_paired) and a `status()` snapshot. The management API (next commit) and
the QUIC accept loop share one handle. CLI `--allow-pairing`/`--require-pairing`
still arm at startup (no expiry, PIN logged) — back-compat. m3 pairing ceremony +
gate and the C-ABI roundtrip stay green; new unit tests for arm/expire/pair.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler