rustfmt pass over the files the deferred-plan items touched (pinned
toolchain 1.96.0); no semantic change. cargo fmt --all --check now clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stage B of the zero-copy host packetize path (networking-audit deferred
plan §1): Packetizer::packetize_each yields (header, shard) pairs in exact
wire order; Session::seal_frame writes seq(8) ‖ header(40) ‖ shard ‖ tag
scratch directly into the pooled wire buffer and seals [8..] in place. The
per-packet intermediate Vec (header ++ body) and its extra memcpy are gone
— with Stage A, every data byte is now copied once (frame → wire) instead
of three times, and the ~2 transient allocs/packet on the send thread are
zero after pool warmup (~180k allocs/s at 1 Gbps rates).
packetize() stays as a thin wrapper over packetize_each — the reference
implementation used by tests and the loss harness.
- wire-equivalence test: pooled path vs wrapper path byte-identical across
multi-block/partial-tail/exact-multiple/empty frames, fec 0%/50%, both
schemes, crypto on/off
- loss-harness sweep: recovery rates identical to the pre-item-1 baseline
- bench pipeline (end-to-end incl. client half) vs pre-item-1 baseline,
stages A+B cumulative: gf16/64K -3.6%, gf16/1M -3.2%; gf8 cases are
Cauchy-math-bound and unchanged within noise
- cargo ndk check (arm64-v8a) green
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two receive-path findings from the networking audit:
1. The anti-replay window (4096 seqs) silently re-tightened the "late ≠ lost"
fix: at 1 Gbps (~125k pkt/s) it spans only ~33 ms, so a Wi-Fi-retry-delayed
shard the reassembler's 120 ms loss window would still use was dropped HERE
first as "older than the window" — recreating the false-loss → recovery-IDR
churn the time-based loss window was built to kill, exactly on the high-rate
links punktfunk targets. Widened to 32768 (covers 120 ms up to ~270k pkt/s,
≈2 Gbps+); the bitmap costs 4 KiB per session and the replay-hiding bound
stays finite.
2. Every received datagram still paid one Vec allocation in the AES-GCM open
(and a to_vec on the plaintext probe path) — ~125k allocs/s of cross-thread
allocator churn at line rate, the same class of overhead that was the
documented single-core wall on the macOS receive path. New
`SessionCrypto::open_in_place` (mirror of seal_in_place; GCM verifies the
tag BEFORE decrypting, so a forged packet never yields plaintext) lets
`poll_frame` decrypt inside the recv ring and hand the reassembler a slice.
Byte-identical semantics, unit-tested against `open` incl. tamper/runt
cases; criterion entry added next to seal_in_place.
Tests: 94 core unit + loopback/c_abi suites green; clippy clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`cargo fmt --all --check` on main flags decode.rs (android dlsym fix),
probe/main.rs (0600 key fix), and session.rs (anti-replay tests). The probe
one is restructured rather than machine-formatted: rustfmt wanted the key-
permissions comment gutter-aligned to the trailing `// the certificate is
public` comment, so fold both into one block comment above the write instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Address findings from a repo security review:
- core: add a sliding-window anti-replay filter over the AEAD-authenticated
sequence in Session (poll_input/poll_frame), closing the input-replay gap the
data plane previously left to the LAN/VPN trust assumption. 4096-deep window,
unit-tested; the encrypted loopback suite confirms no false drops.
- clients: write the mTLS client private key 0600 and lock the config dir 0700
on Unix (it was world-readable at the umask default), re-locking existing
stores on load. pf-client-core::trust plus the probe's own identity writer.
Windows keeps the %APPDATA% ACL; Android/Apple already wrap the key.
- web: fix a post-login open redirect — resolve `next` via URL and require it to
stay same-origin, rejecting `/\evil.com` and tab/encoding variants the old
`!startsWith("//")` guard missed. Also fixes the dead safeNextPath helper.
- ci: SHA-256-pin the BtbN FFmpeg DLLs bundled into the signed Windows installer
(were fetched from the rolling `latest` tag unverified); fails closed on a
re-roll, matching the VB-CABLE gate.
- ci: fail-open fork-guard on the Windows/Apple host-mode PR build jobs that
share runner labels with the signing jobs. Definitive fix stays server-side
(Gitea outside-collaborator approval / isolated PR runners) — see the notes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root-caused live on a phone at 100 Mbps (stream stuck seconds behind, then
oscillating): a stack of transport defects, each amplifying the next.
- MTU-safe shards: shard_payload 1452 overshot the IPv4/1500 budget (the old
math forgot the 40 B header + 24 B crypto ride inside the UDP payload and
counted IP+UDP as 8 B) — the kernel silently split EVERY video datagram into
two IP fragments, doubling per-datagram loss on Wi-Fi. New
config::mtu1500_shard_payload() = 1408 (1472 sealed = the exact ceiling),
negotiated in the Welcome, pinned by a unit test.
- Android batched I/O: recv/send batching was cfg(linux); Android is
target_os="android" and silently fell back to a syscall per datagram. The
libc crate binds neither recvmmsg/sendmmsg nor mmsghdr for Android, so a
local bionic extern binding provides them (API 21+, floor is 28); cbindgen
excludes them from the C header. The pump/runtime threads also get the
Apple-QoS analogue on Android: nice −8 (below the decode thread's −10).
- Latency-bounded receive: packets are consumed strictly in order at exactly
the arrival rate, so a standing queue (Wi-Fi stall, power-save clumping)
NEVER drains — observed as a stream permanently 6-7 s behind with both 32 MB
socket buffers full. The pump now flushes the entire backlog
(Session::flush_backlog: discard ring + kernel queue at memcpy speed, reset
the reassembler) and requests a keyframe when frames keep completing > 400 ms
behind the skew-corrected capture clock (30 consecutive, 2 s cooldown,
logged).
- Time-based loss window: the reassembler declared an incomplete frame lost a
fixed 4 INDICES behind the newest — 33 ms at 120 fps, inside normal Wi-Fi
retry/reorder timescales, so merely-late frames were pruned every few
seconds, each costing a recovery-IDR burst + an inflated loss report.
Now 120 ms of capture time (LOSS_WINDOW_NS), same fuse at every refresh
rate, with a 64-index hard cap bounding memory against hostile pts.
- Adaptive-FEC hysteresis: the controller was memoryless — one clean 750 ms
report dropped FEC from 8 % straight back to the 1 % floor, so periodic burst
loss (Wi-Fi scan / BT coexistence beats) always hit an unprotected stream and
ping-ponged 1↔8 % with a frozen frame per cycle (observed in the host log as
alternating loss_ppm=0/50000). Attack stays instant; decay is now one point
per clean report.
Verified: full core suite (incl. new flush + time-window tests) on macOS +
Linux, host release build, arm64 cargo-ndk build, and a 30 s wired probe run
at 2800x1260@120 — 3559/3559 frames, zero loss, capture→received p50 5.3 ms
(host 5.1 + network 0.3).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On a clean link the flat 20% FEC is pure waste: extra wire bytes AND extra
packets. On a packet-rate-bound uplink (the Steam Deck's WiFi tx caps ~22k pps
regardless of bitrate) those extra packets directly cost goodput — measured at
200 Mbps goodput, 20% FEC drove ~10% loss vs ~2.6% at 0% (it saturated the link).
Adaptive FEC closes the loop:
- Client measures the loss FEC is absorbing each ~750 ms window from session stats
(recovered shards / received, + a bump when a frame went unrecoverable) and sends
a periodic `LossReport { loss_ppm }` on the control stream (new message;
`window_loss_ppm` helper, shared + unit-tested). Connector (Apple/Linux/Windows)
and probe both report; suppressed during a speed test so its filler can't skew it.
- Host maps loss → recovery % (`adapt_fec`: ≈ loss×1.4 + 1pt, clamped 1..50) and
applies it live via `Session::set_fec_percent` (the wire is self-describing — each
packet carries its block's data/recovery counts, so the receiver needs no notice).
A clean link decays to ~1%; loss ramps it up and converges.
- `PUNKTFUNK_FEC_PCT`, when set, now PINS FEC static (disables adaptation) so
speed-test / measurement runs keep a fixed, known overhead. Unset ⇒ adaptive,
starting at 10%.
An older host ignores LossReport (unknown control message) and keeps static FEC;
an older client simply never reports and the host holds its start value. Builds +
clippy + fmt + tests green (adapt_fec / window_loss_ppm / loss_report unit tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two bodies of work in one commit (the rename moved files the fixes also touched).
Naming/structure cleanup (pre-launch):
- Host modules m3.rs->punktfunk1.rs, m0.rs->spike.rs; CLI m3-host->punktfunk1-host,
m0->spike; bare `punktfunk-host` now prints help. Types M3Options/M3Source->
Punktfunk1Options/Punktfunk1Source.
- Clients consolidated out of crates/ into clients/: punktfunk-client-rs->
clients/probe (crate punktfunk-probe), client-linux->clients/linux,
client-windows->clients/windows, punktfunk-android->clients/android/native
(crate punktfunk-client-android; kept [lib] name=punktfunk_android so the JNI
contract is unchanged). crates/ now holds only core + host.
- Milestone codes M0-M4 purged from code/CLI/CLAUDE.md/README/docs/docs-site,
kept only in docs/implementation-plan.md. docs/m2-plan.md->
docs/gamestream-host-plan.md. CI/gradle/flatpak paths updated.
Client loss-recovery (video froze and never recovered after a brief drop):
- Export punktfunk_connection_frames_dropped through the C ABI (the core already
tracked it for the client keyframe-recovery loop; it was never reachable from
the ABI clients). Regenerated punktfunk_core.h.
- Apple (StreamPump + Stage2Pipeline) and Android (decode.rs) now poll
frames_dropped and request a keyframe when it climbs -- the same loss-driven
recovery Linux/Windows already had. Under infinite GOP the decoder silently
conceals reference-missing frames, so the decode-error trigger rarely fires.
Apple rumble robustness (worked then went spotty -- DualSense + Xbox):
- Add CHHapticEngine stopped/reset handlers (rebuild on app background / audio
interruption / server reset) and drop the permanent `broken` latch on a
transient drive failure; latch only when the controller truly has no haptics.
- Surface swallowed SDL set_rumble errors on Linux/Windows + diagnostic logging.
Verified: cargo build/clippy/fmt --workspace, C-ABI harness, header drift.
Not runnable on this box (verify in CI): Gitea workflows, gradle/Android,
flatpak, Swift/decky.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The host sealed every packet with ~3 heap allocations: aes-gcm's convenience
encrypt() allocates the ciphertext Vec, seal_for_wire allocates the seq||ct||tag
wire Vec, and seal_frame allocated a fresh Vec<Vec<u8>> per frame. At line rate
(~250k–500k pkt/s for 2.5–5 Gbps) that's the single-core allocator wall.
- SessionCrypto::seal_in_place uses AeadInPlace::encrypt_in_place_detached to
encrypt into the caller's buffer and write the detached tag at the end —
byte-identical to seal's ciphertext||tag, no allocation (unit-tested for byte
equality + decrypt).
- Session keeps a wire_pool the caller returns via reclaim_wires; seal_frame
seals each packet in place into the reused buffers (clear() keeps capacity), so
after warmup there's no per-packet ciphertext/wire allocation. paced_submit and
submit_frame reclaim the pool after sending.
End-to-end encrypted/lossless multi-frame tests stay green (validates the pool
reuse doesn't corrupt across frames). Next: write packetize directly into a
contiguous send buffer (kills the remaining shard allocs + GSO's coalescing copy).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sendmmsg already batches syscalls but still builds one sk_buff per datagram —
the kernel-side wall above ~1 Gbps. UDP Generic Segmentation Offload hands the
kernel one big buffer it splits into gso_size datagrams, building ~1 GSO skb per
≤64 segments. Research (LWN/Cloudflare/Tailscale) measures ~2.4x throughput at
equal CPU and 17-44x fewer syscalls, and that sendmmsg batching alone is
insufficient — you need true segmentation offload.
Adds Transport::send_gso (default = send_batch) + a UdpTransport Linux override:
coalesces a frame's equal-size wire packets (shards are zero-padded to a constant
size, so a whole frame is one gso_size) into ≤64-segment sendmsg(UDP_SEGMENT)
calls. seal/send routes through it. Opt-in via PUNKTFUNK_GSO (new unsafe hot-path
code) with automatic fallback to sendmmsg on any GSO error (unsupported kernel/
path), latched per process. Loopback unit test validates the cmsg segmentation;
full session over loopback streams clean (0% loss). Linux-only; loopback/non-Linux
keep sendmmsg/scalar.
Next levers: in-place AES-GCM seal (kill per-packet allocs), UDP GRO on recv,
drop the sleep-pacing in favor of the kernel qdisc, jumbo MTU.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
From a bug-hunt + unsafe-audit pass (4 reviewers + adversarial verify). It
confirmed ZERO real bugs in the recent batched/paced data-plane work — these are
the surfaced cleanups + one genuine soundness fix:
- SOUNDNESS (reduce unsafe): inject/gamepad.rs::pump_ff did `ptr::read` of an
InputEventRaw (align 8, holds a timeval) out of a 1-aligned [u8; N] buffer — UB
per the reference (x86_64 tolerates it, but it can miscompile under LTO). Use
ptr::read_unaligned + a SAFETY note. Zero behavior change.
- recv parity: recv_batch (recvmmsg) didn't drop an oversized/truncated datagram
the way scalar recv does — poll_frame now skips a message whose len fills the
buffer (> MAX_DATAGRAM_BYTES), matching recv's `n >= RECV_BUF` drop. (AEAD
already rejected these on encrypted sessions; this restores the documented
invariant on the batched path.)
- dedup unsafe FFI: factor the identical mmsghdr-from-iovec construction out of
send_batch + recv_batch into one `mmsghdrs()` helper — the raw-pointer
scaffolding + its lifetime SAFETY note now live in one place.
- docs: TARGET_SOCKBUF no longer calls paced sending future work (it landed,
m3.rs::paced_submit); gamescope.rs input is no longer "(TODO)" (wired +
live-validated); the PUNKTFUNK_PERF `wire_mbps` field is renamed `tx_mbps` and
noted as attempted/sealed bytes (send_dropped shows what didn't reach the wire).
Full suite (35 + loopback round-trip + 6) + clippy + fmt green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Final increment of the 1 Gbps data-plane rework — the recv counterpart of the
sendmmsg work. The client recv path did one recvfrom + one Vec allocation per
packet (and the pump's 300µs idle sleep could let packets pile up at line rate).
- Transport gains recv_batch(&mut [Vec<u8>], &mut [usize]) -> count; default is
a single scalar recv into out[0] (loopback + non-Linux).
- UdpTransport overrides it on Linux with recvmmsg (MSG_DONTWAIT) draining up to
N datagrams per syscall into the caller's reused buffers — no per-packet alloc.
- Session::poll_frame owns a lazily-allocated recv ring (RECV_BATCH=32) and
consumes it one packet at a time across calls, refilling with one recvmmsg when
drained. Encapsulated: the punktfunk-client-rs + NativeClient pumps are
unchanged, and draining a batch per syscall means the 300µs sleep no longer
underdrains. Added UdpTransport::local_addr (used by the test, generally handy).
~125k → ~4k recv syscalls/sec at line rate, zero per-packet recv allocation.
Verified: new recv_batch_drains_over_loopback test (50 datagrams drained intact
via recvmmsg) + the existing loopback round-trip now runs through the batched
poll_frame; full suite (35 + round-trip + 6) + clippy + fmt green.
Decode-in-place (kill the per-packet open_from_wire alloc) is a separate later
optimization. With A (sendmmsg) + B (paced send) + C (recvmmsg), the native data
plane is batched + paced end to end.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Increment B of the send-path rework — the actual fix for "freezes get more
common over ~150 Mbps, no image at all at 400 Mbps" on the native path. Cause:
the encoder emits a frame and submit_frame blasted ALL its packets at once into
the NIC; a real link drops the line-rate burst (host send buffer EAGAINs), and
under infinite GOP one dropped frame freezes the decode until the next keyframe.
(The speed-test probe showed 0 drops at 400 Mbps because the probe is self-paced;
real video wasn't.)
Adaptive pacing, no extra thread, no regression:
- Session splits into seal_frame (FEC + packetize + seal → wire packets, no
send) and send_sealed (one batched sendmmsg of a chunk, counts drops);
submit_frame is now their composition (synthetic + probe paths unchanged).
- virtual_stream's paced_submit seals a frame then sends it in 16-packet chunks
spread over ~90% of the time until the next frame is due. At 60 fps desktop
(fast encode → lots of slack) the frame spreads across the interval → no NIC
burst → no freeze. At 240 fps@5K (encode ≈ interval → ~0 slack) the budget
collapses and every chunk goes out immediately → never slower than before.
Core suite (34 + loopback round-trip + 6) + clippy + fmt green. The seal/send
split is covered by the existing loopback tests; the pacing is host timing,
verified by review (live-test needs a real NIC — your Mac at a raised bitrate).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First increment of the 1 Gbps send-path rework (the measured bottleneck): the
native data plane did one send() syscall per packet — at ~125k pkt/s (1 Gbps
wire) that burns a core on syscalls. Port the proven GameStream sendmmsg path
into the core Transport seam.
- Transport gains `send_batch(&[&[u8]]) -> usize` (count handed to the kernel;
caller counts the rest as send-buffer drops). Default = the scalar send loop
(loopback transport + non-Linux).
- UdpTransport overrides it on Linux with `sendmmsg` (64 datagrams/syscall);
the connected socket needs no per-message address. Non-blocking-aware: a full
send buffer yields a short count / EAGAIN, and we stop + report what went out
rather than block or retry (same lossy, FEC-protected contract as send()).
- Session::submit_frame seals every shard then hands the whole frame to
send_batch in ONE call instead of looping send() — ~64x fewer syscalls per
frame on the native + GameStream-over-core paths; send_dropped accounting
preserved (total - sent).
~125k → ~2k syscalls/sec at 1 Gbps line rate. Verified: new loopback-UDP test
send_batch_delivers_over_loopback (100 batched packets arrive intact, datagram
boundaries preserved); full core suite + clippy + fmt green.
Next increments: a paced send thread (microburst shaping so a real NIC doesn't
drop line-rate bursts) and recvmmsg on the client.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First step of 1 Gbps+ readiness (the whole point of the GF(2^16) Leopard FEC):
make 1 Gbps configurable and its dominant failure mode observable, before the
real transport work (sendmmsg + paced encode|send split) lands.
Investigation (6-way) verdict: we're ~halfway, and it's mostly clamps plus one
real piece of work. The integer/type path, FEC (a 1 Gbps frame is only a few
hundred shards in one GF(2^16) block, far under the 65535 ceiling), AES-GCM
(AES-NI, ~10-25x headroom), and the M1 reassembler bounds (fully derived from
the negotiated FecConfig) are ALL already 1 Gbps-ready and untouched.
This commit (the configurable + observable foundation):
- m3.rs: MAX_BITRATE_KBPS 500_000 -> 2_000_000 (2 Gbps headroom over the 1 Gbps+
target); MAX_PROBE_KBPS 1_000_000 -> 3_000_000 (probe can demonstrate headroom
ABOVE the session cap so a client can confidently pick a 1 Gbps+ bitrate).
- transport/udp.rs: TARGET_SOCKBUF 8 MB -> 32 MB (a multi-MB IDR keyframe burst
no longer fills the buffer); scripts/99-punktfunk-net.conf bumped to match.
- Observability: Transport::send now returns Ok(true|false) (false = WouldBlock
send-buffer drop, previously a silent Ok(())). Session counts these as a new
`packets_send_dropped` stat (distinct from recv-side packets_dropped) — in
Stats, the C ABI PunktfunkStats (header regenerated), a PUNKTFUNK_PERF periodic
wire-Mbps + drop dump in virtual_stream, and the speed-test probe completion
log. This is the dominant 1 Gbps+ loss mode and was invisible.
Loopback-verified: a probe now runs at 1.2 Gbps target (no longer truncated to
1 Gbps) with the drop counter live. NOT yet a sustained-1-Gbps proof — the
single-send()-per-packet native path is the next, real piece of work (port the
proven GameStream sendmmsg + paced send thread into the core Transport).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full project rename, decided 2026-06-10:
- Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs.
- C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h,
PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl.
PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants).
- Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1.
WIRE BREAK: clients must be rebuilt from this revision.
- Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / ….
- Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the
persistent identity is unchanged, pinned fingerprints stay valid).
- Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection
(Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated.
- scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated.
Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of
"desktop but no apps/settings" over the stream: plasmashell launched without
XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and
rendered an empty menu. The script sets the complete KDE session env (menu prefix,
KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell.
Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS,
zero lumen references left outside .git.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enricobuehler