71f26083a66966842e80f7cb881f977aa9c73222
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 enricobuehler
|
c56b1b455a |
feat(punktfunk/1): request-IDR recovery for a wedged client decode
apple / swift (push) Successful in 1m17s
ci / rust (push) Failing after 31s
ci / web (push) Failing after 42s
ci / docs-site (push) Failing after 40s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Failing after 10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 5s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 6s
docker / deploy-docs (push) Has been skipped
rpm / build-publish (push) Failing after 15s
deb / build-publish (push) Failing after 43s
Fixes the intermittent first-connect freeze. The host streams infinite GOP — one opening IDR, then P-frames only (recovery keyframes just on loss) — so when the client's decoder wedges on the cold first session (a lost/corrupt opening IDR, a bad early P-frame) the picture stays frozen until the far-off next keyframe. The client had no way to ask for one; now it does. Add a RequestKeyframe control message (client -> host, reliable control stream), mirroring Reconfigure: - core: quic.rs RequestKeyframe (type 0x03) + roundtrip test; client.rs CtrlRequest::Keyframe + NativeClient::request_keyframe; abi.rs punktfunk_connection_request_keyframe (header regenerated). - host: m3.rs decodes it in the control loop and signals the encode loop, which coalesces a burst and calls enc.request_keyframe() — wiring the existing NvencEncoder hook (force_kf -> next frame pict_type=I), the same recovery the GameStream path already had via force_idr. - apple: PunktfunkConnection.requestKeyframe(); StreamPump (stage-1) requests on layer.status==.failed; Stage2Pipeline (stage-2) on a sync submit failure and on the async decode-error callback via a thread-safe KeyframeRecovery. All throttled to <=1/250ms (the decode stays wedged for several frames until the IDR lands, so per-frame requests would flood the control stream). Self-healing: a lost recovery IDR is re-requested after the throttle; the host coalesces bursts into a single IDR. Validated: cargo fmt + clippy clean; core + host test suites green (incl. new request_keyframe_roundtrip); swift build + test (39 passed); xcframework rebuilt (all 5 slices), header regenerated with no unrelated drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
6b4de5d738 |
feat(client/speedtest): request the host's full 3 Gbps probe ceiling
The Apple speed test asked for only 400 Mbps, capping the measured throughput
there and hiding the link's real headroom. Request the host's full
MAX_PROBE_KBPS (3 Gbps) instead, and raise the recommended-bitrate clamp from
500 Mbps to the host's 2 Gbps session ceiling so a fast measurement yields a
usable recommendation.
Also fix the stale caps left when the host clamps were raised (
|
||
|
enricobuehler
|
7eb9a927cf |
feat(connector): expose host clock offset over the C ABI for glass-to-glass
ci / rust (push) Has been cancelled
Factor the client-side skew handshake into a shared core helper (quic::clock_sync -> ClockSkew) so both the reference client and the embeddable connector use one implementation. NativeClient now runs the handshake at connect (right after Start, before the control task takes the stream) and stores the host-client offset; it's read over the C ABI via punktfunk_connection_clock_offset_ns (i64 ns, host minus client; 0 = no correction / old host). This is the substrate the Apple client needs for the decode->present (glass-to- glass) term: stamp present time, add the offset to express it in the host's capture clock, subtract the AU pts_ns. client-rs drops its local clock_sync copy and uses the shared helper (behavior unchanged; validated locally). Regenerates include/punktfunk_core.h. Roadmap section 12 + status updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
b8a33e21a2 |
feat(1gbps): raise bitrate/probe clamps + socket buffers, count send-buffer drops
ci / rust (push) Has been cancelled
First step of 1 Gbps+ readiness (the whole point of the GF(2^16) Leopard FEC): make 1 Gbps configurable and its dominant failure mode observable, before the real transport work (sendmmsg + paced encode|send split) lands. Investigation (6-way) verdict: we're ~halfway, and it's mostly clamps plus one real piece of work. The integer/type path, FEC (a 1 Gbps frame is only a few hundred shards in one GF(2^16) block, far under the 65535 ceiling), AES-GCM (AES-NI, ~10-25x headroom), and the M1 reassembler bounds (fully derived from the negotiated FecConfig) are ALL already 1 Gbps-ready and untouched. This commit (the configurable + observable foundation): - m3.rs: MAX_BITRATE_KBPS 500_000 -> 2_000_000 (2 Gbps headroom over the 1 Gbps+ target); MAX_PROBE_KBPS 1_000_000 -> 3_000_000 (probe can demonstrate headroom ABOVE the session cap so a client can confidently pick a 1 Gbps+ bitrate). - transport/udp.rs: TARGET_SOCKBUF 8 MB -> 32 MB (a multi-MB IDR keyframe burst no longer fills the buffer); scripts/99-punktfunk-net.conf bumped to match. - Observability: Transport::send now returns Ok(true|false) (false = WouldBlock send-buffer drop, previously a silent Ok(())). Session counts these as a new `packets_send_dropped` stat (distinct from recv-side packets_dropped) — in Stats, the C ABI PunktfunkStats (header regenerated), a PUNKTFUNK_PERF periodic wire-Mbps + drop dump in virtual_stream, and the speed-test probe completion log. This is the dominant 1 Gbps+ loss mode and was invisible. Loopback-verified: a probe now runs at 1.2 Gbps target (no longer truncated to 1 Gbps) with the drop counter live. NOT yet a sustained-1-Gbps proof — the single-send()-per-packet native path is the next, real piece of work (port the proven GameStream sendmmsg + paced send thread into the core Transport). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
74819b1be8 |
feat(punktfunk/1): negotiable encoder bitrate + bandwidth speed-test probe
ci / rust (push) Has been cancelled
Two related additions to the native protocol, host-side (the client side of
each is exposed over the C ABI so the platform clients can wire it up).
Bitrate negotiation
- Hello/Welcome carry `bitrate_kbps` (appended trailing-byte field, back-compat:
old peers decode 0 = host default). The client requests a rate; the host
clamps it to [500 kbps, 500 Mbps] (or its 20 Mbps default when 0) and echoes
the resolved value in Welcome. Replaces the hardcoded 20 Mbps NVENC bitrate in
m3.rs — threaded through virtual_stream → build_pipeline → open_video, applied
on the initial mode and every reconfigure rebuild.
- C ABI: punktfunk_connect_ex3(..., bitrate_kbps, ...) (ex2 delegates with 0);
punktfunk_connection_bitrate() reads the resolved value.
Speed test (bandwidth probe)
- New typed control messages ProbeRequest{target_kbps,duration_ms} (0x20) /
ProbeResult{bytes_sent,packets_sent,duration_ms} (0x21), plus a FLAG_PROBE
packet flag. The client asks the host to burst zero-filled, FLAG_PROBE-tagged
access units over the data plane at a target goodput for a duration (clamped
≤ 1 Gbps / ≤ 5 s), pacing by a bytes-allowed budget; video pauses for the
burst. The host reports what it actually sent; the client measures received
bytes + window → goodput and loss. Probe filler is never fed to the decoder
(diverted in the connector pump and the reference client's poll loop).
- The host control task now multiplexes Reconfigure + ProbeRequest (inbound)
and ProbeResult (outbound) over select!; a probe channel reaches the
data-plane thread (both virtual and synthetic sources).
- Connector: NativeClient::request_probe()/probe_result() with an internal
accumulator; C ABI punktfunk_connection_speed_test() +
punktfunk_connection_probe_result() → PunktfunkProbeResult.
- punktfunk-client-rs gains `--bitrate KBPS` and `--speed-test KBPS:MS` (its own
loop measures + logs goodput/loss) for loopback verification.
Validated on loopback (synthetic source): a 20 Mbps / 2 s probe measured
20050 kbps at 0% loss, bitrate negotiated (0→20000 and 50000→50000), and the
interleaved probe AUs were correctly excluded from frame verification
(mismatched=0). Wire codecs + trailing-byte back-compat have unit tests. C
header regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
1d605fb781 |
feat(gamepad): controller discovery + client-negotiated pad type + rich DualSense end to end
The Apple client grows full gamepad support and punktfunk/1 learns to negotiate the virtual pad type: - Protocol: Hello carries a GamepadPref byte (offset 21, the same trailing-byte back-compat pattern as the compositor; echoed resolved in Welcome at 54). Host precedence: explicit client choice > PUNKTFUNK_GAMEPAD env > Xbox 360, DualSense (UHID) only where available. ABI: punktfunk_connect_ex2 + punktfunk_connection_gamepad (connect_ex delegates; ABI_VERSION stays 2 — the trailing byte IS the compat mechanism). punktfunk-client-rs gets --gamepad. - Swift client: GamepadManager (app-lifetime discovery + selection — Settings lists every controller with capabilities/battery/"In use"; exactly ONE pad forwards as pad 0, auto = most recently connected, or pinned), GamepadCapture (snapshot-diff button/axis events, DualSense touchpad + ~250 Hz motion on the rich-input plane, held state released on switch/deactivate/stop), GamepadFeedback (rumble → CoreHaptics per-handle engines; lightbar → GCDeviceLight; player LEDs → playerIndex; adaptive-trigger blocks → the table-driven DualSenseTriggerEffect parser → GCDualSenseAdaptiveTrigger, exact for the 10-zone positional modes). The pad type auto-resolves from the physical controller at connect time, user-overridable in Settings. - Host DualSense fixes surfaced by adversarial review against hid-playstation / SDL / Nielk1 ground truth: input-report sensor/touch offsets were off by one (the kernel read garbage motion + phantom touches), the L2/R2 trigger blocks were swapped (the report is right-trigger-first), feedback now gates on the report's valid-flags (a plain rumble write no longer blanks lightbar/ triggers), and the touchpad rescale clamps to the advertised ABS_MT extents. - Tests: Hello/Welcome trailing-byte back-compat, pick_gamepad precedence, byte-exact input-report layout, valid-flag gating, per-mode trigger-parser table (incl. packed 3-bit zones), wire conversions, and a scripted loopback feedback burst (PUNKTFUNK_TEST_FEEDBACK=1) asserted through the xcframework on the rumble + HID-output planes. Validated: cargo test/clippy/fmt green on macOS + Linux (61 host tests), swift build/test green, test-loopback.sh green, tvOS/iOS targets compile. DualSense motion sign/scale is derived from the calibration blob, not yet live-verified (constants isolated in GamepadWire). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
enricobuehler
|
59edeedf07 |
feat(dualsense): Phase C/D/E — virtual DualSense routing + 0xCC/0xCD planes + C ABI
ci / rust (push) Has been cancelled
PUNKTFUNK_GAMEPAD=dualsense now routes a session's gamepad through a real virtual
DualSense (UHID + hid-playstation) end to end:
- host: a `PadBackend` enum (m3.rs) selects `GamepadManager` (uinput xpad, default)
or the new `DualSenseManager` (dualsense.rs) per session. The manager keeps each
pad's full DsState so touchpad + motion (rich-input plane) persist across
button/stick frames, and services the !Send /dev/uhid fd only on the input thread
(which cycles <=4ms, so the GET_REPORT init handshake completes).
- feedback: `service()` now returns `DsFeedback { hidout, rumble }`. Motor rumble
stays on the universal 0xCA plane (so non-DualSense clients still feel it; manager
dedups change); lightbar / player LEDs / adaptive-trigger effects ride the new
0xCD HID-output plane (host->client) as `HidOutput`.
- rich input: touchpad contacts + motion ride the 0xCC plane (client->host) as
`RichInput`, applied via `DualSenseManager::apply_rich` (merged with button state;
touch normalized 0..65535 -> the touchpad resolution).
- connector + C ABI: `NativeClient::next_hidout` / `send_rich_input`, exported as
`punktfunk_connection_next_hidout` (-> PunktfunkHidOutput) and
`punktfunk_connection_send_rich_input` (<- PunktfunkRichInput); header regenerated.
- reference client: `--rich-input-test` drives the DualSense touchpad + motion and
logs the 0xCD feedback that comes back.
Validated live on-box: a synthetic-source m3-host + client-rs created the real
kernel DualSense, drove 0xCC, and decoded 12 live 0xCD events (the kernel's actual
lightbar/trigger init reports) with the data plane unaffected (600/600 frames).
Adversarial review fixes folded in: the input loop no longer skips the rich drain +
feedback pump on a dropped gamepad event, and the touch contact id is clamped to its
slot. Remaining: the Apple client renders triggers/rumble on a real DualSense.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
0755c823a5 |
feat: mic passthrough — client microphone → host virtual PipeWire source
ci / rust (push) Has been cancelled
The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire Audio/Source its apps can record from (voice chat, etc.). Protocol (punktfunk-core): - MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram). - NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task (mirror of send_input); C ABI punktfunk_connection_send_mic for native clients. Host: - audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class= Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM. - MicService: host-lifetime owner of the source + Opus decoder (mirror of InjectorService / the audio capturer slot); lazily opened, persists across sessions, self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a single read_datagram loop (two loops would race). - Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting, so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output was silence; with it, glitch-free across consumer quanta. Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic uplink (opus dep added) for end-to-end validation without a real microphone. Validated live on headless KWin: client tone → host source → pw-record shows the punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001 elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
enricobuehler
|
6fdf7d1511 |
feat: client-selectable compositor (protocol → host → client → C ABI → mgmt → web)
A client can now request which compositor backend the host drives its virtual
output on (gamescope/KWin/Mutter/wlroots). The host honors the request if that
backend is available, else falls back to auto-detect and reports the resolved
choice back — wire-compatible both directions (no ABI bump).
Protocol (punktfunk-core):
- New CompositorPref (config.rs): Auto|Kwin|Wlroots|Mutter|Gamescope with
u8/name mappings. Appended as one optional byte to Hello (client preference)
and Welcome (host's resolved choice). Both decoders already tolerate trailing
bytes, so old↔new interop is preserved — ABI_VERSION stays 2. Round-trip +
back-compat (truncated-message) tests.
- C ABI: punktfunk_connect_ex(compositor) + PUNKTFUNK_COMPOSITOR_* constants;
punktfunk_connect delegates with AUTO, so the existing symbol is unchanged.
NativeClient::connect / worker_main thread the preference through.
Host:
- vdisplay::available() enumerates usable backends via cheap, side-effect-free
probes (KWin zkde global, gamescope binary+version, GNOME/Sway env), plus
Compositor id/label/as_pref/from_pref/all helpers.
- m3 handshake resolves the preference to a concrete backend during the
handshake (pick_compositor pure + resolved logging), reports it in Welcome,
and threads it into virtual_stream (replacing the unconditional detect()).
- mgmt GET /v1/compositors lists every backend with availability + the
auto-detected default (OpenAPI regenerated).
Client:
- punktfunk-client-rs --compositor NAME; logs the host's resolved choice from
the Welcome ("session offer … compositor=…").
Web console:
- Host page gains a Compositors card (availability + default badges) via the
codegen'd useListCompositors hook; en/de strings added.
Also fixes a pre-existing, env-dependent test-isolation bug:
mgmt::tests::paired_clients_list_and_unpair seeded the real
~/.config/punktfunk/paired.json (AppState::new loads it), so a real
GameStream-paired client leaked into body[0] on a dev box — now cleared first.
Live-validated against headless KWin: --compositor kwin honored, --compositor
mutter falls back to kwin (available=[kwin, gamescope]), resolved choice
round-trips to the client. Tests: +6 (wire/back-compat, resolution precedence,
endpoint); workspace green, clippy/fmt clean, C ABI harness PASS at abi_version=2,
web typecheck + build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
4d26ac5c85 |
feat: punktfunk/1 — mid-stream mode renegotiation + PIN pairing ceremony
Renegotiation (no reconnect on resize): the handshake bi-stream stays open; the client
sends Reconfigure{mode} (typed post-handshake message), the host validates + acks
Reconfigured and rebuilds capture/encoder/virtual output at the new mode while the data
plane (keys, ports, FEC) runs untouched — the first new-mode AU is an IDR with in-band
parameter sets. NativeClient::request_mode / punktfunk_connection_request_mode; mode()
reflects the active mode. Validated live on KWin: one continuous stream, 225 frames
@1280x720 then 395 @1920x1080, ~90 ms pipeline rebuild (ffprobe shows both resolutions).
PIN pairing (mutual trust, kills TOFU MITM): clients get persistent self-signed
identities presented via QUIC client auth (generate_identity / client auth offered but
optional server-side — legacy clients still connect). Ceremony on the control stream:
PairRequest{name} → host shows a 4-digit PIN (log) + PairChallenge{salt} → client proves
with HMAC-SHA256(PIN‖salt, client_fp‖host_fp) — binding both certs means a MITM can't
forward a proof, single attempt per PIN, constant-time compare → PairResult; host
persists the fingerprint (~/.config/punktfunk/punktfunk1-paired.json), client pins the
host's. m3-host --require-pairing gates sessions on the paired set.
NativeClient::pair + punktfunk_pair/punktfunk_generate_identity in the ABI; reference
client: --pair PIN --name LABEL + auto-generated persistent identity, --remode for live
renegotiation testing. Swift wrapper: ClientIdentity/generateIdentity()/pair(),
requestMode()/currentMode(); README handoff updated.
Tested: reconfigure/pairing wire roundtrips, C-ABI mode switch ack, full in-process
ceremony (wrong PIN → Crypto, anonymous-vs-gate rejection, success → pinned session);
live wrong-PIN ceremony against the serving host (PIN logged, proof rejected).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
enricobuehler
|
bfd64ce871 |
rename: lumen → punktfunk, everywhere
ci / rust (push) Has been cancelled
Full project rename, decided 2026-06-10: - Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs. - C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h, PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl. PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants). - Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1. WIRE BREAK: clients must be rebuilt from this revision. - Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / …. - Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the persistent identity is unchanged, pinned fingerprints stay valid). - Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated. - scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated. Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of "desktop but no apps/settings" over the stream: plasmashell launched without XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and rendered an empty menu. The script sets the complete KDE session env (menu prefix, KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell. Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS, zero lumen references left outside .git. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |