unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 4 Wiki Activity
496 Commits 2 Branches 5 Tags
24ee05a4d09a4cdd67e4b1d6b18ef1e547a62b67
Commit Graph

163 Commits

Author SHA1 Message Date
enricobuehler 6575dddac7 fix: keep the workspace green on macOS after the mic/touch/rich-input batch 
The new features were Linux-built only and broke the documented macOS gate
(cargo build/test/clippy --workspace) four ways, all fixed following the existing
platform-gating conventions:

- m3.rs: mic_service_thread split into the Linux worker and a non-Linux stub that
  drains and drops (sessions still count the datagrams) — opus/PipeWire are
  Linux-gated deps, same pattern as audio_thread.
- punktfunk-client-rs: the new `opus` dependency moved into the Linux target table and
  --mic-test gated with a warn-and-skip stub (only the synthetic-tone test rig needs
  the encoder; the mic uplink itself is portable).
- gamestream/audio.rs: SAMPLE_RATE import gated to any(linux, test) (the frame_sizing
  test uses it everywhere, the data plane only on Linux).
- tests/c_abi.rs: the harness's macOS link flags gained Security + CoreFoundation —
  the quic feature now pulls rustls's platform verifier into the staticlib.

Also: two clippy match-ref-pats lints in the new rich-input/HID-output decoders
(clippy -D warnings is the repo gate), the regenerated punktfunk_core.h committed (the
checked-in copy predated the rich-input/HID-output constants — CI fails on drift), and
web's inlang cache dir gitignored.

cargo build/test/clippy/fmt --workspace: green on macOS, 122 tests passing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-11 09:07:48 +02:00
enricobuehler dc375668ee feat: touch input — TouchDown/Move/Up + host libei ei_touchscreen injection
ci / rust (push) Has been cancelled
Details 
Roadmap #5 (touch, ahead of the XL UHID DualSense work). Touch fits the existing 18-byte
InputEvent: code = touch id, x/y = client pixels, flags = (w<<16)|h — the same absolute
mapping as MouseMoveAbs.

- core: InputKind::{TouchDown=9, TouchMove=10, TouchUp=11} + from_u8 + roundtrip test.
- host inject/libei.rs: request the RemoteDesktop Touchscreen device type, bind the Touch
  capability, and inject ei_touchscreen down/motion/up (one event = one frame, per the
  protocol rule), mapping coordinates into the device region like the abs pointer. wlroots
  has no virtual-touch protocol wired — no-ops there.
- client-rs --touch-test: drags a synthetic finger (touch id 0) in a circle.

Validated live on headless KWin: the portal GRANTS the Touchscreen device type
(Keyboard|Pointer|Touchscreen), proving the request path — but KWin's EIS server creates no
touchscreen *device*, so touch currently no-ops on this KWin (now logged once, not silent).
The injection code is correct and will land on a backend that exposes ei_touchscreen
(gamescope / a newer compositor / the real touch-client path). Workspace green, clippy/fmt
clean, +1 unit test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 22:38:44 +00:00
enricobuehler 0755c823a5 feat: mic passthrough — client microphone → host virtual PipeWire source
ci / rust (push) Has been cancelled
Details 
The inverse of the host→client audio path: the client's mic, Opus-encoded, rides a
new 0xCB QUIC datagram to the host, which decodes it into a virtual PipeWire
Audio/Source its apps can record from (voice chat, etc.).

Protocol (punktfunk-core):
- MIC_MAGIC 0xCB + encode/decode_mic_datagram (mirror of the 0xC9 audio datagram).
- NativeClient::send_mic(seq, pts_ns, opus) over a new outbound channel + worker task
  (mirror of send_input); C ABI punktfunk_connection_send_mic for native clients.

Host:
- audio::VirtualMic + PwMicSource: a PipeWire output stream tagged media.class=
  Audio/Source (Direction::Output) — a recordable microphone node, fed decoded PCM.
- MicService: host-lifetime owner of the source + Opus decoder (mirror of
  InjectorService / the audio capturer slot); lazily opened, persists across sessions,
  self-heals. The per-session datagram reader now demuxes 0xCB→mic / 0xC8→input over a
  single read_datagram loop (two loops would race).
- Adaptive jitter buffer in the producer: primes to ~3 consumer quanta before emitting,
  so the 5 ms push / N ms pull clock skew never underruns — without it ~58% of output
  was silence; with it, glitch-free across consumer quanta.

Client: punktfunk-client-rs --mic-test streams a synthetic 440 Hz Opus tone as the mic
uplink (opus dep added) for end-to-end validation without a real microphone.

Validated live on headless KWin: client tone → host source → pw-record shows the
punktfunk-mic Audio/Source node, 440 Hz dominant (Goertzel power 20.7 vs <0.001
elsewhere), RMS 0.179 ≈ the ideal 0.177, 0.3–0.4% silence at both 256 ms and 10 ms
consumer quanta. Tests +1 (mic datagram roundtrip); workspace green, clippy/fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 22:15:07 +00:00
enricobuehler a03aae891e fix(m3): persistent host-lifetime input injector — end the RemoteDesktop portal churn
ci / rust (push) Has been cancelled
Details 
Under rapid client reconnects, KWin's libei/EIS input setup intermittently wedged
with "EIS setup timed out", causing total input loss for affected sessions. Root
cause: each punktfunk/1 session opened (and tore down) its own RemoteDesktop-portal
CreateSession for pointer/keyboard injection, and back-to-back reconnects raced a
prior session's portal teardown before it settled.

LibeiInjector is only a Send channel handle to a worker thread that owns the portal
session, so the injector can live for the whole host run instead of per session.
Adds InjectorService: one host-lifetime thread owns the (!Send) injector, opened
ONCE (lazily, on the first event) and reused across every session — the portal grant
is established a single time and held. Sessions forward pointer/keyboard events to it
over a clonable Send channel; gamepads stay per-session (uinput, no portal). The
service self-heals — reopen after a 2s backoff if open fails or the backend worker
dies (covers a gamescope EIS socket that respawns with its nested session).

Mirrors the existing host-lifetime audio-capturer slot; the audio capturer is Send
(a slot works), the injector is !Send (needs the owning thread + channel).

Validated live on headless KWin: 8 rapid back-to-back input sessions →
"input injector ready (host-lifetime)" exactly once, ZERO "EIS setup timed out",
8/8 sessions injected input. Tests green, clippy/fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 21:31:11 +00:00
enricobuehler 6fdf7d1511 feat: client-selectable compositor (protocol → host → client → C ABI → mgmt → web) 
A client can now request which compositor backend the host drives its virtual
output on (gamescope/KWin/Mutter/wlroots). The host honors the request if that
backend is available, else falls back to auto-detect and reports the resolved
choice back — wire-compatible both directions (no ABI bump).

Protocol (punktfunk-core):
- New CompositorPref (config.rs): Auto|Kwin|Wlroots|Mutter|Gamescope with
  u8/name mappings. Appended as one optional byte to Hello (client preference)
  and Welcome (host's resolved choice). Both decoders already tolerate trailing
  bytes, so old↔new interop is preserved — ABI_VERSION stays 2. Round-trip +
  back-compat (truncated-message) tests.
- C ABI: punktfunk_connect_ex(compositor) + PUNKTFUNK_COMPOSITOR_* constants;
  punktfunk_connect delegates with AUTO, so the existing symbol is unchanged.
  NativeClient::connect / worker_main thread the preference through.

Host:
- vdisplay::available() enumerates usable backends via cheap, side-effect-free
  probes (KWin zkde global, gamescope binary+version, GNOME/Sway env), plus
  Compositor id/label/as_pref/from_pref/all helpers.
- m3 handshake resolves the preference to a concrete backend during the
  handshake (pick_compositor pure + resolved logging), reports it in Welcome,
  and threads it into virtual_stream (replacing the unconditional detect()).
- mgmt GET /v1/compositors lists every backend with availability + the
  auto-detected default (OpenAPI regenerated).

Client:
- punktfunk-client-rs --compositor NAME; logs the host's resolved choice from
  the Welcome ("session offer … compositor=…").

Web console:
- Host page gains a Compositors card (availability + default badges) via the
  codegen'd useListCompositors hook; en/de strings added.

Also fixes a pre-existing, env-dependent test-isolation bug:
mgmt::tests::paired_clients_list_and_unpair seeded the real
~/.config/punktfunk/paired.json (AppState::new loads it), so a real
GameStream-paired client leaked into body[0] on a dev box — now cleared first.

Live-validated against headless KWin: --compositor kwin honored, --compositor
mutter falls back to kwin (available=[kwin, gamescope]), resolved choice
round-trips to the client. Tests: +6 (wire/back-compat, resolution precedence,
endpoint); workspace green, clippy/fmt clean, C ABI harness PASS at abi_version=2,
web typecheck + build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 22:45:41 +02:00
enricobuehler 75eb8fa0d6 feat(host): KDE-reliability phase 2 — pipeline retry, graceful capture teardown, refresh reconcile 
Hardens the virtual-display → capture → encode bring-up against the transient
failures that surfaced as black screens / wrong refresh on cold KDE sessions.

- m3: build_pipeline_with_retry wraps the initial vd.create() + first-frame with
  bounded exponential backoff (4 attempts, 500ms→2s). is_permanent_build_error
  classifies config/version/missing-tool failures so they fail fast instead of
  burning the retry budget. Encoder + frame clock now pace to the *achieved*
  refresh reported in VirtualOutput::preferred_mode, not the requested rate.
- capture/linux: PortalCapturer::Drop sends a pipewire channel quit and joins the
  thread, so a dropped/failed/retried capturer releases its PipeWire thread + EGL/
  CUDA context promptly instead of leaking it to process exit. First-frame timeout
  now reports the node id and distinguishes "format never negotiated" from
  "negotiated but no buffers arrived" via a negotiated flag set in param_changed.
- vdisplay/kwin: set_custom_refresh reads back the active mode from kscreen-doctor
  and returns the refresh KWin actually gave us (a rejected custom mode silently
  leaves the output at 60Hz); create() carries it into preferred_mode.
- vdisplay/gamescope: find_gamescope_node requires the Video/Source object (the
  node.name=gamescope tag is on two objects; the other wedges the link); a version
  check warns on <3.16.22 (the PipeWire-1.6 capture-deadlock signature).

Live-validated against headless KWin: 720p120 build with requested=120 achieved=120,
zero-copy CUDA frames, and no per-session thread accumulation across back-to-back
sessions. Tests: +3 unit (retry classifier, gamescope version parse); 49 host tests
green, clippy/fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 22:45:41 +02:00
enricobuehler 9fdc3c3246 feat(headless-kde): reliable bring-up — readiness probe, fix portal ordering/env (roadmap #1 phase 1)
ci / rust (push) Has been cancelled
Details 
Headless KDE startup was a chain of timing-sensitive handoffs gated by a blind `sleep 2`,
the dominant source of black screens. Phase-1 fixes:

- New `punktfunk-host probe-compositor` subcommand: exits 0 iff the detected compositor is
  up AND ready to create a virtual output now. KWin gets a real check (connect + registry
  roundtrip + the privileged zkde_screencast global must be advertised — what the backend
  needs); gamescope/Mutter/wlroots create on demand so the probe just confirms Linux.
  (vdisplay::probe dispatcher + kwin::probe; reuses kwin.rs's existing roundtrip path.)
- run-headless-kde.sh: replace `sleep 2` with an active readiness wait (poll probe-compositor
  until ready, 30s deadline, and bail with kwin's log if kwin_wayland exits during init).
  Move the portal restart to AFTER readiness, and precede it with `systemctl --user
  import-environment` + `dbus-update-activation-environment` (the missing env import — the
  Sway script does this; without it a restarted portal inherits a stale/empty WAYLAND_DISPLAY,
  which is the "streams but eats no input/audio" failure). kwin's stderr → a log file.

Validated: probe-compositor exits 0 "Kwin ready" against the live session, exit 1 with a
clear diagnostic when the compositor is absent. 114 tests green, clippy/fmt clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 19:25:52 +00:00
enricobuehler ff4fe197be fix(punktfunk/1): adversarial-review fixes — SPAKE2 pairing, renegotiation hardening, +more
ci / rust (push) Has been cancelled
Details 
Triaged the multi-agent review of the renegotiation + pairing + Sway + AV1/surround batch
(1 critical, 11 major/minor confirmed). Fixes:

CRITICAL — PIN pairing was offline-brute-forceable. The HMAC-of-PIN proof let an active
MITM who terminates the TOFU ceremony recover the 4-digit PIN by offline dictionary search
(all other inputs observable) and forge a correctly-bound proof. Replaced with **SPAKE2**
(balanced PAKE, `spake2` crate) + key-confirmation MACs, binding both cert fingerprints as
the SPAKE2 identities: an attacker gets exactly ONE online guess, no offline search, and
mismatched cert views (a real MITM) never reach a shared key. Also reworked the UX to an
"arming PIN" — one PIN per arming window shown at host startup (the SPAKE2 client needs the
PIN to build its first message, so it can't be minted per-connection). Validated live:
wrong PIN rejected in 0.1s, right PIN pairs + persists + the paired identity streams.

Pairing hardening: `--allow-pairing`/`--require-pairing` must arm pairing (default rejects
unsolicited ceremonies); per-host cooldown bounds online guessing; the client flushes its
CONNECTION_CLOSE so a refused ceremony can't wedge the sequential host for the full timeout;
atomic (temp+rename) paired-store writes.

Protocol: control/pairing messages use a distinct CTL_MAGIC (PKFc) — fully disjoint from
the positional Hello namespace (a future abi_version can't be misparsed as a control
message); all typed decodes are length-exact. ABI_VERSION → 2 (punktfunk_connect signature
gained the identity params; header regenerated).

Renegotiation: drain the reconfig channel to the NEWEST mode (one rebuild, not one per
stale step); validate refresh_hz; build the new pipeline BEFORE dropping the old so a
rebuild failure keeps the session on its current mode instead of killing it.

GameStream: packetDuration snaps to {5,10} (an in-between value isn't a legal Opus frame
size and would kill audio). Sway: chooser file moved to $XDG_RUNTIME_DIR (was a fixed
world-writable /tmp path — DoS / capture-misdirection by another local user).

Swift: fixed two compile breakers in the new pairing/identity APIs (Int32 status .rawValue,
UInt cap cast). New SPAKE2 + namespace-disjointness + pairing-roundtrip unit tests; the
in-process pairing test now also exercises the arming PIN + cooldown. 114 tests green,
clippy -D warnings clean (both feature sets), fmt, C-ABI harness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 16:26:48 +00:00
enricobuehler 429bd1e6ac Merge branch 'worktree-agent-a6fe98c40d55fd284' into m1-lumen-core 
# Conflicts:
#	CLAUDE.md
 2026-06-10 15:42:48 +00:00
enricobuehler 4d26ac5c85 feat: punktfunk/1 — mid-stream mode renegotiation + PIN pairing ceremony 
Renegotiation (no reconnect on resize): the handshake bi-stream stays open; the client
sends Reconfigure{mode} (typed post-handshake message), the host validates + acks
Reconfigured and rebuilds capture/encoder/virtual output at the new mode while the data
plane (keys, ports, FEC) runs untouched — the first new-mode AU is an IDR with in-band
parameter sets. NativeClient::request_mode / punktfunk_connection_request_mode; mode()
reflects the active mode. Validated live on KWin: one continuous stream, 225 frames
@1280x720 then 395 @1920x1080, ~90 ms pipeline rebuild (ffprobe shows both resolutions).

PIN pairing (mutual trust, kills TOFU MITM): clients get persistent self-signed
identities presented via QUIC client auth (generate_identity / client auth offered but
optional server-side — legacy clients still connect). Ceremony on the control stream:
PairRequest{name} → host shows a 4-digit PIN (log) + PairChallenge{salt} → client proves
with HMAC-SHA256(PIN‖salt, client_fp‖host_fp) — binding both certs means a MITM can't
forward a proof, single attempt per PIN, constant-time compare → PairResult; host
persists the fingerprint (~/.config/punktfunk/punktfunk1-paired.json), client pins the
host's. m3-host --require-pairing gates sessions on the paired set.
NativeClient::pair + punktfunk_pair/punktfunk_generate_identity in the ABI; reference
client: --pair PIN --name LABEL + auto-generated persistent identity, --remode for live
renegotiation testing. Swift wrapper: ClientIdentity/generateIdentity()/pair(),
requestMode()/currentMode(); README handoff updated.

Tested: reconfigure/pairing wire roundtrips, C-ABI mode switch ack, full in-process
ceremony (wrong PIN → Crypto, anonymous-vs-gate rejection, success → pinned session);
live wrong-PIN ceremony against the serving host (PIN logged, proof rejected).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 15:42:29 +00:00
enricobuehler 3cc3c02b42 feat(gamestream): AV1 negotiation + 5.1/7.1 surround audio 
Codec negotiation (M2 polish):
- ServerCodecModeSupport now advertises what we encode: H264|HEVC|AV1_MAIN8
  = 65793 (flags verified against moonlight-common-c Limelight.h). The old
  placeholder 3843 wrongly claimed HEVC Main10 + 4:4:4 and no AV1. Main10
  bits stay off on purpose: Moonlight ties 10-bit to HDR, and capture is
  8-bit SDR BGRx with no HDR metadata path (av1_nvenc -highbitdepth was
  validated working for later).
- RTSP ANNOUNCE: bitStreamFormat 0/1/2 -> H264/HEVC/AV1 (already plumbed to
  av1_nvenc; validated e2e via `m0 --codec av1` + ffprobe av01), and a
  dynamicRangeMode!=0 request now logs + falls back to 8-bit SDR.

Surround audio (M2 polish):
- ANNOUNCE x-nv-audio.surround.{numChannels,AudioQuality} +
  x-nv-aqos.packetDuration -> per-session AudioParams; DESCRIBE advertises
  all six Opus configs (normal before HQ per channel count). Normal-quality
  mappings are pre-rotated for the client's GFE-order LFE swap
  (RtspConnection.c, verified verbatim) so its derived decoder mapping
  equals our encoder mapping — including 7.1, where Sunshine's rotate only
  covers [3,6) and scrambles LFE/SL/SR.
- 5.1/7.1 encode via libopus multistream (audiopus_sys, the sys layer the
  opus crate already links) with Sunshine's layouts/bitrates, RAII wrapper;
  the live-validated stereo wire is byte-identical (plain Opus, no FEC).
- Surround sessions add Sunshine-style RS(4,2) audio FEC (packetType 127 +
  AUDIO_FEC_HEADER, the OpenFEC parity matrix both ends hardcode, nanors
  gemm semantics verified from nanors/rs.c).
- PipeWire capture generalized to the negotiated channel count with explicit
  FL FR FC LFE RL RR [SL SR] positions; missing sink channels are zero-
  filled by the channel-mixer. PwAudioCapturer now tears down cleanly on
  Drop (pipewire channel -> loop quit), so a channel-count change can
  reopen without leaking a capture stream.

Tests: serverinfo mask, RTSP codec/audio param parsing, DESCRIBE contents,
surround-params strings + client-swap round trip, FEC parity self-recovery
and packet layout, real-codec 5.1 channel-identity round trip, and an
ignored live test (ran green against a 6ch null sink monitor).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 15:41:15 +00:00
enricobuehler 7381ba8218 feat(vdisplay): wlroots/Sway backend — swaymsg headless output + xdpw chooser 
The fourth VirtualDisplay backend: `swaymsg create_output` adds a HEADLESS-N
output (name found by diffing get_outputs), `output <NAME> mode --custom
WxH@HzHz` sets the client's exact mode (and the refresh clock a fresh headless
output needs to produce frames at all), and the PipeWire node comes from the
ScreenCast portal. Headless output selection is non-interactive via
xdg-desktop-portal-wlr's chooser hook: a managed config (chooser_type=simple,
chooser_cmd cats /tmp/punktfunk-xdpw-output; portal try-restarted when the
config changes) plus a per-session `Monitor: <NAME>` written to that file.
Teardown is RAII: drop ends the portal thread (zbus connection drop ends the
cast) then `swaymsg output <NAME> unplug`. swaymsg commands go after `--` so
tokens like `--custom` reach sway instead of swaymsg's getopt.

Validated live on headless sway 1.11 (gles2-on-NVIDIA, xdpw 0.8.1), zero-copy
dmabuf→CUDA on both runs: 720p60 257 frames p50 0.77 ms, 1080p60 480/480
frames p50 1.18 ms, output unplugged with the session both times. The
checked-in xdpw.config sample now matches the managed config (the old
chooser_type=none/HEADLESS-1 form would pin capture to the wrong output).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 15:23:28 +00:00
enricobuehler bfd64ce871 rename: lumen → punktfunk, everywhere
ci / rust (push) Has been cancelled
Details 
Full project rename, decided 2026-06-10:
- Crates/binaries: punktfunk-core / punktfunk-host / punktfunk-client-rs.
- C ABI: punktfunk_* symbols, Punktfunk* types, include/punktfunk_core.h,
  PUNKTFUNK_FEATURE_QUIC guard (header regenerated; cbindgen renames updated, incl.
  PUNKTFUNK_BTN_*/PUNKTFUNK_AXIS_* wire constants).
- Protocol: punktfunk/1 — control-plane magic LMN1 → PKF1, nonce salt lmn1 → pkf1.
  WIRE BREAK: clients must be rebuilt from this revision.
- Env knobs: PUNKTFUNK_VIDEO_SOURCE / PUNKTFUNK_COMPOSITOR / PUNKTFUNK_ZEROCOPY / ….
- Host config dir: ~/.config/punktfunk (the box's dir was migrated in place — the
  persistent identity is unchanged, pinned fingerprints stay valid).
- Swift package: PunktfunkKit + PunktfunkCore.xcframework + PunktfunkConnection
  (Sources/PunktfunkClient app + tests renamed with it); build-xcframework.sh updated.
- scripts/: 60-punktfunk.rules, punktfunk-host.service; OpenAPI doc regenerated.

Also: scripts/headless/run-headless-kde.sh — full headless Plasma bringup. Root cause of
"desktop but no apps/settings" over the stream: plasmashell launched without
XDG_MENU_PREFIX=plasma-, so the launcher resolved a nonexistent applications.menu and
rendered an empty menu. The script sets the complete KDE session env (menu prefix,
KDE_FULL_SESSION, session version) and rebuilds ksycoca before starting plasmashell.

Gate: 97/97 tests, clippy -D warnings (both feature sets), fmt, C-ABI harness PASS,
zero lumen references left outside .git.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 13:11:59 +00:00