enricobuehler
59edeedf07
ci / rust (push) Has been cancelled
PUNKTFUNK_GAMEPAD=dualsense now routes a session's gamepad through a real virtual
DualSense (UHID + hid-playstation) end to end:
- host: a `PadBackend` enum (m3.rs) selects `GamepadManager` (uinput xpad, default)
or the new `DualSenseManager` (dualsense.rs) per session. The manager keeps each
pad's full DsState so touchpad + motion (rich-input plane) persist across
button/stick frames, and services the !Send /dev/uhid fd only on the input thread
(which cycles <=4ms, so the GET_REPORT init handshake completes).
- feedback: `service()` now returns `DsFeedback { hidout, rumble }`. Motor rumble
stays on the universal 0xCA plane (so non-DualSense clients still feel it; manager
dedups change); lightbar / player LEDs / adaptive-trigger effects ride the new
0xCD HID-output plane (host->client) as `HidOutput`.
- rich input: touchpad contacts + motion ride the 0xCC plane (client->host) as
`RichInput`, applied via `DualSenseManager::apply_rich` (merged with button state;
touch normalized 0..65535 -> the touchpad resolution).
- connector + C ABI: `NativeClient::next_hidout` / `send_rich_input`, exported as
`punktfunk_connection_next_hidout` (-> PunktfunkHidOutput) and
`punktfunk_connection_send_rich_input` (<- PunktfunkRichInput); header regenerated.
- reference client: `--rich-input-test` drives the DualSense touchpad + motion and
logs the 0xCD feedback that comes back.
Validated live on-box: a synthetic-source m3-host + client-rs created the real
kernel DualSense, drove 0xCC, and decoded 12 live 0xCD events (the kernel's actual
lightbar/trigger init reports) with the data plane unaffected (600/600 frames).
Adversarial review fixes folded in: the input loop no longer skips the rich drain +
feedback pump on a dropped gamepad event, and the touch contact id is clamped to its
slot. Remaining: the Apple client renders triggers/rumble on a real DualSense.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
125 lines
8.9 KiB
Markdown
125 lines
8.9 KiB
Markdown
# punktfunk roadmap — next goals
|
||
|
||
Decided 2026-06-10 (research-grounded; see commit history). Sequence:
|
||
**KDE reliability → client compositor options → mic passthrough → Bazzite COPR RPM (then
|
||
bootc) → touch → full UHID DualSense → iOS** (+ Windows host, scoped & deferred).
|
||
|
||
**Done (2026-06-10):** #1 KDE reliability (Phase 1 + 2), #2 compositor options (full stack incl.
|
||
macOS client), #4 mic passthrough — all on `main`, live-validated. #3 Bazzite packaging written
|
||
(`packaging/`); the COPR/bootc build is operator-run. Remaining: #5 touch → UHID DualSense, #6 iOS,
|
||
and a Windows host (`docs/windows-host.md`).
|
||
|
||
## 1. Reliable headless KDE/compositor spawning ✅ *(done — Phase 1 + 2)*
|
||
|
||
Startup is a chain of timing-sensitive handoffs with no readiness checks — each is a blind
|
||
`sleep`, one-shot timeout, or silent fire-and-forget that fails into a black screen.
|
||
|
||
- **Phase 1 (S):** replace `run-headless-kde.sh`'s blind `sleep 2` with an active readiness
|
||
wait (kwin socket + `wl_display` roundtrip + `zkde_screencast` global advertised +
|
||
KWIN_PID alive); add a `punktfunk-host probe-compositor` subcommand (reuses kwin.rs's
|
||
registry roundtrip); move the portal restart to *after* readiness and precede it with
|
||
`systemctl --user import-environment` + `dbus-update-activation-environment` (the missing
|
||
env import — the Sway script does this, the KDE one doesn't).
|
||
- **Phase 2 (M):** bounded retry-with-backoff around `vd.create()` + first-frame
|
||
(permanent vs transient); a PipeWire negotiation watchdog with zero-copy→CPU auto-fallback
|
||
("no PipeWire frame within 10s" → recovery or precise diagnosis); fix `set_custom_refresh`
|
||
to wait for the output, read back the active mode, reconcile encoder fps; harden gamescope
|
||
node discovery + detect the known-bad-gamescope signature; graceful PipeWire-thread stop.
|
||
- **Phase 3 (L):** supervised systemd user session (kwin + portal + host) with the readiness
|
||
probe as an `ExecStartPost` gate, `Restart=on-failure`.
|
||
|
||
## 2. Offer available compositors in the client ✅ *(done)*
|
||
|
||
Host enumerates which backends are actually available (binary present + version OK:
|
||
gamescope ≥3.16.22, KWin ≥6.5.6, gnome-shell, sway), advertises the list in the punktfunk/1
|
||
Welcome + a mgmt-API field; client sends its pick in the Hello; host honors it per session.
|
||
Picker in the Apple client + web console.
|
||
|
||
## 3. Bazzite / install on other devices ✅ *(packaging written — `packaging/`)*
|
||
|
||
Bazzite already ships gamescope + PipeWire + the NVIDIA driver (incl. `libnvidia-encode`);
|
||
it's Fedora-atomic and the community installs Sunshine via COPR rpm-ostree — the analog.
|
||
Written: `packaging/rpm/punktfunk.spec` (builds the host from source), `packaging/bootc/Containerfile`
|
||
(`FROM bazzite-nvidia`), `packaging/bazzite/host.env` (gamescope default), `packaging/copr/` +
|
||
`packaging/README.md`. The build itself is operator-run (COPR / a Fedora toolbox; not buildable on
|
||
the Ubuntu dev box). `LICENSE-{MIT,APACHE}` added to match the declared dual license.
|
||
|
||
- **M-Bazzite-1:** a **COPR RPM** (primary) — binary + `60-punktfunk.rules` (→
|
||
`/usr/lib/udev/rules.d`) + systemd `--user` unit + `host.env.example`; `Requires` the
|
||
NVENC ffmpeg-libs Bazzite already pulls; links host `libcuda`/`libnvidia-encode` directly.
|
||
Install = `rpm-ostree install` + reboot + add to `input`/`render`. Default backend =
|
||
Bazzite's already-present **gamescope** (minimal session plumbing).
|
||
- **M-Bazzite-2:** wrap the RPM in a **bootc/OCI image layer** (`FROM
|
||
ghcr.io/ublue-os/bazzite-nvidia:stable`) for the appliance/"just rebase" experience.
|
||
- Flatpak only later as an explicitly-degraded convenience build (sandbox fights
|
||
zero-copy NVENC/dmabuf/uinput).
|
||
|
||
## 4. Mic passthrough — client mic → host input device ✅ *(done — host side)*
|
||
|
||
The exact mirror of the host→client desktop-audio path. A PipeWire virtual source apps can
|
||
select = a `pw_stream` with `Direction::Output` + `media.class=Audio/Source`.
|
||
|
||
- New `0xCB` MIC_AUDIO datagram (mirror of `0xC9`) + `NativeClient::send_audio` + ABI
|
||
`punktfunk_send_audio`.
|
||
- `audio/source_linux.rs` — near-copy of the capture file, Direction::Output, fed from a
|
||
jitter buffer (silence-fill underrun, Opus PLC).
|
||
- Host `mic_thread` (Opus decode → ring → source); teardown RAII, set `node.dont-reconnect`.
|
||
- Apple capture (AVAudioEngine → Opus). **Opt-in + paired-only** (a remote mic is a privacy
|
||
surface). punktfunk/1-only.
|
||
|
||
## 5. Touch + rich DualSense *(decision: commit to full UHID DualSense)*
|
||
|
||
- **Touch — implemented (host path), pending a backend that lands it.** `TouchDown/Move/Up`
|
||
InputKinds (reuse the abs-pointer `flags=(w<<16)|h` mapping, `code`=touch id); host
|
||
`inject/libei.rs` requests the `Touchscreen` device type + binds the `Touch` capability and
|
||
injects `ei_touchscreen` down/motion/up; `punktfunk-client-rs --touch-test` drags a finger.
|
||
**Validated:** KWin's RemoteDesktop portal *grants* the Touchscreen device type, but its EIS
|
||
server creates **no touchscreen device** (headless KWin) — so touch currently no-ops on KWin
|
||
(now logged once). The code is correct; it needs a backend that exposes `ei_touchscreen`
|
||
(gamescope / newer KWin / the real iPad client path) to land. wlroots: no virtual-touch wired.
|
||
- **Rich DualSense — HID backend built & validated live.** `inject/dualsense.rs`: a hand-rolled
|
||
`/dev/uhid` codec (no bindgen) presenting a genuine USB DualSense (vendor 054C/0CE6, the 232-byte
|
||
inputtino report descriptor) bound by the kernel `hid-playstation` driver. The mandatory
|
||
GET_REPORT feature handshake (calibration 0x05 / pairing 0x09 / firmware 0x20) is answered, so the
|
||
kernel creates the full device (gamepad/motion/touchpad/lightbar). Input report `0x01` is built
|
||
from gamepad frames; output report `0x02` is parsed for LED RGB, player LEDs, and **adaptive
|
||
trigger effects (L2/R2)**. Protocol carries new side-planes: rich-input `0xCC`
|
||
(touchpad/motion) + HID-output `0xCD` (LED/triggers). `/dev/uhid` udev rule shipped.
|
||
- **Rich DualSense — Phase C/D/E end-to-end, validated live.** `PUNKTFUNK_GAMEPAD=dualsense`
|
||
selects a per-session `DualSenseManager` (the `PadBackend` enum in `m3.rs`): client gamepad frames
|
||
build the DualSense report; the kernel's feedback comes back as `HidOutput` on the **0xCD** plane
|
||
(lightbar / player LEDs / adaptive triggers) while **rumble stays on the universal 0xCA plane**
|
||
(so non-DualSense clients still feel it); touchpad + motion ride the **0xCC** rich-input plane
|
||
(`DualSenseManager::apply_rich`, merged with button state). The connector + C ABI gained
|
||
`punktfunk_connection_next_hidout` (→ `PunktfunkHidOutput`) and `punktfunk_connection_send_rich_input`
|
||
(← `PunktfunkRichInput`); header regenerated. Validated on-box: a synthetic-source `m3-host` +
|
||
`punktfunk-client-rs --rich-input-test` created the real kernel DualSense, drove 0xCC, and decoded
|
||
12 live 0xCD events (the kernel's actual lightbar/trigger init reports) — data plane unaffected
|
||
(600/600 frames). *Remaining:* the Apple client renders adaptive triggers + rumble on a real
|
||
DualSense (`GCDualSenseAdaptiveTrigger`) — handed off to the client agent for the real playtest.
|
||
- **Advanced (audio-driven voice-coil) haptics — scoped, NO-GO for now (`docs/dualsense-haptics.md`).**
|
||
Driven by the DualSense's USB *audio* interface (4-ch, back 2 channels = haptic PCM), not HID — so
|
||
the UHID backend structurally can't carry it. Three independent walls: host capture needs a kernel
|
||
rebuild (`CONFIG_USB_DUMMY_HCD` is off → no UDC for an `f_uac2` gadget); **near-zero Linux supply**
|
||
(only ~5–10 Proton titles via custom Wine patches emit it; `hid-playstation`/Steam Input/RPCS3
|
||
don't); and the Apple client can't faithfully replay PCM haptics (CoreHaptics is discrete/pattern-
|
||
based, no public channel-3/4 routing). Deferred; revisit only if a real DS for capture + a UDC/host
|
||
path + a PCM-capable client all land. Adaptive triggers (HID, above) deliver the reachable 80%.
|
||
|
||
## 6. iOS/iPadOS → tvOS *(deferred)*
|
||
|
||
PunktfunkKit is already platform-shared; iOS needs the `UIViewRepresentable` presenter twin
|
||
+ touch capture (#5) + UI. tvOS later.
|
||
|
||
## 7. Windows as a host *(scoped & deferred — `docs/windows-host.md`)*
|
||
|
||
Architecturally an "add a backend" job, not a parallel port: `punktfunk-core` (protocol/FEC/
|
||
crypto/C-ABI) + QUIC + GameStream + mgmt + the `m3`/pipeline orchestration are all platform-agnostic
|
||
and already `cfg`-isolated (~95% reuse). New `#[cfg(windows)]` backends behind the existing traits:
|
||
capture (DXGI Desktop Duplication), encode (Media Foundation / NVENC-SDK with a D3D11 context),
|
||
input (SendInput + ViGEm), audio (WASAPI loopback + a virtual mic). **The blocker** is the
|
||
virtual-display feature — no user-mode Windows API; it needs a signed kernel-mode **IDD** driver
|
||
(XL). Recommended start: **Phase 0** — a "basic Windows host" capturing an existing monitor (no
|
||
virtual display), proving the whole stack with the smallest surface. Deferred because it's large and
|
||
unbuildable on the Linux dev box; the trait boundaries are already in the right places.
|