enricobuehler
e5b15353c7
ci / rust (push) Has been cancelled
4-agent feasibility read converged on three independent walls, any one fatal:
- host capture needs a kernel rebuild (CONFIG_USB_DUMMY_HCD off → no UDC for an
f_uac2 composite gadget; everything else for the gadget IS present);
- near-zero Linux supply (only ~5-10 Proton titles via custom Wine patches emit
it; hid-playstation/Steam-Input/RPCS3 don't);
- Apple client can't faithfully replay PCM haptics (CoreHaptics is discrete
pattern-based; no public CoreAudio channel-3/4 routing).
Advanced haptics ride the DualSense USB *audio* interface, not HID, so the UHID
backend structurally can't carry them. Defer; the reachable 80% ("real DualSense
feel") is adaptive triggers over the HID 0x02 path we already parse + two-motor
rumble. New docs/dualsense-haptics.md records the walls + conditions for a future
go; roadmap §5 updated (HID DualSense backend built & live-validated).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
116 lines
8.0 KiB
Markdown
116 lines
8.0 KiB
Markdown
# punktfunk roadmap — next goals
|
||
|
||
Decided 2026-06-10 (research-grounded; see commit history). Sequence:
|
||
**KDE reliability → client compositor options → mic passthrough → Bazzite COPR RPM (then
|
||
bootc) → touch → full UHID DualSense → iOS** (+ Windows host, scoped & deferred).
|
||
|
||
**Done (2026-06-10):** #1 KDE reliability (Phase 1 + 2), #2 compositor options (full stack incl.
|
||
macOS client), #4 mic passthrough — all on `main`, live-validated. #3 Bazzite packaging written
|
||
(`packaging/`); the COPR/bootc build is operator-run. Remaining: #5 touch → UHID DualSense, #6 iOS,
|
||
and a Windows host (`docs/windows-host.md`).
|
||
|
||
## 1. Reliable headless KDE/compositor spawning ✅ *(done — Phase 1 + 2)*
|
||
|
||
Startup is a chain of timing-sensitive handoffs with no readiness checks — each is a blind
|
||
`sleep`, one-shot timeout, or silent fire-and-forget that fails into a black screen.
|
||
|
||
- **Phase 1 (S):** replace `run-headless-kde.sh`'s blind `sleep 2` with an active readiness
|
||
wait (kwin socket + `wl_display` roundtrip + `zkde_screencast` global advertised +
|
||
KWIN_PID alive); add a `punktfunk-host probe-compositor` subcommand (reuses kwin.rs's
|
||
registry roundtrip); move the portal restart to *after* readiness and precede it with
|
||
`systemctl --user import-environment` + `dbus-update-activation-environment` (the missing
|
||
env import — the Sway script does this, the KDE one doesn't).
|
||
- **Phase 2 (M):** bounded retry-with-backoff around `vd.create()` + first-frame
|
||
(permanent vs transient); a PipeWire negotiation watchdog with zero-copy→CPU auto-fallback
|
||
("no PipeWire frame within 10s" → recovery or precise diagnosis); fix `set_custom_refresh`
|
||
to wait for the output, read back the active mode, reconcile encoder fps; harden gamescope
|
||
node discovery + detect the known-bad-gamescope signature; graceful PipeWire-thread stop.
|
||
- **Phase 3 (L):** supervised systemd user session (kwin + portal + host) with the readiness
|
||
probe as an `ExecStartPost` gate, `Restart=on-failure`.
|
||
|
||
## 2. Offer available compositors in the client ✅ *(done)*
|
||
|
||
Host enumerates which backends are actually available (binary present + version OK:
|
||
gamescope ≥3.16.22, KWin ≥6.5.6, gnome-shell, sway), advertises the list in the punktfunk/1
|
||
Welcome + a mgmt-API field; client sends its pick in the Hello; host honors it per session.
|
||
Picker in the Apple client + web console.
|
||
|
||
## 3. Bazzite / install on other devices ✅ *(packaging written — `packaging/`)*
|
||
|
||
Bazzite already ships gamescope + PipeWire + the NVIDIA driver (incl. `libnvidia-encode`);
|
||
it's Fedora-atomic and the community installs Sunshine via COPR rpm-ostree — the analog.
|
||
Written: `packaging/rpm/punktfunk.spec` (builds the host from source), `packaging/bootc/Containerfile`
|
||
(`FROM bazzite-nvidia`), `packaging/bazzite/host.env` (gamescope default), `packaging/copr/` +
|
||
`packaging/README.md`. The build itself is operator-run (COPR / a Fedora toolbox; not buildable on
|
||
the Ubuntu dev box). `LICENSE-{MIT,APACHE}` added to match the declared dual license.
|
||
|
||
- **M-Bazzite-1:** a **COPR RPM** (primary) — binary + `60-punktfunk.rules` (→
|
||
`/usr/lib/udev/rules.d`) + systemd `--user` unit + `host.env.example`; `Requires` the
|
||
NVENC ffmpeg-libs Bazzite already pulls; links host `libcuda`/`libnvidia-encode` directly.
|
||
Install = `rpm-ostree install` + reboot + add to `input`/`render`. Default backend =
|
||
Bazzite's already-present **gamescope** (minimal session plumbing).
|
||
- **M-Bazzite-2:** wrap the RPM in a **bootc/OCI image layer** (`FROM
|
||
ghcr.io/ublue-os/bazzite-nvidia:stable`) for the appliance/"just rebase" experience.
|
||
- Flatpak only later as an explicitly-degraded convenience build (sandbox fights
|
||
zero-copy NVENC/dmabuf/uinput).
|
||
|
||
## 4. Mic passthrough — client mic → host input device ✅ *(done — host side)*
|
||
|
||
The exact mirror of the host→client desktop-audio path. A PipeWire virtual source apps can
|
||
select = a `pw_stream` with `Direction::Output` + `media.class=Audio/Source`.
|
||
|
||
- New `0xCB` MIC_AUDIO datagram (mirror of `0xC9`) + `NativeClient::send_audio` + ABI
|
||
`punktfunk_send_audio`.
|
||
- `audio/source_linux.rs` — near-copy of the capture file, Direction::Output, fed from a
|
||
jitter buffer (silence-fill underrun, Opus PLC).
|
||
- Host `mic_thread` (Opus decode → ring → source); teardown RAII, set `node.dont-reconnect`.
|
||
- Apple capture (AVAudioEngine → Opus). **Opt-in + paired-only** (a remote mic is a privacy
|
||
surface). punktfunk/1-only.
|
||
|
||
## 5. Touch + rich DualSense *(decision: commit to full UHID DualSense)*
|
||
|
||
- **Touch — implemented (host path), pending a backend that lands it.** `TouchDown/Move/Up`
|
||
InputKinds (reuse the abs-pointer `flags=(w<<16)|h` mapping, `code`=touch id); host
|
||
`inject/libei.rs` requests the `Touchscreen` device type + binds the `Touch` capability and
|
||
injects `ei_touchscreen` down/motion/up; `punktfunk-client-rs --touch-test` drags a finger.
|
||
**Validated:** KWin's RemoteDesktop portal *grants* the Touchscreen device type, but its EIS
|
||
server creates **no touchscreen device** (headless KWin) — so touch currently no-ops on KWin
|
||
(now logged once). The code is correct; it needs a backend that exposes `ei_touchscreen`
|
||
(gamescope / newer KWin / the real iPad client path) to land. wlroots: no virtual-touch wired.
|
||
- **Rich DualSense — HID backend built & validated live.** `inject/dualsense.rs`: a hand-rolled
|
||
`/dev/uhid` codec (no bindgen) presenting a genuine USB DualSense (vendor 054C/0CE6, the 232-byte
|
||
inputtino report descriptor) bound by the kernel `hid-playstation` driver. The mandatory
|
||
GET_REPORT feature handshake (calibration 0x05 / pairing 0x09 / firmware 0x20) is answered, so the
|
||
kernel creates the full device (gamepad/motion/touchpad/lightbar). Input report `0x01` is built
|
||
from gamepad frames; output report `0x02` is parsed for LED RGB, player LEDs, and **adaptive
|
||
trigger effects (L2/R2)**. Protocol carries new side-planes: rich-input `0xCC`
|
||
(touchpad/motion) + HID-output `0xCD` (LED/triggers). `/dev/uhid` udev rule shipped.
|
||
*Remaining (paused, resume-able):* route gamepad frames → `DualSensePad` behind
|
||
`PUNKTFUNK_GAMEPAD=dualsense`, wire the `0xCC`/`0xCD` back-channel end-to-end (+ C ABI
|
||
`next_hidout`/`send_rich_input`), and render adaptive triggers + rumble on the Apple client.
|
||
- **Advanced (audio-driven voice-coil) haptics — scoped, NO-GO for now (`docs/dualsense-haptics.md`).**
|
||
Driven by the DualSense's USB *audio* interface (4-ch, back 2 channels = haptic PCM), not HID — so
|
||
the UHID backend structurally can't carry it. Three independent walls: host capture needs a kernel
|
||
rebuild (`CONFIG_USB_DUMMY_HCD` is off → no UDC for an `f_uac2` gadget); **near-zero Linux supply**
|
||
(only ~5–10 Proton titles via custom Wine patches emit it; `hid-playstation`/Steam Input/RPCS3
|
||
don't); and the Apple client can't faithfully replay PCM haptics (CoreHaptics is discrete/pattern-
|
||
based, no public channel-3/4 routing). Deferred; revisit only if a real DS for capture + a UDC/host
|
||
path + a PCM-capable client all land. Adaptive triggers (HID, above) deliver the reachable 80%.
|
||
|
||
## 6. iOS/iPadOS → tvOS *(deferred)*
|
||
|
||
PunktfunkKit is already platform-shared; iOS needs the `UIViewRepresentable` presenter twin
|
||
+ touch capture (#5) + UI. tvOS later.
|
||
|
||
## 7. Windows as a host *(scoped & deferred — `docs/windows-host.md`)*
|
||
|
||
Architecturally an "add a backend" job, not a parallel port: `punktfunk-core` (protocol/FEC/
|
||
crypto/C-ABI) + QUIC + GameStream + mgmt + the `m3`/pipeline orchestration are all platform-agnostic
|
||
and already `cfg`-isolated (~95% reuse). New `#[cfg(windows)]` backends behind the existing traits:
|
||
capture (DXGI Desktop Duplication), encode (Media Foundation / NVENC-SDK with a D3D11 context),
|
||
input (SendInput + ViGEm), audio (WASAPI loopback + a virtual mic). **The blocker** is the
|
||
virtual-display feature — no user-mode Windows API; it needs a signed kernel-mode **IDD** driver
|
||
(XL). Recommended start: **Phase 0** — a "basic Windows host" capturing an existing monitor (no
|
||
virtual display), proving the whole stack with the smallest surface. Deferred because it's large and
|
||
unbuildable on the Linux dev box; the trait boundaries are already in the right places.
|