Files
punktfunk/design/steam-deck-passthrough-plan.md
T
enricobuehler 580b1ea7a7
apple / screenshots (push) Has been cancelled
android / android (push) Has been cancelled
apple / swift (push) Has been cancelled
audit / cargo-audit (push) Has been cancelled
ci / web (push) Has been cancelled
ci / docs-site (push) Has been cancelled
ci / bench (push) Has been cancelled
ci / rust (push) Has been cancelled
deb / build-publish (push) Has been cancelled
decky / build-publish (push) Has been cancelled
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Has been cancelled
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Has been cancelled
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Has been cancelled
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Has been cancelled
docker / deploy-docs (push) Has been cancelled
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Has been cancelled
flatpak / build-publish (push) Has been cancelled
release / apple (push) Has been cancelled
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Has been cancelled
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Has been cancelled
windows-host / package (push) Has been cancelled
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Has been cancelled
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Has been cancelled
windows / build (aarch64-pc-windows-msvc) (push) Has been cancelled
windows / build (x86_64-pc-windows-msvc) (push) Has been cancelled
feat(host/steam): shippable usbip/vhci_hcd virtual Deck + client leave-shortcuts
Steam Deck pass-through (design/steam-deck-passthrough-plan.md), code-complete +
all CI checks green on Linux + adversarially reviewed; on-glass validation pending:

- usbip/`vhci_hcd` virtual Deck transport (inject/linux/steam_usbip.rs) for
  non-SteamOS hosts (Bazzite/generic) — presents a real interface-2 USB Deck so
  Steam Input promotes it. In-process vhci attach (loopback OP_REQ_IMPORT handshake
  → sysfs attach) with a bounded `usbip`-CLI fallback; detach on drop.
- Backed by a vendored, libusb-free trim of the `usbip` crate
  (crates/punktfunk-host/vendor/usbip-sim, MIT + NOTICE; host/cdc/hid + rusb/nusb
  removed; interrupt-IN paced by bInterval).
- Selection ladder raw_gadget (SteamOS fast-path) → usbip (universal) → UHID,
  with PUNKTFUNK_STEAM_USBIP / PUNKTFUNK_USBIP_ATTACH knobs.
- Shared Deck descriptors + the 0x83/0xAE feature contract + a Steam-accepted
  serial consolidated into steam_proto.rs; the raw_gadget backend reuses them.
- Linux client leave-shortcuts: Ctrl+Alt+Shift+D + holding the escape chord
  (L1+R1+Start+Select) >=1.5s end the session (short press still exits
  fullscreen); the chord state resets across sessions.

Also bundles in-progress work already staged in the tree:
- host(kwin): xdg-output logical-geometry mapping so the KWin fake_input backend
  places absolute coordinates correctly under display scaling.
- docs: design/README index entries + design/controller-only-mode.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 19:17:37 +00:00

206 lines
15 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Plan: production-ready Steam Deck pass-through (client) + shippable virtual Deck on any Linux host
> **Status (2026-06-29): BUILT — code-complete, all CI checks green on Linux (build · `clippy
> -D warnings` · `fmt` · ~270 tests), adversarially reviewed; NOT yet on-glass validated, NOT pushed.**
> Implemented in one pass:
> - **usbip/`vhci_hcd` transport** (`crates/punktfunk-host/src/inject/linux/steam_usbip.rs`) presenting
> a real interface-2 USB Deck, with **in-process vhci attach** (sysfs `OP_REQ_IMPORT` handshake) and a
> bounded `usbip`-CLI fallback. Backed by a **vendored, libusb-free trim** of the `usbip` crate
> (`crates/punktfunk-host/vendor/usbip-sim/`, MIT, see its NOTICE).
> - **Selection ladder** `raw_gadget` (SteamOS) → `usbip` (`vhci_hcd`, universal) → UHID
> (`steam_controller.rs::open_transport`); `PUNKTFUNK_STEAM_USBIP=0/1`, `PUNKTFUNK_USBIP_ATTACH=inproc|cli`.
> - **Shared Deck device contract** (captured descriptors + `0x83`/`0xAE` `feature_reply` + a
> Steam-accepted serial) consolidated into `steam_proto.rs`; the gadget now reuses it.
> - **Client leave-shortcuts**: keyboard **Ctrl+Alt+Shift+D** + controller **hold the escape chord
> (L1+R1+Start+Select) ≥1.5 s** → disconnect (short press still leaves fullscreen). Steam/QAM are NOT
> in the chord. Linux client only for now (windows/apple/android mirror is future work).
>
> **Decisions taken** (the plan's open questions): vendor-trim the crate (no libusb) ✓; in-process
> attach primary + CLI fallback ✓; escalate the existing escape chord (long-hold) ✓; keep BOTH
> `raw_gadget` (SteamOS fast-path) and usbip (universal) behind the transport ladder ✓.
>
> **Remaining = §6 on-glass validation** (Bazzite `192.168.1.41` + Deck `192.168.1.253`): confirm the
> in-process usbip attach promotes the Deck in game mode (Steam + QAM reach the game-mode UI), the
> raw_gadget path still works on SteamOS (regression), and the leave-shortcuts fire. The dev box has no
> Steam + no root, so this could not be run here.
>
> Companion doc: [`steam-controller-deck-support.md`](steam-controller-deck-support.md)
> (the virtual-Deck design + §11 the interface-2 / gadget story). The virtual Steam Deck that Steam
> Input promotes is **already built, hardware-validated, and default-on for SteamOS**
> (`raw_gadget`/`dummy_hcd`, glass-confirmed in-game on a real Deck).
## Goal
When a Steam Deck (or any Valve controller) is the **client**, streaming to a Linux **host** running
Steam (SteamOS *or* Bazzite/generic), every Deck control — including the **Steam** logo button and the
**QAM "…"** (quick-access) button — passes through and drives the host's game-mode UI, so it feels
native. Plus a leave-shortcut (controller + keyboard) since Steam/QAM now pass through.
## What's already true (do NOT rebuild — verified by investigation `wf_f5e3528b-3ef`)
The **client capture is already correct**, and the **wire + host mapping already carry Steam + QAM**:
- SDL3's HIDAPI Steam Deck driver exposes **Steam → `Button::Guide`** (joystick b5) and **QAM "…" →
`Button::Misc1`** (joystick b11→`misc1` in `SDL_gamepad_db.h:729`; confirmed in `SDL_gamepad.h`:
*"Steam Controller QAM"* = `MISC1`). Paddles → `RightPaddle1/2`,`LeftPaddle1/2`; trackpad clicks →
`Touchpad`/`Misc2`.
- `clients/linux/src/gamepad.rs:173-201` `button_bit()` already maps `Guide → wire::BTN_GUIDE`,
`Misc1 → wire::BTN_MISC1`, all four paddles, touchpad.
- Wire buttons are **`u32`** (`crates/punktfunk-core/src/input.rs:54-86`): `BTN_GUIDE=0x0400` (bit 10),
`BTN_MISC1=0x0020_0000` (bit 21); free bits = **11, 22-31**. Buttons ride as individual
`InputEvent` (0xC8) events (`code`=bit, `x`=1/0); rich input (touchpad/gyro) on `0xCC`.
- Host `steam_proto::from_gamepad` (`crates/punktfunk-host/src/inject/proto/steam_proto.rs:179-233`)
already maps **`BTN_GUIDE → btn::STEAM`** (line 214) and **`BTN_MISC1 → btn::QAM`** (line 230). The
`btn` module: `STEAM = 1<<13`, `QAM = 1<<50`.
- **Caveat that matters:** SDL only surfaces Steam/QAM when **Steam Input is NOT grabbing the
controller on the client** (else Steam consumes them globally and hands the app its virtual Xbox
pad, which lacks Steam/QAM). The fix is *disable Steam Input for the client* — already the Decky
plugin's "Disable Steam Input" UX. SDL's HIDAPI Deck driver is on by default on Linux
(`SDL_HIDAPI_DEFAULT`); `SDL_JOYSTICK_HIDAPI_STEAMDECK=1` is already set in `gamepad.rs:446-456`.
- **Only the Deck host backend carries QAM.** The Xbox/xpad map
(`inject/linux/gamepad.rs:80-97`) and DualSense (`dualsense_proto.rs:214`) map Guide→MODE/PS but
**drop `BTN_MISC1` (QAM)** — they have no slot for it. So QAM-to-game-mode *requires* the virtual
Deck backend (gadget or usbip). This is expected and correct.
**Net:** for SteamOS hosts the whole feature already works today (client capture → gadget Deck →
Steam Input). The remaining work is the *non-SteamOS host* (usbip) + the leave-shortcut + polish.
## Architecture decision: usbip/`vhci_hcd` is the shippable universal transport
Presenting a real interface-2 USB Deck on a generic Linux host is the only gap. Decision matrix:
| Mechanism | Ships? | Why |
|---|---|---|
| `dummy_hcd` + `raw_gadget` | SteamOS only | In-tree on SteamOS (used + validated). **Not** built on Bazzite/Fedora (`CONFIG_USB_DUMMY_HCD`/`RAW_GADGET` unset); building them needs `kernel-devel` **and** MOK-signing under Secure Boot → **not shippable**. |
| **`usbip` + `vhci_hcd`** | **everywhere** | **In-tree + signed** on SteamOS, Bazzite, and ~every distro (it's the standard usbip stack). Loads under Secure Boot, **no module build, no MOK**. A userspace usbip server emulates the Deck; `vhci_hcd` attaches it locally. |
**Both validated on hardware (2026-06-29):**
- `raw_gadget` Deck on a real Steam Deck → Steam promotes it, glass-confirmed in-game.
- `usbip` Deck on **Bazzite**`usbip attach -r 127.0.0.1 -b 0-0-0``vhci_hcd` enumerates the
3-interface Deck, `hid-steam` binds it, reads the serial, makes the `Steam Deck`/`Motion Sensors`
evdevs, **stable (1 connect / 0 disconnect)**, and Steam logs `Interface: 2 … opened for index …
reserving XInput slot 1` + emits an X-Box pad. **Identical recognition to the gadget.**
The working PoC is checked in at `packaging/linux/steam-deck-gadget/usbip-poc/` — the new session
should build on it. It uses the `usbip` crate (jiegec/usbip v0.8.0): a custom `UsbInterfaceHandler`
(`get_class_specific_descriptor` = the 9-byte HID descriptor; `handle_urb` = GET report-descriptor /
HID `GET_REPORT`=`feature_reply` / `SET_REPORT` / interrupt-IN = the 64-byte Deck state), reusing the
exact captured descriptors + feature contract from `steam_gadget.rs`.
## Build steps (ordered)
### 1. Refactor `steam_gadget.rs` into shared Deck-logic + a transport trait
The descriptor set (mouse/kbd/controller report descriptors, the device/config assembly), the
`feature_reply` (0x83 attributes + 0xAE serial), and `serialize_deck_state` are **transport-agnostic**
and already proven. Extract them into a shared module (e.g. `inject/proto/steam_proto.rs` already holds
`serialize_deck_state`/`feature_reply`-equivalents; consolidate the gadget's `feature_reply` +
descriptors there or a new `steam_device.rs`). Define:
```rust
/// A virtual Deck transport: feed it the current 64-byte state, drain feedback.
trait DeckTransport {
fn write_state(&mut self, st: &SteamState);
fn service(&mut self) -> Option<(u16, u16)>; // rumble
}
```
Make the existing `raw_gadget` `SteamDeckGadget` implement it (it already has `write_state`/`service`).
### 2. Add the usbip transport (`SteamDeckUsbip`)
- Reuse the PoC's device definition + handler. Drive the interrupt-IN report from the shared
`SteamState` (a `Arc<Mutex<[u8;64]>>` the handler reads), updated by `write_state`.
- **Dependency decision:** the `usbip` crate hard-depends on `rusb``libusb1-sys` (for its *host*
mode, which we don't use; it also breaks `musl`). For a clean shippable host, **vendor a trimmed
copy** of the crate (keep `lib.rs`, `device.rs`, `interface.rs`, `endpoint.rs`, `setup.rs`,
`usbip_protocol.rs`, `util.rs`, `consts.rs`; drop `host.rs`/`cdc.rs`/`hid.rs` + the `rusb`/`nusb`
deps) under e.g. `crates/punktfunk-host/vendor/usbip-sim/`, or accept the libusb dep if vendoring
is too much churn. Recommendation: vendor-trim (no libusb at runtime).
- **Runtime:** the usbip server is tokio-based. Run it on a dedicated runtime/thread (the host already
uses tokio behind the `quic` feature). Keep it off the per-frame video path (input only — fine).
- **Local attach without the `usbip` CLI (preferred):** don't shell out to `usbip attach`
(avoids an external `usbip-utils` runtime dep). Implement the client side in-process: connect to our
own server (or better, a `socketpair`/unix socket to avoid a TCP port), do the `OP_REQ_IMPORT`
handshake, then write `"<port> <sockfd> <devid> <speed>"` to
`/sys/devices/platform/vhci_hcd.0/attach`. (Acceptable fallback for v1: depend on the `usbip` CLI,
which is widely packaged, and `Command::new("usbip").args(["attach","-r","127.0.0.1","-b","0-0-0"])`.)
- **`ensure_modules`:** `modprobe vhci_hcd` (best-effort) the way the gadget does `dummy_hcd raw_gadget`.
### 3. Transport selection (in `inject/linux/steam_controller.rs` `ensure()` + `gadget_preferred`)
Extend the existing `DeckTransport` enum (currently `Uhid | Gadget`) to `Uhid | Gadget | Usbip` and the
selection ladder to: **`raw_gadget` if `/dev/raw-gadget` usable (SteamOS) → else `usbip` if `vhci_hcd`
loadable (Bazzite/generic) → else UHID/DualSense.** `gadget_preferred()` currently keys on
`ID=steamos`; generalize to "a recognized-by-Steam transport is available" (raw_gadget OR usbip).
Keep the M6 conflict gate (`degrade_steam_on_conflict` in `punktfunk1.rs`) ahead of all this — a host
with a *physical* Deck still degrades `SteamDeck`→DualSense, so two-Decks never happens in production.
### 4. Client leave-shortcut (`clients/linux/src/`)
Steam/QAM now pass through, so add an explicit disconnect:
- **Keyboard:** in `ui_stream.rs:300-310` (next to the `Ctrl+Alt+Shift+Q` capture toggle) add
`Ctrl+Alt+Shift+D``stop.store(true, …)` (the `stop_h` is already in scope), `Propagation::Stop`.
- **Controller:** in `gamepad.rs` (model on `maybe_fire_escape` at `:354-362`, `ESCAPE_CHORD` at `:36`)
add a disconnect chord. Recommended: **hold Start+Select+L1+R1 ≥ ~1.5 s** (escalate the existing
escape chord — short press leaves fullscreen, long-hold disconnects) OR a dedicated combo. Fire a
`disconnect_tx` consumed in `ui_stream.rs` (parallel to the escape future) → set the session `stop`
flag (`session.rs:73,212-214`). Do **not** use Steam/QAM in the chord (they're the marquee
pass-through buttons). Mirror the same to the other clients (windows/apple/android) later.
### 5. Polish
- **Serial format:** Steam flagged `PFDECK0000` as an "Invalid or missing unit serial number" and
substituted `28de-1205-<hash>` (benign, still promoted). Use a serial Steam accepts (a real Deck's
is alphanumeric like `FVZZ4200469B`); derive a per-instance valid-looking serial. The `0xAE`
attr-1 reply + the `0x83` unit-id attrs (`0x0a`/`0x04`) should be consistent.
- Verify the **Decky/client "Disable Steam Input"** path actually frees the Deck controller for SDL on
the client (so Steam/QAM reach SDL). This is the one runtime precondition for capture.
### 6. Validation (glass-to-glass)
- **Bazzite host** (`bazzite@192.168.1.41`): run the host with the usbip transport, connect the Linux
client (a Deck or a machine with a Valve controller, Steam Input disabled), and confirm in **game
mode** that the Steam button opens the Steam menu and the **QAM "…" button opens Quick Access**.
- **SteamOS host** (`deck@192.168.1.253`): confirm `raw_gadget` still selected + works (regression).
- Confirm the leave-shortcut works from both controller and keyboard while Steam/QAM pass through.
## Key findings / gotchas (so they aren't rediscovered)
- **usbip PoC portability:** the glibc build needs `GLIBC_2.34` (Bazzite has 2.42) + libusb (present
or vendored) → a dev-box glibc binary runs on Bazzite. `musl` fails (libusb1-sys). The server runs
as an unprivileged **user** (TCP 3240); only `modprobe vhci_hcd` + the attach need **root**. A
systemd *system* service can't exec from `/home` (perms) — run the server as the user.
- **raw_gadget gotchas** (already solved, see `steam-controller-deck-support.md` §11): 7-vs-9-byte
endpoint descriptor; no-data OUT controls acked via zero-length `EP0_READ`; no-arg ioctls must pass
an explicit `0` (musl); `libc::ioctl` request is `c_ulong`/`c_int` per libc → `as _`.
- **Feature contract** is what stops the gamepad-evdev churn (Steam re-probing): serve the captured
`0x83 GET_ATTRIBUTES` blob + `0xAE` serial (`packaging/linux/steam-deck-gadget/get_deck_attrs.c`
captures them from a physical Deck via hidraw `HIDIOCGFEATURE`; usbmon truncates to 32B). This is
already in `steam_gadget.rs::feature_reply` and the usbip PoC.
- **Captured descriptors** (verbatim from a physical Deck) live in `steam_gadget.rs` + the usbip PoC:
mouse (65B), keyboard (39B), controller (38B, Usage Page `0xFFFF`), endpoints `0x81/0x82/0x83`,
controller `bCountryCode 33`.
## Hardware + recipes
- **Deck (SteamOS)** `ssh deck@192.168.1.253` — has `dummy_hcd`+`raw_gadget`+`vhci_hcd`+`usbip`; a
*physical* Deck controller (so it degrades to DualSense by the M6 gate — for raw_gadget testing
there, de-authorize the physical Deck via `/sys/bus/usb/devices/3-3/authorized`). No `gcc`.
- **Bazzite** `ssh bazzite@192.168.1.41``vhci_hcd`+`usbip` (signed, in-tree), **no** dummy_hcd;
Secure Boot **on**; `gcc`+`kernel-devel` present; Steam runs. This is the usbip test bed.
- Both need passwordless sudo for driving (`/etc/sudoers.d/zz-punktfunk-poc` — remove when done). SSH
via `-o BatchMode=yes`. No `gcc` on the Deck → build static/glibc on the dev box + `scp`.
- usbip quick test (Bazzite): `sudo modprobe vhci_hcd; ./usbip-deck-poc pressa & ; sudo usbip attach
-r 127.0.0.1 -b 0-0-0` then watch `dmesg` + `~/.local/share/Steam/logs/controller.txt` for
`Interface: 2 … reserving XInput slot`.
## Open decisions for the new session
1. **Vendor-trim the `usbip` crate (no libusb) vs. accept the `rusb`/libusb dep.** Recommend trim.
2. **In-process vhci attach (write the sysfs) vs. shell out to the `usbip` CLI.** Recommend in-process
for v1-ship (no external CLI dep); CLI is the quick path to a working build first.
3. **Controller leave-chord**: escalate the escape chord (long-hold) vs. a dedicated combo.
4. Whether to **unify on usbip everywhere** (it works on SteamOS too) and retire `raw_gadget`, vs.
keep `raw_gadget` for SteamOS (already validated). Recommend keep both behind the trait — usbip is
the universal fallback, raw_gadget the validated SteamOS fast-path.
## Commit trail (this work, all on `main`, NOT pushed)
`faea4f1`…`a33c7d3` (M0M6) · `b6b6f27` (raw_gadget Deck) · `9e5112b` (feature contract) ·
`b3bc313` (host backend) · `8c3188d` (glass-confirmed + default-on SteamOS). The usbip PoC +
this plan are the next commits.