docs: rework client/crate READMEs, add missing ones
windows-drivers / probe-and-proto (push) Successful in 24s
windows-drivers / driver-build (push) Successful in 1m18s
apple / swift (push) Successful in 1m5s
android / android (push) Successful in 4m21s
ci / rust (push) Successful in 5m3s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 2m48s
windows-host / package (push) Successful in 7m10s
decky / build-publish (push) Successful in 24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m38s
release / apple (push) Successful in 9m1s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m32s
flatpak / build-publish (push) Successful in 4m59s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m7s
docker / deploy-docs (push) Successful in 25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m49s
windows-drivers / probe-and-proto (push) Successful in 24s
windows-drivers / driver-build (push) Successful in 1m18s
apple / swift (push) Successful in 1m5s
android / android (push) Successful in 4m21s
ci / rust (push) Successful in 5m3s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 2m48s
windows-host / package (push) Successful in 7m10s
decky / build-publish (push) Successful in 24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m38s
release / apple (push) Successful in 9m1s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m32s
flatpak / build-publish (push) Successful in 4m59s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m7s
docker / deploy-docs (push) Successful in 25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m49s
Rework the client READMEs to be accurate and inviting to first-time
visitors, and fill in the gaps where crates and tools had none.
- Rewrite clients/{apple,android,decky} READMEs (features-first, trim
dense internal narrative; drop the stale "one session at a time" /
"renegotiation not implemented" section from the Apple README).
- Add READMEs for clients/{linux,windows,probe}, which had none.
- Add crate READMEs for punktfunk-host, punktfunk-core, pf-driver-proto.
- Add brief READMEs for tools/{loss-harness,latency-probe}.
- Fix packaging/README duplicate "Option B" heading (bootc -> Option C).
- Fix docs-site/README stale docs/ -> design/ reference.
- De-stale packaging/windows/drivers/pf-dualsense README (drop "M0 spike"
/ external-checkout framing; reflect in-tree workspace + shipped +
installer-bundled + multi-pad), keeping the driver-authoring lore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+54
-59
@@ -1,83 +1,78 @@
|
|||||||
# punktfunk Android client
|
# punktfunk — Android client (phone & TV)
|
||||||
|
|
||||||
Native Android client for **punktfunk/1**, targeting **phone + TV** (Compose, D-pad + touch).
|
The native **Android** app for streaming a punktfunk host to your phone, tablet, or Android TV. A
|
||||||
|
Compose app that finds hosts on your network, pairs with a PIN, and streams at the display's own
|
||||||
|
resolution — with hardware HEVC decode, HDR10, and controller support, built for both touch and the
|
||||||
|
couch (D-pad / gamepad focus navigation).
|
||||||
|
|
||||||
## Architecture — Rust-heavy (like the Linux client, not thin-native like Apple)
|
## Features
|
||||||
|
|
||||||
Kotlin cannot `import` the cbindgen C header the way Swift can, so a native bridge is unavoidable.
|
- **Hardware decode** — NDK `AMediaCodec` HEVC → `SurfaceView`, including **HDR10** (Main10 /
|
||||||
We write it in **Rust** and link `punktfunk-core` directly — so the Android client reuses the Linux
|
BT.2020 PQ), with low-latency tuning and a live stats HUD.
|
||||||
|
- **Audio both ways** — Opus + Oboe playback with a jitter ring, plus mic uplink to the host.
|
||||||
|
- **Controller support** — buttons + axes with rumble and HID feedback (lightbar / adaptive
|
||||||
|
triggers); D-pad / gamepad focus navigation for TV and phone.
|
||||||
|
- **Find hosts automatically** — native mDNS discovery; first connect does a one-time **SPAKE2 PIN
|
||||||
|
pairing** (or TOFU on trusted LANs), then reconnects on a Keystore-wrapped, pinned identity.
|
||||||
|
- **Compose UI** — Connect / Settings / Stream screens with Material You theming.
|
||||||
|
|
||||||
|
Built for `arm64-v8a` + `x86_64`.
|
||||||
|
|
||||||
|
## Get it
|
||||||
|
|
||||||
|
Published to **Google Play (Internal Testing)** — join the beta via the
|
||||||
|
[Discord](https://discord.gg/kaPNvzMuGU). Per-device setup and pairing:
|
||||||
|
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
|
||||||
|
|
||||||
|
## How it's built — Rust-heavy
|
||||||
|
|
||||||
|
Kotlin can't `import` the cbindgen C header the way Swift can, so a native bridge is unavoidable. We
|
||||||
|
write it in **Rust** and link `punktfunk-core` directly — so the Android client reuses the Linux
|
||||||
client's orchestration (audio jitter ring, VK keymap inverse, latency/skew math, capture state
|
client's orchestration (audio jitter ring, VK keymap inverse, latency/skew math, capture state
|
||||||
machine, trust logic) instead of re-porting it into Kotlin.
|
machine, trust logic) instead of re-porting it into Kotlin.
|
||||||
|
|
||||||
| Side | Owns |
|
| Side | Owns |
|
||||||
|------|------|
|
|------|------|
|
||||||
| **Rust** (`clients/android/native` → `libpunktfunk_android.so`) | the JNI seam, `NativeClient` (QUIC control + UDP data plane), AnnexB→`AMediaCodec` decode, Opus+Oboe audio, VK keymap, latency math, trust/pairing, **mDNS discovery** (`mdns-sd`, the same browse the Linux/Windows clients use) |
|
| **Rust** (`native/` → `libpunktfunk_android.so`) | the JNI seam, `NativeClient` (QUIC control + UDP data plane), AnnexB → `AMediaCodec` decode (incl. HDR10), Opus + Oboe audio + mic, controller feedback, latency math, trust/pairing, `mdns-sd` discovery |
|
||||||
| **Kotlin** (`clients/android`) | Compose UI (host grid / settings / stream), `SurfaceView` lifecycle, input capture, the Wi-Fi `MulticastLock` + permission UX, Keystore identity, permissions |
|
| **Kotlin** (`app/`, `kit/`) | Compose UI, `SurfaceView` lifecycle, input capture, the Wi-Fi `MulticastLock` + permission UX, Keystore identity |
|
||||||
|
|
||||||
The single seam is `io.unom.punktfunk.kit.NativeBridge` ⇄ `Java_io_unom_punktfunk_kit_NativeBridge_*`.
|
The single seam is `io.unom.punktfunk.kit.NativeBridge` ⇄ `Java_io_unom_punktfunk_kit_NativeBridge_*`.
|
||||||
|
|
||||||
## Layout
|
|
||||||
|
|
||||||
```
|
```
|
||||||
clients/android/native/ Rust cdylib (workspace member) — links punktfunk-core directly
|
native/ Rust cdylib (workspace member) — links punktfunk-core directly
|
||||||
src/lib.rs JNI seam (connect/pair, input, plane getters, abi/core version)
|
src/lib.rs JNI seam (connect/pair, input, plane getters, versions)
|
||||||
src/session.rs session lifecycle + plane pumps
|
src/session.rs session lifecycle + plane pumps
|
||||||
src/decode.rs AnnexB → AMediaCodec HEVC hardware decode → SurfaceView (incl. HDR10)
|
src/decode.rs AnnexB → AMediaCodec HEVC hardware decode → SurfaceView (incl. HDR10)
|
||||||
src/audio.rs · src/mic.rs Opus + Oboe playback / mic uplink (jitter ring)
|
src/audio.rs · src/mic.rs Opus + Oboe playback / mic uplink
|
||||||
src/feedback.rs rumble + HID output (lightbar / adaptive triggers)
|
src/feedback.rs · src/stats.rs rumble + HID feedback; live video stats
|
||||||
src/stats.rs live video stats
|
app/ :app — Compose UI: Connect / Settings / Stream (phone + TV)
|
||||||
|
kit/ :kit — NativeBridge · native mDNS discovery · Gamepad · Keymap · Keystore identity
|
||||||
clients/android/ Gradle project (this dir)
|
|
||||||
settings.gradle.kts · build.gradle.kts · gradle.properties · gradlew
|
|
||||||
app/ :app — Compose UI: Connect / Settings / Stream screens (phone + TV)
|
|
||||||
kit/ :kit — NativeBridge · discovery (native mdns-sd, polled) · Gamepad · Keymap ·
|
|
||||||
security (Keystore identity + known-host store) · cargo-ndk build
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- Android SDK + **NDK r30** (`30.0.14904198`), `platforms;android-37.0`, `build-tools;37.0.0`,
|
|
||||||
**`cmake;3.22.1`** (`sdkmanager "cmake;3.22.1"` — the `cmake` crate builds libopus with it)
|
|
||||||
- **JDK 21** for Gradle/AGP (AGP 9.2 runs on JDK 17–21, *not* a newer default JDK like 25)
|
|
||||||
- Rust + `rustup target add aarch64-linux-android x86_64-linux-android` + `cargo install cargo-ndk`
|
|
||||||
|
|
||||||
Toolchain pinned: AGP 9.2.0 · Gradle 9.4.1 · Kotlin 2.3.21 · Compose BOM 2026.05.01 ·
|
|
||||||
compileSdk 37 · targetSdk 36 · minSdk 31 · ABIs arm64-v8a + x86_64.
|
|
||||||
|
|
||||||
## Build & run
|
## Build & run
|
||||||
|
|
||||||
**Android Studio:** open `clients/android` — it uses its bundled JBR 21 automatically. The
|
**Prerequisites:** Android SDK + **NDK r30** (`30.0.14904198`), `platforms;android-37.0`,
|
||||||
`cargoNdk*` task builds the `.so` as part of the normal build.
|
`build-tools;37.0.0`, **`cmake;3.22.1`** (builds libopus); **JDK 21** (AGP 9.2 runs on JDK 17–21, not
|
||||||
|
a newer default); Rust with `rustup target add aarch64-linux-android x86_64-linux-android` and
|
||||||
|
`cargo install cargo-ndk`. Toolchain is pinned (AGP 9.2 · Gradle 9.4.1 · Kotlin 2.3.21 · Compose BOM
|
||||||
|
2026.05.01 · compileSdk 37 · minSdk 31).
|
||||||
|
|
||||||
**CLI** (point Gradle at a JDK 21 if your machine default is newer, e.g. JDK 25):
|
**Android Studio:** open `clients/android` — it uses its bundled JBR 21, and the `cargoNdk*` task
|
||||||
|
builds the `.so` as part of the normal build.
|
||||||
|
|
||||||
|
**CLI** (point Gradle at JDK 21 if your machine default is newer):
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Adoptium/Temurin 21 (installed by the Android Studio setup, or `brew install temurin@21`):
|
export JAVA_HOME="$(/usr/libexec/java_home -v 21)" # or your Temurin 21 path
|
||||||
export JAVA_HOME="$(/usr/libexec/java_home -v 21)"
|
|
||||||
cd clients/android
|
cd clients/android
|
||||||
./gradlew :app:assembleDebug # cargo-ndk cross-compiles libpunktfunk_android.so first
|
./gradlew :app:assembleDebug # cargo-ndk cross-compiles libpunktfunk_android.so first
|
||||||
./gradlew :app:installDebug # onto a running emulator/device
|
./gradlew :app:installDebug # onto a running emulator/device
|
||||||
|
# emulators from env setup: emulator -avd pf_phone | emulator -avd pf_tv
|
||||||
# Emulators (created during env setup): emulator -avd pf_phone | emulator -avd pf_tv
|
|
||||||
```
|
```
|
||||||
|
|
||||||
The debug APK lands in `app/build/outputs/apk/debug/`. Launch it, pick a host from the list, pair,
|
The debug APK lands in `app/build/outputs/apk/debug/`. Launch it, pick a host, pair, and stream.
|
||||||
and stream.
|
|
||||||
|
|
||||||
## Status
|
## Related
|
||||||
|
|
||||||
A working native client (phone + Android TV), at parity with the Linux and Apple apps for the core
|
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
|
||||||
streaming experience:
|
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
|
||||||
|
|
||||||
- **Video** — `AMediaCodec` hardware HEVC decode → `SurfaceView`, including **HDR10** (Main10 /
|
|
||||||
BT.2020 PQ), with low-latency decode tuning and a live stats HUD.
|
|
||||||
- **Audio** — Opus + Oboe playback with a jitter ring, plus mic uplink to the host.
|
|
||||||
- **Input** — game controllers (buttons + axes) with rumble and HID feedback; D-pad /
|
|
||||||
game-controller focus navigation for the couch (TV + phone).
|
|
||||||
- **Discovery & trust** — native `mdns-sd` mDNS host list (polled over JNI; the same browse the
|
|
||||||
Linux/Windows clients use, not `NsdManager`), SPAKE2 PIN pairing and TOFU, with a
|
|
||||||
Keystore-wrapped client identity and a known-host store.
|
|
||||||
- **UI** — Compose host list / settings / stream screens, Material You theming.
|
|
||||||
- **Shipping** — built for `arm64-v8a` + `x86_64`; published to Google Play (Internal Testing).
|
|
||||||
|
|
||||||
`crates/punktfunk-core` uses the `ring` `rcgen` backend so the client `.so` is aws-lc-free.
|
|
||||||
|
|||||||
+92
-341
@@ -1,364 +1,115 @@
|
|||||||
# punktfunk Apple client (SwiftUI)
|
# punktfunk — Apple client (macOS · iOS · iPadOS · tvOS)
|
||||||
|
|
||||||
The native macOS/iOS client for **`punktfunk/1`** (the post-GameStream protocol). All
|
The native **Apple** app for streaming a punktfunk host to your Mac, iPhone, iPad, or Apple TV. A
|
||||||
networking/protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
|
SwiftUI app that finds hosts on your network, pairs with a PIN, and streams at your display's own
|
||||||
input datagrams, Opus audio, cert pinning — lives in the shared Rust core (statically
|
resolution and refresh rate — with VideoToolbox hardware decode and full controller support.
|
||||||
linked as `PunktfunkCore.xcframework`); this package is the Swift shell: decode
|
|
||||||
(VideoToolbox), present (SwiftUI), input capture.
|
|
||||||
|
|
||||||
## Status — working client (macOS, with iOS / tvOS in the shared build)
|
All the networking and protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
|
||||||
|
Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (statically linked as
|
||||||
|
`PunktfunkCore.xcframework`). This package is the Swift shell: decode, present, input, and UI.
|
||||||
|
|
||||||
A full streaming client: VideoToolbox HEVC decode, controllers incl. DualSense feedback, host
|
## Features
|
||||||
discovery, PIN pairing, and a network speed test. The lower-latency **stage-2 presenter**
|
|
||||||
(`VTDecompressionSession` → `CAMetalLayer`) is built and opt-in (Settings → Presenter); see below.
|
|
||||||
|
|
||||||
First light was achieved 2026-06-10 — validated live, Mac ↔ a Linux host over the LAN: gamescope
|
- **Hardware decode** — VideoToolbox HEVC, with a low-latency **stage-2 presenter**
|
||||||
virtual output → NVENC HEVC →
|
(`VTDecompressionSession` → `CAMetalLayer`, presented off a `CADisplayLink`, ~11 ms p50) as the
|
||||||
`punktfunk/1` (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox →
|
default and an `AVSampleBufferDisplayLayer` fallback.
|
||||||
`AVSampleBufferDisplayLayer` on glass at 1280×720@60, with mouse/keyboard flowing back as
|
- **HDR & 4:4:4** — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR
|
||||||
QUIC datagrams into the host's gamescope EIS injector (thousands of events injected during
|
reconfiguration, and hardware-probed 4:4:4 support.
|
||||||
the session). Headless variant of the same proof: `RemoteFirstLightTests` decoded 60/60
|
- **Your display's native mode** — the host builds a virtual output at exactly your WxH@Hz;
|
||||||
received AUs spanning 983 ms of host capture clock.
|
mid-stream resize renegotiates without reconnecting.
|
||||||
|
- **Audio both ways** — Opus playback (CoreAudio, no bundled libopus) with a jitter ring, plus mic
|
||||||
|
uplink; speaker/mic selectable in Settings.
|
||||||
|
- **Full controller support** — one selected controller forwarded as pad 0, including **DualSense**
|
||||||
|
feedback (rumble → CoreHaptics, lightbar, player LEDs, adaptive triggers) and touchpad/motion. The
|
||||||
|
virtual pad type auto-resolves from your physical controller.
|
||||||
|
- **Mouse & keyboard** — `GCMouse`/`GCKeyboard` capture with click-to-capture and a ⌘⎋ release, plus
|
||||||
|
iPad pointer lock and touch input.
|
||||||
|
- **Find hosts automatically** — mDNS discovery (`NWBrowser` over `_punktfunk._udp`); first connect
|
||||||
|
does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then reconnects on a pinned,
|
||||||
|
Keychain-stored identity.
|
||||||
|
- **Tune the stream** — a fps / Mb·s / **latency** HUD (skew-corrected across machines), a bitrate
|
||||||
|
control, a per-host **network speed test** with a recommended bitrate, and a host-compositor picker.
|
||||||
|
|
||||||
The connector underneath (`punktfunk_core::client::NativeClient` over the C ABI) carries the
|
Runs from one shared codebase across **macOS, iOS, iPadOS, and tvOS**.
|
||||||
full session: video AUs, **Opus audio** (`nextAudio()`), **rumble** (`nextRumble()`),
|
|
||||||
**DualSense feedback** (`nextHidOutput()` — lightbar, player LEDs, adaptive-trigger
|
|
||||||
effects), input incl. gamepads + DualSense touchpad/motion (`sendTouchpad`/`sendMotion`),
|
|
||||||
and **cert pinning + TOFU** (`pinSHA256:`/`hostFingerprint`) — see
|
|
||||||
`punktfunk1.rs::tests::c_abi_connection_roundtrip` (three sequential sessions: TOFU, pinned
|
|
||||||
reconnect, wrong-pin rejection). The host (`punktfunk-host punktfunk1-host`) is a persistent listener:
|
|
||||||
reconnect at will during development.
|
|
||||||
|
|
||||||
What's here, all compiled and tested on macOS (Xcode 26.5 / Swift 6.3):
|
## Get it
|
||||||
|
|
||||||
- **`PunktfunkKit`** (library)
|
Install from the App Store / TestFlight, or build from source below. Per-device install steps and the
|
||||||
- `PunktfunkConnection.swift` — wrapper over the C ABI. AUs/audio are copied into `Data`
|
pairing walkthrough:
|
||||||
(the C pointer is only valid until the next call of the same kind). `close()` is safe
|
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
|
||||||
from any thread: per-plane locks enforce the C contract ("never close with a
|
|
||||||
`next_au`/`next_audio` in flight") instead of leaving it to callers. Pinning + TOFU
|
|
||||||
via `pinSHA256:`/`hostFingerprint`.
|
|
||||||
- `AnnexB.swift` — in-band VPS/SPS/PPS → `CMVideoFormatDescription`; Annex-B → AVCC
|
|
||||||
`CMSampleBuffer` with `DisplayImmediately` set.
|
|
||||||
- `StreamView.swift` — SwiftUI `NSViewRepresentable` over `AVSampleBufferDisplayLayer`
|
|
||||||
(stage-1 presenter: the layer hardware-decodes compressed HEVC itself). One pump
|
|
||||||
thread per view, token-cancelled so reconnects can't double-pump.
|
|
||||||
- `InputCapture.swift` — `GCMouse` raw deltas + `GCKeyboard` HID→VK mapping (the host's
|
|
||||||
`vk_to_evdev` consumes Windows VKs), with fractional-delta accumulation so sub-pixel
|
|
||||||
motion isn't truncated away. Buttons use GameStream ids (1=left … 5=X2). Scroll is
|
|
||||||
WHEEL_DELTA(120)-scaled: macOS via the stream view's `scrollWheel` override, iPad via
|
|
||||||
GCMouse's scroll dpad when pointer-locked and a scroll-only `UIPanGestureRecognizer`
|
|
||||||
otherwise (trackpad gestures never reach GC's scroll dpad).
|
|
||||||
- `GamepadManager.swift` — app-lifetime controller discovery + selection (`.shared`):
|
|
||||||
watches `GCController` connect/disconnect, fingerprints each pad for the Settings UI
|
|
||||||
(name, capabilities, battery), and selects the ONE controller forwarded to the host
|
|
||||||
(user pin via "Use controller", else most recently connected extended gamepad).
|
|
||||||
- `GamepadCapture.swift` — the active controller → wire: snapshot-diff over
|
|
||||||
`GCExtendedGamepad` into incremental `gamepadButton`/`gamepadAxis` events (pad 0),
|
|
||||||
plus DualSense touchpad contacts and ~250 Hz motion samples on the rich-input plane
|
|
||||||
(the GC→DualSense unit conversions live in `GamepadWire`, one place). Held state is
|
|
||||||
released on the wire on controller switch / app deactivation / stop.
|
|
||||||
- `GamepadFeedback.swift` + `DualSenseTriggerEffect.swift` — host feedback → the real
|
|
||||||
controller: one drain thread for `nextRumble()` (→ `CHHapticEngine` per handle
|
|
||||||
locality) and `nextHidOutput()` (lightbar → `GCDeviceLight`, player LEDs →
|
|
||||||
`playerIndex`, adaptive-trigger effect blocks → a total, table-driven parser →
|
|
||||||
`GCDualSenseAdaptiveTrigger`, exact for the 10-zone positional modes).
|
|
||||||
- `HostDiscovery.swift` — LAN auto-discovery: an `NWBrowser` over `_punktfunk._udp`
|
|
||||||
(the host's `crate::discovery` mDNS advert), resolving each service to an IP:port via a
|
|
||||||
throwaway `NWConnection` and parsing the TXT (`fp` advisory cert fingerprint, `pair`,
|
|
||||||
stable `id`). iOS/tvOS need `NSBonjourServices` (`Config/Info.plist`) or the system
|
|
||||||
blocks the browse.
|
|
||||||
- **`PunktfunkClient`** (the app): hosts grid (saved in UserDefaults) with an **On this
|
|
||||||
network** section listing mDNS-discovered hosts (tap to save + connect, or pair if the
|
|
||||||
host requires it), "+" toolbar sheet to add hosts manually, stream mode in Settings (⌘,),
|
|
||||||
two trust flows — the
|
|
||||||
trust-on-first-use fingerprint prompt over the live-but-blurred stream, and SPAKE2 PIN
|
|
||||||
pairing (`PairSheet`, from a host card's context menu or the trust prompt;
|
|
||||||
`ClientIdentityStore` keeps the client identity in the Keychain and presents it on
|
|
||||||
every connect) — then pinned reconnects, fps/Mb-s HUD + a **capture→client-receipt latency**
|
|
||||||
line (`LatencyMeter`, p50/p95): the AU `pts_ns` (host capture clock) to the instant the client
|
|
||||||
received it, **skew-corrected** across machines via `PunktfunkConnection.clockOffsetNs` (the
|
|
||||||
connect-time wall-clock handshake, `punktfunk_connection_clock_offset_ns`). It excludes the
|
|
||||||
layer's decode+present (stage-1 `AVSampleBufferDisplayLayer` has no per-frame present callback);
|
|
||||||
the opt-in **stage-2 presenter** (Settings → Presenter) adds a **capture→present**
|
|
||||||
(glass-to-glass) line via explicit decode + a Metal/display-link present. Settings also picks the HOST
|
|
||||||
compositor (KWin/wlroots/Mutter/gamescope, default automatic — the host honors it
|
|
||||||
only if that backend is available there) and has a **Controllers** section: every
|
|
||||||
detected controller (capability glyphs, battery, "In use" badge), which one to forward
|
|
||||||
("Use controller", default automatic), and the virtual pad type the host creates
|
|
||||||
("Controller type": Automatic / Xbox 360 / DualSense — Automatic matches the physical
|
|
||||||
pad; resolved at connect time, the host pad is fixed per session). Gamepad capture +
|
|
||||||
feedback run with streaming (`SessionModel` owns them, same trust gate as audio).
|
|
||||||
Settings also sets the **Bitrate** (Automatic toggle = host default; manual is a
|
|
||||||
log-scale slider, 2 Mbps – 3 Gbps, snapped to two significant figures — above 1 Gbps
|
|
||||||
an inline warning says to run a speed test first; tvOS uses a preset picker instead,
|
|
||||||
Slider doesn't exist there; negotiated via the Hello on every connect), and a host
|
|
||||||
card's context menu offers **"Test Network Speed…"** (`SpeedTestSheet`): connects, has
|
|
||||||
the host burst probe filler over the real data plane (up to the host's 3 Gbps probe
|
|
||||||
ceiling for 2 s, roadmap §9),
|
|
||||||
shows measured goodput · loss · a recommended bitrate (≈70% of measured), and applies
|
|
||||||
it in one tap. The streaming **statistics overlay** can be turned off and moved to any
|
|
||||||
corner (Settings → Display → Statistics, `DefaultsKey.hudEnabled`/`hudPlacement`), and
|
|
||||||
toggled live with **⌘⇧S** — a Scene-level **"Stream" menu** (`StreamCommands`) that also
|
|
||||||
carries **Disconnect ⌘D**, so disconnect survives the HUD being hidden (on iOS a small
|
|
||||||
exit chip appears instead; on tvOS the Siri-Remote Menu button still disconnects). The
|
|
||||||
macOS Settings window is a **tabbed preferences pane** (General / Display / Audio /
|
|
||||||
Controllers / Advanced) — the sections are shared with the iOS single-Form layout and the
|
|
||||||
tvOS pushed-picker layout, defined once each.
|
|
||||||
- **Tests** (`swift test`): byte-level Annex-B units; a real-codec round trip
|
|
||||||
(VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → `AnnexB` →
|
|
||||||
VTDecompressionSession → pixels); table-driven DualSense trigger-effect parsing
|
|
||||||
(`DualSenseTriggerEffectTests`) and the gamepad wire conversions
|
|
||||||
(`GamepadWireTests`); loopback integration against real local hosts
|
|
||||||
(`test-loopback.sh` — stream round trip incl. gamepad/touchpad/motion sends, a
|
|
||||||
host-scripted feedback burst asserted on the rumble + HID-output planes
|
|
||||||
(`PUNKTFUNK_TEST_FEEDBACK=1`), the bitrate-negotiation echo and a real 20 Mbps
|
|
||||||
bandwidth probe, plus the PIN pairing ceremony and the `--require-pairing` gate
|
|
||||||
against a second, armed host); the remote first-light test above.
|
|
||||||
|
|
||||||
## Build / run / test (on a Mac)
|
## Build / run / test (on a Mac)
|
||||||
|
|
||||||
|
Requires Xcode 26.5 / Swift 6.3. First build the Rust core into an xcframework, then build the app:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
rustup target add aarch64-apple-darwin x86_64-apple-darwin
|
rustup target add aarch64-apple-darwin x86_64-apple-darwin
|
||||||
bash scripts/build-xcframework.sh # → clients/apple/PunktfunkCore.xcframework
|
bash scripts/build-xcframework.sh # → clients/apple/PunktfunkCore.xcframework
|
||||||
# + BUILD_IOS=1 for the iOS slices (rustup target add aarch64-apple-ios{,-sim} x86_64-apple-ios)
|
# BUILD_IOS=1 also builds the iOS slices (add the ios rustup targets)
|
||||||
# + BUILD_TVOS=1 for tvOS — TIER-3 Rust targets, built from source:
|
# BUILD_TVOS=1 also builds tvOS (tier-3 targets, built from source — see below)
|
||||||
# rustup toolchain install nightly && rustup component add rust-src --toolchain nightly
|
|
||||||
cd clients/apple
|
cd clients/apple
|
||||||
swift build && swift test # loopback/remote tests self-skip without a host
|
open Punktfunk.xcodeproj # the real app: ⌘R builds + runs Punktfunk.app
|
||||||
swift run PunktfunkClient # the unbundled dev shell (CLI)
|
swift run PunktfunkClient # or the unbundled dev shell (CLI)
|
||||||
open Punktfunk.xcodeproj # the real app: ⌘R builds + runs Punktfunk.app
|
swift build && swift test # unit + loopback/remote tests (self-skip w/o a host)
|
||||||
|
```
|
||||||
|
|
||||||
bash test-loopback.sh # full loopback proof: builds punktfunk-host
|
tvOS slices are tier-3 Rust targets, built from source:
|
||||||
# (synthetic source — runs on macOS), streams
|
`rustup toolchain install nightly && rustup component add rust-src --toolchain nightly`.
|
||||||
# byte-verified frames into the Swift client
|
|
||||||
|
|
||||||
# against the real host (Linux box, see CLAUDE.md "Running on this box") — punktfunk1-host is a
|
### Test against a host
|
||||||
# persistent listener, reconnect at will:
|
|
||||||
# PUNKTFUNK_COMPOSITOR=gamescope PUNKTFUNK_GAMESCOPE_APP=vkcube PUNKTFUNK_ZEROCOPY=1 \
|
```sh
|
||||||
# cargo run -rp punktfunk-host -- punktfunk1-host --source virtual --seconds 60
|
# full loopback proof — builds punktfunk-host (synthetic source, runs on macOS) and streams
|
||||||
PUNKTFUNK_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless
|
# byte-verified frames into the Swift client, incl. the PIN pairing ceremony:
|
||||||
# (+ PUNKTFUNK_REMOTE_PORT / PUNKTFUNK_REMOTE_COMPOSITOR=gamescope|kwin|… /
|
bash test-loopback.sh
|
||||||
# PUNKTFUNK_REMOTE_PIN=<arming-pin> for the remote pairing test)
|
|
||||||
|
# against a real Linux host on the LAN (see the repo README "Running on this box"):
|
||||||
|
PUNKTFUNK_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless
|
||||||
PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkClient # on glass
|
PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkClient # on glass
|
||||||
```
|
```
|
||||||
|
|
||||||
## Xcode project (`Punktfunk.xcodeproj`)
|
## Project layout
|
||||||
|
|
||||||
The app target **Punktfunk** wraps the same sources as the `swift run` shell
|
- **`PunktfunkKit`** (library) — the reusable pieces:
|
||||||
(`Sources/PunktfunkClient`, a synchronized folder — no duplication) plus `App/` (asset
|
- `PunktfunkConnection` — the wrapper over the C ABI (thread-safe `close()`, per-plane locks,
|
||||||
catalog) and links `PunktfunkKit` from the local package. Generated Info.plist, ad-hoc
|
pinning + TOFU).
|
||||||
signing, bundle id `io.unom.punktfunk`. Notes:
|
- `AnnexB` / `StreamView` / `VideoDecoder` / `MetalVideoPresenter` — format handling, the stage-1
|
||||||
|
(`AVSampleBufferDisplayLayer`) and stage-2 (`VTDecompressionSession` → `CAMetalLayer`) presenters.
|
||||||
|
- `InputCapture` — `GCMouse`/`GCKeyboard` → host VK/mouse, with fractional-delta accumulation.
|
||||||
|
- `GamepadManager` / `GamepadCapture` / `GamepadFeedback` / `DualSenseTriggerEffect` — controller
|
||||||
|
discovery + selection, capture (buttons/axes/touchpad/motion), and host-feedback rendering.
|
||||||
|
- `HostDiscovery` — `NWBrowser` over `_punktfunk._udp`.
|
||||||
|
- **`PunktfunkClient`** (the app) — hosts grid with an *On this network* section, add-host sheet,
|
||||||
|
the two trust flows (TOFU prompt + SPAKE2 `PairSheet`), the stream view with the HUD, a
|
||||||
|
tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed
|
||||||
|
test. A Scene-level **Stream** menu carries Disconnect (⌘D) and the HUD toggle (⌘⇧S).
|
||||||
|
- **Tests** (`swift test`) — Annex-B units, a real-codec VideoToolbox round trip, DualSense
|
||||||
|
trigger-effect and gamepad-wire conversions, loopback integration against real local hosts, and the
|
||||||
|
remote first-light test.
|
||||||
|
|
||||||
- **Entitlements (sandbox)**: the macOS target uses
|
## Notes for contributors
|
||||||
`Config/Punktfunk-macOS.entitlements`; iOS/tvOS use the shared
|
|
||||||
`Config/Punktfunk.entitlements`. The macOS app is **App-Sandboxed** (mandatory for the Mac
|
|
||||||
App Store/TestFlight, and used for the Developer ID DMG too so the local build matches what
|
|
||||||
ships): `com.apple.security.app-sandbox`, `network.client` + **`network.server`** (the
|
|
||||||
sandbox gates `bind()`; quinn + the raw-UDP plane both bind, so receive breaks without it),
|
|
||||||
`device.audio-input` (mic), `device.bluetooth` + `device.usb` (GameController over BT/USB),
|
|
||||||
and the existing `keychain-access-groups`. `app-sandbox` is macOS-only — keep it OUT of the
|
|
||||||
shared iOS/tvOS file (it fails upload validation there). Verify a build is sandboxed with
|
|
||||||
`codesign -d --entitlements :- <built .app>`. Heads-up: `device.usb` draws some App Review
|
|
||||||
scrutiny — justify it in the review notes ("reads input from USB game controllers").
|
|
||||||
- **App icon**: `App/Assets.xcassets` ships an empty `AppIcon` slot. For an Icon Composer
|
|
||||||
`.icon`: add the file to the project (target Punktfunk), set it as the App Icon in the
|
|
||||||
target's General tab, and delete the placeholder `AppIcon.appiconset`. Heads-up: CLI
|
|
||||||
`actool` (Xcode 26.5) crashed compiling `punktfunk_Logo.icon` — if Xcode does the same,
|
|
||||||
suspect the icon bundle (it has a duplicate-named layer, "…Layer-3 2.svg"), not the
|
|
||||||
project.
|
|
||||||
- **Tests from Xcode**: the package tests run with `swift test`; to get them on ⌘U, add
|
|
||||||
`PunktfunkKitTests` once via Edit Scheme → Test → + (Xcode persists it into the shared
|
|
||||||
scheme — a hand-written package-test reference doesn't resolve headlessly).
|
|
||||||
- `xcodebuild -project Punktfunk.xcodeproj -scheme Punktfunk build` works headlessly;
|
|
||||||
same for `-scheme Punktfunk-iOS -destination 'generic/platform=iOS Simulator'` (run it
|
|
||||||
in a simulator via `xcrun simctl install/launch` — `SIMCTL_CHILD_PUNKTFUNK_AUTOCONNECT=…`
|
|
||||||
passes the dev autoconnect env through).
|
|
||||||
|
|
||||||
## App Store screenshots
|
- **Xcode project** (`Punktfunk.xcodeproj`) wraps the same sources as the `swift run` shell (a
|
||||||
|
synchronized folder — no duplication). The macOS target is **App-Sandboxed** (needs
|
||||||
|
`network.server` — the raw-UDP plane and quinn both `bind()`); iOS/tvOS use the shared
|
||||||
|
entitlements file (keep `app-sandbox` **out** of it). Verify with
|
||||||
|
`codesign -d --entitlements :- <built .app>`.
|
||||||
|
- **Decode flow**: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band, and recovery
|
||||||
|
keyframes re-send them — refresh the format description on every IDR; there is no out-of-band
|
||||||
|
extradata, ever.
|
||||||
|
- **ABI threading**: one video pump thread per connection, one optional audio drain thread, and one
|
||||||
|
optional feedback drain thread (rumble + HID-output). `send()` is enqueue-only and safe alongside
|
||||||
|
all of them. The wrapper's per-plane locks make `close()` safe from anywhere.
|
||||||
|
- **DualSense motion scale** (`GamepadWire`) is derived from hid-playstation's math, not yet
|
||||||
|
live-verified — if gyro/accel feel wrong in a game, correct sign/scale there and `evtest` the
|
||||||
|
host's virtual pad.
|
||||||
|
- **App Store screenshots** are automated — `tools/screenshots.sh all` renders the real UI at the
|
||||||
|
required pixel sizes via a DEBUG-only shot mode; the `apple` CI workflow captures the iOS sizes on
|
||||||
|
every main push. See the script header for details.
|
||||||
|
- Deeper design notes live in `docs-site/content/docs/apple-stage2-presenter.md`.
|
||||||
|
|
||||||
Automated, faithful screenshots of the real UI for App Store Connect — one set per platform at
|
## Related
|
||||||
exactly the accepted pixel sizes. Driver: **`tools/screenshots.sh`**.
|
|
||||||
|
|
||||||
```sh
|
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
|
||||||
tools/screenshots.sh all # macOS + (if full Xcode) iOS, iPadOS, tvOS → ./screenshots
|
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
|
||||||
tools/screenshots.sh macos # just macOS
|
|
||||||
OUT=~/Desktop/shots tools/screenshots.sh ios ipad tvos
|
|
||||||
PUNKTFUNK_SHOT_HERO=~/frame.png tools/screenshots.sh ios # real captured frame behind the hero
|
|
||||||
```
|
|
||||||
|
|
||||||
How it works: the app has a DEBUG-only **shot mode** (`Sources/PunktfunkClient/Screenshots/`).
|
|
||||||
Launched with `PUNKTFUNK_SHOT_SCENE=<name>` it renders **one** mock-populated screen full-bleed
|
|
||||||
(`ScreenshotHostView`) instead of `ContentView`, then the OS screenshots the *real, fully-rendered*
|
|
||||||
window — `screencapture` on macOS, `xcrun simctl io booted screenshot` on the Simulators. The five
|
|
||||||
scenes (`ShotScenes.all`): `01-stream` (the stream hero — a synthetic frame + the glass HUD, since
|
|
||||||
`StreamView` needs a live connection), `02-hosts`, `03-pair`, `04-trust`, `05-settings`. Mock data
|
|
||||||
is in `ShotMock`; nothing touches a host.
|
|
||||||
|
|
||||||
Output pixels are App Store Connect's required/largest sizes (Apple auto-derives the smaller ones):
|
|
||||||
`mac` 2880×1800 · `iphone-6.9` 1320×2868 (hero 2868×1320) · `ipad-13` 2064×2752 (hero 2752×2064) ·
|
|
||||||
`appletv` 1920×1080.
|
|
||||||
|
|
||||||
Why not `ImageRenderer` (the obvious offscreen route)? It can't rasterize this app's chrome —
|
|
||||||
`NavigationStack`, `Form`/`TabView`, and Liquid-Glass/`NSVisualEffect` materials all render black or
|
|
||||||
SwiftUI's "can't render" placeholder. Capturing the live window/Simulator avoids that entirely.
|
|
||||||
|
|
||||||
Requirements / gotchas:
|
|
||||||
- **macOS**: only the Swift toolchain is needed, **plus a one-time Screen Recording grant** for
|
|
||||||
your terminal (System Settings → Privacy & Security → Screen Recording) — without it
|
|
||||||
`screencapture -l` fails with "could not create image from window". (A no-permission fallback,
|
|
||||||
`PUNKTFUNK_SHOT_SELFCAPTURE=<dir>`, uses `cacheDisplay` — but it omits material blur and can't
|
|
||||||
read `ScrollView` content, so it's for quick checks, not submission.)
|
|
||||||
- **iOS/iPadOS/tvOS**: needs **full Xcode** (xcodebuild + Simulators), not just Command Line Tools,
|
|
||||||
and the matching device Simulators installed (iPhone 16 Pro Max, iPad Pro 13", Apple TV). Run it
|
|
||||||
on a full-Xcode Mac (e.g. the `macos-arm64` CI mini).
|
|
||||||
- The hero defaults to a synthetic synthwave frame — set `PUNKTFUNK_SHOT_HERO` to a real captured
|
|
||||||
frame for a production-quality lead screenshot.
|
|
||||||
|
|
||||||
**CI**: the `apple` workflow's **`screenshots`** job runs on the `macos-arm64` runner on every main
|
|
||||||
push + manual dispatch (skipped on PRs), and attaches the result as a single zip artifact,
|
|
||||||
**`punktfunk-appstore-screenshots`** (download it from the run's Artifacts; `upload-artifact@v3` —
|
|
||||||
Gitea's backend rejects v4). It captures the two **required iOS sizes — iPhone 6.9" + iPad 13"** —
|
|
||||||
on the Simulator (auto-creating the device if the runner lacks it), and is isolated from the
|
|
||||||
build/test job so a capture hiccup never reds the build.
|
|
||||||
|
|
||||||
**macOS and tvOS are NOT in CI**, by design: the self-hosted runner is **headless** (no
|
|
||||||
window-server session), so the macOS window capture can't run there, and tvOS needs the Tier-3
|
|
||||||
build-std slice. Generate those on a GUI Mac: `tools/screenshots.sh macos tvos`. (If the runner is
|
|
||||||
ever switched to a logged-in GUI session, re-adding macOS to the job's capture step is one line.)
|
|
||||||
|
|
||||||
## Notes for whoever picks this up next
|
|
||||||
|
|
||||||
1. **cbindgen import quirk** (the predicted "small compile fixes", now fixed): the
|
|
||||||
C17-compatible header spells `PunktfunkStatus`/`PunktfunkInputKind` as integer typedefs while
|
|
||||||
the enum *constants* import into Swift as a distinct same-named type — bridge with
|
|
||||||
`.rawValue` (see the top of `PunktfunkConnection.swift`). Don't fight the generated header.
|
|
||||||
2. **ABI contract**: one video pump thread per connection, plus optionally one *separate*
|
|
||||||
audio drain thread for `nextAudio()` and one feedback drain thread for
|
|
||||||
`nextRumble()`/`nextHidOutput()` (the core keeps per-plane borrow slots, so the planes
|
|
||||||
never alias; rumble + HID-output are two planes drained sequentially by the one
|
|
||||||
feedback thread); `send()` is enqueue-only and safe alongside all of them. The
|
|
||||||
wrapper's per-plane locks make `close()` safe from anywhere (it waits out in-flight
|
|
||||||
polls, ≤ their timeouts).
|
|
||||||
3. **Decode flow**: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band
|
|
||||||
and recovery keyframes re-send them — "refresh the format description on every IDR"
|
|
||||||
(what `StreamView` does) is sufficient; there is no out-of-band extradata, ever.
|
|
||||||
4. **Stage 2 — built, opt-in (`punktfunk.presenter == "stage2"`, default stage 1).** Explicit
|
|
||||||
`VTDecompressionSession` decode (`VideoDecoder`) → a `CAMetalLayer` + display-link present
|
|
||||||
(`MetalVideoPresenter`/`Stage2Pipeline`), hosted as a sublayer by the same `StreamView`s with
|
|
||||||
input capture + HUD unchanged. It adds a **capture→present** (glass-to-glass, modulo the host
|
|
||||||
render→capture term) HUD line, skew-corrected via `PunktfunkConnection.clockOffsetNs`. The
|
|
||||||
decode half is unit-tested (`testVideoDecoderAsyncCallbackDeliversPixels`); the Metal present
|
|
||||||
is display-bound — **validate live** (flip the Settings "Presenter" picker, watch the HUD
|
|
||||||
number and that the image looks right) before making it the default. 10-bit/HDR + a smoothing
|
|
||||||
pacer are later. Plan: `docs-site/content/docs/apple-stage2-presenter.md`.
|
|
||||||
5. **Audio — wired, both directions.** Playback: `SessionAudio` drains `nextAudio()`
|
|
||||||
on its own thread, decodes through CoreAudio's built-in Opus codec (`OpusCodec.swift`
|
|
||||||
— kAudioFormatOpus, no bundled libopus; round-trip unit-tested) into a priming
|
|
||||||
jitter ring feeding an `AVAudioSourceNode`. Mic: a second engine taps the input
|
|
||||||
device, resamples to 48 kHz stereo, Opus-encodes 20 ms chunks and `sendMic()`s them
|
|
||||||
(the host's virtual PipeWire source accepts any frame size ≤ 120 ms). Speaker/mic
|
|
||||||
are chosen in Settings (`AudioDevices.swift` — persisted by UID; "System default"
|
|
||||||
leaves the engines unpinned so they follow macOS device changes), mic on/off toggle
|
|
||||||
included; the app asks for mic permission on first use
|
|
||||||
(NSMicrophoneUsageDescription is in the Xcode target). A/V sync and packet-loss
|
|
||||||
concealment beyond silence-fill are still open (AudioPacket.seq/ptsNs carry what's
|
|
||||||
needed). Decode with libopus or `AVAudioConverter`/`kAudioFormatOpus` into an
|
|
||||||
`AVAudioEngine` source node; conceal gaps (drop/dup) rather than blocking — the Rust
|
|
||||||
side buffers 320 ms and drops the newest packet when the puller lags. Wall-clock
|
|
||||||
`ptsNs` shares the host clock with video AUs for A/V sync. Wiring this into
|
|
||||||
`PunktfunkClient` is the next app-side task.
|
|
||||||
6. **Gamepads — wired end to end.** Exactly ONE controller (the `GamepadManager`
|
|
||||||
selection) forwards as pad 0; the host accumulates the incremental events into a
|
|
||||||
virtual pad whose TYPE the client negotiates in the Hello (`gamepad:` connect
|
|
||||||
parameter, echoed resolved in `resolvedGamepad` — Automatic resolves from the physical
|
|
||||||
pad at connect time; host precedence: explicit client choice > host `PUNKTFUNK_GAMEPAD`
|
|
||||||
env > Xbox 360). A DualSense session carries the full feel: adaptive-trigger blocks
|
|
||||||
(`DualSenseTriggerEffect.parse` — mode bytes per the community convention
|
|
||||||
(Nielk1/ds5w/inputtino), total, unknown → `.off`), lightbar, player LEDs, touchpad,
|
|
||||||
motion. **Motion scale constants** (`GamepadWire.gyroLSBPerRadS` = 20 LSB per deg/s,
|
|
||||||
`accelLSBPerG` = 10000) are derived from hid-playstation's math over the host's fixed
|
|
||||||
calibration blob, not yet live-verified — if gyro/accel feel wrong in a real game,
|
|
||||||
correct sign/scale in `GamepadCapture.forwardMotion`/`GamepadWire` and `evtest` the
|
|
||||||
host's virtual pad. Twin identical controllers share a fingerprint base, so a manual
|
|
||||||
pin can swap between them across reconnects (documented in the Settings footer).
|
|
||||||
7. **Trust — the full ceremony exists now (SPAKE2).** `generateIdentity()` once (persist
|
|
||||||
both PEMs in the Keychain), then `pair(host:identity:pin:name:)` with the 4-digit PIN
|
|
||||||
the host prints when it ARMS pairing (`--allow-pairing`/`--require-pairing`; one PIN
|
|
||||||
per arming window, surfaced in the host's web console — port 3000 → Pairing — and
|
|
||||||
printed at startup; the user reads it before pairing). Returns the
|
|
||||||
host's VERIFIED fingerprint; persist it and pass `pinSHA256:` + `identity:` to every
|
|
||||||
connect. Pairing is a real PAKE: a wrong PIN gets ONE online guess (no offline
|
|
||||||
dictionary attack), throwing `.wrongPIN`; a wrong-size pin throws `.invalidPin`. `PunktfunkClient` implements both flows:
|
|
||||||
the TOFU fingerprint sheet keeps working against hosts not running
|
|
||||||
`--require-pairing`, and the PIN ceremony is wired in — `ClientIdentityStore`
|
|
||||||
(Keychain) on every connect, `PairSheet` from a host card's context menu or the trust
|
|
||||||
prompt's "Pair with PIN instead…" (the host's accept loop is sequential, so that path
|
|
||||||
drops the live session before pairing). With `--require-pairing` the host now
|
|
||||||
authorizes clients too (the "other direction" is no longer open, opt-in per host);
|
|
||||||
the whole gate is regression-tested in `testPairingCeremonyAndRequirePairingGate`.
|
|
||||||
7b. **Resize without reconnect**: `requestMode(width:height:refreshHz:)` mid-stream —
|
|
||||||
the host rebuilds at the new mode in ~90 ms; the first new-mode AU is an IDR with
|
|
||||||
fresh parameter sets (the refresh-on-IDR decode flow handles it untouched) and
|
|
||||||
`currentMode()` reflects the switch. Wire it to window-resize events.
|
|
||||||
8. **Input capture** (stage 1): capture is a deliberate, reversible STATE owned by
|
|
||||||
`StreamLayerView`, Moonlight-style. Engaged when the stream starts / trust is
|
|
||||||
confirmed and when the user clicks into the video (that click is suppressed toward
|
|
||||||
the host); released by ⌘⎋ (toggles) or focus loss; NEVER engaged by mere app
|
|
||||||
activation — activating clicks may be title-bar drags or resizes, which used to get
|
|
||||||
their cursor warped away mid-drag. While captured: the local cursor is hidden +
|
|
||||||
frozen mid-view (the host renders its own), all input is forwarded, and the view
|
|
||||||
consumes key events as first responder so unhandled keyDowns don't beep — ⌘-combos
|
|
||||||
still work locally (⌘D disconnect, ⌘Q) *and* reach the host via GC. While released:
|
|
||||||
nothing is forwarded (`InputCapture.forwarding` gates the GC handlers; held
|
|
||||||
keys/buttons are flushed host-side on release so nothing sticks down), the cursor is
|
|
||||||
free, and the HUD shows "Click the stream to capture input". GC handlers only fire
|
|
||||||
while the app has focus, and focus loss also auto-releases everything held. One live capture per process (the GC
|
|
||||||
mouse/keyboard singletons have a single handler slot — ownership is tracked so a stale
|
|
||||||
capture's stop() can't clobber a newer one).
|
|
||||||
9. **iOS/iPadOS — ported and first-lit** (iPad simulator ↔ the real host, 60 fps).
|
|
||||||
`BUILD_IOS=1 bash scripts/build-xcframework.sh` builds device + universal-simulator
|
|
||||||
slices; the Xcode project has a second target, **Punktfunk-iOS**, sharing the same
|
|
||||||
synchronized sources. The iOS `StreamView` (StreamViewIOS.swift — same name/signature
|
|
||||||
as the macOS one, so the SwiftUI shell is identical) hosts the shared `StreamPump` in
|
|
||||||
a view controller for `prefersPointerLocked`: with a hardware mouse/trackpad that is
|
|
||||||
the iPadOS cursor capture (system honors it fullscreen-and-frontmost; in Stage
|
|
||||||
Manager it degrades to absolute-mouse forwarding). Input is routed by kind: DIRECT
|
|
||||||
fingers / Pencil are touches (each gets a wire touch id, coordinates mapped through the
|
|
||||||
aspect-fit letterbox into host-mode pixels — surface == host mode, so the host rescale is
|
|
||||||
the identity), while a mouse/trackpad is a MOUSE — pointer-LOCKED it is GCMouse relative
|
|
||||||
deltas; unlocked it is absolute moves + buttons + scroll over the UIKit pointer path
|
|
||||||
(hover + `.indirectPointer` touches), the local cursor staying visible so you can aim. An
|
|
||||||
indirect pointer is never sent as a touch. Touch is gated on trust (not forwarded under
|
|
||||||
the TOFU prompt), and returning to the foreground restores the capture you had on leaving.
|
|
||||||
`InputCapture` is cross-platform (GC works the same on iPadOS; ⌘⎋ is detected from
|
|
||||||
the HID stream there); audio routes via `AVAudioSession` (the Settings device
|
|
||||||
pickers are macOS-only). For the iPad-with-external-display setup: the target
|
|
||||||
enables multiple scenes + indirect input events — on Stage Manager iPads, drag the
|
|
||||||
punktfunk window onto the external screen and the stream runs there with full
|
|
||||||
keyboard/mouse/touch. While streaming the session is immersive (edge-to-edge,
|
|
||||||
status bar + home indicator hidden) and the iPadOS cursor is hidden over the video only
|
|
||||||
while the scene is actually pointer-LOCKED (`UIPointerInteraction` `.hidden()`); when the
|
|
||||||
lock isn't held it stays visible and the mouse forwards as an absolute cursor instead; on
|
|
||||||
iOS first run the stream mode defaults to the device's native screen so the video
|
|
||||||
fills the display. **tvOS** runs the same app (target **Punktfunk-tvOS**, first-lit
|
|
||||||
in the Apple TV simulator at 720p60): playback-only audio (no mic on tvOS),
|
|
||||||
focus-driven UI (`.card` host tiles), no kb/mouse capture yet — input lands with
|
|
||||||
gamepad support, the natural tvOS input anyway. While streaming there is NO focusable
|
|
||||||
control (a focusable Disconnect button would let the focus engine eat the controller's A
|
|
||||||
before the host sees it); the Siri Remote's **Menu** button disconnects (`.onExitCommand`).
|
|
||||||
Core slices are tier-3 Rust targets (see Build above). Known gaps: true pointer LOCK (`prefersPointerLocked`) isn't
|
|
||||||
consulted through UIHostingController, so the hidden cursor can still drift onto a
|
|
||||||
second screen (fixing it means putting the controller into the UIKit presentation
|
|
||||||
chain); and
|
|
||||||
AVAudioSession interruptions (calls, Siri) don't auto-restart the audio engines yet
|
|
||||||
(reconnect recovers).
|
|
||||||
|
|
||||||
## Known limitations of the current host (relevant to client UX)
|
|
||||||
|
|
||||||
- One session **at a time** (the listener is persistent, but a second concurrent client
|
|
||||||
waits in the accept queue until the current session ends — the virtual output and
|
|
||||||
encoder are single-tenant).
|
|
||||||
- Mid-stream renegotiation (resolution change without reconnect) is designed-for but not
|
|
||||||
implemented (the Welcome is one-shot today).
|
|
||||||
- Host-side gamepad injection needs `/dev/uinput` access on the box (udev rule from
|
|
||||||
`design/linux-setup.md`).
|
|
||||||
|
|||||||
+55
-160
@@ -1,189 +1,84 @@
|
|||||||
# punktfunk Decky plugin (SteamOS / Steam Deck)
|
# punktfunk — Steam Deck plugin (Decky)
|
||||||
|
|
||||||
A **[Decky Loader](https://decky.xyz/)** plugin that adds a **punktfunk** panel to the Steam
|
Stream to your **Steam Deck** without ever leaving Gaming Mode. This
|
||||||
Deck's Quick Access Menu (the QAM, opened with the `…` button), so you can launch the
|
**[Decky Loader](https://decky.xyz/)** plugin adds a **punktfunk** panel to the Quick Access Menu
|
||||||
punktfunk streaming client from **Gaming Mode** without dropping to the desktop.
|
(the `…` button): discover hosts on your network, pair with a PIN, tweak stream settings, and launch
|
||||||
|
a fullscreen, gamescope-focused stream — all from the couch, gamepad-navigable.
|
||||||
|
|
||||||
Because Decky plugins run inside Steam's CEF, the panel is built from real Steam UI
|
The video itself is the native GTK4 Linux client (the `io.unom.Punktfunk` flatpak); the plugin
|
||||||
primitives (`@decky/ui`: `PanelSection`, `PanelSectionRow`, `ButtonItem`, `Field`,
|
discovers, pairs, configures, and *launches it the right way* so gamescope fullscreens it — the same
|
||||||
`Spinner`) — so it looks and feels native to Gaming Mode.
|
Steam-shortcut trick MoonDeck uses. Because it's built from real Steam UI primitives (`@decky/ui`),
|
||||||
|
the panel looks and feels native to Gaming Mode.
|
||||||
> **Full Gaming-Mode client.** Discovery, a fullscreen page, in-UI SPAKE2 PIN pairing,
|
|
||||||
> stream settings, and a stream that actually launches fullscreen under gamescope (via a
|
|
||||||
> Steam shortcut, MoonDeck-style). The video itself is the existing GTK4 flatpak client
|
|
||||||
> (`io.unom.Punktfunk`) — the plugin discovers, pairs, configures, and *launches it the
|
|
||||||
> right way* so gamescope focuses it. The Steam-shortcut launch + pairing need a real Deck
|
|
||||||
> in Gaming Mode to fully confirm.
|
|
||||||
|
|
||||||
## What it does
|
## What it does
|
||||||
|
|
||||||
1. **Discover** — browses the LAN over mDNS for punktfunk/1 hosts (`_punktfunk._udp`,
|
1. **Discover** — browses the LAN over mDNS for punktfunk hosts, in both the QAM panel and a
|
||||||
backend `discover()` via `avahi-browse`). Shown in both the QAM panel and a **fullscreen
|
fullscreen page.
|
||||||
page** (Decky route `/punktfunk`, via `routerHook.addRoute`).
|
2. **Pair** — for a host that requires it, a gamepad-navigable PIN keypad runs the SPAKE2 pairing
|
||||||
2. **Pair** — for a `pair=required` host: a gamepad-navigable PIN keypad. The operator arms
|
ceremony headlessly, then remembers the host so future streams connect silently.
|
||||||
pairing on the host (it shows a 4-digit PIN), the user enters it on the Deck, and the
|
3. **Stream** — launches fullscreen via a hidden Steam shortcut so gamescope focuses it.
|
||||||
backend runs the SPAKE2 ceremony headlessly via the flatpak client's `--pair` mode
|
4. **Settings** — resolution / refresh / bitrate / gamepad / mic, written to the client's config.
|
||||||
(`pair()`), persisting the host as paired so the stream then connects silently.
|
|
||||||
3. **Stream** — launches fullscreen in Gaming Mode. The plugin registers ONE hidden
|
|
||||||
non-Steam shortcut pointing at `bin/punktfunkrun.sh`, passes `PF_HOST` as the shortcut's
|
|
||||||
Steam launch options, and starts it with `SteamClient.Apps.RunGame` — so gamescope
|
|
||||||
focuses + fullscreens it. (A flatpak launched directly from the backend is invisible:
|
|
||||||
gamescope only focuses the process tree Steam launched via `reaper` — gamescope#484.)
|
|
||||||
The wrapper then execs `flatpak run io.unom.Punktfunk --connect <host>`.
|
|
||||||
4. **Settings** — resolution / refresh / bitrate / gamepad / mic, written to the client's
|
|
||||||
`client-gtk-settings.json` (`get_settings`/`set_settings`), which the launched client reads.
|
|
||||||
|
|
||||||
To leave the stream: the in-client controller chord (**L1+R1+Start+Select**) or close the
|
To leave a stream: the in-client controller chord (**L1 + R1 + Start + Select**), or close the
|
||||||
"game" from the Steam overlay — exiting the client ends the Steam game and returns to
|
"game" from the Steam overlay — either returns you to Gaming Mode.
|
||||||
Gaming Mode automatically.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
| File | Role |
|
|
||||||
| --- | --- |
|
|
||||||
| `src/index.tsx` | Frontend: QAM panel + the `/punktfunk` fullscreen page (host list, PIN keypad modal, settings). |
|
|
||||||
| `src/steam.ts` | Steam-shortcut launch (`AddShortcut` / `SetAppLaunchOptions` / `RunGame`) — the focus-correct stream start. |
|
|
||||||
| `src/backend.ts` | Typed `callable` bridges to `main.py`. |
|
|
||||||
| `bin/punktfunkrun.sh` | The launch wrapper the Steam shortcut targets (so the window is focusable). |
|
|
||||||
| `main.py` | Backend: `discover` / `pair` / `runner_info` / `get_settings` / `set_settings` / `kill_stream` / `check_update`. |
|
|
||||||
| `plugin.json` | Decky plugin manifest. |
|
|
||||||
| `update.json` | CI-baked `{channel, manifest}` — where `check_update()` polls (absent on dev builds). |
|
|
||||||
| `decky.pyi` | Type stub for the injected `decky` module (vendored from the template). |
|
|
||||||
|
|
||||||
### Discovery (`discover()`)
|
|
||||||
|
|
||||||
Shells out to **`avahi-browse -rpt _punktfunk._udp`** (SteamOS and Bazzite ship
|
|
||||||
`avahi-daemon`; this avoids bundling python-zeroconf):
|
|
||||||
|
|
||||||
- `-r` resolve services, `-p` parseable output, `-t` terminate after the cache dump.
|
|
||||||
- Resolved records start with `=` and are semicolon-separated:
|
|
||||||
`=;iface;protocol;name;type;domain;hostname;address;port;txt`.
|
|
||||||
- The `txt` column is space-separated, quoted `"key=value"` tokens. We read the keys the
|
|
||||||
host advertises (`crates/punktfunk-host/src/discovery.rs`): `proto`, `fp`, `pair`, `id`.
|
|
||||||
- Records are deduped on the `id` TXT key (a host re-advertises per interface and across
|
|
||||||
IPv4/IPv6), preferring the IPv4 address for the user-facing host string.
|
|
||||||
|
|
||||||
### Client launch (`connect()`)
|
|
||||||
|
|
||||||
The client binary `punktfunk-client` is resolved in order: `PATH` → `/usr/bin` →
|
|
||||||
`/usr/local/bin` → `~/.local/bin` → a `flatpak run io.unom.Punktfunk` fallback. The resolved
|
|
||||||
argv and a clear `client-not-found` error surface to the UI. The child PID is tracked so
|
|
||||||
`disconnect()` (and plugin `_unload`) can terminate it.
|
|
||||||
|
|
||||||
> On the **Steam Deck** the client install is the flatpak `io.unom.Punktfunk`
|
|
||||||
> (`packaging/flatpak/`) — SteamOS `/usr` is read-only and lacks `libadwaita`/`libSDL3`, so
|
|
||||||
> the flatpak (which bundles them) is the canonical path; the resolver's flatpak fallback
|
|
||||||
> launches exactly that.
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- **Decky Loader** installed on the Deck (https://decky.xyz/).
|
|
||||||
- **`punktfunk-client`** (the GTK4/libadwaita Linux client, crate `punktfunk-client-linux`)
|
|
||||||
installed and runnable on the Deck — via `.deb`/RPM/flatpak, or symlinked into
|
|
||||||
`~/.local/bin`.
|
|
||||||
- **avahi** (`avahi-daemon` + `avahi-browse`) for discovery — present on SteamOS/Bazzite.
|
|
||||||
- A punktfunk/1 host on the LAN (`punktfunk-host serve` or `punktfunk1-host`).
|
|
||||||
|
|
||||||
## Build
|
|
||||||
|
|
||||||
```sh
|
|
||||||
pnpm install
|
|
||||||
pnpm build # rollup → dist/index.js
|
|
||||||
```
|
|
||||||
|
|
||||||
(`npm install && npm run build` also works.)
|
|
||||||
|
|
||||||
## Install on the Deck
|
## Install on the Deck
|
||||||
|
|
||||||
### Option A — Decky "install from URL" (recommended; published by CI)
|
You need **[Decky Loader](https://decky.xyz/)** and the **`io.unom.Punktfunk` flatpak**
|
||||||
|
([`packaging/flatpak`](../../packaging/flatpak/README.md)) installed on the Deck — SteamOS `/usr` is
|
||||||
|
read-only, so the flatpak (which bundles libadwaita/SDL3) is the canonical client. Discovery uses
|
||||||
|
`avahi-browse`, which ships on SteamOS/Bazzite.
|
||||||
|
|
||||||
CI (`.gitea/workflows/decky.yml`) builds the plugin into a store-layout zip and publishes it to
|
**Recommended — install from URL** (published by CI): in Decky → Settings → **Developer Mode** →
|
||||||
Gitea's **generic package registry** on every push to `main` and on `v*` tags, exposing a stable
|
**Install Plugin from URL**, paste:
|
||||||
URL. In Decky's settings → **Developer Mode** → **Install Plugin from URL**, paste:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
https://git.unom.io/api/packages/unom/generic/punktfunk-decky/latest/punktfunk.zip
|
https://git.unom.io/api/packages/unom/generic/punktfunk-decky/latest/punktfunk.zip
|
||||||
```
|
```
|
||||||
|
|
||||||
(or a pinned version: `.../punktfunk-decky/<version>/punktfunk.zip`). On tags the same zip is
|
(or a pinned `.../punktfunk-decky/<version>/punktfunk.zip`). The plugin then **self-updates** without
|
||||||
also attached to the Gitea release. The zip's layout is the store-required one — a single
|
the Decky store — when a newer build exists, an **Update to vX** button appears and drives Decky
|
||||||
top-level `punktfunk/` dir holding `plugin.json`, `package.json`, `main.py`, `dist/index.js`,
|
Loader's own (SHA-256-verified) install.
|
||||||
`README.md`, and `LICENSE`.
|
|
||||||
|
|
||||||
### Option B — manual dev copy (sideload)
|
## Build & sideload (development)
|
||||||
|
|
||||||
Decky's `~/homebrew/plugins/` is **root-owned** (PluginLoader runs as root and manages it), so a
|
|
||||||
plain `rsync` into it fails — stage to a writable temp dir, then `sudo`-install and restart the
|
|
||||||
loader. The two helper scripts do exactly this:
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cd clients/decky
|
cd clients/decky
|
||||||
pnpm install
|
pnpm install
|
||||||
|
pnpm build # rollup → dist/index.js
|
||||||
pnpm run package # → out/punktfunk/ + out/punktfunk-v<ver>.zip
|
pnpm run package # → out/punktfunk/ + out/punktfunk-v<ver>.zip
|
||||||
DECK=deck@<deck-ip> pnpm run deploy # rsync → /tmp, sudo cp into plugins/, chown root, restart
|
DECK=deck@<deck-ip> pnpm run deploy # rsync → /tmp, sudo-install into the root-owned plugins dir, restart loader
|
||||||
```
|
```
|
||||||
|
|
||||||
`deploy.sh` prompts for the Deck's sudo password interactively (via `ssh -t`); set `DECKPASS=…`
|
`~/homebrew/plugins/` is root-owned (the loader runs as root), so `deploy.sh` stages to a temp dir
|
||||||
to run it non-interactively. Equivalent by hand:
|
then `sudo`-installs and restarts the loader — set `DECKPASS=…` to run it non-interactively. A loader
|
||||||
|
restart is required for an out-of-band install to appear.
|
||||||
|
|
||||||
```sh
|
## Architecture
|
||||||
cd clients/decky && pnpm build && bash scripts/package.sh
|
|
||||||
rsync -azp --delete out/punktfunk/ deck@<deck-ip>:/tmp/punktfunk/
|
|
||||||
ssh -t deck@<deck-ip> 'sudo sh -c "rm -rf ~deck/homebrew/plugins/punktfunk && \
|
|
||||||
cp -r /tmp/punktfunk ~deck/homebrew/plugins/punktfunk && \
|
|
||||||
chown -R root:root ~deck/homebrew/plugins/punktfunk && systemctl restart plugin_loader"'
|
|
||||||
```
|
|
||||||
|
|
||||||
A loader restart is required for an out-of-band install to appear. The **punktfunk** panel then
|
| File | Role |
|
||||||
shows up in the Quick Access Menu.
|
| --- | --- |
|
||||||
|
| `src/index.tsx` | Frontend: QAM panel + the `/punktfunk` fullscreen page (host list, PIN keypad, settings). |
|
||||||
|
| `src/steam.ts` | Steam-shortcut launch (`AddShortcut` / `SetAppLaunchOptions` / `RunGame`) — the focus-correct stream start. |
|
||||||
|
| `src/backend.ts` | Typed `callable` bridges to `main.py`. |
|
||||||
|
| `bin/punktfunkrun.sh` | The launch wrapper the Steam shortcut targets (so the window is focusable). |
|
||||||
|
| `main.py` | Backend: `discover` (via `avahi-browse`) / `pair` / settings / `kill_stream` / `check_update`. |
|
||||||
|
| `plugin.json` · `update.json` | Decky manifest; CI-baked update channel. |
|
||||||
|
|
||||||
> The plugin launches the client via the flatpak `io.unom.Punktfunk` (see
|
The client binary is resolved `PATH` → `/usr/bin` → `/usr/local/bin` → `~/.local/bin` → a
|
||||||
> [`../../packaging/flatpak/README.md`](../../packaging/flatpak/README.md)) — install that on
|
`flatpak run io.unom.Punktfunk` fallback, so the flatpak install always works.
|
||||||
> the Deck too, or the panel's Connect surfaces a `client-not-found` error.
|
|
||||||
|
|
||||||
## Updating (self-update, no store)
|
|
||||||
|
|
||||||
The plugin updates itself without the official Decky store. CI (`decky.yml`) publishes a tiny
|
|
||||||
per-channel `manifest.json` next to the zip in the Gitea registry:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{"version":"0.3.123","artifact":".../punktfunk-decky/0.3.123/punktfunk.zip","sha256":"…"}
|
|
||||||
```
|
|
||||||
|
|
||||||
and bakes an `update.json` (`{channel, manifest}`) into the plugin so it knows which channel it was
|
|
||||||
installed from. The backend `check_update()` reads the **installed** version from `package.json` —
|
|
||||||
the value Decky itself reports (it does **not** read `plugin.json`) — fetches the channel manifest,
|
|
||||||
and compares. When a newer build exists the frontend shows an **Update to vX** button that drives
|
|
||||||
Decky Loader's own install RPC:
|
|
||||||
|
|
||||||
```ts
|
|
||||||
window.DeckyBackend.callable("utilities/install_plugin")(artifact, "punktfunk", version, hash, /*UPDATE=*/2)
|
|
||||||
```
|
|
||||||
|
|
||||||
The loader (root) downloads the immutable per-version zip, **SHA-256-verifies** it against `hash`,
|
|
||||||
replaces `~/homebrew/plugins/punktfunk`, and hot-reloads — the unprivileged backend never writes the
|
|
||||||
root-owned plugins dir itself. `window.DeckyBackend` / `utilities/install_plugin` are loader
|
|
||||||
internals (not `@decky/api`), so every access is guarded; missing them, the button falls back to a
|
|
||||||
toast pointing at **Install Plugin from URL**.
|
|
||||||
|
|
||||||
> CI stamps a **plain numeric** semver per channel (`0.3.<run>` canary, `X.Y.Z` stable) into
|
|
||||||
> `package.json`. Decky's `compare-versions` orders pre-release identifiers lexically (so `ci10 < ci9`)
|
|
||||||
> — a `-ciN` suffix would mis-detect updates.
|
|
||||||
|
|
||||||
**Optional — native Updates tab:** Decky's store is single-source (a custom store URL *replaces* the
|
|
||||||
official catalog), so punktfunk doesn't ship one by default. A user who wants the native update badge
|
|
||||||
can point Decky → Settings → **Custom store** at a punktfunk-only store JSON — not recommended if you
|
|
||||||
use other plugins, since it hides the official catalog.
|
|
||||||
|
|
||||||
## Limitations / next steps
|
## Limitations / next steps
|
||||||
|
|
||||||
- **Needs on-Deck validation in Gaming Mode**: the Steam-shortcut launch (`AddShortcut` /
|
- **Needs on-Deck validation in Gaming Mode** — the Steam-shortcut launch and headless pairing follow
|
||||||
`RunGame` / the `gameId` encoding) and the headless pairing env are coded to MoonDeck's
|
MoonDeck's proven pattern but are verified only at build time here.
|
||||||
proven pattern but verified only at build time here.
|
- No manual "add host by IP" entry yet (discovery is mDNS-only).
|
||||||
- mDNS discovery depends on `avahi-browse`; no manual "add host by IP" entry yet.
|
- No in-stream overlay inside the plugin — the client owns the session once launched.
|
||||||
- No in-stream overlay (latency/bitrate HUD) inside the plugin — the client owns the session
|
- Pairing needs the operator to **arm pairing on the host** so it shows the PIN; the plugin can't arm
|
||||||
once launched; leave it with the L1+R1+Start+Select chord.
|
it remotely.
|
||||||
- Pairing requires the operator to **arm pairing on the host** (so it shows the PIN); the
|
|
||||||
plugin can't arm it remotely (no host mgmt token on the Deck).
|
## Related
|
||||||
- Settings are written to the flatpak's sandbox config path; if the client ever moves its
|
|
||||||
config location, that path mapping must follow.
|
- **[Documentation](https://docs.punktfunk.unom.io/docs/steam-deck)** — Steam Deck setup guide
|
||||||
|
- **[Linux client](../linux/README.md)** — the app this plugin launches
|
||||||
|
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
|
||||||
|
|||||||
@@ -0,0 +1,77 @@
|
|||||||
|
# punktfunk — Linux client
|
||||||
|
|
||||||
|
The native **Linux** app for streaming a punktfunk host to your desktop, laptop, or Steam Deck.
|
||||||
|
It's a clean GTK4/libadwaita app that finds hosts on your network, pairs with a PIN, and puts a
|
||||||
|
low-latency stream on glass at your display's own resolution and refresh rate.
|
||||||
|
|
||||||
|
Built in Rust, it links the shared **`punktfunk-core`** directly (no C ABI) and speaks the fast
|
||||||
|
**`punktfunk/1`** protocol — QUIC control plane, GF(2¹⁶) FEC + AES-GCM data plane.
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- **Zero-copy hardware decode** — FFmpeg VAAPI decode → DRM-PRIME dmabuf → `GdkDmabufTexture`
|
||||||
|
(Tier-1 zero-copy on Intel and AMD), with an automatic software-HEVC fallback on NVIDIA or when
|
||||||
|
VAAPI is unavailable.
|
||||||
|
- **Your display's native mode** — the host builds a virtual output at exactly your WxH@Hz; no
|
||||||
|
scaling, no letterboxing. Steady 60 fps at 1080p60, ~6 ms capture→decoded on the LAN.
|
||||||
|
- **Audio both ways** — PipeWire playback with a jitter ring, plus mic uplink to the host.
|
||||||
|
- **Full controller support** — SDL3 gamepads with rumble and DualSense fidelity (lightbar, player
|
||||||
|
LEDs, touchpad, motion, adaptive-trigger replay). Click-to-capture keyboard and mouse, with a
|
||||||
|
release chord (Ctrl+Alt+Shift+Q) and focus-loss release.
|
||||||
|
- **Find hosts automatically** — mDNS discovery lists hosts on your LAN; saved hosts persist.
|
||||||
|
First connect does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then reconnects on
|
||||||
|
a pinned identity.
|
||||||
|
- **Per-host speed test** to pick a bitrate, plus compositor and mode preferences in Settings.
|
||||||
|
|
||||||
|
## Get it
|
||||||
|
|
||||||
|
Most people should install a package rather than build from source:
|
||||||
|
|
||||||
|
| Distro | Install |
|
||||||
|
|--------|---------|
|
||||||
|
| **Flatpak** (any distro, Steam Deck) | `io.unom.Punktfunk` — see [`packaging/flatpak`](../../packaging/flatpak/README.md) |
|
||||||
|
| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-client` *(after adding the repo)* |
|
||||||
|
| **Fedora / Bazzite** (rpm) | `rpm-ostree install punktfunk-client` |
|
||||||
|
| **Arch** (PKGBUILD) | see [`packaging/arch`](../../packaging/arch/README.md) |
|
||||||
|
|
||||||
|
Per-device install steps and pairing walkthrough:
|
||||||
|
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
|
||||||
|
|
||||||
|
## Build & run from source
|
||||||
|
|
||||||
|
Requires GTK ≥ 4.16, libadwaita ≥ 1.5, FFmpeg 7 or 8 (with VAAPI for hardware decode), PipeWire,
|
||||||
|
and SDL3 (with hidapi) development packages.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# from the repo root
|
||||||
|
cargo run -p punktfunk-client-linux # launch the app
|
||||||
|
cargo run -p punktfunk-client-linux -- --discover # list hosts on the LAN, then exit
|
||||||
|
cargo run -p punktfunk-client-linux -- --connect HOST[:PORT] # skip the host list and connect
|
||||||
|
```
|
||||||
|
|
||||||
|
The binary is named **`punktfunk-client`**. Handy flags: `--connect host[:port]` (start a session
|
||||||
|
immediately — for scripting and the Steam Deck launcher), `--discover [secs]`, and
|
||||||
|
`--pair <PIN> --connect host[:port]` (run the pairing ceremony headlessly). Force a decoder with
|
||||||
|
`PUNKTFUNK_DECODER=software|vaapi`.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
src/
|
||||||
|
main.rs · app.rs entry point, GTK application, CLI paths
|
||||||
|
ui_hosts.rs host list (mDNS + saved), pairing / trust dialogs
|
||||||
|
ui_settings.rs resolution · refresh · decoder · bitrate · compositor · mic
|
||||||
|
ui_stream.rs the stream window (GtkGraphicsOffload present) + input capture
|
||||||
|
session.rs session lifecycle over the NativeClient connector
|
||||||
|
video.rs FFmpeg VAAPI / software decode → dmabuf / texture
|
||||||
|
audio.rs PipeWire playback + mic uplink
|
||||||
|
gamepad.rs · keymap.rs SDL3 controllers + feedback; keyboard VK mapping
|
||||||
|
trust.rs · discovery.rs persistent identity, TOFU/PIN pairing, mDNS browse
|
||||||
|
tools/screenshots.sh store screenshot capture
|
||||||
|
```
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
|
||||||
|
- **[Steam Deck plugin](../decky/README.md)** — launches this client fullscreen in Gaming Mode
|
||||||
|
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
# punktfunk — probe (reference client)
|
||||||
|
|
||||||
|
`punktfunk-probe` is the **headless reference client** for the `punktfunk/1` protocol — a
|
||||||
|
command-line tool for testing, latency measurement, and validating host behavior. It's not a
|
||||||
|
streaming app you'd watch on; it connects, exercises a plane, and reports numbers. If you want to
|
||||||
|
actually stream, use the [Linux](../linux/README.md), [Windows](../windows/README.md),
|
||||||
|
[Apple](../apple/README.md), or [Android](../android/README.md) clients.
|
||||||
|
|
||||||
|
Because it links the same **`punktfunk-core`** as every other client, it's also the canonical
|
||||||
|
example of driving the protocol end to end: QUIC control plane, UDP data plane, and the side planes
|
||||||
|
(input, audio, rumble) over QUIC datagrams.
|
||||||
|
|
||||||
|
## What it does
|
||||||
|
|
||||||
|
- **Receives a real stream**, writes a playable `.h265`, and reports per-frame
|
||||||
|
**capture→…→reassembled latency** percentiles (the host stamps each frame with its capture clock).
|
||||||
|
- **Verification mode** against a synthetic host — byte-checks deterministic test frames.
|
||||||
|
- **Exercises every plane** with scripted test traffic:
|
||||||
|
`--input-test` (mouse/keyboard), `--mic-test` (a 440 Hz Opus tone up to the host mic),
|
||||||
|
`--touch-test` (a synthetic finger), `--rich-input-test` (DualSense touchpad + motion, logging the
|
||||||
|
HID-output feedback that comes back).
|
||||||
|
- **Trust** — `--pin <64-hex>` pins the host fingerprint; `--pair <PIN>` runs the SPAKE2 pairing
|
||||||
|
ceremony and prints the verified fingerprint to pin from then on. Without a pin it trusts on first
|
||||||
|
use.
|
||||||
|
- **Discovery** — `--discover [secs]` browses the LAN for `_punktfunk._udp` hosts and prints each
|
||||||
|
(name, addr:port, pairing requirement, cert fingerprint), then exits.
|
||||||
|
- **Negotiation knobs** — `--mode WxHxFPS`, `--remode` (mid-stream mode change), `--bitrate`,
|
||||||
|
`--audio-channels` (stereo / 5.1 / 7.1), `--compositor`, `--gamepad`, `--launch`, `--speed-test`.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# stream 720p120 from a host, save the video, and print latency percentiles:
|
||||||
|
cargo run -p punktfunk-probe -- --mode 1280x720x120 --connect HOST:PORT --out /tmp/a.h265
|
||||||
|
|
||||||
|
# list hosts on the LAN:
|
||||||
|
cargo run -p punktfunk-probe -- --discover
|
||||||
|
|
||||||
|
# pair with a host that requires it (read the PIN off the host), then stream:
|
||||||
|
cargo run -p punktfunk-probe -- --connect HOST:PORT --pair 1234
|
||||||
|
cargo run -p punktfunk-probe -- --connect HOST:PORT --pin <64-hex> --input-test
|
||||||
|
```
|
||||||
|
|
||||||
|
Full flag reference is in the module doc-comment at the top of [`src/main.rs`](src/main.rs).
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- **[Project README](../../README.md)** — the host, the streaming clients, and the protocol
|
||||||
|
- **`punktfunk-host punktfunk1-host`** — the persistent native-protocol listener to probe against
|
||||||
|
(see the "Running on this box" section of the repo README / `CLAUDE.md`)
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# punktfunk — Windows client
|
||||||
|
|
||||||
|
The native **Windows** app for streaming a punktfunk host to your PC. A modern WinUI 3 app that
|
||||||
|
discovers hosts on your network, pairs with a PIN, and streams at your display's own resolution and
|
||||||
|
refresh rate — with a hardware-accelerated D3D11 video path and HDR.
|
||||||
|
|
||||||
|
It's **pure Rust**: the UI is WinUI 3 driven through [windows-reactor](https://github.com/microsoft/windows-rs)
|
||||||
|
(a declarative, React-like framework), and it links the shared **`punktfunk-core`** directly to speak
|
||||||
|
the fast **`punktfunk/1`** protocol.
|
||||||
|
|
||||||
|
## Features
|
||||||
|
|
||||||
|
- **Hardware decode, GPU present** — FFmpeg HEVC with a **D3D11VA zero-copy path** (decoder and
|
||||||
|
presenter share one D3D11 device; NV12/P010 textures sampled straight into a `SwapChainPanel`
|
||||||
|
composition swapchain), with a robust software-decode fallback.
|
||||||
|
- **HDR10** — advertise 10-bit/HDR, detect PQ in-band, and flip the swapchain to `R10G10B10A2` +
|
||||||
|
ST.2084 with HDR10 metadata.
|
||||||
|
- **Your display's native mode** — the host builds a virtual display at exactly your WxH@Hz.
|
||||||
|
- **Audio both ways** — WASAPI render + mic capture.
|
||||||
|
- **Full controller support** — SDL3 gamepads with rumble, lightbar, and DualSense feedback.
|
||||||
|
- **Find hosts automatically** — mDNS discovery lists hosts on your LAN, alongside saved and manual
|
||||||
|
entries. First connect does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then
|
||||||
|
reconnects on a pinned identity.
|
||||||
|
- **Polished shell** — host cards, settings (resolution / refresh / decoder / bitrate / HDR / mic),
|
||||||
|
a status-chip stream HUD, and the full trust surface. Stream input uses Win32 low-level hooks with
|
||||||
|
a Ctrl+Alt+Shift+Q capture toggle.
|
||||||
|
|
||||||
|
Builds and ships for both **x64** and **ARM64** as a signed **MSIX**.
|
||||||
|
|
||||||
|
## Get it
|
||||||
|
|
||||||
|
Install the signed MSIX from the package registry — see
|
||||||
|
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
|
||||||
|
A stock [Moonlight](https://moonlight-stream.org/) client also works over GameStream if you prefer.
|
||||||
|
|
||||||
|
## Build from source
|
||||||
|
|
||||||
|
Windows-only (the crate builds as a stub on other platforms so the workspace stays green). You need
|
||||||
|
the MSVC toolchain, an `FFMPEG_DIR` FFmpeg tree, and CMake (SDL3 builds from source). windows-reactor's
|
||||||
|
`build.rs` downloads the Windows App SDK NuGets and needs `CARGO_WORKSPACE_DIR` set.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo build -p punktfunk-client-windows --target x86_64-pc-windows-msvc
|
||||||
|
|
||||||
|
# CLI paths for testing (no window):
|
||||||
|
punktfunk-client --discover # list hosts on the LAN
|
||||||
|
punktfunk-client --headless --connect host[:port] [--pin HEX] # connect, count frames, print stats
|
||||||
|
```
|
||||||
|
|
||||||
|
> `CARGO_HOME` must be an ASCII path — non-ASCII characters break SDL3's MSVC precompiled-header
|
||||||
|
> build. Packaging (MSIX manifest, signing) lives in [`packaging/`](packaging/).
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
src/
|
||||||
|
main.rs · app.rs entry point + CLI paths; WinUI 3 shell (windows-reactor)
|
||||||
|
present.rs · gpu.rs SwapChainPanel D3D11 composition swapchain; shared D3D11 device
|
||||||
|
video.rs FFmpeg HEVC decode (D3D11VA zero-copy + software fallback)
|
||||||
|
audio.rs WASAPI render + mic capture
|
||||||
|
gamepad.rs SDL3 controllers + rumble/lightbar/DualSense feedback
|
||||||
|
input.rs Win32 low-level keyboard/mouse hooks → host input
|
||||||
|
session.rs session lifecycle over the NativeClient connector
|
||||||
|
trust.rs · discovery.rs persistent identity, TOFU/PIN pairing, mDNS browse
|
||||||
|
packaging/ MSIX manifest, signing, pack script
|
||||||
|
```
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
|
||||||
|
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
# pf-driver-proto
|
||||||
|
|
||||||
|
The shared **host ↔ driver binary contract** for punktfunk's Windows **pf-vdisplay** virtual display —
|
||||||
|
the control IOCTLs and the IDD-push frame transport, defined exactly once.
|
||||||
|
|
||||||
|
It's a path dependency of **both** the host workspace ([`crates/punktfunk-host`](../punktfunk-host))
|
||||||
|
and the out-of-workspace driver workspace ([`packaging/windows/drivers/`](../../packaging/windows)),
|
||||||
|
so it must resolve identically from either build graph. That's why it's deliberately self-contained:
|
||||||
|
`no_std` (+ alloc), platform-neutral (GUID/LUID are plain integers each side converts to its own OS
|
||||||
|
type), and free of `*.workspace = true` inheritance.
|
||||||
|
|
||||||
|
Defining every wire struct here — with `const` size/offset asserts and `bytemuck` round-trips — turns
|
||||||
|
host↔driver ABI drift into a **compile error** instead of a silent frame or IOCTL corruption.
|
||||||
|
|
||||||
|
See the crate root ([`src/`](src/)) for the wire types; the Windows virtual-display design is in
|
||||||
|
[`design/windows-virtual-display-rust-port.md`](../../design/windows-virtual-display-rust-port.md).
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
# punktfunk-core
|
||||||
|
|
||||||
|
The **shared protocol core** — the one place where punktfunk's transport, forward error correction,
|
||||||
|
and crypto live. It's linked into the [host](../punktfunk-host/README.md) and every native client, so
|
||||||
|
there's exactly one implementation of the wire format everywhere.
|
||||||
|
|
||||||
|
Written in Rust with **no async on the per-frame path** (native threads only). It exposes both a
|
||||||
|
normal Rust API and a **stable, versioned C ABI**, so the Swift and Kotlin clients — and any C
|
||||||
|
embedder — link the same code as the Rust ones.
|
||||||
|
|
||||||
|
## What's in here
|
||||||
|
|
||||||
|
- **Transport & session** (`session.rs`, `transport/`, `packet.rs`) — the `punktfunk/1` data plane
|
||||||
|
over raw UDP: packetization, reassembly (with attacker-bounded limits), pacing, and socket tuning.
|
||||||
|
- **FEC** (`fec/`) — the wall-breaker. Two codes:
|
||||||
|
- **GF(2⁸)** classic Reed–Solomon with the *Cauchy* generator matrix — byte-identical to the
|
||||||
|
`nanors` library Moonlight uses, so our parity is decodable by a stock Moonlight client.
|
||||||
|
- **GF(2¹⁶) Leopard-RS** (SIMD, O(n log n)) — up to 65535 shards/block, which removes the ~1 Gbps
|
||||||
|
FEC ceiling. `punktfunk/1` negotiates this one.
|
||||||
|
- **Crypto** (`crypto.rs`) — AES-128-GCM session encryption with per-direction nonce salts and
|
||||||
|
sequence-as-AAD; SPAKE2 PIN pairing lives behind the `quic` feature.
|
||||||
|
- **QUIC control plane** (`quic.rs`, `client.rs`, feature `quic`) — the Hello/Welcome/Start handshake,
|
||||||
|
cert pinning/TOFU, reverse audio, and the embeddable `NativeClient` connector. This is the **only**
|
||||||
|
place `tokio`/`quinn` are allowed; the feature is **off by default** so the core stays runtime-free.
|
||||||
|
- **C ABI** (`abi.rs`) — the versioned surface (`punktfunk_abi_version()`, `PunktfunkConfig` carrying
|
||||||
|
its own `struct_size`) that generates [`include/punktfunk_core.h`](../../include/punktfunk_core.h)
|
||||||
|
via cbindgen at build time.
|
||||||
|
|
||||||
|
## Build outputs
|
||||||
|
|
||||||
|
The crate builds three ways at once (`crate-type = ["lib", "cdylib", "staticlib"]`):
|
||||||
|
|
||||||
|
| Output | Used by |
|
||||||
|
|--------|---------|
|
||||||
|
| `lib` (rlib) | the host, probe, and tools link it as a normal Rust crate |
|
||||||
|
| `cdylib` (`.so`/`.dylib`) | the Swift / Kotlin clients via the C ABI |
|
||||||
|
| `staticlib` (`.a`) | the C test harness and static embedding |
|
||||||
|
|
||||||
|
## Test
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test -p punktfunk-core # unit + proptest + loopback
|
||||||
|
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
|
||||||
|
bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip proof
|
||||||
|
```
|
||||||
|
|
||||||
|
## Design invariants (do not regress)
|
||||||
|
|
||||||
|
- **One core, linked everywhere** — protocol/FEC/crypto live only here, behind the stable C ABI.
|
||||||
|
- **No async on the hot path** — the per-frame pipeline is native threads only; `quic` (tokio/quinn)
|
||||||
|
is control-plane only, feature-gated, off by default.
|
||||||
|
- **Security hardening stays intact** — the reassembler bounds attacker-controlled fields before
|
||||||
|
allocating; AES-GCM keeps per-direction nonce salts + seq-as-AAD; the ABI checks `struct_size`.
|
||||||
|
Regression tests exist — keep them green.
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- **[`punktfunk-host`](../punktfunk-host/README.md)** — the streaming host built on this core
|
||||||
|
- **[Clients](../../clients/)** — the apps that link this core over the C ABI (or directly, in Rust)
|
||||||
|
- **[`design/implementation-plan.md`](../../design/implementation-plan.md)** — why GF(2¹⁶) FEC, the
|
||||||
|
latency budget, and the architecture thesis
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
# punktfunk-host
|
||||||
|
|
||||||
|
The **streaming host** — the program you run on the machine whose desktop or games you want to
|
||||||
|
stream. For each client that connects, it spins up a **virtual display sized to that device**,
|
||||||
|
captures it on the GPU, encodes with hardware NVENC/VAAPI/AMF/QSV, and sends it out over a
|
||||||
|
low-latency transport — no physical monitor, no letterboxing, no rearranging your real screens.
|
||||||
|
|
||||||
|
It speaks two protocols from **one process**:
|
||||||
|
|
||||||
|
- **GameStream** — so any [Moonlight](https://moonlight-stream.org/) / Artemis client works day one.
|
||||||
|
- **`punktfunk/1`** — punktfunk's own faster protocol (QUIC control plane, GF(2¹⁶) FEC + AES-GCM data
|
||||||
|
plane) that the native clients use.
|
||||||
|
|
||||||
|
Runs on **Linux** (the primary, most battle-tested path) and **Windows** (x64). The shared protocol,
|
||||||
|
FEC, and crypto live in [`punktfunk-core`](../punktfunk-core/README.md); this crate is everything
|
||||||
|
platform-facing around it.
|
||||||
|
|
||||||
|
## What it does
|
||||||
|
|
||||||
|
- **Per-client virtual displays at the exact WxH@Hz.** Linux uses per-compositor backends — **KWin**,
|
||||||
|
**gamescope**, **Mutter**, and **Sway/wlroots**; Windows uses its own all-Rust IddCx virtual display,
|
||||||
|
even on the secure desktop (UAC / lock screen).
|
||||||
|
- **GPU zero-copy capture → encode.** dmabuf → CUDA/Vulkan → NVENC on Linux; DXGI/WGC → GPU encode on
|
||||||
|
Windows. Encoders auto-select by GPU vendor: **NVENC** (NVIDIA), **VAAPI** (Linux AMD/Intel),
|
||||||
|
**AMF/QSV** (Windows AMD/Intel), or software H.264 as a floor. HDR/10-bit and HEVC 4:4:4 supported.
|
||||||
|
- **Input injection.** Mouse/keyboard (libei / gamescope EIS / wlr / Windows SendInput) and virtual
|
||||||
|
**gamepads** — Xbox 360/One, DualSense, DualShock 4 — with rumble and HID feedback back-channels.
|
||||||
|
- **Audio both ways.** Opus audio host→client, plus a virtual microphone the client can talk into.
|
||||||
|
- **Trust & discovery.** A persistent host identity, **SPAKE2 PIN pairing** (default) or TOFU, and
|
||||||
|
mDNS auto-advertisement so clients find the host without typing an IP.
|
||||||
|
- **Management API + web console.** A REST API (`mgmt.rs`, OpenAPI at
|
||||||
|
[`api/openapi.json`](../../api/openapi.json)) drives status, paired devices, and on-demand pairing;
|
||||||
|
the browser UI is in [`web/`](../../web/README.md).
|
||||||
|
|
||||||
|
## Run it
|
||||||
|
|
||||||
|
`punktfunk-host serve` runs inside your desktop session. Bare `serve` is the **secure native-only
|
||||||
|
default** (`punktfunk/1` + the management API); add `--gamestream` on a trusted LAN to also accept
|
||||||
|
stock Moonlight clients.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Linux, from the repo root (see the repo README "Running on this box" for the headless recipe):
|
||||||
|
cargo run -rp punktfunk-host -- serve # native-only (secure default)
|
||||||
|
cargo run -rp punktfunk-host -- serve --gamestream # + Moonlight compatibility
|
||||||
|
```
|
||||||
|
|
||||||
|
Then pair from the web console (`https://<host-ip>:3000`) or the client app.
|
||||||
|
|
||||||
|
Most people should install a **package** rather than run from source — see
|
||||||
|
[`packaging/`](../../packaging/README.md) (apt · rpm/COPR/bootc · Arch/sysext · Windows installer) and
|
||||||
|
the per-platform guides at **[docs.punktfunk.unom.io/docs/install](https://docs.punktfunk.unom.io/docs/install)**.
|
||||||
|
|
||||||
|
### Subcommands
|
||||||
|
|
||||||
|
| Command | Purpose |
|
||||||
|
|---------|---------|
|
||||||
|
| `serve` | The host (native `punktfunk/1` + mgmt API; `--gamestream` adds Moonlight). |
|
||||||
|
| `punktfunk1-host` | Standalone native-protocol listener for testing/measurement (`--source virtual`, `--max-sessions`). |
|
||||||
|
| `openapi` | Print the management-API OpenAPI spec (regenerates `api/openapi.json`). |
|
||||||
|
| `library` | Inspect the multi-store game library. |
|
||||||
|
| `service` · `driver` · `web` | Windows: SCM service, driver install, bundled web console. |
|
||||||
|
| `*-test` / `*-selftest` / `*-probe` | Diagnostics (input, zero-copy, HDR, compositor, gamepads). |
|
||||||
|
|
||||||
|
`--help` lists them all.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
src/
|
||||||
|
main.rs CLI + subcommand dispatch
|
||||||
|
config.rs · session_plan.rs · session_tuning.rs · pipeline.rs session setup + the frame pipeline
|
||||||
|
vdisplay/ per-compositor virtual outputs (kwin · gamescope · mutter · wlroots)
|
||||||
|
capture/ · capture.rs screen/dmabuf capture (+ Windows DXGI/WGC)
|
||||||
|
encode/ · encode.rs per-GPU encoders (nvenc · vaapi · ffmpeg_win (AMF/QSV) · sw)
|
||||||
|
zerocopy/ dmabuf → CUDA → NVENC bridges (EGL/GL tiled, Vulkan LINEAR)
|
||||||
|
inject/ · inject.rs input backends (libei · wlr · uinput gamepads · UHID DualSense/DS4)
|
||||||
|
audio/ · audio.rs Opus out + virtual mic (PipeWire / WASAPI)
|
||||||
|
gamestream/ Moonlight compat: nvhttp · pairing · rtsp · control · stream · gamepad · apps
|
||||||
|
punktfunk1.rs the native punktfunk/1 host (QUIC control + native-thread UDP data plane)
|
||||||
|
mgmt.rs · native_pairing.rs · stats_recorder.rs management API, pairing, perf capture
|
||||||
|
hdr.rs · library.rs HDR metadata; multi-store game library
|
||||||
|
linux/ · windows/ platform-confined backends
|
||||||
|
```
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- **[`punktfunk-core`](../punktfunk-core/README.md)** — the shared protocol · FEC · crypto core
|
||||||
|
- **[Clients](../../clients/)** — the apps that connect (Apple · Linux · Windows · Android · probe)
|
||||||
|
- **[Packaging](../../packaging/README.md)** & **[docs](https://docs.punktfunk.unom.io)** — install & operate
|
||||||
|
- **[`design/`](../../design/README.md)** — architecture rationale and deep-dive plans
|
||||||
+3
-3
@@ -3,9 +3,9 @@
|
|||||||
The punktfunk documentation site: [Fumadocs](https://fumadocs.dev) on
|
The punktfunk documentation site: [Fumadocs](https://fumadocs.dev) on
|
||||||
[TanStack Start](https://tanstack.com/start) (Vite + Nitro/bun preset).
|
[TanStack Start](https://tanstack.com/start) (Vite + Nitro/bun preset).
|
||||||
|
|
||||||
Content lives in [`content/docs/`](content/docs) as `.md`/`.mdx`. Several pages are imported
|
Content lives in [`content/docs/`](content/docs) as `.md`/`.mdx`. This site is the source of truth
|
||||||
verbatim from the repo's `docs/` design notes (with added frontmatter); edit those there or
|
for the **user-facing** guides; repo-internal design rationale lives in
|
||||||
here as the docs site becomes the source of truth.
|
[`../design/`](../design/README.md).
|
||||||
|
|
||||||
## API reference
|
## API reference
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -73,7 +73,7 @@ rpm-ostree install punktfunk && systemctl reboot
|
|||||||
systemctl reboot
|
systemctl reboot
|
||||||
```
|
```
|
||||||
|
|
||||||
## Option B — bootc (image-based, atomic)
|
## Option C — bootc (image-based, atomic)
|
||||||
|
|
||||||
Layer punktfunk into a Bazzite image once, then rebase any number of hosts onto it — no
|
Layer punktfunk into a Bazzite image once, then rebase any number of hosts onto it — no
|
||||||
per-host drift. See `bootc/Containerfile`:
|
per-host drift. See `bootc/Containerfile`:
|
||||||
|
|||||||
@@ -1,26 +1,27 @@
|
|||||||
# pf-dualsense — virtual DualSense UMDF2 HID minidriver (M0 spike)
|
# pf-dualsense — virtual DualSense UMDF2 HID minidriver
|
||||||
|
|
||||||
A self-authored **Rust UMDF2 HID minidriver** that presents a virtual Sony **DualSense**
|
A self-authored **Rust UMDF2 HID minidriver** that presents a virtual Sony **DualSense**
|
||||||
(VID `054C` / PID `0CE6`) to Windows, so games drive adaptive triggers / lightbar / rumble —
|
(VID `054C` / PID `0CE6`) to Windows, so games drive adaptive triggers / lightbar / rumble —
|
||||||
capabilities ViGEm structurally cannot deliver. This is the M0 feasibility spike for rich
|
capabilities ViGEm structurally cannot deliver. It's how the punktfunk Windows host gives a client's
|
||||||
controller support in the punktfunk Windows host.
|
DualSense a near-native feel with **no external gamepad dependencies** (no ViGEmBus).
|
||||||
|
|
||||||
## Status (2026-06-21)
|
Shipping: the driver is one member of the in-tree driver workspace
|
||||||
|
([`packaging/windows/drivers/`](../../README.md)), built from source in CI, and bundled +
|
||||||
|
`pnputil`-installed by the Windows host [installer](../../README.md). The host feeds it over a shared
|
||||||
|
memory channel from `crates/punktfunk-host/src/inject/dualsense_windows.rs`. The same UMDF driver also
|
||||||
|
serves the **DualShock 4** identity per a `device_type` byte the host stamps.
|
||||||
|
|
||||||
**Load + recognition: DONE.** A self-signed build **loads under Secure Boot ON** and enumerates as a
|
This README captures the driver-authoring lore — the bugs and the signing recipe that make a
|
||||||
genuine DualSense HID game controller (`Status: OK`, VID `054C`, 273-byte DualSense report descriptor,
|
self-signed UMDF HID driver actually load. The authoritative build/sign/package flow (CI + Inno Setup)
|
||||||
PID `0CE6` via `GET_DEVICE_ATTRIBUTES`). Validated live on the RTX box (`192.168.1.173`, Win11 25H2).
|
lives in the [Windows host packaging README](../../README.md).
|
||||||
|
|
||||||
**Remaining:** the real-game `0x02` adaptive-trigger gate (Cyberpunk 2077 on the interactive desktop →
|
## Build workspace
|
||||||
confirm `[pf-ds] *** OUTPUT ...` in the driver log), then wire into the host (M1+).
|
|
||||||
|
|
||||||
## This is a reference snapshot
|
This crate builds as a member of the [`packaging/windows/drivers/`](../../drivers) workspace, which
|
||||||
|
uses the published **crates.io `wdk`/`wdk-sys`/`wdk-build`** (0.4/0.5) — not the old dev-box
|
||||||
The crate's `Cargo.toml` uses path-deps into `microsoft/windows-drivers-rs`
|
`windows-drivers-rs` path-deps. It's a separate cargo workspace from the main tree because driver
|
||||||
(`../../crates/wdk{,-sys,-build}`), so it builds **inside a `windows-drivers-rs` checkout's
|
crates are cdylibs built with the WDK toolchain on Windows only; it path-deps the shared ABI crate
|
||||||
`examples/` dir**, not standalone in this repo. On the dev box it lives at
|
[`crates/pf-driver-proto`](../../../../crates/pf-driver-proto/README.md).
|
||||||
`C:\Users\Public\m0\windows-drivers-rs\examples\pf-dualsense`. These files are checked in for
|
|
||||||
version control / portability of the spike.
|
|
||||||
|
|
||||||
## Build / sign / install recipe (the one that actually loads)
|
## Build / sign / install recipe (the one that actually loads)
|
||||||
|
|
||||||
@@ -76,9 +77,10 @@ silently breaks them:
|
|||||||
zero requests → `EvtIoDeviceControl` never fires → no HID handshake → ~5 s timeout →
|
zero requests → `EvtIoDeviceControl` never fires → no HID handshake → ~5 s timeout →
|
||||||
`CM_PROB_FAILED_START`. Set to `u32::MAX`.
|
`CM_PROB_FAILED_START`. Set to `u32::MAX`.
|
||||||
|
|
||||||
## Known limitations
|
## Notes
|
||||||
|
|
||||||
- Uses **statics, not per-device WDF contexts** → only one device instance per WUDFHost works.
|
- **Multi-pad** works via `UmdfHostProcessSharing=ProcessSharingDisabled` — each pad gets its own
|
||||||
Multi-instance needs proper device contexts.
|
WUDFHost (so the per-instance statics don't collide), and the driver reads its pad index from the
|
||||||
- Port of the WDK `vhidmini2` UMDF2 sample; DualSense identity + 273-byte descriptor + feature blobs
|
device Location (`WdfDeviceAllocAndQueryProperty`) to map its own `*-shm-<index>` channel.
|
||||||
`0x05`/`0x09`/`0x20` from `crates/punktfunk-host/src/inject/dualsense.rs`.
|
- Port of the WDK `vhidmini2` UMDF2 sample; the DualSense identity + 273-byte descriptor + feature
|
||||||
|
blobs `0x05`/`0x09`/`0x20` come from `crates/punktfunk-host/src/inject/dualsense.rs`.
|
||||||
|
|||||||
@@ -0,0 +1,15 @@
|
|||||||
|
# latency-probe
|
||||||
|
|
||||||
|
A **glass-to-glass latency** measurement tool (design/implementation-plan §10): it renders a
|
||||||
|
timestamp/QR on the host, reads it back off the client's capture (or a photodiode, for true photons),
|
||||||
|
and tracks p50/p99 — so latency regressions are quantifiable rather than felt.
|
||||||
|
|
||||||
|
> **Status: scaffold.** The measurement harness is stubbed out and not yet wired to a live pipeline.
|
||||||
|
> For latency numbers today, use the per-frame **capture→…→reassembled** percentiles the
|
||||||
|
> [`probe`](../../clients/probe/README.md) client reports over a real `punktfunk/1` session.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo run -p latency-probe # from the repo root
|
||||||
|
```
|
||||||
|
|
||||||
|
Companion to [`loss-harness`](../loss-harness/README.md) (FEC loss sweep).
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
# loss-harness
|
||||||
|
|
||||||
|
A **FEC loss-resilience sweep** for [`punktfunk-core`](../../crates/punktfunk-core/README.md). It
|
||||||
|
drives access units through the in-process loopback at increasing packet-loss rates — for **both** FEC
|
||||||
|
schemes (GF(2⁸) and GF(2¹⁶)) — and reports how many frames survive.
|
||||||
|
|
||||||
|
It's a pure-software stand-in for `tc netem`: no network, no root, runs anywhere `punktfunk-core`
|
||||||
|
builds. Use it to sanity-check the FEC before reaching for the real `punktfunk/1` harness (which adds
|
||||||
|
`tc netem` jitter/reorder on the UDP path).
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo run -p loss-harness # from the repo root
|
||||||
|
```
|
||||||
|
|
||||||
|
Part of the measurement tooling (design/implementation-plan §10), alongside
|
||||||
|
[`latency-probe`](../latency-probe/README.md).
|
||||||
Reference in New Issue
Block a user