docs: rework client/crate READMEs, add missing ones
windows-drivers / probe-and-proto (push) Successful in 24s
windows-drivers / driver-build (push) Successful in 1m18s
apple / swift (push) Successful in 1m5s
android / android (push) Successful in 4m21s
ci / rust (push) Successful in 5m3s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 2m48s
windows-host / package (push) Successful in 7m10s
decky / build-publish (push) Successful in 24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m38s
release / apple (push) Successful in 9m1s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m32s
flatpak / build-publish (push) Successful in 4m59s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m7s
docker / deploy-docs (push) Successful in 25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m49s
windows-drivers / probe-and-proto (push) Successful in 24s
windows-drivers / driver-build (push) Successful in 1m18s
apple / swift (push) Successful in 1m5s
android / android (push) Successful in 4m21s
ci / rust (push) Successful in 5m3s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 2m48s
windows-host / package (push) Successful in 7m10s
decky / build-publish (push) Successful in 24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m38s
release / apple (push) Successful in 9m1s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m32s
flatpak / build-publish (push) Successful in 4m59s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m7s
docker / deploy-docs (push) Successful in 25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m49s
Rework the client READMEs to be accurate and inviting to first-time
visitors, and fill in the gaps where crates and tools had none.
- Rewrite clients/{apple,android,decky} READMEs (features-first, trim
dense internal narrative; drop the stale "one session at a time" /
"renegotiation not implemented" section from the Apple README).
- Add READMEs for clients/{linux,windows,probe}, which had none.
- Add crate READMEs for punktfunk-host, punktfunk-core, pf-driver-proto.
- Add brief READMEs for tools/{loss-harness,latency-probe}.
- Fix packaging/README duplicate "Option B" heading (bootc -> Option C).
- Fix docs-site/README stale docs/ -> design/ reference.
- De-stale packaging/windows/drivers/pf-dualsense README (drop "M0 spike"
/ external-checkout framing; reflect in-tree workspace + shipped +
installer-bundled + multi-pad), keeping the driver-authoring lore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+50
-55
@@ -1,83 +1,78 @@
|
||||
# punktfunk Android client
|
||||
# punktfunk — Android client (phone & TV)
|
||||
|
||||
Native Android client for **punktfunk/1**, targeting **phone + TV** (Compose, D-pad + touch).
|
||||
The native **Android** app for streaming a punktfunk host to your phone, tablet, or Android TV. A
|
||||
Compose app that finds hosts on your network, pairs with a PIN, and streams at the display's own
|
||||
resolution — with hardware HEVC decode, HDR10, and controller support, built for both touch and the
|
||||
couch (D-pad / gamepad focus navigation).
|
||||
|
||||
## Architecture — Rust-heavy (like the Linux client, not thin-native like Apple)
|
||||
## Features
|
||||
|
||||
Kotlin cannot `import` the cbindgen C header the way Swift can, so a native bridge is unavoidable.
|
||||
We write it in **Rust** and link `punktfunk-core` directly — so the Android client reuses the Linux
|
||||
- **Hardware decode** — NDK `AMediaCodec` HEVC → `SurfaceView`, including **HDR10** (Main10 /
|
||||
BT.2020 PQ), with low-latency tuning and a live stats HUD.
|
||||
- **Audio both ways** — Opus + Oboe playback with a jitter ring, plus mic uplink to the host.
|
||||
- **Controller support** — buttons + axes with rumble and HID feedback (lightbar / adaptive
|
||||
triggers); D-pad / gamepad focus navigation for TV and phone.
|
||||
- **Find hosts automatically** — native mDNS discovery; first connect does a one-time **SPAKE2 PIN
|
||||
pairing** (or TOFU on trusted LANs), then reconnects on a Keystore-wrapped, pinned identity.
|
||||
- **Compose UI** — Connect / Settings / Stream screens with Material You theming.
|
||||
|
||||
Built for `arm64-v8a` + `x86_64`.
|
||||
|
||||
## Get it
|
||||
|
||||
Published to **Google Play (Internal Testing)** — join the beta via the
|
||||
[Discord](https://discord.gg/kaPNvzMuGU). Per-device setup and pairing:
|
||||
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
|
||||
|
||||
## How it's built — Rust-heavy
|
||||
|
||||
Kotlin can't `import` the cbindgen C header the way Swift can, so a native bridge is unavoidable. We
|
||||
write it in **Rust** and link `punktfunk-core` directly — so the Android client reuses the Linux
|
||||
client's orchestration (audio jitter ring, VK keymap inverse, latency/skew math, capture state
|
||||
machine, trust logic) instead of re-porting it into Kotlin.
|
||||
|
||||
| Side | Owns |
|
||||
|------|------|
|
||||
| **Rust** (`clients/android/native` → `libpunktfunk_android.so`) | the JNI seam, `NativeClient` (QUIC control + UDP data plane), AnnexB→`AMediaCodec` decode, Opus+Oboe audio, VK keymap, latency math, trust/pairing, **mDNS discovery** (`mdns-sd`, the same browse the Linux/Windows clients use) |
|
||||
| **Kotlin** (`clients/android`) | Compose UI (host grid / settings / stream), `SurfaceView` lifecycle, input capture, the Wi-Fi `MulticastLock` + permission UX, Keystore identity, permissions |
|
||||
| **Rust** (`native/` → `libpunktfunk_android.so`) | the JNI seam, `NativeClient` (QUIC control + UDP data plane), AnnexB → `AMediaCodec` decode (incl. HDR10), Opus + Oboe audio + mic, controller feedback, latency math, trust/pairing, `mdns-sd` discovery |
|
||||
| **Kotlin** (`app/`, `kit/`) | Compose UI, `SurfaceView` lifecycle, input capture, the Wi-Fi `MulticastLock` + permission UX, Keystore identity |
|
||||
|
||||
The single seam is `io.unom.punktfunk.kit.NativeBridge` ⇄ `Java_io_unom_punktfunk_kit_NativeBridge_*`.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
clients/android/native/ Rust cdylib (workspace member) — links punktfunk-core directly
|
||||
src/lib.rs JNI seam (connect/pair, input, plane getters, abi/core version)
|
||||
native/ Rust cdylib (workspace member) — links punktfunk-core directly
|
||||
src/lib.rs JNI seam (connect/pair, input, plane getters, versions)
|
||||
src/session.rs session lifecycle + plane pumps
|
||||
src/decode.rs AnnexB → AMediaCodec HEVC hardware decode → SurfaceView (incl. HDR10)
|
||||
src/audio.rs · src/mic.rs Opus + Oboe playback / mic uplink (jitter ring)
|
||||
src/feedback.rs rumble + HID output (lightbar / adaptive triggers)
|
||||
src/stats.rs live video stats
|
||||
|
||||
clients/android/ Gradle project (this dir)
|
||||
settings.gradle.kts · build.gradle.kts · gradle.properties · gradlew
|
||||
app/ :app — Compose UI: Connect / Settings / Stream screens (phone + TV)
|
||||
kit/ :kit — NativeBridge · discovery (native mdns-sd, polled) · Gamepad · Keymap ·
|
||||
security (Keystore identity + known-host store) · cargo-ndk build
|
||||
src/audio.rs · src/mic.rs Opus + Oboe playback / mic uplink
|
||||
src/feedback.rs · src/stats.rs rumble + HID feedback; live video stats
|
||||
app/ :app — Compose UI: Connect / Settings / Stream (phone + TV)
|
||||
kit/ :kit — NativeBridge · native mDNS discovery · Gamepad · Keymap · Keystore identity
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Android SDK + **NDK r30** (`30.0.14904198`), `platforms;android-37.0`, `build-tools;37.0.0`,
|
||||
**`cmake;3.22.1`** (`sdkmanager "cmake;3.22.1"` — the `cmake` crate builds libopus with it)
|
||||
- **JDK 21** for Gradle/AGP (AGP 9.2 runs on JDK 17–21, *not* a newer default JDK like 25)
|
||||
- Rust + `rustup target add aarch64-linux-android x86_64-linux-android` + `cargo install cargo-ndk`
|
||||
|
||||
Toolchain pinned: AGP 9.2.0 · Gradle 9.4.1 · Kotlin 2.3.21 · Compose BOM 2026.05.01 ·
|
||||
compileSdk 37 · targetSdk 36 · minSdk 31 · ABIs arm64-v8a + x86_64.
|
||||
|
||||
## Build & run
|
||||
|
||||
**Android Studio:** open `clients/android` — it uses its bundled JBR 21 automatically. The
|
||||
`cargoNdk*` task builds the `.so` as part of the normal build.
|
||||
**Prerequisites:** Android SDK + **NDK r30** (`30.0.14904198`), `platforms;android-37.0`,
|
||||
`build-tools;37.0.0`, **`cmake;3.22.1`** (builds libopus); **JDK 21** (AGP 9.2 runs on JDK 17–21, not
|
||||
a newer default); Rust with `rustup target add aarch64-linux-android x86_64-linux-android` and
|
||||
`cargo install cargo-ndk`. Toolchain is pinned (AGP 9.2 · Gradle 9.4.1 · Kotlin 2.3.21 · Compose BOM
|
||||
2026.05.01 · compileSdk 37 · minSdk 31).
|
||||
|
||||
**CLI** (point Gradle at a JDK 21 if your machine default is newer, e.g. JDK 25):
|
||||
**Android Studio:** open `clients/android` — it uses its bundled JBR 21, and the `cargoNdk*` task
|
||||
builds the `.so` as part of the normal build.
|
||||
|
||||
**CLI** (point Gradle at JDK 21 if your machine default is newer):
|
||||
|
||||
```sh
|
||||
# Adoptium/Temurin 21 (installed by the Android Studio setup, or `brew install temurin@21`):
|
||||
export JAVA_HOME="$(/usr/libexec/java_home -v 21)"
|
||||
export JAVA_HOME="$(/usr/libexec/java_home -v 21)" # or your Temurin 21 path
|
||||
cd clients/android
|
||||
./gradlew :app:assembleDebug # cargo-ndk cross-compiles libpunktfunk_android.so first
|
||||
./gradlew :app:installDebug # onto a running emulator/device
|
||||
|
||||
# Emulators (created during env setup): emulator -avd pf_phone | emulator -avd pf_tv
|
||||
# emulators from env setup: emulator -avd pf_phone | emulator -avd pf_tv
|
||||
```
|
||||
|
||||
The debug APK lands in `app/build/outputs/apk/debug/`. Launch it, pick a host from the list, pair,
|
||||
and stream.
|
||||
The debug APK lands in `app/build/outputs/apk/debug/`. Launch it, pick a host, pair, and stream.
|
||||
|
||||
## Status
|
||||
## Related
|
||||
|
||||
A working native client (phone + Android TV), at parity with the Linux and Apple apps for the core
|
||||
streaming experience:
|
||||
|
||||
- **Video** — `AMediaCodec` hardware HEVC decode → `SurfaceView`, including **HDR10** (Main10 /
|
||||
BT.2020 PQ), with low-latency decode tuning and a live stats HUD.
|
||||
- **Audio** — Opus + Oboe playback with a jitter ring, plus mic uplink to the host.
|
||||
- **Input** — game controllers (buttons + axes) with rumble and HID feedback; D-pad /
|
||||
game-controller focus navigation for the couch (TV + phone).
|
||||
- **Discovery & trust** — native `mdns-sd` mDNS host list (polled over JNI; the same browse the
|
||||
Linux/Windows clients use, not `NsdManager`), SPAKE2 PIN pairing and TOFU, with a
|
||||
Keystore-wrapped client identity and a known-host store.
|
||||
- **UI** — Compose host list / settings / stream screens, Material You theming.
|
||||
- **Shipping** — built for `arm64-v8a` + `x86_64`; published to Google Play (Internal Testing).
|
||||
|
||||
`crates/punktfunk-core` uses the `ring` `rcgen` backend so the client `.so` is aws-lc-free.
|
||||
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
|
||||
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
|
||||
|
||||
+89
-338
@@ -1,364 +1,115 @@
|
||||
# punktfunk Apple client (SwiftUI)
|
||||
# punktfunk — Apple client (macOS · iOS · iPadOS · tvOS)
|
||||
|
||||
The native macOS/iOS client for **`punktfunk/1`** (the post-GameStream protocol). All
|
||||
networking/protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
|
||||
input datagrams, Opus audio, cert pinning — lives in the shared Rust core (statically
|
||||
linked as `PunktfunkCore.xcframework`); this package is the Swift shell: decode
|
||||
(VideoToolbox), present (SwiftUI), input capture.
|
||||
The native **Apple** app for streaming a punktfunk host to your Mac, iPhone, iPad, or Apple TV. A
|
||||
SwiftUI app that finds hosts on your network, pairs with a PIN, and streams at your display's own
|
||||
resolution and refresh rate — with VideoToolbox hardware decode and full controller support.
|
||||
|
||||
## Status — working client (macOS, with iOS / tvOS in the shared build)
|
||||
All the networking and protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
|
||||
Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (statically linked as
|
||||
`PunktfunkCore.xcframework`). This package is the Swift shell: decode, present, input, and UI.
|
||||
|
||||
A full streaming client: VideoToolbox HEVC decode, controllers incl. DualSense feedback, host
|
||||
discovery, PIN pairing, and a network speed test. The lower-latency **stage-2 presenter**
|
||||
(`VTDecompressionSession` → `CAMetalLayer`) is built and opt-in (Settings → Presenter); see below.
|
||||
## Features
|
||||
|
||||
First light was achieved 2026-06-10 — validated live, Mac ↔ a Linux host over the LAN: gamescope
|
||||
virtual output → NVENC HEVC →
|
||||
`punktfunk/1` (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox →
|
||||
`AVSampleBufferDisplayLayer` on glass at 1280×720@60, with mouse/keyboard flowing back as
|
||||
QUIC datagrams into the host's gamescope EIS injector (thousands of events injected during
|
||||
the session). Headless variant of the same proof: `RemoteFirstLightTests` decoded 60/60
|
||||
received AUs spanning 983 ms of host capture clock.
|
||||
- **Hardware decode** — VideoToolbox HEVC, with a low-latency **stage-2 presenter**
|
||||
(`VTDecompressionSession` → `CAMetalLayer`, presented off a `CADisplayLink`, ~11 ms p50) as the
|
||||
default and an `AVSampleBufferDisplayLayer` fallback.
|
||||
- **HDR & 4:4:4** — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR
|
||||
reconfiguration, and hardware-probed 4:4:4 support.
|
||||
- **Your display's native mode** — the host builds a virtual output at exactly your WxH@Hz;
|
||||
mid-stream resize renegotiates without reconnecting.
|
||||
- **Audio both ways** — Opus playback (CoreAudio, no bundled libopus) with a jitter ring, plus mic
|
||||
uplink; speaker/mic selectable in Settings.
|
||||
- **Full controller support** — one selected controller forwarded as pad 0, including **DualSense**
|
||||
feedback (rumble → CoreHaptics, lightbar, player LEDs, adaptive triggers) and touchpad/motion. The
|
||||
virtual pad type auto-resolves from your physical controller.
|
||||
- **Mouse & keyboard** — `GCMouse`/`GCKeyboard` capture with click-to-capture and a ⌘⎋ release, plus
|
||||
iPad pointer lock and touch input.
|
||||
- **Find hosts automatically** — mDNS discovery (`NWBrowser` over `_punktfunk._udp`); first connect
|
||||
does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then reconnects on a pinned,
|
||||
Keychain-stored identity.
|
||||
- **Tune the stream** — a fps / Mb·s / **latency** HUD (skew-corrected across machines), a bitrate
|
||||
control, a per-host **network speed test** with a recommended bitrate, and a host-compositor picker.
|
||||
|
||||
The connector underneath (`punktfunk_core::client::NativeClient` over the C ABI) carries the
|
||||
full session: video AUs, **Opus audio** (`nextAudio()`), **rumble** (`nextRumble()`),
|
||||
**DualSense feedback** (`nextHidOutput()` — lightbar, player LEDs, adaptive-trigger
|
||||
effects), input incl. gamepads + DualSense touchpad/motion (`sendTouchpad`/`sendMotion`),
|
||||
and **cert pinning + TOFU** (`pinSHA256:`/`hostFingerprint`) — see
|
||||
`punktfunk1.rs::tests::c_abi_connection_roundtrip` (three sequential sessions: TOFU, pinned
|
||||
reconnect, wrong-pin rejection). The host (`punktfunk-host punktfunk1-host`) is a persistent listener:
|
||||
reconnect at will during development.
|
||||
Runs from one shared codebase across **macOS, iOS, iPadOS, and tvOS**.
|
||||
|
||||
What's here, all compiled and tested on macOS (Xcode 26.5 / Swift 6.3):
|
||||
## Get it
|
||||
|
||||
- **`PunktfunkKit`** (library)
|
||||
- `PunktfunkConnection.swift` — wrapper over the C ABI. AUs/audio are copied into `Data`
|
||||
(the C pointer is only valid until the next call of the same kind). `close()` is safe
|
||||
from any thread: per-plane locks enforce the C contract ("never close with a
|
||||
`next_au`/`next_audio` in flight") instead of leaving it to callers. Pinning + TOFU
|
||||
via `pinSHA256:`/`hostFingerprint`.
|
||||
- `AnnexB.swift` — in-band VPS/SPS/PPS → `CMVideoFormatDescription`; Annex-B → AVCC
|
||||
`CMSampleBuffer` with `DisplayImmediately` set.
|
||||
- `StreamView.swift` — SwiftUI `NSViewRepresentable` over `AVSampleBufferDisplayLayer`
|
||||
(stage-1 presenter: the layer hardware-decodes compressed HEVC itself). One pump
|
||||
thread per view, token-cancelled so reconnects can't double-pump.
|
||||
- `InputCapture.swift` — `GCMouse` raw deltas + `GCKeyboard` HID→VK mapping (the host's
|
||||
`vk_to_evdev` consumes Windows VKs), with fractional-delta accumulation so sub-pixel
|
||||
motion isn't truncated away. Buttons use GameStream ids (1=left … 5=X2). Scroll is
|
||||
WHEEL_DELTA(120)-scaled: macOS via the stream view's `scrollWheel` override, iPad via
|
||||
GCMouse's scroll dpad when pointer-locked and a scroll-only `UIPanGestureRecognizer`
|
||||
otherwise (trackpad gestures never reach GC's scroll dpad).
|
||||
- `GamepadManager.swift` — app-lifetime controller discovery + selection (`.shared`):
|
||||
watches `GCController` connect/disconnect, fingerprints each pad for the Settings UI
|
||||
(name, capabilities, battery), and selects the ONE controller forwarded to the host
|
||||
(user pin via "Use controller", else most recently connected extended gamepad).
|
||||
- `GamepadCapture.swift` — the active controller → wire: snapshot-diff over
|
||||
`GCExtendedGamepad` into incremental `gamepadButton`/`gamepadAxis` events (pad 0),
|
||||
plus DualSense touchpad contacts and ~250 Hz motion samples on the rich-input plane
|
||||
(the GC→DualSense unit conversions live in `GamepadWire`, one place). Held state is
|
||||
released on the wire on controller switch / app deactivation / stop.
|
||||
- `GamepadFeedback.swift` + `DualSenseTriggerEffect.swift` — host feedback → the real
|
||||
controller: one drain thread for `nextRumble()` (→ `CHHapticEngine` per handle
|
||||
locality) and `nextHidOutput()` (lightbar → `GCDeviceLight`, player LEDs →
|
||||
`playerIndex`, adaptive-trigger effect blocks → a total, table-driven parser →
|
||||
`GCDualSenseAdaptiveTrigger`, exact for the 10-zone positional modes).
|
||||
- `HostDiscovery.swift` — LAN auto-discovery: an `NWBrowser` over `_punktfunk._udp`
|
||||
(the host's `crate::discovery` mDNS advert), resolving each service to an IP:port via a
|
||||
throwaway `NWConnection` and parsing the TXT (`fp` advisory cert fingerprint, `pair`,
|
||||
stable `id`). iOS/tvOS need `NSBonjourServices` (`Config/Info.plist`) or the system
|
||||
blocks the browse.
|
||||
- **`PunktfunkClient`** (the app): hosts grid (saved in UserDefaults) with an **On this
|
||||
network** section listing mDNS-discovered hosts (tap to save + connect, or pair if the
|
||||
host requires it), "+" toolbar sheet to add hosts manually, stream mode in Settings (⌘,),
|
||||
two trust flows — the
|
||||
trust-on-first-use fingerprint prompt over the live-but-blurred stream, and SPAKE2 PIN
|
||||
pairing (`PairSheet`, from a host card's context menu or the trust prompt;
|
||||
`ClientIdentityStore` keeps the client identity in the Keychain and presents it on
|
||||
every connect) — then pinned reconnects, fps/Mb-s HUD + a **capture→client-receipt latency**
|
||||
line (`LatencyMeter`, p50/p95): the AU `pts_ns` (host capture clock) to the instant the client
|
||||
received it, **skew-corrected** across machines via `PunktfunkConnection.clockOffsetNs` (the
|
||||
connect-time wall-clock handshake, `punktfunk_connection_clock_offset_ns`). It excludes the
|
||||
layer's decode+present (stage-1 `AVSampleBufferDisplayLayer` has no per-frame present callback);
|
||||
the opt-in **stage-2 presenter** (Settings → Presenter) adds a **capture→present**
|
||||
(glass-to-glass) line via explicit decode + a Metal/display-link present. Settings also picks the HOST
|
||||
compositor (KWin/wlroots/Mutter/gamescope, default automatic — the host honors it
|
||||
only if that backend is available there) and has a **Controllers** section: every
|
||||
detected controller (capability glyphs, battery, "In use" badge), which one to forward
|
||||
("Use controller", default automatic), and the virtual pad type the host creates
|
||||
("Controller type": Automatic / Xbox 360 / DualSense — Automatic matches the physical
|
||||
pad; resolved at connect time, the host pad is fixed per session). Gamepad capture +
|
||||
feedback run with streaming (`SessionModel` owns them, same trust gate as audio).
|
||||
Settings also sets the **Bitrate** (Automatic toggle = host default; manual is a
|
||||
log-scale slider, 2 Mbps – 3 Gbps, snapped to two significant figures — above 1 Gbps
|
||||
an inline warning says to run a speed test first; tvOS uses a preset picker instead,
|
||||
Slider doesn't exist there; negotiated via the Hello on every connect), and a host
|
||||
card's context menu offers **"Test Network Speed…"** (`SpeedTestSheet`): connects, has
|
||||
the host burst probe filler over the real data plane (up to the host's 3 Gbps probe
|
||||
ceiling for 2 s, roadmap §9),
|
||||
shows measured goodput · loss · a recommended bitrate (≈70% of measured), and applies
|
||||
it in one tap. The streaming **statistics overlay** can be turned off and moved to any
|
||||
corner (Settings → Display → Statistics, `DefaultsKey.hudEnabled`/`hudPlacement`), and
|
||||
toggled live with **⌘⇧S** — a Scene-level **"Stream" menu** (`StreamCommands`) that also
|
||||
carries **Disconnect ⌘D**, so disconnect survives the HUD being hidden (on iOS a small
|
||||
exit chip appears instead; on tvOS the Siri-Remote Menu button still disconnects). The
|
||||
macOS Settings window is a **tabbed preferences pane** (General / Display / Audio /
|
||||
Controllers / Advanced) — the sections are shared with the iOS single-Form layout and the
|
||||
tvOS pushed-picker layout, defined once each.
|
||||
- **Tests** (`swift test`): byte-level Annex-B units; a real-codec round trip
|
||||
(VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → `AnnexB` →
|
||||
VTDecompressionSession → pixels); table-driven DualSense trigger-effect parsing
|
||||
(`DualSenseTriggerEffectTests`) and the gamepad wire conversions
|
||||
(`GamepadWireTests`); loopback integration against real local hosts
|
||||
(`test-loopback.sh` — stream round trip incl. gamepad/touchpad/motion sends, a
|
||||
host-scripted feedback burst asserted on the rumble + HID-output planes
|
||||
(`PUNKTFUNK_TEST_FEEDBACK=1`), the bitrate-negotiation echo and a real 20 Mbps
|
||||
bandwidth probe, plus the PIN pairing ceremony and the `--require-pairing` gate
|
||||
against a second, armed host); the remote first-light test above.
|
||||
Install from the App Store / TestFlight, or build from source below. Per-device install steps and the
|
||||
pairing walkthrough:
|
||||
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
|
||||
|
||||
## Build / run / test (on a Mac)
|
||||
|
||||
Requires Xcode 26.5 / Swift 6.3. First build the Rust core into an xcframework, then build the app:
|
||||
|
||||
```sh
|
||||
rustup target add aarch64-apple-darwin x86_64-apple-darwin
|
||||
bash scripts/build-xcframework.sh # → clients/apple/PunktfunkCore.xcframework
|
||||
# + BUILD_IOS=1 for the iOS slices (rustup target add aarch64-apple-ios{,-sim} x86_64-apple-ios)
|
||||
# + BUILD_TVOS=1 for tvOS — TIER-3 Rust targets, built from source:
|
||||
# rustup toolchain install nightly && rustup component add rust-src --toolchain nightly
|
||||
# BUILD_IOS=1 also builds the iOS slices (add the ios rustup targets)
|
||||
# BUILD_TVOS=1 also builds tvOS (tier-3 targets, built from source — see below)
|
||||
|
||||
cd clients/apple
|
||||
swift build && swift test # loopback/remote tests self-skip without a host
|
||||
swift run PunktfunkClient # the unbundled dev shell (CLI)
|
||||
open Punktfunk.xcodeproj # the real app: ⌘R builds + runs Punktfunk.app
|
||||
swift run PunktfunkClient # or the unbundled dev shell (CLI)
|
||||
swift build && swift test # unit + loopback/remote tests (self-skip w/o a host)
|
||||
```
|
||||
|
||||
bash test-loopback.sh # full loopback proof: builds punktfunk-host
|
||||
# (synthetic source — runs on macOS), streams
|
||||
# byte-verified frames into the Swift client
|
||||
tvOS slices are tier-3 Rust targets, built from source:
|
||||
`rustup toolchain install nightly && rustup component add rust-src --toolchain nightly`.
|
||||
|
||||
# against the real host (Linux box, see CLAUDE.md "Running on this box") — punktfunk1-host is a
|
||||
# persistent listener, reconnect at will:
|
||||
# PUNKTFUNK_COMPOSITOR=gamescope PUNKTFUNK_GAMESCOPE_APP=vkcube PUNKTFUNK_ZEROCOPY=1 \
|
||||
# cargo run -rp punktfunk-host -- punktfunk1-host --source virtual --seconds 60
|
||||
### Test against a host
|
||||
|
||||
```sh
|
||||
# full loopback proof — builds punktfunk-host (synthetic source, runs on macOS) and streams
|
||||
# byte-verified frames into the Swift client, incl. the PIN pairing ceremony:
|
||||
bash test-loopback.sh
|
||||
|
||||
# against a real Linux host on the LAN (see the repo README "Running on this box"):
|
||||
PUNKTFUNK_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless
|
||||
# (+ PUNKTFUNK_REMOTE_PORT / PUNKTFUNK_REMOTE_COMPOSITOR=gamescope|kwin|… /
|
||||
# PUNKTFUNK_REMOTE_PIN=<arming-pin> for the remote pairing test)
|
||||
PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkClient # on glass
|
||||
```
|
||||
|
||||
## Xcode project (`Punktfunk.xcodeproj`)
|
||||
## Project layout
|
||||
|
||||
The app target **Punktfunk** wraps the same sources as the `swift run` shell
|
||||
(`Sources/PunktfunkClient`, a synchronized folder — no duplication) plus `App/` (asset
|
||||
catalog) and links `PunktfunkKit` from the local package. Generated Info.plist, ad-hoc
|
||||
signing, bundle id `io.unom.punktfunk`. Notes:
|
||||
- **`PunktfunkKit`** (library) — the reusable pieces:
|
||||
- `PunktfunkConnection` — the wrapper over the C ABI (thread-safe `close()`, per-plane locks,
|
||||
pinning + TOFU).
|
||||
- `AnnexB` / `StreamView` / `VideoDecoder` / `MetalVideoPresenter` — format handling, the stage-1
|
||||
(`AVSampleBufferDisplayLayer`) and stage-2 (`VTDecompressionSession` → `CAMetalLayer`) presenters.
|
||||
- `InputCapture` — `GCMouse`/`GCKeyboard` → host VK/mouse, with fractional-delta accumulation.
|
||||
- `GamepadManager` / `GamepadCapture` / `GamepadFeedback` / `DualSenseTriggerEffect` — controller
|
||||
discovery + selection, capture (buttons/axes/touchpad/motion), and host-feedback rendering.
|
||||
- `HostDiscovery` — `NWBrowser` over `_punktfunk._udp`.
|
||||
- **`PunktfunkClient`** (the app) — hosts grid with an *On this network* section, add-host sheet,
|
||||
the two trust flows (TOFU prompt + SPAKE2 `PairSheet`), the stream view with the HUD, a
|
||||
tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed
|
||||
test. A Scene-level **Stream** menu carries Disconnect (⌘D) and the HUD toggle (⌘⇧S).
|
||||
- **Tests** (`swift test`) — Annex-B units, a real-codec VideoToolbox round trip, DualSense
|
||||
trigger-effect and gamepad-wire conversions, loopback integration against real local hosts, and the
|
||||
remote first-light test.
|
||||
|
||||
- **Entitlements (sandbox)**: the macOS target uses
|
||||
`Config/Punktfunk-macOS.entitlements`; iOS/tvOS use the shared
|
||||
`Config/Punktfunk.entitlements`. The macOS app is **App-Sandboxed** (mandatory for the Mac
|
||||
App Store/TestFlight, and used for the Developer ID DMG too so the local build matches what
|
||||
ships): `com.apple.security.app-sandbox`, `network.client` + **`network.server`** (the
|
||||
sandbox gates `bind()`; quinn + the raw-UDP plane both bind, so receive breaks without it),
|
||||
`device.audio-input` (mic), `device.bluetooth` + `device.usb` (GameController over BT/USB),
|
||||
and the existing `keychain-access-groups`. `app-sandbox` is macOS-only — keep it OUT of the
|
||||
shared iOS/tvOS file (it fails upload validation there). Verify a build is sandboxed with
|
||||
`codesign -d --entitlements :- <built .app>`. Heads-up: `device.usb` draws some App Review
|
||||
scrutiny — justify it in the review notes ("reads input from USB game controllers").
|
||||
- **App icon**: `App/Assets.xcassets` ships an empty `AppIcon` slot. For an Icon Composer
|
||||
`.icon`: add the file to the project (target Punktfunk), set it as the App Icon in the
|
||||
target's General tab, and delete the placeholder `AppIcon.appiconset`. Heads-up: CLI
|
||||
`actool` (Xcode 26.5) crashed compiling `punktfunk_Logo.icon` — if Xcode does the same,
|
||||
suspect the icon bundle (it has a duplicate-named layer, "…Layer-3 2.svg"), not the
|
||||
project.
|
||||
- **Tests from Xcode**: the package tests run with `swift test`; to get them on ⌘U, add
|
||||
`PunktfunkKitTests` once via Edit Scheme → Test → + (Xcode persists it into the shared
|
||||
scheme — a hand-written package-test reference doesn't resolve headlessly).
|
||||
- `xcodebuild -project Punktfunk.xcodeproj -scheme Punktfunk build` works headlessly;
|
||||
same for `-scheme Punktfunk-iOS -destination 'generic/platform=iOS Simulator'` (run it
|
||||
in a simulator via `xcrun simctl install/launch` — `SIMCTL_CHILD_PUNKTFUNK_AUTOCONNECT=…`
|
||||
passes the dev autoconnect env through).
|
||||
## Notes for contributors
|
||||
|
||||
## App Store screenshots
|
||||
- **Xcode project** (`Punktfunk.xcodeproj`) wraps the same sources as the `swift run` shell (a
|
||||
synchronized folder — no duplication). The macOS target is **App-Sandboxed** (needs
|
||||
`network.server` — the raw-UDP plane and quinn both `bind()`); iOS/tvOS use the shared
|
||||
entitlements file (keep `app-sandbox` **out** of it). Verify with
|
||||
`codesign -d --entitlements :- <built .app>`.
|
||||
- **Decode flow**: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band, and recovery
|
||||
keyframes re-send them — refresh the format description on every IDR; there is no out-of-band
|
||||
extradata, ever.
|
||||
- **ABI threading**: one video pump thread per connection, one optional audio drain thread, and one
|
||||
optional feedback drain thread (rumble + HID-output). `send()` is enqueue-only and safe alongside
|
||||
all of them. The wrapper's per-plane locks make `close()` safe from anywhere.
|
||||
- **DualSense motion scale** (`GamepadWire`) is derived from hid-playstation's math, not yet
|
||||
live-verified — if gyro/accel feel wrong in a game, correct sign/scale there and `evtest` the
|
||||
host's virtual pad.
|
||||
- **App Store screenshots** are automated — `tools/screenshots.sh all` renders the real UI at the
|
||||
required pixel sizes via a DEBUG-only shot mode; the `apple` CI workflow captures the iOS sizes on
|
||||
every main push. See the script header for details.
|
||||
- Deeper design notes live in `docs-site/content/docs/apple-stage2-presenter.md`.
|
||||
|
||||
Automated, faithful screenshots of the real UI for App Store Connect — one set per platform at
|
||||
exactly the accepted pixel sizes. Driver: **`tools/screenshots.sh`**.
|
||||
## Related
|
||||
|
||||
```sh
|
||||
tools/screenshots.sh all # macOS + (if full Xcode) iOS, iPadOS, tvOS → ./screenshots
|
||||
tools/screenshots.sh macos # just macOS
|
||||
OUT=~/Desktop/shots tools/screenshots.sh ios ipad tvos
|
||||
PUNKTFUNK_SHOT_HERO=~/frame.png tools/screenshots.sh ios # real captured frame behind the hero
|
||||
```
|
||||
|
||||
How it works: the app has a DEBUG-only **shot mode** (`Sources/PunktfunkClient/Screenshots/`).
|
||||
Launched with `PUNKTFUNK_SHOT_SCENE=<name>` it renders **one** mock-populated screen full-bleed
|
||||
(`ScreenshotHostView`) instead of `ContentView`, then the OS screenshots the *real, fully-rendered*
|
||||
window — `screencapture` on macOS, `xcrun simctl io booted screenshot` on the Simulators. The five
|
||||
scenes (`ShotScenes.all`): `01-stream` (the stream hero — a synthetic frame + the glass HUD, since
|
||||
`StreamView` needs a live connection), `02-hosts`, `03-pair`, `04-trust`, `05-settings`. Mock data
|
||||
is in `ShotMock`; nothing touches a host.
|
||||
|
||||
Output pixels are App Store Connect's required/largest sizes (Apple auto-derives the smaller ones):
|
||||
`mac` 2880×1800 · `iphone-6.9` 1320×2868 (hero 2868×1320) · `ipad-13` 2064×2752 (hero 2752×2064) ·
|
||||
`appletv` 1920×1080.
|
||||
|
||||
Why not `ImageRenderer` (the obvious offscreen route)? It can't rasterize this app's chrome —
|
||||
`NavigationStack`, `Form`/`TabView`, and Liquid-Glass/`NSVisualEffect` materials all render black or
|
||||
SwiftUI's "can't render" placeholder. Capturing the live window/Simulator avoids that entirely.
|
||||
|
||||
Requirements / gotchas:
|
||||
- **macOS**: only the Swift toolchain is needed, **plus a one-time Screen Recording grant** for
|
||||
your terminal (System Settings → Privacy & Security → Screen Recording) — without it
|
||||
`screencapture -l` fails with "could not create image from window". (A no-permission fallback,
|
||||
`PUNKTFUNK_SHOT_SELFCAPTURE=<dir>`, uses `cacheDisplay` — but it omits material blur and can't
|
||||
read `ScrollView` content, so it's for quick checks, not submission.)
|
||||
- **iOS/iPadOS/tvOS**: needs **full Xcode** (xcodebuild + Simulators), not just Command Line Tools,
|
||||
and the matching device Simulators installed (iPhone 16 Pro Max, iPad Pro 13", Apple TV). Run it
|
||||
on a full-Xcode Mac (e.g. the `macos-arm64` CI mini).
|
||||
- The hero defaults to a synthetic synthwave frame — set `PUNKTFUNK_SHOT_HERO` to a real captured
|
||||
frame for a production-quality lead screenshot.
|
||||
|
||||
**CI**: the `apple` workflow's **`screenshots`** job runs on the `macos-arm64` runner on every main
|
||||
push + manual dispatch (skipped on PRs), and attaches the result as a single zip artifact,
|
||||
**`punktfunk-appstore-screenshots`** (download it from the run's Artifacts; `upload-artifact@v3` —
|
||||
Gitea's backend rejects v4). It captures the two **required iOS sizes — iPhone 6.9" + iPad 13"** —
|
||||
on the Simulator (auto-creating the device if the runner lacks it), and is isolated from the
|
||||
build/test job so a capture hiccup never reds the build.
|
||||
|
||||
**macOS and tvOS are NOT in CI**, by design: the self-hosted runner is **headless** (no
|
||||
window-server session), so the macOS window capture can't run there, and tvOS needs the Tier-3
|
||||
build-std slice. Generate those on a GUI Mac: `tools/screenshots.sh macos tvos`. (If the runner is
|
||||
ever switched to a logged-in GUI session, re-adding macOS to the job's capture step is one line.)
|
||||
|
||||
## Notes for whoever picks this up next
|
||||
|
||||
1. **cbindgen import quirk** (the predicted "small compile fixes", now fixed): the
|
||||
C17-compatible header spells `PunktfunkStatus`/`PunktfunkInputKind` as integer typedefs while
|
||||
the enum *constants* import into Swift as a distinct same-named type — bridge with
|
||||
`.rawValue` (see the top of `PunktfunkConnection.swift`). Don't fight the generated header.
|
||||
2. **ABI contract**: one video pump thread per connection, plus optionally one *separate*
|
||||
audio drain thread for `nextAudio()` and one feedback drain thread for
|
||||
`nextRumble()`/`nextHidOutput()` (the core keeps per-plane borrow slots, so the planes
|
||||
never alias; rumble + HID-output are two planes drained sequentially by the one
|
||||
feedback thread); `send()` is enqueue-only and safe alongside all of them. The
|
||||
wrapper's per-plane locks make `close()` safe from anywhere (it waits out in-flight
|
||||
polls, ≤ their timeouts).
|
||||
3. **Decode flow**: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band
|
||||
and recovery keyframes re-send them — "refresh the format description on every IDR"
|
||||
(what `StreamView` does) is sufficient; there is no out-of-band extradata, ever.
|
||||
4. **Stage 2 — built, opt-in (`punktfunk.presenter == "stage2"`, default stage 1).** Explicit
|
||||
`VTDecompressionSession` decode (`VideoDecoder`) → a `CAMetalLayer` + display-link present
|
||||
(`MetalVideoPresenter`/`Stage2Pipeline`), hosted as a sublayer by the same `StreamView`s with
|
||||
input capture + HUD unchanged. It adds a **capture→present** (glass-to-glass, modulo the host
|
||||
render→capture term) HUD line, skew-corrected via `PunktfunkConnection.clockOffsetNs`. The
|
||||
decode half is unit-tested (`testVideoDecoderAsyncCallbackDeliversPixels`); the Metal present
|
||||
is display-bound — **validate live** (flip the Settings "Presenter" picker, watch the HUD
|
||||
number and that the image looks right) before making it the default. 10-bit/HDR + a smoothing
|
||||
pacer are later. Plan: `docs-site/content/docs/apple-stage2-presenter.md`.
|
||||
5. **Audio — wired, both directions.** Playback: `SessionAudio` drains `nextAudio()`
|
||||
on its own thread, decodes through CoreAudio's built-in Opus codec (`OpusCodec.swift`
|
||||
— kAudioFormatOpus, no bundled libopus; round-trip unit-tested) into a priming
|
||||
jitter ring feeding an `AVAudioSourceNode`. Mic: a second engine taps the input
|
||||
device, resamples to 48 kHz stereo, Opus-encodes 20 ms chunks and `sendMic()`s them
|
||||
(the host's virtual PipeWire source accepts any frame size ≤ 120 ms). Speaker/mic
|
||||
are chosen in Settings (`AudioDevices.swift` — persisted by UID; "System default"
|
||||
leaves the engines unpinned so they follow macOS device changes), mic on/off toggle
|
||||
included; the app asks for mic permission on first use
|
||||
(NSMicrophoneUsageDescription is in the Xcode target). A/V sync and packet-loss
|
||||
concealment beyond silence-fill are still open (AudioPacket.seq/ptsNs carry what's
|
||||
needed). Decode with libopus or `AVAudioConverter`/`kAudioFormatOpus` into an
|
||||
`AVAudioEngine` source node; conceal gaps (drop/dup) rather than blocking — the Rust
|
||||
side buffers 320 ms and drops the newest packet when the puller lags. Wall-clock
|
||||
`ptsNs` shares the host clock with video AUs for A/V sync. Wiring this into
|
||||
`PunktfunkClient` is the next app-side task.
|
||||
6. **Gamepads — wired end to end.** Exactly ONE controller (the `GamepadManager`
|
||||
selection) forwards as pad 0; the host accumulates the incremental events into a
|
||||
virtual pad whose TYPE the client negotiates in the Hello (`gamepad:` connect
|
||||
parameter, echoed resolved in `resolvedGamepad` — Automatic resolves from the physical
|
||||
pad at connect time; host precedence: explicit client choice > host `PUNKTFUNK_GAMEPAD`
|
||||
env > Xbox 360). A DualSense session carries the full feel: adaptive-trigger blocks
|
||||
(`DualSenseTriggerEffect.parse` — mode bytes per the community convention
|
||||
(Nielk1/ds5w/inputtino), total, unknown → `.off`), lightbar, player LEDs, touchpad,
|
||||
motion. **Motion scale constants** (`GamepadWire.gyroLSBPerRadS` = 20 LSB per deg/s,
|
||||
`accelLSBPerG` = 10000) are derived from hid-playstation's math over the host's fixed
|
||||
calibration blob, not yet live-verified — if gyro/accel feel wrong in a real game,
|
||||
correct sign/scale in `GamepadCapture.forwardMotion`/`GamepadWire` and `evtest` the
|
||||
host's virtual pad. Twin identical controllers share a fingerprint base, so a manual
|
||||
pin can swap between them across reconnects (documented in the Settings footer).
|
||||
7. **Trust — the full ceremony exists now (SPAKE2).** `generateIdentity()` once (persist
|
||||
both PEMs in the Keychain), then `pair(host:identity:pin:name:)` with the 4-digit PIN
|
||||
the host prints when it ARMS pairing (`--allow-pairing`/`--require-pairing`; one PIN
|
||||
per arming window, surfaced in the host's web console — port 3000 → Pairing — and
|
||||
printed at startup; the user reads it before pairing). Returns the
|
||||
host's VERIFIED fingerprint; persist it and pass `pinSHA256:` + `identity:` to every
|
||||
connect. Pairing is a real PAKE: a wrong PIN gets ONE online guess (no offline
|
||||
dictionary attack), throwing `.wrongPIN`; a wrong-size pin throws `.invalidPin`. `PunktfunkClient` implements both flows:
|
||||
the TOFU fingerprint sheet keeps working against hosts not running
|
||||
`--require-pairing`, and the PIN ceremony is wired in — `ClientIdentityStore`
|
||||
(Keychain) on every connect, `PairSheet` from a host card's context menu or the trust
|
||||
prompt's "Pair with PIN instead…" (the host's accept loop is sequential, so that path
|
||||
drops the live session before pairing). With `--require-pairing` the host now
|
||||
authorizes clients too (the "other direction" is no longer open, opt-in per host);
|
||||
the whole gate is regression-tested in `testPairingCeremonyAndRequirePairingGate`.
|
||||
7b. **Resize without reconnect**: `requestMode(width:height:refreshHz:)` mid-stream —
|
||||
the host rebuilds at the new mode in ~90 ms; the first new-mode AU is an IDR with
|
||||
fresh parameter sets (the refresh-on-IDR decode flow handles it untouched) and
|
||||
`currentMode()` reflects the switch. Wire it to window-resize events.
|
||||
8. **Input capture** (stage 1): capture is a deliberate, reversible STATE owned by
|
||||
`StreamLayerView`, Moonlight-style. Engaged when the stream starts / trust is
|
||||
confirmed and when the user clicks into the video (that click is suppressed toward
|
||||
the host); released by ⌘⎋ (toggles) or focus loss; NEVER engaged by mere app
|
||||
activation — activating clicks may be title-bar drags or resizes, which used to get
|
||||
their cursor warped away mid-drag. While captured: the local cursor is hidden +
|
||||
frozen mid-view (the host renders its own), all input is forwarded, and the view
|
||||
consumes key events as first responder so unhandled keyDowns don't beep — ⌘-combos
|
||||
still work locally (⌘D disconnect, ⌘Q) *and* reach the host via GC. While released:
|
||||
nothing is forwarded (`InputCapture.forwarding` gates the GC handlers; held
|
||||
keys/buttons are flushed host-side on release so nothing sticks down), the cursor is
|
||||
free, and the HUD shows "Click the stream to capture input". GC handlers only fire
|
||||
while the app has focus, and focus loss also auto-releases everything held. One live capture per process (the GC
|
||||
mouse/keyboard singletons have a single handler slot — ownership is tracked so a stale
|
||||
capture's stop() can't clobber a newer one).
|
||||
9. **iOS/iPadOS — ported and first-lit** (iPad simulator ↔ the real host, 60 fps).
|
||||
`BUILD_IOS=1 bash scripts/build-xcframework.sh` builds device + universal-simulator
|
||||
slices; the Xcode project has a second target, **Punktfunk-iOS**, sharing the same
|
||||
synchronized sources. The iOS `StreamView` (StreamViewIOS.swift — same name/signature
|
||||
as the macOS one, so the SwiftUI shell is identical) hosts the shared `StreamPump` in
|
||||
a view controller for `prefersPointerLocked`: with a hardware mouse/trackpad that is
|
||||
the iPadOS cursor capture (system honors it fullscreen-and-frontmost; in Stage
|
||||
Manager it degrades to absolute-mouse forwarding). Input is routed by kind: DIRECT
|
||||
fingers / Pencil are touches (each gets a wire touch id, coordinates mapped through the
|
||||
aspect-fit letterbox into host-mode pixels — surface == host mode, so the host rescale is
|
||||
the identity), while a mouse/trackpad is a MOUSE — pointer-LOCKED it is GCMouse relative
|
||||
deltas; unlocked it is absolute moves + buttons + scroll over the UIKit pointer path
|
||||
(hover + `.indirectPointer` touches), the local cursor staying visible so you can aim. An
|
||||
indirect pointer is never sent as a touch. Touch is gated on trust (not forwarded under
|
||||
the TOFU prompt), and returning to the foreground restores the capture you had on leaving.
|
||||
`InputCapture` is cross-platform (GC works the same on iPadOS; ⌘⎋ is detected from
|
||||
the HID stream there); audio routes via `AVAudioSession` (the Settings device
|
||||
pickers are macOS-only). For the iPad-with-external-display setup: the target
|
||||
enables multiple scenes + indirect input events — on Stage Manager iPads, drag the
|
||||
punktfunk window onto the external screen and the stream runs there with full
|
||||
keyboard/mouse/touch. While streaming the session is immersive (edge-to-edge,
|
||||
status bar + home indicator hidden) and the iPadOS cursor is hidden over the video only
|
||||
while the scene is actually pointer-LOCKED (`UIPointerInteraction` `.hidden()`); when the
|
||||
lock isn't held it stays visible and the mouse forwards as an absolute cursor instead; on
|
||||
iOS first run the stream mode defaults to the device's native screen so the video
|
||||
fills the display. **tvOS** runs the same app (target **Punktfunk-tvOS**, first-lit
|
||||
in the Apple TV simulator at 720p60): playback-only audio (no mic on tvOS),
|
||||
focus-driven UI (`.card` host tiles), no kb/mouse capture yet — input lands with
|
||||
gamepad support, the natural tvOS input anyway. While streaming there is NO focusable
|
||||
control (a focusable Disconnect button would let the focus engine eat the controller's A
|
||||
before the host sees it); the Siri Remote's **Menu** button disconnects (`.onExitCommand`).
|
||||
Core slices are tier-3 Rust targets (see Build above). Known gaps: true pointer LOCK (`prefersPointerLocked`) isn't
|
||||
consulted through UIHostingController, so the hidden cursor can still drift onto a
|
||||
second screen (fixing it means putting the controller into the UIKit presentation
|
||||
chain); and
|
||||
AVAudioSession interruptions (calls, Siri) don't auto-restart the audio engines yet
|
||||
(reconnect recovers).
|
||||
|
||||
## Known limitations of the current host (relevant to client UX)
|
||||
|
||||
- One session **at a time** (the listener is persistent, but a second concurrent client
|
||||
waits in the accept queue until the current session ends — the virtual output and
|
||||
encoder are single-tenant).
|
||||
- Mid-stream renegotiation (resolution change without reconnect) is designed-for but not
|
||||
implemented (the Welcome is one-shot today).
|
||||
- Host-side gamepad injection needs `/dev/uinput` access on the box (udev rule from
|
||||
`design/linux-setup.md`).
|
||||
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
|
||||
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
|
||||
|
||||
+55
-160
@@ -1,189 +1,84 @@
|
||||
# punktfunk Decky plugin (SteamOS / Steam Deck)
|
||||
# punktfunk — Steam Deck plugin (Decky)
|
||||
|
||||
A **[Decky Loader](https://decky.xyz/)** plugin that adds a **punktfunk** panel to the Steam
|
||||
Deck's Quick Access Menu (the QAM, opened with the `…` button), so you can launch the
|
||||
punktfunk streaming client from **Gaming Mode** without dropping to the desktop.
|
||||
Stream to your **Steam Deck** without ever leaving Gaming Mode. This
|
||||
**[Decky Loader](https://decky.xyz/)** plugin adds a **punktfunk** panel to the Quick Access Menu
|
||||
(the `…` button): discover hosts on your network, pair with a PIN, tweak stream settings, and launch
|
||||
a fullscreen, gamescope-focused stream — all from the couch, gamepad-navigable.
|
||||
|
||||
Because Decky plugins run inside Steam's CEF, the panel is built from real Steam UI
|
||||
primitives (`@decky/ui`: `PanelSection`, `PanelSectionRow`, `ButtonItem`, `Field`,
|
||||
`Spinner`) — so it looks and feels native to Gaming Mode.
|
||||
|
||||
> **Full Gaming-Mode client.** Discovery, a fullscreen page, in-UI SPAKE2 PIN pairing,
|
||||
> stream settings, and a stream that actually launches fullscreen under gamescope (via a
|
||||
> Steam shortcut, MoonDeck-style). The video itself is the existing GTK4 flatpak client
|
||||
> (`io.unom.Punktfunk`) — the plugin discovers, pairs, configures, and *launches it the
|
||||
> right way* so gamescope focuses it. The Steam-shortcut launch + pairing need a real Deck
|
||||
> in Gaming Mode to fully confirm.
|
||||
The video itself is the native GTK4 Linux client (the `io.unom.Punktfunk` flatpak); the plugin
|
||||
discovers, pairs, configures, and *launches it the right way* so gamescope fullscreens it — the same
|
||||
Steam-shortcut trick MoonDeck uses. Because it's built from real Steam UI primitives (`@decky/ui`),
|
||||
the panel looks and feels native to Gaming Mode.
|
||||
|
||||
## What it does
|
||||
|
||||
1. **Discover** — browses the LAN over mDNS for punktfunk/1 hosts (`_punktfunk._udp`,
|
||||
backend `discover()` via `avahi-browse`). Shown in both the QAM panel and a **fullscreen
|
||||
page** (Decky route `/punktfunk`, via `routerHook.addRoute`).
|
||||
2. **Pair** — for a `pair=required` host: a gamepad-navigable PIN keypad. The operator arms
|
||||
pairing on the host (it shows a 4-digit PIN), the user enters it on the Deck, and the
|
||||
backend runs the SPAKE2 ceremony headlessly via the flatpak client's `--pair` mode
|
||||
(`pair()`), persisting the host as paired so the stream then connects silently.
|
||||
3. **Stream** — launches fullscreen in Gaming Mode. The plugin registers ONE hidden
|
||||
non-Steam shortcut pointing at `bin/punktfunkrun.sh`, passes `PF_HOST` as the shortcut's
|
||||
Steam launch options, and starts it with `SteamClient.Apps.RunGame` — so gamescope
|
||||
focuses + fullscreens it. (A flatpak launched directly from the backend is invisible:
|
||||
gamescope only focuses the process tree Steam launched via `reaper` — gamescope#484.)
|
||||
The wrapper then execs `flatpak run io.unom.Punktfunk --connect <host>`.
|
||||
4. **Settings** — resolution / refresh / bitrate / gamepad / mic, written to the client's
|
||||
`client-gtk-settings.json` (`get_settings`/`set_settings`), which the launched client reads.
|
||||
1. **Discover** — browses the LAN over mDNS for punktfunk hosts, in both the QAM panel and a
|
||||
fullscreen page.
|
||||
2. **Pair** — for a host that requires it, a gamepad-navigable PIN keypad runs the SPAKE2 pairing
|
||||
ceremony headlessly, then remembers the host so future streams connect silently.
|
||||
3. **Stream** — launches fullscreen via a hidden Steam shortcut so gamescope focuses it.
|
||||
4. **Settings** — resolution / refresh / bitrate / gamepad / mic, written to the client's config.
|
||||
|
||||
To leave the stream: the in-client controller chord (**L1+R1+Start+Select**) or close the
|
||||
"game" from the Steam overlay — exiting the client ends the Steam game and returns to
|
||||
Gaming Mode automatically.
|
||||
|
||||
## Architecture
|
||||
|
||||
| File | Role |
|
||||
| --- | --- |
|
||||
| `src/index.tsx` | Frontend: QAM panel + the `/punktfunk` fullscreen page (host list, PIN keypad modal, settings). |
|
||||
| `src/steam.ts` | Steam-shortcut launch (`AddShortcut` / `SetAppLaunchOptions` / `RunGame`) — the focus-correct stream start. |
|
||||
| `src/backend.ts` | Typed `callable` bridges to `main.py`. |
|
||||
| `bin/punktfunkrun.sh` | The launch wrapper the Steam shortcut targets (so the window is focusable). |
|
||||
| `main.py` | Backend: `discover` / `pair` / `runner_info` / `get_settings` / `set_settings` / `kill_stream` / `check_update`. |
|
||||
| `plugin.json` | Decky plugin manifest. |
|
||||
| `update.json` | CI-baked `{channel, manifest}` — where `check_update()` polls (absent on dev builds). |
|
||||
| `decky.pyi` | Type stub for the injected `decky` module (vendored from the template). |
|
||||
|
||||
### Discovery (`discover()`)
|
||||
|
||||
Shells out to **`avahi-browse -rpt _punktfunk._udp`** (SteamOS and Bazzite ship
|
||||
`avahi-daemon`; this avoids bundling python-zeroconf):
|
||||
|
||||
- `-r` resolve services, `-p` parseable output, `-t` terminate after the cache dump.
|
||||
- Resolved records start with `=` and are semicolon-separated:
|
||||
`=;iface;protocol;name;type;domain;hostname;address;port;txt`.
|
||||
- The `txt` column is space-separated, quoted `"key=value"` tokens. We read the keys the
|
||||
host advertises (`crates/punktfunk-host/src/discovery.rs`): `proto`, `fp`, `pair`, `id`.
|
||||
- Records are deduped on the `id` TXT key (a host re-advertises per interface and across
|
||||
IPv4/IPv6), preferring the IPv4 address for the user-facing host string.
|
||||
|
||||
### Client launch (`connect()`)
|
||||
|
||||
The client binary `punktfunk-client` is resolved in order: `PATH` → `/usr/bin` →
|
||||
`/usr/local/bin` → `~/.local/bin` → a `flatpak run io.unom.Punktfunk` fallback. The resolved
|
||||
argv and a clear `client-not-found` error surface to the UI. The child PID is tracked so
|
||||
`disconnect()` (and plugin `_unload`) can terminate it.
|
||||
|
||||
> On the **Steam Deck** the client install is the flatpak `io.unom.Punktfunk`
|
||||
> (`packaging/flatpak/`) — SteamOS `/usr` is read-only and lacks `libadwaita`/`libSDL3`, so
|
||||
> the flatpak (which bundles them) is the canonical path; the resolver's flatpak fallback
|
||||
> launches exactly that.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Decky Loader** installed on the Deck (https://decky.xyz/).
|
||||
- **`punktfunk-client`** (the GTK4/libadwaita Linux client, crate `punktfunk-client-linux`)
|
||||
installed and runnable on the Deck — via `.deb`/RPM/flatpak, or symlinked into
|
||||
`~/.local/bin`.
|
||||
- **avahi** (`avahi-daemon` + `avahi-browse`) for discovery — present on SteamOS/Bazzite.
|
||||
- A punktfunk/1 host on the LAN (`punktfunk-host serve` or `punktfunk1-host`).
|
||||
|
||||
## Build
|
||||
|
||||
```sh
|
||||
pnpm install
|
||||
pnpm build # rollup → dist/index.js
|
||||
```
|
||||
|
||||
(`npm install && npm run build` also works.)
|
||||
To leave a stream: the in-client controller chord (**L1 + R1 + Start + Select**), or close the
|
||||
"game" from the Steam overlay — either returns you to Gaming Mode.
|
||||
|
||||
## Install on the Deck
|
||||
|
||||
### Option A — Decky "install from URL" (recommended; published by CI)
|
||||
You need **[Decky Loader](https://decky.xyz/)** and the **`io.unom.Punktfunk` flatpak**
|
||||
([`packaging/flatpak`](../../packaging/flatpak/README.md)) installed on the Deck — SteamOS `/usr` is
|
||||
read-only, so the flatpak (which bundles libadwaita/SDL3) is the canonical client. Discovery uses
|
||||
`avahi-browse`, which ships on SteamOS/Bazzite.
|
||||
|
||||
CI (`.gitea/workflows/decky.yml`) builds the plugin into a store-layout zip and publishes it to
|
||||
Gitea's **generic package registry** on every push to `main` and on `v*` tags, exposing a stable
|
||||
URL. In Decky's settings → **Developer Mode** → **Install Plugin from URL**, paste:
|
||||
**Recommended — install from URL** (published by CI): in Decky → Settings → **Developer Mode** →
|
||||
**Install Plugin from URL**, paste:
|
||||
|
||||
```
|
||||
https://git.unom.io/api/packages/unom/generic/punktfunk-decky/latest/punktfunk.zip
|
||||
```
|
||||
|
||||
(or a pinned version: `.../punktfunk-decky/<version>/punktfunk.zip`). On tags the same zip is
|
||||
also attached to the Gitea release. The zip's layout is the store-required one — a single
|
||||
top-level `punktfunk/` dir holding `plugin.json`, `package.json`, `main.py`, `dist/index.js`,
|
||||
`README.md`, and `LICENSE`.
|
||||
(or a pinned `.../punktfunk-decky/<version>/punktfunk.zip`). The plugin then **self-updates** without
|
||||
the Decky store — when a newer build exists, an **Update to vX** button appears and drives Decky
|
||||
Loader's own (SHA-256-verified) install.
|
||||
|
||||
### Option B — manual dev copy (sideload)
|
||||
|
||||
Decky's `~/homebrew/plugins/` is **root-owned** (PluginLoader runs as root and manages it), so a
|
||||
plain `rsync` into it fails — stage to a writable temp dir, then `sudo`-install and restart the
|
||||
loader. The two helper scripts do exactly this:
|
||||
## Build & sideload (development)
|
||||
|
||||
```sh
|
||||
cd clients/decky
|
||||
pnpm install
|
||||
pnpm build # rollup → dist/index.js
|
||||
pnpm run package # → out/punktfunk/ + out/punktfunk-v<ver>.zip
|
||||
DECK=deck@<deck-ip> pnpm run deploy # rsync → /tmp, sudo cp into plugins/, chown root, restart
|
||||
DECK=deck@<deck-ip> pnpm run deploy # rsync → /tmp, sudo-install into the root-owned plugins dir, restart loader
|
||||
```
|
||||
|
||||
`deploy.sh` prompts for the Deck's sudo password interactively (via `ssh -t`); set `DECKPASS=…`
|
||||
to run it non-interactively. Equivalent by hand:
|
||||
`~/homebrew/plugins/` is root-owned (the loader runs as root), so `deploy.sh` stages to a temp dir
|
||||
then `sudo`-installs and restarts the loader — set `DECKPASS=…` to run it non-interactively. A loader
|
||||
restart is required for an out-of-band install to appear.
|
||||
|
||||
```sh
|
||||
cd clients/decky && pnpm build && bash scripts/package.sh
|
||||
rsync -azp --delete out/punktfunk/ deck@<deck-ip>:/tmp/punktfunk/
|
||||
ssh -t deck@<deck-ip> 'sudo sh -c "rm -rf ~deck/homebrew/plugins/punktfunk && \
|
||||
cp -r /tmp/punktfunk ~deck/homebrew/plugins/punktfunk && \
|
||||
chown -R root:root ~deck/homebrew/plugins/punktfunk && systemctl restart plugin_loader"'
|
||||
```
|
||||
## Architecture
|
||||
|
||||
A loader restart is required for an out-of-band install to appear. The **punktfunk** panel then
|
||||
shows up in the Quick Access Menu.
|
||||
| File | Role |
|
||||
| --- | --- |
|
||||
| `src/index.tsx` | Frontend: QAM panel + the `/punktfunk` fullscreen page (host list, PIN keypad, settings). |
|
||||
| `src/steam.ts` | Steam-shortcut launch (`AddShortcut` / `SetAppLaunchOptions` / `RunGame`) — the focus-correct stream start. |
|
||||
| `src/backend.ts` | Typed `callable` bridges to `main.py`. |
|
||||
| `bin/punktfunkrun.sh` | The launch wrapper the Steam shortcut targets (so the window is focusable). |
|
||||
| `main.py` | Backend: `discover` (via `avahi-browse`) / `pair` / settings / `kill_stream` / `check_update`. |
|
||||
| `plugin.json` · `update.json` | Decky manifest; CI-baked update channel. |
|
||||
|
||||
> The plugin launches the client via the flatpak `io.unom.Punktfunk` (see
|
||||
> [`../../packaging/flatpak/README.md`](../../packaging/flatpak/README.md)) — install that on
|
||||
> the Deck too, or the panel's Connect surfaces a `client-not-found` error.
|
||||
|
||||
## Updating (self-update, no store)
|
||||
|
||||
The plugin updates itself without the official Decky store. CI (`decky.yml`) publishes a tiny
|
||||
per-channel `manifest.json` next to the zip in the Gitea registry:
|
||||
|
||||
```json
|
||||
{"version":"0.3.123","artifact":".../punktfunk-decky/0.3.123/punktfunk.zip","sha256":"…"}
|
||||
```
|
||||
|
||||
and bakes an `update.json` (`{channel, manifest}`) into the plugin so it knows which channel it was
|
||||
installed from. The backend `check_update()` reads the **installed** version from `package.json` —
|
||||
the value Decky itself reports (it does **not** read `plugin.json`) — fetches the channel manifest,
|
||||
and compares. When a newer build exists the frontend shows an **Update to vX** button that drives
|
||||
Decky Loader's own install RPC:
|
||||
|
||||
```ts
|
||||
window.DeckyBackend.callable("utilities/install_plugin")(artifact, "punktfunk", version, hash, /*UPDATE=*/2)
|
||||
```
|
||||
|
||||
The loader (root) downloads the immutable per-version zip, **SHA-256-verifies** it against `hash`,
|
||||
replaces `~/homebrew/plugins/punktfunk`, and hot-reloads — the unprivileged backend never writes the
|
||||
root-owned plugins dir itself. `window.DeckyBackend` / `utilities/install_plugin` are loader
|
||||
internals (not `@decky/api`), so every access is guarded; missing them, the button falls back to a
|
||||
toast pointing at **Install Plugin from URL**.
|
||||
|
||||
> CI stamps a **plain numeric** semver per channel (`0.3.<run>` canary, `X.Y.Z` stable) into
|
||||
> `package.json`. Decky's `compare-versions` orders pre-release identifiers lexically (so `ci10 < ci9`)
|
||||
> — a `-ciN` suffix would mis-detect updates.
|
||||
|
||||
**Optional — native Updates tab:** Decky's store is single-source (a custom store URL *replaces* the
|
||||
official catalog), so punktfunk doesn't ship one by default. A user who wants the native update badge
|
||||
can point Decky → Settings → **Custom store** at a punktfunk-only store JSON — not recommended if you
|
||||
use other plugins, since it hides the official catalog.
|
||||
The client binary is resolved `PATH` → `/usr/bin` → `/usr/local/bin` → `~/.local/bin` → a
|
||||
`flatpak run io.unom.Punktfunk` fallback, so the flatpak install always works.
|
||||
|
||||
## Limitations / next steps
|
||||
|
||||
- **Needs on-Deck validation in Gaming Mode**: the Steam-shortcut launch (`AddShortcut` /
|
||||
`RunGame` / the `gameId` encoding) and the headless pairing env are coded to MoonDeck's
|
||||
proven pattern but verified only at build time here.
|
||||
- mDNS discovery depends on `avahi-browse`; no manual "add host by IP" entry yet.
|
||||
- No in-stream overlay (latency/bitrate HUD) inside the plugin — the client owns the session
|
||||
once launched; leave it with the L1+R1+Start+Select chord.
|
||||
- Pairing requires the operator to **arm pairing on the host** (so it shows the PIN); the
|
||||
plugin can't arm it remotely (no host mgmt token on the Deck).
|
||||
- Settings are written to the flatpak's sandbox config path; if the client ever moves its
|
||||
config location, that path mapping must follow.
|
||||
- **Needs on-Deck validation in Gaming Mode** — the Steam-shortcut launch and headless pairing follow
|
||||
MoonDeck's proven pattern but are verified only at build time here.
|
||||
- No manual "add host by IP" entry yet (discovery is mDNS-only).
|
||||
- No in-stream overlay inside the plugin — the client owns the session once launched.
|
||||
- Pairing needs the operator to **arm pairing on the host** so it shows the PIN; the plugin can't arm
|
||||
it remotely.
|
||||
|
||||
## Related
|
||||
|
||||
- **[Documentation](https://docs.punktfunk.unom.io/docs/steam-deck)** — Steam Deck setup guide
|
||||
- **[Linux client](../linux/README.md)** — the app this plugin launches
|
||||
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
# punktfunk — Linux client
|
||||
|
||||
The native **Linux** app for streaming a punktfunk host to your desktop, laptop, or Steam Deck.
|
||||
It's a clean GTK4/libadwaita app that finds hosts on your network, pairs with a PIN, and puts a
|
||||
low-latency stream on glass at your display's own resolution and refresh rate.
|
||||
|
||||
Built in Rust, it links the shared **`punktfunk-core`** directly (no C ABI) and speaks the fast
|
||||
**`punktfunk/1`** protocol — QUIC control plane, GF(2¹⁶) FEC + AES-GCM data plane.
|
||||
|
||||
## Features
|
||||
|
||||
- **Zero-copy hardware decode** — FFmpeg VAAPI decode → DRM-PRIME dmabuf → `GdkDmabufTexture`
|
||||
(Tier-1 zero-copy on Intel and AMD), with an automatic software-HEVC fallback on NVIDIA or when
|
||||
VAAPI is unavailable.
|
||||
- **Your display's native mode** — the host builds a virtual output at exactly your WxH@Hz; no
|
||||
scaling, no letterboxing. Steady 60 fps at 1080p60, ~6 ms capture→decoded on the LAN.
|
||||
- **Audio both ways** — PipeWire playback with a jitter ring, plus mic uplink to the host.
|
||||
- **Full controller support** — SDL3 gamepads with rumble and DualSense fidelity (lightbar, player
|
||||
LEDs, touchpad, motion, adaptive-trigger replay). Click-to-capture keyboard and mouse, with a
|
||||
release chord (Ctrl+Alt+Shift+Q) and focus-loss release.
|
||||
- **Find hosts automatically** — mDNS discovery lists hosts on your LAN; saved hosts persist.
|
||||
First connect does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then reconnects on
|
||||
a pinned identity.
|
||||
- **Per-host speed test** to pick a bitrate, plus compositor and mode preferences in Settings.
|
||||
|
||||
## Get it
|
||||
|
||||
Most people should install a package rather than build from source:
|
||||
|
||||
| Distro | Install |
|
||||
|--------|---------|
|
||||
| **Flatpak** (any distro, Steam Deck) | `io.unom.Punktfunk` — see [`packaging/flatpak`](../../packaging/flatpak/README.md) |
|
||||
| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-client` *(after adding the repo)* |
|
||||
| **Fedora / Bazzite** (rpm) | `rpm-ostree install punktfunk-client` |
|
||||
| **Arch** (PKGBUILD) | see [`packaging/arch`](../../packaging/arch/README.md) |
|
||||
|
||||
Per-device install steps and pairing walkthrough:
|
||||
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
|
||||
|
||||
## Build & run from source
|
||||
|
||||
Requires GTK ≥ 4.16, libadwaita ≥ 1.5, FFmpeg 7 or 8 (with VAAPI for hardware decode), PipeWire,
|
||||
and SDL3 (with hidapi) development packages.
|
||||
|
||||
```sh
|
||||
# from the repo root
|
||||
cargo run -p punktfunk-client-linux # launch the app
|
||||
cargo run -p punktfunk-client-linux -- --discover # list hosts on the LAN, then exit
|
||||
cargo run -p punktfunk-client-linux -- --connect HOST[:PORT] # skip the host list and connect
|
||||
```
|
||||
|
||||
The binary is named **`punktfunk-client`**. Handy flags: `--connect host[:port]` (start a session
|
||||
immediately — for scripting and the Steam Deck launcher), `--discover [secs]`, and
|
||||
`--pair <PIN> --connect host[:port]` (run the pairing ceremony headlessly). Force a decoder with
|
||||
`PUNKTFUNK_DECODER=software|vaapi`.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
src/
|
||||
main.rs · app.rs entry point, GTK application, CLI paths
|
||||
ui_hosts.rs host list (mDNS + saved), pairing / trust dialogs
|
||||
ui_settings.rs resolution · refresh · decoder · bitrate · compositor · mic
|
||||
ui_stream.rs the stream window (GtkGraphicsOffload present) + input capture
|
||||
session.rs session lifecycle over the NativeClient connector
|
||||
video.rs FFmpeg VAAPI / software decode → dmabuf / texture
|
||||
audio.rs PipeWire playback + mic uplink
|
||||
gamepad.rs · keymap.rs SDL3 controllers + feedback; keyboard VK mapping
|
||||
trust.rs · discovery.rs persistent identity, TOFU/PIN pairing, mDNS browse
|
||||
tools/screenshots.sh store screenshot capture
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
|
||||
- **[Steam Deck plugin](../decky/README.md)** — launches this client fullscreen in Gaming Mode
|
||||
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
|
||||
@@ -0,0 +1,50 @@
|
||||
# punktfunk — probe (reference client)
|
||||
|
||||
`punktfunk-probe` is the **headless reference client** for the `punktfunk/1` protocol — a
|
||||
command-line tool for testing, latency measurement, and validating host behavior. It's not a
|
||||
streaming app you'd watch on; it connects, exercises a plane, and reports numbers. If you want to
|
||||
actually stream, use the [Linux](../linux/README.md), [Windows](../windows/README.md),
|
||||
[Apple](../apple/README.md), or [Android](../android/README.md) clients.
|
||||
|
||||
Because it links the same **`punktfunk-core`** as every other client, it's also the canonical
|
||||
example of driving the protocol end to end: QUIC control plane, UDP data plane, and the side planes
|
||||
(input, audio, rumble) over QUIC datagrams.
|
||||
|
||||
## What it does
|
||||
|
||||
- **Receives a real stream**, writes a playable `.h265`, and reports per-frame
|
||||
**capture→…→reassembled latency** percentiles (the host stamps each frame with its capture clock).
|
||||
- **Verification mode** against a synthetic host — byte-checks deterministic test frames.
|
||||
- **Exercises every plane** with scripted test traffic:
|
||||
`--input-test` (mouse/keyboard), `--mic-test` (a 440 Hz Opus tone up to the host mic),
|
||||
`--touch-test` (a synthetic finger), `--rich-input-test` (DualSense touchpad + motion, logging the
|
||||
HID-output feedback that comes back).
|
||||
- **Trust** — `--pin <64-hex>` pins the host fingerprint; `--pair <PIN>` runs the SPAKE2 pairing
|
||||
ceremony and prints the verified fingerprint to pin from then on. Without a pin it trusts on first
|
||||
use.
|
||||
- **Discovery** — `--discover [secs]` browses the LAN for `_punktfunk._udp` hosts and prints each
|
||||
(name, addr:port, pairing requirement, cert fingerprint), then exits.
|
||||
- **Negotiation knobs** — `--mode WxHxFPS`, `--remode` (mid-stream mode change), `--bitrate`,
|
||||
`--audio-channels` (stereo / 5.1 / 7.1), `--compositor`, `--gamepad`, `--launch`, `--speed-test`.
|
||||
|
||||
## Usage
|
||||
|
||||
```sh
|
||||
# stream 720p120 from a host, save the video, and print latency percentiles:
|
||||
cargo run -p punktfunk-probe -- --mode 1280x720x120 --connect HOST:PORT --out /tmp/a.h265
|
||||
|
||||
# list hosts on the LAN:
|
||||
cargo run -p punktfunk-probe -- --discover
|
||||
|
||||
# pair with a host that requires it (read the PIN off the host), then stream:
|
||||
cargo run -p punktfunk-probe -- --connect HOST:PORT --pair 1234
|
||||
cargo run -p punktfunk-probe -- --connect HOST:PORT --pin <64-hex> --input-test
|
||||
```
|
||||
|
||||
Full flag reference is in the module doc-comment at the top of [`src/main.rs`](src/main.rs).
|
||||
|
||||
## Related
|
||||
|
||||
- **[Project README](../../README.md)** — the host, the streaming clients, and the protocol
|
||||
- **`punktfunk-host punktfunk1-host`** — the persistent native-protocol listener to probe against
|
||||
(see the "Running on this box" section of the repo README / `CLAUDE.md`)
|
||||
@@ -0,0 +1,71 @@
|
||||
# punktfunk — Windows client
|
||||
|
||||
The native **Windows** app for streaming a punktfunk host to your PC. A modern WinUI 3 app that
|
||||
discovers hosts on your network, pairs with a PIN, and streams at your display's own resolution and
|
||||
refresh rate — with a hardware-accelerated D3D11 video path and HDR.
|
||||
|
||||
It's **pure Rust**: the UI is WinUI 3 driven through [windows-reactor](https://github.com/microsoft/windows-rs)
|
||||
(a declarative, React-like framework), and it links the shared **`punktfunk-core`** directly to speak
|
||||
the fast **`punktfunk/1`** protocol.
|
||||
|
||||
## Features
|
||||
|
||||
- **Hardware decode, GPU present** — FFmpeg HEVC with a **D3D11VA zero-copy path** (decoder and
|
||||
presenter share one D3D11 device; NV12/P010 textures sampled straight into a `SwapChainPanel`
|
||||
composition swapchain), with a robust software-decode fallback.
|
||||
- **HDR10** — advertise 10-bit/HDR, detect PQ in-band, and flip the swapchain to `R10G10B10A2` +
|
||||
ST.2084 with HDR10 metadata.
|
||||
- **Your display's native mode** — the host builds a virtual display at exactly your WxH@Hz.
|
||||
- **Audio both ways** — WASAPI render + mic capture.
|
||||
- **Full controller support** — SDL3 gamepads with rumble, lightbar, and DualSense feedback.
|
||||
- **Find hosts automatically** — mDNS discovery lists hosts on your LAN, alongside saved and manual
|
||||
entries. First connect does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then
|
||||
reconnects on a pinned identity.
|
||||
- **Polished shell** — host cards, settings (resolution / refresh / decoder / bitrate / HDR / mic),
|
||||
a status-chip stream HUD, and the full trust surface. Stream input uses Win32 low-level hooks with
|
||||
a Ctrl+Alt+Shift+Q capture toggle.
|
||||
|
||||
Builds and ships for both **x64** and **ARM64** as a signed **MSIX**.
|
||||
|
||||
## Get it
|
||||
|
||||
Install the signed MSIX from the package registry — see
|
||||
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
|
||||
A stock [Moonlight](https://moonlight-stream.org/) client also works over GameStream if you prefer.
|
||||
|
||||
## Build from source
|
||||
|
||||
Windows-only (the crate builds as a stub on other platforms so the workspace stays green). You need
|
||||
the MSVC toolchain, an `FFMPEG_DIR` FFmpeg tree, and CMake (SDL3 builds from source). windows-reactor's
|
||||
`build.rs` downloads the Windows App SDK NuGets and needs `CARGO_WORKSPACE_DIR` set.
|
||||
|
||||
```sh
|
||||
cargo build -p punktfunk-client-windows --target x86_64-pc-windows-msvc
|
||||
|
||||
# CLI paths for testing (no window):
|
||||
punktfunk-client --discover # list hosts on the LAN
|
||||
punktfunk-client --headless --connect host[:port] [--pin HEX] # connect, count frames, print stats
|
||||
```
|
||||
|
||||
> `CARGO_HOME` must be an ASCII path — non-ASCII characters break SDL3's MSVC precompiled-header
|
||||
> build. Packaging (MSIX manifest, signing) lives in [`packaging/`](packaging/).
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
src/
|
||||
main.rs · app.rs entry point + CLI paths; WinUI 3 shell (windows-reactor)
|
||||
present.rs · gpu.rs SwapChainPanel D3D11 composition swapchain; shared D3D11 device
|
||||
video.rs FFmpeg HEVC decode (D3D11VA zero-copy + software fallback)
|
||||
audio.rs WASAPI render + mic capture
|
||||
gamepad.rs SDL3 controllers + rumble/lightbar/DualSense feedback
|
||||
input.rs Win32 low-level keyboard/mouse hooks → host input
|
||||
session.rs session lifecycle over the NativeClient connector
|
||||
trust.rs · discovery.rs persistent identity, TOFU/PIN pairing, mDNS browse
|
||||
packaging/ MSIX manifest, signing, pack script
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
|
||||
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
|
||||
@@ -0,0 +1,16 @@
|
||||
# pf-driver-proto
|
||||
|
||||
The shared **host ↔ driver binary contract** for punktfunk's Windows **pf-vdisplay** virtual display —
|
||||
the control IOCTLs and the IDD-push frame transport, defined exactly once.
|
||||
|
||||
It's a path dependency of **both** the host workspace ([`crates/punktfunk-host`](../punktfunk-host))
|
||||
and the out-of-workspace driver workspace ([`packaging/windows/drivers/`](../../packaging/windows)),
|
||||
so it must resolve identically from either build graph. That's why it's deliberately self-contained:
|
||||
`no_std` (+ alloc), platform-neutral (GUID/LUID are plain integers each side converts to its own OS
|
||||
type), and free of `*.workspace = true` inheritance.
|
||||
|
||||
Defining every wire struct here — with `const` size/offset asserts and `bytemuck` round-trips — turns
|
||||
host↔driver ABI drift into a **compile error** instead of a silent frame or IOCTL corruption.
|
||||
|
||||
See the crate root ([`src/`](src/)) for the wire types; the Windows virtual-display design is in
|
||||
[`design/windows-virtual-display-rust-port.md`](../../design/windows-virtual-display-rust-port.md).
|
||||
@@ -0,0 +1,61 @@
|
||||
# punktfunk-core
|
||||
|
||||
The **shared protocol core** — the one place where punktfunk's transport, forward error correction,
|
||||
and crypto live. It's linked into the [host](../punktfunk-host/README.md) and every native client, so
|
||||
there's exactly one implementation of the wire format everywhere.
|
||||
|
||||
Written in Rust with **no async on the per-frame path** (native threads only). It exposes both a
|
||||
normal Rust API and a **stable, versioned C ABI**, so the Swift and Kotlin clients — and any C
|
||||
embedder — link the same code as the Rust ones.
|
||||
|
||||
## What's in here
|
||||
|
||||
- **Transport & session** (`session.rs`, `transport/`, `packet.rs`) — the `punktfunk/1` data plane
|
||||
over raw UDP: packetization, reassembly (with attacker-bounded limits), pacing, and socket tuning.
|
||||
- **FEC** (`fec/`) — the wall-breaker. Two codes:
|
||||
- **GF(2⁸)** classic Reed–Solomon with the *Cauchy* generator matrix — byte-identical to the
|
||||
`nanors` library Moonlight uses, so our parity is decodable by a stock Moonlight client.
|
||||
- **GF(2¹⁶) Leopard-RS** (SIMD, O(n log n)) — up to 65535 shards/block, which removes the ~1 Gbps
|
||||
FEC ceiling. `punktfunk/1` negotiates this one.
|
||||
- **Crypto** (`crypto.rs`) — AES-128-GCM session encryption with per-direction nonce salts and
|
||||
sequence-as-AAD; SPAKE2 PIN pairing lives behind the `quic` feature.
|
||||
- **QUIC control plane** (`quic.rs`, `client.rs`, feature `quic`) — the Hello/Welcome/Start handshake,
|
||||
cert pinning/TOFU, reverse audio, and the embeddable `NativeClient` connector. This is the **only**
|
||||
place `tokio`/`quinn` are allowed; the feature is **off by default** so the core stays runtime-free.
|
||||
- **C ABI** (`abi.rs`) — the versioned surface (`punktfunk_abi_version()`, `PunktfunkConfig` carrying
|
||||
its own `struct_size`) that generates [`include/punktfunk_core.h`](../../include/punktfunk_core.h)
|
||||
via cbindgen at build time.
|
||||
|
||||
## Build outputs
|
||||
|
||||
The crate builds three ways at once (`crate-type = ["lib", "cdylib", "staticlib"]`):
|
||||
|
||||
| Output | Used by |
|
||||
|--------|---------|
|
||||
| `lib` (rlib) | the host, probe, and tools link it as a normal Rust crate |
|
||||
| `cdylib` (`.so`/`.dylib`) | the Swift / Kotlin clients via the C ABI |
|
||||
| `staticlib` (`.a`) | the C test harness and static embedding |
|
||||
|
||||
## Test
|
||||
|
||||
```sh
|
||||
cargo test -p punktfunk-core # unit + proptest + loopback
|
||||
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
|
||||
bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip proof
|
||||
```
|
||||
|
||||
## Design invariants (do not regress)
|
||||
|
||||
- **One core, linked everywhere** — protocol/FEC/crypto live only here, behind the stable C ABI.
|
||||
- **No async on the hot path** — the per-frame pipeline is native threads only; `quic` (tokio/quinn)
|
||||
is control-plane only, feature-gated, off by default.
|
||||
- **Security hardening stays intact** — the reassembler bounds attacker-controlled fields before
|
||||
allocating; AES-GCM keeps per-direction nonce salts + seq-as-AAD; the ABI checks `struct_size`.
|
||||
Regression tests exist — keep them green.
|
||||
|
||||
## Related
|
||||
|
||||
- **[`punktfunk-host`](../punktfunk-host/README.md)** — the streaming host built on this core
|
||||
- **[Clients](../../clients/)** — the apps that link this core over the C ABI (or directly, in Rust)
|
||||
- **[`design/implementation-plan.md`](../../design/implementation-plan.md)** — why GF(2¹⁶) FEC, the
|
||||
latency budget, and the architecture thesis
|
||||
@@ -0,0 +1,90 @@
|
||||
# punktfunk-host
|
||||
|
||||
The **streaming host** — the program you run on the machine whose desktop or games you want to
|
||||
stream. For each client that connects, it spins up a **virtual display sized to that device**,
|
||||
captures it on the GPU, encodes with hardware NVENC/VAAPI/AMF/QSV, and sends it out over a
|
||||
low-latency transport — no physical monitor, no letterboxing, no rearranging your real screens.
|
||||
|
||||
It speaks two protocols from **one process**:
|
||||
|
||||
- **GameStream** — so any [Moonlight](https://moonlight-stream.org/) / Artemis client works day one.
|
||||
- **`punktfunk/1`** — punktfunk's own faster protocol (QUIC control plane, GF(2¹⁶) FEC + AES-GCM data
|
||||
plane) that the native clients use.
|
||||
|
||||
Runs on **Linux** (the primary, most battle-tested path) and **Windows** (x64). The shared protocol,
|
||||
FEC, and crypto live in [`punktfunk-core`](../punktfunk-core/README.md); this crate is everything
|
||||
platform-facing around it.
|
||||
|
||||
## What it does
|
||||
|
||||
- **Per-client virtual displays at the exact WxH@Hz.** Linux uses per-compositor backends — **KWin**,
|
||||
**gamescope**, **Mutter**, and **Sway/wlroots**; Windows uses its own all-Rust IddCx virtual display,
|
||||
even on the secure desktop (UAC / lock screen).
|
||||
- **GPU zero-copy capture → encode.** dmabuf → CUDA/Vulkan → NVENC on Linux; DXGI/WGC → GPU encode on
|
||||
Windows. Encoders auto-select by GPU vendor: **NVENC** (NVIDIA), **VAAPI** (Linux AMD/Intel),
|
||||
**AMF/QSV** (Windows AMD/Intel), or software H.264 as a floor. HDR/10-bit and HEVC 4:4:4 supported.
|
||||
- **Input injection.** Mouse/keyboard (libei / gamescope EIS / wlr / Windows SendInput) and virtual
|
||||
**gamepads** — Xbox 360/One, DualSense, DualShock 4 — with rumble and HID feedback back-channels.
|
||||
- **Audio both ways.** Opus audio host→client, plus a virtual microphone the client can talk into.
|
||||
- **Trust & discovery.** A persistent host identity, **SPAKE2 PIN pairing** (default) or TOFU, and
|
||||
mDNS auto-advertisement so clients find the host without typing an IP.
|
||||
- **Management API + web console.** A REST API (`mgmt.rs`, OpenAPI at
|
||||
[`api/openapi.json`](../../api/openapi.json)) drives status, paired devices, and on-demand pairing;
|
||||
the browser UI is in [`web/`](../../web/README.md).
|
||||
|
||||
## Run it
|
||||
|
||||
`punktfunk-host serve` runs inside your desktop session. Bare `serve` is the **secure native-only
|
||||
default** (`punktfunk/1` + the management API); add `--gamestream` on a trusted LAN to also accept
|
||||
stock Moonlight clients.
|
||||
|
||||
```sh
|
||||
# Linux, from the repo root (see the repo README "Running on this box" for the headless recipe):
|
||||
cargo run -rp punktfunk-host -- serve # native-only (secure default)
|
||||
cargo run -rp punktfunk-host -- serve --gamestream # + Moonlight compatibility
|
||||
```
|
||||
|
||||
Then pair from the web console (`https://<host-ip>:3000`) or the client app.
|
||||
|
||||
Most people should install a **package** rather than run from source — see
|
||||
[`packaging/`](../../packaging/README.md) (apt · rpm/COPR/bootc · Arch/sysext · Windows installer) and
|
||||
the per-platform guides at **[docs.punktfunk.unom.io/docs/install](https://docs.punktfunk.unom.io/docs/install)**.
|
||||
|
||||
### Subcommands
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `serve` | The host (native `punktfunk/1` + mgmt API; `--gamestream` adds Moonlight). |
|
||||
| `punktfunk1-host` | Standalone native-protocol listener for testing/measurement (`--source virtual`, `--max-sessions`). |
|
||||
| `openapi` | Print the management-API OpenAPI spec (regenerates `api/openapi.json`). |
|
||||
| `library` | Inspect the multi-store game library. |
|
||||
| `service` · `driver` · `web` | Windows: SCM service, driver install, bundled web console. |
|
||||
| `*-test` / `*-selftest` / `*-probe` | Diagnostics (input, zero-copy, HDR, compositor, gamepads). |
|
||||
|
||||
`--help` lists them all.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
src/
|
||||
main.rs CLI + subcommand dispatch
|
||||
config.rs · session_plan.rs · session_tuning.rs · pipeline.rs session setup + the frame pipeline
|
||||
vdisplay/ per-compositor virtual outputs (kwin · gamescope · mutter · wlroots)
|
||||
capture/ · capture.rs screen/dmabuf capture (+ Windows DXGI/WGC)
|
||||
encode/ · encode.rs per-GPU encoders (nvenc · vaapi · ffmpeg_win (AMF/QSV) · sw)
|
||||
zerocopy/ dmabuf → CUDA → NVENC bridges (EGL/GL tiled, Vulkan LINEAR)
|
||||
inject/ · inject.rs input backends (libei · wlr · uinput gamepads · UHID DualSense/DS4)
|
||||
audio/ · audio.rs Opus out + virtual mic (PipeWire / WASAPI)
|
||||
gamestream/ Moonlight compat: nvhttp · pairing · rtsp · control · stream · gamepad · apps
|
||||
punktfunk1.rs the native punktfunk/1 host (QUIC control + native-thread UDP data plane)
|
||||
mgmt.rs · native_pairing.rs · stats_recorder.rs management API, pairing, perf capture
|
||||
hdr.rs · library.rs HDR metadata; multi-store game library
|
||||
linux/ · windows/ platform-confined backends
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- **[`punktfunk-core`](../punktfunk-core/README.md)** — the shared protocol · FEC · crypto core
|
||||
- **[Clients](../../clients/)** — the apps that connect (Apple · Linux · Windows · Android · probe)
|
||||
- **[Packaging](../../packaging/README.md)** & **[docs](https://docs.punktfunk.unom.io)** — install & operate
|
||||
- **[`design/`](../../design/README.md)** — architecture rationale and deep-dive plans
|
||||
+3
-3
@@ -3,9 +3,9 @@
|
||||
The punktfunk documentation site: [Fumadocs](https://fumadocs.dev) on
|
||||
[TanStack Start](https://tanstack.com/start) (Vite + Nitro/bun preset).
|
||||
|
||||
Content lives in [`content/docs/`](content/docs) as `.md`/`.mdx`. Several pages are imported
|
||||
verbatim from the repo's `docs/` design notes (with added frontmatter); edit those there or
|
||||
here as the docs site becomes the source of truth.
|
||||
Content lives in [`content/docs/`](content/docs) as `.md`/`.mdx`. This site is the source of truth
|
||||
for the **user-facing** guides; repo-internal design rationale lives in
|
||||
[`../design/`](../design/README.md).
|
||||
|
||||
## API reference
|
||||
|
||||
|
||||
+1
-1
@@ -73,7 +73,7 @@ rpm-ostree install punktfunk && systemctl reboot
|
||||
systemctl reboot
|
||||
```
|
||||
|
||||
## Option B — bootc (image-based, atomic)
|
||||
## Option C — bootc (image-based, atomic)
|
||||
|
||||
Layer punktfunk into a Bazzite image once, then rebase any number of hosts onto it — no
|
||||
per-host drift. See `bootc/Containerfile`:
|
||||
|
||||
@@ -1,26 +1,27 @@
|
||||
# pf-dualsense — virtual DualSense UMDF2 HID minidriver (M0 spike)
|
||||
# pf-dualsense — virtual DualSense UMDF2 HID minidriver
|
||||
|
||||
A self-authored **Rust UMDF2 HID minidriver** that presents a virtual Sony **DualSense**
|
||||
(VID `054C` / PID `0CE6`) to Windows, so games drive adaptive triggers / lightbar / rumble —
|
||||
capabilities ViGEm structurally cannot deliver. This is the M0 feasibility spike for rich
|
||||
controller support in the punktfunk Windows host.
|
||||
capabilities ViGEm structurally cannot deliver. It's how the punktfunk Windows host gives a client's
|
||||
DualSense a near-native feel with **no external gamepad dependencies** (no ViGEmBus).
|
||||
|
||||
## Status (2026-06-21)
|
||||
Shipping: the driver is one member of the in-tree driver workspace
|
||||
([`packaging/windows/drivers/`](../../README.md)), built from source in CI, and bundled +
|
||||
`pnputil`-installed by the Windows host [installer](../../README.md). The host feeds it over a shared
|
||||
memory channel from `crates/punktfunk-host/src/inject/dualsense_windows.rs`. The same UMDF driver also
|
||||
serves the **DualShock 4** identity per a `device_type` byte the host stamps.
|
||||
|
||||
**Load + recognition: DONE.** A self-signed build **loads under Secure Boot ON** and enumerates as a
|
||||
genuine DualSense HID game controller (`Status: OK`, VID `054C`, 273-byte DualSense report descriptor,
|
||||
PID `0CE6` via `GET_DEVICE_ATTRIBUTES`). Validated live on the RTX box (`192.168.1.173`, Win11 25H2).
|
||||
This README captures the driver-authoring lore — the bugs and the signing recipe that make a
|
||||
self-signed UMDF HID driver actually load. The authoritative build/sign/package flow (CI + Inno Setup)
|
||||
lives in the [Windows host packaging README](../../README.md).
|
||||
|
||||
**Remaining:** the real-game `0x02` adaptive-trigger gate (Cyberpunk 2077 on the interactive desktop →
|
||||
confirm `[pf-ds] *** OUTPUT ...` in the driver log), then wire into the host (M1+).
|
||||
## Build workspace
|
||||
|
||||
## This is a reference snapshot
|
||||
|
||||
The crate's `Cargo.toml` uses path-deps into `microsoft/windows-drivers-rs`
|
||||
(`../../crates/wdk{,-sys,-build}`), so it builds **inside a `windows-drivers-rs` checkout's
|
||||
`examples/` dir**, not standalone in this repo. On the dev box it lives at
|
||||
`C:\Users\Public\m0\windows-drivers-rs\examples\pf-dualsense`. These files are checked in for
|
||||
version control / portability of the spike.
|
||||
This crate builds as a member of the [`packaging/windows/drivers/`](../../drivers) workspace, which
|
||||
uses the published **crates.io `wdk`/`wdk-sys`/`wdk-build`** (0.4/0.5) — not the old dev-box
|
||||
`windows-drivers-rs` path-deps. It's a separate cargo workspace from the main tree because driver
|
||||
crates are cdylibs built with the WDK toolchain on Windows only; it path-deps the shared ABI crate
|
||||
[`crates/pf-driver-proto`](../../../../crates/pf-driver-proto/README.md).
|
||||
|
||||
## Build / sign / install recipe (the one that actually loads)
|
||||
|
||||
@@ -76,9 +77,10 @@ silently breaks them:
|
||||
zero requests → `EvtIoDeviceControl` never fires → no HID handshake → ~5 s timeout →
|
||||
`CM_PROB_FAILED_START`. Set to `u32::MAX`.
|
||||
|
||||
## Known limitations
|
||||
## Notes
|
||||
|
||||
- Uses **statics, not per-device WDF contexts** → only one device instance per WUDFHost works.
|
||||
Multi-instance needs proper device contexts.
|
||||
- Port of the WDK `vhidmini2` UMDF2 sample; DualSense identity + 273-byte descriptor + feature blobs
|
||||
`0x05`/`0x09`/`0x20` from `crates/punktfunk-host/src/inject/dualsense.rs`.
|
||||
- **Multi-pad** works via `UmdfHostProcessSharing=ProcessSharingDisabled` — each pad gets its own
|
||||
WUDFHost (so the per-instance statics don't collide), and the driver reads its pad index from the
|
||||
device Location (`WdfDeviceAllocAndQueryProperty`) to map its own `*-shm-<index>` channel.
|
||||
- Port of the WDK `vhidmini2` UMDF2 sample; the DualSense identity + 273-byte descriptor + feature
|
||||
blobs `0x05`/`0x09`/`0x20` come from `crates/punktfunk-host/src/inject/dualsense.rs`.
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
# latency-probe
|
||||
|
||||
A **glass-to-glass latency** measurement tool (design/implementation-plan §10): it renders a
|
||||
timestamp/QR on the host, reads it back off the client's capture (or a photodiode, for true photons),
|
||||
and tracks p50/p99 — so latency regressions are quantifiable rather than felt.
|
||||
|
||||
> **Status: scaffold.** The measurement harness is stubbed out and not yet wired to a live pipeline.
|
||||
> For latency numbers today, use the per-frame **capture→…→reassembled** percentiles the
|
||||
> [`probe`](../../clients/probe/README.md) client reports over a real `punktfunk/1` session.
|
||||
|
||||
```sh
|
||||
cargo run -p latency-probe # from the repo root
|
||||
```
|
||||
|
||||
Companion to [`loss-harness`](../loss-harness/README.md) (FEC loss sweep).
|
||||
@@ -0,0 +1,16 @@
|
||||
# loss-harness
|
||||
|
||||
A **FEC loss-resilience sweep** for [`punktfunk-core`](../../crates/punktfunk-core/README.md). It
|
||||
drives access units through the in-process loopback at increasing packet-loss rates — for **both** FEC
|
||||
schemes (GF(2⁸) and GF(2¹⁶)) — and reports how many frames survive.
|
||||
|
||||
It's a pure-software stand-in for `tc netem`: no network, no root, runs anywhere `punktfunk-core`
|
||||
builds. Use it to sanity-check the FEC before reaching for the real `punktfunk/1` harness (which adds
|
||||
`tc netem` jitter/reorder on the UDP path).
|
||||
|
||||
```sh
|
||||
cargo run -p loss-harness # from the repo root
|
||||
```
|
||||
|
||||
Part of the measurement tooling (design/implementation-plan §10), alongside
|
||||
[`latency-probe`](../latency-probe/README.md).
|
||||
Reference in New Issue
Block a user