docs: rework client/crate READMEs, add missing ones
windows-drivers / probe-and-proto (push) Successful in 24s
windows-drivers / driver-build (push) Successful in 1m18s
apple / swift (push) Successful in 1m5s
android / android (push) Successful in 4m21s
ci / rust (push) Successful in 5m3s
ci / web (push) Successful in 54s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 2m48s
windows-host / package (push) Successful in 7m10s
decky / build-publish (push) Successful in 24s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 6s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
ci / bench (push) Successful in 4m38s
release / apple (push) Successful in 9m1s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 1m13s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 51s
windows-msix / package (x64, C:\Users\Public\ffmpeg, x86_64-pc-windows-msvc, C:\t) (push) Successful in 1m10s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m42s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 1m0s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 1m0s
apple / screenshots (push) Successful in 5m32s
flatpak / build-publish (push) Successful in 4m59s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 9m7s
docker / deploy-docs (push) Successful in 25s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Successful in 8m49s

Rework the client READMEs to be accurate and inviting to first-time
visitors, and fill in the gaps where crates and tools had none.

- Rewrite clients/{apple,android,decky} READMEs (features-first, trim
  dense internal narrative; drop the stale "one session at a time" /
  "renegotiation not implemented" section from the Apple README).
- Add READMEs for clients/{linux,windows,probe}, which had none.
- Add crate READMEs for punktfunk-host, punktfunk-core, pf-driver-proto.
- Add brief READMEs for tools/{loss-harness,latency-probe}.
- Fix packaging/README duplicate "Option B" heading (bootc -> Option C).
- Fix docs-site/README stale docs/ -> design/ reference.
- De-stale packaging/windows/drivers/pf-dualsense README (drop "M0 spike"
  / external-checkout framing; reflect in-tree workspace + shipped +
  installer-bundled + multi-pad), keeping the driver-authoring lore.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-01 19:31:06 +00:00
parent 7975a95cd6
commit d6596ff81b
14 changed files with 624 additions and 585 deletions
+54 -59
View File
@@ -1,83 +1,78 @@
# punktfunk Android client # punktfunk Android client (phone & TV)
Native Android client for **punktfunk/1**, targeting **phone + TV** (Compose, D-pad + touch). The native **Android** app for streaming a punktfunk host to your phone, tablet, or Android TV. A
Compose app that finds hosts on your network, pairs with a PIN, and streams at the display's own
resolution — with hardware HEVC decode, HDR10, and controller support, built for both touch and the
couch (D-pad / gamepad focus navigation).
## Architecture — Rust-heavy (like the Linux client, not thin-native like Apple) ## Features
Kotlin cannot `import` the cbindgen C header the way Swift can, so a native bridge is unavoidable. - **Hardware decode** — NDK `AMediaCodec` HEVC → `SurfaceView`, including **HDR10** (Main10 /
We write it in **Rust** and link `punktfunk-core` directly — so the Android client reuses the Linux BT.2020 PQ), with low-latency tuning and a live stats HUD.
- **Audio both ways** — Opus + Oboe playback with a jitter ring, plus mic uplink to the host.
- **Controller support** — buttons + axes with rumble and HID feedback (lightbar / adaptive
triggers); D-pad / gamepad focus navigation for TV and phone.
- **Find hosts automatically** — native mDNS discovery; first connect does a one-time **SPAKE2 PIN
pairing** (or TOFU on trusted LANs), then reconnects on a Keystore-wrapped, pinned identity.
- **Compose UI** — Connect / Settings / Stream screens with Material You theming.
Built for `arm64-v8a` + `x86_64`.
## Get it
Published to **Google Play (Internal Testing)** — join the beta via the
[Discord](https://discord.gg/kaPNvzMuGU). Per-device setup and pairing:
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
## How it's built — Rust-heavy
Kotlin can't `import` the cbindgen C header the way Swift can, so a native bridge is unavoidable. We
write it in **Rust** and link `punktfunk-core` directly — so the Android client reuses the Linux
client's orchestration (audio jitter ring, VK keymap inverse, latency/skew math, capture state client's orchestration (audio jitter ring, VK keymap inverse, latency/skew math, capture state
machine, trust logic) instead of re-porting it into Kotlin. machine, trust logic) instead of re-porting it into Kotlin.
| Side | Owns | | Side | Owns |
|------|------| |------|------|
| **Rust** (`clients/android/native``libpunktfunk_android.so`) | the JNI seam, `NativeClient` (QUIC control + UDP data plane), AnnexB`AMediaCodec` decode, Opus+Oboe audio, VK keymap, latency math, trust/pairing, **mDNS discovery** (`mdns-sd`, the same browse the Linux/Windows clients use) | | **Rust** (`native/``libpunktfunk_android.so`) | the JNI seam, `NativeClient` (QUIC control + UDP data plane), AnnexB`AMediaCodec` decode (incl. HDR10), Opus + Oboe audio + mic, controller feedback, latency math, trust/pairing, `mdns-sd` discovery |
| **Kotlin** (`clients/android`) | Compose UI (host grid / settings / stream), `SurfaceView` lifecycle, input capture, the Wi-Fi `MulticastLock` + permission UX, Keystore identity, permissions | | **Kotlin** (`app/`, `kit/`) | Compose UI, `SurfaceView` lifecycle, input capture, the Wi-Fi `MulticastLock` + permission UX, Keystore identity |
The single seam is `io.unom.punktfunk.kit.NativeBridge``Java_io_unom_punktfunk_kit_NativeBridge_*`. The single seam is `io.unom.punktfunk.kit.NativeBridge``Java_io_unom_punktfunk_kit_NativeBridge_*`.
## Layout
``` ```
clients/android/native/ Rust cdylib (workspace member) — links punktfunk-core directly native/ Rust cdylib (workspace member) — links punktfunk-core directly
src/lib.rs JNI seam (connect/pair, input, plane getters, abi/core version) src/lib.rs JNI seam (connect/pair, input, plane getters, versions)
src/session.rs session lifecycle + plane pumps src/session.rs session lifecycle + plane pumps
src/decode.rs AnnexB → AMediaCodec HEVC hardware decode → SurfaceView (incl. HDR10) src/decode.rs AnnexB → AMediaCodec HEVC hardware decode → SurfaceView (incl. HDR10)
src/audio.rs · src/mic.rs Opus + Oboe playback / mic uplink (jitter ring) src/audio.rs · src/mic.rs Opus + Oboe playback / mic uplink
src/feedback.rs rumble + HID output (lightbar / adaptive triggers) src/feedback.rs · src/stats.rs rumble + HID feedback; live video stats
src/stats.rs live video stats app/ :app — Compose UI: Connect / Settings / Stream (phone + TV)
kit/ :kit — NativeBridge · native mDNS discovery · Gamepad · Keymap · Keystore identity
clients/android/ Gradle project (this dir)
settings.gradle.kts · build.gradle.kts · gradle.properties · gradlew
app/ :app — Compose UI: Connect / Settings / Stream screens (phone + TV)
kit/ :kit — NativeBridge · discovery (native mdns-sd, polled) · Gamepad · Keymap ·
security (Keystore identity + known-host store) · cargo-ndk build
``` ```
## Prerequisites
- Android SDK + **NDK r30** (`30.0.14904198`), `platforms;android-37.0`, `build-tools;37.0.0`,
**`cmake;3.22.1`** (`sdkmanager "cmake;3.22.1"` — the `cmake` crate builds libopus with it)
- **JDK 21** for Gradle/AGP (AGP 9.2 runs on JDK 1721, *not* a newer default JDK like 25)
- Rust + `rustup target add aarch64-linux-android x86_64-linux-android` + `cargo install cargo-ndk`
Toolchain pinned: AGP 9.2.0 · Gradle 9.4.1 · Kotlin 2.3.21 · Compose BOM 2026.05.01 ·
compileSdk 37 · targetSdk 36 · minSdk 31 · ABIs arm64-v8a + x86_64.
## Build & run ## Build & run
**Android Studio:** open `clients/android` — it uses its bundled JBR 21 automatically. The **Prerequisites:** Android SDK + **NDK r30** (`30.0.14904198`), `platforms;android-37.0`,
`cargoNdk*` task builds the `.so` as part of the normal build. `build-tools;37.0.0`, **`cmake;3.22.1`** (builds libopus); **JDK 21** (AGP 9.2 runs on JDK 1721, not
a newer default); Rust with `rustup target add aarch64-linux-android x86_64-linux-android` and
`cargo install cargo-ndk`. Toolchain is pinned (AGP 9.2 · Gradle 9.4.1 · Kotlin 2.3.21 · Compose BOM
2026.05.01 · compileSdk 37 · minSdk 31).
**CLI** (point Gradle at a JDK 21 if your machine default is newer, e.g. JDK 25): **Android Studio:** open `clients/android` — it uses its bundled JBR 21, and the `cargoNdk*` task
builds the `.so` as part of the normal build.
**CLI** (point Gradle at JDK 21 if your machine default is newer):
```sh ```sh
# Adoptium/Temurin 21 (installed by the Android Studio setup, or `brew install temurin@21`): export JAVA_HOME="$(/usr/libexec/java_home -v 21)" # or your Temurin 21 path
export JAVA_HOME="$(/usr/libexec/java_home -v 21)"
cd clients/android cd clients/android
./gradlew :app:assembleDebug # cargo-ndk cross-compiles libpunktfunk_android.so first ./gradlew :app:assembleDebug # cargo-ndk cross-compiles libpunktfunk_android.so first
./gradlew :app:installDebug # onto a running emulator/device ./gradlew :app:installDebug # onto a running emulator/device
# emulators from env setup: emulator -avd pf_phone | emulator -avd pf_tv
# Emulators (created during env setup): emulator -avd pf_phone | emulator -avd pf_tv
``` ```
The debug APK lands in `app/build/outputs/apk/debug/`. Launch it, pick a host from the list, pair, The debug APK lands in `app/build/outputs/apk/debug/`. Launch it, pick a host, pair, and stream.
and stream.
## Status ## Related
A working native client (phone + Android TV), at parity with the Linux and Apple apps for the core - **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
streaming experience: - **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
- **Video** — `AMediaCodec` hardware HEVC decode → `SurfaceView`, including **HDR10** (Main10 /
BT.2020 PQ), with low-latency decode tuning and a live stats HUD.
- **Audio** — Opus + Oboe playback with a jitter ring, plus mic uplink to the host.
- **Input** — game controllers (buttons + axes) with rumble and HID feedback; D-pad /
game-controller focus navigation for the couch (TV + phone).
- **Discovery & trust** — native `mdns-sd` mDNS host list (polled over JNI; the same browse the
Linux/Windows clients use, not `NsdManager`), SPAKE2 PIN pairing and TOFU, with a
Keystore-wrapped client identity and a known-host store.
- **UI** — Compose host list / settings / stream screens, Material You theming.
- **Shipping** — built for `arm64-v8a` + `x86_64`; published to Google Play (Internal Testing).
`crates/punktfunk-core` uses the `ring` `rcgen` backend so the client `.so` is aws-lc-free.
+92 -341
View File
@@ -1,364 +1,115 @@
# punktfunk Apple client (SwiftUI) # punktfunk Apple client (macOS · iOS · iPadOS · tvOS)
The native macOS/iOS client for **`punktfunk/1`** (the post-GameStream protocol). All The native **Apple** app for streaming a punktfunk host to your Mac, iPhone, iPad, or Apple TV. A
networking/protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM, SwiftUI app that finds hosts on your network, pairs with a PIN, and streams at your display's own
input datagrams, Opus audio, cert pinning — lives in the shared Rust core (statically resolution and refresh rate — with VideoToolbox hardware decode and full controller support.
linked as `PunktfunkCore.xcframework`); this package is the Swift shell: decode
(VideoToolbox), present (SwiftUI), input capture.
## Status — working client (macOS, with iOS / tvOS in the shared build) All the networking and protocol work — QUIC control plane, UDP data plane, GF(2¹⁶) FEC, AES-GCM,
Opus audio, cert pinning — lives in the shared Rust **`punktfunk-core`** (statically linked as
`PunktfunkCore.xcframework`). This package is the Swift shell: decode, present, input, and UI.
A full streaming client: VideoToolbox HEVC decode, controllers incl. DualSense feedback, host ## Features
discovery, PIN pairing, and a network speed test. The lower-latency **stage-2 presenter**
(`VTDecompressionSession``CAMetalLayer`) is built and opt-in (Settings → Presenter); see below.
First light was achieved 2026-06-10 — validated live, Mac ↔ a Linux host over the LAN: gamescope - **Hardware decode** — VideoToolbox HEVC, with a low-latency **stage-2 presenter**
virtual output → NVENC HEVC → (`VTDecompressionSession``CAMetalLayer`, presented off a `CADisplayLink`, ~11 ms p50) as the
`punktfunk/1` (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox → default and an `AVSampleBufferDisplayLayer` fallback.
`AVSampleBufferDisplayLayer` on glass at 1280×720@60, with mouse/keyboard flowing back as - **HDR & 4:4:4** — PQ passthrough with a correct reference-white anchor, mid-session SDR↔HDR
QUIC datagrams into the host's gamescope EIS injector (thousands of events injected during reconfiguration, and hardware-probed 4:4:4 support.
the session). Headless variant of the same proof: `RemoteFirstLightTests` decoded 60/60 - **Your display's native mode** — the host builds a virtual output at exactly your WxH@Hz;
received AUs spanning 983 ms of host capture clock. mid-stream resize renegotiates without reconnecting.
- **Audio both ways** — Opus playback (CoreAudio, no bundled libopus) with a jitter ring, plus mic
uplink; speaker/mic selectable in Settings.
- **Full controller support** — one selected controller forwarded as pad 0, including **DualSense**
feedback (rumble → CoreHaptics, lightbar, player LEDs, adaptive triggers) and touchpad/motion. The
virtual pad type auto-resolves from your physical controller.
- **Mouse & keyboard** — `GCMouse`/`GCKeyboard` capture with click-to-capture and a ⌘⎋ release, plus
iPad pointer lock and touch input.
- **Find hosts automatically** — mDNS discovery (`NWBrowser` over `_punktfunk._udp`); first connect
does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then reconnects on a pinned,
Keychain-stored identity.
- **Tune the stream** — a fps / Mb·s / **latency** HUD (skew-corrected across machines), a bitrate
control, a per-host **network speed test** with a recommended bitrate, and a host-compositor picker.
The connector underneath (`punktfunk_core::client::NativeClient` over the C ABI) carries the Runs from one shared codebase across **macOS, iOS, iPadOS, and tvOS**.
full session: video AUs, **Opus audio** (`nextAudio()`), **rumble** (`nextRumble()`),
**DualSense feedback** (`nextHidOutput()` — lightbar, player LEDs, adaptive-trigger
effects), input incl. gamepads + DualSense touchpad/motion (`sendTouchpad`/`sendMotion`),
and **cert pinning + TOFU** (`pinSHA256:`/`hostFingerprint`) — see
`punktfunk1.rs::tests::c_abi_connection_roundtrip` (three sequential sessions: TOFU, pinned
reconnect, wrong-pin rejection). The host (`punktfunk-host punktfunk1-host`) is a persistent listener:
reconnect at will during development.
What's here, all compiled and tested on macOS (Xcode 26.5 / Swift 6.3): ## Get it
- **`PunktfunkKit`** (library) Install from the App Store / TestFlight, or build from source below. Per-device install steps and the
- `PunktfunkConnection.swift` — wrapper over the C ABI. AUs/audio are copied into `Data` pairing walkthrough:
(the C pointer is only valid until the next call of the same kind). `close()` is safe **[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
from any thread: per-plane locks enforce the C contract ("never close with a
`next_au`/`next_audio` in flight") instead of leaving it to callers. Pinning + TOFU
via `pinSHA256:`/`hostFingerprint`.
- `AnnexB.swift` — in-band VPS/SPS/PPS → `CMVideoFormatDescription`; Annex-B → AVCC
`CMSampleBuffer` with `DisplayImmediately` set.
- `StreamView.swift` — SwiftUI `NSViewRepresentable` over `AVSampleBufferDisplayLayer`
(stage-1 presenter: the layer hardware-decodes compressed HEVC itself). One pump
thread per view, token-cancelled so reconnects can't double-pump.
- `InputCapture.swift``GCMouse` raw deltas + `GCKeyboard` HID→VK mapping (the host's
`vk_to_evdev` consumes Windows VKs), with fractional-delta accumulation so sub-pixel
motion isn't truncated away. Buttons use GameStream ids (1=left … 5=X2). Scroll is
WHEEL_DELTA(120)-scaled: macOS via the stream view's `scrollWheel` override, iPad via
GCMouse's scroll dpad when pointer-locked and a scroll-only `UIPanGestureRecognizer`
otherwise (trackpad gestures never reach GC's scroll dpad).
- `GamepadManager.swift` — app-lifetime controller discovery + selection (`.shared`):
watches `GCController` connect/disconnect, fingerprints each pad for the Settings UI
(name, capabilities, battery), and selects the ONE controller forwarded to the host
(user pin via "Use controller", else most recently connected extended gamepad).
- `GamepadCapture.swift` — the active controller → wire: snapshot-diff over
`GCExtendedGamepad` into incremental `gamepadButton`/`gamepadAxis` events (pad 0),
plus DualSense touchpad contacts and ~250 Hz motion samples on the rich-input plane
(the GC→DualSense unit conversions live in `GamepadWire`, one place). Held state is
released on the wire on controller switch / app deactivation / stop.
- `GamepadFeedback.swift` + `DualSenseTriggerEffect.swift` — host feedback → the real
controller: one drain thread for `nextRumble()` (→ `CHHapticEngine` per handle
locality) and `nextHidOutput()` (lightbar → `GCDeviceLight`, player LEDs →
`playerIndex`, adaptive-trigger effect blocks → a total, table-driven parser →
`GCDualSenseAdaptiveTrigger`, exact for the 10-zone positional modes).
- `HostDiscovery.swift` — LAN auto-discovery: an `NWBrowser` over `_punktfunk._udp`
(the host's `crate::discovery` mDNS advert), resolving each service to an IP:port via a
throwaway `NWConnection` and parsing the TXT (`fp` advisory cert fingerprint, `pair`,
stable `id`). iOS/tvOS need `NSBonjourServices` (`Config/Info.plist`) or the system
blocks the browse.
- **`PunktfunkClient`** (the app): hosts grid (saved in UserDefaults) with an **On this
network** section listing mDNS-discovered hosts (tap to save + connect, or pair if the
host requires it), "+" toolbar sheet to add hosts manually, stream mode in Settings (⌘,),
two trust flows — the
trust-on-first-use fingerprint prompt over the live-but-blurred stream, and SPAKE2 PIN
pairing (`PairSheet`, from a host card's context menu or the trust prompt;
`ClientIdentityStore` keeps the client identity in the Keychain and presents it on
every connect) — then pinned reconnects, fps/Mb-s HUD + a **capture→client-receipt latency**
line (`LatencyMeter`, p50/p95): the AU `pts_ns` (host capture clock) to the instant the client
received it, **skew-corrected** across machines via `PunktfunkConnection.clockOffsetNs` (the
connect-time wall-clock handshake, `punktfunk_connection_clock_offset_ns`). It excludes the
layer's decode+present (stage-1 `AVSampleBufferDisplayLayer` has no per-frame present callback);
the opt-in **stage-2 presenter** (Settings → Presenter) adds a **capture→present**
(glass-to-glass) line via explicit decode + a Metal/display-link present. Settings also picks the HOST
compositor (KWin/wlroots/Mutter/gamescope, default automatic — the host honors it
only if that backend is available there) and has a **Controllers** section: every
detected controller (capability glyphs, battery, "In use" badge), which one to forward
("Use controller", default automatic), and the virtual pad type the host creates
("Controller type": Automatic / Xbox 360 / DualSense — Automatic matches the physical
pad; resolved at connect time, the host pad is fixed per session). Gamepad capture +
feedback run with streaming (`SessionModel` owns them, same trust gate as audio).
Settings also sets the **Bitrate** (Automatic toggle = host default; manual is a
log-scale slider, 2 Mbps 3 Gbps, snapped to two significant figures — above 1 Gbps
an inline warning says to run a speed test first; tvOS uses a preset picker instead,
Slider doesn't exist there; negotiated via the Hello on every connect), and a host
card's context menu offers **"Test Network Speed…"** (`SpeedTestSheet`): connects, has
the host burst probe filler over the real data plane (up to the host's 3 Gbps probe
ceiling for 2 s, roadmap §9),
shows measured goodput · loss · a recommended bitrate (≈70% of measured), and applies
it in one tap. The streaming **statistics overlay** can be turned off and moved to any
corner (Settings → Display → Statistics, `DefaultsKey.hudEnabled`/`hudPlacement`), and
toggled live with **⌘⇧S** — a Scene-level **"Stream" menu** (`StreamCommands`) that also
carries **Disconnect ⌘D**, so disconnect survives the HUD being hidden (on iOS a small
exit chip appears instead; on tvOS the Siri-Remote Menu button still disconnects). The
macOS Settings window is a **tabbed preferences pane** (General / Display / Audio /
Controllers / Advanced) — the sections are shared with the iOS single-Form layout and the
tvOS pushed-picker layout, defined once each.
- **Tests** (`swift test`): byte-level Annex-B units; a real-codec round trip
(VTCompressionSession-encoded HEVC rebuilt as the host's wire shape → `AnnexB`
VTDecompressionSession → pixels); table-driven DualSense trigger-effect parsing
(`DualSenseTriggerEffectTests`) and the gamepad wire conversions
(`GamepadWireTests`); loopback integration against real local hosts
(`test-loopback.sh` — stream round trip incl. gamepad/touchpad/motion sends, a
host-scripted feedback burst asserted on the rumble + HID-output planes
(`PUNKTFUNK_TEST_FEEDBACK=1`), the bitrate-negotiation echo and a real 20 Mbps
bandwidth probe, plus the PIN pairing ceremony and the `--require-pairing` gate
against a second, armed host); the remote first-light test above.
## Build / run / test (on a Mac) ## Build / run / test (on a Mac)
Requires Xcode 26.5 / Swift 6.3. First build the Rust core into an xcframework, then build the app:
```sh ```sh
rustup target add aarch64-apple-darwin x86_64-apple-darwin rustup target add aarch64-apple-darwin x86_64-apple-darwin
bash scripts/build-xcframework.sh # → clients/apple/PunktfunkCore.xcframework bash scripts/build-xcframework.sh # → clients/apple/PunktfunkCore.xcframework
# + BUILD_IOS=1 for the iOS slices (rustup target add aarch64-apple-ios{,-sim} x86_64-apple-ios) # BUILD_IOS=1 also builds the iOS slices (add the ios rustup targets)
# + BUILD_TVOS=1 for tvOS — TIER-3 Rust targets, built from source: # BUILD_TVOS=1 also builds tvOS (tier-3 targets, built from source — see below)
# rustup toolchain install nightly && rustup component add rust-src --toolchain nightly
cd clients/apple cd clients/apple
swift build && swift test # loopback/remote tests self-skip without a host open Punktfunk.xcodeproj # the real app: ⌘R builds + runs Punktfunk.app
swift run PunktfunkClient # the unbundled dev shell (CLI) swift run PunktfunkClient # or the unbundled dev shell (CLI)
open Punktfunk.xcodeproj # the real app: ⌘R builds + runs Punktfunk.app swift build && swift test # unit + loopback/remote tests (self-skip w/o a host)
```
bash test-loopback.sh # full loopback proof: builds punktfunk-host tvOS slices are tier-3 Rust targets, built from source:
# (synthetic source — runs on macOS), streams `rustup toolchain install nightly && rustup component add rust-src --toolchain nightly`.
# byte-verified frames into the Swift client
# against the real host (Linux box, see CLAUDE.md "Running on this box") — punktfunk1-host is a ### Test against a host
# persistent listener, reconnect at will:
# PUNKTFUNK_COMPOSITOR=gamescope PUNKTFUNK_GAMESCOPE_APP=vkcube PUNKTFUNK_ZEROCOPY=1 \ ```sh
# cargo run -rp punktfunk-host -- punktfunk1-host --source virtual --seconds 60 # full loopback proof — builds punktfunk-host (synthetic source, runs on macOS) and streams
PUNKTFUNK_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless # byte-verified frames into the Swift client, incl. the PIN pairing ceremony:
# (+ PUNKTFUNK_REMOTE_PORT / PUNKTFUNK_REMOTE_COMPOSITOR=gamescope|kwin|… / bash test-loopback.sh
# PUNKTFUNK_REMOTE_PIN=<arming-pin> for the remote pairing test)
# against a real Linux host on the LAN (see the repo README "Running on this box"):
PUNKTFUNK_REMOTE_HOST=<box-ip> swift test --filter RemoteFirstLightTests # headless
PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkClient # on glass PUNKTFUNK_AUTOCONNECT=<box-ip> PUNKTFUNK_MODE=1280x720x60 swift run PunktfunkClient # on glass
``` ```
## Xcode project (`Punktfunk.xcodeproj`) ## Project layout
The app target **Punktfunk** wraps the same sources as the `swift run` shell - **`PunktfunkKit`** (library) — the reusable pieces:
(`Sources/PunktfunkClient`, a synchronized folder — no duplication) plus `App/` (asset - `PunktfunkConnection` — the wrapper over the C ABI (thread-safe `close()`, per-plane locks,
catalog) and links `PunktfunkKit` from the local package. Generated Info.plist, ad-hoc pinning + TOFU).
signing, bundle id `io.unom.punktfunk`. Notes: - `AnnexB` / `StreamView` / `VideoDecoder` / `MetalVideoPresenter` — format handling, the stage-1
(`AVSampleBufferDisplayLayer`) and stage-2 (`VTDecompressionSession``CAMetalLayer`) presenters.
- `InputCapture``GCMouse`/`GCKeyboard` → host VK/mouse, with fractional-delta accumulation.
- `GamepadManager` / `GamepadCapture` / `GamepadFeedback` / `DualSenseTriggerEffect` — controller
discovery + selection, capture (buttons/axes/touchpad/motion), and host-feedback rendering.
- `HostDiscovery``NWBrowser` over `_punktfunk._udp`.
- **`PunktfunkClient`** (the app) — hosts grid with an *On this network* section, add-host sheet,
the two trust flows (TOFU prompt + SPAKE2 `PairSheet`), the stream view with the HUD, a
tabbed Settings pane (General / Display / Audio / Controllers / Advanced), and the network speed
test. A Scene-level **Stream** menu carries Disconnect (⌘D) and the HUD toggle (⌘⇧S).
- **Tests** (`swift test`) — Annex-B units, a real-codec VideoToolbox round trip, DualSense
trigger-effect and gamepad-wire conversions, loopback integration against real local hosts, and the
remote first-light test.
- **Entitlements (sandbox)**: the macOS target uses ## Notes for contributors
`Config/Punktfunk-macOS.entitlements`; iOS/tvOS use the shared
`Config/Punktfunk.entitlements`. The macOS app is **App-Sandboxed** (mandatory for the Mac
App Store/TestFlight, and used for the Developer ID DMG too so the local build matches what
ships): `com.apple.security.app-sandbox`, `network.client` + **`network.server`** (the
sandbox gates `bind()`; quinn + the raw-UDP plane both bind, so receive breaks without it),
`device.audio-input` (mic), `device.bluetooth` + `device.usb` (GameController over BT/USB),
and the existing `keychain-access-groups`. `app-sandbox` is macOS-only — keep it OUT of the
shared iOS/tvOS file (it fails upload validation there). Verify a build is sandboxed with
`codesign -d --entitlements :- <built .app>`. Heads-up: `device.usb` draws some App Review
scrutiny — justify it in the review notes ("reads input from USB game controllers").
- **App icon**: `App/Assets.xcassets` ships an empty `AppIcon` slot. For an Icon Composer
`.icon`: add the file to the project (target Punktfunk), set it as the App Icon in the
target's General tab, and delete the placeholder `AppIcon.appiconset`. Heads-up: CLI
`actool` (Xcode 26.5) crashed compiling `punktfunk_Logo.icon` — if Xcode does the same,
suspect the icon bundle (it has a duplicate-named layer, "…Layer-3 2.svg"), not the
project.
- **Tests from Xcode**: the package tests run with `swift test`; to get them on ⌘U, add
`PunktfunkKitTests` once via Edit Scheme → Test → + (Xcode persists it into the shared
scheme — a hand-written package-test reference doesn't resolve headlessly).
- `xcodebuild -project Punktfunk.xcodeproj -scheme Punktfunk build` works headlessly;
same for `-scheme Punktfunk-iOS -destination 'generic/platform=iOS Simulator'` (run it
in a simulator via `xcrun simctl install/launch``SIMCTL_CHILD_PUNKTFUNK_AUTOCONNECT=…`
passes the dev autoconnect env through).
## App Store screenshots - **Xcode project** (`Punktfunk.xcodeproj`) wraps the same sources as the `swift run` shell (a
synchronized folder — no duplication). The macOS target is **App-Sandboxed** (needs
`network.server` — the raw-UDP plane and quinn both `bind()`); iOS/tvOS use the shared
entitlements file (keep `app-sandbox` **out** of it). Verify with
`codesign -d --entitlements :- <built .app>`.
- **Decode flow**: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band, and recovery
keyframes re-send them — refresh the format description on every IDR; there is no out-of-band
extradata, ever.
- **ABI threading**: one video pump thread per connection, one optional audio drain thread, and one
optional feedback drain thread (rumble + HID-output). `send()` is enqueue-only and safe alongside
all of them. The wrapper's per-plane locks make `close()` safe from anywhere.
- **DualSense motion scale** (`GamepadWire`) is derived from hid-playstation's math, not yet
live-verified — if gyro/accel feel wrong in a game, correct sign/scale there and `evtest` the
host's virtual pad.
- **App Store screenshots** are automated — `tools/screenshots.sh all` renders the real UI at the
required pixel sizes via a DEBUG-only shot mode; the `apple` CI workflow captures the iOS sizes on
every main push. See the script header for details.
- Deeper design notes live in `docs-site/content/docs/apple-stage2-presenter.md`.
Automated, faithful screenshots of the real UI for App Store Connect — one set per platform at ## Related
exactly the accepted pixel sizes. Driver: **`tools/screenshots.sh`**.
```sh - **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
tools/screenshots.sh all # macOS + (if full Xcode) iOS, iPadOS, tvOS → ./screenshots - **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
tools/screenshots.sh macos # just macOS
OUT=~/Desktop/shots tools/screenshots.sh ios ipad tvos
PUNKTFUNK_SHOT_HERO=~/frame.png tools/screenshots.sh ios # real captured frame behind the hero
```
How it works: the app has a DEBUG-only **shot mode** (`Sources/PunktfunkClient/Screenshots/`).
Launched with `PUNKTFUNK_SHOT_SCENE=<name>` it renders **one** mock-populated screen full-bleed
(`ScreenshotHostView`) instead of `ContentView`, then the OS screenshots the *real, fully-rendered*
window — `screencapture` on macOS, `xcrun simctl io booted screenshot` on the Simulators. The five
scenes (`ShotScenes.all`): `01-stream` (the stream hero — a synthetic frame + the glass HUD, since
`StreamView` needs a live connection), `02-hosts`, `03-pair`, `04-trust`, `05-settings`. Mock data
is in `ShotMock`; nothing touches a host.
Output pixels are App Store Connect's required/largest sizes (Apple auto-derives the smaller ones):
`mac` 2880×1800 · `iphone-6.9` 1320×2868 (hero 2868×1320) · `ipad-13` 2064×2752 (hero 2752×2064) ·
`appletv` 1920×1080.
Why not `ImageRenderer` (the obvious offscreen route)? It can't rasterize this app's chrome —
`NavigationStack`, `Form`/`TabView`, and Liquid-Glass/`NSVisualEffect` materials all render black or
SwiftUI's "can't render" placeholder. Capturing the live window/Simulator avoids that entirely.
Requirements / gotchas:
- **macOS**: only the Swift toolchain is needed, **plus a one-time Screen Recording grant** for
your terminal (System Settings → Privacy & Security → Screen Recording) — without it
`screencapture -l` fails with "could not create image from window". (A no-permission fallback,
`PUNKTFUNK_SHOT_SELFCAPTURE=<dir>`, uses `cacheDisplay` — but it omits material blur and can't
read `ScrollView` content, so it's for quick checks, not submission.)
- **iOS/iPadOS/tvOS**: needs **full Xcode** (xcodebuild + Simulators), not just Command Line Tools,
and the matching device Simulators installed (iPhone 16 Pro Max, iPad Pro 13", Apple TV). Run it
on a full-Xcode Mac (e.g. the `macos-arm64` CI mini).
- The hero defaults to a synthetic synthwave frame — set `PUNKTFUNK_SHOT_HERO` to a real captured
frame for a production-quality lead screenshot.
**CI**: the `apple` workflow's **`screenshots`** job runs on the `macos-arm64` runner on every main
push + manual dispatch (skipped on PRs), and attaches the result as a single zip artifact,
**`punktfunk-appstore-screenshots`** (download it from the run's Artifacts; `upload-artifact@v3`
Gitea's backend rejects v4). It captures the two **required iOS sizes — iPhone 6.9" + iPad 13"**
on the Simulator (auto-creating the device if the runner lacks it), and is isolated from the
build/test job so a capture hiccup never reds the build.
**macOS and tvOS are NOT in CI**, by design: the self-hosted runner is **headless** (no
window-server session), so the macOS window capture can't run there, and tvOS needs the Tier-3
build-std slice. Generate those on a GUI Mac: `tools/screenshots.sh macos tvos`. (If the runner is
ever switched to a logged-in GUI session, re-adding macOS to the job's capture step is one line.)
## Notes for whoever picks this up next
1. **cbindgen import quirk** (the predicted "small compile fixes", now fixed): the
C17-compatible header spells `PunktfunkStatus`/`PunktfunkInputKind` as integer typedefs while
the enum *constants* import into Swift as a distinct same-named type — bridge with
`.rawValue` (see the top of `PunktfunkConnection.swift`). Don't fight the generated header.
2. **ABI contract**: one video pump thread per connection, plus optionally one *separate*
audio drain thread for `nextAudio()` and one feedback drain thread for
`nextRumble()`/`nextHidOutput()` (the core keeps per-plane borrow slots, so the planes
never alias; rumble + HID-output are two planes drained sequentially by the one
feedback thread); `send()` is enqueue-only and safe alongside all of them. The
wrapper's per-plane locks make `close()` safe from anywhere (it waits out in-flight
polls, ≤ their timeouts).
3. **Decode flow**: the host opens every stream with an IDR carrying VPS/SPS/PPS in-band
and recovery keyframes re-send them — "refresh the format description on every IDR"
(what `StreamView` does) is sufficient; there is no out-of-band extradata, ever.
4. **Stage 2 — built, opt-in (`punktfunk.presenter == "stage2"`, default stage 1).** Explicit
`VTDecompressionSession` decode (`VideoDecoder`) → a `CAMetalLayer` + display-link present
(`MetalVideoPresenter`/`Stage2Pipeline`), hosted as a sublayer by the same `StreamView`s with
input capture + HUD unchanged. It adds a **capture→present** (glass-to-glass, modulo the host
render→capture term) HUD line, skew-corrected via `PunktfunkConnection.clockOffsetNs`. The
decode half is unit-tested (`testVideoDecoderAsyncCallbackDeliversPixels`); the Metal present
is display-bound — **validate live** (flip the Settings "Presenter" picker, watch the HUD
number and that the image looks right) before making it the default. 10-bit/HDR + a smoothing
pacer are later. Plan: `docs-site/content/docs/apple-stage2-presenter.md`.
5. **Audio — wired, both directions.** Playback: `SessionAudio` drains `nextAudio()`
on its own thread, decodes through CoreAudio's built-in Opus codec (`OpusCodec.swift`
— kAudioFormatOpus, no bundled libopus; round-trip unit-tested) into a priming
jitter ring feeding an `AVAudioSourceNode`. Mic: a second engine taps the input
device, resamples to 48 kHz stereo, Opus-encodes 20 ms chunks and `sendMic()`s them
(the host's virtual PipeWire source accepts any frame size ≤ 120 ms). Speaker/mic
are chosen in Settings (`AudioDevices.swift` — persisted by UID; "System default"
leaves the engines unpinned so they follow macOS device changes), mic on/off toggle
included; the app asks for mic permission on first use
(NSMicrophoneUsageDescription is in the Xcode target). A/V sync and packet-loss
concealment beyond silence-fill are still open (AudioPacket.seq/ptsNs carry what's
needed). Decode with libopus or `AVAudioConverter`/`kAudioFormatOpus` into an
`AVAudioEngine` source node; conceal gaps (drop/dup) rather than blocking — the Rust
side buffers 320 ms and drops the newest packet when the puller lags. Wall-clock
`ptsNs` shares the host clock with video AUs for A/V sync. Wiring this into
`PunktfunkClient` is the next app-side task.
6. **Gamepads — wired end to end.** Exactly ONE controller (the `GamepadManager`
selection) forwards as pad 0; the host accumulates the incremental events into a
virtual pad whose TYPE the client negotiates in the Hello (`gamepad:` connect
parameter, echoed resolved in `resolvedGamepad` — Automatic resolves from the physical
pad at connect time; host precedence: explicit client choice > host `PUNKTFUNK_GAMEPAD`
env > Xbox 360). A DualSense session carries the full feel: adaptive-trigger blocks
(`DualSenseTriggerEffect.parse` — mode bytes per the community convention
(Nielk1/ds5w/inputtino), total, unknown → `.off`), lightbar, player LEDs, touchpad,
motion. **Motion scale constants** (`GamepadWire.gyroLSBPerRadS` = 20 LSB per deg/s,
`accelLSBPerG` = 10000) are derived from hid-playstation's math over the host's fixed
calibration blob, not yet live-verified — if gyro/accel feel wrong in a real game,
correct sign/scale in `GamepadCapture.forwardMotion`/`GamepadWire` and `evtest` the
host's virtual pad. Twin identical controllers share a fingerprint base, so a manual
pin can swap between them across reconnects (documented in the Settings footer).
7. **Trust — the full ceremony exists now (SPAKE2).** `generateIdentity()` once (persist
both PEMs in the Keychain), then `pair(host:identity:pin:name:)` with the 4-digit PIN
the host prints when it ARMS pairing (`--allow-pairing`/`--require-pairing`; one PIN
per arming window, surfaced in the host's web console — port 3000 → Pairing — and
printed at startup; the user reads it before pairing). Returns the
host's VERIFIED fingerprint; persist it and pass `pinSHA256:` + `identity:` to every
connect. Pairing is a real PAKE: a wrong PIN gets ONE online guess (no offline
dictionary attack), throwing `.wrongPIN`; a wrong-size pin throws `.invalidPin`. `PunktfunkClient` implements both flows:
the TOFU fingerprint sheet keeps working against hosts not running
`--require-pairing`, and the PIN ceremony is wired in — `ClientIdentityStore`
(Keychain) on every connect, `PairSheet` from a host card's context menu or the trust
prompt's "Pair with PIN instead…" (the host's accept loop is sequential, so that path
drops the live session before pairing). With `--require-pairing` the host now
authorizes clients too (the "other direction" is no longer open, opt-in per host);
the whole gate is regression-tested in `testPairingCeremonyAndRequirePairingGate`.
7b. **Resize without reconnect**: `requestMode(width:height:refreshHz:)` mid-stream —
the host rebuilds at the new mode in ~90 ms; the first new-mode AU is an IDR with
fresh parameter sets (the refresh-on-IDR decode flow handles it untouched) and
`currentMode()` reflects the switch. Wire it to window-resize events.
8. **Input capture** (stage 1): capture is a deliberate, reversible STATE owned by
`StreamLayerView`, Moonlight-style. Engaged when the stream starts / trust is
confirmed and when the user clicks into the video (that click is suppressed toward
the host); released by ⌘⎋ (toggles) or focus loss; NEVER engaged by mere app
activation — activating clicks may be title-bar drags or resizes, which used to get
their cursor warped away mid-drag. While captured: the local cursor is hidden +
frozen mid-view (the host renders its own), all input is forwarded, and the view
consumes key events as first responder so unhandled keyDowns don't beep — ⌘-combos
still work locally (⌘D disconnect, ⌘Q) *and* reach the host via GC. While released:
nothing is forwarded (`InputCapture.forwarding` gates the GC handlers; held
keys/buttons are flushed host-side on release so nothing sticks down), the cursor is
free, and the HUD shows "Click the stream to capture input". GC handlers only fire
while the app has focus, and focus loss also auto-releases everything held. One live capture per process (the GC
mouse/keyboard singletons have a single handler slot — ownership is tracked so a stale
capture's stop() can't clobber a newer one).
9. **iOS/iPadOS — ported and first-lit** (iPad simulator ↔ the real host, 60 fps).
`BUILD_IOS=1 bash scripts/build-xcframework.sh` builds device + universal-simulator
slices; the Xcode project has a second target, **Punktfunk-iOS**, sharing the same
synchronized sources. The iOS `StreamView` (StreamViewIOS.swift — same name/signature
as the macOS one, so the SwiftUI shell is identical) hosts the shared `StreamPump` in
a view controller for `prefersPointerLocked`: with a hardware mouse/trackpad that is
the iPadOS cursor capture (system honors it fullscreen-and-frontmost; in Stage
Manager it degrades to absolute-mouse forwarding). Input is routed by kind: DIRECT
fingers / Pencil are touches (each gets a wire touch id, coordinates mapped through the
aspect-fit letterbox into host-mode pixels — surface == host mode, so the host rescale is
the identity), while a mouse/trackpad is a MOUSE — pointer-LOCKED it is GCMouse relative
deltas; unlocked it is absolute moves + buttons + scroll over the UIKit pointer path
(hover + `.indirectPointer` touches), the local cursor staying visible so you can aim. An
indirect pointer is never sent as a touch. Touch is gated on trust (not forwarded under
the TOFU prompt), and returning to the foreground restores the capture you had on leaving.
`InputCapture` is cross-platform (GC works the same on iPadOS; ⌘⎋ is detected from
the HID stream there); audio routes via `AVAudioSession` (the Settings device
pickers are macOS-only). For the iPad-with-external-display setup: the target
enables multiple scenes + indirect input events — on Stage Manager iPads, drag the
punktfunk window onto the external screen and the stream runs there with full
keyboard/mouse/touch. While streaming the session is immersive (edge-to-edge,
status bar + home indicator hidden) and the iPadOS cursor is hidden over the video only
while the scene is actually pointer-LOCKED (`UIPointerInteraction` `.hidden()`); when the
lock isn't held it stays visible and the mouse forwards as an absolute cursor instead; on
iOS first run the stream mode defaults to the device's native screen so the video
fills the display. **tvOS** runs the same app (target **Punktfunk-tvOS**, first-lit
in the Apple TV simulator at 720p60): playback-only audio (no mic on tvOS),
focus-driven UI (`.card` host tiles), no kb/mouse capture yet — input lands with
gamepad support, the natural tvOS input anyway. While streaming there is NO focusable
control (a focusable Disconnect button would let the focus engine eat the controller's A
before the host sees it); the Siri Remote's **Menu** button disconnects (`.onExitCommand`).
Core slices are tier-3 Rust targets (see Build above). Known gaps: true pointer LOCK (`prefersPointerLocked`) isn't
consulted through UIHostingController, so the hidden cursor can still drift onto a
second screen (fixing it means putting the controller into the UIKit presentation
chain); and
AVAudioSession interruptions (calls, Siri) don't auto-restart the audio engines yet
(reconnect recovers).
## Known limitations of the current host (relevant to client UX)
- One session **at a time** (the listener is persistent, but a second concurrent client
waits in the accept queue until the current session ends — the virtual output and
encoder are single-tenant).
- Mid-stream renegotiation (resolution change without reconnect) is designed-for but not
implemented (the Welcome is one-shot today).
- Host-side gamepad injection needs `/dev/uinput` access on the box (udev rule from
`design/linux-setup.md`).
+55 -160
View File
@@ -1,189 +1,84 @@
# punktfunk Decky plugin (SteamOS / Steam Deck) # punktfunk — Steam Deck plugin (Decky)
A **[Decky Loader](https://decky.xyz/)** plugin that adds a **punktfunk** panel to the Steam Stream to your **Steam Deck** without ever leaving Gaming Mode. This
Deck's Quick Access Menu (the QAM, opened with the `…` button), so you can launch the **[Decky Loader](https://decky.xyz/)** plugin adds a **punktfunk** panel to the Quick Access Menu
punktfunk streaming client from **Gaming Mode** without dropping to the desktop. (the `…` button): discover hosts on your network, pair with a PIN, tweak stream settings, and launch
a fullscreen, gamescope-focused stream — all from the couch, gamepad-navigable.
Because Decky plugins run inside Steam's CEF, the panel is built from real Steam UI The video itself is the native GTK4 Linux client (the `io.unom.Punktfunk` flatpak); the plugin
primitives (`@decky/ui`: `PanelSection`, `PanelSectionRow`, `ButtonItem`, `Field`, discovers, pairs, configures, and *launches it the right way* so gamescope fullscreens it — the same
`Spinner`) — so it looks and feels native to Gaming Mode. Steam-shortcut trick MoonDeck uses. Because it's built from real Steam UI primitives (`@decky/ui`),
the panel looks and feels native to Gaming Mode.
> **Full Gaming-Mode client.** Discovery, a fullscreen page, in-UI SPAKE2 PIN pairing,
> stream settings, and a stream that actually launches fullscreen under gamescope (via a
> Steam shortcut, MoonDeck-style). The video itself is the existing GTK4 flatpak client
> (`io.unom.Punktfunk`) — the plugin discovers, pairs, configures, and *launches it the
> right way* so gamescope focuses it. The Steam-shortcut launch + pairing need a real Deck
> in Gaming Mode to fully confirm.
## What it does ## What it does
1. **Discover** — browses the LAN over mDNS for punktfunk/1 hosts (`_punktfunk._udp`, 1. **Discover** — browses the LAN over mDNS for punktfunk hosts, in both the QAM panel and a
backend `discover()` via `avahi-browse`). Shown in both the QAM panel and a **fullscreen fullscreen page.
page** (Decky route `/punktfunk`, via `routerHook.addRoute`). 2. **Pair** — for a host that requires it, a gamepad-navigable PIN keypad runs the SPAKE2 pairing
2. **Pair** — for a `pair=required` host: a gamepad-navigable PIN keypad. The operator arms ceremony headlessly, then remembers the host so future streams connect silently.
pairing on the host (it shows a 4-digit PIN), the user enters it on the Deck, and the 3. **Stream** — launches fullscreen via a hidden Steam shortcut so gamescope focuses it.
backend runs the SPAKE2 ceremony headlessly via the flatpak client's `--pair` mode 4. **Settings** — resolution / refresh / bitrate / gamepad / mic, written to the client's config.
(`pair()`), persisting the host as paired so the stream then connects silently.
3. **Stream** — launches fullscreen in Gaming Mode. The plugin registers ONE hidden
non-Steam shortcut pointing at `bin/punktfunkrun.sh`, passes `PF_HOST` as the shortcut's
Steam launch options, and starts it with `SteamClient.Apps.RunGame` — so gamescope
focuses + fullscreens it. (A flatpak launched directly from the backend is invisible:
gamescope only focuses the process tree Steam launched via `reaper` — gamescope#484.)
The wrapper then execs `flatpak run io.unom.Punktfunk --connect <host>`.
4. **Settings** — resolution / refresh / bitrate / gamepad / mic, written to the client's
`client-gtk-settings.json` (`get_settings`/`set_settings`), which the launched client reads.
To leave the stream: the in-client controller chord (**L1+R1+Start+Select**) or close the To leave a stream: the in-client controller chord (**L1 + R1 + Start + Select**), or close the
"game" from the Steam overlay — exiting the client ends the Steam game and returns to "game" from the Steam overlay — either returns you to Gaming Mode.
Gaming Mode automatically.
## Architecture
| File | Role |
| --- | --- |
| `src/index.tsx` | Frontend: QAM panel + the `/punktfunk` fullscreen page (host list, PIN keypad modal, settings). |
| `src/steam.ts` | Steam-shortcut launch (`AddShortcut` / `SetAppLaunchOptions` / `RunGame`) — the focus-correct stream start. |
| `src/backend.ts` | Typed `callable` bridges to `main.py`. |
| `bin/punktfunkrun.sh` | The launch wrapper the Steam shortcut targets (so the window is focusable). |
| `main.py` | Backend: `discover` / `pair` / `runner_info` / `get_settings` / `set_settings` / `kill_stream` / `check_update`. |
| `plugin.json` | Decky plugin manifest. |
| `update.json` | CI-baked `{channel, manifest}` — where `check_update()` polls (absent on dev builds). |
| `decky.pyi` | Type stub for the injected `decky` module (vendored from the template). |
### Discovery (`discover()`)
Shells out to **`avahi-browse -rpt _punktfunk._udp`** (SteamOS and Bazzite ship
`avahi-daemon`; this avoids bundling python-zeroconf):
- `-r` resolve services, `-p` parseable output, `-t` terminate after the cache dump.
- Resolved records start with `=` and are semicolon-separated:
`=;iface;protocol;name;type;domain;hostname;address;port;txt`.
- The `txt` column is space-separated, quoted `"key=value"` tokens. We read the keys the
host advertises (`crates/punktfunk-host/src/discovery.rs`): `proto`, `fp`, `pair`, `id`.
- Records are deduped on the `id` TXT key (a host re-advertises per interface and across
IPv4/IPv6), preferring the IPv4 address for the user-facing host string.
### Client launch (`connect()`)
The client binary `punktfunk-client` is resolved in order: `PATH``/usr/bin`
`/usr/local/bin``~/.local/bin` → a `flatpak run io.unom.Punktfunk` fallback. The resolved
argv and a clear `client-not-found` error surface to the UI. The child PID is tracked so
`disconnect()` (and plugin `_unload`) can terminate it.
> On the **Steam Deck** the client install is the flatpak `io.unom.Punktfunk`
> (`packaging/flatpak/`) — SteamOS `/usr` is read-only and lacks `libadwaita`/`libSDL3`, so
> the flatpak (which bundles them) is the canonical path; the resolver's flatpak fallback
> launches exactly that.
## Prerequisites
- **Decky Loader** installed on the Deck (https://decky.xyz/).
- **`punktfunk-client`** (the GTK4/libadwaita Linux client, crate `punktfunk-client-linux`)
installed and runnable on the Deck — via `.deb`/RPM/flatpak, or symlinked into
`~/.local/bin`.
- **avahi** (`avahi-daemon` + `avahi-browse`) for discovery — present on SteamOS/Bazzite.
- A punktfunk/1 host on the LAN (`punktfunk-host serve` or `punktfunk1-host`).
## Build
```sh
pnpm install
pnpm build # rollup → dist/index.js
```
(`npm install && npm run build` also works.)
## Install on the Deck ## Install on the Deck
### Option A — Decky "install from URL" (recommended; published by CI) You need **[Decky Loader](https://decky.xyz/)** and the **`io.unom.Punktfunk` flatpak**
([`packaging/flatpak`](../../packaging/flatpak/README.md)) installed on the Deck — SteamOS `/usr` is
read-only, so the flatpak (which bundles libadwaita/SDL3) is the canonical client. Discovery uses
`avahi-browse`, which ships on SteamOS/Bazzite.
CI (`.gitea/workflows/decky.yml`) builds the plugin into a store-layout zip and publishes it to **Recommended — install from URL** (published by CI): in Decky → Settings → **Developer Mode**
Gitea's **generic package registry** on every push to `main` and on `v*` tags, exposing a stable **Install Plugin from URL**, paste:
URL. In Decky's settings → **Developer Mode****Install Plugin from URL**, paste:
``` ```
https://git.unom.io/api/packages/unom/generic/punktfunk-decky/latest/punktfunk.zip https://git.unom.io/api/packages/unom/generic/punktfunk-decky/latest/punktfunk.zip
``` ```
(or a pinned version: `.../punktfunk-decky/<version>/punktfunk.zip`). On tags the same zip is (or a pinned `.../punktfunk-decky/<version>/punktfunk.zip`). The plugin then **self-updates** without
also attached to the Gitea release. The zip's layout is the store-required one — a single the Decky store — when a newer build exists, an **Update to vX** button appears and drives Decky
top-level `punktfunk/` dir holding `plugin.json`, `package.json`, `main.py`, `dist/index.js`, Loader's own (SHA-256-verified) install.
`README.md`, and `LICENSE`.
### Option B — manual dev copy (sideload) ## Build & sideload (development)
Decky's `~/homebrew/plugins/` is **root-owned** (PluginLoader runs as root and manages it), so a
plain `rsync` into it fails — stage to a writable temp dir, then `sudo`-install and restart the
loader. The two helper scripts do exactly this:
```sh ```sh
cd clients/decky cd clients/decky
pnpm install pnpm install
pnpm build # rollup → dist/index.js
pnpm run package # → out/punktfunk/ + out/punktfunk-v<ver>.zip pnpm run package # → out/punktfunk/ + out/punktfunk-v<ver>.zip
DECK=deck@<deck-ip> pnpm run deploy # rsync → /tmp, sudo cp into plugins/, chown root, restart DECK=deck@<deck-ip> pnpm run deploy # rsync → /tmp, sudo-install into the root-owned plugins dir, restart loader
``` ```
`deploy.sh` prompts for the Deck's sudo password interactively (via `ssh -t`); set `DECKPASS=…` `~/homebrew/plugins/` is root-owned (the loader runs as root), so `deploy.sh` stages to a temp dir
to run it non-interactively. Equivalent by hand: then `sudo`-installs and restarts the loader — set `DECKPASS=…` to run it non-interactively. A loader
restart is required for an out-of-band install to appear.
```sh ## Architecture
cd clients/decky && pnpm build && bash scripts/package.sh
rsync -azp --delete out/punktfunk/ deck@<deck-ip>:/tmp/punktfunk/
ssh -t deck@<deck-ip> 'sudo sh -c "rm -rf ~deck/homebrew/plugins/punktfunk && \
cp -r /tmp/punktfunk ~deck/homebrew/plugins/punktfunk && \
chown -R root:root ~deck/homebrew/plugins/punktfunk && systemctl restart plugin_loader"'
```
A loader restart is required for an out-of-band install to appear. The **punktfunk** panel then | File | Role |
shows up in the Quick Access Menu. | --- | --- |
| `src/index.tsx` | Frontend: QAM panel + the `/punktfunk` fullscreen page (host list, PIN keypad, settings). |
| `src/steam.ts` | Steam-shortcut launch (`AddShortcut` / `SetAppLaunchOptions` / `RunGame`) — the focus-correct stream start. |
| `src/backend.ts` | Typed `callable` bridges to `main.py`. |
| `bin/punktfunkrun.sh` | The launch wrapper the Steam shortcut targets (so the window is focusable). |
| `main.py` | Backend: `discover` (via `avahi-browse`) / `pair` / settings / `kill_stream` / `check_update`. |
| `plugin.json` · `update.json` | Decky manifest; CI-baked update channel. |
> The plugin launches the client via the flatpak `io.unom.Punktfunk` (see The client binary is resolved `PATH``/usr/bin``/usr/local/bin``~/.local/bin` → a
> [`../../packaging/flatpak/README.md`](../../packaging/flatpak/README.md)) — install that on `flatpak run io.unom.Punktfunk` fallback, so the flatpak install always works.
> the Deck too, or the panel's Connect surfaces a `client-not-found` error.
## Updating (self-update, no store)
The plugin updates itself without the official Decky store. CI (`decky.yml`) publishes a tiny
per-channel `manifest.json` next to the zip in the Gitea registry:
```json
{"version":"0.3.123","artifact":".../punktfunk-decky/0.3.123/punktfunk.zip","sha256":"…"}
```
and bakes an `update.json` (`{channel, manifest}`) into the plugin so it knows which channel it was
installed from. The backend `check_update()` reads the **installed** version from `package.json`
the value Decky itself reports (it does **not** read `plugin.json`) — fetches the channel manifest,
and compares. When a newer build exists the frontend shows an **Update to vX** button that drives
Decky Loader's own install RPC:
```ts
window.DeckyBackend.callable("utilities/install_plugin")(artifact, "punktfunk", version, hash, /*UPDATE=*/2)
```
The loader (root) downloads the immutable per-version zip, **SHA-256-verifies** it against `hash`,
replaces `~/homebrew/plugins/punktfunk`, and hot-reloads — the unprivileged backend never writes the
root-owned plugins dir itself. `window.DeckyBackend` / `utilities/install_plugin` are loader
internals (not `@decky/api`), so every access is guarded; missing them, the button falls back to a
toast pointing at **Install Plugin from URL**.
> CI stamps a **plain numeric** semver per channel (`0.3.<run>` canary, `X.Y.Z` stable) into
> `package.json`. Decky's `compare-versions` orders pre-release identifiers lexically (so `ci10 < ci9`)
> — a `-ciN` suffix would mis-detect updates.
**Optional — native Updates tab:** Decky's store is single-source (a custom store URL *replaces* the
official catalog), so punktfunk doesn't ship one by default. A user who wants the native update badge
can point Decky → Settings → **Custom store** at a punktfunk-only store JSON — not recommended if you
use other plugins, since it hides the official catalog.
## Limitations / next steps ## Limitations / next steps
- **Needs on-Deck validation in Gaming Mode**: the Steam-shortcut launch (`AddShortcut` / - **Needs on-Deck validation in Gaming Mode** the Steam-shortcut launch and headless pairing follow
`RunGame` / the `gameId` encoding) and the headless pairing env are coded to MoonDeck's MoonDeck's proven pattern but are verified only at build time here.
proven pattern but verified only at build time here. - No manual "add host by IP" entry yet (discovery is mDNS-only).
- mDNS discovery depends on `avahi-browse`; no manual "add host by IP" entry yet. - No in-stream overlay inside the plugin — the client owns the session once launched.
- No in-stream overlay (latency/bitrate HUD) inside the plugin — the client owns the session - Pairing needs the operator to **arm pairing on the host** so it shows the PIN; the plugin can't arm
once launched; leave it with the L1+R1+Start+Select chord. it remotely.
- Pairing requires the operator to **arm pairing on the host** (so it shows the PIN); the
plugin can't arm it remotely (no host mgmt token on the Deck). ## Related
- Settings are written to the flatpak's sandbox config path; if the client ever moves its
config location, that path mapping must follow. - **[Documentation](https://docs.punktfunk.unom.io/docs/steam-deck)** — Steam Deck setup guide
- **[Linux client](../linux/README.md)** — the app this plugin launches
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
+77
View File
@@ -0,0 +1,77 @@
# punktfunk — Linux client
The native **Linux** app for streaming a punktfunk host to your desktop, laptop, or Steam Deck.
It's a clean GTK4/libadwaita app that finds hosts on your network, pairs with a PIN, and puts a
low-latency stream on glass at your display's own resolution and refresh rate.
Built in Rust, it links the shared **`punktfunk-core`** directly (no C ABI) and speaks the fast
**`punktfunk/1`** protocol — QUIC control plane, GF(2¹⁶) FEC + AES-GCM data plane.
## Features
- **Zero-copy hardware decode** — FFmpeg VAAPI decode → DRM-PRIME dmabuf → `GdkDmabufTexture`
(Tier-1 zero-copy on Intel and AMD), with an automatic software-HEVC fallback on NVIDIA or when
VAAPI is unavailable.
- **Your display's native mode** — the host builds a virtual output at exactly your WxH@Hz; no
scaling, no letterboxing. Steady 60 fps at 1080p60, ~6 ms capture→decoded on the LAN.
- **Audio both ways** — PipeWire playback with a jitter ring, plus mic uplink to the host.
- **Full controller support** — SDL3 gamepads with rumble and DualSense fidelity (lightbar, player
LEDs, touchpad, motion, adaptive-trigger replay). Click-to-capture keyboard and mouse, with a
release chord (Ctrl+Alt+Shift+Q) and focus-loss release.
- **Find hosts automatically** — mDNS discovery lists hosts on your LAN; saved hosts persist.
First connect does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then reconnects on
a pinned identity.
- **Per-host speed test** to pick a bitrate, plus compositor and mode preferences in Settings.
## Get it
Most people should install a package rather than build from source:
| Distro | Install |
|--------|---------|
| **Flatpak** (any distro, Steam Deck) | `io.unom.Punktfunk` — see [`packaging/flatpak`](../../packaging/flatpak/README.md) |
| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-client` *(after adding the repo)* |
| **Fedora / Bazzite** (rpm) | `rpm-ostree install punktfunk-client` |
| **Arch** (PKGBUILD) | see [`packaging/arch`](../../packaging/arch/README.md) |
Per-device install steps and pairing walkthrough:
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
## Build & run from source
Requires GTK ≥ 4.16, libadwaita ≥ 1.5, FFmpeg 7 or 8 (with VAAPI for hardware decode), PipeWire,
and SDL3 (with hidapi) development packages.
```sh
# from the repo root
cargo run -p punktfunk-client-linux # launch the app
cargo run -p punktfunk-client-linux -- --discover # list hosts on the LAN, then exit
cargo run -p punktfunk-client-linux -- --connect HOST[:PORT] # skip the host list and connect
```
The binary is named **`punktfunk-client`**. Handy flags: `--connect host[:port]` (start a session
immediately — for scripting and the Steam Deck launcher), `--discover [secs]`, and
`--pair <PIN> --connect host[:port]` (run the pairing ceremony headlessly). Force a decoder with
`PUNKTFUNK_DECODER=software|vaapi`.
## Layout
```
src/
main.rs · app.rs entry point, GTK application, CLI paths
ui_hosts.rs host list (mDNS + saved), pairing / trust dialogs
ui_settings.rs resolution · refresh · decoder · bitrate · compositor · mic
ui_stream.rs the stream window (GtkGraphicsOffload present) + input capture
session.rs session lifecycle over the NativeClient connector
video.rs FFmpeg VAAPI / software decode → dmabuf / texture
audio.rs PipeWire playback + mic uplink
gamepad.rs · keymap.rs SDL3 controllers + feedback; keyboard VK mapping
trust.rs · discovery.rs persistent identity, TOFU/PIN pairing, mDNS browse
tools/screenshots.sh store screenshot capture
```
## Related
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
- **[Steam Deck plugin](../decky/README.md)** — launches this client fullscreen in Gaming Mode
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
+50
View File
@@ -0,0 +1,50 @@
# punktfunk — probe (reference client)
`punktfunk-probe` is the **headless reference client** for the `punktfunk/1` protocol — a
command-line tool for testing, latency measurement, and validating host behavior. It's not a
streaming app you'd watch on; it connects, exercises a plane, and reports numbers. If you want to
actually stream, use the [Linux](../linux/README.md), [Windows](../windows/README.md),
[Apple](../apple/README.md), or [Android](../android/README.md) clients.
Because it links the same **`punktfunk-core`** as every other client, it's also the canonical
example of driving the protocol end to end: QUIC control plane, UDP data plane, and the side planes
(input, audio, rumble) over QUIC datagrams.
## What it does
- **Receives a real stream**, writes a playable `.h265`, and reports per-frame
**capture→…→reassembled latency** percentiles (the host stamps each frame with its capture clock).
- **Verification mode** against a synthetic host — byte-checks deterministic test frames.
- **Exercises every plane** with scripted test traffic:
`--input-test` (mouse/keyboard), `--mic-test` (a 440 Hz Opus tone up to the host mic),
`--touch-test` (a synthetic finger), `--rich-input-test` (DualSense touchpad + motion, logging the
HID-output feedback that comes back).
- **Trust** — `--pin <64-hex>` pins the host fingerprint; `--pair <PIN>` runs the SPAKE2 pairing
ceremony and prints the verified fingerprint to pin from then on. Without a pin it trusts on first
use.
- **Discovery** — `--discover [secs]` browses the LAN for `_punktfunk._udp` hosts and prints each
(name, addr:port, pairing requirement, cert fingerprint), then exits.
- **Negotiation knobs** — `--mode WxHxFPS`, `--remode` (mid-stream mode change), `--bitrate`,
`--audio-channels` (stereo / 5.1 / 7.1), `--compositor`, `--gamepad`, `--launch`, `--speed-test`.
## Usage
```sh
# stream 720p120 from a host, save the video, and print latency percentiles:
cargo run -p punktfunk-probe -- --mode 1280x720x120 --connect HOST:PORT --out /tmp/a.h265
# list hosts on the LAN:
cargo run -p punktfunk-probe -- --discover
# pair with a host that requires it (read the PIN off the host), then stream:
cargo run -p punktfunk-probe -- --connect HOST:PORT --pair 1234
cargo run -p punktfunk-probe -- --connect HOST:PORT --pin <64-hex> --input-test
```
Full flag reference is in the module doc-comment at the top of [`src/main.rs`](src/main.rs).
## Related
- **[Project README](../../README.md)** — the host, the streaming clients, and the protocol
- **`punktfunk-host punktfunk1-host`** — the persistent native-protocol listener to probe against
(see the "Running on this box" section of the repo README / `CLAUDE.md`)
+71
View File
@@ -0,0 +1,71 @@
# punktfunk — Windows client
The native **Windows** app for streaming a punktfunk host to your PC. A modern WinUI 3 app that
discovers hosts on your network, pairs with a PIN, and streams at your display's own resolution and
refresh rate — with a hardware-accelerated D3D11 video path and HDR.
It's **pure Rust**: the UI is WinUI 3 driven through [windows-reactor](https://github.com/microsoft/windows-rs)
(a declarative, React-like framework), and it links the shared **`punktfunk-core`** directly to speak
the fast **`punktfunk/1`** protocol.
## Features
- **Hardware decode, GPU present** — FFmpeg HEVC with a **D3D11VA zero-copy path** (decoder and
presenter share one D3D11 device; NV12/P010 textures sampled straight into a `SwapChainPanel`
composition swapchain), with a robust software-decode fallback.
- **HDR10** — advertise 10-bit/HDR, detect PQ in-band, and flip the swapchain to `R10G10B10A2` +
ST.2084 with HDR10 metadata.
- **Your display's native mode** — the host builds a virtual display at exactly your WxH@Hz.
- **Audio both ways** — WASAPI render + mic capture.
- **Full controller support** — SDL3 gamepads with rumble, lightbar, and DualSense feedback.
- **Find hosts automatically** — mDNS discovery lists hosts on your LAN, alongside saved and manual
entries. First connect does a one-time **SPAKE2 PIN pairing** (or TOFU on trusted LANs), then
reconnects on a pinned identity.
- **Polished shell** — host cards, settings (resolution / refresh / decoder / bitrate / HDR / mic),
a status-chip stream HUD, and the full trust surface. Stream input uses Win32 low-level hooks with
a Ctrl+Alt+Shift+Q capture toggle.
Builds and ships for both **x64** and **ARM64** as a signed **MSIX**.
## Get it
Install the signed MSIX from the package registry — see
**[docs.punktfunk.unom.io/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
A stock [Moonlight](https://moonlight-stream.org/) client also works over GameStream if you prefer.
## Build from source
Windows-only (the crate builds as a stub on other platforms so the workspace stays green). You need
the MSVC toolchain, an `FFMPEG_DIR` FFmpeg tree, and CMake (SDL3 builds from source). windows-reactor's
`build.rs` downloads the Windows App SDK NuGets and needs `CARGO_WORKSPACE_DIR` set.
```sh
cargo build -p punktfunk-client-windows --target x86_64-pc-windows-msvc
# CLI paths for testing (no window):
punktfunk-client --discover # list hosts on the LAN
punktfunk-client --headless --connect host[:port] [--pin HEX] # connect, count frames, print stats
```
> `CARGO_HOME` must be an ASCII path — non-ASCII characters break SDL3's MSVC precompiled-header
> build. Packaging (MSIX manifest, signing) lives in [`packaging/`](packaging/).
## Layout
```
src/
main.rs · app.rs entry point + CLI paths; WinUI 3 shell (windows-reactor)
present.rs · gpu.rs SwapChainPanel D3D11 composition swapchain; shared D3D11 device
video.rs FFmpeg HEVC decode (D3D11VA zero-copy + software fallback)
audio.rs WASAPI render + mic capture
gamepad.rs SDL3 controllers + rumble/lightbar/DualSense feedback
input.rs Win32 low-level keyboard/mouse hooks → host input
session.rs session lifecycle over the NativeClient connector
trust.rs · discovery.rs persistent identity, TOFU/PIN pairing, mDNS browse
packaging/ MSIX manifest, signing, pack script
```
## Related
- **[Documentation](https://docs.punktfunk.unom.io)** — quick start, pairing, troubleshooting
- **[Project README](../../README.md)** — the host, the other clients, and how it all fits together
+16
View File
@@ -0,0 +1,16 @@
# pf-driver-proto
The shared **host ↔ driver binary contract** for punktfunk's Windows **pf-vdisplay** virtual display —
the control IOCTLs and the IDD-push frame transport, defined exactly once.
It's a path dependency of **both** the host workspace ([`crates/punktfunk-host`](../punktfunk-host))
and the out-of-workspace driver workspace ([`packaging/windows/drivers/`](../../packaging/windows)),
so it must resolve identically from either build graph. That's why it's deliberately self-contained:
`no_std` (+ alloc), platform-neutral (GUID/LUID are plain integers each side converts to its own OS
type), and free of `*.workspace = true` inheritance.
Defining every wire struct here — with `const` size/offset asserts and `bytemuck` round-trips — turns
host↔driver ABI drift into a **compile error** instead of a silent frame or IOCTL corruption.
See the crate root ([`src/`](src/)) for the wire types; the Windows virtual-display design is in
[`design/windows-virtual-display-rust-port.md`](../../design/windows-virtual-display-rust-port.md).
+61
View File
@@ -0,0 +1,61 @@
# punktfunk-core
The **shared protocol core** — the one place where punktfunk's transport, forward error correction,
and crypto live. It's linked into the [host](../punktfunk-host/README.md) and every native client, so
there's exactly one implementation of the wire format everywhere.
Written in Rust with **no async on the per-frame path** (native threads only). It exposes both a
normal Rust API and a **stable, versioned C ABI**, so the Swift and Kotlin clients — and any C
embedder — link the same code as the Rust ones.
## What's in here
- **Transport & session** (`session.rs`, `transport/`, `packet.rs`) — the `punktfunk/1` data plane
over raw UDP: packetization, reassembly (with attacker-bounded limits), pacing, and socket tuning.
- **FEC** (`fec/`) — the wall-breaker. Two codes:
- **GF(2⁸)** classic ReedSolomon with the *Cauchy* generator matrix — byte-identical to the
`nanors` library Moonlight uses, so our parity is decodable by a stock Moonlight client.
- **GF(2¹⁶) Leopard-RS** (SIMD, O(n log n)) — up to 65535 shards/block, which removes the ~1 Gbps
FEC ceiling. `punktfunk/1` negotiates this one.
- **Crypto** (`crypto.rs`) — AES-128-GCM session encryption with per-direction nonce salts and
sequence-as-AAD; SPAKE2 PIN pairing lives behind the `quic` feature.
- **QUIC control plane** (`quic.rs`, `client.rs`, feature `quic`) — the Hello/Welcome/Start handshake,
cert pinning/TOFU, reverse audio, and the embeddable `NativeClient` connector. This is the **only**
place `tokio`/`quinn` are allowed; the feature is **off by default** so the core stays runtime-free.
- **C ABI** (`abi.rs`) — the versioned surface (`punktfunk_abi_version()`, `PunktfunkConfig` carrying
its own `struct_size`) that generates [`include/punktfunk_core.h`](../../include/punktfunk_core.h)
via cbindgen at build time.
## Build outputs
The crate builds three ways at once (`crate-type = ["lib", "cdylib", "staticlib"]`):
| Output | Used by |
|--------|---------|
| `lib` (rlib) | the host, probe, and tools link it as a normal Rust crate |
| `cdylib` (`.so`/`.dylib`) | the Swift / Kotlin clients via the C ABI |
| `staticlib` (`.a`) | the C test harness and static embedding |
## Test
```sh
cargo test -p punktfunk-core # unit + proptest + loopback
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip proof
```
## Design invariants (do not regress)
- **One core, linked everywhere** — protocol/FEC/crypto live only here, behind the stable C ABI.
- **No async on the hot path** — the per-frame pipeline is native threads only; `quic` (tokio/quinn)
is control-plane only, feature-gated, off by default.
- **Security hardening stays intact** — the reassembler bounds attacker-controlled fields before
allocating; AES-GCM keeps per-direction nonce salts + seq-as-AAD; the ABI checks `struct_size`.
Regression tests exist — keep them green.
## Related
- **[`punktfunk-host`](../punktfunk-host/README.md)** — the streaming host built on this core
- **[Clients](../../clients/)** — the apps that link this core over the C ABI (or directly, in Rust)
- **[`design/implementation-plan.md`](../../design/implementation-plan.md)** — why GF(2¹⁶) FEC, the
latency budget, and the architecture thesis
+90
View File
@@ -0,0 +1,90 @@
# punktfunk-host
The **streaming host** — the program you run on the machine whose desktop or games you want to
stream. For each client that connects, it spins up a **virtual display sized to that device**,
captures it on the GPU, encodes with hardware NVENC/VAAPI/AMF/QSV, and sends it out over a
low-latency transport — no physical monitor, no letterboxing, no rearranging your real screens.
It speaks two protocols from **one process**:
- **GameStream** — so any [Moonlight](https://moonlight-stream.org/) / Artemis client works day one.
- **`punktfunk/1`** — punktfunk's own faster protocol (QUIC control plane, GF(2¹⁶) FEC + AES-GCM data
plane) that the native clients use.
Runs on **Linux** (the primary, most battle-tested path) and **Windows** (x64). The shared protocol,
FEC, and crypto live in [`punktfunk-core`](../punktfunk-core/README.md); this crate is everything
platform-facing around it.
## What it does
- **Per-client virtual displays at the exact WxH@Hz.** Linux uses per-compositor backends — **KWin**,
**gamescope**, **Mutter**, and **Sway/wlroots**; Windows uses its own all-Rust IddCx virtual display,
even on the secure desktop (UAC / lock screen).
- **GPU zero-copy capture → encode.** dmabuf → CUDA/Vulkan → NVENC on Linux; DXGI/WGC → GPU encode on
Windows. Encoders auto-select by GPU vendor: **NVENC** (NVIDIA), **VAAPI** (Linux AMD/Intel),
**AMF/QSV** (Windows AMD/Intel), or software H.264 as a floor. HDR/10-bit and HEVC 4:4:4 supported.
- **Input injection.** Mouse/keyboard (libei / gamescope EIS / wlr / Windows SendInput) and virtual
**gamepads** — Xbox 360/One, DualSense, DualShock 4 — with rumble and HID feedback back-channels.
- **Audio both ways.** Opus audio host→client, plus a virtual microphone the client can talk into.
- **Trust & discovery.** A persistent host identity, **SPAKE2 PIN pairing** (default) or TOFU, and
mDNS auto-advertisement so clients find the host without typing an IP.
- **Management API + web console.** A REST API (`mgmt.rs`, OpenAPI at
[`api/openapi.json`](../../api/openapi.json)) drives status, paired devices, and on-demand pairing;
the browser UI is in [`web/`](../../web/README.md).
## Run it
`punktfunk-host serve` runs inside your desktop session. Bare `serve` is the **secure native-only
default** (`punktfunk/1` + the management API); add `--gamestream` on a trusted LAN to also accept
stock Moonlight clients.
```sh
# Linux, from the repo root (see the repo README "Running on this box" for the headless recipe):
cargo run -rp punktfunk-host -- serve # native-only (secure default)
cargo run -rp punktfunk-host -- serve --gamestream # + Moonlight compatibility
```
Then pair from the web console (`https://<host-ip>:3000`) or the client app.
Most people should install a **package** rather than run from source — see
[`packaging/`](../../packaging/README.md) (apt · rpm/COPR/bootc · Arch/sysext · Windows installer) and
the per-platform guides at **[docs.punktfunk.unom.io/docs/install](https://docs.punktfunk.unom.io/docs/install)**.
### Subcommands
| Command | Purpose |
|---------|---------|
| `serve` | The host (native `punktfunk/1` + mgmt API; `--gamestream` adds Moonlight). |
| `punktfunk1-host` | Standalone native-protocol listener for testing/measurement (`--source virtual`, `--max-sessions`). |
| `openapi` | Print the management-API OpenAPI spec (regenerates `api/openapi.json`). |
| `library` | Inspect the multi-store game library. |
| `service` · `driver` · `web` | Windows: SCM service, driver install, bundled web console. |
| `*-test` / `*-selftest` / `*-probe` | Diagnostics (input, zero-copy, HDR, compositor, gamepads). |
`--help` lists them all.
## Layout
```
src/
main.rs CLI + subcommand dispatch
config.rs · session_plan.rs · session_tuning.rs · pipeline.rs session setup + the frame pipeline
vdisplay/ per-compositor virtual outputs (kwin · gamescope · mutter · wlroots)
capture/ · capture.rs screen/dmabuf capture (+ Windows DXGI/WGC)
encode/ · encode.rs per-GPU encoders (nvenc · vaapi · ffmpeg_win (AMF/QSV) · sw)
zerocopy/ dmabuf → CUDA → NVENC bridges (EGL/GL tiled, Vulkan LINEAR)
inject/ · inject.rs input backends (libei · wlr · uinput gamepads · UHID DualSense/DS4)
audio/ · audio.rs Opus out + virtual mic (PipeWire / WASAPI)
gamestream/ Moonlight compat: nvhttp · pairing · rtsp · control · stream · gamepad · apps
punktfunk1.rs the native punktfunk/1 host (QUIC control + native-thread UDP data plane)
mgmt.rs · native_pairing.rs · stats_recorder.rs management API, pairing, perf capture
hdr.rs · library.rs HDR metadata; multi-store game library
linux/ · windows/ platform-confined backends
```
## Related
- **[`punktfunk-core`](../punktfunk-core/README.md)** — the shared protocol · FEC · crypto core
- **[Clients](../../clients/)** — the apps that connect (Apple · Linux · Windows · Android · probe)
- **[Packaging](../../packaging/README.md)** & **[docs](https://docs.punktfunk.unom.io)** — install & operate
- **[`design/`](../../design/README.md)** — architecture rationale and deep-dive plans
+3 -3
View File
@@ -3,9 +3,9 @@
The punktfunk documentation site: [Fumadocs](https://fumadocs.dev) on The punktfunk documentation site: [Fumadocs](https://fumadocs.dev) on
[TanStack Start](https://tanstack.com/start) (Vite + Nitro/bun preset). [TanStack Start](https://tanstack.com/start) (Vite + Nitro/bun preset).
Content lives in [`content/docs/`](content/docs) as `.md`/`.mdx`. Several pages are imported Content lives in [`content/docs/`](content/docs) as `.md`/`.mdx`. This site is the source of truth
verbatim from the repo's `docs/` design notes (with added frontmatter); edit those there or for the **user-facing** guides; repo-internal design rationale lives in
here as the docs site becomes the source of truth. [`../design/`](../design/README.md).
## API reference ## API reference
+1 -1
View File
@@ -73,7 +73,7 @@ rpm-ostree install punktfunk && systemctl reboot
systemctl reboot systemctl reboot
``` ```
## Option B — bootc (image-based, atomic) ## Option C — bootc (image-based, atomic)
Layer punktfunk into a Bazzite image once, then rebase any number of hosts onto it — no Layer punktfunk into a Bazzite image once, then rebase any number of hosts onto it — no
per-host drift. See `bootc/Containerfile`: per-host drift. See `bootc/Containerfile`:
@@ -1,26 +1,27 @@
# pf-dualsense — virtual DualSense UMDF2 HID minidriver (M0 spike) # pf-dualsense — virtual DualSense UMDF2 HID minidriver
A self-authored **Rust UMDF2 HID minidriver** that presents a virtual Sony **DualSense** A self-authored **Rust UMDF2 HID minidriver** that presents a virtual Sony **DualSense**
(VID `054C` / PID `0CE6`) to Windows, so games drive adaptive triggers / lightbar / rumble — (VID `054C` / PID `0CE6`) to Windows, so games drive adaptive triggers / lightbar / rumble —
capabilities ViGEm structurally cannot deliver. This is the M0 feasibility spike for rich capabilities ViGEm structurally cannot deliver. It's how the punktfunk Windows host gives a client's
controller support in the punktfunk Windows host. DualSense a near-native feel with **no external gamepad dependencies** (no ViGEmBus).
## Status (2026-06-21) Shipping: the driver is one member of the in-tree driver workspace
([`packaging/windows/drivers/`](../../README.md)), built from source in CI, and bundled +
`pnputil`-installed by the Windows host [installer](../../README.md). The host feeds it over a shared
memory channel from `crates/punktfunk-host/src/inject/dualsense_windows.rs`. The same UMDF driver also
serves the **DualShock 4** identity per a `device_type` byte the host stamps.
**Load + recognition: DONE.** A self-signed build **loads under Secure Boot ON** and enumerates as a This README captures the driver-authoring lore — the bugs and the signing recipe that make a
genuine DualSense HID game controller (`Status: OK`, VID `054C`, 273-byte DualSense report descriptor, self-signed UMDF HID driver actually load. The authoritative build/sign/package flow (CI + Inno Setup)
PID `0CE6` via `GET_DEVICE_ATTRIBUTES`). Validated live on the RTX box (`192.168.1.173`, Win11 25H2). lives in the [Windows host packaging README](../../README.md).
**Remaining:** the real-game `0x02` adaptive-trigger gate (Cyberpunk 2077 on the interactive desktop → ## Build workspace
confirm `[pf-ds] *** OUTPUT ...` in the driver log), then wire into the host (M1+).
## This is a reference snapshot This crate builds as a member of the [`packaging/windows/drivers/`](../../drivers) workspace, which
uses the published **crates.io `wdk`/`wdk-sys`/`wdk-build`** (0.4/0.5) — not the old dev-box
The crate's `Cargo.toml` uses path-deps into `microsoft/windows-drivers-rs` `windows-drivers-rs` path-deps. It's a separate cargo workspace from the main tree because driver
(`../../crates/wdk{,-sys,-build}`), so it builds **inside a `windows-drivers-rs` checkout's crates are cdylibs built with the WDK toolchain on Windows only; it path-deps the shared ABI crate
`examples/` dir**, not standalone in this repo. On the dev box it lives at [`crates/pf-driver-proto`](../../../../crates/pf-driver-proto/README.md).
`C:\Users\Public\m0\windows-drivers-rs\examples\pf-dualsense`. These files are checked in for
version control / portability of the spike.
## Build / sign / install recipe (the one that actually loads) ## Build / sign / install recipe (the one that actually loads)
@@ -76,9 +77,10 @@ silently breaks them:
zero requests → `EvtIoDeviceControl` never fires → no HID handshake → ~5 s timeout → zero requests → `EvtIoDeviceControl` never fires → no HID handshake → ~5 s timeout →
`CM_PROB_FAILED_START`. Set to `u32::MAX`. `CM_PROB_FAILED_START`. Set to `u32::MAX`.
## Known limitations ## Notes
- Uses **statics, not per-device WDF contexts** → only one device instance per WUDFHost works. - **Multi-pad** works via `UmdfHostProcessSharing=ProcessSharingDisabled` — each pad gets its own
Multi-instance needs proper device contexts. WUDFHost (so the per-instance statics don't collide), and the driver reads its pad index from the
- Port of the WDK `vhidmini2` UMDF2 sample; DualSense identity + 273-byte descriptor + feature blobs device Location (`WdfDeviceAllocAndQueryProperty`) to map its own `*-shm-<index>` channel.
`0x05`/`0x09`/`0x20` from `crates/punktfunk-host/src/inject/dualsense.rs`. - Port of the WDK `vhidmini2` UMDF2 sample; the DualSense identity + 273-byte descriptor + feature
blobs `0x05`/`0x09`/`0x20` come from `crates/punktfunk-host/src/inject/dualsense.rs`.
+15
View File
@@ -0,0 +1,15 @@
# latency-probe
A **glass-to-glass latency** measurement tool (design/implementation-plan §10): it renders a
timestamp/QR on the host, reads it back off the client's capture (or a photodiode, for true photons),
and tracks p50/p99 — so latency regressions are quantifiable rather than felt.
> **Status: scaffold.** The measurement harness is stubbed out and not yet wired to a live pipeline.
> For latency numbers today, use the per-frame **capture→…→reassembled** percentiles the
> [`probe`](../../clients/probe/README.md) client reports over a real `punktfunk/1` session.
```sh
cargo run -p latency-probe # from the repo root
```
Companion to [`loss-harness`](../loss-harness/README.md) (FEC loss sweep).
+16
View File
@@ -0,0 +1,16 @@
# loss-harness
A **FEC loss-resilience sweep** for [`punktfunk-core`](../../crates/punktfunk-core/README.md). It
drives access units through the in-process loopback at increasing packet-loss rates — for **both** FEC
schemes (GF(2⁸) and GF(2¹⁶)) — and reports how many frames survive.
It's a pure-software stand-in for `tc netem`: no network, no root, runs anywhere `punktfunk-core`
builds. Use it to sanity-check the FEC before reaching for the real `punktfunk/1` harness (which adds
`tc netem` jitter/reorder on the UDP path).
```sh
cargo run -p loss-harness # from the repo root
```
Part of the measurement tooling (design/implementation-plan §10), alongside
[`latency-probe`](../latency-probe/README.md).