docs: update README + docs site for public readiness
apple / swift (push) Successful in 56s
ci / rust (push) Successful in 1m37s
ci / web (push) Successful in 31s
ci / docs-site (push) Successful in 40s
android / android (push) Successful in 3m19s
deb / build-publish (push) Failing after 1m9s
decky / build-publish (push) Successful in 22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m21s
ci / bench (push) Successful in 4m45s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 26s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m22s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m25s
apple / swift (push) Successful in 56s
ci / rust (push) Successful in 1m37s
ci / web (push) Successful in 31s
ci / docs-site (push) Successful in 40s
android / android (push) Successful in 3m19s
deb / build-publish (push) Failing after 1m9s
decky / build-publish (push) Successful in 22s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 2m21s
ci / bench (push) Successful in 4m45s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 26s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m22s
docker / deploy-docs (push) Successful in 18s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Successful in 10m25s
Refresh the README and documentation for public visitors: - README: public-facing rewrite with accurate status for all four native clients (macOS, Linux, Windows, Android) and the Windows host. - docs site: fix stale client status (Android is a full client, not a scaffold; Windows client is stage-1 complete + signed MSIX), add the missing Android client section, correct "which client" guidance. - Windows host: corrected from "deferred/scoped" to implemented & shipping (NVIDIA-only, x64-only) across windows-host, roadmap, status, requirements, running-as-a-service, and the README. - Remove internal infrastructure from public docs (box names, private IPs, SSH/token commands, deploy topology); rewrite status.md as a public project-status page; sanitize ci.md and implementation-plan.md. - Update clients/android and clients/apple READMEs to current state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,98 +1,142 @@
|
||||
# punktfunk
|
||||
|
||||
*A ground-up low-latency desktop streaming stack, built Linux-first, with a shared Rust
|
||||
protocol core and native clients per platform.*
|
||||
**Low-latency desktop and game streaming, Linux-first.** Run the host on a Linux machine — or a
|
||||
Windows PC — with an NVIDIA GPU, connect from a Mac, PC, phone, tablet, or TV, and stream your desktop
|
||||
or games — each device at its **own native resolution and refresh rate**, over your local network.
|
||||
|
||||
`punktfunk` is a placeholder codename. The bet: ship a **Linux virtual-display streaming
|
||||
host** that speaks the existing Moonlight protocol (every Moonlight/Artemis client works
|
||||
day one), then break the ~1 Gbps FEC wall with a **GF(2¹⁶) Leopard-RS** transport as a
|
||||
negotiated extension. See [`docs/implementation-plan.md`](docs/implementation-plan.md).
|
||||
📖 **Documentation: [docs.punktfunk.unom.io](https://docs.punktfunk.unom.io)** — start with
|
||||
[How It Works](https://docs.punktfunk.unom.io/docs/how-it-works) or the
|
||||
[Quick Start](https://docs.punktfunk.unom.io/docs/quickstart).
|
||||
|
||||
punktfunk pairs a **virtual-display streaming host** with native clients on every platform. It speaks
|
||||
the existing **GameStream** protocol, so any [Moonlight](https://moonlight-stream.org/) client works
|
||||
day one — and adds its own faster **`punktfunk/1`** protocol that breaks the ~1 Gbps FEC wall with a
|
||||
**GF(2¹⁶) Leopard-RS** transport. A single shared **Rust core** (`punktfunk-core`) holds the
|
||||
protocol, FEC, and crypto, linked into the host and every client over a stable C ABI.
|
||||
|
||||
## What makes it different
|
||||
|
||||
- **Your device's exact mode.** For each client that connects, the host spins up a virtual display
|
||||
sized to that device — 1080p60 to a laptop, 1440p120 to a desktop, 4K to a TV, all at once. No
|
||||
letterboxing, no scaling, no rearranging your real monitors.
|
||||
- **Low latency, GPU end to end.** Frames go straight from the compositor to the NVENC encoder with
|
||||
zero CPU copies (dmabuf → CUDA/Vulkan → NVENC), over a transport tuned for responsiveness rather
|
||||
than throughput. Stable 240 fps at 5120×1440; sub-millisecond capture-to-reassembly on a LAN.
|
||||
- **Works with what you already have.** Any Moonlight/Artemis client connects over GameStream — and
|
||||
native apps for macOS, Linux, Windows, and Android use the lower-latency `punktfunk/1` protocol.
|
||||
- **Secure by default.** Hosts require a one-time SPAKE2 **PIN pairing**; after that, devices
|
||||
reconnect on a pinned identity. No accounts, no cloud. Hosts auto-advertise over mDNS, so clients
|
||||
find them on the network without typing an IP.
|
||||
|
||||
## Status
|
||||
|
||||
| Milestone | State |
|
||||
| Component | State |
|
||||
|-----------|-------|
|
||||
| **Core — `punktfunk-core` + C ABI** | ✅ done & hardened (FEC, packetization, AES-GCM, session, adversarial-review fixes, `punktfunk_core.h`) |
|
||||
| **GameStream host → stock Moonlight** | ✅ live end-to-end: pairing, RTSP, audio, per-client virtual output at native res, GPU zero-copy NVENC, gamepads |
|
||||
| **Native protocol — `punktfunk/1`** | ✅ validated live: QUIC control + GF(2¹⁶) FEC/AES data plane, SPAKE2 PIN pairing, mid-stream mode renegotiation |
|
||||
| **Native clients — decode + present** | 🟡 macOS first light: AnnexB→VideoToolbox HEVC on glass + input/pairing over `punktfunk/1` (`clients/apple`); iOS + presenter next |
|
||||
| **Web console + management API** | ✅ TanStack web console (`web/`) over the OpenAPI mgmt API: host status, paired devices, on-demand native pairing (arm → show PIN) |
|
||||
| **Core** — `punktfunk-core` + C ABI (protocol · FEC · crypto · QUIC) | ✅ Complete & hardened |
|
||||
| **GameStream host** → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads |
|
||||
| **Native protocol** — `punktfunk/1` | ✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation |
|
||||
| **Windows host** (NVIDIA, x64) | 🟡 Implemented & shipping as a signed installer (DXGI capture · SudoVDA virtual display · NVENC · WASAPI · ViGEm); NVIDIA-only, newer than the Linux host |
|
||||
| **macOS / iOS / tvOS client** (`clients/apple`) | ✅ Streaming live: VideoToolbox decode, controllers incl. DualSense, discovery, pairing, speed test |
|
||||
| **Linux client** (`clients/linux`, GTK4) | ✅ Streaming live: FFmpeg + VAAPI zero-copy decode, PipeWire audio, SDL3 controllers; ships as Flatpak/apt/rpm/Arch |
|
||||
| **Android client** (`clients/android`, phone + TV) | ✅ Streaming live: AMediaCodec decode + HDR10, Oboe audio, controllers, discovery, pairing |
|
||||
| **Windows client** (`clients/windows`, WinUI 3) | 🟡 Stage 1 complete, ships as signed MSIX (x64 + ARM64); D3D11VA decode + HDR present pending on-glass validation |
|
||||
| **Web console + management API** (`web/`) | ✅ TanStack console over the OpenAPI mgmt API: host status, paired devices, on-demand PIN pairing |
|
||||
|
||||
The **GameStream host works with a stock Moonlight client** — validated live on NVIDIA
|
||||
(RTX 5070 Ti & RTX 4090, driver 595): trust-on-first-use pairing that persists, an app
|
||||
catalog, RTSP/ENet/audio, and **video at the client's exact resolution and refresh** via a
|
||||
per-session virtual output (KWin, gamescope, Mutter, Sway backends), encoded with GPU
|
||||
**zero-copy** (dmabuf → CUDA/Vulkan → NVENC) at up to 5120×1440@240. The native
|
||||
**`punktfunk/1`** protocol adds a QUIC control plane and a GF(2¹⁶) Leopard-FEC + AES-GCM data
|
||||
plane (p50 ~0.8 ms capture→reassembled at 720p120). Its trust model is **SPAKE2 PIN pairing by
|
||||
default** — a new host requires the PIN ceremony; trust-on-first-use is an explicit host opt-in
|
||||
(`punktfunk1-host --allow-tofu` / `serve --open`, advertised as `pair=optional`) for fully trusted LANs. Both
|
||||
run from **one process** (`serve --native`), managed through a REST API + web console. Builds
|
||||
against FFmpeg 7 or 8; deployed live on Bazzite. Full status: [`CLAUDE.md`](CLAUDE.md);
|
||||
roadmap, setup guides & progress: the docs site ([`docs-site/`](docs-site) — Fumadocs;
|
||||
`bun run dev`), with the canonical [roadmap](docs-site/content/docs/roadmap.md) and
|
||||
[status](docs-site/content/docs/status.md) there. Design notes stay in [`docs/`](docs).
|
||||
The **GameStream host works with a stock Moonlight client** — validated live on NVIDIA hardware
|
||||
(RTX 5070 Ti, RTX 4090): PIN pairing that persists across restarts, an app catalog, RTSP/ENet/audio,
|
||||
and **video at the client's exact resolution and refresh** via a per-session virtual output (KWin,
|
||||
gamescope, Mutter, and Sway/wlroots backends), encoded with GPU **zero-copy** (dmabuf → CUDA/Vulkan →
|
||||
NVENC) up to 5120×1440@240. The native **`punktfunk/1`** protocol adds a QUIC control plane and a
|
||||
GF(2¹⁶) Leopard-FEC + AES-GCM data plane (p50 ~0.8 ms capture→reassembled at 720p120), with
|
||||
mid-stream mode renegotiation and a wall-clock skew handshake so latency stays valid across machines.
|
||||
Both protocols run from **one process** (`punktfunk-host serve --native`) and are managed through a
|
||||
REST API and web console. Builds against FFmpeg 7 or 8.
|
||||
|
||||
## Install (host)
|
||||
Full milestone status: **[docs.punktfunk.unom.io/docs/status](https://docs.punktfunk.unom.io/docs/status)** ·
|
||||
roadmap: **[/docs/roadmap](https://docs.punktfunk.unom.io/docs/roadmap)**.
|
||||
|
||||
The package registries are the real distribution channel — pick your distro and run one command.
|
||||
Per-distro setup (add the repo, first-run, web console) lives in the linked READMEs.
|
||||
## Install the host
|
||||
|
||||
| Distro | One-command happy path | Details |
|
||||
|--------|------------------------|---------|
|
||||
| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-host` *(after adding the repo)* | [`packaging/debian/README.md`](packaging/debian/README.md) |
|
||||
| **Fedora / Bazzite** (rpm-ostree) | `rpm-ostree install punktfunk punktfunk-web` *(after adding the repo; or the bootc image)* | [`packaging/rpm/README.md`](packaging/rpm/README.md) |
|
||||
| **Arch / Steam Deck** (PKGBUILD / sysext) | `makepkg -si` *(Arch)* · sysext `.raw` *(SteamOS/Deck)* | [`packaging/arch/README.md`](packaging/arch/README.md) |
|
||||
Pick your platform and install from its package registry — the per-platform guide covers adding the
|
||||
repo, first run, and the web console. The Linux host is the primary, most battle-tested path; a
|
||||
Windows host (NVIDIA-only) also ships as a signed installer.
|
||||
|
||||
`punktfunk-host` is the streaming host; `punktfunk-web` is the browser console (pairing + status);
|
||||
`punktfunk-client` is the GTK4 desktop client (also shipped via apt/RPM/Arch/Flatpak). After install,
|
||||
run `punktfunk-host serve --native` inside your desktop session, then pair from the web console.
|
||||
| Platform | Install | Guide |
|
||||
|--------|---------|-------|
|
||||
| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-host` *(after adding the repo)* | [Ubuntu — GNOME](https://docs.punktfunk.unom.io/docs/ubuntu-gnome) · [KDE](https://docs.punktfunk.unom.io/docs/ubuntu-kde) |
|
||||
| **Fedora / Bazzite** (rpm-ostree) | `rpm-ostree install punktfunk punktfunk-web` *(or the bootc image)* | [Fedora — KDE](https://docs.punktfunk.unom.io/docs/fedora-kde) · [Bazzite](https://docs.punktfunk.unom.io/docs/bazzite) |
|
||||
| **Arch / Steam Deck** (PKGBUILD / sysext) | `makepkg -si` *(Arch)* · sysext `.raw` *(SteamOS)* | [packaging/arch](packaging/arch/README.md) |
|
||||
| **Windows** (NVIDIA, x64) | signed `setup.exe` from the package registry | [Windows Host](https://docs.punktfunk.unom.io/docs/windows-host) |
|
||||
|
||||
Building from source (below) is a fallback.
|
||||
`punktfunk-host` is the streaming host; `punktfunk-web` is the browser console (pairing + status).
|
||||
After install, run `punktfunk-host serve --native` inside your desktop session, then pair from the web
|
||||
console. Full instructions: **[docs.punktfunk.unom.io/docs/install](https://docs.punktfunk.unom.io/docs/install)**.
|
||||
|
||||
## Layout
|
||||
## Connect a client
|
||||
|
||||
```
|
||||
crates/
|
||||
punktfunk-core/ protocol · FEC · pacing · crypto · quic — the C ABI (lib + cdylib + staticlib)
|
||||
punktfunk-host/ Linux host: vdisplay · capture · encode · inject · gamestream · punktfunk1 · mgmt · native_pairing
|
||||
clients/
|
||||
probe/ punktfunk/1 reference/probe client (headless test + latency measurement)
|
||||
linux/ windows/ native desktop clients (Rust: GTK4 / WinUI 3, link punktfunk-core directly)
|
||||
apple/ android/ Swift (macOS+iOS) · Kotlin app + native/ Rust JNI core
|
||||
decky/ Steam Deck Decky plugin
|
||||
web/ TanStack web console (host status · paired devices · pairing) over the mgmt API
|
||||
packaging/ Fedora/Bazzite RPM · bootc image · COPR (see packaging/bazzite/README.md)
|
||||
include/punktfunk_core.h cbindgen-generated C header (checked in)
|
||||
tools/{latency-probe,loss-harness}/ measurement (plan §10)
|
||||
docs/{implementation-plan,roadmap,windows-host,dualsense-haptics}.md
|
||||
```
|
||||
| Streaming to… | Use |
|
||||
|---|---|
|
||||
| Mac, iPhone, iPad, Apple TV | The **Apple app** (`clients/apple`) — also on TestFlight |
|
||||
| Linux desktop / laptop, Steam Deck | **`punktfunk-client`** (Flatpak / apt / rpm / Arch) |
|
||||
| Android phone or TV | The **Android app** (`clients/android`) |
|
||||
| Windows | Native **`punktfunk-client`** (signed MSIX) or **Moonlight** |
|
||||
| Anything else (browser, old phone, smart TV) | **Moonlight** over GameStream |
|
||||
|
||||
Each client discovers hosts on the network automatically and does a one-time
|
||||
[PIN pairing](https://docs.punktfunk.unom.io/docs/pairing). Per-device install steps:
|
||||
**[/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**.
|
||||
|
||||
## Build & test (from source)
|
||||
|
||||
For development, or as an install fallback where no package is available:
|
||||
|
||||
```sh
|
||||
cargo build --workspace # green on Linux and macOS
|
||||
cargo build --workspace # the Rust core, host, Linux client, and probe (Linux & macOS)
|
||||
cargo test --workspace # unit + loopback + proptest + C ABI harness
|
||||
cargo clippy --workspace --all-targets
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
cargo fmt --all --check
|
||||
|
||||
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
|
||||
bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link+round-trip proof
|
||||
bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip proof
|
||||
```
|
||||
|
||||
The C header regenerates from `crates/punktfunk-core/src/abi.rs` on every build (cbindgen via
|
||||
`build.rs`) into `include/punktfunk_core.h`.
|
||||
`build.rs`) into `include/punktfunk_core.h`. The Apple, Android, and Windows clients have their own
|
||||
toolchains (Xcode/`swift build`, Gradle, and `cargo` on the MSVC target) — see each client's README
|
||||
and the [docs site](https://docs.punktfunk.unom.io).
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
crates/
|
||||
punktfunk-core/ protocol · FEC · pacing · crypto · QUIC control plane — the C ABI (lib + cdylib + staticlib)
|
||||
punktfunk-host/ Linux host: virtual displays · capture · encode · input · GameStream · punktfunk/1 · mgmt
|
||||
clients/
|
||||
apple/ macOS / iOS / tvOS app (Swift · VideoToolbox · Metal · GameController)
|
||||
linux/ Linux desktop app (Rust · GTK4/libadwaita · FFmpeg/VAAPI · PipeWire · SDL3)
|
||||
windows/ Windows desktop app (Rust · WinUI 3 · D3D11 · WASAPI · SDL3)
|
||||
android/ Android phone + TV app (Kotlin · Rust JNI core · AMediaCodec · Oboe)
|
||||
probe/ headless reference / measurement client for punktfunk/1
|
||||
decky/ Steam Deck Decky plugin
|
||||
web/ web console (TanStack) over the management API — status · devices · pairing
|
||||
packaging/ apt · rpm / COPR · Arch · Flatpak · Bazzite bootc image
|
||||
docs-site/ public documentation site (Fumadocs) — https://docs.punktfunk.unom.io
|
||||
docs/ design notes & deep-dive plans
|
||||
include/punktfunk_core.h cbindgen-generated C header (checked in)
|
||||
tools/ latency-probe · loss-harness (measurement)
|
||||
```
|
||||
|
||||
## Design invariants
|
||||
|
||||
- **One core, linked everywhere.** Protocol/FEC/crypto/pacing live in `punktfunk-core` exactly
|
||||
once, exposed over a stable, versioned C ABI (`punktfunk_abi_version()`, `PunktfunkConfig`
|
||||
carries its own `struct_size`).
|
||||
- **No async on the hot path.** The per-frame pipeline uses native threads only;
|
||||
`tokio`/`quinn` are gated behind the off-by-default `quic` feature (control plane only).
|
||||
- **FEC is the wall-breaker.** GF(2⁸) (≤255 shards/block) for Moonlight compat;
|
||||
GF(2¹⁶) (≤65535 shards/block, SIMD, O(n log n)) to push past ~1 Gbps.
|
||||
- **One core, linked everywhere.** Protocol, FEC, and crypto live in `punktfunk-core` exactly once,
|
||||
exposed over a stable, versioned C ABI (`punktfunk_abi_version()`, `PunktfunkConfig` carries its own
|
||||
`struct_size`). Every native client links the same core.
|
||||
- **No async on the hot path.** The per-frame pipeline uses native threads only; `tokio`/`quinn` are
|
||||
gated behind the off-by-default `quic` feature (control plane only).
|
||||
- **Native client resolution, no scaling.** Each session gets a virtual output at exactly the
|
||||
client's WxH@Hz; each compositor keeps its own backend behind a shared `VirtualDisplay` trait.
|
||||
- **FEC is the wall-breaker.** GF(2⁸) (≤255 shards/block) for Moonlight compatibility; GF(2¹⁶)
|
||||
(≤65535 shards/block, SIMD, O(n log n)) for `punktfunk/1` to push past ~1 Gbps.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
+26
-15
@@ -19,15 +19,19 @@ The single seam is `io.unom.punktfunk.kit.NativeBridge` ⇄ `Java_io_unom_punktf
|
||||
## Layout
|
||||
|
||||
```
|
||||
clients/android/native/ Rust cdylib (workspace member)
|
||||
src/lib.rs JNI_OnLoad + abiVersion/coreVersion (native-link proof)
|
||||
src/session.rs session handle lifecycle (connect/close); plane pumps = TODO
|
||||
clients/android/native/ Rust cdylib (workspace member) — links punktfunk-core directly
|
||||
src/lib.rs JNI seam (connect/pair, input, plane getters, abi/core version)
|
||||
src/session.rs session lifecycle + plane pumps
|
||||
src/decode.rs AnnexB → AMediaCodec HEVC hardware decode → SurfaceView (incl. HDR10)
|
||||
src/audio.rs · src/mic.rs Opus + Oboe playback / mic uplink (jitter ring)
|
||||
src/feedback.rs rumble + HID output (lightbar / adaptive triggers)
|
||||
src/stats.rs live video stats
|
||||
|
||||
clients/android/ Gradle project (this dir)
|
||||
settings.gradle.kts · build.gradle.kts · gradle.properties · gradlew
|
||||
app/ :app — Compose application (MainActivity)
|
||||
kit/ :kit — Android library: NativeBridge + the cargo-ndk build
|
||||
build.gradle.kts cargoNdk{Debug,Release} → src/main/jniLibs/<abi>/*.so
|
||||
app/ :app — Compose UI: Connect / Settings / Stream screens (phone + TV)
|
||||
kit/ :kit — NativeBridge · discovery (NsdManager) · Gamepad · Keymap ·
|
||||
security (Keystore identity + known-host store) · cargo-ndk build
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
@@ -57,15 +61,22 @@ cd clients/android
|
||||
# Emulators (created during env setup): emulator -avd pf_phone | emulator -avd pf_tv
|
||||
```
|
||||
|
||||
The debug APK lands in `app/build/outputs/apk/debug/`. The scaffold screen calls
|
||||
`NativeBridge.abiVersion()` across JNI — a live ABI version proves the whole native stack is wired.
|
||||
The debug APK lands in `app/build/outputs/apk/debug/`. Launch it, pick a host from the list, pair,
|
||||
and stream.
|
||||
|
||||
## Status
|
||||
|
||||
- **Scaffold (done):** Gradle modules, cargo-ndk wiring, JNI native-link proof, phone+TV-installable
|
||||
manifest. `crates/punktfunk-core` `rcgen` switched to the `ring` backend so the client `.so` is
|
||||
aws-lc-free.
|
||||
- **Next (Android stage 1):** video decode (`AMediaCodec` async → `SurfaceView`), audio
|
||||
(Opus + Oboe + jitter ring), input capture → `send_input`, pairing/identity (Keystore-wrapped),
|
||||
mDNS discovery, the phone/TV Compose UI. The Rust-side homes are stubbed in
|
||||
`clients/android/native/src/session.rs` with port pointers to `clients/linux`.
|
||||
A working native client (phone + Android TV), at parity with the Linux and Apple apps for the core
|
||||
streaming experience:
|
||||
|
||||
- **Video** — `AMediaCodec` hardware HEVC decode → `SurfaceView`, including **HDR10** (Main10 /
|
||||
BT.2020 PQ), with low-latency decode tuning and a live stats HUD.
|
||||
- **Audio** — Opus + Oboe playback with a jitter ring, plus mic uplink to the host.
|
||||
- **Input** — game controllers (buttons + axes) with rumble and HID feedback; D-pad /
|
||||
game-controller focus navigation for the couch (TV + phone).
|
||||
- **Discovery & trust** — `NsdManager` mDNS host list, SPAKE2 PIN pairing and TOFU, with a
|
||||
Keystore-wrapped client identity and a known-host store.
|
||||
- **UI** — Compose host list / settings / stream screens, Material You theming.
|
||||
- **Shipping** — built for `arm64-v8a` + `x86_64`; published to Google Play (Internal Testing).
|
||||
|
||||
`crates/punktfunk-core` uses the `ring` `rcgen` backend so the client `.so` is aws-lc-free.
|
||||
|
||||
@@ -6,9 +6,14 @@ input datagrams, Opus audio, cert pinning — lives in the shared Rust core (sta
|
||||
linked as `PunktfunkCore.xcframework`); this package is the Swift shell: decode
|
||||
(VideoToolbox), present (SwiftUI), input capture.
|
||||
|
||||
## Status — first light achieved (2026-06-10)
|
||||
## Status — working client (macOS, with iOS / tvOS in the shared build)
|
||||
|
||||
Validated live, Mac ↔ Linux box over the LAN: gamescope virtual output → NVENC HEVC →
|
||||
A full streaming client: VideoToolbox HEVC decode, controllers incl. DualSense feedback, host
|
||||
discovery, PIN pairing, and a network speed test. The lower-latency **stage-2 presenter**
|
||||
(`VTDecompressionSession` → `CAMetalLayer`) is built and opt-in (Settings → Presenter); see below.
|
||||
|
||||
First light was achieved 2026-06-10 — validated live, Mac ↔ a Linux host over the LAN: gamescope
|
||||
virtual output → NVENC HEVC →
|
||||
`punktfunk/1` (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox →
|
||||
`AVSampleBufferDisplayLayer` on glass at 1280×720@60, with mouse/keyboard flowing back as
|
||||
QUIC datagrams into the host's gamescope EIS injector (thousands of events injected during
|
||||
|
||||
@@ -3,7 +3,11 @@ title: "Apple Stage-2 Presenter (handoff)"
|
||||
description: "Implementation plan for the explicit VTDecompressionSession → CAMetalLayer presenter — hand-paced present + true decode→present (glass-to-glass) measurement. Written so a Mac agent can pick it up."
|
||||
---
|
||||
|
||||
A pickup-ready plan for the **stage-2 Apple presenter**. The current **stage-1** presenter feeds
|
||||
> **Status update:** the stage-2 presenter described here has since been **built and live-validated**,
|
||||
> shipping behind an opt-in flag (`AVSampleBufferDisplayLayer` remains the default known-good path).
|
||||
> This page is preserved as the implementation/handoff record for that work.
|
||||
|
||||
The implementation plan for the **stage-2 Apple presenter**. The **stage-1** presenter feeds
|
||||
compressed HEVC straight into `AVSampleBufferDisplayLayer`, which hardware-decodes **and presents
|
||||
internally with no per-frame callback** — so we can't stamp decode or present, and we can't hand-pace.
|
||||
Stage-2 takes explicit control: decode with `VTDecompressionSession`, present decoded frames through a
|
||||
|
||||
@@ -3,17 +3,19 @@ title: "CI & Docker"
|
||||
description: "Gitea Actions setup — workflows, the dockerized pieces, and the runners."
|
||||
---
|
||||
|
||||
CI runs on **Gitea Actions** (`git.unom.io`, org `unom`). Three workflows in
|
||||
`.gitea/workflows/`, two runners, three images in the Gitea container registry.
|
||||
CI runs on **Gitea Actions** (`git.unom.io`, org `unom`). The workflows live in
|
||||
`.gitea/workflows/`; they run across Linux and macOS runners and push a few images to the
|
||||
Gitea container registry.
|
||||
|
||||
## Workflows
|
||||
|
||||
| Workflow | Trigger | Runner | What it does |
|
||||
|---|---|---|---|
|
||||
| `ci.yml` | push to `main`, PRs | `ubuntu-24.04` | Rust workspace (fmt · clippy `-D warnings` · build · test · C-ABI harness · generated-header drift) inside the `punktfunk-rust-ci` image; `web/` and `docs-site/` build + typecheck in `oven/bun:1` |
|
||||
| `docker.yml` | push to `main`, `v*` tags, manual | `ubuntu-24.04` | Builds + pushes the three images below (`latest` + `sha-<short>` tags) |
|
||||
| `apple.yml` | push to `main`, PRs, manual | `macos-arm64` | Rust core → `PunktfunkCore.xcframework` → `swift build` + `swift test` in `clients/apple` |
|
||||
| `release.yml` | `v*` tags, manual | `macos-arm64` | Production Apple builds: sandboxed macOS `.dmg` (Developer ID, notarized, stapled) attached to the Gitea release + macOS/iOS/tvOS archives uploaded to TestFlight |
|
||||
| `ci.yml` | push to `main`, PRs | Linux | Rust workspace (fmt · clippy `-D warnings` · build · test · C-ABI harness · generated-header drift) inside the `punktfunk-rust-ci` image; `web/` and `docs-site/` build + typecheck in `oven/bun:1` |
|
||||
| `docker.yml` | push to `main`, `v*` tags, manual | Linux | Builds + pushes the images below (`latest` + `sha-<short>` tags) |
|
||||
| `apple.yml` | push to `main`, PRs, manual | macOS | Rust core → `PunktfunkCore.xcframework` → `swift build` + `swift test` in `clients/apple` |
|
||||
| `release.yml` | `v*` tags, manual | macOS | Production Apple builds: sandboxed macOS `.dmg` (Developer ID, notarized, stapled) attached to the Gitea release + macOS/iOS/tvOS archives uploaded to TestFlight |
|
||||
| `windows-msix.yml` | push to `main`, `v*` tags, manual | Windows | Builds the Windows client for `x86_64`/`aarch64` and packages signed MSIX artifacts |
|
||||
|
||||
## Dockerized pieces
|
||||
|
||||
@@ -26,20 +28,20 @@ the GPU/compositor stack of the box it runs on). What is:
|
||||
| `git.unom.io/unom/punktfunk-docs` | `docs-site/Dockerfile` | This site; `PORT` (3000) |
|
||||
| `git.unom.io/unom/punktfunk-rust-ci` | `ci/rust-ci.Dockerfile` | Ubuntu 26.04 + FFmpeg 8/PipeWire/GL/GBM dev libs + a libcuda **link stub** (driver userspace, no kernel module) + pinned rustup — the container `ci.yml`'s Rust job runs in |
|
||||
|
||||
Registry pushes authenticate with the repo Actions secret **`REGISTRY_TOKEN`** (a PAT
|
||||
Registry pushes authenticate with a repo Actions secret holding a registry token (a PAT
|
||||
with `write:package`; the login username in `docker.yml` is the token owner, not the
|
||||
push actor).
|
||||
|
||||
## Runners
|
||||
|
||||
- **`ubuntu-24.04`** — the pre-existing Linux runner; runs the Rust/web/docs jobs (as
|
||||
docker containers) and the image build+push jobs.
|
||||
- **`macos-arm64`** — `home-mac-mini-1` (M-series, macOS 26), a **host-mode**
|
||||
`act_runner` (upstream now ships it as `gitea-runner`) provisioned by
|
||||
- **Linux runner** — runs the Rust/web/docs jobs (as docker containers) and the image
|
||||
build+push jobs.
|
||||
- **macOS runner** — an Apple-silicon Mac running macOS, a **host-mode** `act_runner`
|
||||
(upstream now ships it as `gitea-runner`) provisioned by
|
||||
[`scripts/ci/setup-macos-runner.sh`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/ci/setup-macos-runner.sh):
|
||||
rustup (+ both darwin targets for the universal xcframework), Node.js (host-mode runners
|
||||
execute JS actions via `node` from PATH — nothing auto-provisions it), the runner binary
|
||||
in `~/.local/bin`, state in `~/ci/act-runner/` (config, `.runner` registration,
|
||||
in `~/.local/bin`, state under `~/ci/act-runner/` (config, `.runner` registration,
|
||||
`runner.log`), kept alive by the `io.gitea.act_runner` **root LaunchDaemon** — it cannot
|
||||
be a user LaunchAgent: macOS Local Network privacy silently blocks LAN dials
|
||||
("no route to host") from unbundled CLI binaries in gui/user launchd domains, while
|
||||
@@ -47,14 +49,12 @@ push actor).
|
||||
(CLT alone only covers `swift build/test`); if `xcode-select` still points at CLT, the
|
||||
script auto-detects `/Applications/Xcode*.app` and bakes a `DEVELOPER_DIR` override into
|
||||
the daemon environment — no `xcode-select -s` required.
|
||||
- **Windows runner** — builds and packages the native Windows client (MSIX) for the
|
||||
release matrix.
|
||||
|
||||
Re-provisioning (idempotent) or first-time registration from a dev box:
|
||||
|
||||
```sh
|
||||
# token: org unom → Settings → Actions → Runners → Create new runner
|
||||
ssh enricobuehler@192.168.1.135 GITEA_RUNNER_TOKEN=<token> bash -s \
|
||||
< scripts/ci/setup-macos-runner.sh
|
||||
```
|
||||
Re-provisioning is idempotent — re-running `scripts/ci/setup-macos-runner.sh` on the macOS
|
||||
runner with a fresh `GITEA_RUNNER_TOKEN` (org `unom` → Settings → Actions → Runners →
|
||||
Create new runner) re-registers it without manual cleanup.
|
||||
|
||||
## Apple releases
|
||||
|
||||
@@ -97,18 +97,16 @@ linking) and **refuses iOS/tvOS slices** (CLT has no iOS SDK).
|
||||
## Deployment
|
||||
|
||||
`docker.yml`'s `deploy-docs` job ships this docs site after every image push: it syncs
|
||||
`compose.production.yml` to `~/punktfunk-docs` on **unom-1** (the DMZ services VM
|
||||
website and cms deploy to) and runs `docker compose pull && up -d` there over SSH (same
|
||||
pattern and secret set as `unom/website`: `DEPLOY_HOST` / `DEPLOY_USER` / `DEPLOY_PORT` /
|
||||
`DEPLOY_SSH_KEY`, the `unom-ci-deploy` key). The container binds host port **3220**;
|
||||
Caddy on `home-reverse-proxy-1` serves it as <https://docs.punktfunk.unom.io> (vhost in
|
||||
`unom/reverse-proxy`, UniFi firewall allowlist Caddy→unom-1:3220 in `unom/infra`
|
||||
`proxmox/unom-1`). The host and the web console are NOT deployed — the console
|
||||
fronts a punktfunk host's management API on whatever box runs the host.
|
||||
`compose.production.yml` to the docs server and runs `docker compose pull && up -d` there
|
||||
over SSH, driven by a small set of deploy secrets (`DEPLOY_HOST` / `DEPLOY_USER` /
|
||||
`DEPLOY_PORT` / `DEPLOY_SSH_KEY`). A reverse proxy in front of that server serves the
|
||||
container as <https://docs.punktfunk.unom.io>. The host and the web console are NOT
|
||||
deployed — the console fronts a punktfunk host's management API on whatever box runs the
|
||||
host.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Mac runner offline** — `ssh <mac> tail -50 '~/ci/act-runner/runner.log'`; restart with
|
||||
- **macOS runner offline** — check `~/ci/act-runner/runner.log` on the runner; restart with
|
||||
`sudo launchctl kickstart -k system/io.gitea.act_runner`. "no route to host" in the log
|
||||
means the daemon is running in a gui/user domain again — see the Local Network note
|
||||
above.
|
||||
|
||||
@@ -3,8 +3,9 @@ title: Clients
|
||||
description: The ways to connect to a punktfunk host — the Apple app, Moonlight, or the Linux client.
|
||||
---
|
||||
|
||||
A punktfunk host accepts clients over its own `punktfunk/1` protocol (the Apple and Linux apps) and
|
||||
over GameStream (Moonlight). Pick whichever fits the device you're streaming *to*. Ready to install?
|
||||
A punktfunk host accepts clients over its own `punktfunk/1` protocol (the macOS, Linux, Windows, and
|
||||
Android apps) and over GameStream (Moonlight). Pick whichever fits the device you're streaming *to*.
|
||||
Ready to install?
|
||||
**[Install a Client](/docs/install-client)** has the step-by-step for every device.
|
||||
|
||||
## Apple app (Mac, iPhone, iPad, Apple TV)
|
||||
@@ -24,8 +25,9 @@ Open the app, pick your host, [pair](/docs/pairing) once, and stream. It builds
|
||||
## Moonlight (anything else)
|
||||
|
||||
punktfunk also speaks the **GameStream** protocol, so any [Moonlight](https://moonlight-stream.org/)
|
||||
client — Windows, Android, Steam Deck, a browser, an old phone — connects with no punktfunk-specific
|
||||
software. See [Connect with Moonlight](/docs/moonlight).
|
||||
client — a browser, a smart TV, an old phone, a games console — connects with no punktfunk-specific
|
||||
software. (Most platforms also have a native punktfunk app below — Moonlight is the catch-all.) See
|
||||
[Connect with Moonlight](/docs/moonlight).
|
||||
|
||||
This is the broadest-compatibility option and great for couch gaming. It doesn't use the native
|
||||
protocol's FEC/encryption extensions, but for a healthy LAN that rarely matters.
|
||||
@@ -54,15 +56,31 @@ connect straight away:
|
||||
punktfunk-client --connect <host>:9777 # skip the picker, start a session immediately
|
||||
```
|
||||
|
||||
## Windows desktop client (in development)
|
||||
## Android app (phone + Android TV)
|
||||
|
||||
`punktfunk-client` for Windows (`clients/windows`) is the native graphical client
|
||||
for Windows — pure Rust, the same `punktfunk/1` core as the Apple and Linux apps, with a **WinUI 3**
|
||||
UI (host list, settings, PIN pairing) and the video on a `SwapChainPanel`, plus WASAPI audio, FFmpeg
|
||||
decode, SDL3 controllers, network discovery, and PIN pairing. Launch it and pick a host from the
|
||||
list, just like the Apple and Linux apps. It builds on `x86_64-pc-windows-msvc`; hardware (D3D11VA)
|
||||
decode, 10-bit/HDR present, and packaging are in progress, so it is not yet shipped. A headless CLI
|
||||
path exists for scripting/measurement:
|
||||
The native Android app speaks `punktfunk/1` directly, on both phones and Android TV. It does hardware
|
||||
HEVC decode (including HDR10), Opus audio with a mic uplink, game controllers with rumble and
|
||||
DualSense feedback, automatic host discovery, PIN pairing with pinned reconnects, and a live stats
|
||||
overlay — with D-pad and game-controller focus navigation for the couch. It builds from the
|
||||
`clients/android` directory (Kotlin + a shared Rust core).
|
||||
|
||||
Install it from **Google Play** — see [Install a Client](/docs/install-client#android). Open the app,
|
||||
pick your host, [pair](/docs/pairing) once, and stream.
|
||||
|
||||
## Windows desktop client
|
||||
|
||||
`punktfunk-client` for Windows (`clients/windows`) is the native graphical client for Windows — pure
|
||||
Rust, the same `punktfunk/1` core as the Apple, Linux, and Android apps, with a **WinUI 3** UI (host
|
||||
list, settings, PIN pairing) and the video on a `SwapChainPanel`. It does D3D11VA hardware decode
|
||||
(software fallback), 10-bit/HDR present, WASAPI audio + mic, SDL3 controllers (rumble, lightbar,
|
||||
DualSense), network discovery, and the full PIN-pairing trust surface. It builds for both `x86_64`
|
||||
and `aarch64` and ships as a **signed MSIX**. Launch it and pick a host from the list, just like the
|
||||
other native apps.
|
||||
|
||||
> The hardware-decode and HDR paths are complete but still pending validation on real GPU hardware.
|
||||
> If anything misbehaves, **[Moonlight](/docs/moonlight)** is a proven alternative for Windows.
|
||||
|
||||
A headless CLI path exists for scripting/measurement:
|
||||
|
||||
```sh
|
||||
punktfunk-client # open the WinUI 3 window (host list / settings)
|
||||
@@ -70,7 +88,7 @@ punktfunk-client --discover # list hosts on the network
|
||||
punktfunk-client --headless --connect <host>:9777 # no window: connect, count frames, print stats
|
||||
```
|
||||
|
||||
Until it ships, **Moonlight** remains the recommended way to stream to Windows (see below).
|
||||
Prefer the broadest compatibility, or no install? **Moonlight** also streams to Windows (see below).
|
||||
|
||||
## Linux reference client (headless)
|
||||
|
||||
@@ -89,7 +107,9 @@ punktfunk-probe --connect <host>:9777 --pin <fp> # connect to one
|
||||
|---|---|
|
||||
| A Mac, iPhone, iPad, or Apple TV | The **Apple app** |
|
||||
| A Linux desktop or laptop, or a Steam Deck | **`punktfunk-client`** (GTK4) |
|
||||
| Windows, Android, a browser, a TV | **Moonlight** |
|
||||
| An Android phone or TV | The **Android app** |
|
||||
| Windows | The native **`punktfunk-client`** (signed MSIX) or **Moonlight** |
|
||||
| A browser, a smart TV, or any other device | **Moonlight** |
|
||||
| Automated tests / latency measurement | **`punktfunk-probe`** (headless) |
|
||||
|
||||
Whichever you choose, the first connection needs a one-time [pairing](/docs/pairing).
|
||||
|
||||
@@ -30,15 +30,15 @@ These tell the host which desktop session to attach to. Your setup guide sets th
|
||||
|
||||
You don't set these on the host — **the client chooses them**. When a device connects, the host
|
||||
creates a virtual display at that device's resolution and refresh rate. A 1080p60 laptop and a
|
||||
1440p120 desktop each get their own. (With Moonlight, set the mode in Moonlight's settings; with the
|
||||
Apple app, it uses the device's display.)
|
||||
1440p120 desktop each get their own. (With Moonlight, set the mode in Moonlight's settings; the
|
||||
native clients let you pick a mode or default to the device's display.)
|
||||
|
||||
## Bitrate
|
||||
|
||||
The client requests a bitrate; the host encodes to it. To find a good value for your link:
|
||||
|
||||
- **Apple app:** use the built-in **speed test** (a host card's menu → *Test Network Speed*). It
|
||||
measures your link and suggests a bitrate, then applies it.
|
||||
- **Native clients (Apple, Linux, and more):** use the built-in **speed test** (from a host's menu).
|
||||
It measures your link, suggests a bitrate, and applies it.
|
||||
- **Moonlight:** set the bitrate in Moonlight's settings. Start moderate and raise it.
|
||||
|
||||
## Multiple devices at once
|
||||
|
||||
@@ -17,7 +17,7 @@ punktfunk-host serve --native
|
||||
|
||||
| Flag | Meaning |
|
||||
|---|---|
|
||||
| `--native` | Also run the native `punktfunk/1` server (recommended; enables the Apple app and discovery). |
|
||||
| `--native` | Also run the native `punktfunk/1` server (recommended; enables the native clients and discovery). |
|
||||
| `--native-port <PORT>` | Native QUIC port (default `9777`). |
|
||||
| `--open` | Don't require pairing — serve any device on the network. Off by default; only for trusted single-user setups. |
|
||||
| `--mgmt-bind <IP:PORT>` | Management API address (default loopback `127.0.0.1:47990`). |
|
||||
|
||||
@@ -39,7 +39,8 @@ punktfunk speaks two protocols over the same host:
|
||||
no special software. This is the most compatible way in.
|
||||
- **punktfunk/1 (native)** — a purpose-built protocol with a QUIC control channel and a UDP data
|
||||
channel hardened with forward error correction and encryption. It's lower-latency and more resilient
|
||||
on imperfect networks, and it's what the [Apple app](/docs/clients) uses.
|
||||
on imperfect networks, and it's what the [native clients](/docs/clients) (Apple, Linux, Windows,
|
||||
Android) use.
|
||||
|
||||
Both run from a single host process, so you don't choose up front — Moonlight clients use GameStream,
|
||||
the native clients use punktfunk/1.
|
||||
@@ -53,7 +54,8 @@ cryptographic identity — no PIN, no account, no cloud. See [Pairing & Trust](/
|
||||
## Finding hosts
|
||||
|
||||
Hosts advertise themselves on your local network, so clients can **discover** them automatically
|
||||
instead of needing an IP address. The Apple app and Moonlight both list hosts they find on the LAN.
|
||||
instead of needing an IP address. The native clients and Moonlight both list hosts they find on the
|
||||
LAN.
|
||||
|
||||
## Multiple devices at once
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ description: "The full design: protocol core, milestones, and architecture."
|
||||
|
||||
*A ground-up low-latency desktop streaming stack, built Linux-first, with a shared Rust protocol core and native clients per platform.*
|
||||
|
||||
> `punktfunk` is a placeholder codename — rename freely. It fits the lowercase house style (`unom`, `played`, `remplir`) and reads as "glass-to-glass light," which is the whole point.
|
||||
> The name `punktfunk` fits the lowercase house style (`unom`, `played`, `remplir`) and reads as "glass-to-glass light," which is the whole point.
|
||||
|
||||
---
|
||||
|
||||
@@ -32,7 +32,7 @@ Two concrete gaps justify a new project rather than another fork:
|
||||
|
||||
**Explicit non-goals (at least at first):**
|
||||
- Windows *host* support (Sunshine/Apollo already do this well; no gap to fill).
|
||||
- Internet/NAT-traversal relay infrastructure (LAN/VPN first; you already run Headscale/NetBird — lean on that).
|
||||
- Internet/NAT-traversal relay infrastructure (LAN/VPN first; lean on an existing mesh VPN such as Headscale/NetBird/Tailscale).
|
||||
- Reinventing encoders/decoders (bind to FFmpeg + vendor SDKs; never rewrite codecs).
|
||||
- A bespoke compositor (drive existing ones; only consider a dedicated headless compositor as a *deployment mode*, see §6).
|
||||
|
||||
@@ -169,7 +169,7 @@ This is the differentiator and the most fragmented part. Two deployment models
|
||||
|
||||
**Per-compositor (Model A) runtime virtual-output creation:**
|
||||
|
||||
- **KWin / Plasma 6 (recommended MVP target — matches your CachyOS/KDE daily driver and where the gap is loudest):** KWin can create virtual outputs; KRdp already does this internally for remote sessions. Drive it via the KWin DBus interface; capture via `xdg-desktop-portal-kde` ScreenCast (PipeWire); inject input via the RemoteDesktop portal or `reis`.
|
||||
- **KWin / Plasma 6 (recommended MVP target — a common KDE daily-driver setup, and where the gap is loudest):** KWin can create virtual outputs; KRdp already does this internally for remote sessions. Drive it via the KWin DBus interface; capture via `xdg-desktop-portal-kde` ScreenCast (PipeWire); inject input via the RemoteDesktop portal or `reis`.
|
||||
- **wlroots (Sway/Hyprland — fastest to *prototype* the pipeline):** enable the headless backend (`WLR_BACKENDS=…,headless`), then `swaymsg create_output` / `hyprctl output create headless`. Capture via `wlr-screencopy` or the portal. Simplest API; good for validating capture→encode→send before fighting KWin/Mutter.
|
||||
- **Mutter / GNOME:** virtual monitors via the headless backend; runtime creation via Mutter DBus (`org.gnome.Mutter.*` — partly experimental). Capture via `xdg-desktop-portal-gnome` ScreenCast.
|
||||
|
||||
@@ -227,7 +227,7 @@ Sizing is rough and relative (Spike / S / M / L) for a focused solo dev; treat a
|
||||
|
||||
**M4 — P2 transport: break the wall (L).** Add `punktfunk/1` negotiation; swap to `reed-solomon-simd` GF(2¹⁶) with multi-block per-frame framing; optional QUIC control/audio. Write a minimal **Rust** reference client (decode via VAAPI, present via wgpu/Vulkan) to exercise it. *Acceptance:* a stable stream above 1.4 Gbps at 5120×1440@240 with loss recovery working; latency unchanged vs. M2.
|
||||
|
||||
**M5 — Apple client (L).** Swift + VideoToolbox + Metal + SwiftUI, linking `punktfunk-core` via the C header. *Acceptance:* the Mac Studio plays a stream at native resolution/refresh.
|
||||
**M5 — Apple client (L).** Swift + VideoToolbox + Metal + SwiftUI, linking `punktfunk-core` via the C header. *Acceptance:* a Mac plays a stream at native resolution/refresh.
|
||||
|
||||
**M6 — Feature surface (M, ongoing).** Mic passthrough as a proper encrypted, per-client reverse audio stream (the thing the upstream PR got wrong); HDR signalling; per-client identity/permissions; pause/resume. *Acceptance:* feature parity with Apollo on the items you care about, plus mic done right.
|
||||
|
||||
@@ -243,7 +243,7 @@ Sizing is rough and relative (Spike / S / M / L) for a focused solo dev; treat a
|
||||
| Encoder/decoder can't sustain 1.77 Gpx/s @ 240 | Med | High | Measure in M0/M4 on real silicon; this is a hardware ceiling no rewrite fixes — discover it before P2, not after |
|
||||
| Frame pacing eats more time than expected | High | Med | M3 measurement harness first; treat pacing as a first-class subsystem, not a polish step |
|
||||
| Scope creep into a full Moonlight replacement | High | High | P1 (stock-client compat) is the firewall: it forces you to ship value before writing a client |
|
||||
| Solo bandwidth vs. your other projects (ENRW thesis, played) | High | Med | M2 is a complete, useful artifact on its own; the plan is safe to pause after any milestone |
|
||||
| Solo bandwidth vs. other projects | High | Med | M2 is a complete, useful artifact on its own; the plan is safe to pause after any milestone |
|
||||
|
||||
---
|
||||
|
||||
@@ -254,7 +254,7 @@ Sizing is rough and relative (Spike / S / M / L) for a focused solo dev; treat a
|
||||
- **Loss resilience:** `tc netem` to inject loss/jitter/reorder; verify FEC recovery and graceful degradation.
|
||||
- **Pacing:** log present timestamps vs. client vsync; alert on stalls and duplicate/dropped frames.
|
||||
- **Soak:** multi-hour streams; watch for buffer growth, fd leaks, encoder session exhaustion.
|
||||
- **Hardware matrix:** your NVIDIA box (NVENC), an AMD/Intel box (VAAPI), Mac Studio (VideoToolbox decode). Catch driver quirks early.
|
||||
- **Hardware matrix:** an NVIDIA box (NVENC), an AMD/Intel box (VAAPI), a Mac (VideoToolbox decode). Catch driver quirks early.
|
||||
|
||||
---
|
||||
|
||||
@@ -290,10 +290,10 @@ punktfunk/
|
||||
|
||||
## 12. Immediate next actions (first week)
|
||||
|
||||
1. **Stand up the workspace** with `punktfunk-core` (empty ABI + `cbindgen`) and `punktfunk-host` skeletons; CI on your Gitea (you already have BuildKit pipelines).
|
||||
2. **M0 spike on wlroots:** headless output → PipeWire capture → NVENC/VAAPI encode → playable file. This validates the riskiest *pipeline* assumptions in days, on your real GPU.
|
||||
3. **Read KRdp's source** for how KDE creates virtual outputs and casts them — it's the closest existing reference for the KWin path you'll need in M2.
|
||||
4. **Decide P1 protocol depth:** confirm exactly which `serverinfo`/RTSP/pairing messages a current Moonlight client requires for a successful connect, so M2's compat surface is scoped precisely (this is also the question to take back to the dev who mentioned the 1G limit).
|
||||
1. **Stand up the workspace** with `punktfunk-core` (empty ABI + `cbindgen`) and `punktfunk-host` skeletons; wire up CI (Gitea Actions, BuildKit-based pipelines).
|
||||
2. **M0 spike on wlroots:** headless output → PipeWire capture → NVENC/VAAPI encode → playable file. This validates the riskiest *pipeline* assumptions in days, on real GPU hardware.
|
||||
3. **Read KRdp's source** for how KDE creates virtual outputs and casts them — it's the closest existing reference for the KWin path needed in M2.
|
||||
4. **Decide P1 protocol depth:** confirm exactly which `serverinfo`/RTSP/pairing messages a current Moonlight client requires for a successful connect, so M2's compat surface is scoped precisely.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -17,8 +17,8 @@ It's built for the things that make streaming feel native:
|
||||
- **Low latency, GPU end to end.** Frames go straight from the compositor to the GPU encoder
|
||||
(NVENC) with zero CPU copies, and over a transport tuned for responsiveness rather than throughput.
|
||||
- **Works with the apps you already have.** punktfunk speaks the GameStream protocol, so any
|
||||
**Moonlight** client connects out of the box — and a faster **native protocol** with a dedicated
|
||||
app for Apple devices.
|
||||
**Moonlight** client connects out of the box — and a faster **native protocol** with dedicated apps
|
||||
for **macOS, iOS, tvOS, Linux, Windows, and Android**.
|
||||
- **Secure by default.** Hosts require a one-time PIN pairing; after that, devices reconnect on a
|
||||
pinned identity. No accounts, no cloud.
|
||||
|
||||
@@ -28,15 +28,16 @@ It's built for the things that make streaming feel native:
|
||||
<Card title="How It Works" href="/docs/how-it-works" description="The ideas behind punktfunk in a few minutes — virtual displays, the two protocols, pairing." />
|
||||
<Card title="Quick Start" href="/docs/quickstart" description="From nothing to streaming: set up a host and connect your first client." />
|
||||
<Card title="Host Setup" href="/docs/requirements" description="Install the host on Ubuntu (GNOME or KDE), Fedora (KDE), or Bazzite." />
|
||||
<Card title="Connect a Client" href="/docs/clients" description="Stream with the Apple app, Moonlight, or the Linux client." />
|
||||
<Card title="Connect a Client" href="/docs/clients" description="Stream with the native app for your device — macOS, Linux, Windows, Android — or any Moonlight client." />
|
||||
</Cards>
|
||||
|
||||
## What you need
|
||||
|
||||
- A **Linux host** with an **NVIDIA GPU** (for the NVENC hardware encoder) running one of the
|
||||
[supported setups](/docs/requirements): **Ubuntu** (GNOME or KDE), **Fedora** (KDE), or **Bazzite**.
|
||||
- A **client device** to stream to — a Mac/iPhone/iPad/Apple TV (native app), or anything that runs
|
||||
**Moonlight**.
|
||||
A native [**Windows host**](/docs/windows-host) (NVIDIA-only) is also available.
|
||||
- A **client device** to stream to — there are native apps for **macOS, iOS/iPadOS, tvOS, Linux,
|
||||
Windows, and Android**, plus any device that runs **Moonlight**.
|
||||
- Both on the **same network** (LAN or VPN). punktfunk is designed for a trusted local network.
|
||||
|
||||
Ready? Head to the [Quick Start](/docs/quickstart).
|
||||
|
||||
@@ -85,8 +85,9 @@ certificate, so you import that certificate once before Windows will install the
|
||||
|
||||
3. Launch **Punktfunk** from the Start menu and pick your host.
|
||||
|
||||
> The Windows client is young (software decode; hardware D3D11VA/HDR in progress). If it
|
||||
> misbehaves, **[Moonlight](/docs/moonlight)** is a solid alternative for Windows.
|
||||
> The Windows client's hardware-decode (D3D11VA) and HDR paths are complete but still pending
|
||||
> validation on real GPU hardware. If anything misbehaves, **[Moonlight](/docs/moonlight)** is a
|
||||
> solid alternative for Windows.
|
||||
|
||||
## macOS
|
||||
|
||||
|
||||
@@ -4,8 +4,12 @@ description: Stream from a punktfunk host using any Moonlight client.
|
||||
---
|
||||
|
||||
punktfunk speaks the **GameStream** protocol, so [Moonlight](https://moonlight-stream.org/) connects
|
||||
to it like it would to any GameStream host — no punktfunk-specific app needed. This is the easiest way
|
||||
to stream to Windows, Android, the Steam Deck, a browser, or a TV.
|
||||
to it like it would to any GameStream host — no punktfunk-specific app needed. It's a great option for
|
||||
a browser, a smart TV, or any device without a native client.
|
||||
|
||||
> Many platforms also have a **native punktfunk client** with lower latency and built-in
|
||||
> discovery/pairing — including **Windows** and **Android** (phone and Android TV). See
|
||||
> [Clients](/docs/clients) before reaching for Moonlight.
|
||||
|
||||
## 1. Make sure the host is running
|
||||
|
||||
@@ -34,7 +38,7 @@ it. Mouse, keyboard, and controllers flow back to the host.
|
||||
- **Set your resolution and frame rate in Moonlight's settings** before connecting — the host matches
|
||||
whatever Moonlight asks for, creating the virtual display at that exact mode.
|
||||
- **Codec:** HEVC (H.265) is a good default; AV1 is available if your client supports it.
|
||||
- **Bitrate:** start moderate and raise it. For very high bitrates, the native [Apple
|
||||
app](/docs/clients) has a built-in speed test; with Moonlight, set the bitrate manually.
|
||||
- **Bitrate:** start moderate and raise it. For very high bitrates, the [native
|
||||
clients](/docs/clients) have a built-in speed test; with Moonlight, set the bitrate manually.
|
||||
- Moonlight uses the GameStream protocol, not punktfunk's native FEC/encryption extensions. On a
|
||||
solid LAN this is fine; on a lossy link the [Apple app](/docs/clients) holds up better.
|
||||
solid LAN this is fine; on a lossy link a [native client](/docs/clients) holds up better.
|
||||
|
||||
@@ -50,7 +50,8 @@ command line instead; the production `serve --native` host arms pairing from the
|
||||
|
||||
Then, on the client:
|
||||
|
||||
- **Apple app:** select the host (or use *Pair with PIN…* from its menu) and enter the PIN.
|
||||
- **Native clients (Apple, Linux, Windows, Android):** select the host (or use *Pair with PIN…* from
|
||||
its menu) and enter the PIN the host displays.
|
||||
- **Moonlight:** choose **Pair**; Moonlight shows the PIN to confirm on the host side.
|
||||
|
||||
## Requiring pairing (the default)
|
||||
|
||||
@@ -31,10 +31,11 @@ network, so clients can find it by name. Leave it running. (To start it automati
|
||||
|
||||
## 3. Connect and pair a client
|
||||
|
||||
On the device you want to stream to:
|
||||
On the device you want to stream to, use a [native punktfunk client](/docs/clients) for the lowest
|
||||
latency, or any Moonlight client:
|
||||
|
||||
- **Apple (Mac, iPhone, iPad, Apple TV):** open the punktfunk app — your host appears under *On this
|
||||
network*. Tap it, and when prompted, **pair**.
|
||||
- **Native client (Apple, Linux, Windows, Android):** open the punktfunk app — your host appears in
|
||||
the list of hosts found on your network. Select it, and when prompted, **pair**.
|
||||
- **Anything with Moonlight:** add the host (it should be discovered automatically), then pair.
|
||||
|
||||
To pair, the host needs to show a PIN. Arm pairing from the host's web console — the host displays a
|
||||
|
||||
@@ -5,8 +5,9 @@ description: What you need to run a punktfunk host — GPU, driver, desktop, and
|
||||
|
||||
## Supported setups
|
||||
|
||||
A punktfunk host runs on a Linux machine with an NVIDIA GPU. These are the desktop environments it
|
||||
supports today, each with its own guide:
|
||||
A punktfunk host runs primarily on a Linux machine with an NVIDIA GPU (a native
|
||||
[Windows host](/docs/windows-host) is also available — see below). These are the Linux desktop
|
||||
environments it supports today, each with its own guide:
|
||||
|
||||
| Setup | Desktop / compositor | Guide |
|
||||
|---|---|---|
|
||||
@@ -18,6 +19,10 @@ supports today, each with its own guide:
|
||||
Other wlroots compositors (Sway/Hyprland) also work but aren't a primary target. If your desktop isn't
|
||||
listed, the host still needs one of these compositor backends to create a virtual display.
|
||||
|
||||
> **Windows host:** punktfunk also runs as a native host on **Windows 10/11 (x64) with an NVIDIA GPU**
|
||||
> — a signed installer that registers a service and bundles a virtual-display driver. It's NVIDIA-only
|
||||
> and newer than the Linux host; see [Windows Host](/docs/windows-host).
|
||||
|
||||
## GPU and driver
|
||||
|
||||
- **An NVIDIA GPU** with NVENC — effectively any GeForce RTX or workstation card. NVENC is what
|
||||
@@ -54,5 +59,6 @@ Minimum compositor versions (newer is fine):
|
||||
|
||||
## A client
|
||||
|
||||
You also need something to stream *to* — see [Connect a Client](/docs/clients). The Apple app and any
|
||||
Moonlight client both work; both can discover the host on your network automatically.
|
||||
You also need something to stream *to* — see [Connect a Client](/docs/clients). There are native
|
||||
punktfunk clients for **Apple (macOS, iOS, iPadOS, tvOS), Linux, Windows, and Android**, and any
|
||||
Moonlight client works too. All of them can discover the host on your network automatically.
|
||||
|
||||
@@ -16,14 +16,19 @@ process, with native pairing driven from the **web console** (arm → show PIN),
|
||||
Advanced DualSense (audio-driven voice-coil) haptics **scoped NO-GO** (`docs/dualsense-haptics.md`).
|
||||
**Bazzite dynamic resolution (`c894c6f`):** the host now *manages* a headless `gamescope-session-plus`
|
||||
Steam session at the **client's exact resolution + refresh** — games see it (via injected
|
||||
`--nested-refresh` + generated CVT modes, not the box's TV EDID), relaunched per-connection on a mode
|
||||
`--nested-refresh` + generated CVT modes, not the host's TV EDID), relaunched per-connection on a mode
|
||||
change, reused (no Steam restart) on the same mode. Plus macOS/iPad input fixes (NSEvent motion +
|
||||
iPad pointer-lock) and a 4K/5K one-frame-freeze fix (grow the UDP socket buffers).
|
||||
|
||||
**Next:** **§8 pairing & trust hardening** (mandatory PIN by default + delegated approval), the native
|
||||
client presenter + iOS (§6), and a Windows host (§7 — now **de-risked via SudoVDA**, no custom
|
||||
signed driver needed). **§10 HDR/10-bit is parked — blocked upstream at the compositor** (no
|
||||
gamescope/KWin PipeWire 10-bit producer yet).
|
||||
The four native clients — **macOS/iOS/iPadOS/tvOS**, **Linux**, **Windows**, and **Android** — are
|
||||
all real, working clients; see [Status & Progress](/docs/status) for their current state.
|
||||
|
||||
A native **Windows host** (§7) is also implemented and shipping (NVIDIA-only, x64-only) — see
|
||||
[Windows Host](/docs/windows-host).
|
||||
|
||||
**Next:** **§8 pairing & trust hardening** (mandatory PIN by default + delegated approval) and the
|
||||
native client presenter + iOS (§6). **§10 HDR/10-bit is parked — blocked upstream at the compositor**
|
||||
(no gamescope/KWin PipeWire 10-bit producer yet).
|
||||
|
||||
## 1. Reliable headless KDE/compositor spawning ✅ *(done — Phase 1 + 2)*
|
||||
|
||||
@@ -127,22 +132,26 @@ select = a `pw_stream` with `Direction::Output` + `media.class=Audio/Source`.
|
||||
PunktfunkKit is already platform-shared; iOS needs the `UIViewRepresentable` presenter twin
|
||||
+ touch capture (#5) + UI. tvOS later.
|
||||
|
||||
## 7. Windows as a host *(scoped — `docs/windows-host.md`; de-risked via SudoVDA)*
|
||||
## 7. Windows as a host ✅ *(implemented & shipping — NVIDIA-only, x64-only; see [Windows Host](/docs/windows-host))*
|
||||
|
||||
Architecturally an "add a backend" job, not a parallel port: `punktfunk-core` (protocol/FEC/
|
||||
crypto/C-ABI) + QUIC + GameStream + mgmt + the `m3`/pipeline orchestration are all platform-agnostic
|
||||
and already `cfg`-isolated (~95% reuse). New `#[cfg(windows)]` backends behind the existing traits:
|
||||
capture (DXGI Desktop Duplication / Windows.Graphics.Capture), encode (Media Foundation / NVENC-SDK
|
||||
with a D3D11 context), input (SendInput + ViGEm), audio (WASAPI loopback + a virtual mic).
|
||||
Done as an "add a backend" job, not a parallel port: `punktfunk-core` (protocol/FEC/crypto/C-ABI) +
|
||||
QUIC + GameStream + mgmt + the pipeline orchestration are all platform-agnostic and already
|
||||
`cfg`-isolated (~95% reuse), so the Windows host is a set of `#[cfg(windows)]` backends behind the
|
||||
existing traits:
|
||||
|
||||
**The old blocker is gone.** Rather than author + sign our own kernel IDD for the per-client virtual
|
||||
display, use **SudoVDA** (the Sunshine Virtual Display Adapter) — a pre-built, signed Indirect
|
||||
Display Driver that creates virtual displays at arbitrary WxH@Hz on demand. The `VirtualDisplay`
|
||||
backend becomes *"install + drive SudoVDA's control API"* (M effort), not *"write + WHQL-sign a
|
||||
kernel driver"* (XL). That removes the only hard blocker — the Windows host is now a medium,
|
||||
mostly-mechanical port. Recommended start: **Phase 0** — capture an existing monitor to prove the
|
||||
stack end to end; **Phase 1** wires SudoVDA for the native-resolution output. Deferred only because
|
||||
it's unbuildable on the Linux dev box; the trait boundaries are already in the right places.
|
||||
- **Capture** — DXGI Desktop Duplication → D3D11 texture.
|
||||
- **Virtual display** — **SudoVDA** (the Sunshine Virtual Display Adapter), a pre-built, signed
|
||||
Indirect Display Driver that creates a `WxH@Hz` monitor on demand — so there's no kernel driver to
|
||||
author or WHQL-sign. Bundled and staged by the installer; falls back to capturing an existing
|
||||
monitor if absent.
|
||||
- **Encode** — NVENC with a D3D11 device (`--features nvenc`), so the host is **NVIDIA-only**.
|
||||
- **Input** — SendInput (mouse/keyboard) + ViGEm (virtual gamepads with rumble).
|
||||
- **Audio** — WASAPI loopback capture + a WASAPI virtual mic.
|
||||
|
||||
It ships as a **signed Inno Setup installer** that registers a `LocalSystem` SCM service launching
|
||||
into the interactive session for secure-desktop (UAC / lock-screen) capture, published by
|
||||
`windows-host.yml`. Still open: AMD/Intel encode (none today), broader real-world testing, and
|
||||
bundling ViGEmBus.
|
||||
|
||||
## 8. Pairing & trust hardening *(§8a + §8b-1 done; §8b-2 next)*
|
||||
|
||||
@@ -356,8 +365,8 @@ GameStream already auto-discovered via mDNS (`_nvstream._tcp`). Now both the uni
|
||||
- **Client**: `punktfunk-probe --discover [SECS]` browses and prints each host (name, addr:port,
|
||||
pairing, fingerprint), then exits. Apple clients browse the same service natively via NWBrowser
|
||||
(Bonjour) — no Rust-connector dependency; this section's service type + TXT keys are the contract.
|
||||
- **Validated**: cross-LAN — dev box discovered the GNOME-box appliance
|
||||
(`home-worker-3 192.168.1.248:9777 pair=required fp=1dcf3a…`) and a standalone synthetic host
|
||||
- **Validated**: cross-LAN — a client discovered a GNOME host appliance
|
||||
(`myhost.local 9777 pair=required fp=1dcf3a…`) and a standalone synthetic host
|
||||
(`pair=optional`); fingerprint + pairing state correct in both.
|
||||
- **Next** (not done): wire NWBrowser discovery into the Apple client UI (host picker); the
|
||||
host-side contract above is all it needs.
|
||||
|
||||
@@ -79,6 +79,10 @@ session unit — see [Bazzite](/docs/bazzite).
|
||||
|
||||
## Windows
|
||||
|
||||
> punktfunk is Linux-first, but a native **Windows host** also ships — a signed installer with an SCM
|
||||
> service and a bundled virtual-display driver. It's **NVIDIA-only** (NVENC) and newer than the Linux
|
||||
> host. (Not to be confused with the Windows *client*, which streams *to* a Windows PC.)
|
||||
|
||||
On Windows the host runs as a `LocalSystem` service that launches into the interactive session, so it
|
||||
captures the secure desktop (UAC / lock screen) and survives reboots with nobody logged in — the same
|
||||
model Sunshine/Apollo use.
|
||||
@@ -98,7 +102,7 @@ way you need an NVIDIA GPU + driver (the host is NVENC-only on Windows).
|
||||
After a reboot, from another machine on the network:
|
||||
|
||||
```sh
|
||||
punktfunk-probe --discover # or just look for the host in the Apple app / Moonlight
|
||||
punktfunk-probe --discover # or just look for the host in a native client / Moonlight
|
||||
```
|
||||
|
||||
If the host is listed, it's up. If not, check `journalctl --user -u punktfunk-host` on the host.
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
---
|
||||
title: "Status & Progress"
|
||||
description: "Where the work stands, what's live on each box, and a running progress log."
|
||||
description: "Where the work stands across the core, the host, and the native clients."
|
||||
---
|
||||
|
||||
The living progress tracker. Milestone-level status lives in [`CLAUDE.md`](https://github.com)
|
||||
and the design in the [Implementation Plan](/docs/implementation-plan); this page is the
|
||||
**current state + a dated log** of what landed, kept up to date as work happens. Newest first.
|
||||
A high-level view of where punktfunk stands. The full design lives in the
|
||||
[Implementation Plan](/docs/implementation-plan), the ordered plan of work in the
|
||||
[Roadmap](/docs/roadmap), and milestone-level detail in
|
||||
[`CLAUDE.md`](https://git.unom.io/unom/punktfunk/src/branch/main/CLAUDE.md).
|
||||
|
||||
## Milestones at a glance
|
||||
|
||||
@@ -13,92 +14,105 @@ and the design in the [Implementation Plan](/docs/implementation-plan); this pag
|
||||
|---|---|
|
||||
| **Core** — `punktfunk-core` + C ABI (protocol · FEC · crypto) | ✅ complete & hardened |
|
||||
| **GameStream host** (Moonlight-compatible) | ✅ working end-to-end; HDR/surround-audio polish open |
|
||||
| **Native protocol** — `punktfunk/1` (QUIC control + UDP data) | ✅ full session planes, validated live |
|
||||
| **Native clients** — decode + present (Apple first) | 🟡 macOS stage 1 live; stage-2 presenter built + decode-tested (opt-in, present needs live validation). **Linux GTK client stage 1 live** (2026-06-12) |
|
||||
| **Native protocol** — `punktfunk/1` (QUIC control + UDP data, GF(2¹⁶) Leopard FEC + AES-GCM) | ✅ full session planes, validated live |
|
||||
| **Windows host** (NVIDIA, x64) | 🟡 implemented & shipping as a signed installer; NVIDIA-only, newer than the Linux host |
|
||||
| **macOS / iOS / iPadOS / tvOS client** | ✅ full client; on-glass stage-2 presenter behind an opt-in flag, becoming the default |
|
||||
| **Linux client** (`punktfunk-client`, GTK4/libadwaita) | ✅ full client; VAAPI zero-copy decode + software fallback |
|
||||
| **Windows client** (`punktfunk-client`, WinUI 3) | ✅ stage 1 complete; ships as signed MSIX; on-glass hardware validation pending |
|
||||
| **Android client** (phone + Android TV) | ✅ full client; hardware HEVC decode + HDR10 |
|
||||
| **Web console** (over the management API) | ✅ status · devices · pairing |
|
||||
|
||||
## Live on the boxes
|
||||
## What works today
|
||||
|
||||
| Box | Role | Compositor | Notes |
|
||||
|---|---|---|---|
|
||||
| **home-worker-2** (dev) | KDE/KWin appliance | kwin (headless Plasma) | QEMU VM, passthrough RTX 5070 Ti; `serve --native` user unit |
|
||||
| **home-worker-3** (GNOME) | GNOME/Mutter appliance | mutter (RecordVirtual) | RTX 4090; autologin GNOME Wayland; `serve --native` user unit. See [Ubuntu — GNOME](/docs/ubuntu-gnome) |
|
||||
| **home-bazzite-1** | SteamOS-like host | gamescope | host-managed Steam session at client mode. See [Bazzite Setup](/docs/bazzite) |
|
||||
punktfunk is a low-latency desktop and game streaming **host** — Linux-first (Linux + NVIDIA, NVENC),
|
||||
with a newer **NVIDIA-only Windows host** too — and native **clients** on macOS, iOS/iPadOS/tvOS,
|
||||
Linux, Windows, and Android.
|
||||
|
||||
All three appliances advertise over mDNS (`_punktfunk._udp`) and require PIN pairing by default.
|
||||
- **Two protocols.** The host speaks the **GameStream** protocol, so any **Moonlight**
|
||||
client works out of the box, plus its own lower-latency **`punktfunk/1`** protocol
|
||||
(QUIC control plane + UDP data plane with GF(2¹⁶) Leopard FEC and AES-GCM).
|
||||
- **Native resolution, no scaling.** Every session gets a virtual display at the client's
|
||||
exact resolution and refresh rate, via per-compositor backends for **KWin**,
|
||||
**gamescope**, **Mutter**, and **Sway/wlroots**.
|
||||
- **Zero-copy GPU pipeline.** Captured frames stay on the GPU (dmabuf → CUDA → NVENC) with
|
||||
automatic split-encode at very high resolutions. Stable 240 fps at 5120×1440 has been
|
||||
measured.
|
||||
- **Secure by default.** A **SPAKE2 PIN pairing** ceremony establishes trust (the host
|
||||
shows a 4-digit PIN; an attacker gets a single online guess, no offline dictionary
|
||||
attack). Trust-on-first-use (TOFU) remains an explicit opt-in for fully trusted LANs.
|
||||
- **LAN auto-discovery.** Hosts advertise over mDNS (`_punktfunk._udp`); clients browse and
|
||||
list them automatically.
|
||||
- **Full input.** Mouse, keyboard, and gamepads (including DualSense touchpad, motion,
|
||||
rumble, lightbar, player LEDs, and adaptive triggers) in both directions.
|
||||
- **Audio both ways.** Opus desktop audio host → client, plus an opt-in, paired-only client
|
||||
microphone uplink.
|
||||
- **Management surface.** A REST management API with a checked-in OpenAPI document, plus a
|
||||
web console for status, paired devices, and pairing.
|
||||
|
||||
## Progress log
|
||||
### Native clients
|
||||
|
||||
### 2026-06-12
|
||||
- **Native Linux client — stage 1, first light** (`clients/linux`, binary
|
||||
`punktfunk-client`). GTK4/libadwaita app on the **Option A** architecture picked after a
|
||||
six-angle research pass (toolkits / hw decode / Wayland presentation / input capture /
|
||||
prior art / codebase): links `punktfunk-core` directly as a crate (no C ABI;
|
||||
`NativeClient` is `Sync` now), mDNS host list, TOFU + SPAKE2 PIN pairing dialogs
|
||||
(identity shared with `client-rs`), FFmpeg software HEVC decode (`LOW_DELAY` + slice
|
||||
threads) into a `GtkGraphicsOffload`-wrapped picture, PipeWire playback with the host
|
||||
mic-player's jitter ring inverted, SDL3 gamepad capture + rumble/lightbar feedback,
|
||||
layout-independent keyboard (exact inverse of the host's VK table), absolute mouse +
|
||||
WHEEL_DELTA scroll, compositor-shortcut inhibition, fullscreen, stats overlay.
|
||||
**Validated live** against this box's `serve --native`: 1080p60 at a locked 60 fps,
|
||||
capture→decoded **p50 ≈ 6.4 ms** (software decode, debug build). Next: VAAPI dmabuf →
|
||||
`GdkDmabufTexture` (Tier-1 zero-copy on Intel/AMD clients), DualSense
|
||||
touchpad/motion/trigger replay over SDL3, then the stage-2 raw-Wayland presenter
|
||||
(wp_presentation feedback, tearing-control, Vulkan Video for NVIDIA clients).
|
||||
- **Delegated pairing approval (§8b-1)** — an unpaired device that tries to connect to a
|
||||
pairing-required host now shows up as a **pending request** in the web console's Pairing page;
|
||||
one click approves it (optionally relabeling) and pairs its certificate fingerprint — no PIN
|
||||
fetched out of band. New mgmt endpoints (`/native/pending` + approve/deny), an in-memory pending
|
||||
queue in `NativePairing` (fp-deduped, capped, 10-min expiry), and an optional **device name** in
|
||||
the `Hello` (back-compat trailing field; `client-rs --name` sends it). End-to-end tested.
|
||||
§8b-2 (approve from a paired device's own app) is the client-side follow-up.
|
||||
- **CI + deployment landed** (see the [CI & Docker](/docs/ci) guide). Gitea Actions, three
|
||||
workflows: Rust workspace checks inside the new `punktfunk-rust-ci` builder image (Ubuntu 26.04,
|
||||
full link-dep stack incl. a libcuda stub — 141/141 tests green in-container), web + docs-site
|
||||
build/typecheck, `docker.yml` building+pushing `punktfunk-web`/`punktfunk-docs`/`punktfunk-rust-ci`
|
||||
to the registry, and `apple.yml` (xcframework → `swift build`/`swift test`) on a new **host-mode
|
||||
macOS runner** (`home-mac-mini-1`, provisioned by `scripts/ci/setup-macos-runner.sh`; macOS
|
||||
Local-Network privacy forces it to run as a root LaunchDaemon). Host and native clients stay
|
||||
un-dockerized by design. **This site now deploys automatically**: `deploy-docs` ships it to
|
||||
unom-1:3220, Caddy serves <https://docs.punktfunk.unom.io> — live and verified.
|
||||
- **Concurrent sessions** — the host no longer serves one client at a time. The accept loop spawns
|
||||
each session (`JoinSet`), bounded by `--max-concurrent` (default 4, a NVENC bound; overflow waits
|
||||
in the accept queue). Each session keeps its own virtual output + encoder; they share the
|
||||
host-lifetime input/audio/mic services — i.e. **multiple devices viewing/controlling the same
|
||||
desktop** on kwin/mutter/wlroots. Validated live on the GNOME box: two clients connected at once
|
||||
→ **two independent Mutter virtual outputs (1280×720 + 1920×1080) streaming simultaneously**
|
||||
(39 MB + 48 MB). gamescope's *independent-desktops* (multi-user) isolation — per-session
|
||||
input/audio — is a follow-up.
|
||||
- **Apple client latency HUD** — `PunktfunkConnection.clockOffsetNs` (from the C-ABI getter) +
|
||||
`LatencyMeter` surface a skew-corrected **capture→client-receipt** p50/p95 in the macOS HUD: the
|
||||
first cross-machine latency the real Apple client reports. (Stage-1 `AVSampleBufferDisplayLayer`
|
||||
has no present callback, so decode→present is excluded — that needs the stage-2 presenter.)
|
||||
Needs an `xcframework` rebuild + `swift test` on the Mac to validate.
|
||||
- **Skew handshake in the connector + C ABI** — `quic::clock_sync` is now a shared core helper used
|
||||
by both the reference client and `NativeClient`; the connector runs it at connect and exposes the
|
||||
host clock offset over the C ABI (`punktfunk_connection_clock_offset_ns`). This is the substrate
|
||||
the Apple client needs for the decode→present (glass-to-glass) term.
|
||||
- **Wall-clock skew handshake** (`ClockProbe`/`ClockEcho`, 8 NTP rounds after `Start`) — makes the
|
||||
client's capture→reassembled latency valid **cross-machine**. Validated GNOME box → dev box:
|
||||
offset −1.57 ms removed, **p50 1.30 ms** skew-corrected. (`05bc9ab`)
|
||||
- **Native LAN auto-discovery** — host advertises `_punktfunk._udp` (TXT: fingerprint, pairing,
|
||||
proto); `punktfunk-probe --discover` lists hosts. Validated cross-LAN. (`4fff464`)
|
||||
- **Third test box stood up** — home-worker-3 (Ubuntu 26.04, RTX 4090, GNOME 50): first GNOME/Mutter
|
||||
zero-copy streaming on a real desktop; **1 Gbps probe clean** (625 MB/5 s, `send_dropped=0`).
|
||||
Two physical-NVIDIA gotchas documented in [Ubuntu — GNOME](/docs/ubuntu-gnome).
|
||||
- **Encode|send thread split** validated on real NIC (`send_dropped=0` at 720p60 / 1080p120). (`b295a5b`)
|
||||
| Client | Highlights |
|
||||
|---|---|
|
||||
| **macOS / iOS / iPadOS / tvOS** | VideoToolbox HEVC decode, GameController capture, full DualSense feedback, mDNS discovery, PIN pairing + TOFU, network speed test, latency HUD. Stage-2 presenter (`VTDecompressionSession` → `CAMetalLayer`, ~11 ms p50 capture→present) is built and validated on glass behind an opt-in flag, becoming the default. Ships as one universal TestFlight build / App Store listing. |
|
||||
| **Linux** (`punktfunk-client`) | GTK4/libadwaita. FFmpeg decode with VAAPI → DRM-PRIME dmabuf zero-copy (Intel/AMD; software fallback on NVIDIA), PipeWire audio + mic, SDL3 gamepads incl. DualSense, mDNS discovery, PIN pairing + TOFU, speed test. Ships as Flatpak, apt, rpm, and Arch packages. |
|
||||
| **Windows** (`punktfunk-client`) | WinUI 3. D3D11VA zero-copy decode, HDR10, WASAPI audio + mic, SDL3 gamepads incl. DualSense, mDNS discovery, and the full PIN/TOFU trust surface are all implemented. Ships as a signed MSIX (x86_64 + ARM64). **Stage 1 complete; D3D11VA decode, HDR present, and the GUI are pending on-glass validation on real GPU hardware.** |
|
||||
| **Android** (phone + Android TV) | Kotlin app with a Rust core over JNI. NDK `AMediaCodec` hardware HEVC decode + HDR10 (Main10/BT.2020 PQ), Opus/Oboe audio + mic, gamepad input with rumble/HID feedback, NsdManager mDNS discovery, PIN pairing + TOFU (Keystore identity), live stats HUD, and D-pad/controller focus navigation for TV. Ships to the Google Play Internal Testing track. |
|
||||
|
||||
### Earlier (see roadmap + git log)
|
||||
- **1 Gbps data plane**: batched `sendmmsg`/`recvmmsg` + microburst-cap paced send thread.
|
||||
- **Boot appliance**: headless KDE session + host systemd units (no login).
|
||||
- **Speed test + settable bitrate**: negotiation + bandwidth probe (host side).
|
||||
- **DualSense** UHID + haptics; gamepads live; mic uplink; AV1 + surround (unit/live-capture tested).
|
||||
`punktfunk-probe` is a headless reference and measurement client (for testing and
|
||||
benchmarking, not everyday use).
|
||||
|
||||
## Validated live on
|
||||
|
||||
The stack has been validated live on a range of hosts and clients:
|
||||
|
||||
- **Hosts:** Ubuntu (GNOME / KDE), Fedora KDE, and Bazzite (gamescope) on machines with
|
||||
RTX-class NVIDIA GPUs, across the KWin, gamescope, Mutter, and Sway/wlroots backends.
|
||||
- **Clients:** native macOS, Linux, and Android clients against live hosts, plus stock
|
||||
Moonlight clients over the GameStream path.
|
||||
- **Cross-machine latency** is measured and skew-corrected (a wall-clock handshake removes
|
||||
the inter-machine clock offset), so capture-to-present numbers are valid across the LAN.
|
||||
|
||||
## Highlights
|
||||
|
||||
Notable capabilities that have landed, newest first:
|
||||
|
||||
- **Native Linux client (stage 1).** GTK4/libadwaita app that links `punktfunk-core`
|
||||
directly: mDNS host list, TOFU + SPAKE2 PIN pairing, FFmpeg HEVC decode, PipeWire audio
|
||||
with mic uplink, SDL3 gamepad capture with rumble/lightbar feedback, layout-independent
|
||||
keyboard, absolute mouse, fullscreen, and a stats overlay. VAAPI → `GdkDmabufTexture`
|
||||
zero-copy decode on Intel/AMD with a proven software fallback.
|
||||
- **Delegated pairing approval.** An unpaired device that knocks on a pairing-required host
|
||||
appears as a pending request in the web console's Pairing page; one click approves and
|
||||
pins its certificate — no PIN fetched out of band.
|
||||
- **Concurrent sessions.** The host serves multiple clients at once (bounded by an NVENC
|
||||
limit), each with its own virtual output and encoder — e.g. stream the same desktop to a
|
||||
laptop and a TV simultaneously.
|
||||
- **Cross-machine latency HUD + wall-clock skew handshake.** A short NTP-style handshake
|
||||
aligns client and host clocks, making capture-to-reassembled latency valid across
|
||||
machines; the Apple client surfaces a skew-corrected capture-to-receipt p50/p95 in its
|
||||
HUD.
|
||||
- **Native LAN auto-discovery.** Hosts advertise `_punktfunk._udp` over mDNS (with TXT
|
||||
records carrying the protocol, cert fingerprint, and pairing requirement); clients
|
||||
discover and list them automatically.
|
||||
- **1 Gbps data plane.** Batched `sendmmsg`/`recvmmsg`, a microburst-capped paced send
|
||||
thread, and larger socket buffers, exploiting the GF(2¹⁶) Leopard FEC that breaks the
|
||||
classic ~1 Gbps GameStream ceiling.
|
||||
- **Boot appliance.** A headless compositor session plus host systemd units, so a host can
|
||||
come up and stream with no interactive login.
|
||||
- **Network speed test + settable bitrate.** Bitrate negotiation and an in-band bandwidth
|
||||
probe inform the client's bitrate picker instead of guesswork.
|
||||
- **Rich DualSense.** A full UHID DualSense backend on the host (gamepad, motion, touchpad,
|
||||
lightbar, player LEDs, adaptive triggers) with feedback carried back to the clients.
|
||||
- **AV1 + surround audio** are implemented and unit/live-capture tested.
|
||||
|
||||
## In flight / next
|
||||
|
||||
See the [Roadmap](/docs/roadmap) for the ordered list. Near-term:
|
||||
- **True glass-to-glass**: Apple client present-stamp (decode→present) + host render→capture term.
|
||||
- **Apple stage-2 presenter** (`VTDecompressionSession` → `CAMetalLayer`) — built + decode-unit-tested + live-validated on glass behind the `punktfunk.presenter` flag (capture→present ~11 ms p50); make it the default after a few resolution/HDR checks.
|
||||
- **Mandatory PIN pairing + delegated pairing approval** (an already-paired device approves a new one).
|
||||
- **gamescope multi-user isolation** — per-session input/audio so concurrent sessions are independent
|
||||
desktops (the shared-desktop multi-view case landed).
|
||||
- **bazzite** kept up to date (currently offline; one rebuild behind).
|
||||
|
||||
- **True glass-to-glass latency** — combine the client present-stamp (decode → present)
|
||||
with the host render → capture term for a complete end-to-end number.
|
||||
- **Make the Apple stage-2 presenter the default** after a few more resolution/HDR checks.
|
||||
- **On-glass validation of the Windows client** (D3D11VA decode, HDR present, GUI) on real
|
||||
GPU hardware.
|
||||
- **gamescope multi-user isolation** — per-session input/audio so concurrent sessions can
|
||||
be fully independent desktops (the shared-desktop multi-view case already works).
|
||||
|
||||
@@ -65,8 +65,9 @@ Then log out and back in. On other distros this is `sudo usermod -aG input $USER
|
||||
|
||||
## Stutter, drops, or high latency
|
||||
|
||||
- Lower the **bitrate**. On a busy or Wi-Fi link, the requested bitrate may be too high — the Apple
|
||||
app's [speed test](/docs/configuration#bitrate) picks a safe value; with Moonlight, set it manually.
|
||||
- Lower the **bitrate**. On a busy or Wi-Fi link, the requested bitrate may be too high — the native
|
||||
clients' [speed test](/docs/configuration#bitrate) picks a safe value; with Moonlight, set it
|
||||
manually.
|
||||
- Prefer a **wired** connection or 5 GHz Wi-Fi between host and client.
|
||||
- Streaming to **many devices at once** shares the GPU encoder. The production host
|
||||
(`serve --native`) handles one native session at a time, with extra clients queued; heavy load is
|
||||
|
||||
@@ -1,77 +1,70 @@
|
||||
---
|
||||
title: "Windows Host"
|
||||
description: "Feasibility and scoping for a Windows host backend."
|
||||
description: "Run the punktfunk streaming host on a Windows PC with an NVIDIA GPU."
|
||||
---
|
||||
|
||||
|
||||
**Status: scoped, deferred — but de-risked.** A Windows host is architecturally an *"add a backend"*
|
||||
job, not a parallel port. The one thing that used to make it **large** — the per-client *virtual*
|
||||
output, which has no user-mode Windows API and seemingly needed a self-signed kernel Indirect
|
||||
Display Driver (IDD) — is **solved by reusing [SudoVDA](https://github.com/VirtualDrivers), the
|
||||
Sunshine Virtual Display Adapter**: a pre-built, signed IDD that creates virtual displays at
|
||||
arbitrary `WxH@Hz` on demand. We install it and drive its control interface; **no driver to write or
|
||||
WHQL-sign.** That turns the headline feature from XL into a medium backend. This doc records what's
|
||||
left so the work can be picked up deliberately.
|
||||
**Status: implemented and shipping — NVIDIA-only, x64-only.** punktfunk is Linux-first, but it also
|
||||
runs as a native **Windows host**: a signed installer registers a `LocalSystem` service that streams
|
||||
your Windows desktop or games to any punktfunk or Moonlight client, at the client's exact resolution
|
||||
via a virtual display. It's newer and less battle-tested than the Linux host, and it is built
|
||||
specifically around NVIDIA hardware.
|
||||
|
||||
(Grounded in a 4-agent read of the host crate, 2026-06-10; SudoVDA path added 2026-06-11.)
|
||||
> This page is about the Windows **host** (streaming *from* a Windows PC). To stream *to* a Windows
|
||||
> PC, see the [Windows client](/docs/clients#windows-desktop-client).
|
||||
|
||||
## What's already done for us
|
||||
## Requirements
|
||||
|
||||
punktfunk is cleanly layered. **~95% of the codebase is platform-agnostic and reuses verbatim:**
|
||||
- **Windows 10/11, x64.** ARM64 is not supported — both NVENC and the virtual-display driver are
|
||||
x64-only.
|
||||
- **An NVIDIA GPU + driver.** The host encodes with NVENC (`nvEncodeAPI64.dll`); there is no other
|
||||
encoder backend on Windows.
|
||||
- **(Optional) ViGEmBus** for virtual gamepads — a manual prerequisite for now
|
||||
([releases](https://github.com/nefarius/ViGEmBus/releases)).
|
||||
|
||||
| Reusable as-is | Why |
|
||||
|---|---|
|
||||
| `punktfunk-core` (protocol, FEC, crypto, session, transport, **C ABI**) | Zero platform deps — no `cfg(linux)` anywhere; the C ABI is already cross-platform |
|
||||
| QUIC control plane (`quic.rs`, pairing, mode negotiation) | quinn + tokio are portable |
|
||||
| GameStream P1.1 (mDNS, serverinfo, pairing, RTSP, ENet) — *except* `stream.rs`/`audio.rs` | pure wire logic |
|
||||
| Management REST API (`mgmt.rs`) + OpenAPI | axum/tokio, portable |
|
||||
| Pipeline + `punktfunk1.rs` orchestration | trait-generic — calls `capturer.next_frame()`, `encoder.submit/poll()`; **needs zero changes** |
|
||||
| The **trait boundaries** themselves: `Capturer`, `Encoder`, `VirtualDisplay`, `InputInjector`, `AudioCapturer`, `VirtualMic` | platform-neutral signatures; Linux deps are already isolated under `[target.'cfg(target_os="linux")'.dependencies]` |
|
||||
## Install
|
||||
|
||||
So a Windows host is **new `#[cfg(target_os = "windows")]` backend modules behind the existing
|
||||
traits** — the per-frame path, protocol, and control plane don't move. No architectural refactor is
|
||||
required; the boundaries are already in the right places.
|
||||
Download the signed `punktfunk-host-setup-<ver>.exe` from the package registry and run it — it
|
||||
installs the host into `C:\Program Files\punktfunk`, optionally installs the bundled **SudoVDA**
|
||||
virtual-display driver, and registers + starts the service. Full steps (including the silent install
|
||||
and the CLI `punktfunk-host service install` path) are in
|
||||
[Running as a Service → Windows](/docs/running-as-a-service#windows); packaging internals live in
|
||||
[`packaging/windows`](https://git.unom.io/unom/punktfunk/src/branch/main/packaging/windows/README.md).
|
||||
|
||||
## What a Windows host needs (new code)
|
||||
## How it works
|
||||
|
||||
Each row is a Linux backend that needs a Windows sibling. Effort is the *implementation* effort;
|
||||
all reuse the existing trait.
|
||||
The host installs a **`LocalSystem` SCM service** that runs from Session 0 and launches a worker into
|
||||
the interactive session (`CreateProcessAsUserW`). That lets it **capture the secure desktop** (UAC
|
||||
prompts, the lock screen) and keep streaming across reboots with nobody logged in — the same model
|
||||
Sunshine and Apollo use. Service registration, firewall rules, and the supervisor all live in
|
||||
`punktfunk-host service install`; the installer just lays the exe down and calls it elevated.
|
||||
|
||||
| Subsystem | Linux today | Windows equivalent | Effort | Notes |
|
||||
|---|---|---|---|---|
|
||||
| **Capture** | xdg ScreenCast portal → PipeWire (dmabuf) | **DXGI Desktop Duplication** (or Windows.Graphics.Capture) → D3D11 texture | M | DXGI gives a GPU `B8G8R8A8` texture directly |
|
||||
| **Virtual display** | KWin/Mutter/Sway/gamescope protocols | **SudoVDA** (pre-built signed IDD) — install + drive its control API to add/remove a `WxH@Hz` virtual monitor per session | **M** | ✅ **no longer the blocker**: SudoVDA is the same IDD Sunshine ships, so no driver to author or sign. The `VirtualDisplay` backend = enable the adapter, create a monitor at the client's mode, capture it (DXGI), tear it down on session end. Fallback if SudoVDA is absent: capture an existing monitor (loses native-resolution) |
|
||||
| **Encode** | `ffmpeg-next` NVENC, CUDA hwframes | Media Foundation H.264/HEVC/AV1, **or** NVENC SDK direct with a D3D11 device context (`AVD3D11VADeviceContext`) | M–L | `encode.rs` AU/codec logic + NVENC option strings are portable; only the hwdevice + frame-pool glue swaps |
|
||||
| **Zero-copy bridge** | dmabuf → EGL/Vulkan → CUDA | D3D11 texture → NVENC (shared texture / `cudaImportExternalMemory` + D3D12 fence) | M | **optional** — a portable CPU-copy path already exists, so v1 can skip this |
|
||||
| **Input (ptr/kbd)** | libei (RemoteDesktop portal) / wlr protocols | **SendInput** (`keybd_event`/`mouse_event`) | S | the VK→evdev table just becomes VK→`VIRTUAL_KEY` (already Win32-native) |
|
||||
| **Input (gamepads)** | uinput X-Box-360 pad + FF rumble | **ViGEm** (Virtual Gamepad Emulation) + HID reports | M | rumble back-channel maps to ViGEm notifications |
|
||||
| **Audio capture** | PipeWire sink-monitor | **WASAPI loopback** (`IAudioCaptureClient`) | S–M | also produces interleaved f32 — same `AudioCapturer` contract |
|
||||
| **Virtual mic** | PipeWire `Audio/Source` | virtual audio device (VB-Cable-style WDM driver) or WASAPI render-to-fake-device | M | needs a driver or a bundled 3rd-party cable |
|
||||
| **`sendmmsg` batching** | `gamestream/stream.rs` | already has a `cfg(not(linux))` per-packet fallback | — | nothing to do |
|
||||
### One core, Windows backends
|
||||
|
||||
**Rough total: ~2,000–4,000 LOC of new Rust** (no C++ driver — SudoVDA is reused as-is), spread over
|
||||
capture/encode/vdisplay/input/audio. With the driver problem solved, the overall effort is now
|
||||
**medium**; the input+audio layer alone is *small–medium*.
|
||||
Most of punktfunk is platform-agnostic. `punktfunk-core` (protocol, FEC, crypto, session, transport,
|
||||
the C ABI), the QUIC control plane, the GameStream wire logic, the management API, and the per-frame
|
||||
pipeline orchestration are all shared with the Linux host. The Windows host is a set of
|
||||
`#[cfg(windows)]` backends behind the same traits the Linux host uses:
|
||||
|
||||
## Recommended phasing (when picked up)
|
||||
| Subsystem | Linux backend | Windows backend |
|
||||
|---|---|---|
|
||||
| **Capture** | xdg ScreenCast portal → PipeWire (dmabuf) | **DXGI Desktop Duplication** → D3D11 texture |
|
||||
| **Virtual display** | KWin / Mutter / Sway / gamescope | **SudoVDA** signed IDD — create a `WxH@Hz` monitor per session, capture it, tear it down |
|
||||
| **Encode** | `ffmpeg-next` NVENC (CUDA hwframes) | **NVENC** with a D3D11 device (`--features nvenc`) |
|
||||
| **Input — mouse/keyboard** | libei / wlr protocols | **SendInput** (Win32 VK + absolute mouse) |
|
||||
| **Input — gamepads** | uinput Xbox 360 pad + rumble | **ViGEm** virtual pad + rumble back-channel |
|
||||
| **Audio capture** | PipeWire sink-monitor | **WASAPI loopback** |
|
||||
| **Virtual mic** | PipeWire `Audio/Source` | WASAPI virtual mic |
|
||||
|
||||
1. **Phase 0 — "basic Windows host" (no virtual display).** Capture an *existing* monitor (DXGI
|
||||
Desktop Duplication) → Media Foundation/NVENC encode → SendInput + WASAPI loopback. This proves
|
||||
the whole stack on Windows with the smallest surface, reusing all of core/QUIC/GameStream/mgmt.
|
||||
It loses the per-client native-resolution output but is a working Windows host quickly.
|
||||
2. **Phase 1 — the virtual display via SudoVDA.** A `VirtualDisplay` backend that enables SudoVDA,
|
||||
creates a monitor at the client's exact `WxH@Hz`, captures it (DXGI), and tears it down on session
|
||||
end — restoring punktfunk's headline feature with **no driver authoring or signing**. (Ship/guide
|
||||
the SudoVDA install as a host prerequisite, like the udev rule on Linux.)
|
||||
3. **Phase 2 — input + audio parity.** ViGEm gamepads + rumble; WASAPI virtual mic; D3D11→NVENC
|
||||
zero-copy.
|
||||
The virtual display uses **[SudoVDA](https://github.com/VirtualDrivers)** (the Sunshine Virtual
|
||||
Display Adapter) — a pre-built, signed Indirect Display Driver — so there is **no kernel driver to
|
||||
author or WHQL-sign**. The installer bundles and stages it; if it's absent, the host falls back to
|
||||
capturing an existing monitor (losing the per-client native-resolution output).
|
||||
|
||||
## Why it's deferred (not started now)
|
||||
## Limitations
|
||||
|
||||
- The remaining work is **medium** and mechanical, but **none of it is buildable or testable on the
|
||||
Linux dev box** — it would be unvalidated code until there's a Windows box in the loop.
|
||||
- SudoVDA removed the hard blocker (the signed kernel driver); what's left is a backend port, picked
|
||||
up whenever a Windows target is in scope.
|
||||
|
||||
The architecture is ready whenever the work is scheduled; this doc + the clean trait boundaries are
|
||||
the down payment. Start at **Phase 0** for the fastest path to a working Windows host.
|
||||
- **NVIDIA-only.** NVENC is the only encoder backend — there is no AMD / Intel / software encode path
|
||||
on Windows.
|
||||
- **x64-only.** No ARM64 build (no ARM64 NVIDIA driver, and SudoVDA is x64-only).
|
||||
- **Newer than the Linux host.** The Linux host is the most battle-tested path; the Windows host is
|
||||
more recent, with the virtual-mic and gamepad backends the youngest pieces.
|
||||
|
||||
Reference in New Issue
Block a user