diff --git a/README.md b/README.md index 6145df7..6138592 100644 --- a/README.md +++ b/README.md @@ -1,98 +1,142 @@ # punktfunk -*A ground-up low-latency desktop streaming stack, built Linux-first, with a shared Rust -protocol core and native clients per platform.* +**Low-latency desktop and game streaming, Linux-first.** Run the host on a Linux machine — or a +Windows PC — with an NVIDIA GPU, connect from a Mac, PC, phone, tablet, or TV, and stream your desktop +or games — each device at its **own native resolution and refresh rate**, over your local network. -`punktfunk` is a placeholder codename. The bet: ship a **Linux virtual-display streaming -host** that speaks the existing Moonlight protocol (every Moonlight/Artemis client works -day one), then break the ~1 Gbps FEC wall with a **GF(2¹⁶) Leopard-RS** transport as a -negotiated extension. See [`docs/implementation-plan.md`](docs/implementation-plan.md). +📖 **Documentation: [docs.punktfunk.unom.io](https://docs.punktfunk.unom.io)** — start with +[How It Works](https://docs.punktfunk.unom.io/docs/how-it-works) or the +[Quick Start](https://docs.punktfunk.unom.io/docs/quickstart). + +punktfunk pairs a **virtual-display streaming host** with native clients on every platform. It speaks +the existing **GameStream** protocol, so any [Moonlight](https://moonlight-stream.org/) client works +day one — and adds its own faster **`punktfunk/1`** protocol that breaks the ~1 Gbps FEC wall with a +**GF(2¹⁶) Leopard-RS** transport. A single shared **Rust core** (`punktfunk-core`) holds the +protocol, FEC, and crypto, linked into the host and every client over a stable C ABI. + +## What makes it different + +- **Your device's exact mode.** For each client that connects, the host spins up a virtual display + sized to that device — 1080p60 to a laptop, 1440p120 to a desktop, 4K to a TV, all at once. No + letterboxing, no scaling, no rearranging your real monitors. +- **Low latency, GPU end to end.** Frames go straight from the compositor to the NVENC encoder with + zero CPU copies (dmabuf → CUDA/Vulkan → NVENC), over a transport tuned for responsiveness rather + than throughput. Stable 240 fps at 5120×1440; sub-millisecond capture-to-reassembly on a LAN. +- **Works with what you already have.** Any Moonlight/Artemis client connects over GameStream — and + native apps for macOS, Linux, Windows, and Android use the lower-latency `punktfunk/1` protocol. +- **Secure by default.** Hosts require a one-time SPAKE2 **PIN pairing**; after that, devices + reconnect on a pinned identity. No accounts, no cloud. Hosts auto-advertise over mDNS, so clients + find them on the network without typing an IP. ## Status -| Milestone | State | +| Component | State | |-----------|-------| -| **Core — `punktfunk-core` + C ABI** | ✅ done & hardened (FEC, packetization, AES-GCM, session, adversarial-review fixes, `punktfunk_core.h`) | -| **GameStream host → stock Moonlight** | ✅ live end-to-end: pairing, RTSP, audio, per-client virtual output at native res, GPU zero-copy NVENC, gamepads | -| **Native protocol — `punktfunk/1`** | ✅ validated live: QUIC control + GF(2¹⁶) FEC/AES data plane, SPAKE2 PIN pairing, mid-stream mode renegotiation | -| **Native clients — decode + present** | 🟡 macOS first light: AnnexB→VideoToolbox HEVC on glass + input/pairing over `punktfunk/1` (`clients/apple`); iOS + presenter next | -| **Web console + management API** | ✅ TanStack web console (`web/`) over the OpenAPI mgmt API: host status, paired devices, on-demand native pairing (arm → show PIN) | +| **Core** — `punktfunk-core` + C ABI (protocol · FEC · crypto · QUIC) | ✅ Complete & hardened | +| **GameStream host** → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads | +| **Native protocol** — `punktfunk/1` | ✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation | +| **Windows host** (NVIDIA, x64) | 🟡 Implemented & shipping as a signed installer (DXGI capture · SudoVDA virtual display · NVENC · WASAPI · ViGEm); NVIDIA-only, newer than the Linux host | +| **macOS / iOS / tvOS client** (`clients/apple`) | ✅ Streaming live: VideoToolbox decode, controllers incl. DualSense, discovery, pairing, speed test | +| **Linux client** (`clients/linux`, GTK4) | ✅ Streaming live: FFmpeg + VAAPI zero-copy decode, PipeWire audio, SDL3 controllers; ships as Flatpak/apt/rpm/Arch | +| **Android client** (`clients/android`, phone + TV) | ✅ Streaming live: AMediaCodec decode + HDR10, Oboe audio, controllers, discovery, pairing | +| **Windows client** (`clients/windows`, WinUI 3) | 🟡 Stage 1 complete, ships as signed MSIX (x64 + ARM64); D3D11VA decode + HDR present pending on-glass validation | +| **Web console + management API** (`web/`) | ✅ TanStack console over the OpenAPI mgmt API: host status, paired devices, on-demand PIN pairing | -The **GameStream host works with a stock Moonlight client** — validated live on NVIDIA -(RTX 5070 Ti & RTX 4090, driver 595): trust-on-first-use pairing that persists, an app -catalog, RTSP/ENet/audio, and **video at the client's exact resolution and refresh** via a -per-session virtual output (KWin, gamescope, Mutter, Sway backends), encoded with GPU -**zero-copy** (dmabuf → CUDA/Vulkan → NVENC) at up to 5120×1440@240. The native -**`punktfunk/1`** protocol adds a QUIC control plane and a GF(2¹⁶) Leopard-FEC + AES-GCM data -plane (p50 ~0.8 ms capture→reassembled at 720p120). Its trust model is **SPAKE2 PIN pairing by -default** — a new host requires the PIN ceremony; trust-on-first-use is an explicit host opt-in -(`punktfunk1-host --allow-tofu` / `serve --open`, advertised as `pair=optional`) for fully trusted LANs. Both -run from **one process** (`serve --native`), managed through a REST API + web console. Builds -against FFmpeg 7 or 8; deployed live on Bazzite. Full status: [`CLAUDE.md`](CLAUDE.md); -roadmap, setup guides & progress: the docs site ([`docs-site/`](docs-site) — Fumadocs; -`bun run dev`), with the canonical [roadmap](docs-site/content/docs/roadmap.md) and -[status](docs-site/content/docs/status.md) there. Design notes stay in [`docs/`](docs). +The **GameStream host works with a stock Moonlight client** — validated live on NVIDIA hardware +(RTX 5070 Ti, RTX 4090): PIN pairing that persists across restarts, an app catalog, RTSP/ENet/audio, +and **video at the client's exact resolution and refresh** via a per-session virtual output (KWin, +gamescope, Mutter, and Sway/wlroots backends), encoded with GPU **zero-copy** (dmabuf → CUDA/Vulkan → +NVENC) up to 5120×1440@240. The native **`punktfunk/1`** protocol adds a QUIC control plane and a +GF(2¹⁶) Leopard-FEC + AES-GCM data plane (p50 ~0.8 ms capture→reassembled at 720p120), with +mid-stream mode renegotiation and a wall-clock skew handshake so latency stays valid across machines. +Both protocols run from **one process** (`punktfunk-host serve --native`) and are managed through a +REST API and web console. Builds against FFmpeg 7 or 8. -## Install (host) +Full milestone status: **[docs.punktfunk.unom.io/docs/status](https://docs.punktfunk.unom.io/docs/status)** · +roadmap: **[/docs/roadmap](https://docs.punktfunk.unom.io/docs/roadmap)**. -The package registries are the real distribution channel — pick your distro and run one command. -Per-distro setup (add the repo, first-run, web console) lives in the linked READMEs. +## Install the host -| Distro | One-command happy path | Details | -|--------|------------------------|---------| -| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-host` *(after adding the repo)* | [`packaging/debian/README.md`](packaging/debian/README.md) | -| **Fedora / Bazzite** (rpm-ostree) | `rpm-ostree install punktfunk punktfunk-web` *(after adding the repo; or the bootc image)* | [`packaging/rpm/README.md`](packaging/rpm/README.md) | -| **Arch / Steam Deck** (PKGBUILD / sysext) | `makepkg -si` *(Arch)* · sysext `.raw` *(SteamOS/Deck)* | [`packaging/arch/README.md`](packaging/arch/README.md) | +Pick your platform and install from its package registry — the per-platform guide covers adding the +repo, first run, and the web console. The Linux host is the primary, most battle-tested path; a +Windows host (NVIDIA-only) also ships as a signed installer. -`punktfunk-host` is the streaming host; `punktfunk-web` is the browser console (pairing + status); -`punktfunk-client` is the GTK4 desktop client (also shipped via apt/RPM/Arch/Flatpak). After install, -run `punktfunk-host serve --native` inside your desktop session, then pair from the web console. +| Platform | Install | Guide | +|--------|---------|-------| +| **Ubuntu / Debian** (apt) | `sudo apt install punktfunk-host` *(after adding the repo)* | [Ubuntu — GNOME](https://docs.punktfunk.unom.io/docs/ubuntu-gnome) · [KDE](https://docs.punktfunk.unom.io/docs/ubuntu-kde) | +| **Fedora / Bazzite** (rpm-ostree) | `rpm-ostree install punktfunk punktfunk-web` *(or the bootc image)* | [Fedora — KDE](https://docs.punktfunk.unom.io/docs/fedora-kde) · [Bazzite](https://docs.punktfunk.unom.io/docs/bazzite) | +| **Arch / Steam Deck** (PKGBUILD / sysext) | `makepkg -si` *(Arch)* · sysext `.raw` *(SteamOS)* | [packaging/arch](packaging/arch/README.md) | +| **Windows** (NVIDIA, x64) | signed `setup.exe` from the package registry | [Windows Host](https://docs.punktfunk.unom.io/docs/windows-host) | -Building from source (below) is a fallback. +`punktfunk-host` is the streaming host; `punktfunk-web` is the browser console (pairing + status). +After install, run `punktfunk-host serve --native` inside your desktop session, then pair from the web +console. Full instructions: **[docs.punktfunk.unom.io/docs/install](https://docs.punktfunk.unom.io/docs/install)**. -## Layout +## Connect a client -``` -crates/ - punktfunk-core/ protocol · FEC · pacing · crypto · quic — the C ABI (lib + cdylib + staticlib) - punktfunk-host/ Linux host: vdisplay · capture · encode · inject · gamestream · punktfunk1 · mgmt · native_pairing -clients/ - probe/ punktfunk/1 reference/probe client (headless test + latency measurement) - linux/ windows/ native desktop clients (Rust: GTK4 / WinUI 3, link punktfunk-core directly) - apple/ android/ Swift (macOS+iOS) · Kotlin app + native/ Rust JNI core - decky/ Steam Deck Decky plugin -web/ TanStack web console (host status · paired devices · pairing) over the mgmt API -packaging/ Fedora/Bazzite RPM · bootc image · COPR (see packaging/bazzite/README.md) -include/punktfunk_core.h cbindgen-generated C header (checked in) -tools/{latency-probe,loss-harness}/ measurement (plan §10) -docs/{implementation-plan,roadmap,windows-host,dualsense-haptics}.md -``` +| Streaming to… | Use | +|---|---| +| Mac, iPhone, iPad, Apple TV | The **Apple app** (`clients/apple`) — also on TestFlight | +| Linux desktop / laptop, Steam Deck | **`punktfunk-client`** (Flatpak / apt / rpm / Arch) | +| Android phone or TV | The **Android app** (`clients/android`) | +| Windows | Native **`punktfunk-client`** (signed MSIX) or **Moonlight** | +| Anything else (browser, old phone, smart TV) | **Moonlight** over GameStream | + +Each client discovers hosts on the network automatically and does a one-time +[PIN pairing](https://docs.punktfunk.unom.io/docs/pairing). Per-device install steps: +**[/docs/install-client](https://docs.punktfunk.unom.io/docs/install-client)**. ## Build & test (from source) For development, or as an install fallback where no package is available: ```sh -cargo build --workspace # green on Linux and macOS +cargo build --workspace # the Rust core, host, Linux client, and probe (Linux & macOS) cargo test --workspace # unit + loopback + proptest + C ABI harness -cargo clippy --workspace --all-targets +cargo clippy --workspace --all-targets -- -D warnings +cargo fmt --all --check cargo run -p loss-harness # FEC loss-resilience sweep (no network needed) -bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link+round-trip proof +bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip proof ``` The C header regenerates from `crates/punktfunk-core/src/abi.rs` on every build (cbindgen via -`build.rs`) into `include/punktfunk_core.h`. +`build.rs`) into `include/punktfunk_core.h`. The Apple, Android, and Windows clients have their own +toolchains (Xcode/`swift build`, Gradle, and `cargo` on the MSVC target) — see each client's README +and the [docs site](https://docs.punktfunk.unom.io). + +## Layout + +``` +crates/ + punktfunk-core/ protocol · FEC · pacing · crypto · QUIC control plane — the C ABI (lib + cdylib + staticlib) + punktfunk-host/ Linux host: virtual displays · capture · encode · input · GameStream · punktfunk/1 · mgmt +clients/ + apple/ macOS / iOS / tvOS app (Swift · VideoToolbox · Metal · GameController) + linux/ Linux desktop app (Rust · GTK4/libadwaita · FFmpeg/VAAPI · PipeWire · SDL3) + windows/ Windows desktop app (Rust · WinUI 3 · D3D11 · WASAPI · SDL3) + android/ Android phone + TV app (Kotlin · Rust JNI core · AMediaCodec · Oboe) + probe/ headless reference / measurement client for punktfunk/1 + decky/ Steam Deck Decky plugin +web/ web console (TanStack) over the management API — status · devices · pairing +packaging/ apt · rpm / COPR · Arch · Flatpak · Bazzite bootc image +docs-site/ public documentation site (Fumadocs) — https://docs.punktfunk.unom.io +docs/ design notes & deep-dive plans +include/punktfunk_core.h cbindgen-generated C header (checked in) +tools/ latency-probe · loss-harness (measurement) +``` ## Design invariants -- **One core, linked everywhere.** Protocol/FEC/crypto/pacing live in `punktfunk-core` exactly - once, exposed over a stable, versioned C ABI (`punktfunk_abi_version()`, `PunktfunkConfig` - carries its own `struct_size`). -- **No async on the hot path.** The per-frame pipeline uses native threads only; - `tokio`/`quinn` are gated behind the off-by-default `quic` feature (control plane only). -- **FEC is the wall-breaker.** GF(2⁸) (≤255 shards/block) for Moonlight compat; - GF(2¹⁶) (≤65535 shards/block, SIMD, O(n log n)) to push past ~1 Gbps. +- **One core, linked everywhere.** Protocol, FEC, and crypto live in `punktfunk-core` exactly once, + exposed over a stable, versioned C ABI (`punktfunk_abi_version()`, `PunktfunkConfig` carries its own + `struct_size`). Every native client links the same core. +- **No async on the hot path.** The per-frame pipeline uses native threads only; `tokio`/`quinn` are + gated behind the off-by-default `quic` feature (control plane only). +- **Native client resolution, no scaling.** Each session gets a virtual output at exactly the + client's WxH@Hz; each compositor keeps its own backend behind a shared `VirtualDisplay` trait. +- **FEC is the wall-breaker.** GF(2⁸) (≤255 shards/block) for Moonlight compatibility; GF(2¹⁶) + (≤65535 shards/block, SIMD, O(n log n)) for `punktfunk/1` to push past ~1 Gbps. ## License diff --git a/clients/android/README.md b/clients/android/README.md index 73f2eb6..92fe991 100644 --- a/clients/android/README.md +++ b/clients/android/README.md @@ -19,15 +19,19 @@ The single seam is `io.unom.punktfunk.kit.NativeBridge` ⇄ `Java_io_unom_punktf ## Layout ``` -clients/android/native/ Rust cdylib (workspace member) - src/lib.rs JNI_OnLoad + abiVersion/coreVersion (native-link proof) - src/session.rs session handle lifecycle (connect/close); plane pumps = TODO +clients/android/native/ Rust cdylib (workspace member) — links punktfunk-core directly + src/lib.rs JNI seam (connect/pair, input, plane getters, abi/core version) + src/session.rs session lifecycle + plane pumps + src/decode.rs AnnexB → AMediaCodec HEVC hardware decode → SurfaceView (incl. HDR10) + src/audio.rs · src/mic.rs Opus + Oboe playback / mic uplink (jitter ring) + src/feedback.rs rumble + HID output (lightbar / adaptive triggers) + src/stats.rs live video stats clients/android/ Gradle project (this dir) settings.gradle.kts · build.gradle.kts · gradle.properties · gradlew - app/ :app — Compose application (MainActivity) - kit/ :kit — Android library: NativeBridge + the cargo-ndk build - build.gradle.kts cargoNdk{Debug,Release} → src/main/jniLibs//*.so + app/ :app — Compose UI: Connect / Settings / Stream screens (phone + TV) + kit/ :kit — NativeBridge · discovery (NsdManager) · Gamepad · Keymap · + security (Keystore identity + known-host store) · cargo-ndk build ``` ## Prerequisites @@ -57,15 +61,22 @@ cd clients/android # Emulators (created during env setup): emulator -avd pf_phone | emulator -avd pf_tv ``` -The debug APK lands in `app/build/outputs/apk/debug/`. The scaffold screen calls -`NativeBridge.abiVersion()` across JNI — a live ABI version proves the whole native stack is wired. +The debug APK lands in `app/build/outputs/apk/debug/`. Launch it, pick a host from the list, pair, +and stream. ## Status -- **Scaffold (done):** Gradle modules, cargo-ndk wiring, JNI native-link proof, phone+TV-installable - manifest. `crates/punktfunk-core` `rcgen` switched to the `ring` backend so the client `.so` is - aws-lc-free. -- **Next (Android stage 1):** video decode (`AMediaCodec` async → `SurfaceView`), audio - (Opus + Oboe + jitter ring), input capture → `send_input`, pairing/identity (Keystore-wrapped), - mDNS discovery, the phone/TV Compose UI. The Rust-side homes are stubbed in - `clients/android/native/src/session.rs` with port pointers to `clients/linux`. +A working native client (phone + Android TV), at parity with the Linux and Apple apps for the core +streaming experience: + +- **Video** — `AMediaCodec` hardware HEVC decode → `SurfaceView`, including **HDR10** (Main10 / + BT.2020 PQ), with low-latency decode tuning and a live stats HUD. +- **Audio** — Opus + Oboe playback with a jitter ring, plus mic uplink to the host. +- **Input** — game controllers (buttons + axes) with rumble and HID feedback; D-pad / + game-controller focus navigation for the couch (TV + phone). +- **Discovery & trust** — `NsdManager` mDNS host list, SPAKE2 PIN pairing and TOFU, with a + Keystore-wrapped client identity and a known-host store. +- **UI** — Compose host list / settings / stream screens, Material You theming. +- **Shipping** — built for `arm64-v8a` + `x86_64`; published to Google Play (Internal Testing). + +`crates/punktfunk-core` uses the `ring` `rcgen` backend so the client `.so` is aws-lc-free. diff --git a/clients/apple/README.md b/clients/apple/README.md index a8f5028..99ce099 100644 --- a/clients/apple/README.md +++ b/clients/apple/README.md @@ -6,9 +6,14 @@ input datagrams, Opus audio, cert pinning — lives in the shared Rust core (sta linked as `PunktfunkCore.xcframework`); this package is the Swift shell: decode (VideoToolbox), present (SwiftUI), input capture. -## Status — first light achieved (2026-06-10) +## Status — working client (macOS, with iOS / tvOS in the shared build) -Validated live, Mac ↔ Linux box over the LAN: gamescope virtual output → NVENC HEVC → +A full streaming client: VideoToolbox HEVC decode, controllers incl. DualSense feedback, host +discovery, PIN pairing, and a network speed test. The lower-latency **stage-2 presenter** +(`VTDecompressionSession` → `CAMetalLayer`) is built and opt-in (Settings → Presenter); see below. + +First light was achieved 2026-06-10 — validated live, Mac ↔ a Linux host over the LAN: gamescope +virtual output → NVENC HEVC → `punktfunk/1` (GF(2¹⁶) FEC + AES-GCM over UDP, QUIC control) → VideoToolbox → `AVSampleBufferDisplayLayer` on glass at 1280×720@60, with mouse/keyboard flowing back as QUIC datagrams into the host's gamescope EIS injector (thousands of events injected during diff --git a/docs-site/content/docs/apple-stage2-presenter.md b/docs-site/content/docs/apple-stage2-presenter.md index 3d52db3..b4822bd 100644 --- a/docs-site/content/docs/apple-stage2-presenter.md +++ b/docs-site/content/docs/apple-stage2-presenter.md @@ -3,7 +3,11 @@ title: "Apple Stage-2 Presenter (handoff)" description: "Implementation plan for the explicit VTDecompressionSession → CAMetalLayer presenter — hand-paced present + true decode→present (glass-to-glass) measurement. Written so a Mac agent can pick it up." --- -A pickup-ready plan for the **stage-2 Apple presenter**. The current **stage-1** presenter feeds +> **Status update:** the stage-2 presenter described here has since been **built and live-validated**, +> shipping behind an opt-in flag (`AVSampleBufferDisplayLayer` remains the default known-good path). +> This page is preserved as the implementation/handoff record for that work. + +The implementation plan for the **stage-2 Apple presenter**. The **stage-1** presenter feeds compressed HEVC straight into `AVSampleBufferDisplayLayer`, which hardware-decodes **and presents internally with no per-frame callback** — so we can't stamp decode or present, and we can't hand-pace. Stage-2 takes explicit control: decode with `VTDecompressionSession`, present decoded frames through a diff --git a/docs-site/content/docs/ci.md b/docs-site/content/docs/ci.md index 7ac27cc..68ceefe 100644 --- a/docs-site/content/docs/ci.md +++ b/docs-site/content/docs/ci.md @@ -3,17 +3,19 @@ title: "CI & Docker" description: "Gitea Actions setup — workflows, the dockerized pieces, and the runners." --- -CI runs on **Gitea Actions** (`git.unom.io`, org `unom`). Three workflows in -`.gitea/workflows/`, two runners, three images in the Gitea container registry. +CI runs on **Gitea Actions** (`git.unom.io`, org `unom`). The workflows live in +`.gitea/workflows/`; they run across Linux and macOS runners and push a few images to the +Gitea container registry. ## Workflows | Workflow | Trigger | Runner | What it does | |---|---|---|---| -| `ci.yml` | push to `main`, PRs | `ubuntu-24.04` | Rust workspace (fmt · clippy `-D warnings` · build · test · C-ABI harness · generated-header drift) inside the `punktfunk-rust-ci` image; `web/` and `docs-site/` build + typecheck in `oven/bun:1` | -| `docker.yml` | push to `main`, `v*` tags, manual | `ubuntu-24.04` | Builds + pushes the three images below (`latest` + `sha-` tags) | -| `apple.yml` | push to `main`, PRs, manual | `macos-arm64` | Rust core → `PunktfunkCore.xcframework` → `swift build` + `swift test` in `clients/apple` | -| `release.yml` | `v*` tags, manual | `macos-arm64` | Production Apple builds: sandboxed macOS `.dmg` (Developer ID, notarized, stapled) attached to the Gitea release + macOS/iOS/tvOS archives uploaded to TestFlight | +| `ci.yml` | push to `main`, PRs | Linux | Rust workspace (fmt · clippy `-D warnings` · build · test · C-ABI harness · generated-header drift) inside the `punktfunk-rust-ci` image; `web/` and `docs-site/` build + typecheck in `oven/bun:1` | +| `docker.yml` | push to `main`, `v*` tags, manual | Linux | Builds + pushes the images below (`latest` + `sha-` tags) | +| `apple.yml` | push to `main`, PRs, manual | macOS | Rust core → `PunktfunkCore.xcframework` → `swift build` + `swift test` in `clients/apple` | +| `release.yml` | `v*` tags, manual | macOS | Production Apple builds: sandboxed macOS `.dmg` (Developer ID, notarized, stapled) attached to the Gitea release + macOS/iOS/tvOS archives uploaded to TestFlight | +| `windows-msix.yml` | push to `main`, `v*` tags, manual | Windows | Builds the Windows client for `x86_64`/`aarch64` and packages signed MSIX artifacts | ## Dockerized pieces @@ -26,20 +28,20 @@ the GPU/compositor stack of the box it runs on). What is: | `git.unom.io/unom/punktfunk-docs` | `docs-site/Dockerfile` | This site; `PORT` (3000) | | `git.unom.io/unom/punktfunk-rust-ci` | `ci/rust-ci.Dockerfile` | Ubuntu 26.04 + FFmpeg 8/PipeWire/GL/GBM dev libs + a libcuda **link stub** (driver userspace, no kernel module) + pinned rustup — the container `ci.yml`'s Rust job runs in | -Registry pushes authenticate with the repo Actions secret **`REGISTRY_TOKEN`** (a PAT +Registry pushes authenticate with a repo Actions secret holding a registry token (a PAT with `write:package`; the login username in `docker.yml` is the token owner, not the push actor). ## Runners -- **`ubuntu-24.04`** — the pre-existing Linux runner; runs the Rust/web/docs jobs (as - docker containers) and the image build+push jobs. -- **`macos-arm64`** — `home-mac-mini-1` (M-series, macOS 26), a **host-mode** - `act_runner` (upstream now ships it as `gitea-runner`) provisioned by +- **Linux runner** — runs the Rust/web/docs jobs (as docker containers) and the image + build+push jobs. +- **macOS runner** — an Apple-silicon Mac running macOS, a **host-mode** `act_runner` + (upstream now ships it as `gitea-runner`) provisioned by [`scripts/ci/setup-macos-runner.sh`](https://git.unom.io/unom/punktfunk/src/branch/main/scripts/ci/setup-macos-runner.sh): rustup (+ both darwin targets for the universal xcframework), Node.js (host-mode runners execute JS actions via `node` from PATH — nothing auto-provisions it), the runner binary - in `~/.local/bin`, state in `~/ci/act-runner/` (config, `.runner` registration, + in `~/.local/bin`, state under `~/ci/act-runner/` (config, `.runner` registration, `runner.log`), kept alive by the `io.gitea.act_runner` **root LaunchDaemon** — it cannot be a user LaunchAgent: macOS Local Network privacy silently blocks LAN dials ("no route to host") from unbundled CLI binaries in gui/user launchd domains, while @@ -47,14 +49,12 @@ push actor). (CLT alone only covers `swift build/test`); if `xcode-select` still points at CLT, the script auto-detects `/Applications/Xcode*.app` and bakes a `DEVELOPER_DIR` override into the daemon environment — no `xcode-select -s` required. +- **Windows runner** — builds and packages the native Windows client (MSIX) for the + release matrix. -Re-provisioning (idempotent) or first-time registration from a dev box: - -```sh -# token: org unom → Settings → Actions → Runners → Create new runner -ssh enricobuehler@192.168.1.135 GITEA_RUNNER_TOKEN= bash -s \ - < scripts/ci/setup-macos-runner.sh -``` +Re-provisioning is idempotent — re-running `scripts/ci/setup-macos-runner.sh` on the macOS +runner with a fresh `GITEA_RUNNER_TOKEN` (org `unom` → Settings → Actions → Runners → +Create new runner) re-registers it without manual cleanup. ## Apple releases @@ -97,18 +97,16 @@ linking) and **refuses iOS/tvOS slices** (CLT has no iOS SDK). ## Deployment `docker.yml`'s `deploy-docs` job ships this docs site after every image push: it syncs -`compose.production.yml` to `~/punktfunk-docs` on **unom-1** (the DMZ services VM -website and cms deploy to) and runs `docker compose pull && up -d` there over SSH (same -pattern and secret set as `unom/website`: `DEPLOY_HOST` / `DEPLOY_USER` / `DEPLOY_PORT` / -`DEPLOY_SSH_KEY`, the `unom-ci-deploy` key). The container binds host port **3220**; -Caddy on `home-reverse-proxy-1` serves it as (vhost in -`unom/reverse-proxy`, UniFi firewall allowlist Caddy→unom-1:3220 in `unom/infra` -`proxmox/unom-1`). The host and the web console are NOT deployed — the console -fronts a punktfunk host's management API on whatever box runs the host. +`compose.production.yml` to the docs server and runs `docker compose pull && up -d` there +over SSH, driven by a small set of deploy secrets (`DEPLOY_HOST` / `DEPLOY_USER` / +`DEPLOY_PORT` / `DEPLOY_SSH_KEY`). A reverse proxy in front of that server serves the +container as . The host and the web console are NOT +deployed — the console fronts a punktfunk host's management API on whatever box runs the +host. ## Troubleshooting -- **Mac runner offline** — `ssh tail -50 '~/ci/act-runner/runner.log'`; restart with +- **macOS runner offline** — check `~/ci/act-runner/runner.log` on the runner; restart with `sudo launchctl kickstart -k system/io.gitea.act_runner`. "no route to host" in the log means the daemon is running in a gui/user domain again — see the Local Network note above. diff --git a/docs-site/content/docs/clients.md b/docs-site/content/docs/clients.md index fd4e25f..9f60db4 100644 --- a/docs-site/content/docs/clients.md +++ b/docs-site/content/docs/clients.md @@ -3,8 +3,9 @@ title: Clients description: The ways to connect to a punktfunk host — the Apple app, Moonlight, or the Linux client. --- -A punktfunk host accepts clients over its own `punktfunk/1` protocol (the Apple and Linux apps) and -over GameStream (Moonlight). Pick whichever fits the device you're streaming *to*. Ready to install? +A punktfunk host accepts clients over its own `punktfunk/1` protocol (the macOS, Linux, Windows, and +Android apps) and over GameStream (Moonlight). Pick whichever fits the device you're streaming *to*. +Ready to install? **[Install a Client](/docs/install-client)** has the step-by-step for every device. ## Apple app (Mac, iPhone, iPad, Apple TV) @@ -24,8 +25,9 @@ Open the app, pick your host, [pair](/docs/pairing) once, and stream. It builds ## Moonlight (anything else) punktfunk also speaks the **GameStream** protocol, so any [Moonlight](https://moonlight-stream.org/) -client — Windows, Android, Steam Deck, a browser, an old phone — connects with no punktfunk-specific -software. See [Connect with Moonlight](/docs/moonlight). +client — a browser, a smart TV, an old phone, a games console — connects with no punktfunk-specific +software. (Most platforms also have a native punktfunk app below — Moonlight is the catch-all.) See +[Connect with Moonlight](/docs/moonlight). This is the broadest-compatibility option and great for couch gaming. It doesn't use the native protocol's FEC/encryption extensions, but for a healthy LAN that rarely matters. @@ -54,15 +56,31 @@ connect straight away: punktfunk-client --connect :9777 # skip the picker, start a session immediately ``` -## Windows desktop client (in development) +## Android app (phone + Android TV) -`punktfunk-client` for Windows (`clients/windows`) is the native graphical client -for Windows — pure Rust, the same `punktfunk/1` core as the Apple and Linux apps, with a **WinUI 3** -UI (host list, settings, PIN pairing) and the video on a `SwapChainPanel`, plus WASAPI audio, FFmpeg -decode, SDL3 controllers, network discovery, and PIN pairing. Launch it and pick a host from the -list, just like the Apple and Linux apps. It builds on `x86_64-pc-windows-msvc`; hardware (D3D11VA) -decode, 10-bit/HDR present, and packaging are in progress, so it is not yet shipped. A headless CLI -path exists for scripting/measurement: +The native Android app speaks `punktfunk/1` directly, on both phones and Android TV. It does hardware +HEVC decode (including HDR10), Opus audio with a mic uplink, game controllers with rumble and +DualSense feedback, automatic host discovery, PIN pairing with pinned reconnects, and a live stats +overlay — with D-pad and game-controller focus navigation for the couch. It builds from the +`clients/android` directory (Kotlin + a shared Rust core). + +Install it from **Google Play** — see [Install a Client](/docs/install-client#android). Open the app, +pick your host, [pair](/docs/pairing) once, and stream. + +## Windows desktop client + +`punktfunk-client` for Windows (`clients/windows`) is the native graphical client for Windows — pure +Rust, the same `punktfunk/1` core as the Apple, Linux, and Android apps, with a **WinUI 3** UI (host +list, settings, PIN pairing) and the video on a `SwapChainPanel`. It does D3D11VA hardware decode +(software fallback), 10-bit/HDR present, WASAPI audio + mic, SDL3 controllers (rumble, lightbar, +DualSense), network discovery, and the full PIN-pairing trust surface. It builds for both `x86_64` +and `aarch64` and ships as a **signed MSIX**. Launch it and pick a host from the list, just like the +other native apps. + +> The hardware-decode and HDR paths are complete but still pending validation on real GPU hardware. +> If anything misbehaves, **[Moonlight](/docs/moonlight)** is a proven alternative for Windows. + +A headless CLI path exists for scripting/measurement: ```sh punktfunk-client # open the WinUI 3 window (host list / settings) @@ -70,7 +88,7 @@ punktfunk-client --discover # list hosts on the network punktfunk-client --headless --connect :9777 # no window: connect, count frames, print stats ``` -Until it ships, **Moonlight** remains the recommended way to stream to Windows (see below). +Prefer the broadest compatibility, or no install? **Moonlight** also streams to Windows (see below). ## Linux reference client (headless) @@ -89,7 +107,9 @@ punktfunk-probe --connect :9777 --pin # connect to one |---|---| | A Mac, iPhone, iPad, or Apple TV | The **Apple app** | | A Linux desktop or laptop, or a Steam Deck | **`punktfunk-client`** (GTK4) | -| Windows, Android, a browser, a TV | **Moonlight** | +| An Android phone or TV | The **Android app** | +| Windows | The native **`punktfunk-client`** (signed MSIX) or **Moonlight** | +| A browser, a smart TV, or any other device | **Moonlight** | | Automated tests / latency measurement | **`punktfunk-probe`** (headless) | Whichever you choose, the first connection needs a one-time [pairing](/docs/pairing). diff --git a/docs-site/content/docs/configuration.md b/docs-site/content/docs/configuration.md index 59ab380..e9c1823 100644 --- a/docs-site/content/docs/configuration.md +++ b/docs-site/content/docs/configuration.md @@ -30,15 +30,15 @@ These tell the host which desktop session to attach to. Your setup guide sets th You don't set these on the host — **the client chooses them**. When a device connects, the host creates a virtual display at that device's resolution and refresh rate. A 1080p60 laptop and a -1440p120 desktop each get their own. (With Moonlight, set the mode in Moonlight's settings; with the -Apple app, it uses the device's display.) +1440p120 desktop each get their own. (With Moonlight, set the mode in Moonlight's settings; the +native clients let you pick a mode or default to the device's display.) ## Bitrate The client requests a bitrate; the host encodes to it. To find a good value for your link: -- **Apple app:** use the built-in **speed test** (a host card's menu → *Test Network Speed*). It - measures your link and suggests a bitrate, then applies it. +- **Native clients (Apple, Linux, and more):** use the built-in **speed test** (from a host's menu). + It measures your link, suggests a bitrate, and applies it. - **Moonlight:** set the bitrate in Moonlight's settings. Start moderate and raise it. ## Multiple devices at once diff --git a/docs-site/content/docs/host-cli.md b/docs-site/content/docs/host-cli.md index 83f391e..c5c3f65 100644 --- a/docs-site/content/docs/host-cli.md +++ b/docs-site/content/docs/host-cli.md @@ -17,7 +17,7 @@ punktfunk-host serve --native | Flag | Meaning | |---|---| -| `--native` | Also run the native `punktfunk/1` server (recommended; enables the Apple app and discovery). | +| `--native` | Also run the native `punktfunk/1` server (recommended; enables the native clients and discovery). | | `--native-port ` | Native QUIC port (default `9777`). | | `--open` | Don't require pairing — serve any device on the network. Off by default; only for trusted single-user setups. | | `--mgmt-bind ` | Management API address (default loopback `127.0.0.1:47990`). | diff --git a/docs-site/content/docs/how-it-works.md b/docs-site/content/docs/how-it-works.md index 304c0cf..a155e9f 100644 --- a/docs-site/content/docs/how-it-works.md +++ b/docs-site/content/docs/how-it-works.md @@ -39,7 +39,8 @@ punktfunk speaks two protocols over the same host: no special software. This is the most compatible way in. - **punktfunk/1 (native)** — a purpose-built protocol with a QUIC control channel and a UDP data channel hardened with forward error correction and encryption. It's lower-latency and more resilient - on imperfect networks, and it's what the [Apple app](/docs/clients) uses. + on imperfect networks, and it's what the [native clients](/docs/clients) (Apple, Linux, Windows, + Android) use. Both run from a single host process, so you don't choose up front — Moonlight clients use GameStream, the native clients use punktfunk/1. @@ -53,7 +54,8 @@ cryptographic identity — no PIN, no account, no cloud. See [Pairing & Trust](/ ## Finding hosts Hosts advertise themselves on your local network, so clients can **discover** them automatically -instead of needing an IP address. The Apple app and Moonlight both list hosts they find on the LAN. +instead of needing an IP address. The native clients and Moonlight both list hosts they find on the +LAN. ## Multiple devices at once diff --git a/docs-site/content/docs/implementation-plan.md b/docs-site/content/docs/implementation-plan.md index 64526a8..b61ffee 100644 --- a/docs-site/content/docs/implementation-plan.md +++ b/docs-site/content/docs/implementation-plan.md @@ -6,7 +6,7 @@ description: "The full design: protocol core, milestones, and architecture." *A ground-up low-latency desktop streaming stack, built Linux-first, with a shared Rust protocol core and native clients per platform.* -> `punktfunk` is a placeholder codename — rename freely. It fits the lowercase house style (`unom`, `played`, `remplir`) and reads as "glass-to-glass light," which is the whole point. +> The name `punktfunk` fits the lowercase house style (`unom`, `played`, `remplir`) and reads as "glass-to-glass light," which is the whole point. --- @@ -32,7 +32,7 @@ Two concrete gaps justify a new project rather than another fork: **Explicit non-goals (at least at first):** - Windows *host* support (Sunshine/Apollo already do this well; no gap to fill). -- Internet/NAT-traversal relay infrastructure (LAN/VPN first; you already run Headscale/NetBird — lean on that). +- Internet/NAT-traversal relay infrastructure (LAN/VPN first; lean on an existing mesh VPN such as Headscale/NetBird/Tailscale). - Reinventing encoders/decoders (bind to FFmpeg + vendor SDKs; never rewrite codecs). - A bespoke compositor (drive existing ones; only consider a dedicated headless compositor as a *deployment mode*, see §6). @@ -169,7 +169,7 @@ This is the differentiator and the most fragmented part. Two deployment models **Per-compositor (Model A) runtime virtual-output creation:** -- **KWin / Plasma 6 (recommended MVP target — matches your CachyOS/KDE daily driver and where the gap is loudest):** KWin can create virtual outputs; KRdp already does this internally for remote sessions. Drive it via the KWin DBus interface; capture via `xdg-desktop-portal-kde` ScreenCast (PipeWire); inject input via the RemoteDesktop portal or `reis`. +- **KWin / Plasma 6 (recommended MVP target — a common KDE daily-driver setup, and where the gap is loudest):** KWin can create virtual outputs; KRdp already does this internally for remote sessions. Drive it via the KWin DBus interface; capture via `xdg-desktop-portal-kde` ScreenCast (PipeWire); inject input via the RemoteDesktop portal or `reis`. - **wlroots (Sway/Hyprland — fastest to *prototype* the pipeline):** enable the headless backend (`WLR_BACKENDS=…,headless`), then `swaymsg create_output` / `hyprctl output create headless`. Capture via `wlr-screencopy` or the portal. Simplest API; good for validating capture→encode→send before fighting KWin/Mutter. - **Mutter / GNOME:** virtual monitors via the headless backend; runtime creation via Mutter DBus (`org.gnome.Mutter.*` — partly experimental). Capture via `xdg-desktop-portal-gnome` ScreenCast. @@ -227,7 +227,7 @@ Sizing is rough and relative (Spike / S / M / L) for a focused solo dev; treat a **M4 — P2 transport: break the wall (L).** Add `punktfunk/1` negotiation; swap to `reed-solomon-simd` GF(2¹⁶) with multi-block per-frame framing; optional QUIC control/audio. Write a minimal **Rust** reference client (decode via VAAPI, present via wgpu/Vulkan) to exercise it. *Acceptance:* a stable stream above 1.4 Gbps at 5120×1440@240 with loss recovery working; latency unchanged vs. M2. -**M5 — Apple client (L).** Swift + VideoToolbox + Metal + SwiftUI, linking `punktfunk-core` via the C header. *Acceptance:* the Mac Studio plays a stream at native resolution/refresh. +**M5 — Apple client (L).** Swift + VideoToolbox + Metal + SwiftUI, linking `punktfunk-core` via the C header. *Acceptance:* a Mac plays a stream at native resolution/refresh. **M6 — Feature surface (M, ongoing).** Mic passthrough as a proper encrypted, per-client reverse audio stream (the thing the upstream PR got wrong); HDR signalling; per-client identity/permissions; pause/resume. *Acceptance:* feature parity with Apollo on the items you care about, plus mic done right. @@ -243,7 +243,7 @@ Sizing is rough and relative (Spike / S / M / L) for a focused solo dev; treat a | Encoder/decoder can't sustain 1.77 Gpx/s @ 240 | Med | High | Measure in M0/M4 on real silicon; this is a hardware ceiling no rewrite fixes — discover it before P2, not after | | Frame pacing eats more time than expected | High | Med | M3 measurement harness first; treat pacing as a first-class subsystem, not a polish step | | Scope creep into a full Moonlight replacement | High | High | P1 (stock-client compat) is the firewall: it forces you to ship value before writing a client | -| Solo bandwidth vs. your other projects (ENRW thesis, played) | High | Med | M2 is a complete, useful artifact on its own; the plan is safe to pause after any milestone | +| Solo bandwidth vs. other projects | High | Med | M2 is a complete, useful artifact on its own; the plan is safe to pause after any milestone | --- @@ -254,7 +254,7 @@ Sizing is rough and relative (Spike / S / M / L) for a focused solo dev; treat a - **Loss resilience:** `tc netem` to inject loss/jitter/reorder; verify FEC recovery and graceful degradation. - **Pacing:** log present timestamps vs. client vsync; alert on stalls and duplicate/dropped frames. - **Soak:** multi-hour streams; watch for buffer growth, fd leaks, encoder session exhaustion. -- **Hardware matrix:** your NVIDIA box (NVENC), an AMD/Intel box (VAAPI), Mac Studio (VideoToolbox decode). Catch driver quirks early. +- **Hardware matrix:** an NVIDIA box (NVENC), an AMD/Intel box (VAAPI), a Mac (VideoToolbox decode). Catch driver quirks early. --- @@ -290,10 +290,10 @@ punktfunk/ ## 12. Immediate next actions (first week) -1. **Stand up the workspace** with `punktfunk-core` (empty ABI + `cbindgen`) and `punktfunk-host` skeletons; CI on your Gitea (you already have BuildKit pipelines). -2. **M0 spike on wlroots:** headless output → PipeWire capture → NVENC/VAAPI encode → playable file. This validates the riskiest *pipeline* assumptions in days, on your real GPU. -3. **Read KRdp's source** for how KDE creates virtual outputs and casts them — it's the closest existing reference for the KWin path you'll need in M2. -4. **Decide P1 protocol depth:** confirm exactly which `serverinfo`/RTSP/pairing messages a current Moonlight client requires for a successful connect, so M2's compat surface is scoped precisely (this is also the question to take back to the dev who mentioned the 1G limit). +1. **Stand up the workspace** with `punktfunk-core` (empty ABI + `cbindgen`) and `punktfunk-host` skeletons; wire up CI (Gitea Actions, BuildKit-based pipelines). +2. **M0 spike on wlroots:** headless output → PipeWire capture → NVENC/VAAPI encode → playable file. This validates the riskiest *pipeline* assumptions in days, on real GPU hardware. +3. **Read KRdp's source** for how KDE creates virtual outputs and casts them — it's the closest existing reference for the KWin path needed in M2. +4. **Decide P1 protocol depth:** confirm exactly which `serverinfo`/RTSP/pairing messages a current Moonlight client requires for a successful connect, so M2's compat surface is scoped precisely. --- diff --git a/docs-site/content/docs/index.mdx b/docs-site/content/docs/index.mdx index 51f66a3..377f4e0 100644 --- a/docs-site/content/docs/index.mdx +++ b/docs-site/content/docs/index.mdx @@ -17,8 +17,8 @@ It's built for the things that make streaming feel native: - **Low latency, GPU end to end.** Frames go straight from the compositor to the GPU encoder (NVENC) with zero CPU copies, and over a transport tuned for responsiveness rather than throughput. - **Works with the apps you already have.** punktfunk speaks the GameStream protocol, so any - **Moonlight** client connects out of the box — and a faster **native protocol** with a dedicated - app for Apple devices. + **Moonlight** client connects out of the box — and a faster **native protocol** with dedicated apps + for **macOS, iOS, tvOS, Linux, Windows, and Android**. - **Secure by default.** Hosts require a one-time PIN pairing; after that, devices reconnect on a pinned identity. No accounts, no cloud. @@ -28,15 +28,16 @@ It's built for the things that make streaming feel native: - + ## What you need - A **Linux host** with an **NVIDIA GPU** (for the NVENC hardware encoder) running one of the [supported setups](/docs/requirements): **Ubuntu** (GNOME or KDE), **Fedora** (KDE), or **Bazzite**. -- A **client device** to stream to — a Mac/iPhone/iPad/Apple TV (native app), or anything that runs - **Moonlight**. + A native [**Windows host**](/docs/windows-host) (NVIDIA-only) is also available. +- A **client device** to stream to — there are native apps for **macOS, iOS/iPadOS, tvOS, Linux, + Windows, and Android**, plus any device that runs **Moonlight**. - Both on the **same network** (LAN or VPN). punktfunk is designed for a trusted local network. Ready? Head to the [Quick Start](/docs/quickstart). diff --git a/docs-site/content/docs/install-client.md b/docs-site/content/docs/install-client.md index fb328a7..8b3f5ed 100644 --- a/docs-site/content/docs/install-client.md +++ b/docs-site/content/docs/install-client.md @@ -85,8 +85,9 @@ certificate, so you import that certificate once before Windows will install the 3. Launch **Punktfunk** from the Start menu and pick your host. -> The Windows client is young (software decode; hardware D3D11VA/HDR in progress). If it -> misbehaves, **[Moonlight](/docs/moonlight)** is a solid alternative for Windows. +> The Windows client's hardware-decode (D3D11VA) and HDR paths are complete but still pending +> validation on real GPU hardware. If anything misbehaves, **[Moonlight](/docs/moonlight)** is a +> solid alternative for Windows. ## macOS diff --git a/docs-site/content/docs/moonlight.md b/docs-site/content/docs/moonlight.md index 433c320..81b40ee 100644 --- a/docs-site/content/docs/moonlight.md +++ b/docs-site/content/docs/moonlight.md @@ -4,8 +4,12 @@ description: Stream from a punktfunk host using any Moonlight client. --- punktfunk speaks the **GameStream** protocol, so [Moonlight](https://moonlight-stream.org/) connects -to it like it would to any GameStream host — no punktfunk-specific app needed. This is the easiest way -to stream to Windows, Android, the Steam Deck, a browser, or a TV. +to it like it would to any GameStream host — no punktfunk-specific app needed. It's a great option for +a browser, a smart TV, or any device without a native client. + +> Many platforms also have a **native punktfunk client** with lower latency and built-in +> discovery/pairing — including **Windows** and **Android** (phone and Android TV). See +> [Clients](/docs/clients) before reaching for Moonlight. ## 1. Make sure the host is running @@ -34,7 +38,7 @@ it. Mouse, keyboard, and controllers flow back to the host. - **Set your resolution and frame rate in Moonlight's settings** before connecting — the host matches whatever Moonlight asks for, creating the virtual display at that exact mode. - **Codec:** HEVC (H.265) is a good default; AV1 is available if your client supports it. -- **Bitrate:** start moderate and raise it. For very high bitrates, the native [Apple - app](/docs/clients) has a built-in speed test; with Moonlight, set the bitrate manually. +- **Bitrate:** start moderate and raise it. For very high bitrates, the [native + clients](/docs/clients) have a built-in speed test; with Moonlight, set the bitrate manually. - Moonlight uses the GameStream protocol, not punktfunk's native FEC/encryption extensions. On a - solid LAN this is fine; on a lossy link the [Apple app](/docs/clients) holds up better. + solid LAN this is fine; on a lossy link a [native client](/docs/clients) holds up better. diff --git a/docs-site/content/docs/pairing.md b/docs-site/content/docs/pairing.md index 97d975d..850f81a 100644 --- a/docs-site/content/docs/pairing.md +++ b/docs-site/content/docs/pairing.md @@ -50,7 +50,8 @@ command line instead; the production `serve --native` host arms pairing from the Then, on the client: -- **Apple app:** select the host (or use *Pair with PIN…* from its menu) and enter the PIN. +- **Native clients (Apple, Linux, Windows, Android):** select the host (or use *Pair with PIN…* from + its menu) and enter the PIN the host displays. - **Moonlight:** choose **Pair**; Moonlight shows the PIN to confirm on the host side. ## Requiring pairing (the default) diff --git a/docs-site/content/docs/quickstart.md b/docs-site/content/docs/quickstart.md index dcc5c77..62d9792 100644 --- a/docs-site/content/docs/quickstart.md +++ b/docs-site/content/docs/quickstart.md @@ -31,10 +31,11 @@ network, so clients can find it by name. Leave it running. (To start it automati ## 3. Connect and pair a client -On the device you want to stream to: +On the device you want to stream to, use a [native punktfunk client](/docs/clients) for the lowest +latency, or any Moonlight client: -- **Apple (Mac, iPhone, iPad, Apple TV):** open the punktfunk app — your host appears under *On this - network*. Tap it, and when prompted, **pair**. +- **Native client (Apple, Linux, Windows, Android):** open the punktfunk app — your host appears in + the list of hosts found on your network. Select it, and when prompted, **pair**. - **Anything with Moonlight:** add the host (it should be discovered automatically), then pair. To pair, the host needs to show a PIN. Arm pairing from the host's web console — the host displays a diff --git a/docs-site/content/docs/requirements.md b/docs-site/content/docs/requirements.md index 4fd7b54..42347a7 100644 --- a/docs-site/content/docs/requirements.md +++ b/docs-site/content/docs/requirements.md @@ -5,8 +5,9 @@ description: What you need to run a punktfunk host — GPU, driver, desktop, and ## Supported setups -A punktfunk host runs on a Linux machine with an NVIDIA GPU. These are the desktop environments it -supports today, each with its own guide: +A punktfunk host runs primarily on a Linux machine with an NVIDIA GPU (a native +[Windows host](/docs/windows-host) is also available — see below). These are the Linux desktop +environments it supports today, each with its own guide: | Setup | Desktop / compositor | Guide | |---|---|---| @@ -18,6 +19,10 @@ supports today, each with its own guide: Other wlroots compositors (Sway/Hyprland) also work but aren't a primary target. If your desktop isn't listed, the host still needs one of these compositor backends to create a virtual display. +> **Windows host:** punktfunk also runs as a native host on **Windows 10/11 (x64) with an NVIDIA GPU** +> — a signed installer that registers a service and bundles a virtual-display driver. It's NVIDIA-only +> and newer than the Linux host; see [Windows Host](/docs/windows-host). + ## GPU and driver - **An NVIDIA GPU** with NVENC — effectively any GeForce RTX or workstation card. NVENC is what @@ -54,5 +59,6 @@ Minimum compositor versions (newer is fine): ## A client -You also need something to stream *to* — see [Connect a Client](/docs/clients). The Apple app and any -Moonlight client both work; both can discover the host on your network automatically. +You also need something to stream *to* — see [Connect a Client](/docs/clients). There are native +punktfunk clients for **Apple (macOS, iOS, iPadOS, tvOS), Linux, Windows, and Android**, and any +Moonlight client works too. All of them can discover the host on your network automatically. diff --git a/docs-site/content/docs/roadmap.md b/docs-site/content/docs/roadmap.md index 1fa064d..f9da7b4 100644 --- a/docs-site/content/docs/roadmap.md +++ b/docs-site/content/docs/roadmap.md @@ -16,14 +16,19 @@ process, with native pairing driven from the **web console** (arm → show PIN), Advanced DualSense (audio-driven voice-coil) haptics **scoped NO-GO** (`docs/dualsense-haptics.md`). **Bazzite dynamic resolution (`c894c6f`):** the host now *manages* a headless `gamescope-session-plus` Steam session at the **client's exact resolution + refresh** — games see it (via injected -`--nested-refresh` + generated CVT modes, not the box's TV EDID), relaunched per-connection on a mode +`--nested-refresh` + generated CVT modes, not the host's TV EDID), relaunched per-connection on a mode change, reused (no Steam restart) on the same mode. Plus macOS/iPad input fixes (NSEvent motion + iPad pointer-lock) and a 4K/5K one-frame-freeze fix (grow the UDP socket buffers). -**Next:** **§8 pairing & trust hardening** (mandatory PIN by default + delegated approval), the native -client presenter + iOS (§6), and a Windows host (§7 — now **de-risked via SudoVDA**, no custom -signed driver needed). **§10 HDR/10-bit is parked — blocked upstream at the compositor** (no -gamescope/KWin PipeWire 10-bit producer yet). +The four native clients — **macOS/iOS/iPadOS/tvOS**, **Linux**, **Windows**, and **Android** — are +all real, working clients; see [Status & Progress](/docs/status) for their current state. + +A native **Windows host** (§7) is also implemented and shipping (NVIDIA-only, x64-only) — see +[Windows Host](/docs/windows-host). + +**Next:** **§8 pairing & trust hardening** (mandatory PIN by default + delegated approval) and the +native client presenter + iOS (§6). **§10 HDR/10-bit is parked — blocked upstream at the compositor** +(no gamescope/KWin PipeWire 10-bit producer yet). ## 1. Reliable headless KDE/compositor spawning ✅ *(done — Phase 1 + 2)* @@ -127,22 +132,26 @@ select = a `pw_stream` with `Direction::Output` + `media.class=Audio/Source`. PunktfunkKit is already platform-shared; iOS needs the `UIViewRepresentable` presenter twin + touch capture (#5) + UI. tvOS later. -## 7. Windows as a host *(scoped — `docs/windows-host.md`; de-risked via SudoVDA)* +## 7. Windows as a host ✅ *(implemented & shipping — NVIDIA-only, x64-only; see [Windows Host](/docs/windows-host))* -Architecturally an "add a backend" job, not a parallel port: `punktfunk-core` (protocol/FEC/ -crypto/C-ABI) + QUIC + GameStream + mgmt + the `m3`/pipeline orchestration are all platform-agnostic -and already `cfg`-isolated (~95% reuse). New `#[cfg(windows)]` backends behind the existing traits: -capture (DXGI Desktop Duplication / Windows.Graphics.Capture), encode (Media Foundation / NVENC-SDK -with a D3D11 context), input (SendInput + ViGEm), audio (WASAPI loopback + a virtual mic). +Done as an "add a backend" job, not a parallel port: `punktfunk-core` (protocol/FEC/crypto/C-ABI) + +QUIC + GameStream + mgmt + the pipeline orchestration are all platform-agnostic and already +`cfg`-isolated (~95% reuse), so the Windows host is a set of `#[cfg(windows)]` backends behind the +existing traits: -**The old blocker is gone.** Rather than author + sign our own kernel IDD for the per-client virtual -display, use **SudoVDA** (the Sunshine Virtual Display Adapter) — a pre-built, signed Indirect -Display Driver that creates virtual displays at arbitrary WxH@Hz on demand. The `VirtualDisplay` -backend becomes *"install + drive SudoVDA's control API"* (M effort), not *"write + WHQL-sign a -kernel driver"* (XL). That removes the only hard blocker — the Windows host is now a medium, -mostly-mechanical port. Recommended start: **Phase 0** — capture an existing monitor to prove the -stack end to end; **Phase 1** wires SudoVDA for the native-resolution output. Deferred only because -it's unbuildable on the Linux dev box; the trait boundaries are already in the right places. +- **Capture** — DXGI Desktop Duplication → D3D11 texture. +- **Virtual display** — **SudoVDA** (the Sunshine Virtual Display Adapter), a pre-built, signed + Indirect Display Driver that creates a `WxH@Hz` monitor on demand — so there's no kernel driver to + author or WHQL-sign. Bundled and staged by the installer; falls back to capturing an existing + monitor if absent. +- **Encode** — NVENC with a D3D11 device (`--features nvenc`), so the host is **NVIDIA-only**. +- **Input** — SendInput (mouse/keyboard) + ViGEm (virtual gamepads with rumble). +- **Audio** — WASAPI loopback capture + a WASAPI virtual mic. + +It ships as a **signed Inno Setup installer** that registers a `LocalSystem` SCM service launching +into the interactive session for secure-desktop (UAC / lock-screen) capture, published by +`windows-host.yml`. Still open: AMD/Intel encode (none today), broader real-world testing, and +bundling ViGEmBus. ## 8. Pairing & trust hardening *(§8a + §8b-1 done; §8b-2 next)* @@ -356,8 +365,8 @@ GameStream already auto-discovered via mDNS (`_nvstream._tcp`). Now both the uni - **Client**: `punktfunk-probe --discover [SECS]` browses and prints each host (name, addr:port, pairing, fingerprint), then exits. Apple clients browse the same service natively via NWBrowser (Bonjour) — no Rust-connector dependency; this section's service type + TXT keys are the contract. -- **Validated**: cross-LAN — dev box discovered the GNOME-box appliance - (`home-worker-3 192.168.1.248:9777 pair=required fp=1dcf3a…`) and a standalone synthetic host +- **Validated**: cross-LAN — a client discovered a GNOME host appliance + (`myhost.local 9777 pair=required fp=1dcf3a…`) and a standalone synthetic host (`pair=optional`); fingerprint + pairing state correct in both. - **Next** (not done): wire NWBrowser discovery into the Apple client UI (host picker); the host-side contract above is all it needs. diff --git a/docs-site/content/docs/running-as-a-service.md b/docs-site/content/docs/running-as-a-service.md index 0578cd4..b9b19b4 100644 --- a/docs-site/content/docs/running-as-a-service.md +++ b/docs-site/content/docs/running-as-a-service.md @@ -79,6 +79,10 @@ session unit — see [Bazzite](/docs/bazzite). ## Windows +> punktfunk is Linux-first, but a native **Windows host** also ships — a signed installer with an SCM +> service and a bundled virtual-display driver. It's **NVIDIA-only** (NVENC) and newer than the Linux +> host. (Not to be confused with the Windows *client*, which streams *to* a Windows PC.) + On Windows the host runs as a `LocalSystem` service that launches into the interactive session, so it captures the secure desktop (UAC / lock screen) and survives reboots with nobody logged in — the same model Sunshine/Apollo use. @@ -98,7 +102,7 @@ way you need an NVIDIA GPU + driver (the host is NVENC-only on Windows). After a reboot, from another machine on the network: ```sh -punktfunk-probe --discover # or just look for the host in the Apple app / Moonlight +punktfunk-probe --discover # or just look for the host in a native client / Moonlight ``` If the host is listed, it's up. If not, check `journalctl --user -u punktfunk-host` on the host. diff --git a/docs-site/content/docs/status.md b/docs-site/content/docs/status.md index 9f2cb05..3fc10fc 100644 --- a/docs-site/content/docs/status.md +++ b/docs-site/content/docs/status.md @@ -1,11 +1,12 @@ --- title: "Status & Progress" -description: "Where the work stands, what's live on each box, and a running progress log." +description: "Where the work stands across the core, the host, and the native clients." --- -The living progress tracker. Milestone-level status lives in [`CLAUDE.md`](https://github.com) -and the design in the [Implementation Plan](/docs/implementation-plan); this page is the -**current state + a dated log** of what landed, kept up to date as work happens. Newest first. +A high-level view of where punktfunk stands. The full design lives in the +[Implementation Plan](/docs/implementation-plan), the ordered plan of work in the +[Roadmap](/docs/roadmap), and milestone-level detail in +[`CLAUDE.md`](https://git.unom.io/unom/punktfunk/src/branch/main/CLAUDE.md). ## Milestones at a glance @@ -13,92 +14,105 @@ and the design in the [Implementation Plan](/docs/implementation-plan); this pag |---|---| | **Core** — `punktfunk-core` + C ABI (protocol · FEC · crypto) | ✅ complete & hardened | | **GameStream host** (Moonlight-compatible) | ✅ working end-to-end; HDR/surround-audio polish open | -| **Native protocol** — `punktfunk/1` (QUIC control + UDP data) | ✅ full session planes, validated live | -| **Native clients** — decode + present (Apple first) | 🟡 macOS stage 1 live; stage-2 presenter built + decode-tested (opt-in, present needs live validation). **Linux GTK client stage 1 live** (2026-06-12) | +| **Native protocol** — `punktfunk/1` (QUIC control + UDP data, GF(2¹⁶) Leopard FEC + AES-GCM) | ✅ full session planes, validated live | +| **Windows host** (NVIDIA, x64) | 🟡 implemented & shipping as a signed installer; NVIDIA-only, newer than the Linux host | +| **macOS / iOS / iPadOS / tvOS client** | ✅ full client; on-glass stage-2 presenter behind an opt-in flag, becoming the default | +| **Linux client** (`punktfunk-client`, GTK4/libadwaita) | ✅ full client; VAAPI zero-copy decode + software fallback | +| **Windows client** (`punktfunk-client`, WinUI 3) | ✅ stage 1 complete; ships as signed MSIX; on-glass hardware validation pending | +| **Android client** (phone + Android TV) | ✅ full client; hardware HEVC decode + HDR10 | +| **Web console** (over the management API) | ✅ status · devices · pairing | -## Live on the boxes +## What works today -| Box | Role | Compositor | Notes | -|---|---|---|---| -| **home-worker-2** (dev) | KDE/KWin appliance | kwin (headless Plasma) | QEMU VM, passthrough RTX 5070 Ti; `serve --native` user unit | -| **home-worker-3** (GNOME) | GNOME/Mutter appliance | mutter (RecordVirtual) | RTX 4090; autologin GNOME Wayland; `serve --native` user unit. See [Ubuntu — GNOME](/docs/ubuntu-gnome) | -| **home-bazzite-1** | SteamOS-like host | gamescope | host-managed Steam session at client mode. See [Bazzite Setup](/docs/bazzite) | +punktfunk is a low-latency desktop and game streaming **host** — Linux-first (Linux + NVIDIA, NVENC), +with a newer **NVIDIA-only Windows host** too — and native **clients** on macOS, iOS/iPadOS/tvOS, +Linux, Windows, and Android. -All three appliances advertise over mDNS (`_punktfunk._udp`) and require PIN pairing by default. +- **Two protocols.** The host speaks the **GameStream** protocol, so any **Moonlight** + client works out of the box, plus its own lower-latency **`punktfunk/1`** protocol + (QUIC control plane + UDP data plane with GF(2¹⁶) Leopard FEC and AES-GCM). +- **Native resolution, no scaling.** Every session gets a virtual display at the client's + exact resolution and refresh rate, via per-compositor backends for **KWin**, + **gamescope**, **Mutter**, and **Sway/wlroots**. +- **Zero-copy GPU pipeline.** Captured frames stay on the GPU (dmabuf → CUDA → NVENC) with + automatic split-encode at very high resolutions. Stable 240 fps at 5120×1440 has been + measured. +- **Secure by default.** A **SPAKE2 PIN pairing** ceremony establishes trust (the host + shows a 4-digit PIN; an attacker gets a single online guess, no offline dictionary + attack). Trust-on-first-use (TOFU) remains an explicit opt-in for fully trusted LANs. +- **LAN auto-discovery.** Hosts advertise over mDNS (`_punktfunk._udp`); clients browse and + list them automatically. +- **Full input.** Mouse, keyboard, and gamepads (including DualSense touchpad, motion, + rumble, lightbar, player LEDs, and adaptive triggers) in both directions. +- **Audio both ways.** Opus desktop audio host → client, plus an opt-in, paired-only client + microphone uplink. +- **Management surface.** A REST management API with a checked-in OpenAPI document, plus a + web console for status, paired devices, and pairing. -## Progress log +### Native clients -### 2026-06-12 -- **Native Linux client — stage 1, first light** (`clients/linux`, binary - `punktfunk-client`). GTK4/libadwaita app on the **Option A** architecture picked after a - six-angle research pass (toolkits / hw decode / Wayland presentation / input capture / - prior art / codebase): links `punktfunk-core` directly as a crate (no C ABI; - `NativeClient` is `Sync` now), mDNS host list, TOFU + SPAKE2 PIN pairing dialogs - (identity shared with `client-rs`), FFmpeg software HEVC decode (`LOW_DELAY` + slice - threads) into a `GtkGraphicsOffload`-wrapped picture, PipeWire playback with the host - mic-player's jitter ring inverted, SDL3 gamepad capture + rumble/lightbar feedback, - layout-independent keyboard (exact inverse of the host's VK table), absolute mouse + - WHEEL_DELTA scroll, compositor-shortcut inhibition, fullscreen, stats overlay. - **Validated live** against this box's `serve --native`: 1080p60 at a locked 60 fps, - capture→decoded **p50 ≈ 6.4 ms** (software decode, debug build). Next: VAAPI dmabuf → - `GdkDmabufTexture` (Tier-1 zero-copy on Intel/AMD clients), DualSense - touchpad/motion/trigger replay over SDL3, then the stage-2 raw-Wayland presenter - (wp_presentation feedback, tearing-control, Vulkan Video for NVIDIA clients). -- **Delegated pairing approval (§8b-1)** — an unpaired device that tries to connect to a - pairing-required host now shows up as a **pending request** in the web console's Pairing page; - one click approves it (optionally relabeling) and pairs its certificate fingerprint — no PIN - fetched out of band. New mgmt endpoints (`/native/pending` + approve/deny), an in-memory pending - queue in `NativePairing` (fp-deduped, capped, 10-min expiry), and an optional **device name** in - the `Hello` (back-compat trailing field; `client-rs --name` sends it). End-to-end tested. - §8b-2 (approve from a paired device's own app) is the client-side follow-up. -- **CI + deployment landed** (see the [CI & Docker](/docs/ci) guide). Gitea Actions, three - workflows: Rust workspace checks inside the new `punktfunk-rust-ci` builder image (Ubuntu 26.04, - full link-dep stack incl. a libcuda stub — 141/141 tests green in-container), web + docs-site - build/typecheck, `docker.yml` building+pushing `punktfunk-web`/`punktfunk-docs`/`punktfunk-rust-ci` - to the registry, and `apple.yml` (xcframework → `swift build`/`swift test`) on a new **host-mode - macOS runner** (`home-mac-mini-1`, provisioned by `scripts/ci/setup-macos-runner.sh`; macOS - Local-Network privacy forces it to run as a root LaunchDaemon). Host and native clients stay - un-dockerized by design. **This site now deploys automatically**: `deploy-docs` ships it to - unom-1:3220, Caddy serves — live and verified. -- **Concurrent sessions** — the host no longer serves one client at a time. The accept loop spawns - each session (`JoinSet`), bounded by `--max-concurrent` (default 4, a NVENC bound; overflow waits - in the accept queue). Each session keeps its own virtual output + encoder; they share the - host-lifetime input/audio/mic services — i.e. **multiple devices viewing/controlling the same - desktop** on kwin/mutter/wlroots. Validated live on the GNOME box: two clients connected at once - → **two independent Mutter virtual outputs (1280×720 + 1920×1080) streaming simultaneously** - (39 MB + 48 MB). gamescope's *independent-desktops* (multi-user) isolation — per-session - input/audio — is a follow-up. -- **Apple client latency HUD** — `PunktfunkConnection.clockOffsetNs` (from the C-ABI getter) + - `LatencyMeter` surface a skew-corrected **capture→client-receipt** p50/p95 in the macOS HUD: the - first cross-machine latency the real Apple client reports. (Stage-1 `AVSampleBufferDisplayLayer` - has no present callback, so decode→present is excluded — that needs the stage-2 presenter.) - Needs an `xcframework` rebuild + `swift test` on the Mac to validate. -- **Skew handshake in the connector + C ABI** — `quic::clock_sync` is now a shared core helper used - by both the reference client and `NativeClient`; the connector runs it at connect and exposes the - host clock offset over the C ABI (`punktfunk_connection_clock_offset_ns`). This is the substrate - the Apple client needs for the decode→present (glass-to-glass) term. -- **Wall-clock skew handshake** (`ClockProbe`/`ClockEcho`, 8 NTP rounds after `Start`) — makes the - client's capture→reassembled latency valid **cross-machine**. Validated GNOME box → dev box: - offset −1.57 ms removed, **p50 1.30 ms** skew-corrected. (`05bc9ab`) -- **Native LAN auto-discovery** — host advertises `_punktfunk._udp` (TXT: fingerprint, pairing, - proto); `punktfunk-probe --discover` lists hosts. Validated cross-LAN. (`4fff464`) -- **Third test box stood up** — home-worker-3 (Ubuntu 26.04, RTX 4090, GNOME 50): first GNOME/Mutter - zero-copy streaming on a real desktop; **1 Gbps probe clean** (625 MB/5 s, `send_dropped=0`). - Two physical-NVIDIA gotchas documented in [Ubuntu — GNOME](/docs/ubuntu-gnome). -- **Encode|send thread split** validated on real NIC (`send_dropped=0` at 720p60 / 1080p120). (`b295a5b`) +| Client | Highlights | +|---|---| +| **macOS / iOS / iPadOS / tvOS** | VideoToolbox HEVC decode, GameController capture, full DualSense feedback, mDNS discovery, PIN pairing + TOFU, network speed test, latency HUD. Stage-2 presenter (`VTDecompressionSession` → `CAMetalLayer`, ~11 ms p50 capture→present) is built and validated on glass behind an opt-in flag, becoming the default. Ships as one universal TestFlight build / App Store listing. | +| **Linux** (`punktfunk-client`) | GTK4/libadwaita. FFmpeg decode with VAAPI → DRM-PRIME dmabuf zero-copy (Intel/AMD; software fallback on NVIDIA), PipeWire audio + mic, SDL3 gamepads incl. DualSense, mDNS discovery, PIN pairing + TOFU, speed test. Ships as Flatpak, apt, rpm, and Arch packages. | +| **Windows** (`punktfunk-client`) | WinUI 3. D3D11VA zero-copy decode, HDR10, WASAPI audio + mic, SDL3 gamepads incl. DualSense, mDNS discovery, and the full PIN/TOFU trust surface are all implemented. Ships as a signed MSIX (x86_64 + ARM64). **Stage 1 complete; D3D11VA decode, HDR present, and the GUI are pending on-glass validation on real GPU hardware.** | +| **Android** (phone + Android TV) | Kotlin app with a Rust core over JNI. NDK `AMediaCodec` hardware HEVC decode + HDR10 (Main10/BT.2020 PQ), Opus/Oboe audio + mic, gamepad input with rumble/HID feedback, NsdManager mDNS discovery, PIN pairing + TOFU (Keystore identity), live stats HUD, and D-pad/controller focus navigation for TV. Ships to the Google Play Internal Testing track. | -### Earlier (see roadmap + git log) -- **1 Gbps data plane**: batched `sendmmsg`/`recvmmsg` + microburst-cap paced send thread. -- **Boot appliance**: headless KDE session + host systemd units (no login). -- **Speed test + settable bitrate**: negotiation + bandwidth probe (host side). -- **DualSense** UHID + haptics; gamepads live; mic uplink; AV1 + surround (unit/live-capture tested). +`punktfunk-probe` is a headless reference and measurement client (for testing and +benchmarking, not everyday use). + +## Validated live on + +The stack has been validated live on a range of hosts and clients: + +- **Hosts:** Ubuntu (GNOME / KDE), Fedora KDE, and Bazzite (gamescope) on machines with + RTX-class NVIDIA GPUs, across the KWin, gamescope, Mutter, and Sway/wlroots backends. +- **Clients:** native macOS, Linux, and Android clients against live hosts, plus stock + Moonlight clients over the GameStream path. +- **Cross-machine latency** is measured and skew-corrected (a wall-clock handshake removes + the inter-machine clock offset), so capture-to-present numbers are valid across the LAN. + +## Highlights + +Notable capabilities that have landed, newest first: + +- **Native Linux client (stage 1).** GTK4/libadwaita app that links `punktfunk-core` + directly: mDNS host list, TOFU + SPAKE2 PIN pairing, FFmpeg HEVC decode, PipeWire audio + with mic uplink, SDL3 gamepad capture with rumble/lightbar feedback, layout-independent + keyboard, absolute mouse, fullscreen, and a stats overlay. VAAPI → `GdkDmabufTexture` + zero-copy decode on Intel/AMD with a proven software fallback. +- **Delegated pairing approval.** An unpaired device that knocks on a pairing-required host + appears as a pending request in the web console's Pairing page; one click approves and + pins its certificate — no PIN fetched out of band. +- **Concurrent sessions.** The host serves multiple clients at once (bounded by an NVENC + limit), each with its own virtual output and encoder — e.g. stream the same desktop to a + laptop and a TV simultaneously. +- **Cross-machine latency HUD + wall-clock skew handshake.** A short NTP-style handshake + aligns client and host clocks, making capture-to-reassembled latency valid across + machines; the Apple client surfaces a skew-corrected capture-to-receipt p50/p95 in its + HUD. +- **Native LAN auto-discovery.** Hosts advertise `_punktfunk._udp` over mDNS (with TXT + records carrying the protocol, cert fingerprint, and pairing requirement); clients + discover and list them automatically. +- **1 Gbps data plane.** Batched `sendmmsg`/`recvmmsg`, a microburst-capped paced send + thread, and larger socket buffers, exploiting the GF(2¹⁶) Leopard FEC that breaks the + classic ~1 Gbps GameStream ceiling. +- **Boot appliance.** A headless compositor session plus host systemd units, so a host can + come up and stream with no interactive login. +- **Network speed test + settable bitrate.** Bitrate negotiation and an in-band bandwidth + probe inform the client's bitrate picker instead of guesswork. +- **Rich DualSense.** A full UHID DualSense backend on the host (gamepad, motion, touchpad, + lightbar, player LEDs, adaptive triggers) with feedback carried back to the clients. +- **AV1 + surround audio** are implemented and unit/live-capture tested. ## In flight / next See the [Roadmap](/docs/roadmap) for the ordered list. Near-term: -- **True glass-to-glass**: Apple client present-stamp (decode→present) + host render→capture term. -- **Apple stage-2 presenter** (`VTDecompressionSession` → `CAMetalLayer`) — built + decode-unit-tested + live-validated on glass behind the `punktfunk.presenter` flag (capture→present ~11 ms p50); make it the default after a few resolution/HDR checks. -- **Mandatory PIN pairing + delegated pairing approval** (an already-paired device approves a new one). -- **gamescope multi-user isolation** — per-session input/audio so concurrent sessions are independent - desktops (the shared-desktop multi-view case landed). -- **bazzite** kept up to date (currently offline; one rebuild behind). + +- **True glass-to-glass latency** — combine the client present-stamp (decode → present) + with the host render → capture term for a complete end-to-end number. +- **Make the Apple stage-2 presenter the default** after a few more resolution/HDR checks. +- **On-glass validation of the Windows client** (D3D11VA decode, HDR present, GUI) on real + GPU hardware. +- **gamescope multi-user isolation** — per-session input/audio so concurrent sessions can + be fully independent desktops (the shared-desktop multi-view case already works). diff --git a/docs-site/content/docs/troubleshooting.md b/docs-site/content/docs/troubleshooting.md index 7852c52..70425bf 100644 --- a/docs-site/content/docs/troubleshooting.md +++ b/docs-site/content/docs/troubleshooting.md @@ -65,8 +65,9 @@ Then log out and back in. On other distros this is `sudo usermod -aG input $USER ## Stutter, drops, or high latency -- Lower the **bitrate**. On a busy or Wi-Fi link, the requested bitrate may be too high — the Apple - app's [speed test](/docs/configuration#bitrate) picks a safe value; with Moonlight, set it manually. +- Lower the **bitrate**. On a busy or Wi-Fi link, the requested bitrate may be too high — the native + clients' [speed test](/docs/configuration#bitrate) picks a safe value; with Moonlight, set it + manually. - Prefer a **wired** connection or 5 GHz Wi-Fi between host and client. - Streaming to **many devices at once** shares the GPU encoder. The production host (`serve --native`) handles one native session at a time, with extra clients queued; heavy load is diff --git a/docs-site/content/docs/windows-host.md b/docs-site/content/docs/windows-host.md index 47e87a8..bb73261 100644 --- a/docs-site/content/docs/windows-host.md +++ b/docs-site/content/docs/windows-host.md @@ -1,77 +1,70 @@ --- title: "Windows Host" -description: "Feasibility and scoping for a Windows host backend." +description: "Run the punktfunk streaming host on a Windows PC with an NVIDIA GPU." --- -**Status: scoped, deferred — but de-risked.** A Windows host is architecturally an *"add a backend"* -job, not a parallel port. The one thing that used to make it **large** — the per-client *virtual* -output, which has no user-mode Windows API and seemingly needed a self-signed kernel Indirect -Display Driver (IDD) — is **solved by reusing [SudoVDA](https://github.com/VirtualDrivers), the -Sunshine Virtual Display Adapter**: a pre-built, signed IDD that creates virtual displays at -arbitrary `WxH@Hz` on demand. We install it and drive its control interface; **no driver to write or -WHQL-sign.** That turns the headline feature from XL into a medium backend. This doc records what's -left so the work can be picked up deliberately. +**Status: implemented and shipping — NVIDIA-only, x64-only.** punktfunk is Linux-first, but it also +runs as a native **Windows host**: a signed installer registers a `LocalSystem` service that streams +your Windows desktop or games to any punktfunk or Moonlight client, at the client's exact resolution +via a virtual display. It's newer and less battle-tested than the Linux host, and it is built +specifically around NVIDIA hardware. -(Grounded in a 4-agent read of the host crate, 2026-06-10; SudoVDA path added 2026-06-11.) +> This page is about the Windows **host** (streaming *from* a Windows PC). To stream *to* a Windows +> PC, see the [Windows client](/docs/clients#windows-desktop-client). -## What's already done for us +## Requirements -punktfunk is cleanly layered. **~95% of the codebase is platform-agnostic and reuses verbatim:** +- **Windows 10/11, x64.** ARM64 is not supported — both NVENC and the virtual-display driver are + x64-only. +- **An NVIDIA GPU + driver.** The host encodes with NVENC (`nvEncodeAPI64.dll`); there is no other + encoder backend on Windows. +- **(Optional) ViGEmBus** for virtual gamepads — a manual prerequisite for now + ([releases](https://github.com/nefarius/ViGEmBus/releases)). -| Reusable as-is | Why | -|---|---| -| `punktfunk-core` (protocol, FEC, crypto, session, transport, **C ABI**) | Zero platform deps — no `cfg(linux)` anywhere; the C ABI is already cross-platform | -| QUIC control plane (`quic.rs`, pairing, mode negotiation) | quinn + tokio are portable | -| GameStream P1.1 (mDNS, serverinfo, pairing, RTSP, ENet) — *except* `stream.rs`/`audio.rs` | pure wire logic | -| Management REST API (`mgmt.rs`) + OpenAPI | axum/tokio, portable | -| Pipeline + `punktfunk1.rs` orchestration | trait-generic — calls `capturer.next_frame()`, `encoder.submit/poll()`; **needs zero changes** | -| The **trait boundaries** themselves: `Capturer`, `Encoder`, `VirtualDisplay`, `InputInjector`, `AudioCapturer`, `VirtualMic` | platform-neutral signatures; Linux deps are already isolated under `[target.'cfg(target_os="linux")'.dependencies]` | +## Install -So a Windows host is **new `#[cfg(target_os = "windows")]` backend modules behind the existing -traits** — the per-frame path, protocol, and control plane don't move. No architectural refactor is -required; the boundaries are already in the right places. +Download the signed `punktfunk-host-setup-.exe` from the package registry and run it — it +installs the host into `C:\Program Files\punktfunk`, optionally installs the bundled **SudoVDA** +virtual-display driver, and registers + starts the service. Full steps (including the silent install +and the CLI `punktfunk-host service install` path) are in +[Running as a Service → Windows](/docs/running-as-a-service#windows); packaging internals live in +[`packaging/windows`](https://git.unom.io/unom/punktfunk/src/branch/main/packaging/windows/README.md). -## What a Windows host needs (new code) +## How it works -Each row is a Linux backend that needs a Windows sibling. Effort is the *implementation* effort; -all reuse the existing trait. +The host installs a **`LocalSystem` SCM service** that runs from Session 0 and launches a worker into +the interactive session (`CreateProcessAsUserW`). That lets it **capture the secure desktop** (UAC +prompts, the lock screen) and keep streaming across reboots with nobody logged in — the same model +Sunshine and Apollo use. Service registration, firewall rules, and the supervisor all live in +`punktfunk-host service install`; the installer just lays the exe down and calls it elevated. -| Subsystem | Linux today | Windows equivalent | Effort | Notes | -|---|---|---|---|---| -| **Capture** | xdg ScreenCast portal → PipeWire (dmabuf) | **DXGI Desktop Duplication** (or Windows.Graphics.Capture) → D3D11 texture | M | DXGI gives a GPU `B8G8R8A8` texture directly | -| **Virtual display** | KWin/Mutter/Sway/gamescope protocols | **SudoVDA** (pre-built signed IDD) — install + drive its control API to add/remove a `WxH@Hz` virtual monitor per session | **M** | ✅ **no longer the blocker**: SudoVDA is the same IDD Sunshine ships, so no driver to author or sign. The `VirtualDisplay` backend = enable the adapter, create a monitor at the client's mode, capture it (DXGI), tear it down on session end. Fallback if SudoVDA is absent: capture an existing monitor (loses native-resolution) | -| **Encode** | `ffmpeg-next` NVENC, CUDA hwframes | Media Foundation H.264/HEVC/AV1, **or** NVENC SDK direct with a D3D11 device context (`AVD3D11VADeviceContext`) | M–L | `encode.rs` AU/codec logic + NVENC option strings are portable; only the hwdevice + frame-pool glue swaps | -| **Zero-copy bridge** | dmabuf → EGL/Vulkan → CUDA | D3D11 texture → NVENC (shared texture / `cudaImportExternalMemory` + D3D12 fence) | M | **optional** — a portable CPU-copy path already exists, so v1 can skip this | -| **Input (ptr/kbd)** | libei (RemoteDesktop portal) / wlr protocols | **SendInput** (`keybd_event`/`mouse_event`) | S | the VK→evdev table just becomes VK→`VIRTUAL_KEY` (already Win32-native) | -| **Input (gamepads)** | uinput X-Box-360 pad + FF rumble | **ViGEm** (Virtual Gamepad Emulation) + HID reports | M | rumble back-channel maps to ViGEm notifications | -| **Audio capture** | PipeWire sink-monitor | **WASAPI loopback** (`IAudioCaptureClient`) | S–M | also produces interleaved f32 — same `AudioCapturer` contract | -| **Virtual mic** | PipeWire `Audio/Source` | virtual audio device (VB-Cable-style WDM driver) or WASAPI render-to-fake-device | M | needs a driver or a bundled 3rd-party cable | -| **`sendmmsg` batching** | `gamestream/stream.rs` | already has a `cfg(not(linux))` per-packet fallback | — | nothing to do | +### One core, Windows backends -**Rough total: ~2,000–4,000 LOC of new Rust** (no C++ driver — SudoVDA is reused as-is), spread over -capture/encode/vdisplay/input/audio. With the driver problem solved, the overall effort is now -**medium**; the input+audio layer alone is *small–medium*. +Most of punktfunk is platform-agnostic. `punktfunk-core` (protocol, FEC, crypto, session, transport, +the C ABI), the QUIC control plane, the GameStream wire logic, the management API, and the per-frame +pipeline orchestration are all shared with the Linux host. The Windows host is a set of +`#[cfg(windows)]` backends behind the same traits the Linux host uses: -## Recommended phasing (when picked up) +| Subsystem | Linux backend | Windows backend | +|---|---|---| +| **Capture** | xdg ScreenCast portal → PipeWire (dmabuf) | **DXGI Desktop Duplication** → D3D11 texture | +| **Virtual display** | KWin / Mutter / Sway / gamescope | **SudoVDA** signed IDD — create a `WxH@Hz` monitor per session, capture it, tear it down | +| **Encode** | `ffmpeg-next` NVENC (CUDA hwframes) | **NVENC** with a D3D11 device (`--features nvenc`) | +| **Input — mouse/keyboard** | libei / wlr protocols | **SendInput** (Win32 VK + absolute mouse) | +| **Input — gamepads** | uinput Xbox 360 pad + rumble | **ViGEm** virtual pad + rumble back-channel | +| **Audio capture** | PipeWire sink-monitor | **WASAPI loopback** | +| **Virtual mic** | PipeWire `Audio/Source` | WASAPI virtual mic | -1. **Phase 0 — "basic Windows host" (no virtual display).** Capture an *existing* monitor (DXGI - Desktop Duplication) → Media Foundation/NVENC encode → SendInput + WASAPI loopback. This proves - the whole stack on Windows with the smallest surface, reusing all of core/QUIC/GameStream/mgmt. - It loses the per-client native-resolution output but is a working Windows host quickly. -2. **Phase 1 — the virtual display via SudoVDA.** A `VirtualDisplay` backend that enables SudoVDA, - creates a monitor at the client's exact `WxH@Hz`, captures it (DXGI), and tears it down on session - end — restoring punktfunk's headline feature with **no driver authoring or signing**. (Ship/guide - the SudoVDA install as a host prerequisite, like the udev rule on Linux.) -3. **Phase 2 — input + audio parity.** ViGEm gamepads + rumble; WASAPI virtual mic; D3D11→NVENC - zero-copy. +The virtual display uses **[SudoVDA](https://github.com/VirtualDrivers)** (the Sunshine Virtual +Display Adapter) — a pre-built, signed Indirect Display Driver — so there is **no kernel driver to +author or WHQL-sign**. The installer bundles and stages it; if it's absent, the host falls back to +capturing an existing monitor (losing the per-client native-resolution output). -## Why it's deferred (not started now) +## Limitations -- The remaining work is **medium** and mechanical, but **none of it is buildable or testable on the - Linux dev box** — it would be unvalidated code until there's a Windows box in the loop. -- SudoVDA removed the hard blocker (the signed kernel driver); what's left is a backend port, picked - up whenever a Windows target is in scope. - -The architecture is ready whenever the work is scheduled; this doc + the clean trait boundaries are -the down payment. Start at **Phase 0** for the fastest path to a working Windows host. +- **NVIDIA-only.** NVENC is the only encoder backend — there is no AMD / Intel / software encode path + on Windows. +- **x64-only.** No ARM64 build (no ARM64 NVIDIA driver, and SudoVDA is x64-only). +- **Newer than the Linux host.** The Linux host is the most battle-tested path; the Windows host is + more recent, with the virtual-mic and gamepad backends the youngest pieces.