enricobuehler
e8a7a1e6af
The D3D11VA and Vulkan rungs both decoded into a surface they were predicting from, on 117 of 120 access units of our own host's low-delay H.264 (`1c54d099` for AV1, `834b2443` for H.264). `pf-vaadec` feeds `reference_frames` from the same `plan.dpb_refs` snapshot, releases its whole `removed` list inline exactly as the two broken conversions did, and neither fix commit touched it. It is still exempt — this is the evidence, and the thing that keeps it true. **Measured on the CPU, no GPU needed.** `walk_for_aliasing` drives the planner and `plan_to_va` over both streams and counts four shapes. On `lowdelay-640x480.h264` the aliasing PRECONDITION is fully present: 117 of 120 access units remove a picture their own `dpb_refs` still names, and on the same 117 the setup picture is handed the slot of a picture that access unit READS — the D3D11VA/Vulkan defect verbatim, in this conversion, today. On the vendored conformance vector both counts are 0, which is why that vector proved nothing on two other backends for two milestones. Aliased submissions: **0 on both**. **Why.** A slot is not a surface here. `plan_to_va` never invents one — every reference it can name is read out of the `surfaces` table it is handed — and the decode target is a separate parameter the caller takes from OUTSIDE that table. `setup_surface` reaches the submission at exactly one field per codec (H.264/H.265 `curr_pic.picture_id`, AV1 `current_frame` and `current_display_picture`); HEVC is doubly safe, because its per-slice `RefPicList` stores an INDEX into `reference_frames` rather than a surface. AV1's documented substitution fallback is the one place the target can be named as a reference, and only where the store resolved nothing at all to prefer. **The exemption was incidental; it is structural now.** It needs the reference table and the decode target to come from ONE snapshot of the bindings, and the rung had that only by writing `free_surface()` and `surface_table()` adjacently at three call sites. Split them and this rung acquires the defect exactly: the table must be the PRE-removal one (that is where the references are), while a free list consulted after the removals offers precisely the displaced picture's surface. `Session::acquire_target` now returns the index, the surface and the table together from `&self`, so a later edit cannot move one call and not the other. No behaviour change: same order, same values, same refusal message. Tests. `no_submission_names_its_decode_target_as_one_of_its_own_references` (both streams, 0) with `taking_the_decode_target_from_the_slot_table_aliases_on_the_low_delay_stream` as the counterfactual that reproduces the defect on 117 of 120 — so the walk demonstrably CAN see it when it is there. `the_low_delay_stream_reassigns_slots_whose_pictures_it_still_reads` pins 0/250 and 117/120 so neither can drift silently. `the_decode_target_can_never_be_a_surface_the_reference_table_names` sweeps every binding state a 4-surface/3-slot pool can hold, and `taking_the_free_surface_after_the_removals_would_hand_out_a_referenced_surface` is the ordering counterfactual. ⚠ One existing test lost a VACUOUS half. `the_setup_picture_routinely_inherits_a_just_freed_slot` asserted the decode target was never also a reference while handing every picture its own never-reused surface id — distinct integers cannot collide, so that assertion could not fail whatever the conversion did. Its real measurement (225 of 250 access units reuse a just-freed slot, which is why the target is a parameter) is kept; the collision half is gone, and the doc says where the question is actually answered and why a recycling pool is what it takes to answer it. Gates, run on `.25` (Radeon 780M, radeonsi, Mesa 26.0.3, VA-API 1.23), this rung being Linux-only: `cargo fmt --all -- --check`; `cargo clippy -p pf-client-core -p pf-vaadec --all-targets --features sdl3/build-from-source -- -D warnings`; `cargo test -p pf-client-core --lib --features sdl3/build-from-source` (171 passed); the same filtered to `video_vaapi_native` with `--include-ignored` (18 passed); `cargo test -p pf-vaadec` (48 passed). Plus the pf-lxcheck2 container for the cross-platform half — fmt, clippy and `cargo test -p pf-vaadec`, all clean. All four VAAPI legs still decode with the refactor in place, not one access unit refused: H.264 225 of 250 access units delivering a frame, H.265 204 of 250, HEVC Main 10 45 of 50 (P010), AV1 250 of 250 — the same counts and the same tiled modifier 0x200000010401b04 those legs recorded before it. ⚠ The H.26x legs live on `fix/vaapi-h264-h265-hardware-proof`, not on this branch, so they were run by overlaying that commit's test module onto the scratch tree; only the AV1 leg and the libva probe are reachable from here. This is a decode measurement, not frame-hash parity — the rung exports a driver-tiled DRM-PRIME dmabuf, so there is no CPU-readable image to hash. The alias assertions above are the real evidence and they need no device. ⚠ NOT taken: `finish`'s `outputs.last()`, which ships one frame per access unit and drops the rest of what a bump displaces (225/204/45 against 250/250/50), with no end-of-stream flush. It cannot bite punktfunk — hosts emit zero-reorder output, so `outputs` never holds more than one picture — and fixing it changes `decode()`'s one-frame-per-access-unit contract with the pump (it wants a deliverable queue, which `video_vk_native` already keeps) plus an end-of-stream flush and the `keyframe`-labels-the-access-unit defect in the same function. It is recorded and asserted on that other branch, whose three delivered-count assertions any fix has to move in the same commit; doing that from here, blind to them, would be worse than leaving it.
Low-latency desktop and game streaming with first-class Linux and Windows hosts.
Run the host on a Linux machine or a Windows PC, connect from a Mac, PC, phone, tablet, or TV, and stream your desktop or games — each device at its own native resolution and refresh rate, over your local network.
📖 Documentation: docs.punktfunk.unom.io — start with How It Works or the Quick Start.
💬 Community: Discord — chat, support, and Android beta access · r/Punktfunk.
🔒 Security: found a vulnerability? Report it privately to security@punktfunk.com — see SECURITY.md. Please don't open a public issue.
Punktfunk pairs a virtual-display streaming host with native clients on every platform. It speaks
the existing GameStream protocol, so any Moonlight client works
day one — and adds its own faster punktfunk/1 protocol that breaks the ~1 Gbps FEC wall with a
GF(2¹⁶) Leopard-RS transport. A single shared Rust core (punktfunk-core) holds the
protocol, FEC, and crypto, linked into the host and every native client — directly as a Rust crate
on Linux and Windows, and over a stable C ABI from the Apple and Android apps.
What makes it different
- Your device's exact mode. For each client that connects, the host spins up a virtual display sized to that device — 1080p60 to a laptop, 1440p120 to a desktop, 4K to a TV, all at once. No letterboxing, no scaling, no rearranging your real monitors.
- Displays you configure, not just create. Keep a game's display (and the game) alive across disconnects so a reconnect drops straight back in; make the stream your sole desktop or extend alongside your monitors; let several devices become monitors of one desktop; keep each client's scaling. One-click presets in the console — a dedicated couch box, a shared desktop, a multi-monitor workstation. See Virtual displays.
- A real virtual display on Windows, too. On Linux the host uses per-compositor virtual outputs; on Windows you get the same on-the-fly virtual display — at the client's exact mode, no physical monitor or dummy HDMI plug, even on the secure desktop (UAC / lock screen). It also has its own indirect display driver (IDD) the host pushes finished frames straight into, rather than scraping a screen — tight, push-based integration that's unusual for a Windows streaming host.
- Low latency, GPU end to end. Frames go straight from the compositor to the NVENC encoder with zero CPU copies (dmabuf → CUDA/Vulkan → NVENC), over a transport tuned for responsiveness rather than throughput. Stable 240 fps at 5120×1440; sub-millisecond capture-to-reassembly on-box, ~1.3 ms cross-machine on a LAN. (On Linux AMD/Intel, Vulkan Video for HEVC and AV1 with VAAPI for H.264 and as the fallback; a GPU-less software H.264 encoder exists as a last resort.)
- A library that fills itself. Steam and non-Steam titles show up as a grid on every client, and
plugins add their own sources — ROM Manager (your ROM collection, matched to installed emulators),
Playnite, VirtualHere. Install them from the console's Plugins page or with
punktfunk-host plugins add. See Plugins. - Works with what you already have. Any Moonlight/Artemis client connects over GameStream — and
native apps for macOS, Linux, Windows, and Android use the lower-latency
punktfunk/1protocol. - Secure by default. Hosts require a one-time SPAKE2 PIN pairing; after that, devices reconnect on a pinned identity. No accounts, no cloud. Hosts auto-advertise over mDNS, so clients find them on the network without typing an IP.
Status
| Component | State |
|---|---|
Core — punktfunk-core + C ABI (protocol · FEC · crypto · QUIC) |
✅ Complete & hardened |
| GameStream host → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads |
Native protocol — punktfunk/1 |
✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation |
| Windows host (Windows 11 22H2+, x64) | ✅ Beta — shipping as a signed installer: its own all-Rust IddCx virtual display (secure-desktop capable) with a sealed IDD-push capture path — finished frames pushed straight into its own driver, not screen-scraped (no DDA/WGC) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel, software H.264 without a GPU) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
macOS / iOS / tvOS client (clients/apple) |
✅ Streaming live: VideoToolbox decode (HEVC, and AV1 on hardware that decodes it), controllers incl. DualSense, discovery, pairing, speed test |
Linux client (clients/linux + clients/session) |
✅ Streaming live: relm4/GTK4 launcher shell that spawns a Vulkan session binary — Vulkan Video / VAAPI / software decode, PipeWire audio, SDL3 controllers, Skia console UI; ships as Flatpak/apt/rpm/Arch |
Android client (clients/android, phone + TV) |
✅ Streaming live: AMediaCodec decode + HDR10, AAudio audio, controllers, discovery, pairing |
Windows client (clients/windows, WinUI 3) |
✅ Streaming live: WinUI 3 shell + Vulkan session presenter, hardware decode on all GPU vendors via Vulkan Video → D3D11VA → software (NVIDIA + Intel validated on glass), WASAPI audio, SDL3 controllers, discovery, pairing; ships as signed MSIX (x64 + ARM64). Hardware decode and HDR10 present validated on glass on NVIDIA and Intel, including HDR pass-through on the Intel D3D11VA path |
Web console + management API (web/) |
✅ TanStack console over the OpenAPI mgmt API: host status, paired devices, on-demand PIN pairing, game library, virtual-display presets, plugin store, GPU selection, performance capture graphs, live host logs, host updates |
Every native client also ships a tiered stats overlay (Compact / Normal / Detailed) with a
shared vocabulary across platforms, and the session client carries a full gamepad-driven console
shell (pf-console-ui): host list, PIN pairing, settings, and an on-screen keyboard.
The GameStream host works with a stock Moonlight client — validated live on NVIDIA hardware
(RTX 5070 Ti, RTX 4090): PIN pairing that persists across restarts, an app catalog, RTSP/ENet/audio,
and video at the client's exact resolution and refresh via a per-session virtual output (KWin,
gamescope, Mutter, and Sway/wlroots backends), encoded with GPU zero-copy (dmabuf → CUDA/Vulkan →
NVENC) up to 5120×1440@240. The native punktfunk/1 protocol adds a QUIC control plane and a
GF(2¹⁶) Leopard-FEC + AES-GCM data plane (p50 ~0.8 ms capture→received at 720p120), with
mid-stream mode renegotiation and a wall-clock skew handshake so latency stays valid across machines.
Both run from one process: bare punktfunk-host serve is the secure native-only default
(punktfunk/1 + the management API/web console), and serve --gamestream additionally enables the
GameStream/Moonlight-compat planes (opt-in, trusted-LAN only — GameStream has inherent on-path
weaknesses). The host is managed through a REST API and web console. The host builds against
FFmpeg 7 or 8; the clients link no FFmpeg at all — they decode natively (Vulkan Video, DXVA,
VAAPI, VideoToolbox, MediaCodec, openh264 + rav1d).
What works where: the support matrix · where it's heading: the roadmap.
Install the host
Pick your platform and install from its package registry — the per-platform guide covers adding the repo, first run, and the web console. The Linux host is the primary, most battle-tested path; on SteamOS the host is built on-device by a script instead, and a Windows host ships as a signed installer (all-vendor: NVIDIA, AMD, Intel).
| Platform | Install | Guide |
|---|---|---|
| Ubuntu / Debian (apt) | sudo apt install punktfunk-host (after adding the repo) |
Ubuntu / Debian · packaging/debian |
| Bazzite / Fedora Atomic (systemd-sysext) | curl -fsSLO https://git.unom.io/unom/punktfunk/raw/branch/main/packaging/bazzite/punktfunk-sysext.sh && sudo bash punktfunk-sysext.sh install (no layering, no reboot; rpm-ostree + bootc also supported) |
Bazzite |
| Fedora (dnf) | sudo dnf install punktfunk (after adding the repo; the console comes with it) |
Fedora · packaging/rpm |
| Arch / CachyOS (pacman) | sudo pacman -Syu punktfunk-host (binary repo — always a full -Syu) |
Arch Linux · packaging/arch |
| SteamOS / Steam Deck (on-device build) | bash ~/punktfunk/scripts/steamdeck/install.sh (after cloning this repo to ~/punktfunk) |
SteamOS (Host) |
| Windows (11 22H2+, x64) | winget install unom.PunktfunkHost (after winget source add -n punktfunk https://winget.punktfunk.unom.io -t Microsoft.Rest) · or the signed setup.exe from the package registry |
Windows Host · packaging/winget |
punktfunk-host is the streaming host; punktfunk-web is the browser console (pairing + status).
Linux: every package ships systemd user units, so you don't launch the host by hand. The
host unit won't start until ~/.config/punktfunk/host.env exists, so copy the template your package
installed first:
mkdir -p ~/.config/punktfunk
# /usr/share/punktfunk/ on Fedora/Arch/Bazzite, /usr/share/punktfunk-host/ on Debian/Ubuntu
# (on Bazzite take host.env.bazzite instead)
cp /usr/share/punktfunk/host.env.example ~/.config/punktfunk/host.env
systemctl --user enable --now punktfunk-host # the streaming host
systemctl --user enable --now punktfunk-web # the web console (Arch: install punktfunk-web first)
The shipped host unit runs serve --gamestream — the native punktfunk/1 plane plus the
GameStream/Moonlight-compat planes, which belong on a trusted LAN only; for a native-only host drop
the flag with a systemctl --user edit punktfunk-host drop-in (which needs an empty ExecStart=
line before the replacement — the install guide has the snippet). Then open
https://<host-ip>:47992 and pair.
How the virtual display and input are wired up depends on your desktop — see KDE · GNOME · Steam / gamescope · Sway.
Windows: the installer registers and starts the host as a LocalSystem service, so there is
nothing to run by hand — open the web console and pair. Use
punktfunk-host service start|stop|restart|status if you need to control it. Upgrades happen in
place — the console's Updates card, winget upgrade unom.PunktfunkHost, or the newer
setup.exe over the old install; uninstall from Add/Remove Programs.
Full instructions: docs.punktfunk.unom.io/docs/install.
The console's Host page also shows when a newer host is out, along with the exact command for how this box was installed (or a one-click Update now on Windows) — see Updating the host. To remove it again, or to go back to an earlier version, see Uninstalling and Release Channels.
Connect a client
| Streaming to… | Use |
|---|---|
| Mac, iPhone, iPad, Apple TV | The Apple app (clients/apple) — also on TestFlight |
| Linux desktop / laptop | punktfunk-client (Flatpak / apt / rpm / Arch) |
| Steam Deck | The Decky plugin in Gaming Mode — it launches the client for you (Steam Deck); in Desktop Mode, the Flatpak directly |
| Android phone or TV | The Android app (clients/android) |
| Windows | Native punktfunk-client (signed MSIX) or Moonlight |
| Scripts, automation, another launcher | punktfunk — the headless CLI shipped in the Linux client packages (punktfunk pair, punktfunk hosts list --json, punktfunk launch <host>) |
| Anything else (browser, old phone, smart TV) | Moonlight over GameStream |
Each client discovers hosts on the network automatically and does a one-time PIN pairing. Per-device install steps: /docs/install-client.
Build & test (from source)
For development, or as an install fallback where no package is available:
cargo build --workspace # core, host, tray, shared client crates, Linux shell + session client, the `punktfunk` CLI, probe (Linux & macOS)
cargo test --workspace # unit + loopback + proptest + C ABI harness
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all --check
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip proof
The C header regenerates from crates/punktfunk-core/src/abi.rs on every build (cbindgen via
build.rs) into include/punktfunk_core.h. The Apple, Android, and Windows clients have their own
toolchains (Xcode/swift build, Gradle, and cargo on the MSVC target) — see each client's README
and the docs site.
Layout
crates/
punktfunk-core/ protocol · FEC · pacing · crypto · QUIC control plane — the C ABI (lib + cdylib + staticlib)
punktfunk-host/ the host (Linux + Windows): virtual displays · capture · encode · input · GameStream · punktfunk/1 · mgmt
pf-client-core/ shared client plumbing (Linux + Windows): session pump · native decode ladder · audio · SDL3 gamepads · trust · discovery
pf-presenter/ Vulkan session presenter: SDL3 window · ash swapchain · frame present · input capture
pf-console-ui/ Skia console UI for the session client: gamepad shell · stats OSD · pairing · on-screen keyboard
pf-bitstream/ H.264 / H.265 / AV1 bitstream parsing + per-AU decode plans — the one parser every native rung submits from
pf-vkdecode/ native Vulkan Video decode (H.264 / H.265 / AV1) on the presenter's own device
pf-dxvadec/ native DXVA buffer layouts + AuPlan → picparams conversion (the Windows D3D11VA rung)
pf-vaadec/ native libva buffer layouts + AuPlan → picparams conversion (the Linux VAAPI rung)
pf-driver-proto/ host ↔ pf-vdisplay driver contract: control IOCTLs + IDD-push frame transport (no_std)
punktfunk-tray/ host tray icon (Windows notification area / Linux StatusNotifierItem)
clients/
apple/ macOS / iOS / tvOS app (Swift · VideoToolbox · Metal · GameController)
linux/ Linux launcher shell (Rust · relm4 / GTK4 / libadwaita) — spawns the session client to stream
session/ punktfunk-session, the Vulkan streaming session (Rust · SDL3 · ash · Skia console UI) — also runs standalone (gamescope, Decky)
windows/ Windows desktop app (Rust · WinUI 3 · D3D11 · WASAPI · SDL3)
android/ Android phone + TV app (Kotlin · Rust JNI core · AMediaCodec · AAudio)
cli/ punktfunk, the headless client CLI — pair · hosts · wake · library · launch · punktfunk:// links
probe/ headless reference / measurement client for punktfunk/1
decky/ Steam Deck Decky plugin
web/ web console (TanStack) over the management API — status · devices · pairing · library · displays · plugins · GPUs · performance · logs · updates
api/openapi.json management-API OpenAPI spec (regenerated via `punktfunk-host openapi`, checked in)
sdk/ `@punktfunk/host` — TypeScript management-API client + event stream (Effect)
plugin-kit/ `@punktfunk/plugin-kit` — the plugin authoring kit (bun / TypeScript)
packaging/ apt · rpm / COPR · Arch · Flatpak · Bazzite sysext + bootc · Windows installer + drivers · winget · Nix · gamescope
docs-site/ public documentation site (Fumadocs) — https://docs.punktfunk.unom.io
include/punktfunk_core.h cbindgen-generated C header (checked in)
tools/ latency-probe · loss-harness (measurement)
ci/ CI container images (rust-ci · fedora-rpm)
Design invariants
- One core, linked everywhere. Protocol, FEC, and crypto live in
punktfunk-coreexactly once, exposed over a stable, versioned C ABI (punktfunk_abi_version(),PunktfunkConfigcarries its ownstruct_size). Every native client links the same core. - No async on the hot path. The per-frame pipeline uses native threads only;
tokio/quinnare gated behind the off-by-defaultquicfeature (control plane only). - Native client resolution, no scaling. Each session gets a virtual output at exactly the
client's WxH@Hz; each compositor keeps its own backend behind a shared
VirtualDisplaytrait. - FEC is the wall-breaker. GF(2⁸) (≤255 shards/block) for Moonlight compatibility; GF(2¹⁶)
(≤65535 shards/block, SIMD, O(n log n)) for
punktfunk/1to push past ~1 Gbps.
License
Licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)
at your option — SPDX-License-Identifier: MIT OR Apache-2.0.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions. See CONTRIBUTING.md.
Third-party components
Punktfunk's own source is MIT/Apache-2.0. Shipped binaries additionally link third-party components
under their own (permissive) licenses — see THIRD-PARTY-NOTICES.txt
(regenerate with scripts/gen-third-party-notices.sh). The Windows host build also
bundles FFmpeg under the LGPL v2.1+ (dynamically linked, replaceable DLLs; the license text and
notice ship in the installed licenses/ folder). The clients bundle no FFmpeg — they link
none.
Trademarks
Punktfunk is an independent project and is not affiliated with, endorsed by, or sponsored by NVIDIA, Microsoft, Sony, Valve, or the Moonlight project. "GameStream", "Moonlight", "Xbox", "DualSense", "DualShock", and "PlayStation" are trademarks of their respective owners and are used here only to describe interoperability.