enricobuehler
bab309017d
WP4.2 of the pf-encode audit, validated on RADV PHOENIX (780M, Mesa 26.0.4) under the Vulkan
validation layers rather than by inspection alone. Three of the five reproduce with the
driver naming the violation; one turned out milder than filed and is recorded as such.
1. `ensure_cpu_rgb` cached the staging image on FORMAT ONLY while sizing it to the SOURCE
frame, and the copy uses the CURRENT frame's extent — so a same-format source-size
increase copied past the allocation. Now keyed on (format, width, height). This is the
memory-safety one: on hardware it was 8 x
VUID-vkCmdCopyBufferToImage-imageSubresource-07971 ("extent.width (512) exceeds
imageSubresource width extent (128)") and `submit` returned Ok every single time, so
nothing upstream could notice. Zero after the fix. Note the trigger for anyone re-testing:
the image is cached PER RING SLOT, so the ring must wrap before a slot sees a larger frame
— two differently-sized submits in a row land on different slots and look fine.
2. The RGB-direct CPU padding loop only ever grows a source into the aligned extent; a source
LARGER than the encode extent panicked on the row `copy_from_slice`. Now refused by name,
matching what the DMA-BUF arms already do, so a bad frame takes the encoder-rebuild path
instead of unwinding out of the encode thread with an index message.
3. AV1 wrote `render_width/height_minus_1` but never set
`render_and_frame_size_different`, and AV1 ignores the render size unless that flag says it
differs — so every mode needing 64x16 alignment presented the padding. libdav1d on the
encoded stream now reports `size 1920x1088 ... render 1920x1080`; before, the flag being
unset meant render defaulted to the coded 1088, i.e. 8 rows of duplicated edge pixels
shipped to the client.
4. `read_slot` read the PERF timestamp pool whenever the pool merely EXISTED, but the
padded-RGB path's CPU-upload arm records its own command buffer and writes no timestamps —
reading an unreset query with WAIT is undefined. Now gated on a per-slot `ts_written`,
because "a pool exists" was being used as proof of "the pool was written". SEVERITY
CORRECTION vs the audit: this is not a hang in practice. It needs PUNKTFUNK_PERF *and*
RGB-direct *and* the non-default PUNKTFUNK_VULKAN_RGB_TRUE_EXTENT=0 *and* an unaligned mode
*and* CPU capture, and RADV then fails the read rather than blocking. Another driver may
block, hence the fix.
5. `reset()` re-arms `first_frame`, which was also what decided whether begin-video-coding
declares the rate-control state. But a reset does not rebuild the session, so the CBR
installed earlier is still current when the next frame opens its coding scope: the
declaration was omitted exactly when it was required. Split into `rc_installed`, which
survives reset. On hardware this was
VUID-vkCmdBeginVideoCodingKHR-pBeginInfo-08253 ("no VkVideoEncodeRateControlInfoKHR ... but
the currently set mode is CBR"); zero after.
Also fixes two pre-existing AV1 spec violations that were firing on EVERY frame of a shipped
path and are not in the audit at all — found only because the validation layers were on, and
confirmed untouched by this branch before being attributed as pre-existing:
`constantQIndex` must be 0 unless rate control is DISABLED (VUID-...-10320; this session is
always CBR, so the driver owns Q), and `pStdPictureInfo->pSegmentation` must be NULL
(VUID-...-10350; it pointed at a zeroed struct, and leaving it unset means the same
"segmentation disabled" while being spec-correct).
Items 1 and 5 keep their reproductions as `#[ignore]`d hardware tests, documented as
meaningful only under `VK_LOADER_LAYERS_ENABLE='*validation*'` since the violations are
invisible to the API's return values.
STILL OPEN, filed not fixed (three more pre-existing validation errors the cleanup exposed):
VUID-VkQueryPoolCreateInfo-pNext-pNext on the AV1 and RGB paths, and 2 x
VUID-VkImageViewCreateInfo-image-08336 (the RGB-direct encode-src view's format features lack
VIDEO_ENCODE_INPUT) — the latter may be a validation-layer gap around the VALVE extension
rather than our bug, and wants real analysis rather than a quick patch.
Verified: canonical Linux gate (docker linux/amd64) fmt + clippy --all-targets at default and
at nvenc,vulkan-encode,pyrowave + both test legs; and on RADV PHOENIX all four `vulkan_smoke*`
tests pass at 256x256 and at 1920x1080 with zero validation errors on the paths touched here.
Low-latency desktop and game streaming with first-class Linux and Windows hosts.
Run the host on a Linux machine or a Windows PC, connect from a Mac, PC, phone, tablet, or TV, and stream your desktop or games — each device at its own native resolution and refresh rate, over your local network.
📖 Documentation: docs.punktfunk.unom.io — start with How It Works or the Quick Start.
💬 Community: Discord — chat, support, and Android beta access · r/Punktfunk.
🔒 Security: found a vulnerability? Report it privately to security@punktfunk.com — see SECURITY.md. Please don't open a public issue.
punktfunk pairs a virtual-display streaming host with native clients on every platform. It speaks
the existing GameStream protocol, so any Moonlight client works
day one — and adds its own faster punktfunk/1 protocol that breaks the ~1 Gbps FEC wall with a
GF(2¹⁶) Leopard-RS transport. A single shared Rust core (punktfunk-core) holds the
protocol, FEC, and crypto, linked into the host and every native client — directly as a Rust crate
on Linux and Windows, and over a stable C ABI from the Apple and Android apps.
What makes it different
- Your device's exact mode. For each client that connects, the host spins up a virtual display sized to that device — 1080p60 to a laptop, 1440p120 to a desktop, 4K to a TV, all at once. No letterboxing, no scaling, no rearranging your real monitors.
- Displays you configure, not just create. Keep a game's display (and the game) alive across disconnects so a reconnect drops straight back in; make the stream your sole desktop or extend alongside your monitors; let several devices become monitors of one desktop; keep each client's scaling. One-click presets in the console — a dedicated couch box, a shared desktop, a multi-monitor workstation. See Virtual displays.
- A real virtual display on Windows, too. On Linux the host uses per-compositor virtual outputs; on Windows you get the same on-the-fly virtual display — at the client's exact mode, no physical monitor or dummy HDMI plug, even on the secure desktop (UAC / lock screen). It also has its own indirect display driver (IDD) the host pushes finished frames straight into, rather than scraping a screen — tight, push-based integration that's unusual for a Windows streaming host.
- Low latency, GPU end to end. Frames go straight from the compositor to the NVENC encoder with zero CPU copies (dmabuf → CUDA/Vulkan → NVENC), over a transport tuned for responsiveness rather than throughput. Stable 240 fps at 5120×1440; sub-millisecond capture-to-reassembly on-box, ~1.3 ms cross-machine on a LAN. (AMD/Intel encode via VAAPI, and a GPU-less software H.264 encoder exists as a fallback.)
- Works with what you already have. Any Moonlight/Artemis client connects over GameStream — and
native apps for macOS, Linux, Windows, and Android use the lower-latency
punktfunk/1protocol. - Secure by default. Hosts require a one-time SPAKE2 PIN pairing; after that, devices reconnect on a pinned identity. No accounts, no cloud. Hosts auto-advertise over mDNS, so clients find them on the network without typing an IP.
Status
| Component | State |
|---|---|
Core — punktfunk-core + C ABI (protocol · FEC · crypto · QUIC) |
✅ Complete & hardened |
| GameStream host → stock Moonlight | ✅ Live end-to-end: pairing, RTSP, audio, per-client virtual output at native resolution, GPU zero-copy NVENC, gamepads |
Native protocol — punktfunk/1 |
✅ Validated live: QUIC control + GF(2¹⁶) FEC/AES-GCM data plane, PIN pairing, mDNS discovery, mid-stream mode renegotiation |
| Windows host (Windows 11 22H2+, x64) | 🟡 Implemented & shipping as a signed installer: its own all-Rust IddCx virtual display (secure-desktop capable) with a sealed IDD-push capture path — finished frames pushed straight into its own driver, not screen-scraped (no DDA/WGC) · GPU encode (NVENC on NVIDIA, AMF/QSV on AMD/Intel, software H.264 without a GPU) · WASAPI audio · bundled virtual-gamepad drivers (no ViGEmBus) · HDR incl. Vulkan-game HDR. NVIDIA live-validated; AMD/Intel CI-green |
macOS / iOS / tvOS client (clients/apple) |
✅ Streaming live: VideoToolbox decode (HEVC, and AV1 on hardware that decodes it), controllers incl. DualSense, discovery, pairing, speed test |
Linux client (clients/linux + clients/session) |
✅ Streaming live: relm4/GTK4 launcher shell that spawns a Vulkan session binary — Vulkan Video / VAAPI / software decode, PipeWire audio, SDL3 controllers, Skia console UI; ships as Flatpak/apt/rpm/Arch |
Android client (clients/android, phone + TV) |
✅ Streaming live: AMediaCodec decode + HDR10, AAudio audio, controllers, discovery, pairing |
Windows client (clients/windows, WinUI 3) |
✅ Streaming live: WinUI 3 shell + Vulkan session presenter, hardware decode on all GPU vendors via Vulkan Video → D3D11VA → software (NVIDIA + Intel validated on glass), WASAPI audio, SDL3 controllers, discovery, pairing; ships as signed MSIX (x64 + ARM64). HDR10 implemented, on-glass validation pending |
Web console + management API (web/) |
✅ TanStack console over the OpenAPI mgmt API: host status, paired devices, on-demand PIN pairing, GPU selection, performance capture graphs, live host logs |
Every native client also ships a tiered stats overlay (Compact / Normal / Detailed) with a
shared vocabulary across platforms, and the session client carries a full gamepad-driven console
shell (pf-console-ui): host list, PIN pairing, settings, and an on-screen keyboard.
The GameStream host works with a stock Moonlight client — validated live on NVIDIA hardware
(RTX 5070 Ti, RTX 4090): PIN pairing that persists across restarts, an app catalog, RTSP/ENet/audio,
and video at the client's exact resolution and refresh via a per-session virtual output (KWin,
gamescope, Mutter, and Sway/wlroots backends), encoded with GPU zero-copy (dmabuf → CUDA/Vulkan →
NVENC) up to 5120×1440@240. The native punktfunk/1 protocol adds a QUIC control plane and a
GF(2¹⁶) Leopard-FEC + AES-GCM data plane (p50 ~0.8 ms capture→received at 720p120), with
mid-stream mode renegotiation and a wall-clock skew handshake so latency stays valid across machines.
Both run from one process: bare punktfunk-host serve is the secure native-only default
(punktfunk/1 + the management API/web console), and serve --gamestream additionally enables the
GameStream/Moonlight-compat planes (opt-in, trusted-LAN only — GameStream has inherent on-path
weaknesses). The host is managed through a REST API and web console. Builds against FFmpeg 7 or 8.
Full milestone status: docs.punktfunk.unom.io/docs/status · roadmap: /docs/roadmap.
Install the host
Pick your platform and install from its package registry — the per-platform guide covers adding the repo, first run, and the web console. The Linux host is the primary, most battle-tested path; a Windows host also ships as a signed installer (all-vendor: NVIDIA, AMD, Intel).
| Platform | Install | Guide |
|---|---|---|
| Ubuntu / Debian (apt) | sudo apt install punktfunk-host (after adding the repo) |
Ubuntu — GNOME · KDE |
| Bazzite / Fedora Atomic (systemd-sysext) | sudo bash punktfunk-sysext.sh install (no layering, no reboot; rpm-ostree + bootc also supported) |
Bazzite |
| Fedora (dnf) | dnf install punktfunk punktfunk-web (after adding the repo) |
Fedora — KDE |
| Arch / Steam Deck (pacman / sysext) | pacman -Sy punktfunk-host (binary repo) · sysext .raw (SteamOS) |
packaging/arch |
| Windows (11 22H2+, x64) | signed setup.exe from the package registry |
Windows Host |
punktfunk-host is the streaming host; punktfunk-web is the browser console (pairing + status).
After install, run punktfunk-host serve inside your desktop session (the secure native default;
add --gamestream on a trusted LAN if you also want stock Moonlight clients), then pair from the web
console. Full instructions: docs.punktfunk.unom.io/docs/install.
Connect a client
| Streaming to… | Use |
|---|---|
| Mac, iPhone, iPad, Apple TV | The Apple app (clients/apple) — also on TestFlight |
| Linux desktop / laptop, Steam Deck | punktfunk-client (Flatpak / apt / rpm / Arch) |
| Android phone or TV | The Android app (clients/android) |
| Windows | Native punktfunk-client (signed MSIX) or Moonlight |
| Anything else (browser, old phone, smart TV) | Moonlight over GameStream |
Each client discovers hosts on the network automatically and does a one-time PIN pairing. Per-device install steps: /docs/install-client.
Build & test (from source)
For development, or as an install fallback where no package is available:
cargo build --workspace # core, host, tray, shared client crates, Linux shell + session client, probe (Linux & macOS)
cargo test --workspace # unit + loopback + proptest + C ABI harness
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all --check
cargo run -p loss-harness # FEC loss-resilience sweep (no network needed)
bash crates/punktfunk-core/tests/c/run.sh # standalone C-ABI link + round-trip proof
The C header regenerates from crates/punktfunk-core/src/abi.rs on every build (cbindgen via
build.rs) into include/punktfunk_core.h. The Apple, Android, and Windows clients have their own
toolchains (Xcode/swift build, Gradle, and cargo on the MSVC target) — see each client's README
and the docs site.
Layout
crates/
punktfunk-core/ protocol · FEC · pacing · crypto · QUIC control plane — the C ABI (lib + cdylib + staticlib)
punktfunk-host/ the host (Linux + Windows): virtual displays · capture · encode · input · GameStream · punktfunk/1 · mgmt
pf-client-core/ shared client plumbing (Linux + Windows): session pump · FFmpeg decode · audio · SDL3 gamepads · trust · discovery
pf-presenter/ Vulkan session presenter: SDL3 window · ash swapchain · frame present · input capture
pf-console-ui/ Skia console UI for the session client: gamepad shell · stats OSD · pairing · on-screen keyboard
pf-ffvk/ FFmpeg Vulkan hwcontext bindings (AVVkFrame) for Vulkan Video decode on the presenter's device
pf-driver-proto/ host ↔ pf-vdisplay driver contract: control IOCTLs + IDD-push frame transport (no_std)
punktfunk-tray/ host tray icon (Windows notification area / Linux StatusNotifierItem)
clients/
apple/ macOS / iOS / tvOS app (Swift · VideoToolbox · Metal · GameController)
linux/ Linux launcher shell (Rust · relm4 / GTK4 / libadwaita) — spawns the session client to stream
session/ punktfunk-session, the Vulkan streaming session (Rust · SDL3 · ash · Skia console UI) — also runs standalone (gamescope, Decky)
windows/ Windows desktop app (Rust · WinUI 3 · D3D11 · WASAPI · SDL3)
android/ Android phone + TV app (Kotlin · Rust JNI core · AMediaCodec · AAudio)
probe/ headless reference / measurement client for punktfunk/1
decky/ Steam Deck Decky plugin
web/ web console (TanStack) over the management API — status · devices · pairing · GPUs · performance · logs
api/openapi.json management-API OpenAPI spec (regenerated via `punktfunk-host openapi`, checked in)
packaging/ apt · rpm / COPR · Arch · Flatpak · Bazzite bootc image
docs-site/ public documentation site (Fumadocs) — https://docs.punktfunk.unom.io
include/punktfunk_core.h cbindgen-generated C header (checked in)
tools/ latency-probe · loss-harness (measurement)
ci/ CI container images (rust-ci · fedora-rpm)
Design invariants
- One core, linked everywhere. Protocol, FEC, and crypto live in
punktfunk-coreexactly once, exposed over a stable, versioned C ABI (punktfunk_abi_version(),PunktfunkConfigcarries its ownstruct_size). Every native client links the same core. - No async on the hot path. The per-frame pipeline uses native threads only;
tokio/quinnare gated behind the off-by-defaultquicfeature (control plane only). - Native client resolution, no scaling. Each session gets a virtual output at exactly the
client's WxH@Hz; each compositor keeps its own backend behind a shared
VirtualDisplaytrait. - FEC is the wall-breaker. GF(2⁸) (≤255 shards/block) for Moonlight compatibility; GF(2¹⁶)
(≤65535 shards/block, SIMD, O(n log n)) for
punktfunk/1to push past ~1 Gbps.
License
Licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)
at your option — SPDX-License-Identifier: MIT OR Apache-2.0.
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions. See CONTRIBUTING.md.
Third-party components
punktfunk's own source is MIT/Apache-2.0. Shipped binaries additionally link third-party components
under their own (permissive) licenses — see THIRD-PARTY-NOTICES.txt
(regenerate with scripts/gen-third-party-notices.sh). The Windows host and client builds also
bundle FFmpeg under the LGPL v2.1+ (dynamically linked, replaceable DLLs; the license text and
notice ship in the installed licenses/ folder).
Trademarks
punktfunk is an independent project and is not affiliated with, endorsed by, or sponsored by NVIDIA, Microsoft, Sony, Valve, or the Moonlight project. "GameStream", "Moonlight", "Xbox", "DualSense", "DualShock", and "PlayStation" are trademarks of their respective owners and are used here only to describe interoperability.