enricobuehler
d01a8fd17a
windows-host / package (push) Failing after 4m16s
ci / rust (push) Failing after 4m56s
ci / web (push) Failing after 22s
ci / docs-site (push) Successful in 1m7s
android / android (push) Successful in 9m19s
ci / bench (push) Successful in 4m47s
decky / build-publish (push) Successful in 11s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Failing after 3s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 4s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
docker / deploy-docs (push) Has been skipped
deb / build-publish (push) Failing after 6m29s
rpm / build-publish (bazzite, punktfunk-fedora-rpm) (push) Failing after 7m4s
rpm / build-publish (fedora-44, punktfunk-fedora44-rpm) (push) Failing after 7m17s
apple / swift (push) Successful in 1m13s
apple / screenshots (push) Successful in 5m27s
NVIDIA/AMD Vulkan ICDs refuse to *advertise* an HDR color space for a surface on an
IddCx indirect/virtual display, so Vulkan games (Doom: The Dark Ages, id Tech, Indiana
Jones, …) report "device does not support HDR" — even though Windows HDR, DWM compose,
and the client PQ stream all work, and the ICD happily *accepts + presents* a forced HDR
swapchain there. The whole gap is enumeration; the community (Apollo/Sunshine/VDD) wrote
this off as kernel-side / unfixable.
Add VK_LAYER_PUNKTFUNK_hdr_inject (packaging/windows/pf-vkhdr-layer/): a standalone
cdylib Vulkan implicit layer that appends {A2B10G10R10, HDR10_ST2084} + {RGBA16F, scRGB}
to vkGetPhysicalDeviceSurfaceFormats[2]KHR (no need to hook vkCreateSwapchainKHR — the
ICD doesn't validate the color space there). Self-gated on the surface monitor's actual
advanced-color state (DisplayConfig GET_ADVANCED_COLOR_INFO), so it is a complete no-op
on SDR sessions and real monitors (dedup). Always-on (registry-discovered) so it works
regardless of how a game is launched — env-scoping silently fails for already-running
Steam. Escape hatches: DISABLE_PF_VKHDR, PF_VKHDR_EXCLUDE, and a built-in kernel-anti-
cheat denylist.
The installer builds/signs/stages it and registers it under
HKLM64\SOFTWARE\Khronos\Vulkan\ImplicitLayers (opt-out "Install the HDR Vulkan layer"
task); windows-host CI fmt+clippy-gates it (msvc-only FFI).
Live-validated on the RTX box: Doom: The Dark Ages enables HDR over the pf-vdisplay
virtual display.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
6.1 KiB
6.1 KiB
title, description
| title | description |
|---|---|
| GameStream Host | Stream to a stock Moonlight client on a client-sized virtual display. |
The shippable milestone (plan §8). A stock Moonlight/Artemis client discovers this host,
pairs, launches, and gets video (then input, then audio) on a client-sized virtual display.
Ground-truth protocol reference: research/gamestream-protocol-research.json
(distilled from Sunshine + moonlight-common-c source; cite those for byte-level detail).
Architecture (respects the "one core" invariant)
- punktfunk-core gains a P1 GameStream wire codec (
ProtocolPhase::P1GameStream, the hook already exists): the exact RTP+NV_VIDEO_PACKETframing, the GameStream FEC shard layout, and the video/audio AES-GCM/CBC paths. Hot path, native threads, no async. Kept beside punktfunk's native internal format (P2), selected by phase. - punktfunk-host gains the control plane (tokio/axum OK — I/O-bound, not the hot path): mDNS discovery, nvhttp serverinfo + the 4-phase pairing, the RTSP handshake, the ENet control stream + input injection, the virtual-display lifecycle, and Opus audio encode.
Port map (base 47989; Moonlight derives all by offset)
| Port | Proto | Role |
|---|---|---|
| 47989 | TCP | HTTP nvhttp (unpaired: /serverinfo, /pair PIN flow) |
| 47984 | TCP | HTTPS nvhttp (paired; client-cert pinned) — /launch, /resume, … |
| 48010 | TCP | RTSP (OPTIONS/DESCRIBE/SETUP/ANNOUNCE/PLAY) |
| 47998 | UDP | Video RTP (+ RS-FEC, optional AES-GCM) |
| 47999 | UDP | Audio RTP (Opus, RS-FEC 4+2, optional AES-CBC) |
| 48000 | UDP | ENet control stream (AES-GCM) + remote input |
| 5353 | UDP | mDNS _nvstream._tcp.local advertisement |
Key wire facts (the non-obvious ones)
- Video datagram =
RTP_PACKET(12, BIG-endian)+reserved[4]+NV_VIDEO_PACKET(16, LITTLE-endian)+ payload. Endianness differs within the same packet.header=0x80|0x10. fecInfo(u32 LE) =(dataShards<<22)|(fecIndex<<12)|(fecPercentage<<4); parityShards is recomputed by the client asceil(dataShards*pct/100)— must match exactly.multiFecBlocks=(blockIdx<<4)|((nBlocks-1)<<6); ≤4 FEC blocks/frame, ≤255 shards/block.- Each frame's bitstream is prefixed with an 8-byte
video_short_frame_header_t(headerType=0x01,frameType2=IDR,lastPayloadLen) before striping into shards. - Shard size =
packetSize + 16. Data shards first, then parity, over a contiguous RTP sequence range. Last data shard zero-padded. - Video crypto (when
SS_ENC_VIDEOnegotiated): AES-128-GCM, key = raw 16-byte RIKEY (from/launch?rikey=), IV =counter_le[8]||0,0,0||'V'(0x56), NO AAD, 32-byteENC_VIDEO_HEADER{iv[12],frameNumber,tag[16]}prefix; FEC first, then encrypt per shard. - Pairing: PIN key =
SHA-256(salt[16] || ascii_pin)[..16]; AES-128-ECB (no padding) for the challenge blocks; SHA-256 rolling hashes; RSA-SHA256 signatures over X.509 certs; the client cert is pinned for subsequent HTTPS. 4 phases over/pair?phrase=…. - RTSP
Session: DEADBEEFCAFE;timeout = 90(literal),Transport: server_port=<p>,streamid=video/0/0/control/13/0. ANNOUNCE carries the negotiated config (x-nv-video[0].*,x-nv-vqos[0].*) → maps topunktfunk_core::Config.
The two highest interop risks (validate EARLY)
- RS-FEC matrix compatibility. Sunshine + Moonlight both use nanors (GF(2⁸), poly
0x11d, Vandermonde systematic). punktfunk-core uses
reed-solomon-erasure(Cauchy) — parity bytes likely don't match, so Moonlight silently fails to recover any frame with a lost data shard. Mitigation: on a clean LAN with no loss the client never runs RS decode, so defer this — get a frame decoded first, then FFI/port nanors for loss recovery. - Crypto layout. punktfunk's
SessionCrypto(salt + seq-as-AAD) is wire-incompatible. P1 needs a separate GameStream GCM path. Mitigation: video encryption is negotiated and usually off on LAN — implement plaintext video first, add GCM later.
Phasing (each phase independently testable with a real Moonlight client)
- P1.1 — Discovery + serverinfo + pairing. mDNS
_nvstream._tcp, HTTP/HTTPS nvhttp,/serverinfoXML, the 4-phase pairing + cert pinning. Acceptance: Moonlight discovers, pairs (PIN), and shows the host as ready. ← first slice. - P1.2 — Launch + RTSP + virtual display.
/launch(parse rikey/rikeyid/mode), the RTSP handshake, negotiateConfig, create a wlroots virtual output sized to the client. Acceptance: Moonlight completes RTSP and the host stands up the UDP streams. - P1.3 — Video (punktfunk-core P1 codec), plaintext, clean-LAN. RTP+NV framing + FEC shard layout in punktfunk-core; wire the spike's NVENC AUs → UDP 47998. Acceptance: Moonlight DISPLAYS video.
- P1.4 — Control + input. ENet (
rusty_enet) control stream; decode input →inject.rs(uinput/reis); request-IDR → force NVENC keyframe. Acceptance: mouse/keyboard work. - P1.5 — Robustness: FEC recovery + encryption. nanors-exact FEC; per-shard AES-GCM.
Acceptance: stable under
tc netemloss; encrypted streams. - P1.6 — Audio + polish. Opus + audio RTP/FEC/CBC (UDP 47999); disconnect teardown; KWin backend for the user's KDE box. Acceptance: full game stream with sound — the GameStream-host goal.
Crates (verified available)
mdns-sd 0.20 (discovery) · axum 0.8 + rustls + tokio-rustls (nvhttp/HTTPS, custom
ClientCertVerifier for pinning) · rcgen 0.14 + x509-parser 0.18 + rsa/sha2/aes/
ecb (pairing crypto) · hand-rolled RTSP over tokio::net::TcpListener · rusty_enet 0.4
(control) · opus 0.3 (audio) · reis 0.6 + input-linux (input) · aes-gcm (already in
core) for the P1 video/control GCM path; nanors (FFI/port) for FEC recovery in P1.5.
Testing note
The host is headless; end-to-end needs a stock Moonlight client on the LAN pointed at
this box (manual "add host" by IP works without mDNS). P1.1 is testable with curl against
/serverinfo + the Moonlight pair flow; P1.3+ needs a client that can display.