unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 4 Wiki Activity

feat(host): Apollo-backlog hardening — cert gate, NVENC RFI, media QoS, async injector

Browse Source 
A pass over the apollo-comparison backlog (re-verified against current code).
Lands four items end-to-end plus a Windows-DualSense scoping doc.

- #5/#92/#26 — GameStream paired-cert allow-list. tls.rs surfaces the verified
  peer cert to handlers (serve_https + PeerCertFingerprint, now shared with the
  mgmt API instead of duplicated); nvhttp gates /launch /resume /applist /cancel
  on AppState.paired and reports a real PairStatus; save_paired writes atomically
  (temp+rename). Closes the "mTLS accepts any client cert" hole. + regression test.

- #6/#51/#19/#22 — NVENC caps query -> reference-frame invalidation. nvenc.rs
  query_caps probes nvEncGetEncodeCaps (max dims / 10-bit / custom-VBV / RFI),
  rejecting over-range modes and degrading 10-bit->8-bit instead of an opaque
  InvalidParam. New Encoder::invalidate_ref_frames (default false -> caller
  keyframes); the Windows NVENC path implements real RFI (multi-ref DPB +
  nvEncInvalidateRefFrames, dedup + IDR-on-overflow). control.rs decodes the
  0x0301 lost-frame range (Apollo's IDX_INVALIDATE_REF_FRAMES) -> AppState.rfi_range
  -> encode loop, falling back to a keyframe. NOTE: the Windows NVENC impl is
  RTX-box/CI-pending (can't compile on Linux); adversarially reviewed vs the SDK.

- #43/#72 — media socket QoS + buffer growth. New punktfunk_core::transport::qos:
  grow_socket_buffers (factored out the native plane's 32MB SO_SNDBUF growth so the
  GameStream sockets reuse it) + set_media_qos (opt-in PUNKTFUNK_DSCP=1: DSCP CS5
  video / CS6 audio + Linux SO_PRIORITY, Apollo's scheme). Wired into UdpTransport
  and the GameStream video/audio sockets. Windows IP_TOS needs qWAVE (follow-up).

- #8/#45 — GameStream input injection off the ENet service thread. on_receive no
  longer injects inline (a slow inject head-blocked ENet keepalive/retransmit); it
  forwards to a dedicated injector thread. The hardened InjectorService moved from
  punktfunk1 into crate::inject (shared by both planes) + a coalesce step that sums
  adjacent relative-mouse/scroll deltas while preserving button/key/abs ordering.

Docs: re-verified apollo-comparison.md status (22 items already done/obsolete since
the snapshot) + windows-dualsense-scoping.md (ViGEm can't emulate a DualSense; real
DS5 on Windows needs a VHF virtual-HID driver — web-research pass pending).

fmt + clippy -D warnings clean; full workspace test suite green; no C-ABI/OpenAPI drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-06-21 00:06:30 +00:00
parent a2a6b858f7
commit 450bcf1e7b
20 changed files with 1060 additions and 281 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/apollo-comparison.md
+74 -11
View File
@@ -1601,7 +1601,70 @@ adversarial-verify pass. *Area* is the investigation that surfaced it.
> re-copying the desktop and recompositing the cursor at its new position. `last_present` is repeated
> only on a genuine `WAIT_TIMEOUT` (nothing changed) or a rebuild gap — correct. No stutter from this
> cause. The only real (perf-only) delta is the redundant full-surface copy per pointer update; deferred.
> - **2026-06-20 — re-verified the whole backlog against current code + landed the security & RFI
> chain.** A full re-verification (one agent per subsystem, checked against the live tree rather than
> this snapshot) found **22 of 96 items already done or obsolete since 2026-06-16** — the table below
> is the ORIGINAL snapshot and its blank ✓V cells do NOT reflect that; see **Re-verified status
> (2026-06-20)** immediately below for the authoritative current state.
### Re-verified status (2026-06-20)
The table further down is the 2026-06-16 snapshot. Re-verifying each item against the current tree
(which shipped the in-binary Windows service, two-process secure desktop, DDA born-lost fixes, VAAPI
host, adaptive FEC, etc. in between) gives the current state:
**Done since the snapshot** (gap closed in current code — do not re-do): #1, #2, #4, #13, #16, #20,
#21, #24, #25, #35, #37, #42, #47, #49, #55, #57, #64, #87.
**Obsolete / not-a-bug** (premise no longer applies to punktfunk): #34 (idle dup-lock release), #53
(NvEnc struct-version minimization — handled by the SDK crate), #90 (bitrate-derived pacing —
Apollo paces to a fixed link ceiling, not negotiated bitrate, and punktfunk is pixel-rate-bound by
design), #95 (expired-cert tolerance — n/a to the trust model).
**Landed this pass (2026-06-20, working tree):**
- **#5 + #92 + #26 — GameStream paired-cert allow-list + atomic store.** `gamestream/tls.rs` now
surfaces the verified peer cert to handlers (`serve_https` + `PeerCertFingerprint`, shared with the
mgmt API instead of duplicated); `nvhttp.rs` gates `/launch`/`/resume`/`/applist`/`/cancel` on the
`AppState.paired` fingerprint set and reports a real `PairStatus`; `mod.rs::save_paired` writes
atomically (temp + rename). Regression test `nvhttp::tests::launch_gate_requires_a_pinned_client_cert`.
Compiled + clippy-clean + tested on Linux. (Closes the "GameStream TLS accepts any client cert" hole.)
- **#6 + #51 — NVENC capability query.** `encode/nvenc.rs::query_caps` probes `nvEncGetEncodeCaps`
(WIDTH/HEIGHT_MAX, 10-bit, custom-VBV, ref-pic-invalidation) once before configuring: rejects an
over-range mode with a clear error (instead of an opaque InvalidParam the bitrate-clamp search
misreads), downgrades 10-bit→8-bit when unsupported, gates custom VBV, and records the RFI flag.
Windows-only — adversarially reviewed against the SDK source (verdict SHIP); compile pending the RTX
box / Windows CI.
- **#19 + #22 — reference-frame invalidation instead of always-IDR.** New
`Encoder::invalidate_ref_frames(first, last) -> bool` (default `false` → caller keyframes; only the
Windows NVENC path implements real RFI: a multi-ref DPB gated on caps + `nvEncInvalidateRefFrames`
with dedup + IDR-on-overflow). The GameStream control plane decodes the `0x0301` lost-frame range
(two LE i64, Apollo's `IDX_INVALIDATE_REF_FRAMES`) and routes it via `AppState.rfi_range` to the
encode loop, which prefers invalidation and falls back to a keyframe. Cross-platform wiring compiled
+ tested on Linux (where it degrades to IDR — libavcodec/VAAPI can't express RFI); the NVENC
implementation is RTX-box/CI-pending. (Native punktfunk/1 RFI sites stay `request_keyframe` — the
protocol carries no frame range yet; the trait default keeps that correct.)
- **#43 + #72 — media socket QoS + buffer growth.** New `punktfunk_core::transport::qos`:
`grow_socket_buffers` (the native plane's `SO_SNDBUF`/`SO_RCVBUF`=32 MB growth, factored out so the
GameStream sockets reuse it — kills host-side ENOBUFS at high bitrate) and `set_media_qos`
(opt-in `PUNKTFUNK_DSCP=1`: DSCP CS5 video / CS6 audio via `IP_TOS` + Linux `SO_PRIORITY` 5/6,
Apollo's scheme). Wired into the native `UdpTransport::connect`/`connect_via_punch` and the
GameStream video/audio sockets. Cross-platform; Linux readback test asserts `tos_v4()==0xA0` +
`priority()==5`. Windows note: plain `IP_TOS` is a no-op on the wire without a qWAVE policy (the
qWAVE port is the documented follow-up).
- **#8 + #45 — GameStream input injection off the ENet service thread (+ coalescing).** `on_receive`
no longer injects inline (a slow Wayland/libei/SendInput call head-blocked ENet keepalive/retransmit);
it forwards decoded keyboard/mouse to a dedicated injector thread. The native plane's hardened
`InjectorService` (lazy open + backend-change reopen + failure backoff) was **moved from punktfunk1
into `crate::inject`** so both planes share one impl, and given a `coalesce` step (#45) that sums
adjacent relative-mouse + same-axis scroll deltas while preserving button/key/abs ordering — so a
slow backend never builds a backlog of stale motion. Cross-platform; unit-tested (`coalesce`) +
full native-plane regression suite green.
**Still open / partial:** the remaining ~71 items (table rows not listed above). Highest-value next
steps from this re-verification: **#23 / #89** (Windows DS4/DualSense ViGEm target, honoring the
negotiated pad type), **#9** (actually launch the app on Windows via `CreateProcessAsUserW`), **#7 /
#18** (WASAPI default-device-change + device-invalidated recovery), **#43 / #72** (media QoS/DSCP +
GameStream `SO_SNDBUF`), **#8** (move GameStream input injection off the ENet service thread).
| # | Improvement | Area | Win | Sev | Eff | ✓V |
|---|---|---|---|---|---|---|
@@ -1609,10 +1672,10 @@ adversarial-verify pass. *Area* is the investigation that surfaced it.
| 2 | Detect resolution/format change on the acquire hot path, not only during rebuild | win:capture-dxgi-dd | Y | high | small | |
| 3 | Per-frame IsCurrent() check to catch HDR/GPU/mode changes | win:capture-wgc | Y | high | small | |
| 4 | ✅ **DONE** — Batched/GSO send for the GameStream video plane on Windows | cmp:protocol-streaming | Y | high | medium | ✓ |
| 5 | Gate the GameStream HTTPS plane on the paired-cert allow-list | cmp:gamestream-http-pairing | Y | high | medium | |
| 6 | Query NVENC encode capabilities before init and degrade gracefully | cmp:video-encode | Y | high | medium | |
| 5 | ✅ **DONE** — Gate the GameStream HTTPS plane on the paired-cert allow-list | cmp:gamestream-http-pairing | Y | high | medium | |
| 6 | ✅ **DONE** (CI-pending) — Query NVENC encode capabilities before init and degrade gracefully | cmp:video-encode | Y | high | medium | |
| 7 | Detect default-render-device changes and reinit WASAPI capture | cmp:audio | Y | high | medium | |
| 8 | Move GameStream input injection off the ENet service thread | cmp:input | Y | high | medium | |
| 8 | ✅ **DONE** — Move GameStream input injection off the ENet service thread | cmp:input | Y | high | medium | |
| 9 | Actually launch the app/game on Windows (CreateProcessAsUserW into the user session) | cmp:process-launch | Y | high | medium | |
| 10 | Native system tray with state-driven icon + notifications | cmp:config-management | Y | high | medium | |
| 11 | Treat S_OK-with-no-change frames as timeouts via DXGI update flags | win:capture-dxgi-dd | Y | high | medium | |
@@ -1623,14 +1686,14 @@ adversarial-verify pass. *Area* is the investigation that surfaced it.
| 16 | Add SET_RENDER_ADAPTER (IOCTL 0x802) to bind the IDD render GPU to the capture/encode GPU | win:virtual-display-sudovda | Y | high | medium | |
| 17 | Add streaming_will_start/stop session-level latency tuning on Windows | win:critic | Y | high | medium | |
| 18 | Recover WASAPI loopback from default-device change and AUDCLNT_E_DEVICE_INVALIDATED | win:critic | Y | high | medium | |
| 19 | Implement true reference-frame invalidation with a multi-ref DPB instead of always-full-IDR | cmp:video-encode | Y | high | large | |
| 19 | ✅ **DONE** (CI-pending) — Implement true reference-frame invalidation with a multi-ref DPB instead of always-full-IDR | cmp:video-encode | Y | high | large | |
| 20 | In-binary Windows service install + interactive-session launch | cmp:config-management | Y | high | large | |
| 21 | ⊘ **ALREADY-HANDLED** — Composite the moved cursor onto a clean copy even when DDA returns no new desktop frame | win:cursor-compositing | Y | high | large | |
| 22 | Add real reference-frame invalidation (RFI) instead of always forcing IDR | win:nvenc-d3d11 | Y | high | large | |
| 22 | ✅ **DONE** (CI-pending) — Add real reference-frame invalidation (RFI) instead of always forcing IDR | win:nvenc-d3d11 | Y | high | large | |
| 23 | Add a DS4 (DualShock4) ViGEm target on Windows with type auto-selection, motion, touchpad, battery and timestamp pump | win:input-sendinput-vigem | Y | high | large | |
| 24 | Replace the PsExec scheduled-task launch with a real Windows service that relaunches the host on session change | win:system-secure-desktop | Y | high | large | |
| 25 | Elevate capture/encode/send thread priority on the host hot path | cmp:protocol-streaming | Y | medium | small | ✓ |
| 26 | Atomic temp+rename persistence for the GameStream paired store | cmp:gamestream-http-pairing | Y | medium | small | |
| 26 | ✅ **DONE** — Atomic temp+rename persistence for the GameStream paired store | cmp:gamestream-http-pairing | Y | medium | small | |
| 27 | Always emit explicit SDR color VUI (primaries/transfer/matrix/range), not just HDR | cmp:video-encode | Y | medium | small | |
| 28 | Set repeatSPSPPS=1 and wire slicesPerFrame for the Windows NVENC config | cmp:video-encode | Y | medium | small | |
| 29 | Raise the WASAPI capture thread to MMCSS Pro Audio priority | cmp:audio | Y | medium | small | |
@@ -1647,15 +1710,15 @@ adversarial-verify pass. *Area* is the investigation that surfaced it.
| 40 | Gate on SudoVDA protocol-version compatibility instead of only logging it | win:virtual-display-sudovda | Y | medium | small | |
| 41 | Retry device open with exponential backoff | win:virtual-display-sudovda | Y | medium | small | |
| 42 | Add per-frame IDXGIFactory::IsCurrent reinit detection and switch the host clock to GetSystemTimePreciseAsFileTime | win:system-secure-desktop | Y | medium | small | |
| 43 | Socket QoS / DSCP marking on the media sockets | cmp:protocol-streaming | Y | medium | medium | ✓ |
| 43 | ✅ **DONE** — Socket QoS / DSCP marking on the media sockets | cmp:protocol-streaming | Y | medium | medium | ✓ |
| 44 | Plumb HDR10 static metadata (mastering display + MaxCLL/MaxFALL) | cmp:video-encode | Y | medium | medium | |
| 45 | Coalesce relative-mouse/scroll/controller spam before injection | cmp:input | Y | medium | medium | |
| 45 | ✅ **DONE** (mouse/scroll) — Coalesce relative-mouse/scroll/controller spam before injection | cmp:input | Y | medium | medium | |
| 46 | Display-config apply/revert with a retry scheduler and guaranteed revert on disconnect | cmp:process-launch | Y | medium | medium | |
| 47 | Harden GPU scheduling priority + SetMaximumFrameLatency + NVIDIA-HAGS NVENC-realtime avoidance | win:capture-dxgi-dd | Y | medium | medium | |
| 48 | Use SystemRelativeTime (QPC) as the frame timestamp | win:capture-wgc | Y | medium | medium | |
| 49 | Stop baking the cursor destructively into the repeated gpu_copy texture | win:cursor-compositing | Y | medium | medium | |
| 50 | Gate HDR on (client requested HDR) AND (desktop is actually HDR), and signal the result in Welcome | win:hdr-colorspace | Y | medium | medium | |
| 51 | Query nvEncGetEncodeCaps and gate config on real GPU capabilities | win:nvenc-d3d11 | Y | medium | medium | |
| 51 | ✅ **DONE** (CI-pending) — Query nvEncGetEncodeCaps and gate config on real GPU capabilities | win:nvenc-d3d11 | Y | medium | medium | |
| 52 | Use async encode with a Win32 completion event + timeout | win:nvenc-d3d11 | Y | medium | medium | |
| 53 | Minimize NvEnc API/struct versions per codec for older-driver compatibility | win:nvenc-d3d11 | Y | medium | medium | |
| 54 | Use a canonical US-English VK→scancode table for normalized keys, and fall back to VK when no scancode maps | win:input-sendinput-vigem | Y | medium | medium | |
@@ -1676,7 +1739,7 @@ adversarial-verify pass. *Area* is the investigation that surfaced it.
| 69 | Convert to P010 in a D3D11 shader and feed NVENC YUV instead of ABGR10 RGB | win:hdr-colorspace | Y | medium | large | |
| 70 | Add an NvAPI driver-settings manager (PREFERRED_PSTATE_MAX + OGL_CPL_PREFER_DXPRESENT) with a crash-safe undo file | win:system-secure-desktop | Y | medium | large | |
| 71 | Install/select a virtual audio sink so a headless Windows host has audio with no physical device | win:critic | Y | medium | large | |
| 72 | Grow SO_SNDBUF on the GameStream video/audio sockets | cmp:protocol-streaming | Y | low | small | |
| 72 | ✅ **DONE** — Grow SO_SNDBUF on the GameStream video/audio sockets | cmp:protocol-streaming | Y | low | small | |
| 73 | Decode NVENCSTATUS into readable names and detect InvalidParam structurally | cmp:video-encode | Y | low | small | |
| 74 | Surface WASAPI data-discontinuity as a glitch diagnostic | cmp:audio | Y | low | small | |
| 75 | Inject per-app launch env (client res/fps/HDR/audio + status) for launch scripts | cmp:process-launch | Y | low | small | |
@@ -1696,7 +1759,7 @@ adversarial-verify pass. *Area* is the investigation that surfaced it.
| 89 | Support DualSense/DS4 ViGEm target + feedback on Windows, honoring negotiated pad type | win:critic | Y | low | large | |
| 90 | Bitrate-derived rate-control pacing (vs frame-interval-only) | cmp:protocol-streaming | | medium | medium | ✓ |
| 91 | Named, permissioned paired-device records for the GameStream store | cmp:gamestream-http-pairing | | medium | medium | |
| 92 | Actually reject unpaired GameStream client certs (close the unpair gap) | cmp:config-management | | medium | medium | |
| 92 | ✅ **DONE** — Actually reject unpaired GameStream client certs (close the unpair gap) | cmp:config-management | | medium | medium | |
| 93 | Persisted host config + read/write config API endpoint | cmp:config-management | | medium | large | |
| 94 | Consume the GameStream client loss-stats report | cmp:protocol-streaming | | low | small | ✓ |
| 95 | Tolerate not-yet-valid/expired client certs during verification | cmp:gamestream-http-pairing | | low | small | |