The Moonlight plane stops paying for protection it doesn't need, starts listening to the client, and encrypts its video #414

Merged

enricobuehler merged 7 commits from worktree-gamestream-wp1 into main 2026-08-27 15:05:16 +00:00

Conversation 0 Commits 7 Files Changed 15

+1798 -350

enricobuehler commented 2026-08-27 14:50:28 +00:00

Owner

Copy Link

Copy Source

Seven commits taking the Moonlight-compat plane from "works" to "competitive", each gated on the Linux container and Windows .133 before the next was started. The investigation that produced them is in the planning repo as design/gamestream-competitive-program.md (uncommitted there — ask if you want it landed).

The finding the whole programme rests on: the gap was integration, not architecture. Arrival-driven capture, microburst pacing, buffer pools, adaptive FEC and encoder-rate derivation all already existed in shared code — the GameStream plane simply never called them, because it was written before the native plane grew them and nobody went back.

What changes for a Moonlight user

• Latency drops by roughly 15–19 ms per frame at 60 fps, from two parameter choices.
• The host reacts to their link — packet loss now reaches it, and it answers with parity and bitrate.
• The bitrate they set is honoured. It used to be exceeded by ~23 %.
• Resume and Quit exist in Moonlight's UI, and resume actually re-attaches.
• Wake-on-LAN works from the client.
• Video is encrypted, negotiated per session — verified against a real client (below).

The commits

ab5e642f	Latency + wire budget. The encode loop slept out its tick and then sampled (≈8 ms of stale frame, every frame); the sender passed burst_bytes: None, so a 3-packet P-frame spread over ¾ of a frame interval exactly like a 4K keyframe (≈11 ms tail). Both now use the native plane's shape. Packetizer buffers are pooled (the whole-AU copy and ~3300 allocations per 4K IDR are gone), the ENet control loop wakes on packets instead of spinning at 2 ms, and the encoder rate is derived under the client's bitrate so FEC and framing fit inside it instead of on top. Plus the measurement this needed: the wire's host-latency field is stamped (it was hardcoded 0, so Moonlight's overlay always read 0), and a client-shaped reassembler with loss sweeps locks the wire layout.
7df36a1b	The host listens. Moonlight's periodic 0x0201 loss report used to fall through the control decoder and be dropped. It now drives adaptive FEC (climbs fast under loss, decays slow) and, under sustained loss, eases the wire budget down with gradual recovery. Every step re-derives the encoder rate, so the wire never exceeds what the client asked for. PUNKTFUNK_GS_ADAPT=0 pins it.
31eb91e2	Session truth. serverinfo reported state=FREE/currentgame=0 unconditionally, so no client ever showed Resume or Quit. Now truthful — to the session owner only, because showing it to others would route their taps into owner-only /resume and lose the reject/join/steal admission they get via /launch. /resume read no query parameters at all, so it ran on stale crypto keys; it now re-keys, re-binds the source IP, and restarts the media planes with a proper stop-and-wait handoff (this also closes a pre-existing race where a superseded thread's teardown could stomp its successor).
20310665	Wake-on-LAN. <mac> was the constant 01:02:03:04:05:06, and Moonlight persists that field as its wake target — so every client-side wake was a silent no-op. Now the real routed-NIC MAC, cached on first success so a cold-booted host retries instead of latching zeros.
24783201	Media. The capture format now comes from the shared SessionPlan (a gamescope session gets nv12_native, skipping a per-frame colour conversion the native plane already skips); the encoder receives the source's real HDR grade for in-band SEI (this plane never called set_hdr_meta, so HDR sessions shipped no grade at all); and stereo audio finally gets FEC — the old channels > 2 gate left the most common configuration with no loss protection.
47d9a7d2 + ac14b655	Video encryption (SS_ENC_VIDEO), offered but never required. See below.

Encryption, and why it is on by default

Per-shard AES-128-GCM under the launch key, shipped as [iv][frameNumber][tag] ‖ ciphertext. Two design points carried it:

• FEC first, then encrypt per shard. The client decrypts what it received and then runs Reed–Solomon recovery over those plaintexts — parity computed over ciphertext would recover nothing. This has its own test.
• The nonce counter is process-global and never resets. A session-scoped counter would repeat the moment a keyless /resume (which 31eb91e2 defines as keeping the current keys) started a fresh packetizer on the same key. That is the one catastrophic GCM failure, so it is excluded structurally rather than by argument.

It shipped dark in 47d9a7d2 on purpose — it is the compat plane's video hot path and a wire mistake there is a black screen. ac14b655 flips it after an on-glass run against .173 (Win11, RTX 4090) from Moonlight on macOS:

Leg	Result
encryption off	streams normally — no regression
supported	client opts in by itself, decodes, 0 errors
required	negotiated + decoded in hardware, 0 errors
required + 5 % injected loss	27 s, 8 keyframes decoded, zero keyframe re-requests

The last row is the one that mattered: at 5 % loss the client never once asked for a keyframe, so FEC recovered every dropped sealed shard — the ordering property, proven on hardware.

Two confirmations of the wire format fell out of the run on their own: the client negotiated packet_size 1360 where the plaintext leg used 1392, having subtracted the 32-byte header exactly as the reference says (so the datagram is 1408 either way and still fits the MTU it sized for); and FFmpeg reported nal_unit_type: 19 (IDR_W_RADL) → Decoded frame with POC 0/0 on a Main 10 VideoToolbox path — a real decoder reading real HEVC out of the decrypted, reassembled payload.

Leg 2 also overturned my own reasoning. I had argued clients would likely only enable video encryption on WAN/relay links — that is why the require lever exists — and it opted in on a LAN unprompted. Worth stating plainly: default-on delivers encryption in practice, and that is measured, not assumed.

PUNKTFUNK_GS_ENCRYPT=0 is the escape hatch. SS_ENC_CONTROL_V2 and SS_ENC_AUDIO are deliberately not offered: control-v2 also re-frames RTSP itself, and the audio-GCM layout is not in the sanctioned wire reference.

Verification

Every commit was gated before the next began: Linux container fmt + clippy --all-targets -D warnings (non-vacuity checked) + tests, and Windows .133 clippy on the full nvenc,amf-qsv,qsv matrix + tests. Final state: Linux 14/14 pacing + 90/90 gamestream; Windows 14/14 + 89/89; check-docs-drift.sh and check-docs-links.sh clean. Test count rose 81 → 90 on the gamestream module.

.173 was restored afterwards to its original binary and host.env (verified by size and timestamp), service running, all three GameStream ports back up, build tree removed.

Worth a reviewer's attention

• The docs were wrong in our favour and are now corrected. moonlight.md claimed the GameStream path "doesn't use the native protocol's FEC/encryption extensions" — Moonlight-compatible FEC had shipped months ago. The "control encryption is weaker" caveat is also softened to what is still true (pairing is plain HTTP; the control channel uses the older scheme), now that video and audio are both sealed.
• PUNKTFUNK_IDD_ADAPTIVE was pruned from the undocumented-env baseline. It is documented in configuration.md but was never pruned, so that ratchet is red on main independently of this branch — check-docs-drift.sh refuses to pass otherwise.
• One behavioural asymmetry is deliberate: currentgame is owner-scoped and fails closed on unknown fingerprints, unlike peer_may_control_session, which fails open so a client can never be locked out of controlling its own session. An advertisement and a gate want opposite defaults.

Not in this PR

Trigger rumble, gamepad motion/battery/LED, the 4:4:4 advertisement bits and honouring encoderCscMode are capture-blocked: the sanctioned wire reference carries no control-input detail (its research agent failed mid-run, logged inside the JSON) and names but does not value the SCM_*_444 flags. These need packet captures or a sanctioned protocol-facts pass, not more reading. The Windows GameStream-teardown abort (seen twice on .173, never filed) also remains open and needs its own reproduction session.

Seven commits taking the Moonlight-compat plane from "works" to "competitive", each gated on the Linux container **and** Windows `.133` before the next was started. The investigation that produced them is in the planning repo as `design/gamestream-competitive-program.md` (uncommitted there — ask if you want it landed). The finding the whole programme rests on: **the gap was integration, not architecture.** Arrival-driven capture, microburst pacing, buffer pools, adaptive FEC and encoder-rate derivation all already existed in shared code — the GameStream plane simply never called them, because it was written before the native plane grew them and nobody went back. ## What changes for a Moonlight user * **Latency drops by roughly 15–19 ms per frame at 60 fps**, from two parameter choices. * **The host reacts to their link** — packet loss now reaches it, and it answers with parity and bitrate. * **The bitrate they set is honoured.** It used to be exceeded by ~23 %. * **Resume and Quit exist in Moonlight's UI**, and resume actually re-attaches. * **Wake-on-LAN works** from the client. * **Video is encrypted**, negotiated per session — verified against a real client (below). ## The commits | | | |---|---| | `ab5e642f` | **Latency + wire budget.** The encode loop slept out its tick and then *sampled* (≈8 ms of stale frame, every frame); the sender passed `burst_bytes: None`, so a 3-packet P-frame spread over ¾ of a frame interval exactly like a 4K keyframe (≈11 ms tail). Both now use the native plane's shape. Packetizer buffers are pooled (the whole-AU copy and ~3300 allocations per 4K IDR are gone), the ENet control loop wakes on packets instead of spinning at 2 ms, and the encoder rate is derived *under* the client's bitrate so FEC and framing fit inside it instead of on top. Plus the measurement this needed: the wire's host-latency field is stamped (it was hardcoded 0, so Moonlight's overlay always read 0), and a client-shaped reassembler with loss sweeps locks the wire layout. | | `7df36a1b` | **The host listens.** Moonlight's periodic `0x0201` loss report used to fall through the control decoder and be dropped. It now drives adaptive FEC (climbs fast under loss, decays slow) and, under *sustained* loss, eases the wire budget down with gradual recovery. Every step re-derives the encoder rate, so the wire never exceeds what the client asked for. `PUNKTFUNK_GS_ADAPT=0` pins it. | | `31eb91e2` | **Session truth.** `serverinfo` reported `state=FREE`/`currentgame=0` unconditionally, so no client ever showed Resume or Quit. Now truthful — **to the session owner only**, because showing it to others would route their taps into owner-only `/resume` and lose the reject/join/steal admission they get via `/launch`. `/resume` read *no query parameters at all*, so it ran on stale crypto keys; it now re-keys, re-binds the source IP, and restarts the media planes with a proper stop-and-wait handoff (this also closes a pre-existing race where a superseded thread's teardown could stomp its successor). | | `20310665` | **Wake-on-LAN.** `<mac>` was the constant `01:02:03:04:05:06`, and Moonlight persists that field as its wake target — so every client-side wake was a silent no-op. Now the real routed-NIC MAC, cached on first success so a cold-booted host retries instead of latching zeros. | | `24783201` | **Media.** The capture format now comes from the shared `SessionPlan` (a gamescope session gets `nv12_native`, skipping a per-frame colour conversion the native plane already skips); the encoder receives the source's real HDR grade for in-band SEI (this plane never called `set_hdr_meta`, so HDR sessions shipped no grade at all); and stereo audio finally gets FEC — the old `channels > 2` gate left the *most common* configuration with no loss protection. | | `47d9a7d2` + `ac14b655` | **Video encryption** (`SS_ENC_VIDEO`), offered but never required. See below. | ## Encryption, and why it is on by default Per-shard AES-128-GCM under the launch key, shipped as `[iv][frameNumber][tag] ‖ ciphertext`. Two design points carried it: * **FEC first, then encrypt per shard.** The client decrypts what it received and *then* runs Reed–Solomon recovery over those plaintexts — parity computed over ciphertext would recover nothing. This has its own test. * **The nonce counter is process-global and never resets.** A session-scoped counter would repeat the moment a *keyless* `/resume` (which `31eb91e2` defines as keeping the current keys) started a fresh packetizer on the same key. That is the one catastrophic GCM failure, so it is excluded structurally rather than by argument. It shipped dark in `47d9a7d2` on purpose — it is the compat plane's video hot path and a wire mistake there is a black screen. `ac14b655` flips it after an **on-glass run against `.173` (Win11, RTX 4090) from Moonlight on macOS**: | Leg | Result | |---|---| | encryption off | streams normally — no regression | | supported | **client opts in by itself**, decodes, 0 errors | | required | negotiated + decoded in hardware, 0 errors | | required + 5 % injected loss | 27 s, 8 keyframes decoded, **zero** keyframe re-requests | The last row is the one that mattered: at 5 % loss the client never once asked for a keyframe, so FEC recovered every dropped *sealed* shard — the ordering property, proven on hardware. Two confirmations of the wire format fell out of the run on their own: the client negotiated `packet_size` **1360** where the plaintext leg used **1392**, having subtracted the 32-byte header exactly as the reference says (so the datagram is 1408 either way and still fits the MTU it sized for); and FFmpeg reported `nal_unit_type: 19 (IDR_W_RADL)` → `Decoded frame with POC 0/0` on a Main 10 VideoToolbox path — a real decoder reading real HEVC out of the decrypted, reassembled payload. **Leg 2 also overturned my own reasoning.** I had argued clients would likely only enable video encryption on WAN/relay links — that is why the `require` lever exists — and it opted in on a LAN unprompted. Worth stating plainly: default-on delivers encryption in practice, and that is measured, not assumed. `PUNKTFUNK_GS_ENCRYPT=0` is the escape hatch. `SS_ENC_CONTROL_V2` and `SS_ENC_AUDIO` are deliberately **not** offered: control-v2 also re-frames RTSP itself, and the audio-GCM layout is not in the sanctioned wire reference. ## Verification Every commit was gated before the next began: Linux container `fmt` + `clippy --all-targets -D warnings` (non-vacuity checked) + tests, and Windows `.133` clippy on the full `nvenc,amf-qsv,qsv` matrix + tests. Final state: **Linux 14/14 pacing + 90/90 gamestream; Windows 14/14 + 89/89**; `check-docs-drift.sh` and `check-docs-links.sh` clean. Test count rose 81 → 90 on the gamestream module. `.173` was restored afterwards to its original binary and `host.env` (verified by size and timestamp), service running, all three GameStream ports back up, build tree removed. ## Worth a reviewer's attention * **The docs were wrong in our favour and are now corrected.** `moonlight.md` claimed the GameStream path "doesn't use the native protocol's FEC/encryption extensions" — Moonlight-compatible FEC had shipped months ago. The "control encryption is weaker" caveat is also softened to what is still true (pairing is plain HTTP; the control channel uses the older scheme), now that video and audio are both sealed. * **`PUNKTFUNK_IDD_ADAPTIVE` was pruned from the undocumented-env baseline.** It is documented in `configuration.md` but was never pruned, so that ratchet is **red on main independently of this branch** — `check-docs-drift.sh` refuses to pass otherwise. * **One behavioural asymmetry is deliberate**: `currentgame` is owner-scoped and fails *closed* on unknown fingerprints, unlike `peer_may_control_session`, which fails open so a client can never be locked out of controlling its own session. An advertisement and a gate want opposite defaults. ## Not in this PR Trigger rumble, gamepad motion/battery/LED, the 4:4:4 advertisement bits and honouring `encoderCscMode` are **capture-blocked**: the sanctioned wire reference carries no control-input detail (its research agent failed mid-run, logged inside the JSON) and names but does not value the `SCM_*_444` flags. These need packet captures or a sanctioned protocol-facts pass, not more reading. The Windows GameStream-teardown abort (seen twice on `.173`, never filed) also remains open and needs its own reproduction session.

enricobuehler added 7 commits 2026-08-27 14:50:31 +00:00

perf(gamestream): the compat plane stops paying for congestion protection it doesn't need ... ab5e642f84

GS competitive program WP0/WP1/WP2.1 (design/gamestream-competitive-program.md in the
planning repo). Four latency/overshoot gaps between the Moonlight-compat plane and the
native plane, all closed by wiring GS into machinery the native plane already had:

- WP1.1 arrival-driven capture: the GS encode loop slept out its whole tick and then
  SAMPLED, holding a frame that arrived just after the previous sample for up to a full
  interval (~half on average, ~8 ms at 60 fps). It now sleeps to a 0.9-interval floor and
  wakes on the capture's actual arrival, exactly the native T1.1 shape — with the same
  credit bucket pinning the long-run average at the negotiated rate, so a mirrored panel
  running faster than the session cannot overdrive the wire. The bucket
  (CaptureCredit, ex the native loop's private PaceBudget) and the PUNKTFUNK_FRAME_DRIVEN
  gate move to send_pacing, shared by both planes.

- WP1.2 microburst pacing: the GS sender passed burst_bytes: None into the shared pacer,
  so EVERY frame — a 3-packet P-frame included — spread across 0.75 of the frame interval
  (an ~11 ms tail at 60 fps), and small frames still slept out the full budget. It now
  runs the native parameterization: an auto-sized burst leaves immediately (a normal
  frame goes out whole, unpaced) and only an oversized frame's overflow spreads, across
  the time it needs at ~3x the stream rate, bounded to ~2 intervals. The chunking stays
  BOUNDED (<=12 steps) — on that non-RT thread the per-step sleep overshoot must stay
  independent of bitrate.

- WP1.3 pooled packetization: the packetizer copied the whole AU into a staging Vec and
  allocated one Vec per shard (~3300 for a 4 MB IDR) plus unpooled parity, every frame.
  Datagram buffers now cycle sender -> packetizer -> wire through a recycle channel,
  parity goes through the pooled encode_into, and each payload byte is copied exactly
  once, straight from the encoder's buffer into its datagram. A poisoned-pool equivalence
  test pins the pooled path byte-identical to a fresh packetizer.

- WP1.5 wake-on-packet control: the ENet service loop was a 2 ms spin-sleep, so every
  client input/IDR-request datagram waited out the rest of the tick. The control socket
  now blocks with a 2 ms read timeout (same idle cadence and CPU); an arriving datagram
  wakes it immediately. rusty_enet's receive loop returns on the first produced event
  (verified in its transpiled enet_protocol_receive_incoming_commands), so the block only
  ever sits where the sleep used to.

- WP2.1 budget honesty: the encoder was handed the full negotiated bitrate and then 20%
  FEC parity plus per-shard framing went ON TOP — every session carried ~1.23x what the
  client configured, and on the constrained links where the setting matters the overshoot
  was the failure. The encoder rate is now derived UNDER the client's number
  (gs_encoder_bps, the compat twin of the native encoder_kbps_for_budget from ABR
  overhaul Phase 4), so the wire fits the budget the Moonlight user actually set.

- WP0 measurement: the wire's frame_processing_latency field (1/10 ms, the Sunshine
  extension Moonlight's overlay shows as 'Host processing latency') is now stamped per
  frame instead of hardcoded 0; the 1 Hz stats sample grows a send_spread stage measured
  on the sender thread; and the video tests gain a GS-wire loopback harness — a
  client-shaped reassembler running Cauchy RS recovery over lossy datagram subsets —
  locking layout + recovery against every later packetizer change.

Also: the mirror source's capture-format 'gpu' predicate now matches the virtual source's
(resolved_backend_is_gpu, not the zerocopy env knob).

Gates: Linux container (punktfunk-rust-ci) fmt --all --check clean, clippy --all-targets
-D warnings clean and non-vacuous, send_pacing 14/14, gamestream 81/81 (incl. the new
harness + equivalence + schedule pins). Windows .133 clippy/tests with nvenc,amf-qsv,qsv
run in parallel with this commit; macOS spot-check shows only the known cfg(linux) stub
errors. On-glass A/B (WP0.3 baseline matrix) still owed.

feat(gamestream): the host finally listens when the client reports loss ... 7df36a1b3d

GS competitive program WP2.2-2.4 (design/gamestream-competitive-program.md). The
Moonlight plane's only in-stream quality signal — the periodic 0x0201 loss-stats
report — used to fall through the control decoder and be silently dropped; the host
was blind to client-observed loss and its only recovery was serving IDR round-trips.
No Sunshine-class host does more than log this signal. Now:

- WP2.2 the control stream decodes 0x0201 (LE i32s: loss count, window ms,
  last-good frame — the verified IDX_LOSS_STATS reading, apollo-comparison #94) into
  cumulative per-session counters on AppState (GsLossStats), read as window deltas.

- WP2.3 adaptive FEC: the parity percent climbs fast under loss (+max(5, pct/2) per
  lossy window, cap 50) and decays slowly on clean windows (-5 per 8, floored at the
  configured base). Applied live through the packetizer between frames — safe because
  block geometry is derived per frame and the client reads each block's parity count
  from the per-packet fecInfo wire percent.

- WP2.4 bitrate de-rating: SUSTAINED loss (2+ consecutive lossy windows) de-rates the
  wire budget x0.85 per window, floored at max(1/4 of the negotiated rate, 5 Mbps);
  clean windows climb it back at 1/20 of the negotiated rate per 4 — division of
  labour as on the native plane: parity answers random/burst loss, fewer bits answer
  congestion. The client's negotiated number stays the hard ceiling.

- The WP2.1 invariant is kept live: every step re-derives the ENCODER rate under
  (budget, percent) and retargets it in place (Encoder::reconfigure_bitrate); the send
  pacer's rate follows through an atomic. An encoder that refuses the in-place
  retarget (software paths) turns adaptation OFF for the session — raising parity
  over a frozen encoder rate would push the wire back over budget. The stats sample
  reports the LIVE budget, so the console shows what the host actually targets.

PUNKTFUNK_GS_ADAPT=0 pins both levers at their configured values (the A/B lever).
Constants are deliberately conservative first values — the WP0.3 netem matrix tunes
them. Also: the stale module/status doc headers the completeness inventory flagged.

Gates: Linux container fmt + clippy --all-targets -D warnings (non-vacuous) +
send_pacing 14/14 + gamestream 83/83 (incl. the two new adaptation state-machine
tests); Windows .133 clippy with nvenc,amf-qsv,qsv clean + send_pacing 14/14 +
gamestream 82/82 (amf-qsv,qsv — the box has no NVENC import lib for test linking).
On-glass netem verification (WP0.3) owed.

feat(gamestream): the host tells its owner the truth, and a resume actually resumes ... 31eb91e2b1

GS competitive program WP3 (design/gamestream-competitive-program.md). Three fixes
that together make Moonlight's Resume/Quit UX exist against this host for the first
time — serverinfo hard-coded state=FREE/currentgame=0, so no stock client ever showed
Resume or Quit, and /resume read no query params at all, so even a reached resume ran
on stale crypto keys.

- serverinfo now reports the live session's appid + SUNSHINE_SERVER_BUSY — to the
  session OWNER's pinned cert ONLY. Owner-scoped on purpose (owner_current_game, pure
  + tested): a non-owner shown the truth would route same-app taps into the owner-only
  /resume and /cancel and lose the reject/join/steal admission it gets via /launch
  today; and a busy signal over plain HTTP would leak what's running to the LAN.
  Advertisement fails CLOSED on unknown fingerprints (unlike the control gate, which
  deliberately fails open).

- /resume parses rikey/rikeyid and RE-KEYS the live session (a resuming Moonlight
  mints fresh keys; the control GCM and audio CBC derive from them — the old keys made
  every post-resume control packet undecryptable). Malformed rikey refuses the resume;
  keyless keeps the current keys. The media/RTSP source-IP binds re-anchor to the
  address the client resumes from, so a device that moved networks can come back.

- /resume RESTARTS the media planes for the new connection. A PLAY that found
  'streaming' still true took its already-running branch: the old threads kept
  streaming at the vanished endpoint and the resumed client got no media. The handler
  clears the run flags and WAITS (bounded, 2 s) for the old threads' FULL exit — each
  media thread now bumps AppState::media_exited as its last act, after its teardown
  (capturer re-pool, lease guard, events) has run — so the successor threads never
  race the predecessor over the pooled capturer and the old exit path cannot stomp
  the new session's flags. The launch surviving that wait is re-keyed; one cleared by
  the old threads' client-unreachable teardown falls back to a clean /launch. The
  ENet control plane already handles the fast-reconnect half (a new Connect replaces
  the tracked peer; the old peer's late timeout is gated off it).

Gates: Linux container fmt + clippy --all-targets -D warnings (non-vacuous) +
send_pacing 14/14 + gamestream 86/86 (new: owner-scoped currentgame matrix, resume
re-key round-trip incl. malformed/keyless arms, busy-state XML pin); Windows .133
clippy nvenc,amf-qsv,qsv clean + 14/14 + 85/85. On-glass resume with a stock
Moonlight client owed (WP0.3 pass).

fix(gamestream): the MAC Moonlight wakes is finally the host's own ... 2031066539

GS competitive program WP6.1. serverinfo's <mac> was a fake constant
(01:02:03:04:05:06) over HTTPS — and Moonlight PERSISTS that field as the target for
its Wake-on-LAN, so every client-side wake against this host was a silent no-op. It
now carries the real routed-NIC MAC via the same crate::wol::wake_macs the native
plane's mDNS advert uses (routed NIC first, lowercase colon-hex — the wire reference's
documented shape: real MAC on HTTPS, zeros on plain HTTP).

Cached on first SUCCESS only: /serverinfo is polled, so per-poll NIC enumeration is
waste — but a cold-booted host may not have a routable address yet (the #366 boot-race
lesson), so a failed read retries on the next poll instead of latching zeros forever.
Plain HTTP keeps hiding the MAC.

Gates: Linux container fmt + clippy -D warnings + send_pacing 14/14 + gamestream
87/87 (new pin: never the fake MAC, hidden over HTTP); Windows .133 clippy
nvenc,amf-qsv,qsv clean + 14/14 + 86/86.

feat(gamestream): the compat plane joins the shared session plan, ships the real HDR grade, and stops streaming unprotected stereo ... 247832014a

GS competitive program WP5 (the media items implementable without new wire research —
4:4:4 and encoderCscMode stay follow-ups: the 4:4:4 SCM extension bit values are not in
the sanctioned wire reference, and CscMode honor is an encoder-wide colorspace plumb).

- WP5.1 the capture format comes from the shared SessionPlan (gs_session_plan: 4:2:0,
  depth 10 only with HDR, no cursor-forward — this plane has no client cursor channel)
  instead of the hand-hardcoded OutputFormat::resolve(hdr, gpu). One resolver for both
  planes, and the visible win: a gamescope GameStream session (cursor blend off there)
  now resolves nv12_native, so the producer's NV12 feeds Vulkan Video directly and the
  per-frame RGB→NV12 CSC the native plane already skips is skipped here too. Also the
  structural prerequisite for 4:4:4 when its advertisement bits are researched.

- WP5.4 the encoder gets the source's REAL HDR grade every frame (set_hdr_meta from the
  capturer, exactly the native loop's shape): an HDR backend embeds it as in-band
  mastering/CLL SEI on keyframes — the channel a stock Moonlight decoder tone-maps
  from. This plane never called it, so an HDR GameStream session shipped no grade at
  all; the 0x010e control cue only flips the display mode and carries generic fallback
  metadata.

- WP5.5 RS(4,2) audio FEC on every layout — stereo included. The parity math is
  layout-agnostic (shards are opaque encrypted packets) and a stock client runs the
  same fixed RS(4,2) recovery regardless of channel count; the old channels>2 gate was
  bring-up caution that left the MOST COMMON configuration with zero audio loss
  protection.

Gates: Linux container fmt + clippy --all-targets -D warnings + send_pacing 14/14 +
gamestream 87/87; Windows .133 clippy nvenc,amf-qsv,qsv clean + 14/14 + 86/86.
On-glass owed with the WP0.3 pass: gamescope NV12 path on .41, HDR SEI in a client
bitstream dump, stereo FEC under netem loss.

feat(gamestream): video encryption exists on the wire, behind a knob until glass confirms it ... 47d9a7d2fa

GS competitive program WP7 — the largest remaining wire divergence from a Sunshine-class
host, and the last item the docs' "weaker than the native protocol" caveat rested on.

- RTSP DESCRIBE advertises SS_ENC_VIDEO (0x02) as encryptionSupported and NEVER as
  encryptionRequested: requiring encryption would refuse every client that doesn't do
  it. ANNOUNCE's x-ss-general.encryptionEnabled echo decides the session, and is
  honored only for a bit the host actually offered.

- Each finished datagram is AES-128-GCM-sealed under the /launch rikey and shipped as
  [iv 12][frameNumber u32 LE][tag 16] || ciphertext(blocksize) — the ENC_VIDEO_HEADER
  layout, whose 32 bytes sit OUTSIDE the FEC blocksize (the client already subtracted
  them from the packetSize it negotiated, so the datagram still fits its MTU).

- The order is FEC first, THEN encrypt per shard. That is the load-bearing property,
  and it has its own test: the client decrypts what it received and runs RS recovery
  over those plaintexts, so parity computed over ciphertext would recover nothing.
  Sealing is in-place into a buffer that reserved the prefix, so the pooled
  no-allocation path (WP1.3) survives; parity, which encode_into sizes exactly, costs
  one memcpy on the ~20 % of packets that are parity.

- The GCM nonce counter is PROCESS-global and monotonic, never reset. (key, nonce)
  reuse is the one catastrophic GCM failure, and a session-scoped counter would repeat
  the moment a KEYLESS /resume — which WP3 defines as keeping the current keys —
  started a fresh packetizer on the same rikey.

DEFAULT OFF, opt in with PUNKTFUNK_GS_ENCRYPT=1. This is the compat plane's video hot
path and a wire mistake there is a black screen for any client that opts in; I cannot
run a stock Moonlight client in this environment, so it ships dark and the WP0.3
on-glass pass flips the default. SS_ENC_CONTROL_V2 and SS_ENC_AUDIO stay unoffered:
control-v2 also re-frames RTSP itself, and the audio-GCM layout is not in the
sanctioned wire reference.

Also WP8's docs pass, now that the claims are false in the user's favour:
moonlight.md said the GameStream path "doesn't use the native protocol's
FEC/encryption extensions" — Moonlight-compatible FEC has shipped for months and the
host now adapts it to reported loss. Rewrote that (and the clients.md twin) to say
what Moonlight actually does and doesn't get, documented the bitrate as the wire
budget it became in WP2.1, and documented the three new knobs (PUNKTFUNK_FRAME_DRIVEN,
PUNKTFUNK_GS_ADAPT, PUNKTFUNK_GS_ENCRYPT) — check-docs-drift.sh gates that.

⚠ The drift gate also demanded PUNKTFUNK_IDD_ADAPTIVE be pruned from the undocumented
baseline: it is documented in configuration.md but was never pruned, so that ratchet is
red on main independently of this branch. Pruned here since the gate refuses to pass
otherwise.

Gates: Linux container fmt + clippy --all-targets -D warnings (non-vacuous) +
send_pacing 14/14 + gamestream 89/89 (new: encrypted round-trip incl. tamper + wrong
key + IV uniqueness, and RS recovery THROUGH encryption); Windows .133 clippy
nvenc,amf-qsv,qsv clean + 14/14 + 88/88; check-docs-drift.sh and check-docs-links.sh
both clean.

feat(gamestream): video encryption is on by default — a real Moonlight client negotiates and decodes it ... ac14b6554e

Review of 47d9a7d2 plus the on-glass pass it was waiting for, run 2026-08-27 against
.173 (Win11, RTX 4090) from Moonlight on macOS. Four legs, all pass:

  1. encryption off      -> streams, encrypt_video: false (no regression)
  2. =1 (supported)      -> the CLIENT OPTS IN BY ITSELF, decodes, 0 errors
  3. =require            -> negotiated, decoded in hardware, 0 errors
  4. =require + 5% loss  -> 27 s, 8 keyframes decoded, ZERO IDR re-requests

Leg 2 is why the default moves: I had argued a client might only enable video
encryption on WAN/relay links — which is why the `require` lever exists at all — and
it opted in on a LAN unprompted. Default-on therefore delivers encryption in
practice rather than being a no-op offer.

Leg 4 is the one that mattered most: at 5 % injected wire loss the client never asked
for a single keyframe, so FEC recovered every dropped SEALED shard. That is the
FEC-then-encrypt ordering proven on hardware, not just in the unit test.

Two independent confirmations of the wire format fell out of the run:
  * the client negotiated packet_size 1360 where the plaintext leg used 1392 — it
    subtracted sizeof(ENC_VIDEO_HEADER) exactly as the reference says, so the on-wire
    datagram is 1408 either way and still fits the MTU it sized for;
  * FFmpeg parsed nal_unit_type 19 (IDR_W_RADL) and reported 'Decoded frame with POC
    0/0' on a Main 10 VideoToolbox path — a real decoder reading real HEVC out of the
    decrypted, FEC-reassembled payload.

Default is now Supported; PUNKTFUNK_GS_ENCRYPT=0 is the escape hatch back to the
plaintext wire, `require` stays as the test lever that forces the negotiation.

Review fixes in the same change (found reading 47d9a7d2 back, before the test):
- A failed seal pushed the CLEARED buffer, putting a 0-byte datagram on the wire.
  Unreachable in practice (GCM only refuses absurd lengths) but the wrong failure
  mode: the shard is now dropped, which is what FEC is there for. seal_shard returns
  bool and — per clippy, correctly — takes &mut [u8] now that it never clears.
- `off` and `key` derive separately from self.enc_key; they cannot desync today, but
  if they ever did, sealing would encrypt from offset 32 INTO the shard body and
  corrupt every packet with no error anywhere. A debug_assert states the invariant.
- Stale rtsp module header (it still said plaintext-only, P1.5).

Docs follow the behaviour: moonlight.md/clients.md now say video is encrypted, and
the 'legacy control encryption is weaker' caveat is corrected — it was overstating
things now that video and audio are both sealed; what remains true is that pairing is
plain HTTP and the control channel uses the older GameStream scheme.

Gates: Linux container fmt + clippy --all-targets -D warnings (non-vacuous) +
send_pacing 14/14 + gamestream 90/90; Windows .133 clippy nvenc,amf-qsv,qsv clean +
14/14 + 89/89; check-docs-drift.sh + check-docs-links.sh clean. .173 restored to its
original binary and host.env, build tree removed.

enricobuehler merged commit d2286da955 into main 2026-08-27 15:05:16 +00:00

enricobuehler deleted branch worktree-gamestream-wp1 2026-08-27 15:05:23 +00:00

enricobuehler referenced this pull request 2026-08-27 16:38:20 +00:00

The control channel stops reusing a nonce, and the media ports stop trusting whoever knocks first #417

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

FernandoMX FlipperDan HitFrostbite bluelightspecial enricobuehler (Enrico Bühler) kszlim luxus nannku neoney

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: unom/punktfunk#414