punktfunk/design/hdr-pipeline-plan.md

# HDR pipeline — investigation & implementation plan

Goal: **true, correct HDR glass-to-glass** for punktfunk, across the host (Windows today;
Linux blocked upstream) and every client (Windows / Apple / Android / Linux).

This is an audit of the current state, the gap list, and a phased plan. It was produced from
a full read of every HDR-touching subsystem cross-checked against the HDR10 standards
(CICP/H.273 VUI, SMPTE ST.2086 mastering, CEA-861.3 MaxCLL/MaxFALL) and the
Sunshine/Apollo/Moonlight reference implementation.

> Status legend: **blocker** (HDR can't work) · **correctness** (HDR works but looks wrong) ·
> **quality** (correct-ish, missing refinement) · **ok**.

---

## TL;DR

Our HDR is **correct in isolated islands but broken end-to-end.** The pixel math and the HEVC
VUI we *do* emit are right (self-test validated, matches Apollo). What's missing is the
**metadata chain**: nothing measures, signals, transports, or applies the *static HDR metadata*
(mastering display colour volume + content light level) that tells a display how to tone-map —
so every client hardcodes generic values or infers from the bitstream, and one line
(`abi.rs:896`, `video_caps = 0`) makes the entire (correct) Apple HDR pipeline dead code.

---

## What's already correct (the islands)

| Stage | Where |
|---|---|
| Windows host HEVC **VUI** — primaries=9 (BT.2020) / transfer=16 (PQ) / matrix=9 (BT.2020-NCL) / limited range | `encode/nvenc.rs:307-316` |
| Windows host **scRGB→BT.2020 PQ** shader (×80 nits → BT.709→2020 → ST.2084 OETF, 10000-nit peak) | `capture/dxgi.rs` — self-test `<1` code error, matches Apollo |
| Windows client **P010 decode + YUV→RGB** (BT.2020-NCL, limited→full) + **R10G10B10A2 / G2084-P2020 swapchain** | `present.rs:66-77, 320-370` |
| Android client **Main10 decode + reactive DataSpace** (BT2020-PQ/HLG) | `decode.rs:210-227` |
| Apple client decode/present **code** (P010 VideoToolbox, BT.2020 PQ Metal, `itur_2100_PQ` + EDR) | correct — but never runs (blocker #2) |

## Gap list

### Blockers
1. **No color-metadata transport in the protocol** *(the keystone).* The wire carries only
   `Hello.video_caps` (10BIT/HDR bits) and `Welcome.bit_depth` (8/10) — `quic.rs:127-128`
   explicitly defers color. No primaries/transfer/matrix/range, **no ST.2086 mastering, no
   MaxCLL/MaxFALL**. ST.2086/CLL host→client is impossible by construction today.
2. **C ABI hardcodes `video_caps = 0`** (`abi.rs:896`) → Apple's complete HDR pipeline is dead
   code; no ABI embedder can request HDR. One-line root cause.
3. **H.264 and AV1 emit zero color signaling on Windows** — the `if self.hdr` VUI block in
   `nvenc.rs` only writes `hevcConfig`. Any H.264+10-bit or AV1+HDR stream decodes as BT.709 SDR.
   *(AV1 is **not** a "copy the HEVC VUI" fix — AV1 has no VUI/SEI; it carries
   primaries/transfer/matrix in the sequence-header `color_config` and mastering/CLL in
   **METADATA OBUs** `HDR_MDCV`/`HDR_CLL`. Verify whether NVENC's AV1 path accepts them.)*
4. **Linux host is 8-bit only end to end** — capture offers only 8-bit PipeWire formats
   (`capture/linux.rs:443-453, 594-654`; gamescope #2126, portals don't wire PipeWire 1.6
   BT.2020/PQ); encode downgrades 10-bit (`encode/linux.rs:153-162` TODO, `vaapi.rs:719`) with
   BT.709 hardcoded. The Windows-style 8-bit→Main10 upconvert shim is not implemented here.
5. **Linux client HDR is a complete non-feature** — `video_caps=0`, P010 decode path dead
   (`video.rs:379`), CICP hardcoded BT.709 (`ui_stream.rs:239-243`), no Wayland
   color-management (GTK4 0.11 too old).

### Correctness
6. **No host ever emits the ST.2086 mastering or CEA-861.3 CLL SEI.** Windows never reads
   `IDXGIOutput6::GetDesc1`; `nvenc.rs` never builds an `NV_ENC_SEI_PAYLOAD`; Linux attaches no
   libavcodec `side_data`. Apollo reads `GetDesc1` and attaches it.
7. **Clients hardcode mastering metadata.** `present.rs:584-595` ships fixed
   `1000-nit / MaxCLL 1000 / MaxFALL 400` (with the literal "the protocol doesn't carry the
   stream's real mastering metadata yet" comment). Apple/Android set none.
8. **HDR→SDR tone-mapping is unaddressed — and it's the common case.** Most client displays are
   SDR. No client queries display peak; silent `SetColorSpace1`/`SetHDRMetaData` failures present
   PQ as SDR gamma (crushed/dark). We lean entirely on OS auto-fallback.
9. **Windows secure desktop drops HDR to SDR** on lock/UAC (`dxgi.rs:325-368`,
   `sudovda.rs:234-277`).
10. **GameStream silently streams SDR** on a Moonlight HDR request (`mod.rs:48-56`,
    `rtsp.rs:288-293`) — logged, but no negotiated error. Real Apollo parity needs the Moonlight
    `SS_HDR_METADATA` blob on the **ENet control channel**, not just in-band.
11. **Linux client software path is color-wrong even for SDR** — BT.601 applied to BT.709
    (`video.rs:162-167`, no `color_state` on the texture). Standalone bug.

### Quality
12. No per-content MaxCLL/MaxFALL (`GetDesc1` doesn't expose it). No encoder-CSC-range vs
    signaled-range reconciliation (black-crush risk). No automated 10-bit test — `probe` never
    even reads `Welcome.bit_depth` (`main.rs:396-406`).

### Out of scope (call out, don't build)
- Dynamic metadata: HDR10+ (ST.2094-40) and Dolby Vision RPU. We handle *static* ST.2086 only,
  with mid-stream changes carried by re-sending the static block (below).
- HLG: the colorimetry transfer enum carries `18` from day one (free), but the `0xCE` mastering
  datagram is **omitted for HLG** (scene-referred, no mastering metadata).

---

## Protocol design (the keystone — pure-additive, hardware-free, CI-testable)

Two layers, both back-compat-safe via the established trailing-bytes / new-datagram-tag patterns.

### (A) Per-session colorimetry — 4 trailing bytes on `Welcome`

After the existing `bit_depth` (offset 59), append a fixed 4-byte CICP block at offsets 60..64.
(A future mirror on `Reconfigured` will announce a mid-stream SDR↔HDR / BT.709↔BT.2020 flip on the
control stream we already use for renegotiation — deferred to Step 1 with the mid-stream-flip work;
today a mode switch never changes the colour, and the `0xCE` re-send covers mastering changes.)

```
[60] colour_primaries          (CICP: 1=BT.709, 9=BT.2020)
[61] transfer_characteristics  (1=BT.709, 16=PQ/SMPTE2084, 18=HLG)
[62] matrix_coeffs             (1=BT.709, 9=BT.2020-NCL)   ← never emit 10 (CL): no client decodes it
[63] video_full_range_flag     (0=limited, 1=full)
```

Decode with `b.get(60).unwrap_or(1)` etc. — an older host omits them → BT.709 limited SDR
(today's behavior). `Welcome` stays `Copy`. Modeled as a `ColorInfo` struct on the wire types
and exposed on `NativeClient` (with `bit_depth`) so clients *know* the colorimetry instead of
inferring it.

### (B) Per-change mastering + CLL — a new host→client datagram, tag `0xCE`

ST.2086 is variable and changes mid-stream, so it rides a datagram (next tag after `0xCD`
HIDOUT), demuxed in `client.rs` like AUDIO/RUMBLE/HIDOUT. 28 bytes, standard SEI fixed-point:

```
[0]   = 0xCE
G.x G.y B.x B.y R.x R.y   6 × u16 LE   display primaries, 1/50000 units
wp.x wp.y                 2 × u16 LE   white point,        1/50000 units
max_display_mastering_luminance  u32 LE   0.0001 cd/m²
min_display_mastering_luminance  u32 LE   0.0001 cd/m²
max_cll                   u16 LE       nits
max_fall                  u16 LE       nits
```

- Sent on session start and whenever `GetDesc1`/source mastering changes; **re-sent on every
  IDR/RFI keyframe** so a client that dropped the (best-effort) datagram converges within a GOP.
  Until first receipt the client uses the Welcome transfer + a documented generic default.
- **Bounds-check length before reading** (reassembler-bounds security invariant) — truncation
  test required.
- **Omitted entirely for HLG.**
- Units note: these map straight to DXGI `DXGI_HDR_METADATA_HDR10`, Android `KEY_HDR_STATIC_INFO`,
  and Apple `CAEDRMetadata.hdr10`. On the **libavcodec/Linux** side they need conversion —
  `AVMasteringDisplayMetadata` stores `AVRational`, not raw fixed-point.

### (C) C ABI

- `punktfunk_connect_ex5(... video_caps: u8)` (ex4 delegates with 0); **fix `abi.rs:896`.**
- `punktfunk_connection_next_hdr_meta(c, *mut PunktfunkHdrMeta, timeout_ms)` — new plane,
  one-puller contract like `next_audio`.
- `punktfunk_connection_color_info(c, *mut prim, *mut trc, *mut matrix, *mut range, *mut bit_depth)`.
- Regenerate `include/punktfunk_core.h` (cbindgen); `struct_size`/repr(C) guards on new structs.

---

## Phases

### Step 0 — Protocol + ABI carry color metadata end to end *(this change)*
The dominant cross-cutting blocker; everything else is downstream. No rendering changes, no
hardware, CI-testable.

- **core:** `ColorInfo` + 4 Welcome bytes; `HdrMeta` + `0xCE` codec (bounds-checked);
  `NativeClient` `color`/`bit_depth` fields + HdrMeta receiver + demux + `next_hdr_meta`.
- **C ABI:** `connect_ex5`, `next_hdr_meta`, `color_info`, fix caps=0; regen header.
- **host:** populate `Welcome.color` from the negotiated bit-depth/HDR decision; send a `0xCE`
  (generic default for now) when HDR is negotiated.
- **clients:** Windows/Android inherit the demux via shared core; Apple flips to `ex5`.
- **validation:** `quic.rs` round-trip + truncation + **SDR back-compat** tests; `probe` logs
  `bit_depth` + colorimetry; loopback asserts a 10-bit Welcome carries trc=16 and a `0xCE` lands.

### Step 1 — Host emits correct in-band SEI + complete VUI on all codecs *(landed; RTX-validation pending)*
In-band SEI is read directly by decoders, so it fixes correctness even before clients consume
the protocol, and gives an Apollo/Moonlight on-glass parity gate.

- **Single source of truth:** the capturer learns the source display's mastering metadata and
  exposes it via `Capturer::hdr_meta() -> Option<HdrMeta>`. The stream loop forwards it to the
  encoder (`Encoder::set_hdr_meta` → in-band SEI) **and** the client (real `0xCE`, re-sent on each
  keyframe). Pure byte-level logic (float→fixed conversion + the HEVC/H.264 SEI payloads) lives in
  the unit-tested, cross-platform `src/hdr.rs` (`hdr_meta_from_display`, `hevc_mastering_display_sei`
  type **137**, `hevc_content_light_level_sei` type **144** — note: NOT "type 4", that was a
  drafting error).
- **Windows (done, CI-compiled / RTX on-glass pending):** `dxgi.rs` + `wgc.rs` read
  `IDXGIOutput6::GetDesc1` at capture init / output change → `HdrMeta` (MaxCLL/MaxFALL left 0 —
  GetDesc1 has none, like Apollo). `nvenc.rs` attaches the mastering + CLL SEI on every IDR for
  HEVC/H.264. (AV1 mastering rides METADATA OBUs, not SEI — follow-up; AV1 `color_config` already
  lands in Step 0's quick win.)
- **Linux encode-ready — DEFERRED into Step 4:** Linux capture is 8-bit only, so signalling
  BT.2020 PQ + attaching mastering side-data on a downconverted 8-bit stream would be *incorrect*.
  The libavcodec `side_data` path (with the `AVRational` conversion) lands together with the
  8-bit→Main10 shim / true 10-bit capture in Step 4.
- **Windows secure-desktop relay** (`virtual_stream_relay`) still sends only the generic baseline
  `0xCE`; the helper's in-band SEI carries the real grade. Wiring the relay's `0xCE` is a follow-up.
- **validation (RTX box):** `ffprobe -show_frames` shows mastering + CLL side-data with the
  display's real luminance and VUI 9/16/9; stock Moonlight shows correct (not washed-out) HDR.
  Add **encoder-CSC-range == signaled-range** check.

### Step 2 — Clients apply the metadata *(landed; CI/on-glass validation pending)*
All three clients now drain the protocol's `HdrMeta` (`next_hdr_meta` / `nextHdrMeta`) and apply it,
each remapping from the wire form (ST.2086 G,B,R order, mastering luminance in 0.0001 cd/m²) to the
platform's expected layout:
- **Windows (Rust, CI-compiled):** session pump drains `next_hdr_meta` into a `LATEST_HDR_META`
  slot; `present_newest` applies it via `Presenter::set_hdr_metadata` → real `SetHDRMetaData`
  (`hdr_meta_to_dxgi`: G,B,R→R,G,B reorder, 0.0001-nit→nit for `MaxMasteringLuminance`), dropping
  the 1000/1000/400 hardcode. `SetColorSpace1`/`SetHDRMetaData` failures + an SDR-display
  colour-space rejection are now **logged**, not swallowed.
- **Apple (Swift, mac-runner CI):** connect now advertises caps via `punktfunk_connect_ex5`
  (`SessionModel` computes `videoCap10Bit|videoCapHDR` from `hdrEnabled`) — *this is the fix that
  resurrects Apple's previously-dead HDR pipeline*. `nextHdrMeta`/`colorInfo` wrappers added; the
  pump drains `nextHdrMeta` → `VideoDecoder.setHdrMeta` → `CVBufferSetAttachment` of
  `kCVImageBufferMasteringDisplayColorVolumeKey` (24-byte BE SEI) +
  `kCVImageBufferContentLightLevelInfoKey` (4-byte BE) on each HDR pixel buffer (the correct path
  for the itur_2100_PQ layer; `CAEDRMetadata` on a PQ layer is ambiguous and was avoided).
- **Android (Rust `decode.rs`, cargo-ndk verified):** when `client.color.is_hdr()`, drain the first
  `next_hdr_meta` and set `MediaFormat` `hdr-static-info` (`KEY_HDR_STATIC_INFO`) before
  `configure()` — `android_hdr_static_info` builds the 25-byte CTA-861.3 Type-1 blob (LE, **R,G,B**
  order, max-lum in **nits-u16**). `Display.getHdrCapabilities` gate deferred (the Surface DataSpace
  already drives SurfaceFlinger tone-mapping on non-HDR displays).

### Step 3 — Display-capability gate *(landed; CI/on-glass validation pending)*
The common-case correctness step — most client displays are SDR. **Chosen approach: capability-gate**
(not an in-shader BT.2390 tone-map). Rationale: with Steps 1–2 the host sends *correct* mastering
metadata, so an HDR display self-tone-maps from it; the real remaining gap is SDR displays, best
fixed by **not advertising HDR you can't present** — the host then sends a proper BT.709 SDR stream
instead of PQ the panel would mis-tone-map (washed-out/dark). No guessed tone-map curve, deterministic.
- **Windows** (`present::display_supports_hdr` via DXGI: any `IDXGIOutput6` colour space ==
  `G2084`): `session.rs` ANDs it with the user's HDR setting before advertising caps; logs when it
  drops to SDR.
- **Apple** (`SessionModel`, main-actor): `NSScreen.maximumExtendedDynamicRangeColorComponentValue
  > 1` (macOS) / `UIScreen.main.potentialEDRHeadroom > 1` (iOS) ANDed with `hdrEnabled`.
- **Android** (`Settings.displaySupportsHdr` via `Display.getHdrCapabilities` HDR10/HDR10+): Kotlin
  passes it to `nativeConnect`; `session.rs` gates the caps on the new `hdr_enabled` jboolean
  (cargo-ndk-verified).
- **Deferred** (need on-glass / the RTX box): the **mid-session `Reconfigure` "downgrade to SDR"**
  for a monitor move HDR↔SDR; and confirming the **host produces SDR for an SDR client even off an
  HDR desktop** — on the native path the per-session SudoVDA follows the negotiated depth (SDR
  client → SDR virtual display → SDR stream), so it should hold end-to-end; verify the
  stale-HDR-SudoVDA edge case on the RTX box.

### Step 4 — Linux (last; capture blocked upstream)
- **8-bit→Main10 NVENC upconvert shim** (`encode/linux.rs`) — Main10 transport with correct
  VUI/SEI without HDR capture (gate so we don't claim HDR transfer on SDR content).
- **Linux encode color + side-data (the deferred Step 1c):** set
  `color_primaries/trc/colorspace/range` from the negotiated `ColorInfo` and attach
  `AV_FRAME_DATA_MASTERING_DISPLAY_METADATA` / `CONTENT_LIGHT_LEVEL` side-data (with the
  `AVRational` conversion) in `encode/linux.rs` + `vaapi.rs` — only once the encoder actually
  produces 10-bit, so the signalling matches the bits.
- True 10-bit capture: offer `ABGR2101010`/`P010` PipeWire formats + read colorimetry; pilot on
  Sway/wlroots; track gamescope #2126. **Don't block the rest of the plan on it.**
- Linux client: `ex5` caps, P010 decode, GdkDmabufTexture CICP from Welcome,
  `wp_color_management` when GTK ≥ 4.14.

## Quick wins (independent, land in parallel)
1. `connect_ex5` + fix `abi.rs:896` — resurrects Apple's pipeline *(Step 0)*.
2. H.264 VUI + AV1 `color_config` on `nvenc.rs` — closes two latent blockers *(Windows-only,
   validated in CI / on the RTX box)*.
3. `probe` logs `bit_depth` + colorimetry — observability for every later round-trip assertion.
4. Linux client BT.601→BT.709 sws + texture `color_state` — standalone SDR correctness bug.
5. GameStream silent-downgrade already warns (`rtsp.rs:289`) — keep observable.

## Open questions
- **MaxCLL source:** `GetDesc1` doesn't expose it (Apollo zeroes). Static default, or measure
  per-frame peak in the PQ shader (only truly-correct, adds a readback)?
- **GameStream:** implement `SS_HDR_METADATA` for Moonlight parity, or keep it deliberately SDR
  and steer HDR users to punktfunk/1?
- **HLG:** carry the enum from day one (free) — but do any sources actually produce HLG?
- **Linux:** is shipping the 8-bit→Main10 shim as "HDR-capable transport" acceptable, or does it
  risk advertising HDR we can't truly deliver?

## Ordering rationale
Step 0 first: it's the keystone (metadata transport is the dominant cross-cutter; the ABI line
is a one-line root cause) and needs no hardware. Step 1 next: in-band SEI is read directly by
decoders, so it fixes correctness even before our clients consume the protocol, and gives an
Apollo-parity on-glass gate. Steps 2–3 are mechanical per-client wiring once metadata flows.
Linux is last because capture is gated on upstream we don't control; the shim delivers Main10
transport without that dependency.

Hardware dependencies: Step 0 = none (CI); Step 1 = RTX Windows host; Steps 2–3 = a real HDR
display per platform; Step 4 = a Linux GPU box + HDR-capable Wayland compositor.