feat(clients): HDR Steps 2-3 — apply mastering metadata + display capability-gate

Continues docs/hdr-pipeline-plan.md. Steps 0/1 + Step 2 (Windows/Android) already
landed in 3526517; this is Step 2 (Apple) + Step 3 (all clients). Client-only — no
core/host/ABI change (the 0xCE/next_hdr_meta/color_info surfaces shipped in Step 0).

Step 2 — clients APPLY the host's HDR metadata (each remaps from the wire form: ST.2086
G,B,R order, mastering luminance in 0.0001 cd/m2):
- Apple: connect via punktfunk_connect_ex5 (resurrects the previously-dead HDR pipeline);
  nextHdrMeta/colorInfo wrappers + HdrMeta SEI-blob builders; the pump drains nextHdrMeta
  -> VideoDecoder.setHdrMeta -> CVBufferSetAttachment of MasteringDisplayColorVolume (24B
  BE) + ContentLightLevelInfo (4B BE) on each HDR pixel buffer (correct for the
  itur_2100_PQ layer; CAEDRMetadata avoided as ambiguous there).

Step 3 — capability-gate: advertise HDR caps ONLY when the display can present it, so an
SDR display gets a proper BT.709 stream instead of PQ it would mis-tone-map; an HDR
display self-tone-maps from the Step-1/2 mastering metadata.
- Windows: present::display_supports_hdr() (DXGI any IDXGIOutput6 colour space == G2084),
  ANDed with the user HDR setting in session.rs; logs the SDR drop.
- Apple: NSScreen.maximumExtendedDynamicRangeColorComponentValue>1 (macOS) /
  UIScreen.main.potentialEDRHeadroom>1 (iOS) in SessionModel.
- Android: Settings.displaySupportsHdr (Display.getHdrCapabilities HDR10/HDR10+) passed
  through a new hdr_enabled jboolean on nativeConnect; session.rs gates the caps.

Validation: Android native (incl. the jboolean gate) builds + clippy clean via cargo-ndk;
fmt clean. Windows (MSVC), Apple (Swift) and the Kotlin side are CI/on-glass validated —
not compilable on the Linux dev box. Deferred to the RTX box: mid-session Reconfigure
SDR-downgrade on monitor move, and confirming the host emits SDR for an SDR client off an
HDR desktop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-21 09:46:58 +00:00
parent 3526517eb1
commit 551012bb43
12 changed files with 193 additions and 27 deletions
+40 -13
View File
@@ -186,20 +186,47 @@ the protocol, and gives an Apollo/Moonlight on-glass parity gate.
display's real luminance and VUI 9/16/9; stock Moonlight shows correct (not washed-out) HDR.
Add **encoder-CSC-range == signaled-range** check.
### Step 2 — Clients apply the metadata (Windows + Apple + Android, parallelizable)
- **Windows:** feed `hdr10_metadata()` from the received `HdrMeta` (drop the hardcode); **log**
`SetColorSpace1`/`SetHDRMetaData` failures.
- **Apple:** attach `kCVImageBufferMasteringDisplayColorVolumeKey` + `ContentLightLevelInfoKey`
/ `CAEDRMetadata` from `HdrMeta`; CV color attachments from Welcome.
- **Android:** set `MediaFormat KEY_HDR_STATIC_INFO` from `HdrMeta`.
### Step 2 — Clients apply the metadata *(landed; CI/on-glass validation pending)*
All three clients now drain the protocol's `HdrMeta` (`next_hdr_meta` / `nextHdrMeta`) and apply it,
each remapping from the wire form (ST.2086 G,B,R order, mastering luminance in 0.0001 cd/m²) to the
platform's expected layout:
- **Windows (Rust, CI-compiled):** session pump drains `next_hdr_meta` into a `LATEST_HDR_META`
slot; `present_newest` applies it via `Presenter::set_hdr_metadata` → real `SetHDRMetaData`
(`hdr_meta_to_dxgi`: G,B,R→R,G,B reorder, 0.0001-nit→nit for `MaxMasteringLuminance`), dropping
the 1000/1000/400 hardcode. `SetColorSpace1`/`SetHDRMetaData` failures + an SDR-display
colour-space rejection are now **logged**, not swallowed.
- **Apple (Swift, mac-runner CI):** connect now advertises caps via `punktfunk_connect_ex5`
(`SessionModel` computes `videoCap10Bit|videoCapHDR` from `hdrEnabled`) — *this is the fix that
resurrects Apple's previously-dead HDR pipeline*. `nextHdrMeta`/`colorInfo` wrappers added; the
pump drains `nextHdrMeta``VideoDecoder.setHdrMeta``CVBufferSetAttachment` of
`kCVImageBufferMasteringDisplayColorVolumeKey` (24-byte BE SEI) +
`kCVImageBufferContentLightLevelInfoKey` (4-byte BE) on each HDR pixel buffer (the correct path
for the itur_2100_PQ layer; `CAEDRMetadata` on a PQ layer is ambiguous and was avoided).
- **Android (Rust `decode.rs`, cargo-ndk verified):** when `client.color.is_hdr()`, drain the first
`next_hdr_meta` and set `MediaFormat` `hdr-static-info` (`KEY_HDR_STATIC_INFO`) before
`configure()``android_hdr_static_info` builds the 25-byte CTA-861.3 Type-1 blob (LE, **R,G,B**
order, max-lum in **nits-u16**). `Display.getHdrCapabilities` gate deferred (the Surface DataSpace
already drives SurfaceFlinger tone-mapping on non-HDR displays).
### Step 3 — Display-capability query + client tone-mapping + robust fallback
The common-case correctness step — most displays are SDR.
- **HDR→SDR on every client** (defined BT.2390 EETF / Hable), not silent OS fallback.
- Content-peak > display-peak roll-off (`GetDesc1` / `NSScreen.maximumEDR…` /
`Display.getHdrCapabilities`); explicit SDR fallback when HDR present fails.
- Optional client→host "send me SDR" downgrade as a trailing field on `Reconfigure`.
### Step 3 — Display-capability gate *(landed; CI/on-glass validation pending)*
The common-case correctness step — most client displays are SDR. **Chosen approach: capability-gate**
(not an in-shader BT.2390 tone-map). Rationale: with Steps 12 the host sends *correct* mastering
metadata, so an HDR display self-tone-maps from it; the real remaining gap is SDR displays, best
fixed by **not advertising HDR you can't present** — the host then sends a proper BT.709 SDR stream
instead of PQ the panel would mis-tone-map (washed-out/dark). No guessed tone-map curve, deterministic.
- **Windows** (`present::display_supports_hdr` via DXGI: any `IDXGIOutput6` colour space ==
`G2084`): `session.rs` ANDs it with the user's HDR setting before advertising caps; logs when it
drops to SDR.
- **Apple** (`SessionModel`, main-actor): `NSScreen.maximumExtendedDynamicRangeColorComponentValue
> 1` (macOS) / `UIScreen.main.potentialEDRHeadroom > 1` (iOS) ANDed with `hdrEnabled`.
- **Android** (`Settings.displaySupportsHdr` via `Display.getHdrCapabilities` HDR10/HDR10+): Kotlin
passes it to `nativeConnect`; `session.rs` gates the caps on the new `hdr_enabled` jboolean
(cargo-ndk-verified).
- **Deferred** (need on-glass / the RTX box): the **mid-session `Reconfigure` "downgrade to SDR"**
for a monitor move HDR↔SDR; and confirming the **host produces SDR for an SDR client even off an
HDR desktop** — on the native path the per-session SudoVDA follows the negotiated depth (SDR
client → SDR virtual display → SDR stream), so it should hold end-to-end; verify the
stale-HDR-SudoVDA edge case on the RTX box.
### Step 4 — Linux (last; capture blocked upstream)
- **8-bit→Main10 NVENC upconvert shim** (`encode/linux.rs`) — Main10 transport with correct