feat(clients): HDR Steps 2-3 — apply mastering metadata + display capability-gate
Continues docs/hdr-pipeline-plan.md. Steps 0/1 + Step 2 (Windows/Android) already landed in 3526517; this is Step 2 (Apple) + Step 3 (all clients). Client-only — no core/host/ABI change (the 0xCE/next_hdr_meta/color_info surfaces shipped in Step 0). Step 2 — clients APPLY the host's HDR metadata (each remaps from the wire form: ST.2086 G,B,R order, mastering luminance in 0.0001 cd/m2): - Apple: connect via punktfunk_connect_ex5 (resurrects the previously-dead HDR pipeline); nextHdrMeta/colorInfo wrappers + HdrMeta SEI-blob builders; the pump drains nextHdrMeta -> VideoDecoder.setHdrMeta -> CVBufferSetAttachment of MasteringDisplayColorVolume (24B BE) + ContentLightLevelInfo (4B BE) on each HDR pixel buffer (correct for the itur_2100_PQ layer; CAEDRMetadata avoided as ambiguous there). Step 3 — capability-gate: advertise HDR caps ONLY when the display can present it, so an SDR display gets a proper BT.709 stream instead of PQ it would mis-tone-map; an HDR display self-tone-maps from the Step-1/2 mastering metadata. - Windows: present::display_supports_hdr() (DXGI any IDXGIOutput6 colour space == G2084), ANDed with the user HDR setting in session.rs; logs the SDR drop. - Apple: NSScreen.maximumExtendedDynamicRangeColorComponentValue>1 (macOS) / UIScreen.main.potentialEDRHeadroom>1 (iOS) in SessionModel. - Android: Settings.displaySupportsHdr (Display.getHdrCapabilities HDR10/HDR10+) passed through a new hdr_enabled jboolean on nativeConnect; session.rs gates the caps. Validation: Android native (incl. the jboolean gate) builds + clippy clean via cargo-ndk; fmt clean. Windows (MSVC), Apple (Swift) and the Kotlin side are CI/on-glass validated — not compilable on the Linux dev box. Deferred to the RTX box: mid-session Reconfigure SDR-downgrade on monitor move, and confirming the host emits SDR for an SDR client off an HDR desktop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
+40
-13
@@ -186,20 +186,47 @@ the protocol, and gives an Apollo/Moonlight on-glass parity gate.
|
||||
display's real luminance and VUI 9/16/9; stock Moonlight shows correct (not washed-out) HDR.
|
||||
Add **encoder-CSC-range == signaled-range** check.
|
||||
|
||||
### Step 2 — Clients apply the metadata (Windows + Apple + Android, parallelizable)
|
||||
- **Windows:** feed `hdr10_metadata()` from the received `HdrMeta` (drop the hardcode); **log**
|
||||
`SetColorSpace1`/`SetHDRMetaData` failures.
|
||||
- **Apple:** attach `kCVImageBufferMasteringDisplayColorVolumeKey` + `ContentLightLevelInfoKey`
|
||||
/ `CAEDRMetadata` from `HdrMeta`; CV color attachments from Welcome.
|
||||
- **Android:** set `MediaFormat KEY_HDR_STATIC_INFO` from `HdrMeta`.
|
||||
### Step 2 — Clients apply the metadata *(landed; CI/on-glass validation pending)*
|
||||
All three clients now drain the protocol's `HdrMeta` (`next_hdr_meta` / `nextHdrMeta`) and apply it,
|
||||
each remapping from the wire form (ST.2086 G,B,R order, mastering luminance in 0.0001 cd/m²) to the
|
||||
platform's expected layout:
|
||||
- **Windows (Rust, CI-compiled):** session pump drains `next_hdr_meta` into a `LATEST_HDR_META`
|
||||
slot; `present_newest` applies it via `Presenter::set_hdr_metadata` → real `SetHDRMetaData`
|
||||
(`hdr_meta_to_dxgi`: G,B,R→R,G,B reorder, 0.0001-nit→nit for `MaxMasteringLuminance`), dropping
|
||||
the 1000/1000/400 hardcode. `SetColorSpace1`/`SetHDRMetaData` failures + an SDR-display
|
||||
colour-space rejection are now **logged**, not swallowed.
|
||||
- **Apple (Swift, mac-runner CI):** connect now advertises caps via `punktfunk_connect_ex5`
|
||||
(`SessionModel` computes `videoCap10Bit|videoCapHDR` from `hdrEnabled`) — *this is the fix that
|
||||
resurrects Apple's previously-dead HDR pipeline*. `nextHdrMeta`/`colorInfo` wrappers added; the
|
||||
pump drains `nextHdrMeta` → `VideoDecoder.setHdrMeta` → `CVBufferSetAttachment` of
|
||||
`kCVImageBufferMasteringDisplayColorVolumeKey` (24-byte BE SEI) +
|
||||
`kCVImageBufferContentLightLevelInfoKey` (4-byte BE) on each HDR pixel buffer (the correct path
|
||||
for the itur_2100_PQ layer; `CAEDRMetadata` on a PQ layer is ambiguous and was avoided).
|
||||
- **Android (Rust `decode.rs`, cargo-ndk verified):** when `client.color.is_hdr()`, drain the first
|
||||
`next_hdr_meta` and set `MediaFormat` `hdr-static-info` (`KEY_HDR_STATIC_INFO`) before
|
||||
`configure()` — `android_hdr_static_info` builds the 25-byte CTA-861.3 Type-1 blob (LE, **R,G,B**
|
||||
order, max-lum in **nits-u16**). `Display.getHdrCapabilities` gate deferred (the Surface DataSpace
|
||||
already drives SurfaceFlinger tone-mapping on non-HDR displays).
|
||||
|
||||
### Step 3 — Display-capability query + client tone-mapping + robust fallback
|
||||
The common-case correctness step — most displays are SDR.
|
||||
|
||||
- **HDR→SDR on every client** (defined BT.2390 EETF / Hable), not silent OS fallback.
|
||||
- Content-peak > display-peak roll-off (`GetDesc1` / `NSScreen.maximumEDR…` /
|
||||
`Display.getHdrCapabilities`); explicit SDR fallback when HDR present fails.
|
||||
- Optional client→host "send me SDR" downgrade as a trailing field on `Reconfigure`.
|
||||
### Step 3 — Display-capability gate *(landed; CI/on-glass validation pending)*
|
||||
The common-case correctness step — most client displays are SDR. **Chosen approach: capability-gate**
|
||||
(not an in-shader BT.2390 tone-map). Rationale: with Steps 1–2 the host sends *correct* mastering
|
||||
metadata, so an HDR display self-tone-maps from it; the real remaining gap is SDR displays, best
|
||||
fixed by **not advertising HDR you can't present** — the host then sends a proper BT.709 SDR stream
|
||||
instead of PQ the panel would mis-tone-map (washed-out/dark). No guessed tone-map curve, deterministic.
|
||||
- **Windows** (`present::display_supports_hdr` via DXGI: any `IDXGIOutput6` colour space ==
|
||||
`G2084`): `session.rs` ANDs it with the user's HDR setting before advertising caps; logs when it
|
||||
drops to SDR.
|
||||
- **Apple** (`SessionModel`, main-actor): `NSScreen.maximumExtendedDynamicRangeColorComponentValue
|
||||
> 1` (macOS) / `UIScreen.main.potentialEDRHeadroom > 1` (iOS) ANDed with `hdrEnabled`.
|
||||
- **Android** (`Settings.displaySupportsHdr` via `Display.getHdrCapabilities` HDR10/HDR10+): Kotlin
|
||||
passes it to `nativeConnect`; `session.rs` gates the caps on the new `hdr_enabled` jboolean
|
||||
(cargo-ndk-verified).
|
||||
- **Deferred** (need on-glass / the RTX box): the **mid-session `Reconfigure` "downgrade to SDR"**
|
||||
for a monitor move HDR↔SDR; and confirming the **host produces SDR for an SDR client even off an
|
||||
HDR desktop** — on the native path the per-session SudoVDA follows the negotiated depth (SDR
|
||||
client → SDR virtual display → SDR stream), so it should hold end-to-end; verify the
|
||||
stale-HDR-SudoVDA edge case on the RTX box.
|
||||
|
||||
### Step 4 — Linux (last; capture blocked upstream)
|
||||
- **8-bit→Main10 NVENC upconvert shim** (`encode/linux.rs`) — Main10 transport with correct
|
||||
|
||||
Reference in New Issue
Block a user