docs: the 4:4:4 story catches up with the code that ships it
The support matrix said the desktop clients' Full chroma switch "has no effect today" and that only the Apple client asks for 4:4:4. Both stopped being true in July: `clients/session/src/main.rs` advertises VIDEO_CAP_444 whenever the setting is on, deliberately with no client-side probe, because every desktop decode rung can display full chroma — the Vulkan presenter samples the 2-plane 4:4:4 pool formats and swscale converts for the software rung. So Linux, Windows and Apple all ask; Android is the one that genuinely doesn't implement it. The other half was HDR. `9f72a3b6` gave the Windows IDD-push capturer a packed 10-bit BT.2020 PQ RGB output, so NVENC encodes HEVC Main 4:4:4 10 and the two compose — the matrix still said "4:4:4 and HDR together is refused", and hdr.md still called PyroWave the only exception. Linux is the side that keeps the trade: handshake.rs resolves the depth back to 8 for a 4:4:4 session, so full chroma wins and the stream is SDR. Three cells move ❌ → ⚠️ rather than ✅ on purpose. The client half is unconditional, but the host half is not: HEVC 4:4:4 means an NVIDIA host, or PyroWave on any vendor. The notes say which, and point at the stats overlay's `4:4:4→4:2:0` tag — this negotiation is the one that fails loudly. Also: C ABI version 13 → 14; PyroWave's ≈8K 4:4:4 block-index ceiling now has a note; and the roadmap no longer calls Intel 4:4:4 a hardware limit, which the matrix and vaapi.rs both contradict — VCN can't, VAAPI hasn't. Spot-checked and left alone as still accurate: the Linux client clipboard stub, VAAPI declining 4:4:4, Android having no 4:4:4 at all, and the wire / driver / gamepad-channel versions.
This commit is contained in:
@@ -39,10 +39,12 @@ told HDR, so that is the one place a Punktfunk label can outrun the picture. The
|
||||
|
||||
Two details worth knowing:
|
||||
|
||||
- **HDR usually beats 4:4:4.** For HEVC and AV1 there is no 10-bit full-chroma capture source, so an
|
||||
HDR session drops to 4:2:0 and says so. If you want [full chroma](/docs/client-settings) with
|
||||
those codecs, turn HDR off for that profile. [PyroWave](/docs/pyrowave) is the exception: its
|
||||
Windows capture path writes full-resolution 10-bit chroma, so it can carry HDR and 4:4:4 together.
|
||||
- **HDR and 4:4:4 compose on Windows, not on Linux.** A **Windows** host carries both: the capture
|
||||
path writes full-resolution 10-bit chroma and NVENC encodes HEVC Main 4:4:4 10, so
|
||||
[full chroma](/docs/client-settings) costs you nothing on an HDR desktop.
|
||||
[PyroWave](/docs/pyrowave) does the same there, in 16-bit planes. On **Linux** the 4:4:4 route is
|
||||
8-bit, so a session that negotiates both resolves back down to SDR — full chroma wins. AV1 never
|
||||
carries 4:4:4 anywhere: Range Extensions are HEVC-only.
|
||||
- **Vulkan games need the bundled layer.** NVIDIA and AMD Vulkan drivers refuse to advertise any HDR
|
||||
colour space for a surface on an indirect (virtual) display, so Vulkan games decide the device
|
||||
"does not support HDR" — even though the driver happily presents an HDR swapchain there. The host
|
||||
@@ -139,8 +141,9 @@ swapchain without a tone-map, which looks washed out. Turn the client's HDR sett
|
||||
is SDR. Use HEVC or AV1 for HDR from Linux.
|
||||
|
||||
One more rule if you also use full chroma: a **Linux** host encodes 4:4:4 at 8 bits, so a session
|
||||
that negotiates both resolves back down to SDR before the stream starts. On Linux 4:4:4 wins; on
|
||||
Windows HDR does. Full chroma is off until you turn it on, so this only bites if you did.
|
||||
that negotiates both resolves back down to SDR before the stream starts — on Linux, 4:4:4 wins. A
|
||||
**Windows** host has no such trade: it carries HDR and full chroma at once. Full chroma is off until
|
||||
you turn it on, so this only bites if you did.
|
||||
|
||||
## Check it
|
||||
|
||||
|
||||
@@ -92,7 +92,11 @@ head-tracked remote spatial audio that no streaming stack does today.
|
||||
[matrix](/docs/support-matrix#input-cursor-and-hdr).
|
||||
- **Hosting on macOS, iOS, tvOS or Android.** Client-only platforms by construction: every host
|
||||
entry point fails at compile time. There is no setting that changes this.
|
||||
- **4:4:4 on AMD and Intel encoders.** A limitation of those encode blocks, not a gap in Punktfunk.
|
||||
- **HEVC 4:4:4 on the AMD encode block.** AMD's VCN never encodes 4:4:4, so there is nothing to
|
||||
implement. Intel is a different story and *is* a gap rather than a wall — the VAAPI backend
|
||||
simply has no 4:4:4 path yet, and it waits on hardware that advertises a HEVC 4:4:4 encode
|
||||
entrypoint to build and validate against. On either vendor, [PyroWave](/docs/pyrowave) already
|
||||
carries full chroma today.
|
||||
- **DualSense voice-coil haptics.** Scoped and shelved — it rides the controller's USB audio
|
||||
interface and has near-zero game support on Linux. Rumble, adaptive triggers and the lightbar
|
||||
already work.
|
||||
|
||||
@@ -37,7 +37,10 @@ into a "no" on your machine:
|
||||
answer. The backend supports it; **your device may still refuse it**. Most codec, 10-bit and
|
||||
4:4:4 cells are this kind.
|
||||
- **Negotiated** — both ends must advertise it before it happens. The clipboard, pen input, 4:4:4
|
||||
and client-drawn cursors all die quietly if either side says no.
|
||||
and client-drawn cursors all die if either side says no. Mostly quietly — 4:4:4 is the exception,
|
||||
and deliberately so: the host resolves the chroma *before* the Welcome and names the losing gate
|
||||
in its log, and the client's stats overlay prints `4:4:4→4:2:0` rather than letting you assume
|
||||
you got what you asked for.
|
||||
- **Default on / opt-in / operator-gated** — HDR and 10-bit are attempted by default; the game
|
||||
library is off by default on the desktop clients; the shared clipboard is off on the host until
|
||||
an operator turns it on.
|
||||
@@ -199,14 +202,21 @@ newer on AMD, Arc and newer on Intel).
|
||||
1. H.264, HEVC and AV1, intersected with what the driver reports. If the probe cannot run (no
|
||||
driver, or a build without NVENC), the host advertises the full set rather than nothing — so an
|
||||
advertised codec is not always a *confirmed* codec.
|
||||
2. HEVC only, and only when the GPU's 4:4:4 capability bit says yes. 4:4:4 **and** HDR together is
|
||||
refused.
|
||||
2. HEVC only, and only when the GPU's 4:4:4 capability bit says yes. **HDR and 4:4:4 together
|
||||
depend on the platform.** On Windows they compose: the IDD-push capturer converts the FP16
|
||||
desktop to packed 10-bit BT.2020 PQ RGB and NVENC encodes HEVC Main 4:4:4 10, so a session gets
|
||||
both. On Linux 4:4:4 rides an 8-bit `YUV444P` route, so a session that negotiates both resolves
|
||||
the bit depth back to 8 — full chroma wins and the stream is SDR. Either way the answer is
|
||||
settled before the Welcome, so the client is never told one thing and sent another.
|
||||
3. A hardware limitation of AMD's encode block, not a gap in Punktfunk. VCN never encodes 4:4:4,
|
||||
so there is nothing to probe.
|
||||
4. Only in a build that includes the native QSV backend — which the shipped installer does. In a
|
||||
hand build without it, 10-bit is honestly reported as unavailable rather than guessed.
|
||||
5. [PyroWave](/docs/pyrowave) is a wavelet codec, not H.26x. It is never picked automatically: your
|
||||
client has to ask for it by name in its codec setting.
|
||||
client has to ask for it by name in its codec setting. Its 4:4:4 is the one that needs no GPU
|
||||
encode probe — it does its own full-chroma colour conversion, so it resolves on any vendor —
|
||||
with a single ceiling: the vendored rate controller packs its block index into 16 bits, which an
|
||||
≈8K-class 4:4:4 mode overflows, so those modes are downgraded to 4:2:0 before the Welcome.
|
||||
6. Requires the direct-SDK NVENC path (which every shipped Linux package builds). Without it the
|
||||
frame takes a slower CPU route to reach 10-bit.
|
||||
7. H.264 never uses this backend, and it exists only in a build carrying the Vulkan-encode feature
|
||||
@@ -224,10 +234,11 @@ newer on AMD, Arc and newer on Intel).
|
||||
`PUNKTFUNK_ENCODER=software` deliberately if that is what you want. On Windows, by contrast, an
|
||||
unrecognised adapter does resolve to software on its own.
|
||||
|
||||
**4:4:4 across the whole project:** only HEVC and PyroWave can carry it, only NVENC and PyroWave can
|
||||
produce it, and only the Apple client asks for it. The Linux and Windows desktop clients have a
|
||||
4:4:4 setting that currently does nothing — see [Client settings](/docs/client-settings). GameStream
|
||||
sessions are always 4:2:0.
|
||||
**4:4:4 across the whole project:** only HEVC and PyroWave can carry it, and only NVENC and
|
||||
PyroWave can produce it — so on the HEVC side full chroma means an NVIDIA host. Asking for it is a
|
||||
client setting, off by default, and the **Linux, Windows and Apple** clients all have a working one;
|
||||
**Android does not implement 4:4:4 at all**, and GameStream sessions are always 4:2:0. See
|
||||
[Client settings](/docs/client-settings).
|
||||
|
||||
### How the host picks a backend
|
||||
|
||||
@@ -288,9 +299,9 @@ This is a **GPU and encoder** question, not a compositor one, which is why it is
|
||||
|
||||
| Client | Decode path (in order) | Codecs | 10-bit / HDR | 4:4:4 |
|
||||
|---|---|---|---|---|
|
||||
| Linux desktop | Vulkan Video → VAAPI → software ¹ | probed ² | ✅ ³ | ❌ ⁴ |
|
||||
| Windows desktop | Vulkan Video → D3D11VA → software ¹ | probed ² | ✅ ³ | ❌ ⁴ |
|
||||
| Steam Deck (via Decky) | as Linux desktop ⁵ | probed ² | ✅ | ❌ |
|
||||
| Linux desktop | Vulkan Video → VAAPI → software ¹ | probed ² | ✅ ³ | ⚠️ ⁴ |
|
||||
| Windows desktop | Vulkan Video → D3D11VA → software ¹ | probed ² | ✅ ³ | ⚠️ ⁴ |
|
||||
| Steam Deck (via Decky) | as Linux desktop ⁵ | probed ² | ✅ | ⚠️ ⁴ |
|
||||
| macOS · iOS · tvOS | VideoToolbox only | H.264, HEVC, AV1 ⁶ | ⚠️ ⁷ | ⚠️ ⁸ |
|
||||
| Android · Android TV | MediaCodec only ⁹ | H.264, HEVC, AV1 ¹⁰ | ⚠️ ⁷ | ❌ |
|
||||
| Moonlight | your Moonlight app's | negotiated | ⚠️ ¹¹ | ❌ |
|
||||
@@ -308,8 +319,14 @@ This is a **GPU and encoder** question, not a compositor one, which is why it is
|
||||
3. On by default. It is presented on a real HDR10 surface where your desktop offers one (KDE HDR,
|
||||
gamescope), and tone-mapped in-shader otherwise. Software-decoded frames never take the HDR
|
||||
surface.
|
||||
4. The 4:4:4 setting is stored and shown but is never advertised to the host, so these clients
|
||||
always receive 4:2:0.
|
||||
4. Opt-in (Settings ▸ **Full chroma**), off by default, and advertised whenever you ask — unlike
|
||||
Apple, **no client-side probe gates it**, because every rung here can display full chroma: the
|
||||
Vulkan presenter samples the 2-plane 4:4:4 pool formats where the driver decodes RExt in
|
||||
hardware (NVIDIA today), and swscale converts for the software rung otherwise. So the setting
|
||||
always reaches the host; what varies is the *cost*, and that cost is visible rather than silent
|
||||
— the Detailed [stats overlay](/docs/stats) prints the resolved chroma (`4:4:4→4:2:0` when the
|
||||
host declined) and the decode path frames really took. Whether you get it is then the host's
|
||||
answer: HEVC 4:4:4 means an NVIDIA host, or pick PyroWave, which does it on any vendor.
|
||||
5. The Decky plugin does not decode anything — it launches the Linux client, so the decode path is
|
||||
identical, including the Mesa `RADV_PERFTEST=video_decode` opt-in the session binary sets before
|
||||
any Vulkan call (without it RADV exposes no decode queue and the Deck silently falls back to
|
||||
@@ -320,8 +337,11 @@ This is a **GPU and encoder** question, not a compositor one, which is why it is
|
||||
7. Runtime-probed against the actual display: EDR headroom on iPhone/iPad, HDR eligibility on Mac
|
||||
and Apple TV, HDR capabilities on Android. On an SDR panel the client advertises no HDR at all
|
||||
so the host sends a correct 8-bit picture instead of PQ your screen would mangle.
|
||||
8. The only client that asks for 4:4:4 — opt-in, and gated on a real hardware-decode probe (both
|
||||
8-bit and 10-bit when HDR is also on). In practice it only ever resolves against an NVIDIA host.
|
||||
8. Opt-in, and the only client that **probes before asking**: it advertises 4:4:4 only where
|
||||
VideoToolbox really hardware-decodes it (both 8-bit and 10-bit when HDR is also on), because
|
||||
VideoToolbox's software 4:4:4 decode is far too slow for a real-time stream — validated on M3.
|
||||
The desktop clients need no such probe (note 4). For HEVC it only ever resolves against an
|
||||
NVIDIA host.
|
||||
9. Chosen by name from a ranked device list that prefers hardware, real SoC vendors and low-latency
|
||||
decoders, and blocks the known-bad software ones. There is no software rung.
|
||||
10. H.264 and HEVC are assumed universal on Android hardware; AV1 is probed.
|
||||
@@ -445,8 +465,8 @@ text from an IME), are covered in [Input](/docs/input).
|
||||
|
||||
| Client | HDR | 4:4:4 | Surround 5.1 / 7.1 | Microphone | Clipboard | Stats overlay |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Linux desktop | ✅ ¹ | ❌ ¹³ | ✅ ² | ✅ | ❌ ³ | ✅ |
|
||||
| Windows desktop | ✅ ¹ | ❌ ¹³ | ✅ ² | ✅ | ⚠️ ⁴ | ✅ |
|
||||
| Linux desktop | ✅ ¹ | ⚠️ ¹³ | ✅ ² | ✅ | ❌ ³ | ✅ |
|
||||
| Windows desktop | ✅ ¹ | ⚠️ ¹³ | ✅ ² | ✅ | ⚠️ ⁴ | ✅ |
|
||||
| macOS | ⚠️ ⁵ | ⚠️ ⁶ | ✅ ² | ✅ | ✅ ⁷ | ✅ |
|
||||
| iPhone · iPad | ⚠️ ⁵ | ⚠️ ⁶ | ✅ ² | ✅ | ❌ ⁸ | ✅ |
|
||||
| Apple TV | ⚠️ ⁵ | ⚠️ ⁶ | ✅ ² | ❌ ⁹ | ❌ ⁸ | ✅ |
|
||||
@@ -475,9 +495,12 @@ text from an IME), are covered in [Input](/docs/input).
|
||||
until the host also enables its clipboard, but the client-side consent is pre-granted.
|
||||
11. Decided entirely by the host and layered into what Moonlight is offered.
|
||||
12. Moonlight has its own overlay; [stats](/docs/stats) here describes Punktfunk's.
|
||||
13. The desktop settings still show a **Full chroma (4:4:4)** switch, and it is a per-profile field
|
||||
— but the session binary never advertises the 4:4:4 capability, so the switch has no effect
|
||||
today and the stream stays 4:2:0.
|
||||
13. Opt-in — the per-profile **Full chroma (4:4:4)** switch, off by default — and advertised with
|
||||
no client-side probe, because every desktop decode rung can display it (note 4 under
|
||||
[Client decode](#client-decode)). The ⚠️ is the host half: full chroma needs HEVC on an
|
||||
**NVIDIA** host, or the PyroWave codec, which carries it on any vendor. On Windows it composes
|
||||
with HDR; on a Linux host 4:4:4 is 8-bit, so asking for both gives you full chroma in SDR. The
|
||||
stats overlay tells you which you got.
|
||||
|
||||
**File transfer through the clipboard does not exist yet** on any client. The wire format and the
|
||||
host-side policy for it are in place, but no client offers files, so a copied file never crosses.
|
||||
@@ -531,7 +554,7 @@ own so that adding a client-side feature never locks a client out of a deployed
|
||||
| Contract | Current | What it governs |
|
||||
|---|---|---|
|
||||
| `punktfunk/1` wire version | **2** | The `Hello`/`Welcome` handshake and the session planes. Hosts equality-check it, so this is the one that must match. |
|
||||
| C ABI version | **13** | The embeddable C surface a client links against. It grows far more often than the wire does. |
|
||||
| C ABI version | **14** | The embeddable C surface a client links against. It grows far more often than the wire does. |
|
||||
| Virtual-display driver protocol (Windows) | **6** (accepts **3** and up) | Between the Windows host and its display driver, so an older driver keeps working after a host update. |
|
||||
| Windows virtual-gamepad channel | **3** | Between the host and its pad driver. |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user