feat(protocol,host): negotiate video codec + add a GPU-less software (openh264) encode path

Phase 1 of codec negotiation, and the Linux software H.264 encode path it unblocks.

**Codec negotiation (core `quic`):**
- `Hello.video_codecs` (bitfield: CODEC_H264/HEVC/AV1) — the client advertises what it can
  decode; appended as a trailing byte (older client → 0 = HEVC-only, back-compat).
- `Welcome.codec` — the single codec the host resolved and will emit; trailing byte (older
  host → HEVC).
- `resolve_codec(client, host_capable)` picks the shared codec (precedence HEVC > AV1 > H.264)
  or `None` → the host refuses honestly rather than sending an undecodable stream.
- Roundtrip + back-compat tests; cbindgen exports the CODEC_* constants.

**Software encoder (host):**
- The openh264 `OpenH264Encoder` (was Windows-only) is now built on Linux too — it's
  platform-agnostic (consumes CPU RGB `CapturedFrame`s, statically-bundled openh264). `openh264`
  moved to the shared linux+windows Cargo target.
- `PUNKTFUNK_ENCODER=software` selects it: `open_video` gains a `software` branch (H.264 only),
  and `session_plan::resolve_encoder` / `capture::gpu_encode` resolve `EncoderBackend::Software`
  → `output_format().gpu = false`, so the portal capturer delivers CPU RGB. Explicit-only (auto
  never picks it — a box with a dead driver still has /dev/nvidiactl and would mis-resolve NVENC).

**Host codec resolution (`punktfunk1`):**
- The native path no longer hardcodes HEVC: it resolves the codec from the client's advertised
  set ∩ the host's capability (`Codec::host_wire_caps`: software→H.264, else HEVC), threads it
  through `SessionPlan.codec`, and opens the encoder + validates reconfigures at that codec. A
  software host + HEVC-only client is refused with a clear error.
- 4:4:4 is gated on HEVC (it's HEVC-only).

**Probe:** advertises H264|HEVC|AV1 and logs the resolved codec.

Validated on the GPU-less dev box: negotiation is live end-to-end (probe advertises 0x07 → host
resolves H.264 → Welcome reports it → plan = Software/H264), and the openh264 unit test (CPU RGB →
AnnexB IDR) now runs on Linux. Full capture→encode still needs a GPU on this box — every
compositor screencast path (KWin GL, gamescope VK_EXT_physical_device_drm, wlroots EGL) requires
one; software render (llvmpipe/pixman) can't be captured — so this box exercises negotiation +
encoder, not live capture. The software path unblocks GPU-less-*encode* boxes that still have a
display GPU. Phase 2 (clients advertising real codecs + decoding per Welcome.codec) is a follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-01 23:13:39 +00:00
parent e7b07d2363
commit ffc0b07b46
10 changed files with 331 additions and 30 deletions
+19 -4
View File
@@ -94,12 +94,19 @@ pub struct SessionPlan {
/// Handshake-negotiated chroma subsampling (4:2:0, or full-chroma 4:4:4 when the client + host +
/// GPU all support it). Resolved before the Welcome; `Yuv420` on every backend that declined it.
pub chroma: crate::encode::ChromaFormat,
/// Handshake-negotiated video codec the encoder emits — HEVC by default, H.264 for a GPU-less
/// software host (`resolve_codec` over the client's advertised codecs ∩ the host's capability).
pub codec: crate::encode::Codec,
}
impl SessionPlan {
/// Resolve the whole plan once from [`config`](crate::config) + the negotiated `bit_depth` and
/// `chroma`.
pub fn resolve(bit_depth: u8, chroma: crate::encode::ChromaFormat) -> Self {
/// Resolve the whole plan once from [`config`](crate::config) + the negotiated `bit_depth`,
/// `chroma`, and `codec`.
pub fn resolve(
bit_depth: u8,
chroma: crate::encode::ChromaFormat,
codec: crate::encode::Codec,
) -> Self {
SessionPlan {
capture: CaptureBackend::resolve(),
topology: resolve_topology(),
@@ -107,6 +114,7 @@ impl SessionPlan {
bit_depth,
hdr: bit_depth >= 10,
chroma,
codec,
}
}
@@ -154,5 +162,12 @@ fn resolve_encoder() -> EncoderBackend {
#[cfg(not(target_os = "windows"))]
fn resolve_encoder() -> EncoderBackend {
EncoderBackend::PlatformAuto
// `PUNKTFUNK_ENCODER=software` forces the GPU-less openh264 path — which must take CPU-staged
// capture (`EncoderBackend::Software.is_gpu() == false` → `output_format().gpu = false`), so the
// portal capturer delivers CPU RGB. Everything else stays `PlatformAuto` (NVENC/VAAPI resolved
// inside `encode::open_video`).
match crate::config::config().encoder_pref.as_str() {
"software" | "sw" | "openh264" => EncoderBackend::Software,
_ => EncoderBackend::PlatformAuto,
}
}