unom/punktfunk
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 8 Wiki Activity

feat(protocol,clients): codec preference negotiation + Linux client decodes per Welcome (Phase 2a)

Browse Source 
Adds a client-selectable **preferred codec** and wires the core + ABI + probe + Linux client to
negotiate and decode it. (Windows/Apple/Android follow in 2b.)

**Core:**
- `Hello.preferred_codec` (a single CODEC_* bit, 0 = auto) — a soft hint appended after
  `video_codecs`. `resolve_codec(client, host, preferred)` now honors the preference when the host
  can also emit it, else falls back to precedence (HEVC > AV1 > H.264). Roundtrip + preference tests.
- `NativeClient::connect` takes `video_codecs` + `preferred_codec`; `NativeClient.codec` exposes the
  resolved `Welcome.codec`.
- ABI: `punktfunk_connect_ex7` (adds the two codec params; `ex6` delegates to it advertising
  HEVC-only) + `punktfunk_connection_codec` getter + `PUNKTFUNK_CODEC_{H264,HEVC,AV1}` constants
  (drift-guarded against the wire values). Header regenerated.

**Host:** passes `hello.preferred_codec` into `resolve_codec`.

**probe:** `--codec h264|hevc|av1|auto` sets the preference (still advertises it can decode all
three); the dump extension already follows the resolved codec.

**Linux client:** advertises the codecs FFmpeg can actually decode (`decodable_codecs()`), threads
the user's `codec` setting as the preference, and builds the decoder — both the software and VAAPI
paths, plus the mid-session VAAPI→software demotion — from the negotiated `Welcome.codec` instead of
hardcoding HEVC. New "Video codec" dropdown in Preferences (Automatic/HEVC/H.264/AV1).

Live-validated on the dev box: probe `--codec hevc` against a software (H.264-only) host resolves to
H.264 (graceful soft-preference fallback), no failure. clippy + core (57) + host (133) tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Enrico Bühler
2026-07-02 00:13:26 +00:00
parent ffc0b07b46
commit 12843fe253
36 changed files with 529 additions and 144 deletions
Download Patch File Download Diff File Expand all files Collapse all files
include/punktfunk_core.h
+55 -2
View File
@@ -133,6 +133,16 @@
// [`punktfunk_connection_chroma_format`] reports the real value. (Mirrors `quic::VIDEO_CAP_444`.)
#define PUNKTFUNK_VIDEO_CAP_444 4
// Codec bit for [`punktfunk_connect_ex7`] (`video_codecs` / `preferred_codec`) and the value
// [`punktfunk_connection_codec`] returns: H.264 / AVC. (Mirrors `quic::CODEC_H264`.)
#define PUNKTFUNK_CODEC_H264 1
// Codec bit: H.265 / HEVC — the default codec. (Mirrors `quic::CODEC_HEVC`.)
#define PUNKTFUNK_CODEC_HEVC 2
// Codec bit: AV1. (Mirrors `quic::CODEC_AV1`.)
#define PUNKTFUNK_CODEC_AV1 4
// 16-byte AEAD authentication tag appended by GCM.
#define TAG_LEN 16
@@ -986,8 +996,8 @@ PunktfunkConnection *punktfunk_connect_ex5(const char *host,
// Like [`punktfunk_connect_ex5`], but additionally requests the audio channel count:
// `2` (stereo, the default behaviour of every earlier variant), `6` (5.1) or `8` (7.1). The host
// clamps the request to what it can actually capture and echoes the resolved count via
// [`punktfunk_connection_audio_channels`]; the `0xC9` audio frames are Opus-(multi)stream encoded
// for that layout. A client that wants surround calls this; everything else inherits stereo.
// [`punktfunk_connection_audio_channels`]. Advertises HEVC-only with no codec preference (call
// [`punktfunk_connect_ex7`] to negotiate the codec).
//
// # Safety
// Same as [`punktfunk_connect`].
@@ -1009,6 +1019,36 @@ PunktfunkConnection *punktfunk_connect_ex6(const char *host,
uint32_t timeout_ms);
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Like [`punktfunk_connect_ex6`], but additionally advertises the codecs the client can decode
// (`video_codecs` — a bitfield of [`PUNKTFUNK_CODEC_H264`] / [`PUNKTFUNK_CODEC_HEVC`] /
// [`PUNKTFUNK_CODEC_AV1`]) and a soft `preferred_codec` (a single codec bit, `0` = no preference).
// The host resolves the codec it emits from these (preference honored when it can also produce it,
// else best shared codec) and reports it via [`punktfunk_connection_codec`]. A client that omits
// this (calls `ex6`) advertises HEVC-only, no preference — the pre-negotiation behaviour.
//
// # Safety
// Same as [`punktfunk_connect`].
PunktfunkConnection *punktfunk_connect_ex7(const char *host,
uint16_t port,
uint32_t width,
uint32_t height,
uint32_t refresh_hz,
uint32_t compositor,
uint32_t gamepad,
uint32_t bitrate_kbps,
uint8_t video_caps,
uint8_t audio_channels,
uint8_t video_codecs,
uint8_t preferred_codec,
const char *launch_id,
const uint8_t *pin_sha256,
uint8_t *observed_sha256_out,
const char *client_cert_pem,
const char *client_key_pem,
uint32_t timeout_ms);
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Generate a persistent client identity: a self-signed certificate + private key, both
// PEM, NUL-terminated, written into the caller's buffers. Generate ONCE, store both
@@ -1179,6 +1219,19 @@ PunktfunkStatus punktfunk_connection_color_info(PunktfunkConnection *c,
PunktfunkStatus punktfunk_connection_chroma_format(PunktfunkConnection *c, uint8_t *out);
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Read the video codec the host resolved for this session (from its Welcome): one of
// [`PUNKTFUNK_CODEC_H264`] / [`PUNKTFUNK_CODEC_HEVC`] / [`PUNKTFUNK_CODEC_AV1`]. The embedder builds
// its decoder from THIS (never assuming HEVC). `*out` is filled when non-NULL. Available
// immediately after a successful connect (it doesn't change without a reconfigure). An older host
// that didn't negotiate a codec reports [`PUNKTFUNK_CODEC_HEVC`].
//
// # Safety
// `c` is a valid connection handle; `out` is NULL or writable for one `u8`.
PunktfunkStatus punktfunk_connection_codec(PunktfunkConnection *c,
uint8_t *out);
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// Send one input event to the host as a QUIC datagram (non-blocking enqueue).
//