feat(codec): make client codec selection real — GPU-aware native advertisement

The clients' codec pickers sent a preference the host threw away: host_wire_caps() hardcoded HEVC for every GPU backend, so the native path never emitted H.264/AV1 regardless of what the user chose. Host: advertise what the backend actually encodes, mirroring the GameStream serverinfo logic — software openh264 emits H.264; probed backends (Linux VAAPI, Windows AMF/QSV) advertise their per-GPU CodecSupport via the new wire_mask(); NVENC keeps the Moonlight-validated H.264|HEVC|AV1 static superset; an empty probe falls back to the superset so auto clients still resolve HEVC. Gate 10-bit to HEVC (like the 4:4:4 gate) now that a client can steer the codec — Main10 is the only 10-bit encode path. Fix the web-console stats label hardcoded to "hevc" (new Codec::label(), shared with the GameStream register_session mapping). Android: replace the hardcoded H264|HEVC Hello advertisement with a videoCodecs param fed by VideoDecoders.decodableCodecBits() (AV1 bit only when a real hardware video/av01 decoder exists — the decode loop was already mime-driven); offer AV1 in the codec picker on capable devices. Decky: add the missing "Video codec" dropdown (auto/hevc/h264/av1) to the plugin settings, round-tripping the same codec key the flatpak client reads. Apple: unchanged by design (AnnexB.swift is NAL-only, AV1 is never advertised); refresh the stale "hosts don't emit AV1 on the native path yet" comments here and host-side. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 16:45:30 +02:00
parent 0290bf7285
commit 838a1239cf
15 changed files with 225 additions and 41 deletions
@@ -194,9 +194,10 @@ final class SessionModel: ObservableObject {
            if want444, canDecode444 {
                videoCaps |= PunktfunkConnection.videoCap444
            }
-            // This client's VideoToolbox path decodes H.264 and HEVC (AV1 isn't wired — hosts don't
-            // emit it on the native path yet). The host resolves the emitted codec from these + the
-            // soft `preferredCodec`; `resolvedCodec` reflects what it chose.
+            // This client's VideoToolbox path decodes H.264 and HEVC (AV1 depacketization isn't
+            // wired — AnnexB.swift is NAL-only — so it must never be advertised here). The host
+            // resolves the emitted codec from these + the soft `preferredCodec`; `resolvedCodec`
+            // reflects what it chose.
            let videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
            let result = Result { try PunktfunkConnection(
                host: host.address, port: host.port,
@@ -38,8 +38,8 @@ enum SettingsOptions {
        HUDPlacement.allCases.map { ($0.label, $0.rawValue) }

    /// Video-codec preference (`DefaultsKey.codec`) — a soft preference the host falls back from.
-    /// No AV1: this client's VideoToolbox path decodes H.264/HEVC only (hosts don't emit AV1 on
-    /// the native path yet).
+    /// No AV1: this client's VideoToolbox path decodes H.264/HEVC only (AnnexB.swift is NAL-only),
+    /// so it never advertises AV1 — offering it here would be a dead setting.
    static let codecs: [(label: String, tag: String)] = [
        ("Automatic", "auto"),
        ("HEVC (H.265)", "hevc"),
@@ -6,8 +6,8 @@
 // buffers whose NALs are 4-byte-length-prefixed. This file converts between the two, for
 // the codec the host resolved in the Welcome (`connection.videoCodec`) — HEVC and H.264
 // differ only in NAL-header layout and which parameter sets exist (HEVC adds a VPS). AV1
-// is not an Annex-B/NAL codec and isn't handled here (hosts don't emit it on the native
-// path yet).
+// is not an Annex-B/NAL codec and isn't handled here — this client never advertises it in
+// the Hello, so a host never emits it at us.
 //
 // HOT PATH: both pumps run `formatDescription(fromIDR:codec:)` + `sampleBuffer(au:format:codec:)`
 // once per AU, so the conversion is built on `forEachNAL` — a zero-copy scan over the AU's bytes