feat(clients): codec preference on Windows/Apple/Android clients (Phase 2b)
Rounds out codec negotiation across the last three clients — each advertises what it can decode, builds its decoder from the resolved Welcome.codec, and exposes a "Video codec" preference picker. **Windows** (Rust, mirrors Linux): `decodable_codecs()` + `ffmpeg_codec_id()`; the D3D11VA and software FFmpeg decoders (and the mid-session D3D11VA→software demotion) open the negotiated codec instead of hardcoding HEVC; settings gain a `codec` field + reactor ComboBox; `--codec` CLI flag. **Apple** (Swift/C-ABI): AnnexB is now codec-aware — a `VideoCodec` enum drives H.264 vs HEVC NAL parsing / parameter-set extraction (`CMVideoFormatDescriptionCreateFromH264ParameterSets` for H.264, no VPS) and AVCC repacking; `PunktfunkConnection` advertises H264|HEVC via `punktfunk_connect_ex7`, reads `resolvedCodec` (`punktfunk_connection_codec`), and threads `videoCodec` into the stage-1/2 pipelines + `VideoDecoder`; SettingsView "Video codec" Picker (auto/HEVC/H.264). AV1 is left out (hosts don't emit it on the native path, and it's not an AnnexB codec). Test call sites updated. **Android** (Kotlin + Rust JNI): the JNI `nativeConnect` gains `preferredCodec`; the native decode loop picks the AMediaCodec MIME (`video/hevc`|`video/avc`) from `connector.codec` and advertises H264|HEVC; Settings `codec` field + Compose dropdown. Core/host/probe/Linux clippy + tests green (unchanged from 2a). Windows/Apple/Android compile on their platform CI (this Linux box can't build them — Windows toolchain / Xcode / the Android NDK's opus-cmake toolchain). All follow the Linux client's validated pattern. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -54,6 +54,10 @@ public final class VideoDecoder: @unchecked Sendable {
|
||||
/// depth / HDR). Read inside `createSessionLocked` under `lock`.
|
||||
private var chroma444 = false
|
||||
|
||||
/// The negotiated codec (`connection.videoCodec`), set once at session start. Drives the AnnexB
|
||||
/// NAL parsing (H.264 vs HEVC parameter sets). Read under `lock`.
|
||||
private var codec: VideoCodec = .hevc
|
||||
|
||||
public init(
|
||||
onDecoded: @escaping @Sendable (ReadyFrame) -> Void,
|
||||
onDecodeError: @escaping @Sendable (OSStatus) -> Void = { _ in }
|
||||
@@ -73,6 +77,14 @@ public final class VideoDecoder: @unchecked Sendable {
|
||||
lock.unlock()
|
||||
}
|
||||
|
||||
/// Select the negotiated codec (H.264 vs HEVC). Call once at session start, before decoding,
|
||||
/// from `connection.videoCodec`. Thread-safe.
|
||||
public func setCodec(_ c: VideoCodec) {
|
||||
lock.lock()
|
||||
codec = c
|
||||
lock.unlock()
|
||||
}
|
||||
|
||||
/// Submit one AU for asynchronous decode, (re)creating the session if `format` changed. The
|
||||
/// caller resolves `format` from the IDR exactly as stage-1 does (`AnnexB.formatDescription`).
|
||||
/// Returns false if the session couldn't be created or the frame couldn't be submitted.
|
||||
@@ -93,7 +105,7 @@ public final class VideoDecoder: @unchecked Sendable {
|
||||
// invalidate the session between here and DecodeFrame. The VT output callback takes the
|
||||
// ring lock, not this one, so there's no re-entrancy. DecodeFrame is async — non-blocking.
|
||||
guard let session,
|
||||
let sample = AnnexB.sampleBuffer(au: au, format: newFormat)
|
||||
let sample = AnnexB.sampleBuffer(au: au, format: newFormat, codec: codec)
|
||||
else { lock.unlock(); return false }
|
||||
var infoOut = VTDecodeInfoFlags()
|
||||
let status = VTDecompressionSessionDecodeFrame(
|
||||
|
||||
Reference in New Issue
Block a user