feat(codec): make client codec selection real — GPU-aware native advertisement
apple / swift (push) Successful in 1m7s
release / apple (push) Successful in 8m27s
windows-host / package (push) Successful in 7m59s
apple / screenshots (push) Failing after 2m38s
android / android (push) Successful in 4m44s
ci / rust (push) Failing after 46s
arch / build-publish (push) Successful in 5m32s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 4m44s
ci / bench (push) Successful in 5m7s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
decky / build-publish (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m14s
docker / deploy-docs (push) Successful in 13s
apple / swift (push) Successful in 1m7s
release / apple (push) Successful in 8m27s
windows-host / package (push) Successful in 7m59s
apple / screenshots (push) Failing after 2m38s
android / android (push) Successful in 4m44s
ci / rust (push) Failing after 46s
arch / build-publish (push) Successful in 5m32s
ci / web (push) Successful in 50s
ci / docs-site (push) Successful in 1m2s
deb / build-publish (push) Successful in 4m44s
ci / bench (push) Successful in 5m7s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 5s
decky / build-publish (push) Successful in 13s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 4s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 3s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 4s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 3s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 11m9s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 11m14s
docker / deploy-docs (push) Successful in 13s
The clients' codec pickers sent a preference the host threw away: host_wire_caps() hardcoded HEVC for every GPU backend, so the native path never emitted H.264/AV1 regardless of what the user chose. Host: advertise what the backend actually encodes, mirroring the GameStream serverinfo logic — software openh264 emits H.264; probed backends (Linux VAAPI, Windows AMF/QSV) advertise their per-GPU CodecSupport via the new wire_mask(); NVENC keeps the Moonlight-validated H.264|HEVC|AV1 static superset; an empty probe falls back to the superset so auto clients still resolve HEVC. Gate 10-bit to HEVC (like the 4:4:4 gate) now that a client can steer the codec — Main10 is the only 10-bit encode path. Fix the web-console stats label hardcoded to "hevc" (new Codec::label(), shared with the GameStream register_session mapping). Android: replace the hardcoded H264|HEVC Hello advertisement with a videoCodecs param fed by VideoDecoders.decodableCodecBits() (AV1 bit only when a real hardware video/av01 decoder exists — the decode loop was already mime-driven); offer AV1 in the codec picker on capable devices. Decky: add the missing "Video codec" dropdown (auto/hevc/h264/av1) to the plugin settings, round-tripping the same codec key the flatpak client reads. Apple: unchanged by design (AnnexB.swift is NAL-only, AV1 is never advertised); refresh the stale "hosts don't emit AV1 on the native path yet" comments here and host-side. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -3,6 +3,7 @@ package io.unom.punktfunk
|
||||
import android.content.Context
|
||||
import io.unom.punktfunk.kit.Gamepad
|
||||
import io.unom.punktfunk.kit.NativeBridge
|
||||
import io.unom.punktfunk.kit.VideoDecoders
|
||||
import io.unom.punktfunk.kit.security.ClientIdentity
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.withContext
|
||||
@@ -41,7 +42,10 @@ suspend fun connectToHost(
|
||||
host, port, w, h, hz,
|
||||
identity.certPem, identity.privateKeyPem, pinHex,
|
||||
settings.bitrateKbps, settings.compositor, gamepadPref,
|
||||
hdrEnabled, settings.audioChannels, settings.preferredCodec(), timeoutMs,
|
||||
hdrEnabled, settings.audioChannels,
|
||||
// What this device can decode (H.264|HEVC always, AV1 when a real decoder exists) +
|
||||
// the user's soft codec preference — the host resolves the emitted codec from both.
|
||||
VideoDecoders.decodableCodecBits(), settings.preferredCodec(), timeoutMs,
|
||||
launch,
|
||||
)
|
||||
}
|
||||
|
||||
@@ -26,8 +26,9 @@ data class Settings(
|
||||
/** Requested audio channel count: 2 (stereo), 6 (5.1) or 8 (7.1). The host clamps to what it
|
||||
* can capture; the resolved count drives the decoder + AAudio layout. */
|
||||
val audioChannels: Int = 2,
|
||||
/** Preferred video codec: `"auto"` (host decides), `"hevc"`, or `"h264"`. A soft preference — the
|
||||
* host emits it when it can, else falls back. AMediaCodec decodes whichever the host resolves. */
|
||||
/** Preferred video codec: `"auto"` (host decides), `"hevc"`, `"h264"`, or `"av1"`. A soft
|
||||
* preference — the host emits it when it can, else falls back. AMediaCodec decodes whichever
|
||||
* the host resolves (AV1 is only advertised/offered when the device has a real AV1 decoder). */
|
||||
val codec: String = "auto",
|
||||
val micEnabled: Boolean = false,
|
||||
/**
|
||||
@@ -271,14 +272,17 @@ val AUDIO_CHANNEL_OPTIONS = listOf(
|
||||
8 to "7.1 Surround",
|
||||
)
|
||||
|
||||
/** (stored value, label) for the preferred video codec. `"auto"` = host decides. */
|
||||
/** (stored value, label) for the preferred video codec. `"auto"` = host decides. The `"av1"` row
|
||||
* only makes sense on a device with a real AV1 decoder — SettingsScreen filters it out otherwise. */
|
||||
val CODEC_OPTIONS = listOf(
|
||||
"auto" to "Automatic",
|
||||
"hevc" to "HEVC (H.265)",
|
||||
"h264" to "H.264 (AVC)",
|
||||
"av1" to "AV1",
|
||||
)
|
||||
|
||||
/** The [Settings.codec] string as a `quic::CODEC_*` preference byte (`0` = auto). H264=1, HEVC=2. */
|
||||
/** The [Settings.codec] string as a `quic::CODEC_*` preference byte (`0` = auto). H264=1, HEVC=2,
|
||||
* AV1=4. */
|
||||
fun Settings.preferredCodec(): Int = when (codec) {
|
||||
"h264" -> 1
|
||||
"hevc" -> 2
|
||||
|
||||
@@ -65,6 +65,7 @@ import androidx.compose.ui.graphics.vector.ImageVector
|
||||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.core.content.ContextCompat
|
||||
import io.unom.punktfunk.kit.VideoDecoders
|
||||
|
||||
/**
|
||||
* Stream settings, organised as an iOS-Settings / Android-system-settings style list of category
|
||||
@@ -300,7 +301,12 @@ private fun DisplaySettings(s: Settings, update: (Settings) -> Unit, context: an
|
||||
update(s.copy(bitrateKbps = kbps))
|
||||
}
|
||||
|
||||
SettingDropdown(label = "Video codec", options = CODEC_OPTIONS, selected = s.codec) { c ->
|
||||
// AV1 is only offered when the device has a real AV1 decoder (it's never advertised to the
|
||||
// host otherwise, so preferring it would be a dead setting). A stored "av1" from a capable
|
||||
// device stays visible so the selection is always representable.
|
||||
val av1Capable = remember { VideoDecoders.pickDecoder("video/av01") != null }
|
||||
val codecOptions = CODEC_OPTIONS.filter { (v, _) -> v != "av1" || av1Capable || s.codec == "av1" }
|
||||
SettingDropdown(label = "Video codec", options = codecOptions, selected = s.codec) { c ->
|
||||
update(s.copy(codec = c))
|
||||
}
|
||||
|
||||
|
||||
@@ -48,6 +48,9 @@ object NativeBridge {
|
||||
gamepadPref: Int,
|
||||
hdrEnabled: Boolean,
|
||||
audioChannels: Int,
|
||||
/** `quic::CODEC_*` bitfield of codecs this device decodes ([VideoDecoders.decodableCodecBits]);
|
||||
* `0` falls back to H.264|HEVC. The host resolves the emitted codec from this ∩ its GPU. */
|
||||
videoCodecs: Int,
|
||||
/** Preferred video codec as a `quic::CODEC_*` bit (`0` = auto). Soft — the host falls back. */
|
||||
preferredCodec: Int,
|
||||
timeoutMs: Int,
|
||||
|
||||
@@ -44,6 +44,15 @@ object VideoDecoders {
|
||||
* Pick the best decoder for [mime] (`"video/hevc"` / `"video/avc"` / `"video/av01"`), or `null`
|
||||
* to let the platform resolve its default. Enumerates once — call at stream start.
|
||||
*/
|
||||
/**
|
||||
* The `quic::CODEC_*` bitfield of codecs this device can decode, advertised in the Hello so the
|
||||
* host never emits a codec the decode loop can't open: H.264 (1) and HEVC (2) always (universal
|
||||
* on Android hardware), plus AV1 (4) only when [pickDecoder] finds a real (hardware, non-blocked)
|
||||
* `video/av01` decoder. Enumerates `MediaCodecList` — call at connect time, not per frame.
|
||||
*/
|
||||
fun decodableCodecBits(): Int =
|
||||
1 or 2 or (if (pickDecoder("video/av01") != null) 4 else 0)
|
||||
|
||||
fun pickDecoder(mime: String): DecoderChoice? {
|
||||
if (mime.isEmpty()) return null
|
||||
val infos = runCatching { MediaCodecList(MediaCodecList.REGULAR_CODECS).codecInfos }
|
||||
|
||||
@@ -76,6 +76,7 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
|
||||
gamepad_pref: jint,
|
||||
hdr_enabled: jboolean,
|
||||
audio_channels: jint,
|
||||
video_codecs: jint,
|
||||
preferred_codec: jint,
|
||||
timeout_ms: jint,
|
||||
launch: JString<'local>,
|
||||
@@ -142,10 +143,23 @@ pub extern "system" fn Java_io_unom_punktfunk_kit_NativeBridge_nativeConnect<'lo
|
||||
// decoder + AAudio layout (read in `crate::audio::AudioPlayback::start`). Anything else
|
||||
// normalizes to stereo here.
|
||||
punktfunk_core::audio::normalize_channels(audio_channels.clamp(0, u8::MAX as jint) as u8),
|
||||
// Codecs this device can decode — AMediaCodec decodes both HEVC and H.264 (AV1 isn't wired;
|
||||
// hosts don't emit it on the native path yet). The host resolves the emitted codec from these
|
||||
// + the soft `preferred_codec` and echoes it in `connector.codec`, which drives the mime below.
|
||||
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC,
|
||||
// Codecs this device can decode, ranked on the Kotlin side (`VideoDecoders.decodableCodecBits`:
|
||||
// H.264 + HEVC always, AV1 when a real `video/av01` decoder exists — AMediaCodec is
|
||||
// mime-driven, see `codec_mime`). Mask to the known bits and fall back to the pre-AV1
|
||||
// H.264|HEVC pair on 0 so a bogus value can't advertise nothing and kill the handshake.
|
||||
// The host resolves the emitted codec from these + the soft `preferred_codec` and echoes it
|
||||
// in `connector.codec`, which drives the mime below.
|
||||
{
|
||||
let bits = (video_codecs.clamp(0, u8::MAX as jint) as u8)
|
||||
& (punktfunk_core::quic::CODEC_H264
|
||||
| punktfunk_core::quic::CODEC_HEVC
|
||||
| punktfunk_core::quic::CODEC_AV1);
|
||||
if bits == 0 {
|
||||
punktfunk_core::quic::CODEC_H264 | punktfunk_core::quic::CODEC_HEVC
|
||||
} else {
|
||||
bits
|
||||
}
|
||||
},
|
||||
preferred_codec.clamp(0, u8::MAX as jint) as u8,
|
||||
launch, // a store-qualified library id to boot into a game, or None for the desktop
|
||||
pin, // Some → Crypto on host-fp mismatch
|
||||
|
||||
@@ -194,9 +194,10 @@ final class SessionModel: ObservableObject {
|
||||
if want444, canDecode444 {
|
||||
videoCaps |= PunktfunkConnection.videoCap444
|
||||
}
|
||||
// This client's VideoToolbox path decodes H.264 and HEVC (AV1 isn't wired — hosts don't
|
||||
// emit it on the native path yet). The host resolves the emitted codec from these + the
|
||||
// soft `preferredCodec`; `resolvedCodec` reflects what it chose.
|
||||
// This client's VideoToolbox path decodes H.264 and HEVC (AV1 depacketization isn't
|
||||
// wired — AnnexB.swift is NAL-only — so it must never be advertised here). The host
|
||||
// resolves the emitted codec from these + the soft `preferredCodec`; `resolvedCodec`
|
||||
// reflects what it chose.
|
||||
let videoCodecs = PunktfunkConnection.codecH264 | PunktfunkConnection.codecHEVC
|
||||
let result = Result { try PunktfunkConnection(
|
||||
host: host.address, port: host.port,
|
||||
|
||||
@@ -38,8 +38,8 @@ enum SettingsOptions {
|
||||
HUDPlacement.allCases.map { ($0.label, $0.rawValue) }
|
||||
|
||||
/// Video-codec preference (`DefaultsKey.codec`) — a soft preference the host falls back from.
|
||||
/// No AV1: this client's VideoToolbox path decodes H.264/HEVC only (hosts don't emit AV1 on
|
||||
/// the native path yet).
|
||||
/// No AV1: this client's VideoToolbox path decodes H.264/HEVC only (AnnexB.swift is NAL-only),
|
||||
/// so it never advertises AV1 — offering it here would be a dead setting.
|
||||
static let codecs: [(label: String, tag: String)] = [
|
||||
("Automatic", "auto"),
|
||||
("HEVC (H.265)", "hevc"),
|
||||
|
||||
@@ -6,8 +6,8 @@
|
||||
// buffers whose NALs are 4-byte-length-prefixed. This file converts between the two, for
|
||||
// the codec the host resolved in the Welcome (`connection.videoCodec`) — HEVC and H.264
|
||||
// differ only in NAL-header layout and which parameter sets exist (HEVC adds a VPS). AV1
|
||||
// is not an Annex-B/NAL codec and isn't handled here (hosts don't emit it on the native
|
||||
// path yet).
|
||||
// is not an Annex-B/NAL codec and isn't handled here — this client never advertises it in
|
||||
// the Hello, so a host never emits it at us.
|
||||
//
|
||||
// HOT PATH: both pumps run `formatDescription(fromIDR:codec:)` + `sampleBuffer(au:format:codec:)`
|
||||
// once per AU, so the conversion is built on `forEachNAL` — a zero-copy scan over the AU's bytes
|
||||
|
||||
@@ -669,7 +669,7 @@ class Plugin:
|
||||
# The client's own defaults (native display, host-default bitrate, auto pad).
|
||||
return {
|
||||
"width": 0, "height": 0, "refresh_hz": 0, "bitrate_kbps": 0,
|
||||
"gamepad": "auto", "compositor": "auto",
|
||||
"codec": "auto", "gamepad": "auto", "compositor": "auto",
|
||||
"inhibit_shortcuts": True, "mic_enabled": False,
|
||||
}
|
||||
|
||||
|
||||
@@ -61,13 +61,14 @@ export interface RunnerInfo {
|
||||
}
|
||||
|
||||
// The slice of the flatpak client's settings JSON this UI surfaces. The file can hold more
|
||||
// keys (codec, decoder, … set from the desktop client's own UI) — they round-trip untouched
|
||||
// keys (decoder, … set from the desktop client's own UI) — they round-trip untouched
|
||||
// because get_settings returns the whole parsed file and patches are object spreads.
|
||||
export interface StreamSettings {
|
||||
width: number; // 0 = native
|
||||
height: number; // 0 = native
|
||||
refresh_hz: number; // 0 = native
|
||||
bitrate_kbps: number; // 0 = host default
|
||||
codec?: string; // "auto" | "hevc" | "h264" | "av1" — soft preference (absent in pre-codec files)
|
||||
gamepad: string; // "auto" | "xbox360" | "xboxone" | "dualsense" | "dualshock4" | "steamdeck"
|
||||
compositor: string; // "auto" | "kwin" | "wlroots" | "mutter" | "gamescope"
|
||||
inhibit_shortcuts: boolean;
|
||||
|
||||
@@ -34,6 +34,15 @@ const GAMEPAD_LABELS: Record<string, string> = {
|
||||
dualshock4: "DualShock 4",
|
||||
steamdeck: "Steam Deck",
|
||||
};
|
||||
// Mirrors the desktop client's picker (ui_settings.rs CODECS) — a soft preference the host
|
||||
// falls back from when its GPU can't encode it.
|
||||
const CODECS = ["auto", "hevc", "h264", "av1"];
|
||||
const CODEC_LABELS: Record<string, string> = {
|
||||
auto: "Automatic",
|
||||
hevc: "HEVC (H.265)",
|
||||
h264: "H.264 (AVC)",
|
||||
av1: "AV1",
|
||||
};
|
||||
const COMPOSITORS = ["auto", "kwin", "wlroots", "mutter", "gamescope"];
|
||||
const COMPOSITOR_LABELS: Record<string, string> = {
|
||||
auto: "Automatic",
|
||||
@@ -108,6 +117,21 @@ export const SettingsSection: FC = () => {
|
||||
valueSuffix=" Mbit/s"
|
||||
onChange={(v) => patch({ bitrate_kbps: v * 1000 })}
|
||||
/>
|
||||
<Field
|
||||
label="Video codec"
|
||||
description="Preferred stream codec — the host falls back when its GPU can't encode it"
|
||||
childrenContainerWidth="max"
|
||||
>
|
||||
<RowActions>
|
||||
<div style={selectShell}>
|
||||
<Dropdown
|
||||
rgOptions={CODECS.map((c) => ({ data: c, label: CODEC_LABELS[c] ?? c }))}
|
||||
selectedOption={s.codec ?? "auto"}
|
||||
onChange={(o) => patch({ codec: o.data as string })}
|
||||
/>
|
||||
</div>
|
||||
</RowActions>
|
||||
</Field>
|
||||
<Field
|
||||
label="Gamepad type"
|
||||
description="Which virtual controller the host creates for your inputs"
|
||||
|
||||
@@ -77,14 +77,68 @@ impl Codec {
|
||||
}
|
||||
|
||||
/// The `quic` codec bitfield the host can currently **emit** on the punktfunk/1 native path,
|
||||
/// given the resolved encode backend. The GPU-less software encoder (openh264) produces H.264
|
||||
/// only; every GPU backend emits HEVC today (per-GPU H.264/AV1 negotiation on the native path is
|
||||
/// future work — GameStream already negotiates codecs with Moonlight separately). Fed to
|
||||
/// given the resolved encode backend — the same GPU-aware advertisement GameStream builds for
|
||||
/// Moonlight ([`crate::gamestream::serverinfo`]), in `quic::CODEC_*` bits. The GPU-less software
|
||||
/// encoder (openh264) produces H.264 only; the probed backends (Linux VAAPI, Windows AMF/QSV)
|
||||
/// advertise exactly what the GPU encodes ([`vaapi_codec_support`] / [`windows_codec_support`] —
|
||||
/// AV1 encode is narrow, an old iGPU might lack HEVC); NVENC keeps the Moonlight-validated
|
||||
/// static superset. An empty probe means the GPU wasn't usable at probe time (GPU-less CI,
|
||||
/// wrong-vendor pref), not that it encodes nothing — fall back to the superset so `resolve_codec`
|
||||
/// still lands on HEVC for an auto client, exactly the pre-probe behaviour. Fed to
|
||||
/// [`punktfunk_core::quic::resolve_codec`] against the client's advertised codecs.
|
||||
pub fn host_wire_caps() -> u8 {
|
||||
match crate::config::config().encoder_pref.as_str() {
|
||||
"software" | "sw" | "openh264" => punktfunk_core::quic::CODEC_H264,
|
||||
_ => punktfunk_core::quic::CODEC_HEVC,
|
||||
/// The static GPU superset (H.264 | HEVC | AV1) — mirrors the GameStream
|
||||
/// `SERVER_CODEC_MODE_SUPPORT` advertisement for the unprobed backends.
|
||||
const GPU_SUPERSET: u8 = punktfunk_core::quic::CODEC_H264
|
||||
| punktfunk_core::quic::CODEC_HEVC
|
||||
| punktfunk_core::quic::CODEC_AV1;
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
if matches!(
|
||||
crate::config::config().encoder_pref.as_str(),
|
||||
"software" | "sw" | "openh264"
|
||||
) {
|
||||
return punktfunk_core::quic::CODEC_H264;
|
||||
}
|
||||
if linux_zero_copy_is_vaapi() {
|
||||
if let Some(m) = vaapi_codec_support().wire_mask() {
|
||||
return m;
|
||||
}
|
||||
}
|
||||
// NVENC (static superset, like GameStream) — or an empty VAAPI probe (see above).
|
||||
GPU_SUPERSET
|
||||
}
|
||||
#[cfg(target_os = "windows")]
|
||||
{
|
||||
if windows_resolved_backend() == WindowsBackend::Software {
|
||||
return punktfunk_core::quic::CODEC_H264;
|
||||
}
|
||||
if windows_backend_is_probed() {
|
||||
if let Some(m) = windows_codec_support().wire_mask() {
|
||||
return m;
|
||||
}
|
||||
}
|
||||
// NVENC (static superset, like GameStream) — or an empty AMF/QSV probe (see above).
|
||||
GPU_SUPERSET
|
||||
}
|
||||
// The macOS dev/test host has no GPU encode backend — keep the pre-probe advertisement.
|
||||
#[cfg(not(any(target_os = "linux", target_os = "windows")))]
|
||||
{
|
||||
let _ = GPU_SUPERSET;
|
||||
match crate::config::config().encoder_pref.as_str() {
|
||||
"software" | "sw" | "openh264" => punktfunk_core::quic::CODEC_H264,
|
||||
_ => punktfunk_core::quic::CODEC_HEVC,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Lowercase stats/console label (`"h264"` / `"hevc"` / `"av1"`) — the codec string seeded into
|
||||
/// the web console's session meta ([`crate::stats_recorder::StatsRecorder::register_session`]).
|
||||
pub fn label(self) -> &'static str {
|
||||
match self {
|
||||
Codec::H264 => "h264",
|
||||
Codec::H265 => "hevc",
|
||||
Codec::Av1 => "av1",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -741,6 +795,27 @@ pub struct CodecSupport {
|
||||
pub av1: bool,
|
||||
}
|
||||
|
||||
#[cfg(any(target_os = "linux", target_os = "windows"))]
|
||||
impl CodecSupport {
|
||||
/// The probed codecs as a `quic::CODEC_*` bitfield, or `None` when the probe found nothing —
|
||||
/// meaning the GPU wasn't usable at probe time (GPU-less CI, a wrong-vendor pref), NOT that it
|
||||
/// encodes zero codecs; the caller then falls back to the static superset (the native-path
|
||||
/// analogue of `gamestream::serverinfo::probed_mask`).
|
||||
pub fn wire_mask(self) -> Option<u8> {
|
||||
let mut m = 0u8;
|
||||
if self.h264 {
|
||||
m |= punktfunk_core::quic::CODEC_H264;
|
||||
}
|
||||
if self.h265 {
|
||||
m |= punktfunk_core::quic::CODEC_HEVC;
|
||||
}
|
||||
if self.av1 {
|
||||
m |= punktfunk_core::quic::CODEC_AV1;
|
||||
}
|
||||
(m != 0).then_some(m)
|
||||
}
|
||||
}
|
||||
|
||||
/// Probe the active Linux GPU backend for its encodable codecs (cached; opens a tiny encoder per
|
||||
/// codec, once). Only the VAAPI (AMD/Intel) backend is probed — NVENC keeps its Moonlight-validated
|
||||
/// static advertisement (callers gate on [`linux_zero_copy_is_vaapi`]).
|
||||
@@ -1042,4 +1117,42 @@ mod tests {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The probed-capability → wire-bitfield mapping the native codec advertisement is built from.
|
||||
#[cfg(any(target_os = "linux", target_os = "windows"))]
|
||||
#[test]
|
||||
fn codec_support_wire_mask() {
|
||||
use punktfunk_core::quic::{CODEC_AV1, CODEC_H264, CODEC_HEVC};
|
||||
let all = CodecSupport {
|
||||
h264: true,
|
||||
h265: true,
|
||||
av1: true,
|
||||
};
|
||||
assert_eq!(all.wire_mask(), Some(CODEC_H264 | CODEC_HEVC | CODEC_AV1));
|
||||
let hevc_only = CodecSupport {
|
||||
h264: false,
|
||||
h265: true,
|
||||
av1: false,
|
||||
};
|
||||
assert_eq!(hevc_only.wire_mask(), Some(CODEC_HEVC));
|
||||
// An all-false probe means "GPU unusable at probe time", not "zero codecs" — `None` tells
|
||||
// the caller to advertise the static superset instead of refusing every handshake.
|
||||
let none = CodecSupport {
|
||||
h264: false,
|
||||
h265: false,
|
||||
av1: false,
|
||||
};
|
||||
assert_eq!(none.wire_mask(), None);
|
||||
}
|
||||
|
||||
/// Wire round-trip and the stats label stay in lockstep with the `quic::CODEC_*` bits.
|
||||
#[test]
|
||||
fn codec_wire_roundtrip_and_label() {
|
||||
for c in [Codec::H264, Codec::H265, Codec::Av1] {
|
||||
assert_eq!(Codec::from_wire(c.to_wire()), c);
|
||||
}
|
||||
assert_eq!(Codec::H264.label(), "h264");
|
||||
assert_eq!(Codec::H265.label(), "hevc");
|
||||
assert_eq!(Codec::Av1.label(), "av1");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -663,11 +663,7 @@ fn stream_body(
|
||||
// Web-console stats accumulation (active when `perf` OR a capture is armed): per-stage vectors
|
||||
// for p50/p99, the goodput bytes queued to the sender this window, the previous window's
|
||||
// dropped-frame count for delta computation, and the registration id cached on the first sample.
|
||||
let codec_name = match cfg.codec {
|
||||
Codec::H264 => "h264",
|
||||
Codec::H265 => "hevc",
|
||||
Codec::Av1 => "av1",
|
||||
};
|
||||
let codec_name = cfg.codec.label();
|
||||
let mut sid: Option<u32> = None;
|
||||
let (mut v_cap, mut v_enc, mut v_pkt, mut v_send): (Vec<u32>, Vec<u32>, Vec<u32>, Vec<u32>) =
|
||||
(Vec::new(), Vec::new(), Vec::new(), Vec::new());
|
||||
|
||||
@@ -732,9 +732,10 @@ async fn serve_session(
|
||||
// The pairing gate (require_pairing → paired? else park for delegated approval) ran above,
|
||||
// before this future, so a client reaching here is paired (or the host is `--open`).
|
||||
|
||||
// Codec negotiation: pick the one codec this host will emit (its backend capability ∩ the
|
||||
// client's advertised codecs). A GPU-less software host emits H.264, so an HEVC-only client
|
||||
// shares nothing with it → refuse honestly rather than send a stream it can't decode.
|
||||
// Codec negotiation: pick the one codec this host will emit (its GPU-probed backend
|
||||
// capability ∩ the client's advertised codecs, honoring the client's soft preference).
|
||||
// A GPU-less software host emits H.264 only, so an HEVC-only client shares nothing with
|
||||
// it → refuse honestly rather than send a stream it can't decode.
|
||||
let host_codecs = crate::encode::Codec::host_wire_caps();
|
||||
let codec_bit =
|
||||
punktfunk_core::quic::resolve_codec(hello.video_codecs, host_codecs, hello.preferred_codec)
|
||||
@@ -888,10 +889,16 @@ async fn serve_session(
|
||||
// Resolve the encode bit depth: HEVC Main10 only when the client advertised it AND the host
|
||||
// opted in (PUNKTFUNK_10BIT). A client that can't decode 10-bit (caps bit clear, or an older
|
||||
// client) always gets the 8-bit stream. PUNKTFUNK_10BIT is the host policy gate until a
|
||||
// mgmt/console toggle replaces it.
|
||||
// mgmt/console toggle replaces it. 10-bit is HEVC-only (like the 4:4:4 gate below): now that
|
||||
// the client can steer the codec to H.264/AV1, a non-HEVC session must stay 8-bit — the
|
||||
// encoders' 10-bit path is Main10, and AV1 10-bit stays off until live-confirmed (the same
|
||||
// stance as the GameStream Main10 advertisement).
|
||||
let host_wants_10bit = crate::config::config().ten_bit;
|
||||
let client_supports_10bit = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_10BIT != 0;
|
||||
let bit_depth: u8 = if host_wants_10bit && client_supports_10bit {
|
||||
let bit_depth: u8 = if host_wants_10bit
|
||||
&& client_supports_10bit
|
||||
&& codec == crate::encode::Codec::H265
|
||||
{
|
||||
10
|
||||
} else {
|
||||
8
|
||||
@@ -1010,8 +1017,9 @@ async fn serve_session(
|
||||
// The resolved audio channel count the audio thread will capture + Opus-(multi)stream
|
||||
// encode (2/6/8). The client builds its decoder from this echoed value.
|
||||
audio_channels,
|
||||
// The negotiated codec the encoder will emit (H.264 for a software host, else HEVC). The
|
||||
// client builds its decoder from this instead of assuming HEVC.
|
||||
// The negotiated codec the encoder will emit (client preference ∩ GPU capability;
|
||||
// HEVC-precedence tie-break). The client builds its decoder from this instead of
|
||||
// assuming HEVC.
|
||||
codec: codec_bit,
|
||||
};
|
||||
io::write_msg(&mut send, &welcome.encode()).await?;
|
||||
@@ -1027,7 +1035,7 @@ async fn serve_session(
|
||||
.await
|
||||
.map_err(|_| anyhow!("handshake timed out after {HANDSHAKE_TIMEOUT:?}"))??;
|
||||
let (mut ctrl_send, mut ctrl_recv) = (send, recv);
|
||||
// Negotiated codec (HEVC / H.264), derived from the Welcome. `Copy`, so the control task's
|
||||
// Negotiated codec (HEVC / H.264 / AV1), derived from the Welcome. `Copy`, so the control task's
|
||||
// `async move` captures a copy and it stays usable for the data-plane SessionContext below.
|
||||
let codec = crate::encode::Codec::from_wire(welcome.codec);
|
||||
let client_udp = std::net::SocketAddr::new(peer.ip(), start.client_udp_port);
|
||||
@@ -3080,8 +3088,9 @@ struct SessionContext {
|
||||
bit_depth: u8,
|
||||
/// Negotiated chroma subsampling (4:2:0, or 4:4:4 when the client + host + GPU all support it).
|
||||
chroma: crate::encode::ChromaFormat,
|
||||
/// Negotiated video codec the encoder emits (HEVC by default; H.264 for a software host). Also
|
||||
/// used to rebuild the encoder at the same codec across a mid-stream mode reconfigure.
|
||||
/// Negotiated video codec the encoder emits (HEVC by default; H.264 / AV1 when the client
|
||||
/// prefers one the GPU encodes; H.264 for a software host). Also used to rebuild the encoder
|
||||
/// at the same codec across a mid-stream mode reconfigure.
|
||||
codec: crate::encode::Codec,
|
||||
/// Speed-test burst requests (see [`service_probes`]).
|
||||
probe_rx: std::sync::mpsc::Receiver<ProbeRequest>,
|
||||
@@ -3236,7 +3245,7 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
|
||||
width: mode.width,
|
||||
height: mode.height,
|
||||
fps: mode.refresh_hz,
|
||||
codec: "hevc",
|
||||
codec: plan.codec.label(),
|
||||
client: client_label,
|
||||
bitrate_kbps,
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user