fix(encode): gate multi-slice frames on the client's decoder — the 0.17.0 Chromecast crash
ci / docs-site (push) Successful in 1m9s
apple / swift (push) Successful in 1m15s
ci / web (push) Successful in 3m16s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
deb / build-publish-client-arm64 (push) Successful in 2m13s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
ci / rust-arm64 (push) Successful in 3m34s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
deb / build-publish-host (push) Successful in 4m20s
docker / builders-arm64cross (push) Successful in 6s
docker / deploy-docs (push) Successful in 28s
deb / build-publish (push) Successful in 6m17s
arch / build-publish (push) Failing after 7m58s
flatpak / build-publish (push) Successful in 9m6s
ci / rust (push) Successful in 13m37s
release / apple (push) Successful in 16m32s
windows-host / package (push) Successful in 18m21s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m21s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m44s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m24s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m27s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 2m56s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 3m54s
apple / screenshots (push) Successful in 20m45s
android / android (push) Successful in 5m0s
ci / docs-site (push) Successful in 1m9s
apple / swift (push) Successful in 1m15s
ci / web (push) Successful in 3m16s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
deb / build-publish-client-arm64 (push) Successful in 2m13s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
ci / rust-arm64 (push) Successful in 3m34s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
deb / build-publish-host (push) Successful in 4m20s
docker / builders-arm64cross (push) Successful in 6s
docker / deploy-docs (push) Successful in 28s
deb / build-publish (push) Successful in 6m17s
arch / build-publish (push) Failing after 7m58s
flatpak / build-publish (push) Successful in 9m6s
ci / rust (push) Successful in 13m37s
release / apple (push) Successful in 16m32s
windows-host / package (push) Successful in 18m21s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m21s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m44s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m24s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m27s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 2m56s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 3m54s
apple / screenshots (push) Successful in 20m45s
android / android (push) Successful in 5m0s
Field report: since 0.17.0 a stream to a Chromecast with Google TV 4K freezes on the first frame and ~80% of the time crashes + reboots the DEVICE — with both the Punktfunk app and Moonlight, while an Xbox Series S is fine. Root cause: LN1 Phase 3 (67b79810) defaulted Linux direct-NVENC to 4 slices per frame for EVERY session. Amlogic HEVC decoders wedge on multi-slice AUs — exactly why moonlight-android requests slicesPerFrame=1 for every hardware decoder (4 only for software slice-threading) — and our RTSP parser never read the request. The Phase-3 commit recorded the untested leg ("a live Moonlight re-test joins the standing owed Moonlight item"); this report is that re-test. The slicing ceiling now belongs to the CLIENT, threaded as open_video's new max_slices from both planes: - GameStream: parse x-nv-video[0].videoEncoderSlicesPerFrame into StreamConfig and honor it; absent/out-of-range (pre-auth input) => 1. - punktfunk/1: new Hello cap VIDEO_CAP_MULTI_SLICE (0x80 — the byte's LAST free bit; the next cap needs a second byte + ABI bump). SessionPlan.max_slices = 32 with the bit, 1 without, applied to every encoder the plan opens so rebuilds can't change the wire shape. The desktop session client advertises it (FFmpeg/D3D11VA/Vulkan decode stacks are fine); Android/Apple stay off until they can decide per-decoder like Moonlight does — the cap is embedder-set decoder truth, never OR'd in by the shared pump. - Linux direct-NVENC clamps its Phase-3 default to the ceiling (resolve_slices(codec, 4.min(max_slices))) and logs slices/max_slices in the caps-probe line; PUNKTFUNK_NVENC_SLICES stays the explicit operator override in both directions. Windows keeps its single-slice default untouched. Also repairs the nvenc_cuda #[ignore] hardware tests:d2c46eafadded open()'s cursor_blend param without updating them, invisible because CI never compiles tests with the nvenc feature. Verified on .21 (RTX 5070 Ti): pf-encode + host check/clippy clean with nvenc; host unit suite 263/0; rtsp announce tests 7/7 incl. the new slicesPerFrame coverage; on-hardware smokes — default e2e still 4 chunks/frame, NEW single-slice client-ceiling test clamps + disarms chunked poll with no env involved, env escape unchanged. rustfmt clean. Windows leg (prepare_display param) is CI-only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -240,11 +240,17 @@ mod session_main {
|
||||
audio_channels: settings.audio_channels,
|
||||
preferred_codec: settings.preferred_codec(),
|
||||
// HDR off = don't advertise 10-bit/HDR at all; the host then never upgrades.
|
||||
video_caps: if settings.hdr_enabled {
|
||||
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
|
||||
} else {
|
||||
0
|
||||
},
|
||||
// MULTI_SLICE is decoder truth for THIS embedder: every desktop decode stack
|
||||
// (FFmpeg software, VAAPI, D3D11VA, Vulkan Video) handles AUs carrying several
|
||||
// slice NALs, so the host may keep its multi-slice low-latency default (§7 LN1).
|
||||
// The mobile/TV embedders must NOT copy this blindly — Amlogic MediaCodec wedges
|
||||
// on multi-slice AUs (see `VIDEO_CAP_MULTI_SLICE`), so they advertise per-decoder.
|
||||
video_caps: punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE
|
||||
| if settings.hdr_enabled {
|
||||
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
|
||||
} else {
|
||||
0
|
||||
},
|
||||
// No portable Wayland/X11 display-volume query yet, so the host keeps its EDID
|
||||
// defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session
|
||||
// pump) pins one manually.
|
||||
|
||||
@@ -795,10 +795,17 @@ pub struct NvencCudaEncoder {
|
||||
/// [`stream_ordered_requested`]). The per-frame gate additionally requires `pending` empty.
|
||||
stream_ordered: bool,
|
||||
/// Slice count the live session was configured with ([`resolve_slices`] — env override,
|
||||
/// else the Linux direct-NVENC default of 4 since Phase 3; 1 = the preset's single slice).
|
||||
/// else the Linux direct-NVENC default of 4 since Phase 3 clamped to
|
||||
/// [`max_slices`](Self::max_slices); 1 = the preset's single slice).
|
||||
/// Chunked poll needs ≥ 2 to have boundaries to cut at. Latched at init, consumed by
|
||||
/// `build_config` (so an in-place reconfigure presents the same slicing).
|
||||
slices: u32,
|
||||
/// Ceiling on the per-frame slice count the session's CLIENT decoder accepts (from
|
||||
/// negotiation: `VIDEO_CAP_MULTI_SLICE`, or GameStream's `videoEncoderSlicesPerFrame`).
|
||||
/// 1 = single-slice only — the safe shape toward decoders that never asked (Amlogic TV
|
||||
/// SoCs wedge on multi-slice AUs). Clamps the Phase-3 default; the explicit
|
||||
/// `PUNKTFUNK_NVENC_SLICES` env override still wins in both directions.
|
||||
max_slices: u32,
|
||||
/// `NV_ENC_CAPS_SUPPORT_SUBFRAME_READBACK` from the caps probe — gates the DEFAULT-on
|
||||
/// sub-frame arming (an unsupported GPU must not have `enableSubFrameWrite` forced into its
|
||||
/// init params, which could fail the session open). `PUNKTFUNK_NVENC_SUBFRAME=1` overrides.
|
||||
@@ -847,6 +854,7 @@ impl NvencCudaEncoder {
|
||||
bit_depth: u8,
|
||||
chroma: ChromaFormat,
|
||||
cursor_blend: bool,
|
||||
max_slices: u32,
|
||||
) -> Result<Self> {
|
||||
// The runtime `.so` load is the real "is NVENC possible here" gate: fail the open with a
|
||||
// clear reason instead of an opaque session error on the first frame.
|
||||
@@ -895,6 +903,8 @@ impl NvencCudaEncoder {
|
||||
io_stream: ptr::null_mut(),
|
||||
stream_ordered: false,
|
||||
slices: 1,
|
||||
// A zero from a misbehaving caller must not zero the resolver's default arithmetic.
|
||||
max_slices: max_slices.max(1),
|
||||
subframe_cap: false,
|
||||
subframe_on: false,
|
||||
subframe_forced: false,
|
||||
@@ -1094,9 +1104,14 @@ impl NvencCudaEncoder {
|
||||
// every Linux direct-NVENC session, resolved HERE (before the session opens) so the
|
||||
// config author, the init params and the chunked-poll latch all agree; the caps probe
|
||||
// gates the sub-frame default so a GPU without SUBFRAME_READBACK never has it forced
|
||||
// into its init params. PUNKTFUNK_NVENC_SLICES=1 / PUNKTFUNK_NVENC_SUBFRAME=0 are the
|
||||
// escapes.
|
||||
self.slices = resolve_slices(self.codec, 4);
|
||||
// into its init params. The Phase-3 default is CLAMPED to the session's negotiated
|
||||
// `max_slices` — the client-decoder ceiling (`VIDEO_CAP_MULTI_SLICE`, or GameStream's
|
||||
// `videoEncoderSlicesPerFrame`): a client that never asked for multi-slice AUs gets
|
||||
// single-slice frames, because TV-SoC decoders (Amlogic — Chromecast with Google TV)
|
||||
// wedge the whole device on frames carrying several slice NALs (the 0.17.0 field
|
||||
// regression). PUNKTFUNK_NVENC_SLICES / PUNKTFUNK_NVENC_SUBFRAME stay the explicit
|
||||
// operator overrides in both directions.
|
||||
self.slices = resolve_slices(self.codec, 4.min(self.max_slices));
|
||||
self.subframe_on = resolve_subframe(self.subframe_cap);
|
||||
self.subframe_forced = subframe_env_forced();
|
||||
tracing::info!(
|
||||
@@ -1105,6 +1120,8 @@ impl NvencCudaEncoder {
|
||||
yuv444 = self.yuv444_supported,
|
||||
subframe_readback = subframe != 0,
|
||||
dynamic_slice = dyn_slice != 0,
|
||||
slices = self.slices,
|
||||
max_slices = self.max_slices,
|
||||
max = %format!("{wmax}x{hmax}"),
|
||||
"NVENC (Linux direct) capabilities probed"
|
||||
);
|
||||
@@ -2559,6 +2576,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA session");
|
||||
|
||||
@@ -2657,6 +2675,7 @@ mod tests {
|
||||
10,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA session");
|
||||
|
||||
@@ -2737,6 +2756,7 @@ mod tests {
|
||||
10,
|
||||
ChromaFormat::Yuv420,
|
||||
true, // cursor_blend: bring up the Vulkan slot ring + the 10-bit blend
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA session");
|
||||
let cursor = |serial: u64, x: i32, y: i32| pf_frame::CursorOverlay {
|
||||
@@ -2802,6 +2822,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv444,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA 4:4:4 session");
|
||||
|
||||
@@ -2848,6 +2869,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA session");
|
||||
|
||||
@@ -2906,6 +2928,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
) else {
|
||||
eprintln!(
|
||||
"skipping rfi_declines_impossible_ranges: NVENC unavailable (no NVIDIA driver)"
|
||||
@@ -2933,6 +2956,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA encoder")
|
||||
}
|
||||
@@ -2968,6 +2992,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("open");
|
||||
for f in 0..4u32 {
|
||||
@@ -3106,6 +3131,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA session");
|
||||
let frame = nv12_frame(W, H, 0);
|
||||
@@ -3149,6 +3175,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
true, // cursor_blend: bring up the Vulkan slot ring + blend
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA session");
|
||||
let cursor = |serial: u64, x: i32, y: i32| pf_frame::CursorOverlay {
|
||||
@@ -3227,6 +3254,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA session");
|
||||
// Steady sync frames first (stream-ordered mode).
|
||||
@@ -3307,6 +3335,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA session");
|
||||
|
||||
@@ -3413,6 +3442,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("open NVENC CUDA session");
|
||||
|
||||
@@ -3483,6 +3513,49 @@ mod tests {
|
||||
assert!(!au.data.is_empty());
|
||||
}
|
||||
|
||||
/// ON-HARDWARE (RTX box `.21`): a session whose CLIENT ceiling is 1 slice (`max_slices` from
|
||||
/// negotiation — no cap bit / Moonlight `videoEncoderSlicesPerFrame:1`, the Chromecast field
|
||||
/// regression's fix) must encode single-slice frames with chunked poll disarmed, with NO env
|
||||
/// knobs involved. Run with `--test-threads=1` (env vars are process-global).
|
||||
#[test]
|
||||
#[ignore = "requires an NVIDIA GPU + driver — run manually on the RTX box (.21)"]
|
||||
fn nvenc_cuda_single_slice_client_ceiling() {
|
||||
const W: u32 = 1920;
|
||||
const H: u32 = 1080;
|
||||
// The ceiling under test is the negotiated one, not the operator override.
|
||||
std::env::remove_var("PUNKTFUNK_NVENC_SLICES");
|
||||
std::env::remove_var("PUNKTFUNK_NVENC_SUBFRAME");
|
||||
pf_zerocopy::cuda::make_current().expect("shared CUDA context current");
|
||||
let mut enc = NvencCudaEncoder::open(
|
||||
Codec::H265,
|
||||
PixelFormat::Nv12,
|
||||
W,
|
||||
H,
|
||||
60,
|
||||
20_000_000,
|
||||
true,
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
1, // the client never advertised multi-slice tolerance
|
||||
)
|
||||
.expect("open NVENC CUDA session");
|
||||
for i in 0..4u32 {
|
||||
let frame = nv12_frame(W, H, i);
|
||||
enc.submit_indexed(&frame, i).expect("submit");
|
||||
assert_eq!(
|
||||
enc.slices, 1,
|
||||
"a 1-slice client ceiling must clamp the Phase-3 default"
|
||||
);
|
||||
assert!(
|
||||
!enc.supports_chunked_poll(),
|
||||
"single-slice sessions have no boundaries — chunked poll must stay disarmed"
|
||||
);
|
||||
let au = enc.poll().expect("poll").expect("one AU per sync frame");
|
||||
assert!(!au.data.is_empty(), "frame {i} produced an empty AU");
|
||||
}
|
||||
}
|
||||
|
||||
/// ON-HARDWARE (RTX box `.21`): the Phase-3 default-on ESCAPES — `PUNKTFUNK_NVENC_SLICES=1`
|
||||
/// must fully disarm chunked poll (and `poll_chunk` degrades
|
||||
/// to exactly one self-closing whole-AU chunk (the default-path contract every non-chunked
|
||||
|
||||
@@ -45,11 +45,14 @@ pub(super) fn codec_guid(codec: Codec) -> nv::GUID {
|
||||
/// Resolved per-frame slice count for a session (latency plan §7 LN1, Phase 3): the
|
||||
/// `PUNKTFUNK_NVENC_SLICES` env override wins (1..=32; **1 = the explicit single-slice
|
||||
/// escape**, needed now that a backend can default higher), else the backend's
|
||||
/// `default_slices` — 4 on Linux direct-NVENC since the Phase-3 default-on, 1 everywhere else
|
||||
/// (the Windows async path is deliberately untouched). H.264/HEVC only (AV1 partitions via
|
||||
/// tiles). ONE parse shared by the config author ([`apply_low_latency_config`] via
|
||||
/// [`LowLatencyConfig::slices`]) and the Linux backend's chunked-poll arming, so the two can
|
||||
/// never disagree about whether a session is multi-slice.
|
||||
/// `default_slices` — on Linux direct-NVENC the Phase-3 default of 4 CLAMPED to the session's
|
||||
/// negotiated client-decoder ceiling (`VIDEO_CAP_MULTI_SLICE` / GameStream's
|
||||
/// `videoEncoderSlicesPerFrame` — a client that never asked stays single-slice: Amlogic TV
|
||||
/// SoCs wedge on multi-slice AUs), 1 everywhere else (the Windows async path is deliberately
|
||||
/// untouched). H.264/HEVC only (AV1 partitions via tiles). ONE parse shared by the config
|
||||
/// author ([`apply_low_latency_config`] via [`LowLatencyConfig::slices`]) and the Linux
|
||||
/// backend's chunked-poll arming, so the two can never disagree about whether a session is
|
||||
/// multi-slice.
|
||||
pub(super) fn resolve_slices(codec: Codec, default_slices: u32) -> u32 {
|
||||
if !matches!(codec, Codec::H264 | Codec::H265) {
|
||||
return 1;
|
||||
|
||||
@@ -174,6 +174,13 @@ pub fn open_video(
|
||||
// The session may hand this encoder cursor bitmaps to composite (cursor-as-metadata
|
||||
// captures). Backends whose fast path can't blend (Vulkan EFC RGB-direct) key off it.
|
||||
cursor_blend: bool,
|
||||
// Ceiling on the per-frame slice count this session's CLIENT decoder accepts: 1 =
|
||||
// single-slice only (the safe default toward decoders that never asked — Amlogic TV SoCs
|
||||
// wedge on multi-slice AUs), a Moonlight `videoEncoderSlicesPerFrame` request, or 32 (= no
|
||||
// client-side limit) for a `VIDEO_CAP_MULTI_SLICE` punktfunk/1 client. Backends that split
|
||||
// frames (Linux direct-NVENC, §7 LN1) clamp their default against it;
|
||||
// `PUNKTFUNK_NVENC_SLICES` stays the explicit operator override in both directions.
|
||||
max_slices: u32,
|
||||
) -> Result<Box<dyn Encoder>> {
|
||||
let (inner, backend) = open_video_backend(
|
||||
codec,
|
||||
@@ -186,6 +193,7 @@ pub fn open_video(
|
||||
bit_depth,
|
||||
chroma,
|
||||
cursor_blend,
|
||||
max_slices,
|
||||
)?;
|
||||
// Record what this session encodes on (the mgmt API's "currently used GPU"): the backend label
|
||||
// is reported by `open_video_backend` from the branch that ACTUALLY opened — not re-derived by
|
||||
@@ -337,6 +345,7 @@ fn open_video_backend_linux(
|
||||
bit_depth: u8,
|
||||
chroma: ChromaFormat,
|
||||
cursor_blend: bool,
|
||||
max_slices: u32,
|
||||
) -> Result<(Box<dyn Encoder>, &'static str)> {
|
||||
// A NEGOTIATED PyroWave session (client advertised + preferred it, plan §3) routes
|
||||
// straight to that backend — the PUNKTFUNK_ENCODER pref below stays a lab override.
|
||||
@@ -446,6 +455,7 @@ fn open_video_backend_linux(
|
||||
bit_depth,
|
||||
chroma,
|
||||
cursor_blend,
|
||||
max_slices,
|
||||
)
|
||||
.map(|e| (e, "nvenc"))
|
||||
};
|
||||
@@ -562,8 +572,11 @@ fn open_video_backend(
|
||||
bit_depth: u8,
|
||||
chroma: ChromaFormat,
|
||||
cursor_blend: bool,
|
||||
max_slices: u32,
|
||||
) -> Result<(Box<dyn Encoder>, &'static str)> {
|
||||
let _ = cursor_blend; // consumed only by the Linux vulkan-encode + direct-NVENC arms below
|
||||
// Consumed only by the Linux vulkan-encode + direct-NVENC arms below (max_slices: the
|
||||
// direct-NVENC arm — the one backend that splits frames).
|
||||
let _ = (cursor_blend, max_slices);
|
||||
validate_dimensions(codec, width, height)?;
|
||||
// Refresh/fps must be positive and sane: fps feeds the encoder time_base (`Rational(1, fps)`)
|
||||
// and the pts→ns conversion (`pts * 1e9 / fps`), so 0 builds a 1/0 rational / divides by zero.
|
||||
@@ -600,6 +613,7 @@ fn open_video_backend(
|
||||
bit_depth,
|
||||
chroma,
|
||||
cursor_blend,
|
||||
max_slices,
|
||||
)
|
||||
}
|
||||
#[cfg(target_os = "windows")]
|
||||
@@ -816,6 +830,7 @@ fn open_video_backend(
|
||||
cuda,
|
||||
bit_depth,
|
||||
chroma,
|
||||
max_slices,
|
||||
);
|
||||
anyhow::bail!("video encode requires Linux or Windows")
|
||||
}
|
||||
@@ -840,14 +855,16 @@ fn open_nvenc_probed(
|
||||
bit_depth: u8,
|
||||
chroma: ChromaFormat,
|
||||
cursor_blend: bool,
|
||||
max_slices: u32,
|
||||
) -> Result<Box<dyn Encoder>> {
|
||||
// Consumed by the direct-SDK arm below.
|
||||
#[cfg(not(feature = "nvenc"))]
|
||||
let _ = cursor_blend; // consumed by the direct-SDK arm below
|
||||
// Direct-SDK NVENC (design/linux-direct-nvenc.md): the DEFAULT on NVIDIA, and only for a CUDA
|
||||
// capture payload (it registers CUDADEVICEPTR inputs — a CPU/dmabuf frame can't feed it, so those
|
||||
// keep the libav path; and `cuda` is false on AMD/Intel, so they stay on VAAPI). Set
|
||||
// PUNKTFUNK_NVENC_DIRECT=0 to fall back to libav. It self-clamps the bitrate internally (its own
|
||||
// level-ceiling binary search at session open), so it skips the probe-loop stepping below.
|
||||
let _ = (cursor_blend, max_slices);
|
||||
// Direct-SDK NVENC (design/linux-direct-nvenc.md): the DEFAULT on NVIDIA, and only for a CUDA
|
||||
// capture payload (it registers CUDADEVICEPTR inputs — a CPU/dmabuf frame can't feed it, so those
|
||||
// keep the libav path; and `cuda` is false on AMD/Intel, so they stay on VAAPI). Set
|
||||
// PUNKTFUNK_NVENC_DIRECT=0 to fall back to libav. It self-clamps the bitrate internally (its own
|
||||
// level-ceiling binary search at session open), so it skips the probe-loop stepping below.
|
||||
#[cfg(feature = "nvenc")]
|
||||
if cuda && nvenc_direct_enabled() {
|
||||
tracing::info!(
|
||||
@@ -865,6 +882,7 @@ fn open_nvenc_probed(
|
||||
bit_depth,
|
||||
chroma,
|
||||
cursor_blend,
|
||||
max_slices,
|
||||
)?) as Box<dyn Encoder>);
|
||||
}
|
||||
// The silent-degrade trap: a build without `--features nvenc` compiles the direct-SDK
|
||||
@@ -2317,6 +2335,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
)
|
||||
.expect("software arm must open GPU-free");
|
||||
assert_eq!(label, "software");
|
||||
@@ -2335,6 +2354,7 @@ mod tests {
|
||||
8,
|
||||
ChromaFormat::Yuv420,
|
||||
false,
|
||||
4,
|
||||
) {
|
||||
// `expect_err` needs `Ok: Debug` and `Box<dyn Encoder>` isn't — match instead.
|
||||
Ok(_) => panic!("software emits H.264 only; an H.265 session must be refused"),
|
||||
|
||||
@@ -136,6 +136,9 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
|
||||
// video frame indexes. STREAMED_AU the same way: the shared reassembler accepts
|
||||
// sentinel-headed streamed blocks (retro-validated at the final block), so the host
|
||||
// may overlap a multi-slice encode's tail with packetize/FEC/pacing.
|
||||
// VIDEO_CAP_MULTI_SLICE must NOT join this unconditional set: it is DECODER
|
||||
// truth (Amlogic MediaCodec wedges on multi-slice AUs — the 0.17.0 Chromecast
|
||||
// regression), so only the embedder can set it, per its decode stack.
|
||||
video_caps: video_caps
|
||||
| crate::quic::VIDEO_CAP_HOST_TIMING
|
||||
| crate::quic::VIDEO_CAP_PROBE_SEQ
|
||||
|
||||
@@ -55,6 +55,19 @@ pub const VIDEO_CAP_STREAMED_AU: u8 = 0x20;
|
||||
/// performance choice — both AEADs are full-strength, and Hello/Welcome ride the pinned-TLS
|
||||
/// control channel, so there is no downgrade surface.
|
||||
pub const VIDEO_CAP_CHACHA20: u8 = 0x40;
|
||||
/// [`Hello::video_caps`] bit: the client's decoder accepts **multi-slice access units** — H.264/
|
||||
/// HEVC frames carrying several slice NALs (latency plan §7 LN1: the encoder splits frames so
|
||||
/// sub-frame readback can ship early slices while the tail encodes). Decoder-level, so the
|
||||
/// EMBEDDER sets it from what its decode stack actually handles: the desktop clients' FFmpeg/
|
||||
/// D3D11VA/Vulkan-video decoders are fine, but mobile/TV MediaCodec is per-SoC — Amlogic HEVC
|
||||
/// decoders (Chromecast with Google TV, Fire TV) wedge the whole DEVICE on multi-slice frames
|
||||
/// (the 0.17.0 field regression: the 4-slice Linux default froze streams on first frame and
|
||||
/// watchdog-rebooted the CCwGTV), which is exactly why Moonlight requests 1 slice per frame for
|
||||
/// every hardware decoder. The host defaults to >1 slice ONLY toward a client that sets this
|
||||
/// bit (`PUNKTFUNK_NVENC_SLICES` stays the explicit operator override in both directions);
|
||||
/// every other client gets single-slice frames — the pre-0.17 wire shape. NOTE: this takes the
|
||||
/// video_caps byte's last free bit — the next video cap needs a second byte (ABI bump).
|
||||
pub const VIDEO_CAP_MULTI_SLICE: u8 = 0x80;
|
||||
|
||||
/// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
|
||||
/// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
|
||||
|
||||
@@ -627,6 +627,7 @@ mod session_tests {
|
||||
codec: crate::encode::Codec::H265,
|
||||
min_fec: 0,
|
||||
hdr: false,
|
||||
slices: 1, // the no-request default — hardware decoders get single-slice AUs
|
||||
});
|
||||
|
||||
assert!(state.end_session("test"), "video was live");
|
||||
|
||||
@@ -556,6 +556,17 @@ fn stream_config(map: &HashMap<String, String>) -> Option<StreamConfig> {
|
||||
let min_fec = parse_u("x-nv-vqos[0].fec.minRequiredFecPackets")
|
||||
.unwrap_or(2)
|
||||
.min(16) as u8;
|
||||
// The client's requested per-frame slice count (moonlight-common-c SdpGenerator.c:
|
||||
// `videoEncoderSlicesPerFrame`) — 1 for every HARDWARE decoder, 4 only for software
|
||||
// decoders (slice-threading). Honor it as the encoder's slicing ceiling: GFE/Sunshine
|
||||
// encode what was asked, and hardware TV decoders (Amlogic — Chromecast with Google TV)
|
||||
// wedge the whole DEVICE on multi-slice AUs they never requested — the 0.17.0 field
|
||||
// regression, where the Linux direct-NVENC 4-slice default (§7 LN1) ignored this key and
|
||||
// froze + watchdog-rebooted CCwGTV clients on the first frame. Absent or out-of-range
|
||||
// (attacker-controlled pre-auth input) ⇒ 1, the universally-safe single-slice shape.
|
||||
let slices = parse_u("x-nv-video[0].videoEncoderSlicesPerFrame")
|
||||
.filter(|n| (1..=32).contains(n))
|
||||
.unwrap_or(1);
|
||||
Some(StreamConfig {
|
||||
width,
|
||||
height,
|
||||
@@ -565,6 +576,7 @@ fn stream_config(map: &HashMap<String, String>) -> Option<StreamConfig> {
|
||||
codec,
|
||||
min_fec,
|
||||
hdr,
|
||||
slices,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -727,6 +739,30 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
/// `videoEncoderSlicesPerFrame` is honored as the encoder's slicing ceiling: Moonlight sends
|
||||
/// 1 for every hardware decoder (Amlogic TV SoCs wedge on multi-slice AUs — the 0.17.0
|
||||
/// Chromecast regression) and 4 for software decoders. Absent or out-of-range (pre-auth,
|
||||
/// attacker-controlled) must fall back to the universally-safe 1, never reject the session.
|
||||
#[test]
|
||||
fn announce_slices_per_frame() {
|
||||
// Absent (very old client) → single-slice.
|
||||
assert_eq!(stream_config(&announce(&[])).unwrap().slices, 1);
|
||||
// The two real Moonlight values pass through verbatim.
|
||||
for want in ["1", "4"] {
|
||||
let map = announce(&[("x-nv-video[0].videoEncoderSlicesPerFrame", want)]);
|
||||
assert_eq!(
|
||||
stream_config(&map).unwrap().slices,
|
||||
want.parse::<u32>().unwrap()
|
||||
);
|
||||
}
|
||||
// Garbage / out-of-range degrades to 1 (still streams — this key must never kill a session).
|
||||
for bad in ["0", "33", "999999", "-1", "x"] {
|
||||
let map = announce(&[("x-nv-video[0].videoEncoderSlicesPerFrame", bad)]);
|
||||
let cfg = stream_config(&map).expect("session must still negotiate");
|
||||
assert_eq!(cfg.slices, 1, "slicesPerFrame {bad} must degrade to 1");
|
||||
}
|
||||
}
|
||||
|
||||
/// Audio negotiation: numChannels/AudioQuality/packetDuration, with Moonlight defaults.
|
||||
#[test]
|
||||
fn announce_audio_params() {
|
||||
|
||||
@@ -31,6 +31,12 @@ pub struct StreamConfig {
|
||||
/// Drives the capturer's proactive advanced-color enable; the encoder picks Main10 from the captured
|
||||
/// (P010) frame format. Always `false` on a non-HDR host, so the SDR path is unchanged.
|
||||
pub hdr: bool,
|
||||
/// Client's `x-nv-video[0].videoEncoderSlicesPerFrame` — the per-frame slice count its
|
||||
/// decoder wants (moonlight-common-c requests 1 for every HARDWARE decoder and 4 only for
|
||||
/// software slice-threading). Honored as the encoder's slicing ceiling: hardware TV decoders
|
||||
/// (Amlogic — Chromecast with Google TV) wedge the whole device on multi-slice AUs they
|
||||
/// never asked for (the 0.17.0 4-slice-default field regression). Absent ⇒ 1.
|
||||
pub slices: u32,
|
||||
}
|
||||
|
||||
/// A pooled capturer plus the three properties reuse must match on — its HDR-ness, its
|
||||
@@ -1040,6 +1046,8 @@ fn stream_body(
|
||||
// True only when THIS session's capture negotiated cursor-as-metadata — which the
|
||||
// callers grant only where the resolved backend composites (`cursor_blend_capable`).
|
||||
cursor_blend,
|
||||
// The client's requested slices-per-frame (its DECODER's ceiling) — see `StreamConfig`.
|
||||
cfg.slices,
|
||||
)
|
||||
.context("open video encoder for stream")?;
|
||||
// Tell the encoder how deep the capturer lets it pipeline. Without this an in-place backend
|
||||
@@ -1212,6 +1220,7 @@ fn stream_body(
|
||||
gs_bit_depth(frame.format),
|
||||
encode::ChromaFormat::Yuv420, // GameStream stays 4:2:0
|
||||
cursor_blend, // same capture cursor mode — see the first open
|
||||
cfg.slices, // client slicing ceiling — see the first open
|
||||
)
|
||||
.context("reopen encoder after rebuild")?;
|
||||
// A rebuilt encoder starts unconfigured — same reason as the first open above.
|
||||
|
||||
@@ -1454,6 +1454,10 @@ async fn serve_session(
|
||||
// Streamed-AU capability: the client's reassembler accepts sentinel-headed streamed blocks,
|
||||
// so a chunked encoder session may ship an AU's early FEC blocks while its tail encodes.
|
||||
let streamed_au = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_STREAMED_AU != 0;
|
||||
// Multi-slice capability: the client's DECODER accepts AUs carrying several slice NALs, so
|
||||
// the encoder may keep its multi-slice default (§7 LN1). Absent ⇒ single-slice frames —
|
||||
// TV-SoC decoders (Amlogic: Chromecast with Google TV) wedge the device on multi-slice AUs.
|
||||
let multi_slice = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE != 0;
|
||||
let stats_dp = stats; // data-plane handle to the shared stats recorder
|
||||
// Short label for web-console stats captures: the client's cert-fingerprint prefix, else its
|
||||
// peer IP (no fingerprint = anonymous TOFU/--open client).
|
||||
@@ -1571,6 +1575,7 @@ async fn serve_session(
|
||||
cursor_client_draws: cursor_client_draws_dp,
|
||||
probe_seq,
|
||||
streamed_au,
|
||||
multi_slice,
|
||||
stats: stats_dp,
|
||||
client_label,
|
||||
client_name,
|
||||
|
||||
@@ -579,6 +579,10 @@ pub(super) async fn negotiate(
|
||||
// The bit the Welcome just advertised — read back rather than recomputed, so the
|
||||
// prepared display and the session wiring cannot disagree with it.
|
||||
let cursor_fw = welcome.host_caps & punktfunk_core::quic::HOST_CAP_CURSOR != 0;
|
||||
// Same bit the data plane's SessionContext reads — the prepared plan and the
|
||||
// session wiring must agree on the slicing ceiling (an encoder rebuilt from the
|
||||
// prepared plan with a DIFFERENT max_slices would change the wire shape mid-flow).
|
||||
let multi_slice = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE != 0;
|
||||
let (mode, shard_payload) = (hello.mode, welcome.shard_payload);
|
||||
// "Automatic" — `bitrate_kbps` above is the host's own answer for `mode`, so the build
|
||||
// may re-resolve it if the source turns out to deliver a different size. Sampled here
|
||||
@@ -594,6 +598,7 @@ pub(super) async fn negotiate(
|
||||
client_identity,
|
||||
client_hdr,
|
||||
cursor_fw,
|
||||
multi_slice,
|
||||
bitrate_kbps,
|
||||
bitrate_auto,
|
||||
bit_depth,
|
||||
|
||||
@@ -992,6 +992,12 @@ pub(super) struct SessionContext {
|
||||
/// AU's FEC blocks under sentinel headers as the slices complete instead of waiting for the
|
||||
/// whole AU. `false` = older client — chunks (if any) are drained whole-AU, zero wire change.
|
||||
pub(super) streamed_au: bool,
|
||||
/// The client advertised [`punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE`]: its decoder
|
||||
/// accepts multi-slice AUs, so the session's encoder may keep its multi-slice default
|
||||
/// (§7 LN1 — becomes [`SessionPlan::max_slices`](crate::session_plan::SessionPlan)).
|
||||
/// `false` = single-slice frames, the pre-0.17 wire shape TV-SoC decoders (Amlogic —
|
||||
/// Chromecast with Google TV) require to not wedge.
|
||||
pub(super) multi_slice: bool,
|
||||
/// Shared streaming-stats recorder. The capture loop reads `is_armed()` per frame to decide
|
||||
/// whether to measure the per-stage split; the send thread builds + pushes the aggregated
|
||||
/// `StatsSample` at its 2 s boundary.
|
||||
@@ -1092,6 +1098,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
|
||||
ctx.bit_depth,
|
||||
),
|
||||
ctx.cursor_forward,
|
||||
ctx.multi_slice,
|
||||
);
|
||||
// gamescope: the XFixes cursor source feeds the host-side composite (Phase C) — unless the
|
||||
// spawned gamescope paints the pointer into its node itself, in which case the reader would
|
||||
@@ -1141,6 +1148,8 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
|
||||
cursor_client_draws,
|
||||
probe_seq,
|
||||
streamed_au,
|
||||
// Already folded into `plan.max_slices` by the resolve above — nothing below reads it.
|
||||
multi_slice: _,
|
||||
stats,
|
||||
client_label,
|
||||
client_name,
|
||||
@@ -2044,6 +2053,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
|
||||
bit_depth,
|
||||
plan.chroma,
|
||||
plan.cursor_blend,
|
||||
plan.max_slices,
|
||||
) {
|
||||
Ok(mut new_enc) => {
|
||||
// The fresh encoder may have clamped to its codec-level ceiling —
|
||||
@@ -3393,6 +3403,7 @@ fn try_inplace_resize(
|
||||
bit_depth,
|
||||
plan.chroma,
|
||||
plan.cursor_blend,
|
||||
plan.max_slices,
|
||||
) {
|
||||
Ok(e) => e,
|
||||
Err(e) => {
|
||||
@@ -3450,6 +3461,7 @@ pub(super) fn prepare_display(
|
||||
client_identity: Option<[u8; 32]>,
|
||||
client_hdr: Option<punktfunk_core::quic::HdrMeta>,
|
||||
cursor_forward: bool,
|
||||
multi_slice: bool,
|
||||
bitrate_kbps: u32,
|
||||
// Passed through to [`build_pipeline`] — see its parameter of the same name.
|
||||
bitrate_auto: bool,
|
||||
@@ -3478,6 +3490,7 @@ pub(super) fn prepare_display(
|
||||
bit_depth,
|
||||
),
|
||||
cursor_forward,
|
||||
multi_slice,
|
||||
);
|
||||
plan.gamescope_cursor =
|
||||
crate::session_plan::gamescope_cursor_for(compositor == pf_vdisplay::Compositor::Gamescope);
|
||||
@@ -3899,6 +3912,7 @@ fn build_pipeline(
|
||||
bit_depth,
|
||||
plan.chroma,
|
||||
plan.cursor_blend,
|
||||
plan.max_slices,
|
||||
)
|
||||
.context("open video encoder")?;
|
||||
if let Some(t) = trace {
|
||||
|
||||
@@ -132,6 +132,13 @@ pub struct SessionPlan {
|
||||
/// redundant work producing a SECOND pointer. Resolved by [`cursor_blend_for`]'s sibling so
|
||||
/// the two answers cannot disagree.
|
||||
pub gamescope_cursor: bool,
|
||||
/// Ceiling on the encoder's per-frame slice count, from the client's
|
||||
/// [`VIDEO_CAP_MULTI_SLICE`](punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE): 32 (= no
|
||||
/// client-side limit, the backend picks its own multi-slice default, §7 LN1) when the bit is
|
||||
/// set, 1 (single-slice frames — the pre-0.17 wire shape TV-SoC decoders like Amlogic
|
||||
/// require) when it isn't. Applied to EVERY encoder this plan opens (initial + all rebuilds)
|
||||
/// so the slicing can never change shape across a mode/bitrate/stall rebuild.
|
||||
pub max_slices: u32,
|
||||
}
|
||||
|
||||
impl SessionPlan {
|
||||
@@ -143,6 +150,7 @@ impl SessionPlan {
|
||||
codec: crate::encode::Codec,
|
||||
cursor_blend: bool,
|
||||
cursor_forward: bool,
|
||||
multi_slice: bool,
|
||||
) -> Self {
|
||||
SessionPlan {
|
||||
capture: CaptureBackend::resolve(),
|
||||
@@ -158,6 +166,7 @@ impl SessionPlan {
|
||||
// Set by the resolve callers (they know the compositor); default off keeps every
|
||||
// non-gamescope plan unchanged.
|
||||
gamescope_cursor: false,
|
||||
max_slices: if multi_slice { 32 } else { 1 },
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -150,6 +150,7 @@ pub fn run(opts: Options) -> Result<()> {
|
||||
8, // spike synthetic harness: 8-bit
|
||||
encode::ChromaFormat::Yuv420, // ...and 4:2:0
|
||||
false, // synthetic frames carry no cursor
|
||||
4, // no client decoder — keep the backend's multi-slice default
|
||||
)
|
||||
.context("open encoder")?;
|
||||
|
||||
|
||||
@@ -528,6 +528,22 @@
|
||||
#define VIDEO_CAP_CHACHA20 64
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// [`Hello::video_caps`] bit: the client's decoder accepts **multi-slice access units** — H.264/
|
||||
// HEVC frames carrying several slice NALs (latency plan §7 LN1: the encoder splits frames so
|
||||
// sub-frame readback can ship early slices while the tail encodes). Decoder-level, so the
|
||||
// EMBEDDER sets it from what its decode stack actually handles: the desktop clients' FFmpeg/
|
||||
// D3D11VA/Vulkan-video decoders are fine, but mobile/TV MediaCodec is per-SoC — Amlogic HEVC
|
||||
// decoders (Chromecast with Google TV, Fire TV) wedge the whole DEVICE on multi-slice frames
|
||||
// (the 0.17.0 field regression: the 4-slice Linux default froze streams on first frame and
|
||||
// watchdog-rebooted the CCwGTV), which is exactly why Moonlight requests 1 slice per frame for
|
||||
// every hardware decoder. The host defaults to >1 slice ONLY toward a client that sets this
|
||||
// bit (`PUNKTFUNK_NVENC_SLICES` stays the explicit operator override in both directions);
|
||||
// every other client gets single-slice frames — the pre-0.17 wire shape. NOTE: this takes the
|
||||
// video_caps byte's last free bit — the next video cap needs a second byte (ABI bump).
|
||||
#define VIDEO_CAP_MULTI_SLICE 128
|
||||
#endif
|
||||
|
||||
#if defined(PUNKTFUNK_FEATURE_QUIC)
|
||||
// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
|
||||
// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
|
||||
|
||||
Reference in New Issue
Block a user