fix(encode): gate multi-slice frames on the client's decoder — the 0.17.0 Chromecast crash
ci / docs-site (push) Successful in 1m9s
apple / swift (push) Successful in 1m15s
ci / web (push) Successful in 3m16s
docker / builders (--build-arg FEDORA_VERSION=44, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm, -f44) (push) Successful in 7s
deb / build-publish-client-arm64 (push) Successful in 2m13s
docker / builders (ci/arch-ci.Dockerfile, punktfunk-arch-ci) (push) Successful in 5s
docker / builders (ci/android-ci.Dockerfile, punktfunk-android-ci) (push) Successful in 7s
ci / rust-arm64 (push) Successful in 3m34s
docker / builders (ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 5s
docker / builders (ci/rust-ci-noble.Dockerfile, punktfunk-rust-ci-noble) (push) Successful in 8s
docker / builders (ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / apps (., web/Dockerfile, punktfunk-web) (push) Successful in 11s
docker / apps (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 13s
deb / build-publish-host (push) Successful in 4m20s
docker / builders-arm64cross (push) Successful in 6s
docker / deploy-docs (push) Successful in 28s
deb / build-publish (push) Successful in 6m17s
arch / build-publish (push) Failing after 7m58s
flatpak / build-publish (push) Successful in 9m6s
ci / rust (push) Successful in 13m37s
release / apple (push) Successful in 16m32s
windows-host / package (push) Successful in 18m21s
windows-host / winget-source (push) Skipped
windows-host / canary-manifest (push) Successful in 32s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 17m21s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m44s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 20m24s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m27s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 2m56s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 3m54s
apple / screenshots (push) Successful in 20m45s
android / android (push) Successful in 5m0s

Field report: since 0.17.0 a stream to a Chromecast with Google TV 4K
freezes on the first frame and ~80% of the time crashes + reboots the
DEVICE — with both the Punktfunk app and Moonlight, while an Xbox
Series S is fine. Root cause: LN1 Phase 3 (67b79810) defaulted Linux
direct-NVENC to 4 slices per frame for EVERY session. Amlogic HEVC
decoders wedge on multi-slice AUs — exactly why moonlight-android
requests slicesPerFrame=1 for every hardware decoder (4 only for
software slice-threading) — and our RTSP parser never read the request.
The Phase-3 commit recorded the untested leg ("a live Moonlight re-test
joins the standing owed Moonlight item"); this report is that re-test.

The slicing ceiling now belongs to the CLIENT, threaded as open_video's
new max_slices from both planes:
- GameStream: parse x-nv-video[0].videoEncoderSlicesPerFrame into
  StreamConfig and honor it; absent/out-of-range (pre-auth input) => 1.
- punktfunk/1: new Hello cap VIDEO_CAP_MULTI_SLICE (0x80 — the byte's
  LAST free bit; the next cap needs a second byte + ABI bump).
  SessionPlan.max_slices = 32 with the bit, 1 without, applied to every
  encoder the plan opens so rebuilds can't change the wire shape. The
  desktop session client advertises it (FFmpeg/D3D11VA/Vulkan decode
  stacks are fine); Android/Apple stay off until they can decide
  per-decoder like Moonlight does — the cap is embedder-set decoder
  truth, never OR'd in by the shared pump.
- Linux direct-NVENC clamps its Phase-3 default to the ceiling
  (resolve_slices(codec, 4.min(max_slices))) and logs slices/max_slices
  in the caps-probe line; PUNKTFUNK_NVENC_SLICES stays the explicit
  operator override in both directions. Windows keeps its single-slice
  default untouched.

Also repairs the nvenc_cuda #[ignore] hardware tests: d2c46eaf added
open()'s cursor_blend param without updating them, invisible because CI
never compiles tests with the nvenc feature.

Verified on .21 (RTX 5070 Ti): pf-encode + host check/clippy clean with
nvenc; host unit suite 263/0; rtsp announce tests 7/7 incl. the new
slicesPerFrame coverage; on-hardware smokes — default e2e still 4
chunks/frame, NEW single-slice client-ceiling test clamps + disarms
chunked poll with no env involved, env escape unchanged. rustfmt clean.
Windows leg (prepare_display param) is CI-only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-30 22:23:42 +02:00
co-authored by Claude Fable 5
parent c04c5be224
commit f87c1e6cec
15 changed files with 235 additions and 21 deletions
+11 -5
View File
@@ -240,11 +240,17 @@ mod session_main {
audio_channels: settings.audio_channels,
preferred_codec: settings.preferred_codec(),
// HDR off = don't advertise 10-bit/HDR at all; the host then never upgrades.
video_caps: if settings.hdr_enabled {
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
} else {
0
},
// MULTI_SLICE is decoder truth for THIS embedder: every desktop decode stack
// (FFmpeg software, VAAPI, D3D11VA, Vulkan Video) handles AUs carrying several
// slice NALs, so the host may keep its multi-slice low-latency default (§7 LN1).
// The mobile/TV embedders must NOT copy this blindly — Amlogic MediaCodec wedges
// on multi-slice AUs (see `VIDEO_CAP_MULTI_SLICE`), so they advertise per-decoder.
video_caps: punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE
| if settings.hdr_enabled {
punktfunk_core::quic::VIDEO_CAP_10BIT | punktfunk_core::quic::VIDEO_CAP_HDR
} else {
0
},
// No portable Wayland/X11 display-volume query yet, so the host keeps its EDID
// defaults for Linux clients; `PUNKTFUNK_CLIENT_PEAK_NITS` (read in the session
// pump) pins one manually.
+77 -4
View File
@@ -795,10 +795,17 @@ pub struct NvencCudaEncoder {
/// [`stream_ordered_requested`]). The per-frame gate additionally requires `pending` empty.
stream_ordered: bool,
/// Slice count the live session was configured with ([`resolve_slices`] — env override,
/// else the Linux direct-NVENC default of 4 since Phase 3; 1 = the preset's single slice).
/// else the Linux direct-NVENC default of 4 since Phase 3 clamped to
/// [`max_slices`](Self::max_slices); 1 = the preset's single slice).
/// Chunked poll needs ≥ 2 to have boundaries to cut at. Latched at init, consumed by
/// `build_config` (so an in-place reconfigure presents the same slicing).
slices: u32,
/// Ceiling on the per-frame slice count the session's CLIENT decoder accepts (from
/// negotiation: `VIDEO_CAP_MULTI_SLICE`, or GameStream's `videoEncoderSlicesPerFrame`).
/// 1 = single-slice only — the safe shape toward decoders that never asked (Amlogic TV
/// SoCs wedge on multi-slice AUs). Clamps the Phase-3 default; the explicit
/// `PUNKTFUNK_NVENC_SLICES` env override still wins in both directions.
max_slices: u32,
/// `NV_ENC_CAPS_SUPPORT_SUBFRAME_READBACK` from the caps probe — gates the DEFAULT-on
/// sub-frame arming (an unsupported GPU must not have `enableSubFrameWrite` forced into its
/// init params, which could fail the session open). `PUNKTFUNK_NVENC_SUBFRAME=1` overrides.
@@ -847,6 +854,7 @@ impl NvencCudaEncoder {
bit_depth: u8,
chroma: ChromaFormat,
cursor_blend: bool,
max_slices: u32,
) -> Result<Self> {
// The runtime `.so` load is the real "is NVENC possible here" gate: fail the open with a
// clear reason instead of an opaque session error on the first frame.
@@ -895,6 +903,8 @@ impl NvencCudaEncoder {
io_stream: ptr::null_mut(),
stream_ordered: false,
slices: 1,
// A zero from a misbehaving caller must not zero the resolver's default arithmetic.
max_slices: max_slices.max(1),
subframe_cap: false,
subframe_on: false,
subframe_forced: false,
@@ -1094,9 +1104,14 @@ impl NvencCudaEncoder {
// every Linux direct-NVENC session, resolved HERE (before the session opens) so the
// config author, the init params and the chunked-poll latch all agree; the caps probe
// gates the sub-frame default so a GPU without SUBFRAME_READBACK never has it forced
// into its init params. PUNKTFUNK_NVENC_SLICES=1 / PUNKTFUNK_NVENC_SUBFRAME=0 are the
// escapes.
self.slices = resolve_slices(self.codec, 4);
// into its init params. The Phase-3 default is CLAMPED to the session's negotiated
// `max_slices` — the client-decoder ceiling (`VIDEO_CAP_MULTI_SLICE`, or GameStream's
// `videoEncoderSlicesPerFrame`): a client that never asked for multi-slice AUs gets
// single-slice frames, because TV-SoC decoders (Amlogic — Chromecast with Google TV)
// wedge the whole device on frames carrying several slice NALs (the 0.17.0 field
// regression). PUNKTFUNK_NVENC_SLICES / PUNKTFUNK_NVENC_SUBFRAME stay the explicit
// operator overrides in both directions.
self.slices = resolve_slices(self.codec, 4.min(self.max_slices));
self.subframe_on = resolve_subframe(self.subframe_cap);
self.subframe_forced = subframe_env_forced();
tracing::info!(
@@ -1105,6 +1120,8 @@ impl NvencCudaEncoder {
yuv444 = self.yuv444_supported,
subframe_readback = subframe != 0,
dynamic_slice = dyn_slice != 0,
slices = self.slices,
max_slices = self.max_slices,
max = %format!("{wmax}x{hmax}"),
"NVENC (Linux direct) capabilities probed"
);
@@ -2559,6 +2576,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
)
.expect("open NVENC CUDA session");
@@ -2657,6 +2675,7 @@ mod tests {
10,
ChromaFormat::Yuv420,
false,
4,
)
.expect("open NVENC CUDA session");
@@ -2737,6 +2756,7 @@ mod tests {
10,
ChromaFormat::Yuv420,
true, // cursor_blend: bring up the Vulkan slot ring + the 10-bit blend
4,
)
.expect("open NVENC CUDA session");
let cursor = |serial: u64, x: i32, y: i32| pf_frame::CursorOverlay {
@@ -2802,6 +2822,7 @@ mod tests {
8,
ChromaFormat::Yuv444,
false,
4,
)
.expect("open NVENC CUDA 4:4:4 session");
@@ -2848,6 +2869,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
)
.expect("open NVENC CUDA session");
@@ -2906,6 +2928,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
) else {
eprintln!(
"skipping rfi_declines_impossible_ranges: NVENC unavailable (no NVIDIA driver)"
@@ -2933,6 +2956,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
)
.expect("open NVENC CUDA encoder")
}
@@ -2968,6 +2992,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
)
.expect("open");
for f in 0..4u32 {
@@ -3106,6 +3131,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
)
.expect("open NVENC CUDA session");
let frame = nv12_frame(W, H, 0);
@@ -3149,6 +3175,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
true, // cursor_blend: bring up the Vulkan slot ring + blend
4,
)
.expect("open NVENC CUDA session");
let cursor = |serial: u64, x: i32, y: i32| pf_frame::CursorOverlay {
@@ -3227,6 +3254,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
)
.expect("open NVENC CUDA session");
// Steady sync frames first (stream-ordered mode).
@@ -3307,6 +3335,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
)
.expect("open NVENC CUDA session");
@@ -3413,6 +3442,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
)
.expect("open NVENC CUDA session");
@@ -3483,6 +3513,49 @@ mod tests {
assert!(!au.data.is_empty());
}
/// ON-HARDWARE (RTX box `.21`): a session whose CLIENT ceiling is 1 slice (`max_slices` from
/// negotiation — no cap bit / Moonlight `videoEncoderSlicesPerFrame:1`, the Chromecast field
/// regression's fix) must encode single-slice frames with chunked poll disarmed, with NO env
/// knobs involved. Run with `--test-threads=1` (env vars are process-global).
#[test]
#[ignore = "requires an NVIDIA GPU + driver — run manually on the RTX box (.21)"]
fn nvenc_cuda_single_slice_client_ceiling() {
const W: u32 = 1920;
const H: u32 = 1080;
// The ceiling under test is the negotiated one, not the operator override.
std::env::remove_var("PUNKTFUNK_NVENC_SLICES");
std::env::remove_var("PUNKTFUNK_NVENC_SUBFRAME");
pf_zerocopy::cuda::make_current().expect("shared CUDA context current");
let mut enc = NvencCudaEncoder::open(
Codec::H265,
PixelFormat::Nv12,
W,
H,
60,
20_000_000,
true,
8,
ChromaFormat::Yuv420,
false,
1, // the client never advertised multi-slice tolerance
)
.expect("open NVENC CUDA session");
for i in 0..4u32 {
let frame = nv12_frame(W, H, i);
enc.submit_indexed(&frame, i).expect("submit");
assert_eq!(
enc.slices, 1,
"a 1-slice client ceiling must clamp the Phase-3 default"
);
assert!(
!enc.supports_chunked_poll(),
"single-slice sessions have no boundaries — chunked poll must stay disarmed"
);
let au = enc.poll().expect("poll").expect("one AU per sync frame");
assert!(!au.data.is_empty(), "frame {i} produced an empty AU");
}
}
/// ON-HARDWARE (RTX box `.21`): the Phase-3 default-on ESCAPES — `PUNKTFUNK_NVENC_SLICES=1`
/// must fully disarm chunked poll (and `poll_chunk` degrades
/// to exactly one self-closing whole-AU chunk (the default-path contract every non-chunked
+8 -5
View File
@@ -45,11 +45,14 @@ pub(super) fn codec_guid(codec: Codec) -> nv::GUID {
/// Resolved per-frame slice count for a session (latency plan §7 LN1, Phase 3): the
/// `PUNKTFUNK_NVENC_SLICES` env override wins (1..=32; **1 = the explicit single-slice
/// escape**, needed now that a backend can default higher), else the backend's
/// `default_slices` — 4 on Linux direct-NVENC since the Phase-3 default-on, 1 everywhere else
/// (the Windows async path is deliberately untouched). H.264/HEVC only (AV1 partitions via
/// tiles). ONE parse shared by the config author ([`apply_low_latency_config`] via
/// [`LowLatencyConfig::slices`]) and the Linux backend's chunked-poll arming, so the two can
/// never disagree about whether a session is multi-slice.
/// `default_slices` — on Linux direct-NVENC the Phase-3 default of 4 CLAMPED to the session's
/// negotiated client-decoder ceiling (`VIDEO_CAP_MULTI_SLICE` / GameStream's
/// `videoEncoderSlicesPerFrame` — a client that never asked stays single-slice: Amlogic TV
/// SoCs wedge on multi-slice AUs), 1 everywhere else (the Windows async path is deliberately
/// untouched). H.264/HEVC only (AV1 partitions via tiles). ONE parse shared by the config
/// author ([`apply_low_latency_config`] via [`LowLatencyConfig::slices`]) and the Linux
/// backend's chunked-poll arming, so the two can never disagree about whether a session is
/// multi-slice.
pub(super) fn resolve_slices(codec: Codec, default_slices: u32) -> u32 {
if !matches!(codec, Codec::H264 | Codec::H265) {
return 1;
+27 -7
View File
@@ -174,6 +174,13 @@ pub fn open_video(
// The session may hand this encoder cursor bitmaps to composite (cursor-as-metadata
// captures). Backends whose fast path can't blend (Vulkan EFC RGB-direct) key off it.
cursor_blend: bool,
// Ceiling on the per-frame slice count this session's CLIENT decoder accepts: 1 =
// single-slice only (the safe default toward decoders that never asked — Amlogic TV SoCs
// wedge on multi-slice AUs), a Moonlight `videoEncoderSlicesPerFrame` request, or 32 (= no
// client-side limit) for a `VIDEO_CAP_MULTI_SLICE` punktfunk/1 client. Backends that split
// frames (Linux direct-NVENC, §7 LN1) clamp their default against it;
// `PUNKTFUNK_NVENC_SLICES` stays the explicit operator override in both directions.
max_slices: u32,
) -> Result<Box<dyn Encoder>> {
let (inner, backend) = open_video_backend(
codec,
@@ -186,6 +193,7 @@ pub fn open_video(
bit_depth,
chroma,
cursor_blend,
max_slices,
)?;
// Record what this session encodes on (the mgmt API's "currently used GPU"): the backend label
// is reported by `open_video_backend` from the branch that ACTUALLY opened — not re-derived by
@@ -337,6 +345,7 @@ fn open_video_backend_linux(
bit_depth: u8,
chroma: ChromaFormat,
cursor_blend: bool,
max_slices: u32,
) -> Result<(Box<dyn Encoder>, &'static str)> {
// A NEGOTIATED PyroWave session (client advertised + preferred it, plan §3) routes
// straight to that backend — the PUNKTFUNK_ENCODER pref below stays a lab override.
@@ -446,6 +455,7 @@ fn open_video_backend_linux(
bit_depth,
chroma,
cursor_blend,
max_slices,
)
.map(|e| (e, "nvenc"))
};
@@ -562,8 +572,11 @@ fn open_video_backend(
bit_depth: u8,
chroma: ChromaFormat,
cursor_blend: bool,
max_slices: u32,
) -> Result<(Box<dyn Encoder>, &'static str)> {
let _ = cursor_blend; // consumed only by the Linux vulkan-encode + direct-NVENC arms below
// Consumed only by the Linux vulkan-encode + direct-NVENC arms below (max_slices: the
// direct-NVENC arm — the one backend that splits frames).
let _ = (cursor_blend, max_slices);
validate_dimensions(codec, width, height)?;
// Refresh/fps must be positive and sane: fps feeds the encoder time_base (`Rational(1, fps)`)
// and the pts→ns conversion (`pts * 1e9 / fps`), so 0 builds a 1/0 rational / divides by zero.
@@ -600,6 +613,7 @@ fn open_video_backend(
bit_depth,
chroma,
cursor_blend,
max_slices,
)
}
#[cfg(target_os = "windows")]
@@ -816,6 +830,7 @@ fn open_video_backend(
cuda,
bit_depth,
chroma,
max_slices,
);
anyhow::bail!("video encode requires Linux or Windows")
}
@@ -840,14 +855,16 @@ fn open_nvenc_probed(
bit_depth: u8,
chroma: ChromaFormat,
cursor_blend: bool,
max_slices: u32,
) -> Result<Box<dyn Encoder>> {
// Consumed by the direct-SDK arm below.
#[cfg(not(feature = "nvenc"))]
let _ = cursor_blend; // consumed by the direct-SDK arm below
// Direct-SDK NVENC (design/linux-direct-nvenc.md): the DEFAULT on NVIDIA, and only for a CUDA
// capture payload (it registers CUDADEVICEPTR inputs — a CPU/dmabuf frame can't feed it, so those
// keep the libav path; and `cuda` is false on AMD/Intel, so they stay on VAAPI). Set
// PUNKTFUNK_NVENC_DIRECT=0 to fall back to libav. It self-clamps the bitrate internally (its own
// level-ceiling binary search at session open), so it skips the probe-loop stepping below.
let _ = (cursor_blend, max_slices);
// Direct-SDK NVENC (design/linux-direct-nvenc.md): the DEFAULT on NVIDIA, and only for a CUDA
// capture payload (it registers CUDADEVICEPTR inputs — a CPU/dmabuf frame can't feed it, so those
// keep the libav path; and `cuda` is false on AMD/Intel, so they stay on VAAPI). Set
// PUNKTFUNK_NVENC_DIRECT=0 to fall back to libav. It self-clamps the bitrate internally (its own
// level-ceiling binary search at session open), so it skips the probe-loop stepping below.
#[cfg(feature = "nvenc")]
if cuda && nvenc_direct_enabled() {
tracing::info!(
@@ -865,6 +882,7 @@ fn open_nvenc_probed(
bit_depth,
chroma,
cursor_blend,
max_slices,
)?) as Box<dyn Encoder>);
}
// The silent-degrade trap: a build without `--features nvenc` compiles the direct-SDK
@@ -2317,6 +2335,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
)
.expect("software arm must open GPU-free");
assert_eq!(label, "software");
@@ -2335,6 +2354,7 @@ mod tests {
8,
ChromaFormat::Yuv420,
false,
4,
) {
// `expect_err` needs `Ok: Debug` and `Box<dyn Encoder>` isn't — match instead.
Ok(_) => panic!("software emits H.264 only; an H.265 session must be refused"),
@@ -136,6 +136,9 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
// video frame indexes. STREAMED_AU the same way: the shared reassembler accepts
// sentinel-headed streamed blocks (retro-validated at the final block), so the host
// may overlap a multi-slice encode's tail with packetize/FEC/pacing.
// VIDEO_CAP_MULTI_SLICE must NOT join this unconditional set: it is DECODER
// truth (Amlogic MediaCodec wedges on multi-slice AUs — the 0.17.0 Chromecast
// regression), so only the embedder can set it, per its decode stack.
video_caps: video_caps
| crate::quic::VIDEO_CAP_HOST_TIMING
| crate::quic::VIDEO_CAP_PROBE_SEQ
+13
View File
@@ -55,6 +55,19 @@ pub const VIDEO_CAP_STREAMED_AU: u8 = 0x20;
/// performance choice — both AEADs are full-strength, and Hello/Welcome ride the pinned-TLS
/// control channel, so there is no downgrade surface.
pub const VIDEO_CAP_CHACHA20: u8 = 0x40;
/// [`Hello::video_caps`] bit: the client's decoder accepts **multi-slice access units** — H.264/
/// HEVC frames carrying several slice NALs (latency plan §7 LN1: the encoder splits frames so
/// sub-frame readback can ship early slices while the tail encodes). Decoder-level, so the
/// EMBEDDER sets it from what its decode stack actually handles: the desktop clients' FFmpeg/
/// D3D11VA/Vulkan-video decoders are fine, but mobile/TV MediaCodec is per-SoC — Amlogic HEVC
/// decoders (Chromecast with Google TV, Fire TV) wedge the whole DEVICE on multi-slice frames
/// (the 0.17.0 field regression: the 4-slice Linux default froze streams on first frame and
/// watchdog-rebooted the CCwGTV), which is exactly why Moonlight requests 1 slice per frame for
/// every hardware decoder. The host defaults to >1 slice ONLY toward a client that sets this
/// bit (`PUNKTFUNK_NVENC_SLICES` stays the explicit operator override in both directions);
/// every other client gets single-slice frames — the pre-0.17 wire shape. NOTE: this takes the
/// video_caps byte's last free bit — the next video cap needs a second byte (ABI bump).
pub const VIDEO_CAP_MULTI_SLICE: u8 = 0x80;
/// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
/// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
@@ -627,6 +627,7 @@ mod session_tests {
codec: crate::encode::Codec::H265,
min_fec: 0,
hdr: false,
slices: 1, // the no-request default — hardware decoders get single-slice AUs
});
assert!(state.end_session("test"), "video was live");
@@ -556,6 +556,17 @@ fn stream_config(map: &HashMap<String, String>) -> Option<StreamConfig> {
let min_fec = parse_u("x-nv-vqos[0].fec.minRequiredFecPackets")
.unwrap_or(2)
.min(16) as u8;
// The client's requested per-frame slice count (moonlight-common-c SdpGenerator.c:
// `videoEncoderSlicesPerFrame`) — 1 for every HARDWARE decoder, 4 only for software
// decoders (slice-threading). Honor it as the encoder's slicing ceiling: GFE/Sunshine
// encode what was asked, and hardware TV decoders (Amlogic — Chromecast with Google TV)
// wedge the whole DEVICE on multi-slice AUs they never requested — the 0.17.0 field
// regression, where the Linux direct-NVENC 4-slice default (§7 LN1) ignored this key and
// froze + watchdog-rebooted CCwGTV clients on the first frame. Absent or out-of-range
// (attacker-controlled pre-auth input) ⇒ 1, the universally-safe single-slice shape.
let slices = parse_u("x-nv-video[0].videoEncoderSlicesPerFrame")
.filter(|n| (1..=32).contains(n))
.unwrap_or(1);
Some(StreamConfig {
width,
height,
@@ -565,6 +576,7 @@ fn stream_config(map: &HashMap<String, String>) -> Option<StreamConfig> {
codec,
min_fec,
hdr,
slices,
})
}
@@ -727,6 +739,30 @@ mod tests {
}
}
/// `videoEncoderSlicesPerFrame` is honored as the encoder's slicing ceiling: Moonlight sends
/// 1 for every hardware decoder (Amlogic TV SoCs wedge on multi-slice AUs — the 0.17.0
/// Chromecast regression) and 4 for software decoders. Absent or out-of-range (pre-auth,
/// attacker-controlled) must fall back to the universally-safe 1, never reject the session.
#[test]
fn announce_slices_per_frame() {
// Absent (very old client) → single-slice.
assert_eq!(stream_config(&announce(&[])).unwrap().slices, 1);
// The two real Moonlight values pass through verbatim.
for want in ["1", "4"] {
let map = announce(&[("x-nv-video[0].videoEncoderSlicesPerFrame", want)]);
assert_eq!(
stream_config(&map).unwrap().slices,
want.parse::<u32>().unwrap()
);
}
// Garbage / out-of-range degrades to 1 (still streams — this key must never kill a session).
for bad in ["0", "33", "999999", "-1", "x"] {
let map = announce(&[("x-nv-video[0].videoEncoderSlicesPerFrame", bad)]);
let cfg = stream_config(&map).expect("session must still negotiate");
assert_eq!(cfg.slices, 1, "slicesPerFrame {bad} must degrade to 1");
}
}
/// Audio negotiation: numChannels/AudioQuality/packetDuration, with Moonlight defaults.
#[test]
fn announce_audio_params() {
@@ -31,6 +31,12 @@ pub struct StreamConfig {
/// Drives the capturer's proactive advanced-color enable; the encoder picks Main10 from the captured
/// (P010) frame format. Always `false` on a non-HDR host, so the SDR path is unchanged.
pub hdr: bool,
/// Client's `x-nv-video[0].videoEncoderSlicesPerFrame` — the per-frame slice count its
/// decoder wants (moonlight-common-c requests 1 for every HARDWARE decoder and 4 only for
/// software slice-threading). Honored as the encoder's slicing ceiling: hardware TV decoders
/// (Amlogic — Chromecast with Google TV) wedge the whole device on multi-slice AUs they
/// never asked for (the 0.17.0 4-slice-default field regression). Absent ⇒ 1.
pub slices: u32,
}
/// A pooled capturer plus the three properties reuse must match on — its HDR-ness, its
@@ -1040,6 +1046,8 @@ fn stream_body(
// True only when THIS session's capture negotiated cursor-as-metadata — which the
// callers grant only where the resolved backend composites (`cursor_blend_capable`).
cursor_blend,
// The client's requested slices-per-frame (its DECODER's ceiling) — see `StreamConfig`.
cfg.slices,
)
.context("open video encoder for stream")?;
// Tell the encoder how deep the capturer lets it pipeline. Without this an in-place backend
@@ -1212,6 +1220,7 @@ fn stream_body(
gs_bit_depth(frame.format),
encode::ChromaFormat::Yuv420, // GameStream stays 4:2:0
cursor_blend, // same capture cursor mode — see the first open
cfg.slices, // client slicing ceiling — see the first open
)
.context("reopen encoder after rebuild")?;
// A rebuilt encoder starts unconfigured — same reason as the first open above.
+5
View File
@@ -1454,6 +1454,10 @@ async fn serve_session(
// Streamed-AU capability: the client's reassembler accepts sentinel-headed streamed blocks,
// so a chunked encoder session may ship an AU's early FEC blocks while its tail encodes.
let streamed_au = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_STREAMED_AU != 0;
// Multi-slice capability: the client's DECODER accepts AUs carrying several slice NALs, so
// the encoder may keep its multi-slice default (§7 LN1). Absent ⇒ single-slice frames —
// TV-SoC decoders (Amlogic: Chromecast with Google TV) wedge the device on multi-slice AUs.
let multi_slice = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE != 0;
let stats_dp = stats; // data-plane handle to the shared stats recorder
// Short label for web-console stats captures: the client's cert-fingerprint prefix, else its
// peer IP (no fingerprint = anonymous TOFU/--open client).
@@ -1571,6 +1575,7 @@ async fn serve_session(
cursor_client_draws: cursor_client_draws_dp,
probe_seq,
streamed_au,
multi_slice,
stats: stats_dp,
client_label,
client_name,
@@ -579,6 +579,10 @@ pub(super) async fn negotiate(
// The bit the Welcome just advertised — read back rather than recomputed, so the
// prepared display and the session wiring cannot disagree with it.
let cursor_fw = welcome.host_caps & punktfunk_core::quic::HOST_CAP_CURSOR != 0;
// Same bit the data plane's SessionContext reads — the prepared plan and the
// session wiring must agree on the slicing ceiling (an encoder rebuilt from the
// prepared plan with a DIFFERENT max_slices would change the wire shape mid-flow).
let multi_slice = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE != 0;
let (mode, shard_payload) = (hello.mode, welcome.shard_payload);
// "Automatic" — `bitrate_kbps` above is the host's own answer for `mode`, so the build
// may re-resolve it if the source turns out to deliver a different size. Sampled here
@@ -594,6 +598,7 @@ pub(super) async fn negotiate(
client_identity,
client_hdr,
cursor_fw,
multi_slice,
bitrate_kbps,
bitrate_auto,
bit_depth,
@@ -992,6 +992,12 @@ pub(super) struct SessionContext {
/// AU's FEC blocks under sentinel headers as the slices complete instead of waiting for the
/// whole AU. `false` = older client — chunks (if any) are drained whole-AU, zero wire change.
pub(super) streamed_au: bool,
/// The client advertised [`punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE`]: its decoder
/// accepts multi-slice AUs, so the session's encoder may keep its multi-slice default
/// (§7 LN1 — becomes [`SessionPlan::max_slices`](crate::session_plan::SessionPlan)).
/// `false` = single-slice frames, the pre-0.17 wire shape TV-SoC decoders (Amlogic —
/// Chromecast with Google TV) require to not wedge.
pub(super) multi_slice: bool,
/// Shared streaming-stats recorder. The capture loop reads `is_armed()` per frame to decide
/// whether to measure the per-stage split; the send thread builds + pushes the aggregated
/// `StatsSample` at its 2 s boundary.
@@ -1092,6 +1098,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
ctx.bit_depth,
),
ctx.cursor_forward,
ctx.multi_slice,
);
// gamescope: the XFixes cursor source feeds the host-side composite (Phase C) — unless the
// spawned gamescope paints the pointer into its node itself, in which case the reader would
@@ -1141,6 +1148,8 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
cursor_client_draws,
probe_seq,
streamed_au,
// Already folded into `plan.max_slices` by the resolve above — nothing below reads it.
multi_slice: _,
stats,
client_label,
client_name,
@@ -2044,6 +2053,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
bit_depth,
plan.chroma,
plan.cursor_blend,
plan.max_slices,
) {
Ok(mut new_enc) => {
// The fresh encoder may have clamped to its codec-level ceiling —
@@ -3393,6 +3403,7 @@ fn try_inplace_resize(
bit_depth,
plan.chroma,
plan.cursor_blend,
plan.max_slices,
) {
Ok(e) => e,
Err(e) => {
@@ -3450,6 +3461,7 @@ pub(super) fn prepare_display(
client_identity: Option<[u8; 32]>,
client_hdr: Option<punktfunk_core::quic::HdrMeta>,
cursor_forward: bool,
multi_slice: bool,
bitrate_kbps: u32,
// Passed through to [`build_pipeline`] — see its parameter of the same name.
bitrate_auto: bool,
@@ -3478,6 +3490,7 @@ pub(super) fn prepare_display(
bit_depth,
),
cursor_forward,
multi_slice,
);
plan.gamescope_cursor =
crate::session_plan::gamescope_cursor_for(compositor == pf_vdisplay::Compositor::Gamescope);
@@ -3899,6 +3912,7 @@ fn build_pipeline(
bit_depth,
plan.chroma,
plan.cursor_blend,
plan.max_slices,
)
.context("open video encoder")?;
if let Some(t) = trace {
@@ -132,6 +132,13 @@ pub struct SessionPlan {
/// redundant work producing a SECOND pointer. Resolved by [`cursor_blend_for`]'s sibling so
/// the two answers cannot disagree.
pub gamescope_cursor: bool,
/// Ceiling on the encoder's per-frame slice count, from the client's
/// [`VIDEO_CAP_MULTI_SLICE`](punktfunk_core::quic::VIDEO_CAP_MULTI_SLICE): 32 (= no
/// client-side limit, the backend picks its own multi-slice default, §7 LN1) when the bit is
/// set, 1 (single-slice frames — the pre-0.17 wire shape TV-SoC decoders like Amlogic
/// require) when it isn't. Applied to EVERY encoder this plan opens (initial + all rebuilds)
/// so the slicing can never change shape across a mode/bitrate/stall rebuild.
pub max_slices: u32,
}
impl SessionPlan {
@@ -143,6 +150,7 @@ impl SessionPlan {
codec: crate::encode::Codec,
cursor_blend: bool,
cursor_forward: bool,
multi_slice: bool,
) -> Self {
SessionPlan {
capture: CaptureBackend::resolve(),
@@ -158,6 +166,7 @@ impl SessionPlan {
// Set by the resolve callers (they know the compositor); default off keeps every
// non-gamescope plan unchanged.
gamescope_cursor: false,
max_slices: if multi_slice { 32 } else { 1 },
}
}
+1
View File
@@ -150,6 +150,7 @@ pub fn run(opts: Options) -> Result<()> {
8, // spike synthetic harness: 8-bit
encode::ChromaFormat::Yuv420, // ...and 4:2:0
false, // synthetic frames carry no cursor
4, // no client decoder — keep the backend's multi-slice default
)
.context("open encoder")?;
+16
View File
@@ -528,6 +528,22 @@
#define VIDEO_CAP_CHACHA20 64
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_caps`] bit: the client's decoder accepts **multi-slice access units** — H.264/
// HEVC frames carrying several slice NALs (latency plan §7 LN1: the encoder splits frames so
// sub-frame readback can ship early slices while the tail encodes). Decoder-level, so the
// EMBEDDER sets it from what its decode stack actually handles: the desktop clients' FFmpeg/
// D3D11VA/Vulkan-video decoders are fine, but mobile/TV MediaCodec is per-SoC — Amlogic HEVC
// decoders (Chromecast with Google TV, Fire TV) wedge the whole DEVICE on multi-slice frames
// (the 0.17.0 field regression: the 4-slice Linux default froze streams on first frame and
// watchdog-rebooted the CCwGTV), which is exactly why Moonlight requests 1 slice per frame for
// every hardware decoder. The host defaults to >1 slice ONLY toward a client that sets this
// bit (`PUNKTFUNK_NVENC_SLICES` stays the explicit operator override in both directions);
// every other client gets single-slice frames — the pre-0.17 wire shape. NOTE: this takes the
// video_caps byte's last free bit — the next video cap needs a second byte (ABI bump).
#define VIDEO_CAP_MULTI_SLICE 128
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder