fix(encode/pyrowave): the RDO block-index cap must cover 4:2:0, not just 4:4:4

The vendored rate controller packs the 32x32 block index into the low 16 bits of
`RDOperation::block_offset_saving` (pyrowave-sys patches/0002). Past u16::MAX the
index collides with the `saving` field, the resolve over-credits, and the emitted
payload can overshoot the buffer `pyrowave_encoder_packetize` writes into — whose
only bounds check is an `assert` that the Release (NDEBUG) vendored build compiles
out. There is no Rust-side guard behind it.

All three callers of `pyrowave_mode_fits_rdo` hardcoded 4:4:4, leaving 4:2:0
unchecked — but 4:2:0 overflows too, just later: 8192x6144 is 73728 blocks and
8192x8192 is 98304, while `Codec::PyroWave.max_dimension()` permits 8192 per axis,
so both are reachable from a client-requested `mode=WxHxFPS`. Worse, the host's
only use of the helper (native/handshake.rs) is a 4:4:4 -> 4:2:0 downgrade, so an
oversized mode was actively routed INTO the unguarded branch.

The cap now lives in `validate_dimensions`, checked against 4:2:0 — the most
permissive chroma, so a mode that cannot fit there fits no PyroWave session at any
chroma. That is the single chokepoint both the negotiator (handshake.rs:180) and
`open_video_backend` already run. Both encoder opens additionally check the chroma
actually being opened: the 4:4:4 half, plus defence in depth for the
`PUNKTFUNK_ENCODER=pyrowave` lab override.

Not dormant: punktfunk-host has `default = ["pyrowave"]` and no build passes
`--no-default-features`, so these backends ship in every host binary.

Tests pin the arithmetic (8192x6144 = 73728, 8192x8192 = 98304, 7680x4320 still
fits) and assert the cap does not leak to H.265/AV1, which have no such controller.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-25 00:32:37 +02:00
co-authored by Claude Opus 5
parent 16fa43da40
commit 6c97c00add
4 changed files with 71 additions and 10 deletions
+12 -5
View File
@@ -227,12 +227,19 @@ impl PyroWaveEncoder {
if !chroma.is_444() && (width % 2 != 0 || height % 2 != 0) {
bail!("pyrowave 4:2:0 needs even dimensions (got {width}x{height})");
}
if chroma.is_444() && !crate::pyrowave_mode_fits_rdo(width, height, true) {
// The negotiator downgrades these modes to 4:2:0 pre-Welcome; refuse if one
// slips through (e.g. the lab override) rather than wrap the RDO block index.
// Checked against the chroma actually being opened, NOT hardcoded 4:4:4. The 4:2:0 block
// count is ~half of 4:4:4's but still unbounded (8192×6144 4:2:0 = 73728 > u16::MAX), and
// the negotiator's 4:4:4 → 4:2:0 downgrade hands oversized modes to this open AS 4:2:0 —
// so a `chroma.is_444()`-gated check is skipped exactly when it is needed. Wrapping the
// index lets the resolve over-credit and `packetize` overshoot our bitstream buffer
// (its own bounds `assert` is compiled out by the Release vendored build).
// `validate_dimensions` rejects the impossible-at-any-chroma modes earlier; this is the
// 4:4:4-specific half plus defence in depth for the lab override.
if !crate::pyrowave_mode_fits_rdo(width, height, chroma.is_444()) {
bail!(
"pyrowave 4:4:4 at {width}x{height} exceeds the rate controller's 16-bit \
block index (see pyrowave-sys patches/0002 note) — use 4:2:0 at this size"
"pyrowave {} at {width}x{height} exceeds the rate controller's 16-bit block \
index (see pyrowave-sys patches/0002 note) — lower the resolution",
if chroma.is_444() { "4:4:4" } else { "4:2:0" }
);
}
// SAFETY: `open_inner` only issues Vulkan/pyrowave calls whose preconditions it