fix(encode/pyrowave): the RDO block-index cap must cover 4:2:0, not just 4:4:4

The vendored rate controller packs the 32x32 block index into the low 16 bits of
`RDOperation::block_offset_saving` (pyrowave-sys patches/0002). Past u16::MAX the
index collides with the `saving` field, the resolve over-credits, and the emitted
payload can overshoot the buffer `pyrowave_encoder_packetize` writes into — whose
only bounds check is an `assert` that the Release (NDEBUG) vendored build compiles
out. There is no Rust-side guard behind it.

All three callers of `pyrowave_mode_fits_rdo` hardcoded 4:4:4, leaving 4:2:0
unchecked — but 4:2:0 overflows too, just later: 8192x6144 is 73728 blocks and
8192x8192 is 98304, while `Codec::PyroWave.max_dimension()` permits 8192 per axis,
so both are reachable from a client-requested `mode=WxHxFPS`. Worse, the host's
only use of the helper (native/handshake.rs) is a 4:4:4 -> 4:2:0 downgrade, so an
oversized mode was actively routed INTO the unguarded branch.

The cap now lives in `validate_dimensions`, checked against 4:2:0 — the most
permissive chroma, so a mode that cannot fit there fits no PyroWave session at any
chroma. That is the single chokepoint both the negotiator (handshake.rs:180) and
`open_video_backend` already run. Both encoder opens additionally check the chroma
actually being opened: the 4:4:4 half, plus defence in depth for the
`PUNKTFUNK_ENCODER=pyrowave` lab override.

Not dormant: punktfunk-host has `default = ["pyrowave"]` and no build passes
`--no-default-features`, so these backends ship in every host binary.

Tests pin the arithmetic (8192x6144 = 73728, 8192x8192 = 98304, 7680x4320 still
fits) and assert the cap does not leak to H.265/AV1, which have no such controller.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-25 00:32:37 +02:00
co-authored by Claude Opus 5
parent 16fa43da40
commit 6c97c00add
4 changed files with 71 additions and 10 deletions
+11
View File
@@ -246,6 +246,17 @@ mod tests {
assert!(block_count_32x32(3840, 2160, true) <= u16::MAX as u32);
assert!(block_count_32x32(7680, 4320, true) > u16::MAX as u32);
assert!(block_count_32x32(7680, 4320, false) <= u16::MAX as u32);
// …and 4:2:0 wraps it too, just later — the hole the old 4:4:4-only open guard left.
// `Codec::max_dimension()` allows PyroWave 8192px per axis, so these modes were
// reachable from a client-requested `mode=WxHxFPS`, and the negotiator's 4:4:4 → 4:2:0
// downgrade routed oversized modes straight into the unguarded branch.
// `validate_dimensions` now rejects them against this 4:2:0 count.
assert_eq!(block_count_32x32(8192, 6144, false), 73728);
assert_eq!(block_count_32x32(8192, 8192, false), 98304);
assert!(block_count_32x32(8192, 6144, false) > u16::MAX as u32);
assert!(block_count_32x32(8192, 8192, false) > u16::MAX as u32);
// The largest 4:2:0 mode that still fits, for the boundary the validator enforces.
assert!(block_count_32x32(7680, 4320, false) <= u16::MAX as u32);
}
#[test]