forked from unom/punktfunk
enricobuehlerandClaude Opus 5
6c97c00add
The vendored rate controller packs the 32x32 block index into the low 16 bits of `RDOperation::block_offset_saving` (pyrowave-sys patches/0002). Past u16::MAX the index collides with the `saving` field, the resolve over-credits, and the emitted payload can overshoot the buffer `pyrowave_encoder_packetize` writes into — whose only bounds check is an `assert` that the Release (NDEBUG) vendored build compiles out. There is no Rust-side guard behind it. All three callers of `pyrowave_mode_fits_rdo` hardcoded 4:4:4, leaving 4:2:0 unchecked — but 4:2:0 overflows too, just later: 8192x6144 is 73728 blocks and 8192x8192 is 98304, while `Codec::PyroWave.max_dimension()` permits 8192 per axis, so both are reachable from a client-requested `mode=WxHxFPS`. Worse, the host's only use of the helper (native/handshake.rs) is a 4:4:4 -> 4:2:0 downgrade, so an oversized mode was actively routed INTO the unguarded branch. The cap now lives in `validate_dimensions`, checked against 4:2:0 — the most permissive chroma, so a mode that cannot fit there fits no PyroWave session at any chroma. That is the single chokepoint both the negotiator (handshake.rs:180) and `open_video_backend` already run. Both encoder opens additionally check the chroma actually being opened: the 4:4:4 half, plus defence in depth for the `PUNKTFUNK_ENCODER=pyrowave` lab override. Not dormant: punktfunk-host has `default = ["pyrowave"]` and no build passes `--no-default-features`, so these backends ship in every host binary. Tests pin the arithmetic (8192x6144 = 73728, 8192x8192 = 98304, 7680x4320 still fits) and assert the cap does not leak to H.265/AV1, which have no such controller. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>