feat(core): zero-copy pooled reassembly — shards land at their final AU offset

Rewrite the client Reassembler around one whole-frame buffer per frame:
frame_bytes rides in every header and packetize geometry is
deterministic (every non-final block is exactly max_data_per_block data
shards), so a data shard's final AU offset is computable on arrival —
copy it there once, straight from the decrypt ring. New
ErasureCoder::reconstruct_into decodes ONLY the missing shards directly
into the frame buffer's holes (gf16 native; gf8 legacy shim); received
recovery shards ride pooled shard-sized buffers. The completed buffer
IS Frame::data.

Deletes the per-shard to_vec + per-block concat + final AU concat
(~178k allocs and a double copy of every byte per second at 2 Gbps —
the pump wall the 2026-07-14 sweeps measured at 98.9% of an M3 Ultra
core). Reassembly now costs ~0.4 µs/packet in-stream.

The eager buffer changes the hostile-header exposure, so two new
firewalls: derived-geometry validation (a header lying about its
data_shards/block_count vs its own frame_bytes is dropped before it can
scribble across another shard's range) and an in-flight allocation
budget (IN_FLIGHT_BUF_FACTOR × max_frame_bytes) so a window of tiny
first-shards can't commit gigabytes.

Behavior parity pinned by the existing suite (all green unchanged) plus
new end-to-end roundtrips through the real Packetizer (multi-block +
partial tail, loss within budget, reversed delivery, duplicates, empty
frame, unrecoverable block ages out, budget enforcement). loss-harness
recovery curve identical; pipeline bench: gf8/1MB +42%, gf16 neutral
(host-encode dominated). Known pre-existing quirk kept as-is: reversed
delivery reconstructs early (data+recovery ≥ k) and counts late-not-lost
shards into fec_recovered_shards.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-14 19:08:15 +02:00
parent f2fa7828d6
commit ed0ce5dc6d
4 changed files with 626 additions and 98 deletions
+143
View File
@@ -43,6 +43,25 @@ pub trait ErasureCoder: Send + Sync {
recovery_count: usize,
received: &mut [Option<Vec<u8>>],
) -> Result<Vec<Vec<u8>>, FecError>;
/// Reconstruct ONLY the missing data shards of a block, writing each straight into its final
/// slot in the caller's buffer — the receive-side half of [`encode`](Self::encode)'s ref-based
/// contract (the reassembler's slots are slices of one contiguous frame buffer, so recovery
/// lands at its final AU offset with no per-shard `Vec`s and no block/AU concat copies).
///
/// `data` holds the block's K equal-length shard slots; `have[i]` marks the slots whose bytes
/// were received (valid codec input — a missing slot's contents are unspecified on entry).
/// `recovery` is the received parity as `(recovery_index, bytes)` with `recovery_index <
/// recovery_count` (the block's declared M, which the codec math needs even when not all M
/// arrived). On success every missing slot has been filled; on error missing slots are
/// unspecified and the caller must discard the block.
fn reconstruct_into(
&self,
recovery_count: usize,
data: &mut [&mut [u8]],
have: &[bool],
recovery: &[(usize, &[u8])],
) -> Result<(), FecError>;
}
/// Construct the coder for a scheme.
@@ -80,6 +99,43 @@ pub(crate) fn validate_block_shape(
Ok(())
}
/// Validate the shape [`ErasureCoder::reconstruct_into`] promises: `have` matches `data`, one
/// shard length across data slots and recovery shards, recovery indices within the declared M,
/// and enough shards present to reconstruct at all. Both backends call this first.
pub(crate) fn validate_into_shape(
data: &[&mut [u8]],
have: &[bool],
recovery: &[(usize, &[u8])],
recovery_count: usize,
) -> Result<(), FecError> {
if data.is_empty() {
return Err(FecError::Config("no data shards"));
}
if have.len() != data.len() {
return Err(FecError::Config("have length must equal data length"));
}
let len = data[0].len();
if data.iter().any(|s| s.len() != len) {
return Err(FecError::Config("shards in a block must be equal length"));
}
for &(j, bytes) in recovery {
if j >= recovery_count {
return Err(FecError::Config("recovery index out of range"));
}
if bytes.len() != len {
return Err(FecError::Config("shards in a block must be equal length"));
}
}
let present = have.iter().filter(|h| **h).count();
if present + recovery.len() < data.len() {
return Err(FecError::TooFewShards {
have: present + recovery.len(),
need: data.len(),
});
}
Ok(())
}
/// Validate `encode` inputs: at least one data shard, all of equal length.
pub(crate) fn validate_encode_shape(data: &[&[u8]]) -> Result<(), FecError> {
let first = data
@@ -117,6 +173,93 @@ mod tests {
assert_eq!(restored, data);
}
/// Round-trip through `reconstruct_into`: encode, zero out `lose_data` slots in a contiguous
/// buffer (the reassembler's frame-buffer shape), drop `lose_recovery` parity shards, and
/// assert the missing slots are restored in place while the present ones are untouched.
fn roundtrip_into(
coder: &dyn ErasureCoder,
k: usize,
m: usize,
shard_len: usize,
lose_data: &[usize],
lose_recovery: &[usize],
) {
let src: Vec<Vec<u8>> = (0..k)
.map(|i| (0..shard_len).map(|b| (i * 31 + b * 7) as u8).collect())
.collect();
let refs: Vec<&[u8]> = src.iter().map(|s| s.as_slice()).collect();
let parity = coder.encode(&refs, m).unwrap();
let mut buf = vec![0u8; k * shard_len];
let mut have = vec![true; k];
for (i, s) in src.iter().enumerate() {
if lose_data.contains(&i) {
have[i] = false; // slot stays zeroed — codec must fill it
} else {
buf[i * shard_len..(i + 1) * shard_len].copy_from_slice(s);
}
}
let recovery: Vec<(usize, &[u8])> = parity
.iter()
.enumerate()
.filter(|(j, _)| !lose_recovery.contains(j))
.map(|(j, p)| (j, p.as_slice()))
.collect();
let mut slots: Vec<&mut [u8]> = buf.chunks_mut(shard_len).collect();
coder
.reconstruct_into(m, &mut slots, &have, &recovery)
.unwrap();
for (i, s) in src.iter().enumerate() {
assert_eq!(
&buf[i * shard_len..(i + 1) * shard_len],
s.as_slice(),
"shard {i}"
);
}
}
#[test]
fn gf16_reconstruct_into_fills_only_the_holes() {
roundtrip_into(&Gf16Coder, 16, 4, 256, &[1, 9], &[3]);
roundtrip_into(&Gf16Coder, 4, 2, 16, &[0, 3], &[]);
roundtrip_into(&Gf16Coder, 4, 2, 16, &[], &[0, 1]); // nothing missing, no parity needed
}
#[test]
fn gf8_reconstruct_into_fills_only_the_holes() {
roundtrip_into(&Gf8Coder, 16, 4, 256, &[0, 7], &[1]);
roundtrip_into(&Gf8Coder, 4, 2, 16, &[2], &[1]);
}
#[test]
fn reconstruct_into_rejects_bad_shapes() {
let mut buf = [0u8; 4 * 8];
// Too few shards: 2 of 4 data present, no recovery.
let mut slots: Vec<&mut [u8]> = buf.chunks_mut(8).collect();
let have = [true, true, false, false];
assert!(Gf16Coder
.reconstruct_into(2, &mut slots, &have, &[])
.is_err());
// Recovery index out of the declared range.
let parity = [0u8; 8];
let mut slots: Vec<&mut [u8]> = buf.chunks_mut(8).collect();
assert!(Gf16Coder
.reconstruct_into(2, &mut slots, &have, &[(2, &parity), (3, &parity)])
.is_err());
// Mismatched recovery shard length.
let short = [0u8; 6];
let mut slots: Vec<&mut [u8]> = buf.chunks_mut(8).collect();
assert!(Gf8Coder
.reconstruct_into(2, &mut slots, &have, &[(0, &short), (1, &parity)])
.is_err());
// `have` length disagreeing with `data`.
let mut slots: Vec<&mut [u8]> = buf.chunks_mut(8).collect();
assert!(Gf8Coder
.reconstruct_into(2, &mut slots, &[true; 3], &[(0, &parity)])
.is_err());
}
#[test]
fn gf8_recovers_within_budget() {
// 16 data + 4 recovery; lose 2 data + 2 recovery (== budget).