feat(core+host): LN1 phase-2 — VIDEO_CAP_STREAMED_AU streamed access units

The wire half of sub-frame slice output (latency §7 LN1, planning
design/nvenc-subframe-slice-output.md Phase 2): toward a client that
advertises the new cap bit (0x20), a chunked-poll encoder session ships each
AU's completed FEC blocks while the tail of the frame is still encoding —
the AU's last packet leaves the host as the encode finishes instead of
after it.

Wire semantics (negotiated; zero change for anyone else):
- Non-final blocks ride SENTINEL headers: block_count = 0 (a value no legacy
  sender emits) + frame_bytes = 0 + exactly max_data_per_block data shards,
  so the receiver's shard-offset formula needs no total.
- The final block's headers carry the real frame_bytes/block_count (+
  FLAG_EOF) and RETRO-VALIDATE the whole frame: totals under which a
  received sentinel block is out of range or not full-K kill the frame
  wholesale (no spliced delivery) and the index can't be resurrected.
- Firewall: sentinels are bounded by the negotiated limits (full-K exactly,
  never the last block the limits allow, no total to lie about); the exact
  derived-geometry check runs unchanged on every non-sentinel packet and
  retroactively at pinning. Sentinel opens commit a max_frame_bytes buffer,
  bounded by the existing IN_FLIGHT_BUF_FACTOR budget (amplification test).
- Order-agnostic like legacy: a reversed frame (final block first) opens
  legacy-shaped and still accepts its sentinels against the pinned totals.
- Small/empty streamed AUs degenerate to byte-identical legacy headers.

Host: Packetizer::{begin,push,finish}_streamed seal full-K blocks (data +
parity per block) as chunks arrive; Session::seal_streamed_* share the
pooled-wire + two-lane seal machinery via the new seal_run; the send thread
paces each flush under the frame's existing deadline (pace_sealed split out
of paced_submit) and runs the whole-AU accounting at the last chunk; the
encode pump forwards poll_chunk output as ChunkMsg when the client has the
cap AND the encoder chunks (re-queried per AU — an escalation falls back
seamlessly). Probes never run mid-AU. PUNKTFUNK_STREAMED_AU=0 = host escape
hatch. Client core ORs the cap into Hello (the shared reassembler carries
the support). Sampled first_slice_us vs encode_us PERF log measures the
overlap; the 0xCF stage-field extension stays a follow-up.

Core tests: streamed round-trip (clean/loss/reorder/duplicate, both orders),
sentinel firewall bounds, lying-final wholesale kill + no-resurrect,
open-amplification budget, header-shape pins. Gates still owed before
default-on: security review pass, loss-harness curve, GameStream smoke
(plane untouched structurally), bitrate A/B.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-20 22:53:53 +02:00
parent 9d3b114fd6
commit c5402cb1f7
9 changed files with 1111 additions and 110 deletions
@@ -83,10 +83,13 @@ pub(super) async fn connect_and_handshake(args: &WorkerArgs) -> Result<Handshake
// bit only asks the host for observability datagrams (never changes the encode). // bit only asks the host for observability datagrams (never changes the encode).
// PROBE_SEQ likewise: the shared reassembler keeps probe filler in its own window // PROBE_SEQ likewise: the shared reassembler keeps probe filler in its own window
// (every embedder inherits it), so the host may burst speed tests without consuming // (every embedder inherits it), so the host may burst speed tests without consuming
// video frame indexes. // video frame indexes. STREAMED_AU the same way: the shared reassembler accepts
// sentinel-headed streamed blocks (retro-validated at the final block), so the host
// may overlap a multi-slice encode's tail with packetize/FEC/pacing.
video_caps: video_caps video_caps: video_caps
| crate::quic::VIDEO_CAP_HOST_TIMING | crate::quic::VIDEO_CAP_HOST_TIMING
| crate::quic::VIDEO_CAP_PROBE_SEQ, | crate::quic::VIDEO_CAP_PROBE_SEQ
| crate::quic::VIDEO_CAP_STREAMED_AU,
// Requested surround channel count; the host echoes the resolved value in Welcome. // Requested surround channel count; the host echoes the resolved value in Welcome.
audio_channels, audio_channels,
// The codecs this client can decode + its soft preference (0 = auto). The host // The codecs this client can decode + its soft preference (0 = auto). The host
@@ -47,11 +47,50 @@ pub struct Packetizer {
/// where every packet of the block is dropped wholesale, the frame never completes, and the /// where every packet of the block is dropped wholesale, the frame never completes, and the
/// resulting loss pushes adaptive FEC *higher*. See the `recovery_for` clamp in `packetize_each`. /// resulting loss pushes adaptive FEC *higher*. See the `recovery_for` clamp in `packetize_each`.
max_total_shards: usize, max_total_shards: usize,
/// The peer's per-frame block ceiling, mirroring [`ReassemblerLimits::from_config`]'s
/// `max_blocks` — the streamed path's bound on how many sentinel blocks it may emit (a
/// streamed AU's size isn't known up front, so this is the only pre-emission guard against
/// producing a frame the receiver must reject).
max_blocks: usize,
}
/// One in-progress **streamed** access unit (design/nvenc-subframe-slice-output.md Phase 2):
/// the caller feeds encoder chunks in as they exist ([`Packetizer::push_streamed`]) and every
/// completed `max_data_per_block × shard_payload` block leaves under SENTINEL headers
/// (`frame_bytes = 0`, `block_count = 0` — "not final yet") before the AU's total size is
/// known; [`Packetizer::finish_streamed`] seals the tail block with the real totals and
/// `FLAG_EOF`. Only ever sent to a peer that advertised
/// [`crate::quic::VIDEO_CAP_STREAMED_AU`].
pub struct StreamedAu {
frame_index: u32,
pts_ns: u64,
user_flags: u32,
/// Bytes not yet sealed into a block. Kept ≤ one block by `push_streamed` (it flushes only
/// while STRICTLY more than a block is buffered, so the final block always has ≥ 1 byte and
/// a sentinel block is never retroactively the frame's last).
pending: Vec<u8>,
/// Sentinel blocks already emitted.
blocks_out: u16,
/// Total AU bytes accumulated so far (`pending` included).
total_bytes: u64,
/// The frame's first packet (block 0, shard 0 — carries `FLAG_SOF`) has been emitted.
opened: bool,
}
impl StreamedAu {
/// The wire frame index this AU is sealed with (the caller's RFI bookkeeping domain).
pub fn frame_index(&self) -> u32 {
self.frame_index
}
} }
impl Packetizer { impl Packetizer {
pub fn new(config: &Config) -> Self { pub fn new(config: &Config) -> Self {
let max_data = config.fec.max_data_per_block as usize; let max_data = config.fec.max_data_per_block as usize;
let total_data_max = config
.max_frame_bytes
.div_ceil(config.shard_payload.max(1))
.max(1);
Packetizer { Packetizer {
next_frame_index: 0, next_frame_index: 0,
next_probe_index: 0, next_probe_index: 0,
@@ -64,6 +103,7 @@ impl Packetizer {
// Mirrors `ReassemblerLimits::from_config` — keep the two in step. // Mirrors `ReassemblerLimits::from_config` — keep the two in step.
max_total_shards: (max_data + config.fec.recovery_for(max_data)) max_total_shards: (max_data + config.fec.recovery_for(max_data))
.min(config.fec.scheme.max_total_shards()), .min(config.fec.scheme.max_total_shards()),
max_blocks: total_data_max.div_ceil(max_data).max(1),
} }
} }
@@ -277,4 +317,202 @@ impl Packetizer {
self.next_seq = next_seq; self.next_seq = next_seq;
Ok(()) Ok(())
} }
/// Open a **streamed** access unit (see [`StreamedAu`]). `frame_index` follows the same
/// contract as [`packetize_each`](Self::packetize_each): `Some(i)` = the caller owns the
/// video numbering; `None` draws from the internal counter.
pub fn begin_streamed(
&mut self,
pts_ns: u64,
user_flags: u32,
frame_index: Option<u32>,
) -> StreamedAu {
let frame_index = frame_index.unwrap_or_else(|| {
let i = self.next_frame_index;
self.next_frame_index = i.wrapping_add(1);
i
});
StreamedAu {
frame_index,
pts_ns,
user_flags,
pending: Vec::new(),
blocks_out: 0,
total_bytes: 0,
opened: false,
}
}
/// Feed one encoder chunk into a streamed AU, emitting every block that COMPLETES under
/// sentinel headers (`frame_bytes = 0`, `block_count = 0`, exactly `max_data_per_block`
/// data shards — the receiver's offset formula needs no total). Flushes only while
/// STRICTLY more than one block is buffered, so the frame's last block — whose header must
/// carry the real totals — is never emitted here. Packets reach `emit` in wire order (the
/// caller's nonce order), data then parity per block.
pub fn push_streamed(
&mut self,
au: &mut StreamedAu,
chunk: &[u8],
coder: &dyn ErasureCoder,
mut emit: impl FnMut(&PacketHeader, &[u8]) -> Result<()>,
) -> Result<()> {
au.total_bytes += chunk.len() as u64;
au.pending.extend_from_slice(chunk);
let block_bytes = self.fec.max_data_per_block as usize * self.shard_payload;
while au.pending.len() > block_bytes {
// Room for this sentinel block AND the final block after it, within the peer's
// per-frame block ceiling and the u16 wire field.
if au.blocks_out as usize + 2 > self.max_blocks.min(u16::MAX as usize) {
return Err(PunktfunkError::Unsupported(
"streamed AU exceeds the negotiated max_frame_bytes",
));
}
let sof = !au.opened;
let (bi, pts, uf) = (au.blocks_out, au.pts_ns, au.user_flags);
let fi = au.frame_index;
self.emit_streamed_block(
fi,
pts,
uf,
bi,
&au.pending[..block_bytes],
0,
0,
sof,
false,
coder,
&mut emit,
)?;
au.pending.drain(..block_bytes);
au.blocks_out += 1;
au.opened = true;
}
Ok(())
}
/// Close a streamed AU: seal the final block — its headers carry the REAL
/// `frame_bytes`/`block_count`, which retro-validate the whole frame at the receiver — with
/// `FLAG_EOF` on the last emitted packet. An empty AU degenerates to today's single
/// zero-padded-shard frame (`block_count = 1`, never a sentinel).
pub fn finish_streamed(
&mut self,
au: StreamedAu,
coder: &dyn ErasureCoder,
mut emit: impl FnMut(&PacketHeader, &[u8]) -> Result<()>,
) -> Result<()> {
let frame_bytes = u32::try_from(au.total_bytes)
.map_err(|_| PunktfunkError::Unsupported("streamed AU exceeds u32 bytes"))?;
let block_count = au.blocks_out + 1;
self.emit_streamed_block(
au.frame_index,
au.pts_ns,
au.user_flags,
au.blocks_out,
&au.pending,
frame_bytes,
block_count,
!au.opened,
true,
coder,
&mut emit,
)
}
/// Seal ONE streamed block (data shards + its parity, in wire order). Sentinel blocks pass
/// `frame_bytes = 0` / `block_count = 0`; the final block passes the real totals. `sof`
/// marks the frame's very first packet, `eof` its very last.
#[allow(clippy::too_many_arguments)]
fn emit_streamed_block(
&mut self,
frame_index: u32,
pts_ns: u64,
user_flags: u32,
block_index: u16,
bytes: &[u8],
frame_bytes: u32,
block_count: u16,
sof: bool,
eof: bool,
coder: &dyn ErasureCoder,
emit: &mut impl FnMut(&PacketHeader, &[u8]) -> Result<()>,
) -> Result<()> {
let payload = self.shard_payload;
if payload > u16::MAX as usize {
return Err(PunktfunkError::InvalidArg("shard_payload exceeds u16"));
}
// At least one (zero-padded) data shard even for an empty final block (empty AU).
let k = bytes.len().div_ceil(payload).max(1);
let m = self
.fec
.recovery_for(k)
.min(self.max_total_shards.saturating_sub(k));
if k + m > u16::MAX as usize {
return Err(PunktfunkError::Unsupported("block shard count exceeds u16"));
}
// Stage the one possibly-partial (or empty-frame) shard in the zero-padded scratch.
let full_shards = bytes.len() / payload;
self.tail.clear();
self.tail.resize(payload, 0);
let rem = bytes.len() % payload;
if rem > 0 {
self.tail[..rem].copy_from_slice(&bytes[full_shards * payload..]);
}
let tail = &self.tail;
let shard_at = |s: usize| -> &[u8] {
if s < full_shards {
&bytes[s * payload..(s + 1) * payload]
} else {
tail.as_slice()
}
};
let data_shards: Vec<&[u8]> = (0..k).map(shard_at).collect();
if self.recovery.is_empty() {
self.recovery.push(Vec::new());
}
coder.encode_into(&data_shards, m, &mut self.recovery[0])?;
let mut next_seq = self.next_seq;
let mut emit_one = |next_seq: &mut u32, shard_index: usize, body: &[u8], flags: u8| {
let seq = *next_seq;
*next_seq = next_seq.wrapping_add(1);
let hdr = PacketHeader {
pts_ns,
frame_index,
stream_seq: seq,
frame_bytes,
user_flags,
block_index,
block_count,
data_shards: k as u16,
recovery_shards: m as u16,
shard_index: shard_index as u16,
shard_bytes: payload as u16,
magic: PUNKTFUNK_MAGIC,
version: self.version,
fec_scheme: coder.scheme() as u8,
flags,
};
emit(&hdr, body)
};
for (shard_index, body) in data_shards.iter().enumerate() {
let mut flags = FLAG_PIC;
if sof && shard_index == 0 {
flags |= FLAG_SOF;
}
if eof && m == 0 && shard_index + 1 == k {
flags |= FLAG_EOF;
}
emit_one(&mut next_seq, shard_index, body, flags)?;
}
for r in 0..m {
let mut flags = FLAG_PIC;
if eof && r + 1 == m {
flags |= FLAG_EOF;
}
let body: &[u8] = &self.recovery[0][r];
emit_one(&mut next_seq, k + r, body, flags)?;
}
self.next_seq = next_seq;
Ok(())
}
} }
+115 -16
View File
@@ -70,7 +70,12 @@ struct BlockState {
} }
struct FrameBuf { struct FrameBuf {
/// Exact AU size. 0 = unknown: the frame was opened by a streamed-AU SENTINEL packet
/// ([`crate::quic::VIDEO_CAP_STREAMED_AU`]) and the final block's real totals haven't
/// arrived yet — the frame can't complete before they do (and retro-validate).
frame_bytes: usize, frame_bytes: usize,
/// Block count; 0 = unknown (sentinel-opened, totals not yet pinned). A legacy-opened
/// frame always has ≥ 1 here, so 0 doubles as the "unpinned streamed" marker.
block_count: usize, block_count: usize,
pts_ns: u64, pts_ns: u64,
user_flags: u32, user_flags: u32,
@@ -277,25 +282,49 @@ impl Reassembler {
|| total == 0 || total == 0
|| total > lim.max_total_shards || total > lim.max_total_shards
|| shard_index >= total || shard_index >= total
|| block_count == 0
|| block_count > lim.max_blocks
|| hdr.block_index as usize >= block_count
|| frame_bytes > lim.max_frame_bytes || frame_bytes > lim.max_frame_bytes
{ {
drop(stats); drop(stats);
return Ok(None); return Ok(None);
} }
// Derived-geometry firewall: every sender (our Packetizer, any version) slices a frame // Streamed-AU sentinel ([`crate::quic::VIDEO_CAP_STREAMED_AU`]): `block_count == 0` — a
// into consecutive blocks of exactly `max_data_per_block` data shards with only the LAST // value no legacy sender ever emits — marks a NON-FINAL block of an AU whose total size
// block smaller, and stamps the exact `frame_bytes` in every header. That makes every // doesn't exist yet (the host is still encoding its tail). The exact derived-geometry
// data shard's final AU offset computable on arrival — // check below can't run without a total, so a sentinel is bounded by the negotiated
// limits instead — and by construction: full-K exactly (the offset formula needs it),
// never the last block the limits allow (the real final block must still fit after it),
// and no total to lie about. The frame-wide exact check runs retroactively the moment
// the final block's real totals arrive (the pinning arm below).
let sentinel = block_count == 0;
let block_idx = hdr.block_index as usize;
// For a sentinel-opened frame the buffer must hold ANY final geometry the totals may
// later pin — the maximum the negotiated limits allow (the design's "allocate at
// max_frame_bytes"; the existing in-flight budget bounds the amplification).
let total_data_max = lim.max_frame_bytes.div_ceil(shard_bytes).max(1);
let total_data = frame_bytes.div_ceil(shard_bytes).max(1);
if sentinel {
if frame_bytes != 0
|| data_shards != lim.max_data_shards
|| block_idx + 1 >= lim.max_blocks
{
drop(stats);
return Ok(None);
}
} else {
if block_count > lim.max_blocks || block_idx >= block_count {
drop(stats);
return Ok(None);
}
// Derived-geometry firewall: every sender (our Packetizer, any version) slices a
// frame into consecutive blocks of exactly `max_data_per_block` data shards with
// only the LAST block smaller, and stamps the exact `frame_bytes` in every
// non-sentinel header. That makes every data shard's final AU offset computable on
// arrival —
// offset = (block_index × max_data_per_block + shard_index) × shard_bytes // offset = (block_index × max_data_per_block + shard_index) × shard_bytes
// — which is what lets shards land straight in the frame buffer below. Enforce the // — which is what lets shards land straight in the frame buffer below. Enforce the
// invariant so a header lying about its geometry is dropped instead of scribbling into // invariant so a header lying about its geometry is dropped instead of scribbling
// another shard's range. // into another shard's range.
let total_data = frame_bytes.div_ceil(shard_bytes).max(1);
let expect_blocks = total_data.div_ceil(lim.max_data_shards).max(1); let expect_blocks = total_data.div_ceil(lim.max_data_shards).max(1);
let block_idx = hdr.block_index as usize;
let expect_data_shards = if block_idx + 1 == expect_blocks { let expect_data_shards = if block_idx + 1 == expect_blocks {
total_data - (expect_blocks - 1) * lim.max_data_shards total_data - (expect_blocks - 1) * lim.max_data_shards
} else { } else {
@@ -305,6 +334,7 @@ impl Reassembler {
drop(stats); drop(stats);
return Ok(None); return Ok(None);
} }
}
let body = &pkt[HEADER_LEN..HEADER_LEN + shard_bytes]; let body = &pkt[HEADER_LEN..HEADER_LEN + shard_bytes];
// Route by index space: speed-test probe filler (FLAG_PROBE in user_flags) reassembles in // Route by index space: speed-test probe filler (FLAG_PROBE in user_flags) reassembles in
@@ -350,8 +380,13 @@ impl Reassembler {
} }
// First packet of a frame allocates its whole (zeroed) buffer, budget-gated; later // First packet of a frame allocates its whole (zeroed) buffer, budget-gated; later
// packets must agree with its geometry. // packets must agree with its geometry. A sentinel-opened (streamed) frame allocates at
let buf_len = total_data * shard_bytes; // the limits' maximum — its real size doesn't exist yet.
let buf_len = if sentinel {
total_data_max * shard_bytes
} else {
total_data * shard_bytes
};
let frame = match win.frames.entry(hdr.frame_index) { let frame = match win.frames.entry(hdr.frame_index) {
std::collections::hash_map::Entry::Occupied(e) => e.into_mut(), std::collections::hash_map::Entry::Occupied(e) => e.into_mut(),
std::collections::hash_map::Entry::Vacant(e) => { std::collections::hash_map::Entry::Vacant(e) => {
@@ -374,7 +409,66 @@ impl Reassembler {
}) })
} }
}; };
if frame.block_count != block_count || frame.frame_bytes != frame_bytes { if sentinel {
// Sentinel packets carry no totals to cross-check: while the frame is unpinned
// (`block_count == 0`) they match by construction, and once the totals are pinned —
// whichever packet arrived first, sentinel or final (reorder is normal) — a
// sentinel block must still be NON-final under them. Its full-K shape was already
// enforced by the firewall, which is exactly what the pinned geometry demands of
// every non-final block, and its shard offsets are within the pinned range by
// `block_idx + 1 < block_count`.
if frame.block_count != 0 && block_idx + 1 >= frame.block_count {
drop(stats);
return Ok(None);
}
} else if frame.block_count == 0 {
// A streamed frame meets its FINAL block's real totals: retro-validate every block
// the sentinels created against the exact geometry these totals derive (the same
// invariant the firewall enforces per-packet for legacy frames), then PIN them. A
// header that lies — totals under which an already-received sentinel block is
// out of range or not full-K — kills the WHOLE frame: its landed shards can't be
// trusted to be at the offsets this geometry means, so delivering any of it would
// hand the decoder spliced bytes.
let expect_blocks = total_data.div_ceil(lim.max_data_shards).max(1);
let final_k = total_data - (expect_blocks - 1) * lim.max_data_shards;
let lied = frame.blocks.iter().any(|(&bi, b)| {
let bi = bi as usize;
bi >= expect_blocks
|| (bi + 1 < expect_blocks && b.data_shards != lim.max_data_shards)
|| (bi + 1 == expect_blocks && b.data_shards != final_k)
});
if lied {
let mut f = win
.frames
.remove(&hdr.frame_index)
.expect("frame entry exists");
*in_flight_bytes -= f.buf.len();
// Remember the index (with its late-shard memory, exactly like an aged-out
// frame) so stragglers can't resurrect it, reclaim the parity buffers, and
// count the loss — the client's recovery request is the right outcome for a
// frame that was destroyed by a lying header.
win.completed.insert(
hdr.frame_index,
reconstructed_shards(&f.blocks, lim.max_data_shards),
);
for block in f.blocks.values_mut() {
for slot in block.recovery.iter_mut() {
if let Some(rb) = slot.take() {
if recovery_pool.len() < RECOVERY_POOL_MAX {
recovery_pool.push(rb);
}
}
}
}
if !is_probe {
StatsCounters::add(&stats.frames_dropped, 1);
}
drop(stats);
return Ok(None);
}
frame.frame_bytes = frame_bytes;
frame.block_count = block_count;
} else if frame.block_count != block_count || frame.frame_bytes != frame_bytes {
drop(stats); drop(stats);
return Ok(None); return Ok(None);
} }
@@ -382,6 +476,7 @@ impl Reassembler {
buf, buf,
blocks, blocks,
blocks_ok, blocks_ok,
block_count: frame_block_count,
.. ..
} = frame; } = frame;
@@ -485,8 +580,12 @@ impl Reassembler {
} }
} }
// Whole frame ready? // Whole frame ready? Judged against the FRAME's pinned block count, not this packet's
if *blocks_ok == block_count { // header — a streamed frame can complete on a reordered sentinel packet (header says 0)
// after the final block already pinned the real totals, and can never complete before
// they're pinned (`0` never equals a non-zero `blocks_ok`).
let block_count = *frame_block_count;
if block_count != 0 && *blocks_ok == block_count {
let mut done = win.frames.remove(&hdr.frame_index).unwrap(); let mut done = win.frames.remove(&hdr.frame_index).unwrap();
win.completed.insert( win.completed.insert(
hdr.frame_index, hdr.frame_index,
+291
View File
@@ -640,3 +640,294 @@ fn adaptive_fec_ramp_keeps_maximal_blocks_within_the_peers_ceiling() {
src src
); );
} }
// ---------------------------------------------------------------------------
// Streamed access units (VIDEO_CAP_STREAMED_AU — nvenc-subframe-slice-output.md Phase 2)
// ---------------------------------------------------------------------------
/// Packetize one streamed AU from `chunks` via begin/push/finish, returning the emitted wire
/// packets (header ++ shard) and the concatenated source bytes.
fn streamed_packets(
scheme: FecScheme,
fec_percent: u8,
chunks: &[&[u8]],
) -> (Vec<Vec<u8>>, Vec<u8>) {
let cfg = e2e_config(scheme, fec_percent);
let coder = coder_for(scheme);
let mut pk = Packetizer::new(&cfg);
let mut au = pk.begin_streamed(12345, 0, Some(0));
let mut pkts: Vec<Vec<u8>> = Vec::new();
let mut src = Vec::new();
for c in chunks {
src.extend_from_slice(c);
pk.push_streamed(&mut au, c, coder.as_ref(), |h: &PacketHeader, b: &[u8]| {
let mut p = Vec::with_capacity(HEADER_LEN + b.len());
p.extend_from_slice(h.as_bytes());
p.extend_from_slice(b);
pkts.push(p);
Ok(())
})
.unwrap();
}
pk.finish_streamed(au, coder.as_ref(), |h: &PacketHeader, b: &[u8]| {
let mut p = Vec::with_capacity(HEADER_LEN + b.len());
p.extend_from_slice(h.as_bytes());
p.extend_from_slice(b);
pkts.push(p);
Ok(())
})
.unwrap();
(pkts, src)
}
/// Deliver a streamed AU's packets (optionally with kills within the FEC budget, reversed
/// order, and a duplicate) and assert byte-identical completion. Reversed order is the
/// critical case: the FINAL block's real-total headers arrive FIRST, the frame opens
/// legacy-shaped, and the sentinels must still be accepted against the pinned totals.
fn streamed_roundtrip(scheme: FecScheme, kill: &[usize], reverse: bool) {
let chunks: Vec<Vec<u8>> = (0..3)
.map(|c| (0..50).map(|i| (c * 57 + i * 131 + 7) as u8).collect())
.collect();
let chunk_refs: Vec<&[u8]> = chunks.iter().map(|c| c.as_slice()).collect();
let (pkts, src) = streamed_packets(scheme, 50, &chunk_refs);
// 150 B / 16 B shards / 4-shard blocks → sentinel blocks 0,1 (4 data + 2 rec each) +
// final block (2 data + 1 rec) = 15 packets.
assert_eq!(
pkts.len(),
15,
"expected geometry changed — update the kills"
);
let mut delivery: Vec<Vec<u8>> = pkts
.iter()
.enumerate()
.filter(|(i, _)| !kill.contains(i))
.map(|(_, p)| p.clone())
.collect();
if reverse {
delivery.reverse();
}
if let Some(dup) = delivery.first().cloned() {
delivery.push(dup);
}
let cfg = e2e_config(scheme, 50);
let mut r = Reassembler::new(ReassemblerLimits::from_config(&cfg));
let coder = coder_for(scheme);
let stats = StatsCounters::default();
let mut got = None;
for p in &delivery {
if let Some(f) = r.push(p, coder.as_ref(), &stats).unwrap() {
assert!(got.is_none(), "frame must complete exactly once");
got = Some(f);
}
}
let f = got.expect("streamed frame must complete within the FEC budget");
assert_eq!(
f.data, src,
"reassembled streamed AU must be byte-identical"
);
assert_eq!(f.pts_ns, 12345);
assert!(f.complete);
}
#[test]
fn streamed_roundtrip_clean_and_reversed() {
streamed_roundtrip(FecScheme::Gf16, &[], false);
streamed_roundtrip(FecScheme::Gf16, &[], true);
streamed_roundtrip(FecScheme::Gf8, &[], true);
}
/// Loss within each block's FEC budget: one data shard from a sentinel block, one from the
/// final block — in both delivery orders.
#[test]
fn streamed_roundtrip_survives_loss_and_reorder() {
// Wire order: blk0 = 0..4 data + 4..6 rec, blk1 = 6..10 data + 10..12 rec,
// final = 12..14 data + 14 rec.
streamed_roundtrip(FecScheme::Gf16, &[1, 12], false);
streamed_roundtrip(FecScheme::Gf16, &[1, 12], true);
}
/// The wire shape of a streamed AU: sentinel headers (block_count = 0, frame_bytes = 0,
/// full-K) on every non-final block, real totals + EOF on the final block, SOF on the very
/// first packet only.
#[test]
fn streamed_headers_sentinel_then_final() {
let chunks: Vec<Vec<u8>> = (0..3).map(|_| vec![0xA5u8; 50]).collect();
let chunk_refs: Vec<&[u8]> = chunks.iter().map(|c| c.as_slice()).collect();
let (pkts, src) = streamed_packets(FecScheme::Gf16, 50, &chunk_refs);
let mut saw_final = false;
for (i, p) in pkts.iter().enumerate() {
let h = PacketHeader::read_from_bytes(&p[..HEADER_LEN]).unwrap();
assert_eq!(
h.flags & FLAG_SOF != 0,
i == 0,
"SOF exactly on the first packet"
);
assert_eq!(
h.flags & FLAG_EOF != 0,
i + 1 == pkts.len(),
"EOF exactly on the last packet"
);
if h.block_index < 2 {
assert_eq!(
h.block_count, 0,
"non-final block must ride sentinel headers"
);
assert_eq!(h.frame_bytes, 0);
assert_eq!(h.data_shards, 4, "sentinel blocks are exactly full-K");
} else {
saw_final = true;
assert_eq!(h.block_count, 3, "final block carries the real block count");
assert_eq!(h.frame_bytes as usize, src.len(), "and the real AU size");
}
}
assert!(saw_final);
}
/// A streamed AU smaller than one block emits NO sentinels — its single (final) block is
/// byte-identical in shape to a legacy frame, so small frames pay zero streaming overhead
/// and any receiver accepts them.
#[test]
fn streamed_small_frame_degenerates_to_legacy() {
let (pkts, src) = streamed_packets(FecScheme::Gf16, 50, &[&[0x5Au8; 40]]);
for p in &pkts {
let h = PacketHeader::read_from_bytes(&p[..HEADER_LEN]).unwrap();
assert_eq!(
h.block_count, 1,
"single-block streamed AU must be legacy-shaped"
);
assert_eq!(h.frame_bytes as usize, src.len());
}
}
/// Sentinel firewall: a sentinel header that is not exactly full-K, or claims a non-zero
/// total, or sits where the final block could no longer follow, is dropped before any
/// allocation happens.
#[test]
fn streamed_sentinel_firewall_bounds() {
let mut r = Reassembler::new(limits());
let coder = coder_for(FecScheme::Gf8);
let stats = StatsCounters::default();
let sentinel = |f: fn(&mut PacketHeader)| {
let mut h = base_header();
h.block_count = 0;
h.frame_bytes = 0;
h.data_shards = 8; // limits().max_data_shards — the only legal sentinel K
h.recovery_shards = 0;
f(&mut h);
h
};
// Not full-K.
let h = sentinel(|h| h.data_shards = 7);
assert!(r
.push(&packet(h), coder.as_ref(), &stats)
.unwrap()
.is_none());
// Claims a total.
let h = sentinel(|h| h.frame_bytes = 64);
assert!(r
.push(&packet(h), coder.as_ref(), &stats)
.unwrap()
.is_none());
// Sits on the last block the limits allow (no room for the final block after it).
let h = sentinel(|h| h.block_index = 3); // limits().max_blocks == 4
assert!(r
.push(&packet(h), coder.as_ref(), &stats)
.unwrap()
.is_none());
assert_eq!(stats.snapshot().packets_dropped, 3);
// A conformant sentinel IS accepted (proves the rejections above weren't vacuous).
let h = sentinel(|_| {});
assert!(r
.push(&packet(h), coder.as_ref(), &stats)
.unwrap()
.is_none());
assert_eq!(
stats.snapshot().packets_dropped,
3,
"conformant sentinel accepted"
);
}
/// Retro-validation: final-block totals under which an already-received sentinel block is
/// out of range (or mis-sized) kill the WHOLE frame — no spliced delivery — and the killed
/// index cannot be resurrected by stragglers.
#[test]
fn streamed_lying_final_totals_kill_the_frame_wholesale() {
let mut r = Reassembler::new(limits());
let coder = coder_for(FecScheme::Gf8);
let stats = StatsCounters::default();
// Two sentinel blocks (indexes 0 and 1) open the frame and land shards.
for bi in 0..2u16 {
let mut h = base_header();
h.block_count = 0;
h.frame_bytes = 0;
h.data_shards = 8;
h.recovery_shards = 0;
h.block_index = bi;
assert!(r
.push(&packet(h), coder.as_ref(), &stats)
.unwrap()
.is_none());
}
// A "final" header claiming the whole AU is ONE 16-byte shard: geometry-valid on its own
// (expect_blocks = 1, K = 1), but it disowns both sentinel blocks → the frame dies.
let mut lying = base_header();
lying.block_count = 1;
lying.frame_bytes = 16;
lying.data_shards = 1;
lying.recovery_shards = 0;
assert!(r
.push(&packet(lying), coder.as_ref(), &stats)
.unwrap()
.is_none());
let snap = stats.snapshot();
assert_eq!(
snap.frames_dropped, 1,
"the lying frame must be counted lost"
);
// A straggler sentinel for the killed index must not resurrect it.
let mut h = base_header();
h.block_count = 0;
h.frame_bytes = 0;
h.data_shards = 8;
h.recovery_shards = 0;
let before = stats.snapshot().packets_dropped;
assert!(r
.push(&packet(h), coder.as_ref(), &stats)
.unwrap()
.is_none());
assert_eq!(
stats.snapshot().packets_dropped,
before + 1,
"straggler for a killed frame must be dropped, not re-open it"
);
}
/// A sentinel first-packet commits a MAX-sized frame buffer, so the in-flight budget must
/// bite after IN_FLIGHT_BUF_FACTOR frames — the amplification bound for one-datagram opens.
#[test]
fn streamed_open_amplification_is_budget_bounded() {
let mut r = Reassembler::new(limits());
let coder = coder_for(FecScheme::Gf8);
let stats = StatsCounters::default();
// limits(): max_frame_bytes 4096 → each sentinel open commits 4096 B; budget = 4×4096.
for fi in 0..5u32 {
let mut h = base_header();
h.block_count = 0;
h.frame_bytes = 0;
h.data_shards = 8;
h.recovery_shards = 0;
h.frame_index = fi;
assert!(r
.push(&packet(h), coder.as_ref(), &stats)
.unwrap()
.is_none());
}
assert_eq!(
stats.snapshot().packets_dropped,
1,
"the fifth max-sized open must be refused by the in-flight budget"
);
}
+12
View File
@@ -32,6 +32,18 @@ pub const VIDEO_CAP_HOST_TIMING: u8 = 0x08;
/// older client gets a declined (zeroed) [`ProbeResult`] instead of a measurement its single-window /// older client gets a declined (zeroed) [`ProbeResult`] instead of a measurement its single-window
/// reassembler would silently drop as stale. /// reassembler would silently drop as stale.
pub const VIDEO_CAP_PROBE_SEQ: u8 = 0x10; pub const VIDEO_CAP_PROBE_SEQ: u8 = 0x10;
/// [`Hello::video_caps`] bit: the client's reassembler accepts **streamed access units**
/// (design/nvenc-subframe-slice-output.md Phase 2): the host may ship an AU's early FEC blocks
/// before the AU's total size exists — while the tail of the frame is still encoding — so the
/// AU's last packet leaves the host sooner (latency plan §7 LN1). Non-final blocks ride
/// SENTINEL headers (`block_count == 0` — a value no legacy sender emits — with
/// `frame_bytes == 0` and exactly `max_data_per_block` data shards, so the shard-offset
/// formula needs no total); the FINAL block's headers carry the real
/// `frame_bytes`/`block_count` (+ `FLAG_EOF`), which retro-validate the whole frame's geometry
/// — a mismatch drops the frame wholesale. The host streams ONLY to clients advertising this
/// bit; every other client gets today's whole-AU path (chunks concatenated before sealing), so
/// the fallback is zero-risk.
pub const VIDEO_CAP_STREAMED_AU: u8 = 0x20;
/// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`] /// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
/// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder /// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder
+73 -6
View File
@@ -13,7 +13,9 @@ use crate::crypto::SessionCrypto;
use crate::error::{PunktfunkError, Result}; use crate::error::{PunktfunkError, Result};
use crate::fec::{coder_for, ErasureCoder}; use crate::fec::{coder_for, ErasureCoder};
use crate::input::InputEvent; use crate::input::InputEvent;
use crate::packet::{Packetizer, Reassembler, ReassemblerLimits, MAX_DATAGRAM_BYTES}; use crate::packet::{
PacketHeader, Packetizer, Reassembler, ReassemblerLimits, StreamedAu, MAX_DATAGRAM_BYTES,
};
use crate::stats::{Stats, StatsCounters}; use crate::stats::{Stats, StatsCounters};
use crate::transport::Transport; use crate::transport::Transport;
use zerocopy::IntoBytes; use zerocopy::IntoBytes;
@@ -274,6 +276,68 @@ impl Session {
pts_ns: u64, pts_ns: u64,
user_flags: u32, user_flags: u32,
frame_index: Option<u32>, frame_index: Option<u32>,
) -> Result<Vec<Vec<u8>>> {
self.seal_run(true, |p, coder, emit| {
p.packetize_each(data, pts_ns, user_flags, frame_index, coder, emit)
})
}
/// Host: open a **streamed** access unit ([`crate::quic::VIDEO_CAP_STREAMED_AU`] — only
/// toward a client that advertised it; anyone else must get the whole-AU
/// [`seal_frame_at`](Self::seal_frame_at) path). The AU's bytes are fed in with
/// [`seal_streamed_chunk`](Self::seal_streamed_chunk) as the encoder produces them and
/// closed with [`seal_streamed_finish`](Self::seal_streamed_finish); the three calls'
/// returned wire batches are ONE frame, and the nonce order is emission order across the
/// calls — send each batch as it is returned, before sealing the next.
pub fn begin_streamed_frame_at(
&mut self,
pts_ns: u64,
user_flags: u32,
frame_index: u32,
) -> Result<StreamedAu> {
if self.config.role != Role::Host {
return Err(PunktfunkError::InvalidArg(
"seal_frame called on a client session",
));
}
Ok(self
.packetizer
.begin_streamed(pts_ns, user_flags, Some(frame_index)))
}
/// Feed one encoder chunk into a streamed AU, sealing every FEC block it completes under
/// sentinel headers (see [`begin_streamed_frame_at`](Self::begin_streamed_frame_at)). The
/// returned batch is often EMPTY (the chunk is buffered until a block fills) — that's
/// normal, not an error.
pub fn seal_streamed_chunk(
&mut self,
au: &mut StreamedAu,
chunk: &[u8],
) -> Result<Vec<Vec<u8>>> {
self.seal_run(false, |p, coder, emit| {
p.push_streamed(au, chunk, coder, emit)
})
}
/// Close a streamed AU: seal the final block with the real totals (+ `FLAG_EOF`), which
/// retro-validate the frame at the receiver. Counts the frame as submitted.
pub fn seal_streamed_finish(&mut self, au: StreamedAu) -> Result<Vec<Vec<u8>>> {
self.seal_run(true, |p, coder, emit| p.finish_streamed(au, coder, emit))
}
/// The shared packetize → pooled-wire → seal machinery behind [`seal_frame`](Self::seal_frame)
/// and the streamed sealers: `run` drives the packetizer against an emit sink that writes
/// each packet's plaintext at its final wire offset; the seal pass (two-lane for large runs)
/// then encrypts in place. `count_frame` gates the per-frame stats — a streamed AU counts
/// once, at its finish call.
fn seal_run(
&mut self,
count_frame: bool,
run: impl FnOnce(
&mut Packetizer,
&dyn ErasureCoder,
&mut dyn FnMut(&PacketHeader, &[u8]) -> Result<()>,
) -> Result<()>,
) -> Result<Vec<Vec<u8>>> { ) -> Result<Vec<Vec<u8>>> {
if self.config.role != Role::Host { if self.config.role != Role::Host {
return Err(PunktfunkError::InvalidArg( return Err(PunktfunkError::InvalidArg(
@@ -320,10 +384,10 @@ impl Session {
// sealing itself is a separate pass so it can split across lanes. // sealing itself is a separate pass so it can split across lanes.
let seq_base = *next_seq; let seq_base = *next_seq;
let encrypting = crypto.is_some(); let encrypting = crypto.is_some();
let result = packetizer.packetize_each(data, pts_ns, user_flags, frame_index, coder_ref, { let result = {
let wires = &mut wires; let wires = &mut wires;
let used = &mut used; let used = &mut used;
move |hdr, body| { let mut emit = move |hdr: &PacketHeader, body: &[u8]| -> Result<()> {
if *used == wires.len() { if *used == wires.len() {
wires.push(Vec::new()); wires.push(Vec::new());
} }
@@ -342,8 +406,9 @@ impl Session {
wire.extend_from_slice(body); wire.extend_from_slice(body);
} }
Ok(()) Ok(())
} };
}); run(packetizer, coder_ref, &mut emit)
};
result?; result?;
// A smaller frame uses fewer buffers than the pool holds: drop the unused tail, same // A smaller frame uses fewer buffers than the pool holds: drop the unused tail, same
// as the previous `resize_with(packets.len(), ..)` did. (Before the seal phase, so a // as the previous `resize_with(packets.len(), ..)` did. (Before the seal phase, so a
@@ -421,10 +486,12 @@ impl Session {
if let Some(p) = self.seal_perf.as_mut() { if let Some(p) = self.seal_perf.as_mut() {
p.fec_ns += fec_ns.load(std::sync::atomic::Ordering::Relaxed); p.fec_ns += fec_ns.load(std::sync::atomic::Ordering::Relaxed);
p.seal_ns += seal_ns; p.seal_ns += seal_ns;
p.frames += 1; p.frames += count_frame as u64;
p.packets += used as u64; p.packets += used as u64;
} }
if count_frame {
StatsCounters::add(&self.stats.frames_submitted, 1); StatsCounters::add(&self.stats.frames_submitted, 1);
}
let bytes: u64 = wires.iter().map(|w| w.len() as u64).sum(); let bytes: u64 = wires.iter().map(|w| w.len() as u64).sum();
StatsCounters::add(&self.stats.packets_sent, wires.len() as u64); StatsCounters::add(&self.stats.packets_sent, wires.len() as u64);
StatsCounters::add(&self.stats.bytes_sent, bytes); StatsCounters::add(&self.stats.bytes_sent, bytes);
+4
View File
@@ -1238,6 +1238,9 @@ async fn serve_session(
// so mid-session bursts don't consume video frame indexes. An older client (bit clear) gets // so mid-session bursts don't consume video frame indexes. An older client (bit clear) gets
// mid-session probes declined instead — see `run_probe_burst`. // mid-session probes declined instead — see `run_probe_burst`.
let probe_seq = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ != 0; let probe_seq = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_PROBE_SEQ != 0;
// Streamed-AU capability: the client's reassembler accepts sentinel-headed streamed blocks,
// so a chunked encoder session may ship an AU's early FEC blocks while its tail encodes.
let streamed_au = hello.video_caps & punktfunk_core::quic::VIDEO_CAP_STREAMED_AU != 0;
let stats_dp = stats; // data-plane handle to the shared stats recorder let stats_dp = stats; // data-plane handle to the shared stats recorder
// Short label for web-console stats captures: the client's cert-fingerprint prefix, else its // Short label for web-console stats captures: the client's cert-fingerprint prefix, else its
// peer IP (no fingerprint = anonymous TOFU/--open client). // peer IP (no fingerprint = anonymous TOFU/--open client).
@@ -1330,6 +1333,7 @@ async fn serve_session(
conn: conn_stream, conn: conn_stream,
timing_conn, timing_conn,
probe_seq, probe_seq,
streamed_au,
stats: stats_dp, stats: stats_dp,
client_label, client_label,
launch: launch_for_dp, launch: launch_for_dp,
+283 -11
View File
@@ -252,6 +252,19 @@ fn paced_submit(
let wires = session let wires = session
.seal_frame_at(data, pts_ns, flags, frame_index) .seal_frame_at(data, pts_ns, flags, frame_index)
.map_err(|e| anyhow!("seal_frame: {e:?}"))?; .map_err(|e| anyhow!("seal_frame: {e:?}"))?;
pace_sealed(session, wires, deadline, burst_cap, pace_rate_bps)
}
/// The pace-and-send half of [`paced_submit`], for wires that are ALREADY sealed — shared with
/// the streamed-AU path, whose seal happens per encoder chunk ([`handle_chunk`]) under the same
/// microburst policy and frame deadline.
fn pace_sealed(
session: &mut Session,
wires: Vec<Vec<u8>>,
deadline: std::time::Instant,
burst_cap: Option<usize>,
pace_rate_bps: u64,
) -> Result<PaceStat> {
let mut refs: Vec<&[u8]> = wires.iter().map(|w| w.as_slice()).collect(); let mut refs: Vec<&[u8]> = wires.iter().map(|w| w.as_slice()).collect();
// FEC/recovery test knob (PUNKTFUNK_VIDEO_DROP) — same knob the GameStream plane honors. // FEC/recovery test knob (PUNKTFUNK_VIDEO_DROP) — same knob the GameStream plane honors.
crate::send_pacing::inject_video_drop(&mut refs); crate::send_pacing::inject_video_drop(&mut refs);
@@ -335,6 +348,116 @@ struct FrameMsg {
was_measured: bool, was_measured: bool,
} }
/// What the encode thread hands the send thread: a whole AU (the legacy path — every session
/// shape except a chunked encoder toward a streamed-capable client), or one slice-boundary
/// chunk of a streamed AU (§7 LN1 Phase 2 — the send thread seals/paces each chunk's completed
/// FEC blocks while the encoder still produces the AU's tail).
enum SendMsg {
Frame(FrameMsg),
Chunk(ChunkMsg),
}
/// One encoder chunk of a streamed AU. AU-level fields (`capture_ns`/`flags`/`frame_index`/
/// `deadline`) are identical on every chunk of one AU (the send thread opens the streamed seal
/// at `first`); the perf split fields are meaningful on `last` (whole-AU figures, exactly like
/// [`FrameMsg`]'s).
struct ChunkMsg {
data: Vec<u8>,
first: bool,
last: bool,
capture_ns: u64,
flags: u32,
frame_index: u32,
deadline: std::time::Instant,
encode_us: u32,
queue_us: u32,
cap_us: u32,
submit_us: u32,
wait_us: u32,
repeat: bool,
was_measured: bool,
}
/// A streamed AU the send thread has open: the core's incremental sealer plus the pace
/// aggregation across its per-chunk flushes (the accounting the whole-AU path reads off one
/// [`paced_submit`] call).
struct StreamedOpen {
au: punktfunk_core::packet::StreamedAu,
spread_us: u32,
paced: bool,
}
/// Feed one [`ChunkMsg`] through the streamed sealer: open at `first`, seal + pace every FEC
/// block the chunk completes, close (+ final block, real totals) at `last`. Returns
/// `Some((accounting, aggregated PaceStat))` when the AU finished — the caller runs the same
/// per-AU accounting as the whole-frame path — and `None` mid-AU.
fn handle_chunk(
session: &mut Session,
open: &mut Option<StreamedOpen>,
c: ChunkMsg,
burst_cap: Option<usize>,
pace_rate_bps: u64,
) -> Result<Option<(FrameMsg, PaceStat)>> {
if c.first {
if open.take().is_some() {
// The encode loop abandoned a mid-flight AU (an encoder stall/rebuild forfeits the
// in-flight frame). Its sentinel packets are already on the wire — the client ages
// that frame out and the rebuild's IDR re-anchors; just don't leak the open state.
tracing::warn!(
"streamed AU abandoned mid-flight (encoder rebuild) — client ages it out"
);
}
*open = Some(StreamedOpen {
au: session
.begin_streamed_frame_at(c.capture_ns, c.flags, c.frame_index)
.map_err(|e| anyhow!("begin_streamed_frame: {e:?}"))?,
spread_us: 0,
paced: false,
});
}
let Some(s) = open.as_mut() else {
return Err(anyhow!(
"streamed chunk without an open AU (encode-loop bug)"
));
};
let wires = session
.seal_streamed_chunk(&mut s.au, &c.data)
.map_err(|e| anyhow!("seal_streamed_chunk: {e:?}"))?;
if !wires.is_empty() {
let stat = pace_sealed(session, wires, c.deadline, burst_cap, pace_rate_bps)?;
s.spread_us = s.spread_us.saturating_add(stat.spread_us);
s.paced |= stat.paced;
}
if !c.last {
return Ok(None);
}
let s = open.take().expect("checked above");
let tail = session
.seal_streamed_finish(s.au)
.map_err(|e| anyhow!("seal_streamed_finish: {e:?}"))?;
let stat = pace_sealed(session, tail, c.deadline, burst_cap, pace_rate_bps)?;
Ok(Some((
FrameMsg {
data: Vec::new(), // already on the wire — accounting only
capture_ns: c.capture_ns,
flags: c.flags,
frame_index: c.frame_index,
deadline: c.deadline,
encode_us: c.encode_us,
queue_us: c.queue_us,
cap_us: c.cap_us,
submit_us: c.submit_us,
wait_us: c.wait_us,
repeat: c.repeat,
was_measured: c.was_measured,
},
PaceStat {
spread_us: s.spread_us.saturating_add(stat.spread_us),
paced: s.paced || stat.paced,
},
)))
}
/// The dedicated send thread: it owns the whole [`Session`] (so no socket clone or shared stats are /// The dedicated send thread: it owns the whole [`Session`] (so no socket clone or shared stats are
/// needed) and does FEC+seal + microburst-paced send OFF the capture/encode thread, plus the /// needed) and does FEC+seal + microburst-paced send OFF the capture/encode thread, plus the
/// speed-test probe bursts (which also need the Session). Decoupling the paced send from encoding /// speed-test probe bursts (which also need the Session). Decoupling the paced send from encoding
@@ -380,7 +503,7 @@ pub(super) fn reconfig_allowed(
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
fn send_loop( fn send_loop(
mut session: Session, mut session: Session,
frame_rx: std::sync::mpsc::Receiver<FrameMsg>, frame_rx: std::sync::mpsc::Receiver<SendMsg>,
probe_rx: std::sync::mpsc::Receiver<ProbeRequest>, probe_rx: std::sync::mpsc::Receiver<ProbeRequest>,
probe_result_tx: tokio::sync::mpsc::UnboundedSender<ProbeResult>, probe_result_tx: tokio::sync::mpsc::UnboundedSender<ProbeResult>,
stop: Arc<AtomicBool>, stop: Arc<AtomicBool>,
@@ -425,18 +548,33 @@ fn send_loop(
let mut last_frames_dropped = 0u64; let mut last_frames_dropped = 0u64;
let mut last_packets_dropped = 0u64; let mut last_packets_dropped = 0u64;
let mut last_fec_recovered = 0u64; let mut last_fec_recovered = 0u64;
// The streamed AU currently open (VIDEO_CAP_STREAMED_AU chunked sends) — `Some` strictly
// between a `ChunkMsg::first` and its `last`.
let mut streamed: Option<StreamedOpen> = None;
loop { loop {
if stop.load(Ordering::SeqCst) { if stop.load(Ordering::SeqCst) {
break; break;
} }
// Probes run here (they need the Session); a burst pauses video — the encode thread blocks // Probes run here (they need the Session); a burst pauses video — the encode thread blocks
// on the full frame channel meanwhile, which is exactly the intended pause. // on the full frame channel meanwhile, which is exactly the intended pause. Never mid-AU:
// a streamed frame's chunks are already leaving the socket, so a burst spliced between
// them would push the AU's tail past its deadline (the exact latency the mode removes).
if streamed.is_none() {
service_probes(&mut session, &stop, &probe_rx, &probe_result_tx, probe_seq); service_probes(&mut session, &stop, &probe_rx, &probe_result_tx, probe_seq);
}
// Adaptive FEC: pick up any new recovery target the control task set from client LossReports. // Adaptive FEC: pick up any new recovery target the control task set from client LossReports.
apply_fec_target(&mut session, &fec_target); apply_fec_target(&mut session, &fec_target);
// Short timeout so we keep re-checking `stop` + probes when no frames are flowing. // Short timeout so we keep re-checking `stop` + probes when no frames are flowing.
match frame_rx.recv_timeout(std::time::Duration::from_millis(50)) { match frame_rx.recv_timeout(std::time::Duration::from_millis(50)) {
Ok(msg) => match paced_submit( Ok(send_msg) => {
// Live ABR-tracked encoder bitrate → pace rate; 0 (not yet known) = uncapped.
let pace_rate = (stats.bitrate_kbps.load(Ordering::Relaxed) as f64
* 1000.0
* pace_factor) as u64;
// `Ok(Some(..))` = an AU fully left the socket (a whole frame, or a streamed
// AU's last chunk) — run the per-AU accounting; `Ok(None)` = mid-AU chunk.
let outcome = match send_msg {
SendMsg::Frame(msg) => paced_submit(
&mut session, &mut session,
&msg.data, &msg.data,
msg.capture_ns, msg.capture_ns,
@@ -444,10 +582,17 @@ fn send_loop(
msg.frame_index, msg.frame_index,
msg.deadline, msg.deadline,
burst_cap, burst_cap,
// Live ABR-tracked encoder bitrate → pace rate; 0 (not yet known) = uncapped. pace_rate,
(stats.bitrate_kbps.load(Ordering::Relaxed) as f64 * 1000.0 * pace_factor) as u64, )
) { .map(|stat| Some((msg, stat))),
Ok(stat) => { SendMsg::Chunk(c) => {
handle_chunk(&mut session, &mut streamed, c, burst_cap, pace_rate)
}
};
match outcome {
// Mid-AU chunk: sealed + on the wire; the per-AU accounting runs at `last`.
Ok(None) => {}
Ok(Some((msg, stat))) => {
// First VIDEO packets are on the wire — complete the bring-up trace (P0.1; // First VIDEO packets are on the wire — complete the bring-up trace (P0.1;
// once-only, no-op on every later frame). Speed-test filler isn't video. // once-only, no-op on every later frame). Speed-test filler isn't video.
if msg.flags & FLAG_PROBE as u32 == 0 { if msg.flags & FLAG_PROBE as u32 == 0 {
@@ -514,7 +659,8 @@ fn send_loop(
tracing::error!(error = %format!("{e:#}"), "send failed — stopping stream"); tracing::error!(error = %format!("{e:#}"), "send failed — stopping stream");
break; break;
} }
}, }
}
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {} Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {}
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => break, // encode thread done Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => break, // encode thread done
} }
@@ -798,6 +944,11 @@ pub(super) struct SessionContext {
/// stale — mid-session probes are DECLINED for it (a zeroed [`ProbeResult`]) rather than /// stale — mid-session probes are DECLINED for it (a zeroed [`ProbeResult`]) rather than
/// consuming video frame indexes its gap detectors can't see (the phantom-gap freeze). /// consuming video frame indexes its gap detectors can't see (the phantom-gap freeze).
pub(super) probe_seq: bool, pub(super) probe_seq: bool,
/// The client advertised [`punktfunk_core::quic::VIDEO_CAP_STREAMED_AU`]: when the session's
/// encoder runs chunked poll (multi-slice sub-frame readback, §7 LN1), the host streams each
/// AU's FEC blocks under sentinel headers as the slices complete instead of waiting for the
/// whole AU. `false` = older client — chunks (if any) are drained whole-AU, zero wire change.
pub(super) streamed_au: bool,
/// Shared streaming-stats recorder. The capture loop reads `is_armed()` per frame to decide /// Shared streaming-stats recorder. The capture loop reads `is_armed()` per frame to decide
/// whether to measure the per-stage split; the send thread builds + pushes the aggregated /// whether to measure the per-stage split; the send thread builds + pushes the aggregated
/// `StatsSample` at its 2 s boundary. /// `StatsSample` at its 2 s boundary.
@@ -866,6 +1017,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
conn, conn,
timing_conn, timing_conn,
probe_seq, probe_seq,
streamed_au,
stats, stats,
client_label, client_label,
launch, launch,
@@ -873,6 +1025,16 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
bringup, bringup,
resize_ms, resize_ms,
} = ctx; } = ctx;
// Streamed-AU wire mode: the client's cap AND the host escape hatch (`PUNKTFUNK_STREAMED_AU=0`
// reverts to whole-AU sends without touching the encoder's slicing knobs). The third gate —
// whether the ENCODER actually chunks — is dynamic (`supports_chunked_poll`, per AU).
let streamed_wire = streamed_au && std::env::var("PUNKTFUNK_STREAMED_AU").as_deref() != Ok("0");
if streamed_wire {
tracing::info!(
"client accepts streamed AUs (VIDEO_CAP_STREAMED_AU) — chunked encoder output \
will stream per-slice"
);
}
tracing::info!( tracing::info!(
compositor = compositor.id(), compositor = compositor.id(),
?mode, ?mode,
@@ -1032,7 +1194,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// encode of frame N+1 overlaps the paced transmit of frame N instead of waiting behind its tail. // encode of frame N+1 overlaps the paced transmit of frame N instead of waiting behind its tail.
// The bounded channel applies backpressure (the encode thread blocks if the send falls behind, // The bounded channel applies backpressure (the encode thread blocks if the send falls behind,
// so frames slow down rather than a dropped frame freezing the infinite-GOP stream). // so frames slow down rather than a dropped frame freezing the infinite-GOP stream).
let (frame_tx, frame_rx) = std::sync::mpsc::sync_channel::<FrameMsg>(3); let (frame_tx, frame_rx) = std::sync::mpsc::sync_channel::<SendMsg>(3);
// Live encoder bitrate, shared with the send thread's stats sample: a mid-stream adaptive // Live encoder bitrate, shared with the send thread's stats sample: a mid-stream adaptive
// bitrate change (bitrate_rx below) updates it so the console shows the actual target. // bitrate change (bitrate_rx below) updates it so the console shows the actual target.
let live_bitrate = Arc::new(AtomicU32::new(bitrate_kbps)); let live_bitrate = Arc::new(AtomicU32::new(bitrate_kbps));
@@ -1932,6 +2094,116 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// carry it to the shared stall recovery below instead of killing the session outright. // carry it to the shared stall recovery below instead of killing the session outright.
let mut poll_err: Option<anyhow::Error> = None; let mut poll_err: Option<anyhow::Error> = None;
while inflight.len() >= depth { while inflight.len() >= depth {
// Streamed chunked drain (§7 LN1 Phase 2): toward a STREAMED_AU client with the
// encoder's chunked poll live, forward each slice chunk to the send thread the
// moment it's readable, so packetize/FEC/pacing overlap the encode tail. Re-queried
// per AU (never cached): a pipelined-retrieve escalation or a session rebuild turns
// the mode off and the next AU falls back to the whole-AU path below.
if streamed_wire && enc.supports_chunked_poll() {
let t_wait = std::time::Instant::now();
let mut first_chunk_us = 0u32;
let mut au_flags = 0u32;
let mut au_done = false;
loop {
let c = match enc.poll_chunk() {
Ok(Some(c)) => c,
Ok(None) => break, // defensive: nothing in flight
Err(e) => {
poll_err = Some(e);
break;
}
};
// Every chunk proves the encoder is alive.
last_au_at = std::time::Instant::now();
encoder_resets = 0;
if c.first {
first_chunk_us = t_wait.elapsed().as_micros() as u32;
au_flags = if c.keyframe {
(FLAG_PIC | FLAG_SOF) as u32
} else {
FLAG_PIC as u32
};
let caps = enc.caps();
if caps.intra_refresh_recovery
&& caps.intra_refresh_period > 0
&& mark_recovery_boundary(
&mut ir_wave_pos,
c.keyframe,
caps.intra_refresh_period,
)
{
au_flags |= punktfunk_core::packet::USER_FLAG_RECOVERY_POINT;
}
if c.recovery_anchor {
au_flags |= punktfunk_core::packet::USER_FLAG_RECOVERY_ANCHOR;
}
if c.chunk_aligned {
au_flags |= punktfunk_core::packet::USER_FLAG_CHUNK_ALIGNED;
}
if let Some(m) = last_hdr_meta {
if c.keyframe || resend_meta {
let _ = conn.send_datagram(
punktfunk_core::quic::encode_hdr_meta_datagram(&m).into(),
);
resend_meta = false;
}
}
bringup.mark("first_au");
}
let last = c.last;
let (cap_ns, sub_ns, deadline) = *inflight.front().expect("inflight non-empty");
let wait_total_us = t_wait.elapsed().as_micros() as u32;
let encode_us = (now_ns().saturating_sub(sub_ns) / 1000) as u32;
let msg = ChunkMsg {
data: c.data,
first: c.first,
last,
capture_ns: cap_ns,
flags: au_flags,
frame_index: au_seq,
deadline,
encode_us,
queue_us,
cap_us,
submit_us,
wait_us: if measure { wait_total_us } else { 0 },
repeat,
was_measured: measure,
};
if frame_tx.send(SendMsg::Chunk(msg)).is_err() {
send_gone = true;
break;
}
if last {
inflight.pop_front();
au_seq = au_seq.wrapping_add(1);
sent += 1;
au_done = true;
if perf {
st_wait.push(wait_total_us);
// The overlap measurement the Phase-3 gate needs (sampled): how
// early the first slice reached the send thread vs. the whole
// encode — the win is roughly their difference per AU.
if sent % 120 == 0 {
tracing::info!(
first_slice_us = first_chunk_us,
encode_us,
"streamed AU (sampled): first slice handed to send at \
first_slice_us; encode finished at encode_us"
);
}
}
break;
}
}
if send_gone || poll_err.is_some() {
break;
}
if au_done {
continue; // drain the next in-flight frame, if depth allows
}
break; // defensive Ok(None): leave the frame in flight, re-poll next tick
}
let t_wait = std::time::Instant::now(); let t_wait = std::time::Instant::now();
let polled = enc.poll(); let polled = enc.poll();
let wait_us = if measure { let wait_us = if measure {
@@ -2012,7 +2284,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
// Hand to the send thread; this blocks (backpressure) if it's behind. An Err means it // Hand to the send thread; this blocks (backpressure) if it's behind. An Err means it
// exited (send failure / stop) — end the encode loop too. // exited (send failure / stop) — end the encode loop too.
bringup.mark("first_au"); // P0.1 (first-crossing only; free afterwards) bringup.mark("first_au"); // P0.1 (first-crossing only; free afterwards)
if frame_tx.send(msg).is_err() { if frame_tx.send(SendMsg::Frame(msg)).is_err() {
send_gone = true; send_gone = true;
break; break;
} }
@@ -2157,7 +2429,7 @@ pub(super) fn virtual_stream(ctx: SessionContext, prepared: Option<PreparedDispl
repeat: false, repeat: false,
was_measured: false, was_measured: false,
}; };
if frame_tx.send(msg).is_err() { if frame_tx.send(SendMsg::Frame(msg)).is_err() {
break; break;
} }
au_seq = au_seq.wrapping_add(1); au_seq = au_seq.wrapping_add(1);
+15
View File
@@ -450,6 +450,21 @@
#define VIDEO_CAP_PROBE_SEQ 16 #define VIDEO_CAP_PROBE_SEQ 16
#endif #endif
#if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Hello::video_caps`] bit: the client's reassembler accepts **streamed access units**
// (design/nvenc-subframe-slice-output.md Phase 2): the host may ship an AU's early FEC blocks
// before the AU's total size exists — while the tail of the frame is still encoding — so the
// AU's last packet leaves the host sooner (latency plan §7 LN1). Non-final blocks ride
// SENTINEL headers (`block_count == 0` — a value no legacy sender emits — with
// `frame_bytes == 0` and exactly `max_data_per_block` data shards, so the shard-offset
// formula needs no total); the FINAL block's headers carry the real
// `frame_bytes`/`block_count` (+ `FLAG_EOF`), which retro-validate the whole frame's geometry
// — a mismatch drops the frame wholesale. The host streams ONLY to clients advertising this
// bit; every other client gets today's whole-AU path (chunks concatenated before sealing), so
// the fallback is zero-risk.
#define VIDEO_CAP_STREAMED_AU 32
#endif
#if defined(PUNKTFUNK_FEATURE_QUIC) #if defined(PUNKTFUNK_FEATURE_QUIC)
// [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`] // [`Welcome::host_caps`] bit: the host applies [`InputKind::GamepadState`]
// (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder // (crate::input::InputKind::GamepadState) snapshot events — full per-pad state with a reorder