feat(1gbps): raise bitrate/probe clamps + socket buffers, count send-buffer drops
ci / rust (push) Has been cancelled

First step of 1 Gbps+ readiness (the whole point of the GF(2^16) Leopard FEC):
make 1 Gbps configurable and its dominant failure mode observable, before the
real transport work (sendmmsg + paced encode|send split) lands.

Investigation (6-way) verdict: we're ~halfway, and it's mostly clamps plus one
real piece of work. The integer/type path, FEC (a 1 Gbps frame is only a few
hundred shards in one GF(2^16) block, far under the 65535 ceiling), AES-GCM
(AES-NI, ~10-25x headroom), and the M1 reassembler bounds (fully derived from
the negotiated FecConfig) are ALL already 1 Gbps-ready and untouched.

This commit (the configurable + observable foundation):
- m3.rs: MAX_BITRATE_KBPS 500_000 -> 2_000_000 (2 Gbps headroom over the 1 Gbps+
  target); MAX_PROBE_KBPS 1_000_000 -> 3_000_000 (probe can demonstrate headroom
  ABOVE the session cap so a client can confidently pick a 1 Gbps+ bitrate).
- transport/udp.rs: TARGET_SOCKBUF 8 MB -> 32 MB (a multi-MB IDR keyframe burst
  no longer fills the buffer); scripts/99-punktfunk-net.conf bumped to match.
- Observability: Transport::send now returns Ok(true|false) (false = WouldBlock
  send-buffer drop, previously a silent Ok(())). Session counts these as a new
  `packets_send_dropped` stat (distinct from recv-side packets_dropped) — in
  Stats, the C ABI PunktfunkStats (header regenerated), a PUNKTFUNK_PERF periodic
  wire-Mbps + drop dump in virtual_stream, and the speed-test probe completion
  log. This is the dominant 1 Gbps+ loss mode and was invisible.

Loopback-verified: a probe now runs at 1.2 Gbps target (no longer truncated to
1 Gbps) with the drop counter live. NOT yet a sustained-1-Gbps proof — the
single-send()-per-packet native path is the next, real piece of work (port the
proven GameStream sendmmsg + paced send thread into the core Transport).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-11 20:45:49 +00:00
parent 902cc162f7
commit b8a33e21a2
9 changed files with 98 additions and 32 deletions
+4
View File
@@ -136,6 +136,9 @@ pub struct PunktfunkStats {
pub packets_sent: u64,
pub packets_received: u64,
pub packets_dropped: u64,
/// Packets dropped on the host send path because the kernel buffer was full (WouldBlock) — the
/// dominant loss mode at very high bitrate; distinct from `packets_dropped` (recv-side).
pub packets_send_dropped: u64,
pub fec_recovered_shards: u64,
pub bytes_sent: u64,
pub bytes_received: u64,
@@ -150,6 +153,7 @@ impl From<Stats> for PunktfunkStats {
packets_sent: s.packets_sent,
packets_received: s.packets_received,
packets_dropped: s.packets_dropped,
packets_send_dropped: s.packets_send_dropped,
fec_recovered_shards: s.fec_recovered_shards,
bytes_sent: s.bytes_sent,
bytes_received: s.bytes_received,
+6 -2
View File
@@ -122,7 +122,9 @@ impl Session {
let wire = self.seal_for_wire(&pkt)?;
StatsCounters::add(&self.stats.packets_sent, 1);
StatsCounters::add(&self.stats.bytes_sent, wire.len() as u64);
self.transport.send(&wire)?;
if !self.transport.send(&wire)? {
StatsCounters::add(&self.stats.packets_send_dropped, 1);
}
}
Ok(())
}
@@ -192,7 +194,9 @@ impl Session {
let wire = self.seal_for_wire(&pkt)?;
StatsCounters::add(&self.stats.packets_sent, 1);
StatsCounters::add(&self.stats.bytes_sent, wire.len() as u64);
self.transport.send(&wire)?;
if !self.transport.send(&wire)? {
StatsCounters::add(&self.stats.packets_send_dropped, 1);
}
Ok(())
}
}
+7
View File
@@ -11,6 +11,11 @@ pub struct Stats {
pub packets_sent: u64,
pub packets_received: u64,
pub packets_dropped: u64,
/// Packets the host could NOT hand to the kernel because the send buffer was full (WouldBlock)
/// — the dominant loss mode at very high bitrate. Distinct from `packets_dropped` (recv-side
/// reassembler rejects). A non-zero, growing value means the link/encoder is outrunning the
/// send path; raise `net.core.wmem_max` / lower the bitrate, or wait for paced batched sending.
pub packets_send_dropped: u64,
pub fec_recovered_shards: u64,
pub bytes_sent: u64,
pub bytes_received: u64,
@@ -27,6 +32,7 @@ pub struct StatsCounters {
pub packets_sent: AtomicU64,
pub packets_received: AtomicU64,
pub packets_dropped: AtomicU64,
pub packets_send_dropped: AtomicU64,
pub fec_recovered_shards: AtomicU64,
pub bytes_sent: AtomicU64,
pub bytes_received: AtomicU64,
@@ -47,6 +53,7 @@ impl StatsCounters {
packets_sent: self.packets_sent.load(l),
packets_received: self.packets_received.load(l),
packets_dropped: self.packets_dropped.load(l),
packets_send_dropped: self.packets_send_dropped.load(l),
fec_recovered_shards: self.fec_recovered_shards.load(l),
bytes_sent: self.bytes_sent.load(l),
bytes_received: self.bytes_received.load(l),
@@ -57,15 +57,18 @@ pub fn loopback_pair(
}
impl Transport for LoopbackTransport {
fn send(&self, packet: &[u8]) -> std::io::Result<()> {
fn send(&self, packet: &[u8]) -> std::io::Result<bool> {
let n = self.tx.sent.fetch_add(1, Ordering::Relaxed);
if self.tx.drop_period != 0 && (n % self.tx.drop_period as u64) == 0 {
// Deterministically drop in flight (the 1st of each `drop_period` group).
// Deterministically drop in flight (the 1st of each `drop_period` group). This models
// NETWORK loss (the packet left the sender, then vanished), not a local send-buffer
// drop — so it still reports `Ok(true)`: the host sent it; the recv/FEC side handles
// the loss. (`Ok(false)` is reserved for a real WouldBlock send-buffer overflow.)
self.tx.dropped.fetch_add(1, Ordering::Relaxed);
return Ok(());
return Ok(true);
}
self.tx.queue.lock().unwrap().push_back(packet.to_vec());
Ok(())
Ok(true)
}
fn recv(&self) -> std::io::Result<Option<Vec<u8>>> {
+5 -1
View File
@@ -10,6 +10,10 @@ pub use udp::UdpTransport;
/// A datagram transport. `recv` is non-blocking: it returns `Ok(None)` when no packet
/// is currently available, so the caller (decode/present thread) never blocks here.
pub trait Transport: Send + Sync {
fn send(&self, packet: &[u8]) -> std::io::Result<()>;
/// Send one packet. `Ok(true)` = handed to the kernel; `Ok(false)` = dropped locally because
/// the send buffer was momentarily full (WouldBlock) — a non-fatal loss the FEC/keyframe path
/// recovers, surfaced so the caller can count it (`packets_send_dropped`) instead of it being
/// invisible. `Err` = a real send failure.
fn send(&self, packet: &[u8]) -> std::io::Result<bool>;
fn recv(&self) -> std::io::Result<Option<Vec<u8>>>;
}
+15 -8
View File
@@ -25,7 +25,12 @@ impl UdpTransport {
/// packets) or drops on the client recv — and with infinite-GOP a single lost frame
/// freezes the decode until the next RFI refresh. Requested large; the OS clamps to
/// `net.core.{wmem,rmem}_max` (Linux) / `kern.ipc.maxsockbuf` (macOS).
const TARGET_SOCKBUF: usize = 8 * 1024 * 1024;
///
/// Sized for 1 Gbps+: at ~1.2 Gbps on the wire an 8 MB buffer is only ~49 ms of steady state,
/// and a single multi-MB IDR keyframe (~4 MB ≈ 3300 packets) instantly fills most of it. 32 MB
/// gives ~200 ms of headroom and absorbs a keyframe burst without EAGAIN drops. (Paced sending
/// will reduce the buffer actually needed once it lands — see the 1 Gbps roadmap work.)
const TARGET_SOCKBUF: usize = 32 * 1024 * 1024;
/// Bind `local` and `connect` to `peer`, so `send`/`recv` need no address and the
/// kernel filters to this peer. Non-blocking, matching the [`Transport`] contract.
@@ -60,16 +65,18 @@ impl UdpTransport {
}
impl Transport for UdpTransport {
fn send(&self, packet: &[u8]) -> std::io::Result<()> {
fn send(&self, packet: &[u8]) -> std::io::Result<bool> {
match self.socket.send(packet) {
Ok(_) => Ok(()),
Ok(_) => Ok(true),
// The kernel UDP send buffer is momentarily full (a frame burst saturated the
// tx queue — common right after attaching to an already-running source that
// emits at full rate). Drop this packet rather than fail the whole stream: the
// data plane is lossy + FEC-protected and the next frame/RFI keyframe recovers,
// whereas blocking would queue stale frames and add latency, and erroring tears
// the session down. Mirrors the `recv` WouldBlock handling above.
Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => Ok(()),
// emits at full rate, and the dominant failure mode at 1 Gbps+). Drop this packet
// rather than fail the whole stream: the data plane is lossy + FEC-protected and the
// next frame/RFI keyframe recovers, whereas blocking would queue stale frames and add
// latency, and erroring tears the session down. `Ok(false)` surfaces the drop so the
// session counts it (`packets_send_dropped`) instead of it being invisible. Mirrors
// the `recv` WouldBlock handling above.
Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => Ok(false),
Err(e) => Err(e),
}
}