feat(core): scale the receive path to the new multi-Gbps ceiling
- REPLAY_WINDOW 32768 -> 131072: the anti-replay bitmap covered the 120 ms loss window only to ~2 Gbps; the client now delivers ~4.8 Gbps wire, where a late-but-valid Wi-Fi-retried datagram would have been dropped as 'older than the window' — false loss. 16 KiB/session covers ~12 Gbps. - RECV_BATCH 32 -> 128: syscall rate stays ~3.4k/s at 430k pkt/s and each pump iteration drains the kernel buffer deeper (ring 64->256 KB, client sessions only). flush_backlog's iteration cap rescaled to keep its ~190 MB guard equivalent. - PUNKTFUNK_GSO gate is now value-aware: '=0' used to ENABLE GSO on Linux (presence check) while disabling Windows USO. GSO stays OPT-IN, deliberately: A/B'd twice today — it cuts send-thread CPU ~30% but its 16-packet line-rate trains cost delivered throughput on a constrained fabric (2.5GbE-hop pair: peak 2453 -> 1908 Mbps and 0.4% loss at a rate sendmmsg carries clean). Flipping the default belongs with pace-aware chunk spacing (plan Phase 1.2/1.3). docs-site row corrected to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -83,9 +83,11 @@ pub struct PumpPerf {
|
||||
pub packets: u64,
|
||||
}
|
||||
|
||||
/// Datagrams drained per `recvmmsg` syscall on the client (the reused ring's size). At ~125k
|
||||
/// pkt/s this is ~4k syscalls/s instead of 125k; the buffers cost `RECV_BATCH × RECV_BUF` (~64 KB).
|
||||
const RECV_BATCH: usize = 32;
|
||||
/// Datagrams drained per `recvmmsg` syscall on the client (the reused ring's size). 128 keeps
|
||||
/// the syscall rate ≤ ~3.4k/s even at the ~430k pkt/s the post-2026-07-14 receive path delivers
|
||||
/// (~4.8 Gbps wire), and gives the kernel buffer a deeper drain per pump iteration; the buffers
|
||||
/// cost `RECV_BATCH × RECV_BUF` (~256 KB, client sessions only).
|
||||
const RECV_BATCH: usize = 128;
|
||||
|
||||
impl Session {
|
||||
pub fn new(config: Config, transport: Box<dyn Transport>) -> Result<Session> {
|
||||
@@ -482,8 +484,8 @@ impl Session {
|
||||
/// (observed live: a stream stuck 6–7 s behind, socket buffers full end to end). Discarding
|
||||
/// is memcpy-speed (no decrypt/reassembly/allocation), so this empties even a 32 MB buffer in
|
||||
/// milliseconds; the caller then requests a keyframe and the stream resumes live. The iteration
|
||||
/// cap (4096 batches ≈ 128k datagrams ≈ 190 MB) only guards against a line-rate sender
|
||||
/// outpacing the discard loop indefinitely.
|
||||
/// cap (1024 batches ≈ 131k datagrams ≈ 190 MB at the 128-deep ring) only guards against a
|
||||
/// line-rate sender outpacing the discard loop indefinitely.
|
||||
pub fn flush_backlog(&mut self) -> Result<u64> {
|
||||
if self.config.role != Role::Client {
|
||||
return Err(PunktfunkError::InvalidArg(
|
||||
@@ -495,7 +497,7 @@ impl Session {
|
||||
self.recv_count = 0;
|
||||
self.recv_idx = 0;
|
||||
if !self.recv_scratch.is_empty() {
|
||||
for _ in 0..4096 {
|
||||
for _ in 0..1024 {
|
||||
let n = self
|
||||
.transport
|
||||
.recv_batch(&mut self.recv_scratch, &mut self.recv_lens)?;
|
||||
@@ -541,10 +543,12 @@ fn seq_of(wire: &[u8]) -> u64 {
|
||||
/// ([`LOSS_WINDOW_NS`](crate::packet)) at line-rate packet rates — otherwise the replay filter
|
||||
/// silently re-tightens the "late ≠ lost" fix: a Wi-Fi-retry-delayed shard the reassembler would
|
||||
/// still use gets dropped here as "older than the window" first (4096 was only ~33 ms at the
|
||||
/// ~125k pkt/s of a 1 Gbps stream). 32768 covers 120 ms up to ~270k pkt/s (≈2 Gbps+) and is
|
||||
/// effectively unbounded for the sparse input stream, while still bounding how far back a replay
|
||||
/// could hide; the bitmap costs 4 KiB per session.
|
||||
const REPLAY_WINDOW: u64 = 32768;
|
||||
/// ~125k pkt/s of a 1 Gbps stream; 32768 topped out around ~2 Gbps — which the client now
|
||||
/// exceeds: the 2026-07-14 zero-copy + hardware-AES work measured ~4.8 Gbps wire ≈ 430k pkt/s
|
||||
/// delivered). 131072 covers 120 ms up to ~1.09M pkt/s (≈12 Gbps wire) and is effectively
|
||||
/// unbounded for the sparse input stream, while still bounding how far back a replay could
|
||||
/// hide; the bitmap costs 16 KiB per session.
|
||||
const REPLAY_WINDOW: u64 = 131072;
|
||||
const REPLAY_WORDS: usize = (REPLAY_WINDOW / 64) as usize;
|
||||
|
||||
/// Sliding-window anti-replay filter over the AEAD-authenticated wire sequence. The sender counts
|
||||
|
||||
@@ -46,7 +46,8 @@ pub trait Transport: Send + Sync {
|
||||
/// ~1 GSO skb per ≤64 segments instead of one skb per packet. This is the multi-Gbps lever —
|
||||
/// research shows ~2.4× throughput at equal CPU and ~40× fewer syscalls, and that `sendmmsg`
|
||||
/// batching alone is insufficient (it still builds one skb per datagram). The
|
||||
/// [`UdpTransport`](super::UdpTransport) Linux override implements it (opt-in via `PUNKTFUNK_GSO`,
|
||||
/// [`UdpTransport`](super::UdpTransport) Linux override implements it (opt-in via
|
||||
/// `PUNKTFUNK_GSO=1` pending pace-aware chunk spacing — see the `gso` module doc — with
|
||||
/// auto-fallback on any GSO error); the default just delegates to [`send_batch`](Self::send_batch),
|
||||
/// correct for loopback and non-Linux. Same lossy, FEC-protected short-count contract as `send_batch`.
|
||||
fn send_gso(&self, packets: &[&[u8]]) -> std::io::Result<usize> {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
//! Real UDP datagram transport — native sockets, no async runtime.
|
||||
//!
|
||||
//! Send is batched via `sendmmsg` ([`Transport::send_batch`], ≤64/syscall) and recv via `recvmmsg`
|
||||
//! ([`Transport::recv_batch`], ≤32/syscall into a reused ring) on Linux AND Android (which is
|
||||
//! ([`Transport::recv_batch`], ≤128/syscall into a reused ring) on Linux AND Android (which is
|
||||
//! `target_os = "android"`, not `"linux"` — it needs its own bionic binding, see [`android_mmsg`])
|
||||
//! — the 1 Gbps+ syscall lever (~125k → a few-k syscalls/sec at line rate). The host additionally
|
||||
//! paces each frame's send across the frame interval (see `punktfunk1.rs::paced_submit`) so a real
|
||||
@@ -111,8 +111,14 @@ fn mmsghdrs(iovs: &mut [libc::iovec]) -> Vec<mmsghdr> {
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// UDP GSO enable state (process-wide). Opt-in via `PUNKTFUNK_GSO` — it's new unsafe hot-path code,
|
||||
/// and the auto-fallback (latch off on any GSO syscall error) covers kernels/paths without support.
|
||||
/// UDP GSO enable state (process-wide). **Opt-in** (`PUNKTFUNK_GSO=1`) — and deliberately so,
|
||||
/// measured twice on 2026-07-14: GSO cuts send-thread CPU ~30% at 1250 Mbps, but its 16-packet
|
||||
/// line-rate trains cost real delivered throughput on a constrained fabric (the 2.5GbE-hop pair:
|
||||
/// peak 2453 → 1908 Mbps, and 0.4% loss appeared at a rate the sendmmsg path carries clean).
|
||||
/// Flipping the default belongs together with pace-aware chunk scaling (plan Phase 1.2/1.3 in
|
||||
/// `design/throughput-beyond-1gbps.md`), which spaces the super-buffers instead of skipping
|
||||
/// sub-floor sleeps. NOTE the gate is value-aware: `PUNKTFUNK_GSO=0` explicitly disables (it
|
||||
/// used to key on env *presence*, so `=0` ENABLED it here while disabling Windows USO).
|
||||
#[cfg(target_os = "linux")]
|
||||
mod gso {
|
||||
use std::sync::atomic::{AtomicU8, Ordering};
|
||||
@@ -123,7 +129,8 @@ mod gso {
|
||||
1 => true,
|
||||
2 => false,
|
||||
_ => {
|
||||
let on = std::env::var_os("PUNKTFUNK_GSO").is_some();
|
||||
// Opt-in: on only when PUNKTFUNK_GSO is set to something other than "0".
|
||||
let on = std::env::var("PUNKTFUNK_GSO").is_ok_and(|v| v != "0");
|
||||
STATE.store(if on { 1 } else { 2 }, Ordering::Relaxed);
|
||||
on
|
||||
}
|
||||
|
||||
@@ -138,7 +138,7 @@ notes for context.
|
||||
|
||||
| Setting | Values | Meaning |
|
||||
|---|---|---|
|
||||
| `PUNKTFUNK_GSO` | `1` · `0` | UDP Generic Segmentation Offload on the send path (coalesce a frame's packets into kernel super-buffers) — the dominant lever above ~1 Gbps. On by default; auto-falls back to `sendmmsg`. Set `0` if a NIC/middlebox mishandles GSO. |
|
||||
| `PUNKTFUNK_GSO` | `1` · `0` | UDP Generic Segmentation Offload on the send path (coalesce a frame's packets into kernel super-buffers) — cuts send CPU ~30%, but its line-rate packet trains can cost delivered throughput on constrained links (measured on a 2.5GbE hop). Off by default until send pacing spaces the super-buffers; set `1` to opt in (auto-falls back to `sendmmsg` on kernels/paths without support). |
|
||||
| `PUNKTFUNK_SPLIT_ENCODE` | `0`/`disable` · `1`/`auto` · `2` · `3` | NVENC N-way split-encode for very high pixel rates (5K@240). `auto` picks automatically above ~1 Gpix/s. |
|
||||
| `PUNKTFUNK_GPU_PRIORITY_CLASS` | `off` · `normal` · `high` · `realtime` | **(Windows)** GPU scheduling priority for capture/encode under a GPU-saturating game. Default `high`; `realtime` is the strongest lever but can freeze NVENC on some setups. |
|
||||
| `PUNKTFUNK_IDD_DEPTH` | `N` (default `2`) | **(Windows)** IDD-push pipeline depth. `1` cuts latency once GPU priority is raised; higher smooths a contended GPU. |
|
||||
|
||||
Reference in New Issue
Block a user