unom/punktfunk
5
0
Fork 0
Code Issues Pull Requests 2 Actions 1 Packages Projects Releases 36 Wiki Activity

fix(encode/nvenc): cache the level ceiling, expose the applied bitrate, force 2-way split at 4K120

Browse Source 
Three encode-time fixes from the 4K120 field analysis (sessions pinned ~107 of
120 fps, 14-15 ms reported encode):

- Ceiling truth: the codec-level bitrate clamp (binary search at session open)
  now records its result in a process-lifetime advisory cache keyed by
  GPU/config, so an ABR overshoot opens — or in-place reconfigures — straight
  AT the ceiling instead of re-running the ~6-open search (and a full rebuild
  + IDR) on every overshoot. New `Encoder::applied_bitrate_bps()` exposes the
  post-clamp rate so the session loop can stop pacing/acking a phantom
  requested rate (consumed host-side in a follow-up).

- Clamp-search hygiene: only NVENC parameter/caps rejections steer the search
  now (`NvCallError` keeps the raw status downcastable); a transient failure
  (busy engine, session limit, OOM, driver skew) propagates instead of
  shrinking into — and now caching — a bogus ceiling. The floor fallback also
  records the split mode it actually opened with, so a later reconfigure
  re-presents the real session params.

- Split-frame selection: one shared `resolve_split_mode` replaces the two
  byte-identical direct-SDK copies; the force-2-way threshold moves to
  `SPLIT_FORCE_PIXEL_RATE` (950 Mpix/s, shared with the libav path) because
  4K120 = 995,328,000 px/s missed the old `> 1e9` gate by 0.47% and stayed on
  AUTO — which never engages at 2160 px height, leaving the second NVENC
  engine idle in exactly the mode the threshold existed for. The Linux
  session-ready info! line now carries the final split mode (journals are
  INFO+; this was undiagnosable from user logs).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
enricobuehler
2026-07-25 01:44:54 +02:00
co-authored by Claude Fable 5
parent 42e5f5ad1e
commit c1d54b835b
10 changed files with 498 additions and 111 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/pf-encode/src/enc/linux/nvenc_cuda.rs
+100 -40
View File
@@ -61,8 +61,9 @@
#![deny(clippy::undocumented_unsafe_blocks)]
use super::nvenc_core::{
apply_low_latency_config, build_init_params, codec_guid, resolve_slices, resolve_subframe,
LowLatencyConfig, NvStatusExt, RFI_DPB,
apply_low_latency_config, build_init_params, cached_ceiling, codec_guid, resolve_slices,
resolve_split_mode, resolve_subframe, store_ceiling, CeilingKey, LowLatencyConfig, NvStatusExt,
RFI_DPB,
};
use super::nvenc_status;
use super::{AuChunk, ChromaFormat, Codec, EncodedFrame, Encoder, EncoderCaps};
@@ -1000,6 +1001,23 @@ impl NvencCudaEncoder {
Ok(cfg)
}
/// This session config's identity in the process-lifetime bitrate-ceiling cache
/// (`nvenc_core::{cached_ceiling, store_ceiling}`). GPU identity is the process-global shared
/// `CUcontext` pointer — one context per process, stable for its lifetime; only valid once
/// `cu_ctx` is bound (`init_session` start), which every caller is downstream of.
fn ceiling_key(&self, split_mode: u32) -> CeilingKey {
CeilingKey {
gpu: self.cu_ctx as u64,
codec: self.codec,
width: self.width,
height: self.height,
fps: self.fps,
bit_depth: self.bit_depth,
chroma_444: self.chroma_444,
split_mode,
}
}
/// Open + configure + initialize ONE NVENC CUDA session at `bitrate` (bps) and `split_mode`.
/// Returns the session handle, or destroys it and returns the error.
unsafe fn try_open_session(&self, bitrate: u64, split_mode: u32) -> Result<*mut c_void> {
@@ -1074,58 +1092,71 @@ impl NvencCudaEncoder {
}
const FLOOR_BPS: u64 = 10_000_000;
let requested_bps = self.bitrate_bps;
// 2-way NVENC split-frame encoding (Ada dual-NVENC) above ~1 Gpix/s; env override
// PUNKTFUNK_SPLIT_ENCODE = 0/disable | 1/auto | 2 | 3. HEVC/AV1 only.
// 2-way NVENC split-frame encoding (Ada dual-NVENC) — shared selector, see
// [`resolve_split_mode`] for the precedence (env override / 10-bit / pixel rate).
let pixel_rate = self.width as u64 * self.height as u64 * self.fps.max(1) as u64;
let mut split_mode: u32 = match std::env::var("PUNKTFUNK_SPLIT_ENCODE").ok().as_deref()
{
Some("0") | Some("disable") => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32
}
Some("1") | Some("auto") => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_AUTO_FORCED_MODE as u32
}
Some("3") => nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_THREE_FORCED_MODE as u32,
Some("2") => nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_TWO_FORCED_MODE as u32,
_ if self.bit_depth >= 10 => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32
}
_ if pixel_rate > 1_000_000_000 => {
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_TWO_FORCED_MODE as u32
}
_ => nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_AUTO_MODE as u32,
};
let split_mode: u32 = resolve_split_mode(self.bit_depth, pixel_rate);
const CLAMP_TOL_BPS: u64 = 20_000_000;
let mut probe = self.try_open_session(requested_bps, split_mode);
// Disambiguate a forced-split rejection from a bitrate-cap rejection.
// Ceiling cache (process lifetime, `nvenc_core`): a prior clamp search already found
// this config's max accepted rate — open straight AT the ceiling instead of paying
// the ~6-open binary search (and its session churn) on every ABR overshoot.
let mut target_bps = requested_bps;
if let Some(ceiling) = cached_ceiling(&self.ceiling_key(split_mode)) {
if requested_bps > ceiling {
tracing::info!(
requested_mbps = requested_bps / 1_000_000,
ceiling_mbps = ceiling / 1_000_000,
"NVENC (Linux): requested bitrate above the cached codec-level ceiling — \
opening at the ceiling"
);
target_bps = ceiling;
}
}
let mut probe = self.try_open_session(target_bps, split_mode);
// The cache is advisory: a stale entry (driver change, identity collision) must not
// wedge the open — retry the requested rate and let the search below rediscover.
if probe.is_err() && target_bps < requested_bps {
target_bps = requested_bps;
probe = self.try_open_session(requested_bps, split_mode);
}
// Disambiguate a forced-split rejection from a bitrate-cap rejection. `used_split`
// tracks the mode sessions ACTUALLY open with from here on — it feeds
// `self.split_mode` (a reconfigure must re-present it) and the ceiling-cache key.
let mut used_split = split_mode;
let split_on =
split_mode != nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32;
if probe.is_err() && split_on {
let no_split = nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32;
if let Ok(e) = self.try_open_session(requested_bps, no_split) {
if let Ok(e) = self.try_open_session(target_bps, no_split) {
tracing::warn!(
"NVENC (Linux): split-encode rejected by codec/config — disabled"
);
split_mode = no_split;
used_split = no_split;
probe = Ok(e);
}
}
let enc = match probe {
Ok(enc) => {
self.bitrate_bps = requested_bps;
self.bitrate_bps = target_bps;
enc
}
// Only a parameter/caps rejection means "the bitrate is above the codec-level
// ceiling". A transient failure (busy engine, session limit, OOM, device loss,
// version skew) must propagate — a search steered by it would discover, and
// cache, a bogus ceiling.
Err(e) if !nvenc_status::is_param_rejection(&e) => return Err(e),
Err(_) => {
// Requested bitrate exceeds the codec-level ceiling — binary-search the max accepted.
let mut lo = FLOOR_BPS;
let mut hi = requested_bps;
let mut hi = target_bps;
let mut best: *mut c_void = ptr::null_mut();
let mut best_bps = 0u64;
while hi > lo + CLAMP_TOL_BPS {
let mid = lo + (hi - lo) / 2;
match self.try_open_session(mid, split_mode) {
match self.try_open_session(mid, used_split) {
Ok(e) => {
if !best.is_null() {
let _ = (api().destroy_encoder)(best);
@@ -1134,18 +1165,30 @@ impl NvencCudaEncoder {
best_bps = mid;
lo = mid;
}
Err(_) => hi = mid,
Err(e) if nvenc_status::is_param_rejection(&e) => hi = mid,
Err(e) => {
// Environmental mid-search failure: don't let it shrink the
// search — release the partial result and propagate.
if !best.is_null() {
let _ = (api().destroy_encoder)(best);
}
return Err(e);
}
}
}
if best.is_null() {
let no_split =
nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32;
best = self
.try_open_session(FLOOR_BPS, split_mode)
.or_else(|_| self.try_open_session(FLOOR_BPS, no_split))
.context(
"NVENC initialize_encoder rejected even at the floor bitrate",
)?;
best = match self.try_open_session(FLOOR_BPS, used_split) {
Ok(e) => e,
Err(_) => {
let e = self.try_open_session(FLOOR_BPS, no_split).context(
"NVENC initialize_encoder rejected even at the floor bitrate",
)?;
used_split = no_split;
e
}
};
best_bps = FLOOR_BPS;
}
tracing::warn!(
@@ -1153,15 +1196,13 @@ impl NvencCudaEncoder {
clamped_mbps = best_bps / 1_000_000,
"NVENC (Linux): requested bitrate above the GPU codec-level ceiling — clamped"
);
store_ceiling(self.ceiling_key(used_split), best_bps);
self.bitrate_bps = best_bps;
best
}
};
self.encoder = enc;
// (Best effort: the floor fallback above may have succeeded split-disabled without
// updating `split_mode` — a later reconfigure then presents the forced mode, NVENC
// rejects it, and the caller's rebuild fallback covers the mismatch.)
self.split_mode = split_mode;
self.split_mode = used_split;
// Output bitstream pool.
for _ in 0..POOL {
@@ -1345,6 +1386,10 @@ impl NvencCudaEncoder {
mbps = self.bitrate_bps / 1_000_000,
codec = ?self.codec_guid,
fmt = ?self.buffer_fmt,
// The FINAL split mode (post any rejection fallback) at INFO — journals run
// INFO+, and "did 4K120 actually split across engines?" was undiagnosable from
// a user log without it (Windows only had a debug! at selection time).
split_mode = self.split_mode,
"NVENC CUDA session ready"
);
Ok(())
@@ -2061,6 +2106,15 @@ impl Encoder for NvencCudaEncoder {
self.bitrate_bps = bps;
return true;
}
// Cached codec-level ceiling: clamp the target BEFORE the driver call, so a known
// overshoot retargets to the ceiling IN PLACE instead of bouncing off the driver into
// the caller's full-rebuild fallback (an IDR plus ~half a second of session churn per
// ABR overshoot on the pre-cache path). The caller reads the clamp back through
// [`Encoder::applied_bitrate_bps`].
let bps = match cached_ceiling(&self.ceiling_key(self.split_mode)) {
Some(ceiling) => bps.min(ceiling),
None => bps,
};
// SAFETY: `inited` ⟹ `self.encoder` is the live session and every call here runs on the
// encode thread with no NVENC call in flight (the session loop calls this between
// submit/poll). `build_config` only queries the preset on that session; `cfg` outlives
@@ -2108,6 +2162,12 @@ impl Encoder for NvencCudaEncoder {
}
}
fn applied_bitrate_bps(&self) -> Option<u64> {
// `bitrate_bps` is the post-clamp truth: the open path's ceiling search and the
// reconfigure path's cache clamp both write what the session ACTUALLY targets.
Some(self.bitrate_bps)
}
fn flush(&mut self) -> Result<()> {
Ok(()) // P1/ULL + frameIntervalP=1: each submit yields its AU; no internal queue to drain.
}