feat(host): in-place encoder rate reconfigure — ABR steps no longer cost an IDR
ci / web (push) Successful in 56s
ci / docs-site (push) Successful in 1m8s
ci / bench (push) Successful in 5m25s
decky / build-publish (push) Successful in 21s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
arch / build-publish (push) Successful in 11m11s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 8s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 41s
android / android (push) Successful in 14m53s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 6m24s
deb / build-publish (push) Successful in 11m50s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Successful in 13m5s
ci / rust (push) Successful in 23m4s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Successful in 16m54s
docker / deploy-docs (push) Successful in 22s
apple / swift (push) Successful in 4m31s
windows-host / package (push) Successful in 14m46s
apple / screenshots (push) Successful in 18m26s

Every adaptive-bitrate step used to tear the encoder down and rebuild it,
opening on a full IDR (a 20-40x frame-size spike, in-flight AU forfeit and
an IDR-cooldown anchor) — exactly when the Automatic controller is climbing.
Encoder::reconfigure_bitrate(bps) retargets the LIVE encoder instead
(default false, so libavcodec/software paths keep the rebuild fallback,
which also still owns the bitrate clamping):

- Linux + Windows direct NVENC: nvEncReconfigureEncoder (added to the
  hand-rolled runtime EncodeApi tables) with resetEncoder=0 / forceIDR=0;
  the same init/config is re-authored via the new shared build_config/
  build_init_params with only avg/max bitrate + VBV (PUNKTFUNK_VBV_FRAMES)
  moved. On-hardware test: 20→60→10 Mbps in place, zero IDRs (RTX 5070 Ti).
- Native AMF: TargetBitrate/PeakBitrate/VBVBufferSize are dynamic
  properties — SetProperty on the live component, no Terminate/re-Init.
- Vulkan Video (HEVC + AV1): stage the rate and emit an
  ENCODE_RATE_CONTROL control command on the next recorded frame (begin
  keeps declaring the session's current state, as the spec requires).

The session glue tries the in-place retarget first and skips the rebuild/
inflight-clear/IDR-cooldown bookkeeping when it succeeds — the reference
chain and the wire-index prediction survive, so RFI keeps working across
rate steps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-14 19:53:18 +02:00
parent 46b7ffc001
commit a1af916e38
6 changed files with 478 additions and 93 deletions
+11
View File
@@ -301,6 +301,17 @@ pub trait Encoder: Send {
fn reset(&mut self) -> bool {
false
}
/// Retarget the encoder's rate control to `bps` (average == max, CBR) **in place** — same
/// codec/resolution/fps, only the bitrate and its derived VBV move. Returns `true` when the
/// live encoder accepted the change: the reference chain, the in-flight frames and the
/// caller's wire-index prediction all survive, so an adaptive-bitrate step costs *nothing* on
/// the wire (no IDR, no in-flight forfeit — the whole point vs. a rebuild). `false` = the
/// backend can't (or the driver rejected the new rate, e.g. above the codec-level ceiling) —
/// the caller falls back to its full rebuild path, which also owns the bitrate clamping.
/// Default: no in-place retarget (the libavcodec/software paths).
fn reconfigure_bitrate(&mut self, _bps: u64) -> bool {
false
}
/// Signal end-of-stream. After this, drain the remaining AUs with [`poll`](Self::poll)
/// until it returns `None` — NVENC buffers frames internally even at `delay=0`.
fn flush(&mut self) -> Result<()>;
@@ -61,6 +61,8 @@ struct EncodeApi {
) -> nv::NVENCSTATUS,
initialize_encoder:
unsafe extern "C" fn(*mut c_void, *mut nv::NV_ENC_INITIALIZE_PARAMS) -> nv::NVENCSTATUS,
reconfigure_encoder:
unsafe extern "C" fn(*mut c_void, *mut nv::NV_ENC_RECONFIGURE_PARAMS) -> nv::NVENCSTATUS,
destroy_encoder: unsafe extern "C" fn(*mut c_void) -> nv::NVENCSTATUS,
get_encode_caps: unsafe extern "C" fn(
*mut c_void,
@@ -187,6 +189,7 @@ fn load_api() -> std::result::Result<EncodeApi, String> {
let api = EncodeApi {
open_encode_session_ex: list.nvEncOpenEncodeSessionEx.ok_or(MISSING)?,
initialize_encoder: list.nvEncInitializeEncoder.ok_or(MISSING)?,
reconfigure_encoder: list.nvEncReconfigureEncoder.ok_or(MISSING)?,
destroy_encoder: list.nvEncDestroyEncoder.ok_or(MISSING)?,
get_encode_caps: list.nvEncGetEncodeCaps.ok_or(MISSING)?,
get_encode_preset_config_ex: list.nvEncGetEncodePresetConfigEx.ok_or(MISSING)?,
@@ -294,6 +297,10 @@ pub struct NvencCudaEncoder {
/// GPU capabilities probed once via `nvEncGetEncodeCaps` before configuring.
rfi_supported: bool,
custom_vbv: bool,
/// The split-encode mode the live session was initialized with — `reconfigure_bitrate` must
/// present the SAME init params as the open (only the config's rate fields may move).
/// Meaningless while `inited` is false.
split_mode: u32,
/// The last reference-frame range we invalidated — dedupes repeated RFI requests for one loss.
last_rfi_range: Option<(i64, i64)>,
}
@@ -361,6 +368,7 @@ impl NvencCudaEncoder {
inited: false,
rfi_supported: false,
custom_vbv: false,
split_mode: nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32,
last_rfi_range: None,
})
}
@@ -465,21 +473,11 @@ impl NvencCudaEncoder {
Ok(())
}
/// Open + configure + initialize ONE NVENC CUDA session at `bitrate` (bps) and `split_mode`.
/// Returns the session handle, or destroys it and returns the error.
unsafe fn try_open_session(&self, bitrate: u64, split_mode: u32) -> Result<*mut c_void> {
let mut params = nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS {
version: nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER,
deviceType: nv::NV_ENC_DEVICE_TYPE::NV_ENC_DEVICE_TYPE_CUDA,
device: self.cu_ctx,
apiVersion: nv::NVENCAPI_VERSION,
..Default::default()
};
let mut enc: *mut c_void = ptr::null_mut();
(api().open_encode_session_ex)(&mut params, &mut enc)
.nv_ok()
.map_err(|e| anyhow!("NVENC open_encode_session_ex: {e:?} (no NVIDIA GPU?)"))?;
/// Author the session's `NV_ENC_CONFIG` at `bitrate` (bps): the P1/ULL preset (queried on
/// `enc`) seeded with the RC/tier/chroma/VUI/DPB shape this backend always runs. ONE builder
/// shared by [`try_open_session`] and [`Encoder::reconfigure_bitrate`], so an in-place rate
/// retarget re-authors the exact same config with only the bitrate + derived VBV moved.
unsafe fn build_config(&self, enc: *mut c_void, bitrate: u64) -> Result<nv::NV_ENC_CONFIG> {
// Seed the P1 + ultra-low-latency preset config.
let mut preset = nv::NV_ENC_PRESET_CONFIG {
version: nv::NV_ENC_PRESET_CONFIG_VER,
@@ -489,7 +487,7 @@ impl NvencCudaEncoder {
},
..Default::default()
};
if let Err(e) = (api().get_encode_preset_config_ex)(
(api().get_encode_preset_config_ex)(
enc,
self.codec_guid,
nv::NV_ENC_PRESET_P1_GUID,
@@ -497,10 +495,7 @@ impl NvencCudaEncoder {
&mut preset,
)
.nv_ok()
{
let _ = (api().destroy_encoder)(enc);
return Err(anyhow!("get_encode_preset_config_ex: {e:?}"));
}
.map_err(|e| anyhow!("get_encode_preset_config_ex: {e:?}"))?;
let mut cfg = preset.presetCfg;
// CBR, infinite GOP, P-only, ~1-frame VBV (mirror the Windows/Linux-libav RC config).
@@ -623,7 +618,18 @@ impl NvencCudaEncoder {
}
}
}
Ok(cfg)
}
/// Author the `NV_ENC_INITIALIZE_PARAMS` pointing at `cfg`. Shared by [`try_open_session`]
/// and [`Encoder::reconfigure_bitrate`] — a reconfigure must present the SAME init params as
/// the open. The returned struct borrows `cfg` raw; the caller keeps `cfg` alive across the
/// NVENC call it feeds this into.
fn build_init_params(
&self,
cfg: &mut nv::NV_ENC_CONFIG,
split_mode: u32,
) -> nv::NV_ENC_INITIALIZE_PARAMS {
let mut init = nv::NV_ENC_INITIALIZE_PARAMS {
version: nv::NV_ENC_INITIALIZE_PARAMS_VER,
encodeGUID: self.codec_guid,
@@ -636,10 +642,36 @@ impl NvencCudaEncoder {
frameRateNum: self.fps,
frameRateDen: 1,
enablePTD: 1,
encodeConfig: &mut cfg,
encodeConfig: cfg,
..Default::default()
};
init.set_splitEncodeMode(split_mode);
init
}
/// Open + configure + initialize ONE NVENC CUDA session at `bitrate` (bps) and `split_mode`.
/// Returns the session handle, or destroys it and returns the error.
unsafe fn try_open_session(&self, bitrate: u64, split_mode: u32) -> Result<*mut c_void> {
let mut params = nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS {
version: nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER,
deviceType: nv::NV_ENC_DEVICE_TYPE::NV_ENC_DEVICE_TYPE_CUDA,
device: self.cu_ctx,
apiVersion: nv::NVENCAPI_VERSION,
..Default::default()
};
let mut enc: *mut c_void = ptr::null_mut();
(api().open_encode_session_ex)(&mut params, &mut enc)
.nv_ok()
.map_err(|e| anyhow!("NVENC open_encode_session_ex: {e:?} (no NVIDIA GPU?)"))?;
let mut cfg = match self.build_config(enc, bitrate) {
Ok(cfg) => cfg,
Err(e) => {
let _ = (api().destroy_encoder)(enc);
return Err(e);
}
};
let mut init = self.build_init_params(&mut cfg, split_mode);
match (api().initialize_encoder)(enc, &mut init).nv_ok() {
Ok(()) => Ok(enc),
@@ -752,6 +784,10 @@ impl NvencCudaEncoder {
}
};
self.encoder = enc;
// (Best effort: the floor fallback above may have succeeded split-disabled without
// updating `split_mode` — a later reconfigure then presents the forced mode, NVENC
// rejects it, and the caller's rebuild fallback covers the mismatch.)
self.split_mode = split_mode;
// Output bitstream pool.
for _ in 0..POOL {
@@ -1116,6 +1152,50 @@ impl Encoder for NvencCudaEncoder {
true
}
fn reconfigure_bitrate(&mut self, bps: u64) -> bool {
if !self.inited {
// No live session yet — the lazy init simply opens at the new rate.
self.bitrate_bps = bps;
return true;
}
// SAFETY: `inited` ⟹ `self.encoder` is the live session and every call here runs on the
// encode thread with no NVENC call in flight (the session loop calls this between
// submit/poll). `build_config` only queries the preset on that session; `cfg` outlives
// the synchronous reconfigure call whose `reInitEncodeParams.encodeConfig` points at it.
unsafe {
let mut cfg = match self.build_config(self.encoder, bps) {
Ok(cfg) => cfg,
Err(e) => {
tracing::warn!(error = %format!("{e:#}"),
"NVENC reconfigure: config re-author failed — falling back to a rebuild");
return false;
}
};
let mut params = nv::NV_ENC_RECONFIGURE_PARAMS {
version: nv::NV_ENC_RECONFIGURE_PARAMS_VER,
reInitEncodeParams: self.build_init_params(&mut cfg, self.split_mode),
..Default::default()
};
// Keep the encoder's RC state and reference chain: no reset, no IDR — the in-flight
// frames and the caller's wire-index prediction survive the retarget.
params.set_resetEncoder(0);
params.set_forceIDR(0);
match (api().reconfigure_encoder)(self.encoder, &mut params).nv_ok() {
Ok(()) => {
self.bitrate_bps = bps;
true
}
Err(e) => {
// E.g. the new rate is above the codec-level ceiling — the caller's rebuild
// fallback owns the clamp search.
tracing::warn!(status = ?e, mbps = bps / 1_000_000,
"nvEncReconfigureEncoder rejected — falling back to a rebuild");
false
}
}
}
}
fn flush(&mut self) -> Result<()> {
Ok(()) // P1/ULL + frameIntervalP=1: each submit yields its AU; no internal queue to drain.
}
@@ -1271,6 +1351,68 @@ mod tests {
println!("nvenc_cuda 4:4:4 smoke: {aus} AUs, caps.chroma_444=true");
}
/// ON-HARDWARE (RTX box `.21`): the Phase 3.2 in-place rate retarget — encode a few frames,
/// `reconfigure_bitrate` mid-stream (up AND down), keep encoding, and assert every
/// post-reconfigure AU is a P-frame: `nvEncReconfigureEncoder` with `resetEncoder=0` /
/// `forceIDR=0` must NOT restart the stream (the whole point vs. the rebuild path). Run:
/// cargo test -p punktfunk-host --features nvenc -- --ignored nvenc_cuda_reconfigure --nocapture
#[test]
#[ignore = "requires an NVIDIA GPU + driver — run manually on the RTX box (.21)"]
fn nvenc_cuda_reconfigure_no_idr() {
const W: u32 = 1280;
const H: u32 = 720;
crate::zerocopy::cuda::make_current().expect("shared CUDA context current");
let mut enc = NvencCudaEncoder::open(
Codec::H265,
PixelFormat::Nv12,
W,
H,
60,
20_000_000,
true,
8,
ChromaFormat::Yuv420,
)
.expect("open NVENC CUDA session");
let submit_and_poll = |enc: &mut NvencCudaEncoder, range: std::ops::Range<u32>| {
let mut keyframes = 0usize;
let mut aus = 0usize;
for i in range {
let frame = nv12_frame(W, H, i);
enc.submit_indexed(&frame, i).expect("submit");
while let Some(au) = enc.poll().expect("poll") {
aus += 1;
keyframes += au.keyframe as usize;
}
}
(aus, keyframes)
};
let (aus, kfs) = submit_and_poll(&mut enc, 0..4);
assert!(aus > 0, "no AUs before the reconfigure");
assert_eq!(kfs, 1, "exactly the opening IDR before the reconfigure");
assert!(
enc.reconfigure_bitrate(60_000_000),
"in-place reconfigure to 60 Mbps must succeed on RTX NVENC"
);
let (aus, kfs) = submit_and_poll(&mut enc, 4..8);
assert!(aus > 0, "no AUs after the up-reconfigure");
assert_eq!(kfs, 0, "an in-place rate retarget must not emit an IDR");
assert!(
enc.reconfigure_bitrate(10_000_000),
"in-place reconfigure down to 10 Mbps must succeed"
);
let (aus, kfs) = submit_and_poll(&mut enc, 8..12);
assert!(aus > 0, "no AUs after the down-reconfigure");
assert_eq!(kfs, 0, "an in-place rate retarget must not emit an IDR");
enc.flush().ok();
println!("nvenc_cuda reconfigure smoke: 20→60→10 Mbps in place, zero IDRs");
}
/// A pre-session RFI request and nonsense ranges all correctly decline (→ caller forces IDR).
/// Needs no GPU session (it short-circuits on the null encoder / range checks), so it runs in the
/// normal suite — but `open` gates on the NVENC `.so`, so it skips gracefully where the NVIDIA
@@ -192,6 +192,12 @@ pub struct VulkanVideoEncoder {
// --- rate control (CBR), rebuilt-safe ---
bitrate: u64,
fps: u32,
/// A [`reconfigure_bitrate`](Encoder::reconfigure_bitrate) rate not yet installed in the video
/// session. The next `record_submit` emits an `ENCODE_RATE_CONTROL` control command carrying it
/// (mid-stream) or folds it into the first frame's RESET+RC install, then promotes it into
/// `bitrate` — which must keep naming the session's CURRENT state, because every begin-coding
/// declares it (the spec requires the declared state to match).
pending_bitrate: Option<u64>,
// --- state ---
width: u32,
@@ -654,6 +660,7 @@ impl VulkanVideoEncoder {
compute_pool,
bitrate,
fps,
pending_bitrate: None,
width: w,
height: h,
render_w: rw,
@@ -901,6 +908,15 @@ impl VulkanVideoEncoder {
let nv12_view = self.frames[slot].nv12_view;
// ---- 1. decide frame type + reference (RFI) ----
// Mid-stream rate retarget (`reconfigure_bitrate`): a first frame installs its RC state
// fresh (RESET + ENCODE_RATE_CONTROL in the record fns), so a pending rate folds straight
// into it; mid-stream it stays pending — the record fns emit an ENCODE_RATE_CONTROL
// control command against the declared current state, and step 5 promotes it.
if self.first_frame {
if let Some(nb) = self.pending_bitrate.take() {
self.bitrate = nb;
}
}
let mut is_idr = self.first_frame || self.force_kf;
let mut ref_slot = self.prev_slot;
let mut recovery = false;
@@ -1202,6 +1218,15 @@ impl VulkanVideoEncoder {
self.enc_count += 1;
self.first_frame = false;
self.force_kf = false;
if let Some(nb) = self.pending_bitrate.take() {
// The retarget control command is recorded (execution follows submission order): the
// session's RC state IS the new rate from this frame on — later begins declare it.
self.bitrate = nb;
tracing::info!(
mbps = nb / 1_000_000,
"vulkan-encode: rate control retargeted in place (no IDR)"
);
}
Ok(())
}
@@ -1436,6 +1461,27 @@ impl VulkanVideoEncoder {
);
ctrl.p_next = rc_ptr;
(self.vq_dev.fp().cmd_control_video_coding_khr)(cmd, &ctrl);
} else if let Some(nb) = self.pending_bitrate {
// Mid-stream retarget (`reconfigure_bitrate`): `begin` above declared the session's
// CURRENT rate-control state (the spec requires the match); this control command
// installs the NEW rate — the same CBR shape with only the bitrate moved. No RESET,
// no IDR: the DPB and reference chain carry straight on. `record_submit` promotes
// `nb` into `self.bitrate` after recording, so later begins declare the new state.
let rc_layer2 = [vk::VideoEncodeRateControlLayerInfoKHR::default()
.average_bitrate(nb)
.max_bitrate(nb)
.frame_rate_numerator(self.fps)
.frame_rate_denominator(1)];
let mut rc2 = vk::VideoEncodeRateControlInfoKHR::default()
.rate_control_mode(vk::VideoEncodeRateControlModeFlagsKHR::CBR)
.layers(&rc_layer2)
.virtual_buffer_size_in_ms(1000)
.initial_virtual_buffer_size_in_ms(500);
rc2.p_next = &h265_rc as *const _ as *const c_void;
let mut ctrl = vk::VideoCodingControlInfoKHR::default()
.flags(vk::VideoCodingControlFlagsKHR::ENCODE_RATE_CONTROL);
ctrl.p_next = &rc2 as *const _ as *const c_void;
(self.vq_dev.fp().cmd_control_video_coding_khr)(cmd, &ctrl);
}
dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty());
let src_res = vk::VideoPictureResourceInfoKHR::default()
@@ -1674,6 +1720,25 @@ impl VulkanVideoEncoder {
);
ctrl.p_next = rc_ptr;
(self.vq_dev.fp().cmd_control_video_coding_khr)(cmd, &ctrl);
} else if let Some(nb) = self.pending_bitrate {
// Mid-stream retarget (`reconfigure_bitrate`) — see the HEVC twin for the state
// discipline (begin declares CURRENT, this control installs NEW, `record_submit`
// promotes after recording). No RESET, no IDR.
let rc_layer2 = [vk::VideoEncodeRateControlLayerInfoKHR::default()
.average_bitrate(nb)
.max_bitrate(nb)
.frame_rate_numerator(self.fps)
.frame_rate_denominator(1)];
let mut rc2 = vk::VideoEncodeRateControlInfoKHR::default()
.rate_control_mode(vk::VideoEncodeRateControlModeFlagsKHR::CBR)
.layers(&rc_layer2)
.virtual_buffer_size_in_ms(1000)
.initial_virtual_buffer_size_in_ms(500);
rc2.p_next = &av1_rc as *const _ as *const c_void;
let mut ctrl = vk::VideoCodingControlInfoKHR::default()
.flags(vk::VideoCodingControlFlagsKHR::ENCODE_RATE_CONTROL);
ctrl.p_next = &rc2 as *const _ as *const c_void;
(self.vq_dev.fp().cmd_control_video_coding_khr)(cmd, &ctrl);
}
dev.cmd_begin_query(cmd, query_pool, 0, vk::QueryControlFlags::empty());
let src_res = vk::VideoPictureResourceInfoKHR::default()
@@ -1832,6 +1897,16 @@ impl Encoder for VulkanVideoEncoder {
self.poc = 0;
self.slot_wire.iter_mut().for_each(|s| *s = -1);
self.slot_poc.iter_mut().for_each(|s| *s = -1);
// A pending `reconfigure_bitrate` rate deliberately survives: the restart's first frame
// folds it into the fresh RESET + rate-control install.
true
}
fn reconfigure_bitrate(&mut self, bps: u64) -> bool {
// The RC block is re-declared on every recorded frame, so the retarget is just a staged
// rate: the next `record_submit` emits an ENCODE_RATE_CONTROL control command carrying it
// — no session churn, no IDR. Same floor as `open` (a 0-rate CBR layer is rejected).
self.pending_bitrate = Some(bps.max(1_000_000));
true
}
@@ -1328,6 +1328,14 @@ impl AmfEncoder {
!ltr_disabled() && matches!(self.codec, Codec::H264 | Codec::H265)
}
/// The VBV/HRD buffer (bits) at `bps`: ~1 frame interval, `PUNKTFUNK_VBV_FRAMES`-scaled — the
/// same shape every backend ships. Shared by [`apply_static_props`](Self::apply_static_props)
/// and [`Encoder::reconfigure_bitrate`] so a dynamic retarget rescales the buffer it opened with.
fn vbv_bits(&self, bps: u64) -> i64 {
((bps as f64 / self.fps.max(1) as f64) * crate::encode::vbv_frames_env())
.clamp(1.0, i32::MAX as f64) as i64
}
/// Apply the static encoder configuration (design §3.4 — the native mirror of the ffmpeg
/// opts block in `open_win_encoder`). Called before `Init`, and again on a `reset()`
/// re-`Init` (Terminate does not guarantee property retention across every driver).
@@ -1357,14 +1365,12 @@ impl AmfEncoder {
true,
)?;
// ~1-frame VBV (PUNKTFUNK_VBV_FRAMES override, same knob as the ffmpeg path).
let vbv_frames = std::env::var("PUNKTFUNK_VBV_FRAMES")
.ok()
.and_then(|s| s.parse::<f32>().ok())
.filter(|v| v.is_finite() && *v > 0.0)
.unwrap_or(1.0);
let vbv_bits = ((self.bitrate_bps as f64 / self.fps.max(1) as f64) * vbv_frames as f64)
.clamp(1.0, i32::MAX as f64) as i64;
set_prop(comp, p.vbv_size, AmfVariant::from_i64(vbv_bits), false)?;
set_prop(
comp,
p.vbv_size,
AmfVariant::from_i64(self.vbv_bits(self.bitrate_bps)),
false,
)?;
set_prop(comp, p.enforce_hrd, AmfVariant::from_bool(true), false)?;
set_prop(comp, p.filler_data, AmfVariant::from_bool(false), false)?;
// Latency-first quality; low-latency submission mode (optional — newer VCN/drivers).
@@ -2499,6 +2505,47 @@ impl Encoder for AmfEncoder {
true
}
fn reconfigure_bitrate(&mut self, bps: u64) -> bool {
let bps_i = bps.min(i64::MAX as u64) as i64;
let vbv = self.vbv_bits(bps);
let Some(inner) = self.inner.as_ref() else {
// Nothing live yet — the lazy open applies the new rate via `apply_static_props`.
self.bitrate_bps = bps;
return true;
};
// `TargetBitrate`/`PeakBitrate`/`VBVBufferSize` are DYNAMIC AMF properties (runtime-
// changeable on AVC/HEVC/AV1 alike): a SetProperty on the live component retargets the
// rate controller with no Terminate/re-Init — the reference chain, LTR slots and
// in-flight frames all survive (no IDR).
// SAFETY: `inner.comp.0` is the live component, used only on this thread with no AMF
// call in flight (the session loop is synchronous); `set_prop` is a prefix-vtable call.
let applied = unsafe {
let p = &self.props;
let comp = inner.comp.0;
let ok = set_prop(comp, p.target_bitrate, AmfVariant::from_i64(bps_i), false)
.unwrap_or(false)
&& set_prop(comp, p.peak_bitrate, AmfVariant::from_i64(bps_i), false)
.unwrap_or(false);
if ok {
// Rescale the VBV with the rate. Optional, like at open — a driver that declines
// keeps the old buffer (a size mismatch the HRD absorbs), not worth a rebuild.
let _ = set_prop(comp, p.vbv_size, AmfVariant::from_i64(vbv), false);
}
ok
};
if !applied {
// A half-applied pair doesn't matter: the caller's rebuild fallback re-authors
// everything from scratch.
tracing::warn!(
mbps = bps / 1_000_000,
"AMF declined the dynamic bitrate retarget — falling back to a rebuild"
);
return false;
}
self.bitrate_bps = bps; // future reset()/re-Init paths re-apply the new rate
true
}
fn flush(&mut self) -> Result<()> {
let Some(inner) = self.inner.as_mut() else {
return Ok(());
+118 -28
View File
@@ -74,6 +74,8 @@ struct EncodeApi {
) -> nv::NVENCSTATUS,
initialize_encoder:
unsafe extern "C" fn(*mut c_void, *mut nv::NV_ENC_INITIALIZE_PARAMS) -> nv::NVENCSTATUS,
reconfigure_encoder:
unsafe extern "C" fn(*mut c_void, *mut nv::NV_ENC_RECONFIGURE_PARAMS) -> nv::NVENCSTATUS,
destroy_encoder: unsafe extern "C" fn(*mut c_void) -> nv::NVENCSTATUS,
get_encode_caps: unsafe extern "C" fn(
*mut c_void,
@@ -207,6 +209,7 @@ fn load_api() -> std::result::Result<EncodeApi, String> {
Ok(EncodeApi {
open_encode_session_ex: list.nvEncOpenEncodeSessionEx.ok_or(MISSING)?,
initialize_encoder: list.nvEncInitializeEncoder.ok_or(MISSING)?,
reconfigure_encoder: list.nvEncReconfigureEncoder.ok_or(MISSING)?,
destroy_encoder: list.nvEncDestroyEncoder.ok_or(MISSING)?,
get_encode_caps: list.nvEncGetEncodeCaps.ok_or(MISSING)?,
get_encode_preset_config_ex: list.nvEncGetEncodePresetConfigEx.ok_or(MISSING)?,
@@ -454,6 +457,11 @@ pub struct NvencD3d11Encoder {
/// of failing later as an opaque `InvalidParam`. Set by [`query_caps`](Self::query_caps).
rfi_supported: bool,
custom_vbv: bool,
/// The split-encode mode + async-retrieve flag the live session was initialized with —
/// `reconfigure_bitrate` must present the SAME init params as the open (only the config's
/// rate fields may move). Meaningless while `inited` is false.
split_mode: u32,
session_async: bool,
/// The last reference-frame range we invalidated — dedupes repeated RFI requests for the same
/// loss event (the client resends until it sees recovery).
last_rfi_range: Option<(i64, i64)>,
@@ -526,6 +534,8 @@ impl NvencD3d11Encoder {
inited: false,
rfi_supported: false,
custom_vbv: false,
split_mode: nv::NV_ENC_SPLIT_ENCODE_MODE::NV_ENC_SPLIT_DISABLE_MODE as u32,
session_async: false,
last_rfi_range: None,
init_device: ptr::null_mut(),
session_units: 0,
@@ -679,28 +689,11 @@ impl NvencD3d11Encoder {
Ok(())
}
/// Open + configure + initialize ONE NVENC session at `bitrate` (bps) and `split_mode`. Returns
/// the session handle, or destroys it and returns the error. NVENC has no re-init after a failed
/// `initialize_encoder`, so the bitrate-clamp search in `init_session` calls this once per probe.
unsafe fn try_open_session(
&self,
device: &ID3D11Device,
bitrate: u64,
split_mode: u32,
enable_async: bool,
) -> Result<*mut c_void> {
let mut params = nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS {
version: nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER,
deviceType: nv::NV_ENC_DEVICE_TYPE::NV_ENC_DEVICE_TYPE_DIRECTX,
device: device.as_raw(),
apiVersion: nv::NVENCAPI_VERSION,
..Default::default()
};
let mut enc: *mut c_void = ptr::null_mut();
(api().open_encode_session_ex)(&mut params, &mut enc)
.nv_ok()
.map_err(|e| anyhow!("NVENC open_encode_session_ex: {e:?} (no NVIDIA GPU?)"))?;
/// Author the session's `NV_ENC_CONFIG` at `bitrate` (bps): the P1/ULL preset (queried on
/// `enc`) seeded with the RC/tier/chroma/VUI/DPB shape this backend always runs. ONE builder
/// shared by [`try_open_session`] and [`Encoder::reconfigure_bitrate`], so an in-place rate
/// retarget re-authors the exact same config with only the bitrate + derived VBV moved.
unsafe fn build_config(&self, enc: *mut c_void, bitrate: u64) -> Result<nv::NV_ENC_CONFIG> {
// Seed the P1 + ultra-low-latency preset config.
let mut preset = nv::NV_ENC_PRESET_CONFIG {
version: nv::NV_ENC_PRESET_CONFIG_VER,
@@ -710,7 +703,7 @@ impl NvencD3d11Encoder {
},
..Default::default()
};
if let Err(e) = (api().get_encode_preset_config_ex)(
(api().get_encode_preset_config_ex)(
enc,
self.codec_guid,
nv::NV_ENC_PRESET_P1_GUID,
@@ -718,10 +711,7 @@ impl NvencD3d11Encoder {
&mut preset,
)
.nv_ok()
{
let _ = (api().destroy_encoder)(enc);
return Err(anyhow!("get_encode_preset_config_ex: {e:?}"));
}
.map_err(|e| anyhow!("get_encode_preset_config_ex: {e:?}"))?;
let mut cfg = preset.presetCfg;
// Mirror the Linux RC config: CBR, infinite GOP, P-only, ~1-frame VBV.
@@ -897,7 +887,19 @@ impl NvencD3d11Encoder {
}
}
}
Ok(cfg)
}
/// Author the `NV_ENC_INITIALIZE_PARAMS` pointing at `cfg`. Shared by [`try_open_session`]
/// and [`Encoder::reconfigure_bitrate`] — a reconfigure must present the SAME init params as
/// the open. The returned struct borrows `cfg` raw; the caller keeps `cfg` alive across the
/// NVENC call it feeds this into.
fn build_init_params(
&self,
cfg: &mut nv::NV_ENC_CONFIG,
split_mode: u32,
enable_async: bool,
) -> nv::NV_ENC_INITIALIZE_PARAMS {
let mut init = nv::NV_ENC_INITIALIZE_PARAMS {
version: nv::NV_ENC_INITIALIZE_PARAMS_VER,
encodeGUID: self.codec_guid,
@@ -913,11 +915,44 @@ impl NvencD3d11Encoder {
// Two-thread async retrieve (§5.B): completion events signal the retrieve thread
// instead of `lock_bitstream` blocking the submit thread.
enableEncodeAsync: enable_async as u32,
encodeConfig: &mut cfg,
encodeConfig: cfg,
..Default::default()
};
// splitEncodeMode is a C bitfield — set via the generated accessor, not a struct field.
init.set_splitEncodeMode(split_mode);
init
}
/// Open + configure + initialize ONE NVENC session at `bitrate` (bps) and `split_mode`. Returns
/// the session handle, or destroys it and returns the error. NVENC has no re-init after a failed
/// `initialize_encoder`, so the bitrate-clamp search in `init_session` calls this once per probe.
unsafe fn try_open_session(
&self,
device: &ID3D11Device,
bitrate: u64,
split_mode: u32,
enable_async: bool,
) -> Result<*mut c_void> {
let mut params = nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS {
version: nv::NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER,
deviceType: nv::NV_ENC_DEVICE_TYPE::NV_ENC_DEVICE_TYPE_DIRECTX,
device: device.as_raw(),
apiVersion: nv::NVENCAPI_VERSION,
..Default::default()
};
let mut enc: *mut c_void = ptr::null_mut();
(api().open_encode_session_ex)(&mut params, &mut enc)
.nv_ok()
.map_err(|e| anyhow!("NVENC open_encode_session_ex: {e:?} (no NVIDIA GPU?)"))?;
let mut cfg = match self.build_config(enc, bitrate) {
Ok(cfg) => cfg,
Err(e) => {
let _ = (api().destroy_encoder)(enc);
return Err(e);
}
};
let mut init = self.build_init_params(&mut cfg, split_mode, enable_async);
match (api().initialize_encoder)(enc, &mut init).nv_ok() {
Ok(()) => Ok(enc),
@@ -1071,6 +1106,12 @@ impl NvencD3d11Encoder {
}
};
self.encoder = enc;
// Session init params a later `reconfigure_bitrate` must re-present verbatim. (Best
// effort: the floor fallback above may have succeeded split-disabled without updating
// `split_mode` — a reconfigure then presents the forced mode, NVENC rejects it, and
// the caller's rebuild fallback covers the mismatch.)
self.split_mode = split_mode;
self.session_async = use_async;
// Session-budget accounting (Stage W3): record what this open holds so admission can
// decline a parallel display the hardware can't afford. Weighted by the FINAL split
// mode (a split session occupies one hardware session per engine).
@@ -1621,6 +1662,55 @@ impl Encoder for NvencD3d11Encoder {
true
}
fn reconfigure_bitrate(&mut self, bps: u64) -> bool {
if !self.inited {
// No live session yet — the lazy init simply opens at the new rate.
self.bitrate_bps = bps;
return true;
}
// SAFETY: `inited` ⟹ `self.encoder` is the live session and this runs on the encode
// thread between submit/poll (`nvEncReconfigureEncoder` is a submit-side call, the
// sanctioned side of the two-thread async split — the retrieve thread only ever locks
// bitstreams). `build_config` only queries the preset on that session; `cfg` outlives the
// synchronous reconfigure call whose `reInitEncodeParams.encodeConfig` points at it.
unsafe {
let mut cfg = match self.build_config(self.encoder, bps) {
Ok(cfg) => cfg,
Err(e) => {
tracing::warn!(error = %format!("{e:#}"),
"NVENC reconfigure: config re-author failed — falling back to a rebuild");
return false;
}
};
let mut params = nv::NV_ENC_RECONFIGURE_PARAMS {
version: nv::NV_ENC_RECONFIGURE_PARAMS_VER,
reInitEncodeParams: self.build_init_params(
&mut cfg,
self.split_mode,
self.session_async,
),
..Default::default()
};
// Keep the encoder's RC state and reference chain: no reset, no IDR — the in-flight
// frames and the caller's wire-index prediction survive the retarget.
params.set_resetEncoder(0);
params.set_forceIDR(0);
match (api().reconfigure_encoder)(self.encoder, &mut params).nv_ok() {
Ok(()) => {
self.bitrate_bps = bps;
true
}
Err(e) => {
// E.g. the new rate is above the codec-level ceiling — the caller's rebuild
// fallback owns the clamp search.
tracing::warn!(status = ?e, mbps = bps / 1_000_000,
"nvEncReconfigureEncoder rejected — falling back to a rebuild");
false
}
}
}
}
fn flush(&mut self) -> Result<()> {
Ok(()) // P1/ULL + frameIntervalP=1: each submit yields its AU; no internal queue to drain.
}
+56 -36
View File
@@ -4274,47 +4274,67 @@ fn virtual_stream(ctx: SessionContext) -> Result<()> {
}
}
// Adaptive bitrate: drain to the NEWEST requested rate (the client's controller may step
// several times while we stream) and rebuild the ENCODER ONLY in place — the mode didn't
// change, so capture and the virtual output are untouched and the switch costs exactly the
// IDR the fresh encoder opens with (the same resync discipline as a mode switch, minus the
// pipeline churn). Rates arrive pre-clamped by the control task (`resolve_bitrate_kbps`).
// several times while we stream) and retarget the ENCODER ONLY — the mode didn't change,
// so capture and the virtual output are untouched. Preferred lever: an IN-PLACE
// `reconfigure_bitrate` (Phase 3.2 — NVENC nvEncReconfigureEncoder / AMF dynamic props /
// Vulkan RC control), which keeps the encoder, its reference chain and the in-flight AUs,
// so the step costs NOTHING on the wire (no IDR, no forfeit — exactly what the Automatic
// controller's doubling climb wants). A backend that can't (libavcodec paths) or a driver
// rejection falls back to the full rebuild, which costs the IDR the fresh encoder opens
// with (the same resync discipline as a mode switch, minus the pipeline churn) and owns
// the bitrate clamping. Rates arrive pre-clamped by the control task
// (`resolve_bitrate_kbps`).
let mut want_kbps = None;
while let Ok(k) = bitrate_rx.try_recv() {
want_kbps = Some(k);
}
if let Some(new_kbps) = want_kbps.filter(|&k| k != bitrate_kbps) {
// `interval` was built as 1/effective_hz, so the round-trip recovers the integer rate.
let hz = interval_hz(interval);
match crate::encode::open_video(
plan.codec,
frame.format,
frame.width,
frame.height,
hz,
new_kbps as u64 * 1000,
frame.is_cuda(),
bit_depth,
plan.chroma,
) {
Ok(new_enc) => {
tracing::info!(
from_kbps = bitrate_kbps,
to_kbps = new_kbps,
"encoder rebuilt at new bitrate (adaptive bitrate)"
);
enc = new_enc;
bitrate_kbps = new_kbps;
live_bitrate.store(new_kbps, Ordering::Relaxed);
// The owed AUs died with the old encoder — same bookkeeping as a mode-switch
// rebuild; the fresh encoder opens on an IDR, so anchor the IDR cooldown too.
inflight.clear();
last_au_at = std::time::Instant::now();
encoder_resets = 0;
last_forced_idr = Some(std::time::Instant::now());
}
Err(e) => {
tracing::error!(error = %format!("{e:#}"), to_kbps = new_kbps,
"bitrate-change encoder rebuild failed — keeping the current rate");
if enc.reconfigure_bitrate(new_kbps as u64 * 1000) {
tracing::info!(
from_kbps = bitrate_kbps,
to_kbps = new_kbps,
"encoder bitrate reconfigured in place (adaptive bitrate — no IDR)"
);
bitrate_kbps = new_kbps;
live_bitrate.store(new_kbps, Ordering::Relaxed);
// Same encoder, same stream: the in-flight AUs and the wire-index prediction
// stay valid — no inflight forfeit, no IDR-cooldown anchor.
} else {
// `interval` was built as 1/effective_hz, so the round-trip recovers the integer
// rate.
let hz = interval_hz(interval);
match crate::encode::open_video(
plan.codec,
frame.format,
frame.width,
frame.height,
hz,
new_kbps as u64 * 1000,
frame.is_cuda(),
bit_depth,
plan.chroma,
) {
Ok(new_enc) => {
tracing::info!(
from_kbps = bitrate_kbps,
to_kbps = new_kbps,
"encoder rebuilt at new bitrate (adaptive bitrate)"
);
enc = new_enc;
bitrate_kbps = new_kbps;
live_bitrate.store(new_kbps, Ordering::Relaxed);
// The owed AUs died with the old encoder — same bookkeeping as a
// mode-switch rebuild; the fresh encoder opens on an IDR, so anchor the
// IDR cooldown too.
inflight.clear();
last_au_at = std::time::Instant::now();
encoder_resets = 0;
last_forced_idr = Some(std::time::Instant::now());
}
Err(e) => {
tracing::error!(error = %format!("{e:#}"), to_kbps = new_kbps,
"bitrate-change encoder rebuild failed — keeping the current rate");
}
}
}
}