fix(nvenc): stop telling operators to reboot when the driver version is fine
`NV_ENC_ERR_INVALID_VERSION` is how the driver reports two opposite failures, and we only ever explained one of them. The message told the operator to update the NVIDIA driver or reboot — correct for a genuine header/kernel-module skew, and actively misleading for the field case behind it: a host that streams once per boot and then fails every later session at the caps probe, until the *process* restarts. A version skew is static; it cannot come and go inside one process, so that advice cost the reporter a reboot per stream. Split the two on the only fact that tells them apart: whether a session has already opened in this process. `nvenc_status` gains a `SESSION_OPENED` latch set right after every successful `open_encode_session_ex` — both backends' caps probe and real open, plus the Windows availability probe. No session yet, the version word really is in question and the existing skew advice stands. A session already opened, and the kernel module demonstrably accepted this build's version word, so the message now names per-process driver state and points at the cheap fix (restart the host service, no reboot) plus a request for the log. The load-time gate cannot serve as this discriminator: `NvEncodeAPIGetMaxSupportedVersion` is a pure userspace query, so the classic "updated the driver, didn't reboot" skew sails through it and only fails later at the open. Only a session that actually opened proves the kernel module agreed. The split lives in a pure `invalid_version(bool)` so both halves are unit-tested without touching the process-wide latch. This is a diagnosis change only — it does not fix the underlying field bug, whose root cause is still open. Verified on Linux (192.168.1.25): clippy `-p pf-encode --features nvenc` clean, `cargo test -p pf-encode --features nvenc --lib` 15 passed, rustfmt clean.
This commit is contained in:
@@ -870,6 +870,9 @@ impl NvencCudaEncoder {
|
||||
e,
|
||||
));
|
||||
}
|
||||
// The handshake with the kernel module just succeeded — from here on, an
|
||||
// `NV_ENC_ERR_INVALID_VERSION` in this process cannot be a driver version skew.
|
||||
nvenc_status::note_session_opened();
|
||||
let wmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_WIDTH_MAX);
|
||||
let hmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_HEIGHT_MAX);
|
||||
let yuv444 = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_YUV444_ENCODE);
|
||||
@@ -1067,6 +1070,7 @@ impl NvencCudaEncoder {
|
||||
}
|
||||
return Err(nvenc_status::call_err("open_encode_session_ex", e));
|
||||
}
|
||||
nvenc_status::note_session_opened();
|
||||
|
||||
let mut cfg = match self.build_config(enc, bitrate) {
|
||||
Ok(cfg) => cfg,
|
||||
|
||||
@@ -8,27 +8,77 @@
|
||||
//! means and what the operator should do, and folds that cause into the `anyhow::Error` at
|
||||
//! construction, so every downstream `{e:#}` log (the encode-recovery loop, session teardown) says
|
||||
//! the useful thing without extra plumbing.
|
||||
//!
|
||||
//! One status needs process state to explain honestly: the driver reports BOTH "your headers are
|
||||
//! newer than my kernel module" and "I can no longer hand this process a session" as
|
||||
//! `NV_ENC_ERR_INVALID_VERSION`. [`note_session_opened`] latches the fact that a session already
|
||||
//! opened here, which tells the two apart — see [`explain`].
|
||||
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
|
||||
use nvidia_video_codec_sdk::sys::nvEncodeAPI as nv;
|
||||
|
||||
/// Latched the first time `nvEncOpenEncodeSessionEx` succeeds in this process (the caps probe, a
|
||||
/// real session open, or the Windows availability probe — every one of them completes the
|
||||
/// userspace↔kernel-module handshake).
|
||||
///
|
||||
/// The load-time gate (`NvEncodeAPIGetMaxSupportedVersion`, both backends' `load_api`) can NOT
|
||||
/// serve this purpose: it is a pure userspace query, so the genuine "updated the driver, didn't
|
||||
/// reboot" skew sails through it and fails later, at the open. Only a session that actually opened
|
||||
/// proves the kernel module agreed.
|
||||
static SESSION_OPENED: AtomicBool = AtomicBool::new(false);
|
||||
|
||||
/// Record that an NVENC session opened. Call right after every successful
|
||||
/// `open_encode_session_ex`, so [`explain`] can rule a version skew out for the rest of the
|
||||
/// process.
|
||||
pub(super) fn note_session_opened() {
|
||||
SESSION_OPENED.store(true, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
/// The two very different failures the driver reports as `NV_ENC_ERR_INVALID_VERSION`, split on
|
||||
/// whether a session has already opened here (`session_opened`). Pure, so both halves are testable
|
||||
/// without touching the process-wide latch.
|
||||
fn invalid_version(session_opened: bool) -> String {
|
||||
if session_opened {
|
||||
// Same status, opposite cause: a session ALREADY opened in this process, so the driver's
|
||||
// kernel module accepted this exact build's version word minutes ago. A version skew is
|
||||
// static — it cannot come and go — so "update the driver / reboot" is the wrong advice
|
||||
// here, and following it costs the operator a reboot per stream (2026-07 field report: one
|
||||
// stream works, the next fails at the caps probe, forever, until the PROCESS restarts).
|
||||
// What is left is per-process driver state: a resource the last session did not give back,
|
||||
// or a wedged/lost device. Say that, and point at the cheap fix.
|
||||
// Worded for ANY call (`explain` also serves `lock_bitstream`); `call_err` already names
|
||||
// the entry point ahead of this text, so it must not assume the session open.
|
||||
return "this process already opened an NVENC session successfully, so this is NOT a driver \
|
||||
version mismatch — that cannot come and go within a process, and a reboot is not \
|
||||
the fix. The NVIDIA driver state in THIS process is exhausted or wedged: restart \
|
||||
the Punktfunk host service to clear it, and please report this with the host log \
|
||||
so it can be fixed properly"
|
||||
.to_string();
|
||||
}
|
||||
// No session has ever opened here, so the version word really is in question. Either the
|
||||
// driver is genuinely older than our headers, or (the sneaky case) the userspace
|
||||
// `libnvidia-encode` reports a new-enough version to the pre-flight probe but the running
|
||||
// kernel module is older and rejects the session — the classic "updated the driver, didn't
|
||||
// reboot" skew. Both heal the same way.
|
||||
format!(
|
||||
"the NVIDIA driver is older than this build's NVENC headers (needs NVENC API {}.{} or \
|
||||
newer), or the userspace and kernel-module driver versions are mismatched — common right \
|
||||
after a driver update without a reboot. Update the NVIDIA driver, or reboot if you just \
|
||||
updated it (a host restart is the usual fix).",
|
||||
nv::NVENCAPI_MAJOR_VERSION,
|
||||
nv::NVENCAPI_MINOR_VERSION,
|
||||
)
|
||||
}
|
||||
|
||||
/// A one-line, operator-actionable cause for an NVENC status. Does not repeat the raw code —
|
||||
/// callers print that alongside (see [`call_err`]). Public for the few sites that build a
|
||||
/// `String`/`format!` error instead of an `anyhow::Error`.
|
||||
pub(super) fn explain(status: nv::NVENCSTATUS) -> String {
|
||||
match status {
|
||||
// The one this whole module exists for: a version word the driver rejects. Either the
|
||||
// driver is genuinely older than our headers, or (the sneaky case) the userspace
|
||||
// `libnvidia-encode` reports a new-enough version to the pre-flight probe but the running
|
||||
// kernel module is older and rejects the session — the classic "updated the driver, didn't
|
||||
// reboot" skew. Both heal the same way.
|
||||
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_VERSION => format!(
|
||||
"the NVIDIA driver is older than this build's NVENC headers (needs NVENC API {}.{} or \
|
||||
newer), or the userspace and kernel-module driver versions are mismatched — common \
|
||||
right after a driver update without a reboot. Update the NVIDIA driver, or reboot if \
|
||||
you just updated it (a host restart is the usual fix).",
|
||||
nv::NVENCAPI_MAJOR_VERSION,
|
||||
nv::NVENCAPI_MINOR_VERSION,
|
||||
),
|
||||
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_VERSION => {
|
||||
invalid_version(SESSION_OPENED.load(Ordering::Relaxed))
|
||||
}
|
||||
nv::NVENCSTATUS::NV_ENC_ERR_NO_ENCODE_DEVICE => {
|
||||
"this GPU exposes no usable NVENC engine — it has no hardware video encoder, or NVENC is \
|
||||
disabled on this card"
|
||||
@@ -110,3 +160,49 @@ pub(super) fn is_param_rejection(err: &anyhow::Error) -> bool {
|
||||
pub(super) fn call_err(call: &str, status: nv::NVENCSTATUS) -> anyhow::Error {
|
||||
anyhow::Error::new(NvCallError(status)).context(format!("NVENC {call} failed"))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// Before any session has opened, the version word IS in question — keep the skew advice.
|
||||
#[test]
|
||||
fn invalid_version_before_any_session_blames_the_driver_version() {
|
||||
let msg = invalid_version(false);
|
||||
assert!(
|
||||
msg.contains("older than this build's NVENC headers"),
|
||||
"{msg}"
|
||||
);
|
||||
assert!(msg.contains("reboot if you just updated it"), "{msg}");
|
||||
}
|
||||
|
||||
/// Once a session has opened here, a skew is impossible — the message must stop sending
|
||||
/// operators to reboot (2026-07 field report: one stream per boot, forever).
|
||||
#[test]
|
||||
fn invalid_version_after_a_session_blames_process_state_not_the_driver() {
|
||||
let msg = invalid_version(true);
|
||||
assert!(msg.contains("NOT a driver version mismatch"), "{msg}");
|
||||
assert!(msg.contains("restart the Punktfunk host service"), "{msg}");
|
||||
assert!(
|
||||
!msg.contains("older than this build's NVENC headers"),
|
||||
"must not repeat the version-skew advice: {msg}"
|
||||
);
|
||||
assert!(
|
||||
!msg.contains("Update the NVIDIA driver"),
|
||||
"must not tell the operator to update a driver that just worked: {msg}"
|
||||
);
|
||||
}
|
||||
|
||||
/// The latch is one-way and only touches this status.
|
||||
#[test]
|
||||
fn note_session_opened_latches() {
|
||||
note_session_opened();
|
||||
assert!(SESSION_OPENED.load(Ordering::Relaxed));
|
||||
note_session_opened();
|
||||
assert!(SESSION_OPENED.load(Ordering::Relaxed));
|
||||
assert_eq!(
|
||||
explain(nv::NVENCSTATUS::NV_ENC_ERR_OUT_OF_MEMORY),
|
||||
"the GPU is out of memory"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -642,6 +642,9 @@ impl NvencD3d11Encoder {
|
||||
e,
|
||||
));
|
||||
}
|
||||
// The handshake with the kernel-mode driver just succeeded — from here on, an
|
||||
// `NV_ENC_ERR_INVALID_VERSION` in this process cannot be a driver version skew.
|
||||
nvenc_status::note_session_opened();
|
||||
let wmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_WIDTH_MAX);
|
||||
let hmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_HEIGHT_MAX);
|
||||
let ten_bit = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_10BIT_ENCODE);
|
||||
@@ -802,6 +805,7 @@ impl NvencD3d11Encoder {
|
||||
}
|
||||
return Err(nvenc_status::call_err("open_encode_session_ex", e));
|
||||
}
|
||||
nvenc_status::note_session_opened();
|
||||
|
||||
let mut cfg = match self.build_config(enc, bitrate) {
|
||||
Ok(cfg) => cfg,
|
||||
@@ -1788,6 +1792,9 @@ fn probe_encode_cap(codec: Codec, cap: nv::NV_ENC_CAPS) -> bool {
|
||||
}
|
||||
return false;
|
||||
}
|
||||
// Availability probe, but a real session open all the same: it proves the driver accepted
|
||||
// this build's version word, which is what rules a skew out later (see `nvenc_status`).
|
||||
nvenc_status::note_session_opened();
|
||||
let mut param = nv::NV_ENC_CAPS_PARAM {
|
||||
version: nv::NV_ENC_CAPS_PARAM_VER,
|
||||
capsToQuery: cap,
|
||||
|
||||
@@ -1488,7 +1488,9 @@ mod nvenc;
|
||||
#[path = "enc/linux/nvenc_cuda.rs"]
|
||||
mod nvenc_cuda;
|
||||
// Actionable `NVENCSTATUS` → cause mapping shared by both direct-NVENC backends, so a failed
|
||||
// session open logs "update/reboot the driver" instead of the old misleading "(no NVIDIA GPU?)".
|
||||
// session open names its real cause instead of the old misleading "(no NVIDIA GPU?)" — including
|
||||
// splitting the two opposite failures the driver reports as the SAME `INVALID_VERSION` (a genuine
|
||||
// driver skew vs. this process's driver state going bad after a session already opened).
|
||||
#[cfg(all(any(target_os = "linux", target_os = "windows"), feature = "nvenc"))]
|
||||
#[path = "enc/nvenc_status.rs"]
|
||||
mod nvenc_status;
|
||||
|
||||
Reference in New Issue
Block a user