fix(nvenc): stop telling operators to reboot when the driver version is fine

`NV_ENC_ERR_INVALID_VERSION` is how the driver reports two opposite failures, and we only
ever explained one of them. The message told the operator to update the NVIDIA driver or
reboot — correct for a genuine header/kernel-module skew, and actively misleading for the
field case behind it: a host that streams once per boot and then fails every later session
at the caps probe, until the *process* restarts. A version skew is static; it cannot come
and go inside one process, so that advice cost the reporter a reboot per stream.

Split the two on the only fact that tells them apart: whether a session has already opened
in this process. `nvenc_status` gains a `SESSION_OPENED` latch set right after every
successful `open_encode_session_ex` — both backends' caps probe and real open, plus the
Windows availability probe. No session yet, the version word really is in question and the
existing skew advice stands. A session already opened, and the kernel module demonstrably
accepted this build's version word, so the message now names per-process driver state and
points at the cheap fix (restart the host service, no reboot) plus a request for the log.

The load-time gate cannot serve as this discriminator: `NvEncodeAPIGetMaxSupportedVersion`
is a pure userspace query, so the classic "updated the driver, didn't reboot" skew sails
through it and only fails later at the open. Only a session that actually opened proves the
kernel module agreed.

The split lives in a pure `invalid_version(bool)` so both halves are unit-tested without
touching the process-wide latch. This is a diagnosis change only — it does not fix the
underlying field bug, whose root cause is still open.

Verified on Linux (192.168.1.25): clippy `-p pf-encode --features nvenc` clean, `cargo test
-p pf-encode --features nvenc --lib` 15 passed, rustfmt clean.
This commit is contained in:
2026-07-25 12:40:51 +02:00
parent ead37d066f
commit 6c2183ceec
4 changed files with 123 additions and 14 deletions
@@ -870,6 +870,9 @@ impl NvencCudaEncoder {
e, e,
)); ));
} }
// The handshake with the kernel module just succeeded — from here on, an
// `NV_ENC_ERR_INVALID_VERSION` in this process cannot be a driver version skew.
nvenc_status::note_session_opened();
let wmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_WIDTH_MAX); let wmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_WIDTH_MAX);
let hmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_HEIGHT_MAX); let hmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_HEIGHT_MAX);
let yuv444 = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_YUV444_ENCODE); let yuv444 = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_YUV444_ENCODE);
@@ -1067,6 +1070,7 @@ impl NvencCudaEncoder {
} }
return Err(nvenc_status::call_err("open_encode_session_ex", e)); return Err(nvenc_status::call_err("open_encode_session_ex", e));
} }
nvenc_status::note_session_opened();
let mut cfg = match self.build_config(enc, bitrate) { let mut cfg = match self.build_config(enc, bitrate) {
Ok(cfg) => cfg, Ok(cfg) => cfg,
+109 -13
View File
@@ -8,27 +8,77 @@
//! means and what the operator should do, and folds that cause into the `anyhow::Error` at //! means and what the operator should do, and folds that cause into the `anyhow::Error` at
//! construction, so every downstream `{e:#}` log (the encode-recovery loop, session teardown) says //! construction, so every downstream `{e:#}` log (the encode-recovery loop, session teardown) says
//! the useful thing without extra plumbing. //! the useful thing without extra plumbing.
//!
//! One status needs process state to explain honestly: the driver reports BOTH "your headers are
//! newer than my kernel module" and "I can no longer hand this process a session" as
//! `NV_ENC_ERR_INVALID_VERSION`. [`note_session_opened`] latches the fact that a session already
//! opened here, which tells the two apart — see [`explain`].
use std::sync::atomic::{AtomicBool, Ordering};
use nvidia_video_codec_sdk::sys::nvEncodeAPI as nv; use nvidia_video_codec_sdk::sys::nvEncodeAPI as nv;
/// Latched the first time `nvEncOpenEncodeSessionEx` succeeds in this process (the caps probe, a
/// real session open, or the Windows availability probe — every one of them completes the
/// userspace↔kernel-module handshake).
///
/// The load-time gate (`NvEncodeAPIGetMaxSupportedVersion`, both backends' `load_api`) can NOT
/// serve this purpose: it is a pure userspace query, so the genuine "updated the driver, didn't
/// reboot" skew sails through it and fails later, at the open. Only a session that actually opened
/// proves the kernel module agreed.
static SESSION_OPENED: AtomicBool = AtomicBool::new(false);
/// Record that an NVENC session opened. Call right after every successful
/// `open_encode_session_ex`, so [`explain`] can rule a version skew out for the rest of the
/// process.
pub(super) fn note_session_opened() {
SESSION_OPENED.store(true, Ordering::Relaxed);
}
/// The two very different failures the driver reports as `NV_ENC_ERR_INVALID_VERSION`, split on
/// whether a session has already opened here (`session_opened`). Pure, so both halves are testable
/// without touching the process-wide latch.
fn invalid_version(session_opened: bool) -> String {
if session_opened {
// Same status, opposite cause: a session ALREADY opened in this process, so the driver's
// kernel module accepted this exact build's version word minutes ago. A version skew is
// static — it cannot come and go — so "update the driver / reboot" is the wrong advice
// here, and following it costs the operator a reboot per stream (2026-07 field report: one
// stream works, the next fails at the caps probe, forever, until the PROCESS restarts).
// What is left is per-process driver state: a resource the last session did not give back,
// or a wedged/lost device. Say that, and point at the cheap fix.
// Worded for ANY call (`explain` also serves `lock_bitstream`); `call_err` already names
// the entry point ahead of this text, so it must not assume the session open.
return "this process already opened an NVENC session successfully, so this is NOT a driver \
version mismatch — that cannot come and go within a process, and a reboot is not \
the fix. The NVIDIA driver state in THIS process is exhausted or wedged: restart \
the Punktfunk host service to clear it, and please report this with the host log \
so it can be fixed properly"
.to_string();
}
// No session has ever opened here, so the version word really is in question. Either the
// driver is genuinely older than our headers, or (the sneaky case) the userspace
// `libnvidia-encode` reports a new-enough version to the pre-flight probe but the running
// kernel module is older and rejects the session — the classic "updated the driver, didn't
// reboot" skew. Both heal the same way.
format!(
"the NVIDIA driver is older than this build's NVENC headers (needs NVENC API {}.{} or \
newer), or the userspace and kernel-module driver versions are mismatched — common right \
after a driver update without a reboot. Update the NVIDIA driver, or reboot if you just \
updated it (a host restart is the usual fix).",
nv::NVENCAPI_MAJOR_VERSION,
nv::NVENCAPI_MINOR_VERSION,
)
}
/// A one-line, operator-actionable cause for an NVENC status. Does not repeat the raw code — /// A one-line, operator-actionable cause for an NVENC status. Does not repeat the raw code —
/// callers print that alongside (see [`call_err`]). Public for the few sites that build a /// callers print that alongside (see [`call_err`]). Public for the few sites that build a
/// `String`/`format!` error instead of an `anyhow::Error`. /// `String`/`format!` error instead of an `anyhow::Error`.
pub(super) fn explain(status: nv::NVENCSTATUS) -> String { pub(super) fn explain(status: nv::NVENCSTATUS) -> String {
match status { match status {
// The one this whole module exists for: a version word the driver rejects. Either the nv::NVENCSTATUS::NV_ENC_ERR_INVALID_VERSION => {
// driver is genuinely older than our headers, or (the sneaky case) the userspace invalid_version(SESSION_OPENED.load(Ordering::Relaxed))
// `libnvidia-encode` reports a new-enough version to the pre-flight probe but the running }
// kernel module is older and rejects the session — the classic "updated the driver, didn't
// reboot" skew. Both heal the same way.
nv::NVENCSTATUS::NV_ENC_ERR_INVALID_VERSION => format!(
"the NVIDIA driver is older than this build's NVENC headers (needs NVENC API {}.{} or \
newer), or the userspace and kernel-module driver versions are mismatched — common \
right after a driver update without a reboot. Update the NVIDIA driver, or reboot if \
you just updated it (a host restart is the usual fix).",
nv::NVENCAPI_MAJOR_VERSION,
nv::NVENCAPI_MINOR_VERSION,
),
nv::NVENCSTATUS::NV_ENC_ERR_NO_ENCODE_DEVICE => { nv::NVENCSTATUS::NV_ENC_ERR_NO_ENCODE_DEVICE => {
"this GPU exposes no usable NVENC engine — it has no hardware video encoder, or NVENC is \ "this GPU exposes no usable NVENC engine — it has no hardware video encoder, or NVENC is \
disabled on this card" disabled on this card"
@@ -110,3 +160,49 @@ pub(super) fn is_param_rejection(err: &anyhow::Error) -> bool {
pub(super) fn call_err(call: &str, status: nv::NVENCSTATUS) -> anyhow::Error { pub(super) fn call_err(call: &str, status: nv::NVENCSTATUS) -> anyhow::Error {
anyhow::Error::new(NvCallError(status)).context(format!("NVENC {call} failed")) anyhow::Error::new(NvCallError(status)).context(format!("NVENC {call} failed"))
} }
#[cfg(test)]
mod tests {
use super::*;
/// Before any session has opened, the version word IS in question — keep the skew advice.
#[test]
fn invalid_version_before_any_session_blames_the_driver_version() {
let msg = invalid_version(false);
assert!(
msg.contains("older than this build's NVENC headers"),
"{msg}"
);
assert!(msg.contains("reboot if you just updated it"), "{msg}");
}
/// Once a session has opened here, a skew is impossible — the message must stop sending
/// operators to reboot (2026-07 field report: one stream per boot, forever).
#[test]
fn invalid_version_after_a_session_blames_process_state_not_the_driver() {
let msg = invalid_version(true);
assert!(msg.contains("NOT a driver version mismatch"), "{msg}");
assert!(msg.contains("restart the Punktfunk host service"), "{msg}");
assert!(
!msg.contains("older than this build's NVENC headers"),
"must not repeat the version-skew advice: {msg}"
);
assert!(
!msg.contains("Update the NVIDIA driver"),
"must not tell the operator to update a driver that just worked: {msg}"
);
}
/// The latch is one-way and only touches this status.
#[test]
fn note_session_opened_latches() {
note_session_opened();
assert!(SESSION_OPENED.load(Ordering::Relaxed));
note_session_opened();
assert!(SESSION_OPENED.load(Ordering::Relaxed));
assert_eq!(
explain(nv::NVENCSTATUS::NV_ENC_ERR_OUT_OF_MEMORY),
"the GPU is out of memory"
);
}
}
@@ -642,6 +642,9 @@ impl NvencD3d11Encoder {
e, e,
)); ));
} }
// The handshake with the kernel-mode driver just succeeded — from here on, an
// `NV_ENC_ERR_INVALID_VERSION` in this process cannot be a driver version skew.
nvenc_status::note_session_opened();
let wmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_WIDTH_MAX); let wmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_WIDTH_MAX);
let hmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_HEIGHT_MAX); let hmax = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_HEIGHT_MAX);
let ten_bit = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_10BIT_ENCODE); let ten_bit = self.get_cap(enc, nv::NV_ENC_CAPS::NV_ENC_CAPS_SUPPORT_10BIT_ENCODE);
@@ -802,6 +805,7 @@ impl NvencD3d11Encoder {
} }
return Err(nvenc_status::call_err("open_encode_session_ex", e)); return Err(nvenc_status::call_err("open_encode_session_ex", e));
} }
nvenc_status::note_session_opened();
let mut cfg = match self.build_config(enc, bitrate) { let mut cfg = match self.build_config(enc, bitrate) {
Ok(cfg) => cfg, Ok(cfg) => cfg,
@@ -1788,6 +1792,9 @@ fn probe_encode_cap(codec: Codec, cap: nv::NV_ENC_CAPS) -> bool {
} }
return false; return false;
} }
// Availability probe, but a real session open all the same: it proves the driver accepted
// this build's version word, which is what rules a skew out later (see `nvenc_status`).
nvenc_status::note_session_opened();
let mut param = nv::NV_ENC_CAPS_PARAM { let mut param = nv::NV_ENC_CAPS_PARAM {
version: nv::NV_ENC_CAPS_PARAM_VER, version: nv::NV_ENC_CAPS_PARAM_VER,
capsToQuery: cap, capsToQuery: cap,
+3 -1
View File
@@ -1488,7 +1488,9 @@ mod nvenc;
#[path = "enc/linux/nvenc_cuda.rs"] #[path = "enc/linux/nvenc_cuda.rs"]
mod nvenc_cuda; mod nvenc_cuda;
// Actionable `NVENCSTATUS` → cause mapping shared by both direct-NVENC backends, so a failed // Actionable `NVENCSTATUS` → cause mapping shared by both direct-NVENC backends, so a failed
// session open logs "update/reboot the driver" instead of the old misleading "(no NVIDIA GPU?)". // session open names its real cause instead of the old misleading "(no NVIDIA GPU?)" — including
// splitting the two opposite failures the driver reports as the SAME `INVALID_VERSION` (a genuine
// driver skew vs. this process's driver state going bad after a session already opened).
#[cfg(all(any(target_os = "linux", target_os = "windows"), feature = "nvenc"))] #[cfg(all(any(target_os = "linux", target_os = "windows"), feature = "nvenc"))]
#[path = "enc/nvenc_status.rs"] #[path = "enc/nvenc_status.rs"]
mod nvenc_status; mod nvenc_status;