Every NVIDIA host advertised a static H.264|HEVC|AV1 superset, so a 1st-gen
Maxwell (GTX 960M, no HEVC/AV1 encode) offered HEVC — a client that believed
it got ~15 s of blank video and a disconnect instead of a stream. Both OSes
now ask the driver itself (nvEncGetEncodeGUIDs) on one throwaway direct-SDK
session: Linux on the shared CUDA context, Windows on the selected render
adapter, wired into host_wire_caps AND the GameStream serverinfo mask (which
had been left on the superset for NVIDIA on both OSes). Fails open — an
unanswerable probe keeps the historical superset, so it can only ever narrow
the advertisement to codecs the GPU really encodes.
The HEVC 4:4:4 answer rides the same session on Linux instead of opening a
libav hevc_nvenc FREXT probe: that open is the prime suspect for the field
bug where one probe wedges NVENC process-wide (NV_ENC_ERR_INVALID_VERSION on
every later session until a host restart), and the direct backend re-checks
the same caps bit at session open anyway. The ffmpeg probe remains only for
hosts that really stream over libav (PUNKTFUNK_NVENC_DIRECT=0 or a build
without the nvenc feature), where ffmpeg's NVENC client runs regardless. The
10-bit probe deliberately stays libav — Linux HDR rides the libav P010 path.
On-hardware: .136 (RTX 5070 Ti) 14/14 nvenc tests in one process incl. the
probe followed by real sessions and dirty teardown; .173 (Windows RTX) probe
+ 47 release lib tests. The Windows probe test documents the pre-existing
MSVC debug-link failure (LNK2019 via the sdk crate's unused lazy loader) —
run it with --release, the same reason windows-host.yml gates with clippy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 0346ec8090568eb499e8cb7d735305b28471185e)
enricobuehler
188f55d3b1