unom/punktfunk
6
0
Fork 1
Code Issues 3 Pull Requests 1 Actions 1 Packages Projects Releases 47 Wiki Activity
Files
2bf571a5ad824b303fadeee1775e2e792cab5938
punktfunk/crates/pyrowave-sys/patches/0005-global-priority-queue.patch
T
enricobuehler 2bf571a5ad feat(pf-encode): PyroWave's Linux encode device never asked for the priority its own patch requests 
Wave-2 PW1, first half = Wave-1 WP14 step 4, executed as specced.

PyroWave encodes on the same GPU shader cores a game saturates, and that is measured to hurt:
patch 0005's header records `encode_gpu_synchronous` going from ~2 ms to 15-18 ms at 95 % game
load, with the stream frame rate collapsing. NVENC is immune because it has its own ASIC. The
lever for a compute workload is an elevated global-priority QUEUE — a process-priority raise only
reorders submission, not hardware preemption.

The vendored patch requests exactly that. It is gated `if (!inherit_info)`, and only Windows
leaves `inherit_info` null (`pyrowave_create_device_by_compat`, where Granite builds the device
itself). Linux passes its own create-infos into `pyrowave_device_create_info`, Granite's
`get_existing_create_info()` hands them back, `create_device` takes the inherit branch — and the
whole block is skipped. On Linux the knob has never done anything at all. Meanwhile pf-zerocopy's
VkBridge has shipped the identical ladder on Linux for some time and calls it "the actual NVIDIA
compute-preemption lever"; the encoder that needs it most did not have it.

This wires it natively in `open_inner`'s `DeviceHold`:

- The extension probe reuses the `dev_ext_props` already fetched for queue_family_foreign, and
  takes KHR or the EXT alias — the same spelling pf-zerocopy probes, so the two cannot disagree.
- `queue_priority_candidates` is a pure fn with the grammar copied from the C patch: unset →
  realtime, ASCII-lowercased, `off` alone disables, `high` asks for HIGH only, junk falls back to
  the ladder rather than to off. One env var must not mean two things on two platforms — that is
  the documentation trap this package exists to close — so the grammar is unit-tested against the
  patch's, including where they are both deliberately un-clever (neither trims).
- The create ladder is REALTIME → HIGH → no-priority, stepping only on a refusal. A refused class
  can never fail the open, which matters more here than on Windows: this path is reached only by a
  NEGOTIATED PyroWave session, so a hard error is a dead stream, not a fallback to another encoder.

The subtle part is the write-back. `pyrowave_create_device` RETAINS `device_create_info` for the
device's lifetime and Granite reads the chain back. If the ladder ends on the no-priority attempt
while `_queue_ci[0].p_next` still points at the global-priority struct, Granite is handed a chain
the device was not created with. The `None` arm therefore nulls `p_next` before the final create,
and the field's doc says why. The enabled extension deliberately STAYS in the list: it really is
enabled on the device, it just carries no request.

One deviation from the plan, stated because it is a deviation: the ladder also steps down on
`ERROR_INITIALIZATION_FAILED`, not only `ERROR_NOT_PERMITTED_KHR`. The plan and the C patch handle
only the latter; pf-zerocopy's shipped ladder accepts both. Given a hard error here kills a
negotiated session, treating one extra driver-specific refusal as a downgrade is the cheap side of
that asymmetry.

Also corrects the two vendored notes, which claimed a Linux behaviour the gate made impossible,
and records that patch 0005's negative RTX-4090 result is Windows/WDDM and does not transfer to a
different driver stack. Patch hunks are byte-identical (header prose only) and
`git diff crates/pyrowave-sys/vendor/` is PUNKTFUNK-VENDOR.txt alone.

`PYROWAVE_QUEUE_PRIORITY` is now reachable on Linux, so it is documented in the same PR.

MEASURED ON GLASS, and it changes what this package is worth on its own — .21, RTX 5070 Ti,
NVIDIA 610.43.02, same binary in both arms:

  as packaged (no capability)     every class refused, REALTIME *and* HIGH -> default priority
  same binary, cap_sys_nice+ep    granted REALTIME on the FIRST attempt, no downgrade

So the lever is INERT on an unprivileged host, and that is not the RADV-specific downgrade the
plan predicted — on NVIDIA it is a downgrade to nothing at all. The ladder itself is proven good
across all three legs (unset / high / off): a refused class never fails the open, and `off`
enables no extension and logs nothing. It simply has nothing to grant yet.

The privilege needed is CAP_SYS_NICE on the host binary, which is NOT what Wave-1 WP3 ships
(RLIMIT_NICE, PAM limits, CPUWeight — all different things). That grant is a security-posture
change on a network-facing daemon, so it is deliberately NOT in this commit; the warn line now
names the capability so an operator is not left guessing, and the docs row says the setting has no
effect on most hosts today rather than implying it works.

The loaded-GPU encode_us p99 A/B is therefore not run: it needs a GPU-saturating game (hence a
desktop session the box does not currently have) and it is pointless before the capability lands,
since the unprivileged arm has no priority to measure.

NO unit test is possible for the device-create ladder itself — it needs a real Vulkan device. Its
coverage is the clippy pass, the grammar tests, and the on-glass log line. Stated here rather than
left for a reviewer to wonder about.
2026-08-08 14:37:41 +02:00

135 lines
6.7 KiB
Diff
Raw Blame History

Elevated global-priority encode queue — PUNKTFUNK LOCAL PATCH.
Not upstream. PyroWave's wavelet encode runs on the GPU's compute/shader cores, so a GPU-bound
game starves it: `pyrowave_encoder_encode_gpu_synchronous` spikes from ~2 ms to ~15 ms under a
95%+ game load and the stream framerate collapses (NVENC is immune — it uses the separate encoder
ASIC). A WDDM *process* scheduling-priority raise does not help (it orders packet submission, not
hardware preemption). This requests a global-priority Vulkan *queue* — the actual NVIDIA compute-
preemption lever — on the pyrowave encode device.
`Context::create_device` (the pyrowave path uses plain vkCreateDevice, not the vpCreateDevice
profile path): enable VK_KHR_global_priority (or the EXT alias), chain a
VkDeviceQueueGlobalPriorityCreateInfoKHR into every queue-create-info, and a create loop that tries
`PYROWAVE_QUEUE_PRIORITY` (off|high|realtime, default realtime) → HIGH → no-priority, downgrading on
VK_ERROR_NOT_PERMITTED_KHR so a refused class NEVER regresses the encoder. Gated on !inherit_info
(only the fresh encoder device). This Granite copy is vendored solely for the PyroWave codec.
NOTE: on an RTX 4090 / Windows / WDDM this did not reduce the spikes (the graphics-vs-compute
preemption granularity is the wall) — kept because it is correct, harmless (graceful fallback), and
may help other GPUs/drivers. Reduce the encode's GPU cost (4:2:0/8-bit) or use H.265 for a
GPU-saturated game. **That measurement is Windows/WDDM and does NOT transfer to Linux** — a
different driver stack with a different preemption model; treat Linux as unmeasured until its own
A/B exists.
NOTE 2 — WHERE THIS PATCH IS ACTUALLY LIVE. It is gated `if (!inherit_info)`, and only the WINDOWS
path leaves `inherit_info` null: `crates/pf-encode/src/enc/windows/pyrowave.rs` calls
`pyrowave_create_device_by_compat`, so Granite builds the device itself and this block runs.
**On LINUX it has never done anything.** `crates/pf-encode/src/enc/linux/pyrowave.rs::open_inner`
passes its own instance/device create-infos into `pyrowave_device_create_info`, Granite's
`MyDeviceFactory::get_existing_create_info()` returns them, `create_device` takes the inherit
branch, and the whole block above is skipped. The Linux request is therefore wired natively in
**`crates/pf-encode/src/enc/linux/pyrowave.rs`** (search `queue_priority_candidates`), which
implements the SAME env grammar and the SAME downgrade ladder so one knob means one thing on both
platforms. If you change the grammar here, change it there in the same commit.
diff --git a/crates/pyrowave-sys/vendor/pyrowave/Granite/vulkan/context.cpp b/crates/pyrowave-sys/vendor/pyrowave/Granite/vulkan/context.cpp
index 5257fc33..479eeded 100644
--- a/crates/pyrowave-sys/vendor/pyrowave/Granite/vulkan/context.cpp
+++ b/crates/pyrowave-sys/vendor/pyrowave/Granite/vulkan/context.cpp
@@ -2115,6 +2115,51 @@ bool Context::create_device(VkPhysicalDevice gpu_, VkSurfaceKHR surface,
vpGetProfileProperties(profile.profile, &props);
#endif
+ // PUNKTFUNK PATCH (patches/0005-global-priority-queue.patch, NOT upstream): request a high
+ // global-priority queue so PyroWave's compute-shader encode can PREEMPT a GPU-bound game on
+ // the shared shader cores. A WDDM process-priority raise does not help (it only orders packet
+ // submission, not hardware preemption); a global-priority queue is the actual NVIDIA compute-
+ // preemption lever. `PYROWAVE_QUEUE_PRIORITY` = off | high | realtime (default realtime). The
+ // device-create loop below downgrades on NOT_PERMITTED, so a refused class NEVER fails the
+ // encoder — it just runs at default priority. Only the fresh encoder device (no inherit_info)
+ // is affected; this Granite copy is vendored solely for the PyroWave codec.
+ Util::SmallVector<VkQueueGlobalPriorityKHR> pf_priority_candidates;
+ Util::SmallVector<VkDeviceQueueGlobalPriorityCreateInfoKHR> pf_global_priority_infos;
+ if (!inherit_info)
+ {
+ const char *pf_gp_ext = nullptr;
+ if (has_extension(VK_KHR_GLOBAL_PRIORITY_EXTENSION_NAME))
+ pf_gp_ext = VK_KHR_GLOBAL_PRIORITY_EXTENSION_NAME;
+ else if (has_extension(VK_EXT_GLOBAL_PRIORITY_EXTENSION_NAME))
+ pf_gp_ext = VK_EXT_GLOBAL_PRIORITY_EXTENSION_NAME;
+ std::string pf_prio;
+ if (!Util::get_environment("PYROWAVE_QUEUE_PRIORITY", pf_prio))
+ pf_prio = "realtime";
+ for (auto &pf_ch : pf_prio)
+ if (pf_ch >= 'A' && pf_ch <= 'Z')
+ pf_ch = char(pf_ch + 32);
+ if (pf_gp_ext && pf_prio != "off")
+ {
+ if (pf_prio == "high")
+ {
+ pf_priority_candidates.push_back(VK_QUEUE_GLOBAL_PRIORITY_HIGH_KHR);
+ }
+ else
+ {
+ pf_priority_candidates.push_back(VK_QUEUE_GLOBAL_PRIORITY_REALTIME_KHR);
+ pf_priority_candidates.push_back(VK_QUEUE_GLOBAL_PRIORITY_HIGH_KHR);
+ }
+ enabled_extensions.push_back(pf_gp_ext);
+ pf_global_priority_infos.resize(queue_infos.size());
+ for (size_t pf_i = 0; pf_i < queue_infos.size(); pf_i++)
+ {
+ pf_global_priority_infos[pf_i] = { VK_STRUCTURE_TYPE_DEVICE_QUEUE_GLOBAL_PRIORITY_CREATE_INFO_KHR };
+ pf_global_priority_infos[pf_i].globalPriority = pf_priority_candidates.front();
+ queue_infos[pf_i].pNext = &pf_global_priority_infos[pf_i];
+ }
+ }
+ }
+
if (inherit_info)
{
device_info.enabledExtensionCount = inherit_info->enabledExtensionCount;
@@ -2144,8 +2189,41 @@ bool Context::create_device(VkPhysicalDevice gpu_, VkSurfaceKHR surface,
if (device == VK_NULL_HANDLE)
return false;
}
- else if (vkCreateDevice(gpu, &device_info, nullptr, &device) != VK_SUCCESS)
- return false;
+ else
+ {
+ // PUNKTFUNK: try the requested global-priority class, downgrade through the
+ // candidate list on NOT_PERMITTED, then finally create with no global priority.
+ VkResult pf_res;
+ if (pf_priority_candidates.empty())
+ {
+ pf_res = vkCreateDevice(gpu, &device_info, nullptr, &device);
+ }
+ else
+ {
+ pf_res = VK_ERROR_NOT_PERMITTED_KHR;
+ for (size_t pf_a = 0; pf_a < pf_priority_candidates.size(); pf_a++)
+ {
+ for (auto &pf_gp : pf_global_priority_infos)
+ pf_gp.globalPriority = pf_priority_candidates[pf_a];
+ pf_res = vkCreateDevice(gpu, &device_info, nullptr, &device);
+ if (pf_res != VK_ERROR_NOT_PERMITTED_KHR)
+ break;
+ LOGW("PyroWave: global queue priority %u not permitted; downgrading.\n",
+ unsigned(pf_priority_candidates[pf_a]));
+ }
+ if (pf_res == VK_ERROR_NOT_PERMITTED_KHR)
+ {
+ for (auto &pf_qi : queue_infos)
+ pf_qi.pNext = nullptr;
+ pf_res = vkCreateDevice(gpu, &device_info, nullptr, &device);
+ LOGW("PyroWave: all global queue priorities refused; default priority.\n");
+ }
+ else
+ LOGI("PyroWave: encode device created with an elevated global queue priority.\n");
+ }
+ if (pf_res != VK_SUCCESS)
+ return false;
+ }
}
if (inherit_info)
Reference in New Issue View Git Blame Copy Permalink