enricobuehler
2bf571a5ad
Wave-2 PW1, first half = Wave-1 WP14 step 4, executed as specced. PyroWave encodes on the same GPU shader cores a game saturates, and that is measured to hurt: patch 0005's header records `encode_gpu_synchronous` going from ~2 ms to 15-18 ms at 95 % game load, with the stream frame rate collapsing. NVENC is immune because it has its own ASIC. The lever for a compute workload is an elevated global-priority QUEUE — a process-priority raise only reorders submission, not hardware preemption. The vendored patch requests exactly that. It is gated `if (!inherit_info)`, and only Windows leaves `inherit_info` null (`pyrowave_create_device_by_compat`, where Granite builds the device itself). Linux passes its own create-infos into `pyrowave_device_create_info`, Granite's `get_existing_create_info()` hands them back, `create_device` takes the inherit branch — and the whole block is skipped. On Linux the knob has never done anything at all. Meanwhile pf-zerocopy's VkBridge has shipped the identical ladder on Linux for some time and calls it "the actual NVIDIA compute-preemption lever"; the encoder that needs it most did not have it. This wires it natively in `open_inner`'s `DeviceHold`: - The extension probe reuses the `dev_ext_props` already fetched for queue_family_foreign, and takes KHR or the EXT alias — the same spelling pf-zerocopy probes, so the two cannot disagree. - `queue_priority_candidates` is a pure fn with the grammar copied from the C patch: unset → realtime, ASCII-lowercased, `off` alone disables, `high` asks for HIGH only, junk falls back to the ladder rather than to off. One env var must not mean two things on two platforms — that is the documentation trap this package exists to close — so the grammar is unit-tested against the patch's, including where they are both deliberately un-clever (neither trims). - The create ladder is REALTIME → HIGH → no-priority, stepping only on a refusal. A refused class can never fail the open, which matters more here than on Windows: this path is reached only by a NEGOTIATED PyroWave session, so a hard error is a dead stream, not a fallback to another encoder. The subtle part is the write-back. `pyrowave_create_device` RETAINS `device_create_info` for the device's lifetime and Granite reads the chain back. If the ladder ends on the no-priority attempt while `_queue_ci[0].p_next` still points at the global-priority struct, Granite is handed a chain the device was not created with. The `None` arm therefore nulls `p_next` before the final create, and the field's doc says why. The enabled extension deliberately STAYS in the list: it really is enabled on the device, it just carries no request. One deviation from the plan, stated because it is a deviation: the ladder also steps down on `ERROR_INITIALIZATION_FAILED`, not only `ERROR_NOT_PERMITTED_KHR`. The plan and the C patch handle only the latter; pf-zerocopy's shipped ladder accepts both. Given a hard error here kills a negotiated session, treating one extra driver-specific refusal as a downgrade is the cheap side of that asymmetry. Also corrects the two vendored notes, which claimed a Linux behaviour the gate made impossible, and records that patch 0005's negative RTX-4090 result is Windows/WDDM and does not transfer to a different driver stack. Patch hunks are byte-identical (header prose only) and `git diff crates/pyrowave-sys/vendor/` is PUNKTFUNK-VENDOR.txt alone. `PYROWAVE_QUEUE_PRIORITY` is now reachable on Linux, so it is documented in the same PR. MEASURED ON GLASS, and it changes what this package is worth on its own — .21, RTX 5070 Ti, NVIDIA 610.43.02, same binary in both arms: as packaged (no capability) every class refused, REALTIME *and* HIGH -> default priority same binary, cap_sys_nice+ep granted REALTIME on the FIRST attempt, no downgrade So the lever is INERT on an unprivileged host, and that is not the RADV-specific downgrade the plan predicted — on NVIDIA it is a downgrade to nothing at all. The ladder itself is proven good across all three legs (unset / high / off): a refused class never fails the open, and `off` enables no extension and logs nothing. It simply has nothing to grant yet. The privilege needed is CAP_SYS_NICE on the host binary, which is NOT what Wave-1 WP3 ships (RLIMIT_NICE, PAM limits, CPUWeight — all different things). That grant is a security-posture change on a network-facing daemon, so it is deliberately NOT in this commit; the warn line now names the capability so an operator is not left guessing, and the docs row says the setting has no effect on most hosts today rather than implying it works. The loaded-GPU encode_us p99 A/B is therefore not run: it needs a GPU-saturating game (hence a desktop session the box does not currently have) and it is pointless before the capability lands, since the unprivileged arm has no priority to measure. NO unit test is possible for the device-create ladder itself — it needs a real Vulkan device. Its coverage is the clippy pass, the grammar tests, and the on-glass log line. Stated here rather than left for a reviewer to wonder about.
PyroWave
PyroWave is an intra-only video codec (practially speaking a still-image codec) that is optimized for extremely fast GPU encode (< ~0.1 ms encode and decode at 1080p, < ~0.2 ms at 4K). It is fully implemented in Vulkan compute shaders.
The targeted bit-rates are quite high (~200+ mbit/s) and the intended use case is local network game streaming over ethernet with absolute minimum latency where bandwidth is less of a concern.
It is integrated as part of my pyrofling project.
It is similar in scope to my master thesis from 2014, except the use case now is game streaming instead of adaptation of raw video to ethernet links.
Overview
Colorspace
Currently implemented YCbCr 4:2:0 and 4:4:4.
Wavelets
The images are transformed with the Discrete Wavelet Transform using CDF 9/7 filter. This is basically the exact same as JPEG2000.
Exact rate control
The encoder can target exact maximum bitrate for an encoded image.
Trivial "entropy" coding
The encoding of coefficients is trivial compared to normal image codecs, and is responsible for a large increase in bit-rate, especially at higher compression ratios. This simplicity massively improves encode/decode performance, since it's extremely parallelizable. It also makes it possible to do a single-pass exact rate control for an entire image.
It should be possible in theory to add a more proper entropy coder to the encoded bit-planes to achieve somewhat competent compression, but that has not been explored and is out of scope. High Throughput JPEG2000 is likely a good place to look for that anyway.
Robustness against packet loss
Being intra-only and encoding 64x64 blocks of coefficients in isolation ensures great recovery against packet loss. PyroWave has been battled tested over long distance streaming over fiber links.
Bitstream definition
Building
PyroWave is intended to be built alongside PyroFling with Granite in the normal case.
Standalone C API
NOTE: This API is still under development and the API/ABI is not yet quite stable.
A small portion of Granite needs to be checked out.
bash checkout_granite.sh
Build normally with CMake and a C API is installed.
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=output -DCMAKE_BUILD_TYPE=Release -G Ninja
$ ninja install
[0/1] Install the project...
-- Install configuration: "Release"
-- Installing: ...../build/output/include/pyrowave/pyrowave.h
-- Installing: ...../build/output/lib/libpyrowave-shared.so.0.0.0
-- Installing: ...../build/output/lib/libpyrowave-shared.so.0
-- Installing: ...../build/output/lib/libpyrowave-shared.so
-- Installing: ...../build/output/share/pyrowave-shared/cmake/pyrowave-sharedConfig.cmake
-- Installing: ...../build/output/share/pyrowave-shared/cmake/pyrowave-sharedConfig-release.cmake
-- Installing: ...../build/output/share/pkgconfig/pyrowave-shared.pc
The build is tested on Linux, MinGW, msys2 and MSVC.
See pyrowave-c-test which unit tests the shared C API.
That test also serves as a basic user guide for the API.
build-steamrt.sh builds against the Sniper SDK and is also supported.
Local development and CLI
For the sample and test applications in this repo however, check out
the full https://github.com/Themaister/Granite before invoking CMake.
Build with -DPYROWAVE_DEVEL=ON to get the "full" build.
git clone --depth 1 --recursive --shallow-submodules https://github.com/Themaister/Granite Granite
Basic encoder/decoder CLI
The basic CLI takes a y4m and dumps out a raw bitstream. This is just intended for local testing.
pyrowave-encode test.y4m out.wave 400000
This encodes a raw bitstream where each frame consumes maximum 400 KB. To decode back to y4m:
pyrowave-decode out.wave out.y4m