From 767e67caf45b3d9b2f6f66d385255db6a66e4a75 Mon Sep 17 00:00:00 2001 From: enricobuehler Date: Sat, 8 Aug 2026 15:24:54 +0200 Subject: [PATCH] feat(packaging): grant the host CAP_SYS_NICE, without which the GPU-priority lever does nothing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Wave-2 PW1, second half. The companion commit wires `PYROWAVE_QUEUE_PRIORITY` into the Linux PyroWave device; this is what makes it work on a packaged host. Measured on .21 (RTX 5070 Ti, NVIDIA 610.43.02), same binary in both arms: as packaged (no capability) every class refused, REALTIME *and* HIGH -> default priority same binary, cap_sys_nice+ep granted REALTIME on the FIRST attempt, no downgrade RADV behaves the same way. So this is not the RADV-specific "expect one downgrade to HIGH" the plan predicted — without the capability there is no elevated priority at all, on any vendor, and the knob is decoration. Worth being precise about what is being granted, because it is a network-facing daemon. CAP_SYS_NICE permits raising scheduling priority (nice, ioprio, affinity, RT class) and nothing else: no filesystem access, no network privilege, no user switching, and it is NOT setuid. The repo already ships exactly this capability on its gamescope binary for the same reason. Two side effects that will otherwise confuse someone debugging: a capability-carrying binary is AT_SECURE, so the loader ignores LD_LIBRARY_PATH/LD_PRELOAD for it (note this box was propped up by exactly such a shim during the ffmpeg-9 soname break — that workaround would now be silently ignored), and core dumps are suppressed by default. Per packaging path, because none of them are the same: - Arch: a `_grant_sched_capability` in the scriptlet, called from post_install AND post_upgrade — a replaced binary is a new inode, so the capability does not survive an upgrade by itself. - Debian: the same setcap in the postinst `configure` branch. - RPM: `%caps(cap_sys_nice=ep)` on the binary in `%files`, which is the rpm-native form — rpm then applies it on install, restores it on upgrade, and verifies it. A `%post setcap` does none of those. - NixOS: `security.wrappers`, because a store path is read-only and shared and cannot be setcap'd. The unit's ExecStart moves to `config.security.wrapperDir` — without that the wrapper exists and the service still runs the uncapped store path, which is the whole failure this fixes. - Steam Deck: setcap in the installer's sudo block. That box needs it most (one small Van Gogh GPU shared between the game and the encode). The binary lives under $HOME, so unlike the /etc drop-ins it survives a SteamOS A/B update on its own and needs no atomic-keep entry — but it does need re-applying after each rebuild, which re-running the installer does. - Bazzite sysext: at IMAGE BUILD time, before mksquashfs. It cannot be done in the merge hook (a merged sysext's /usr is read-only squashfs) and it cannot ride in from the RPM either — rpm keeps capabilities in its own header and `rpm2cpio | cpio` carries only the payload, so the staged file arrives with none. mksquashfs does record security.capability (only security.selinux is excluded), so a setcap on the staging tree is what lands in the image. Needs root/CAP_SETFCAP; a plain-user CI build warns and ships without it rather than failing a release over a performance lever. Every one of them is best-effort and cannot fail an install: a box without libcap, or a filesystem that cannot store capabilities, simply runs at default priority exactly as it does today. Documented in the same PR — the configuration row now says the packages grant it, and running-as-a-service gets a section explaining what it is, how to check it (`getcap`), and how to remove it (`setcap -r`, or just `PYROWAVE_QUEUE_PRIORITY=off`), including the two debugging side effects. Verified: the Arch scriptlet grants the capability from a fake package root exactly as pacman would invoke it, and the resulting binary reaches REALTIME end to end on the RTX 5070 Ti; the RPM spec's %caps line parses under rpmspec in a Fedora 41 container; the NixOS module parses under nix-instantiate; all five edited shell scripts pass `bash -n`. No Rust file changed in this commit, so the CI-parity Rust gates from the companion commit still stand. --- docs-site/content/docs/configuration.md | 2 +- .../content/docs/running-as-a-service.md | 30 +++++++++++++++++++ packaging/arch/punktfunk-host.install | 26 ++++++++++++++++ packaging/bazzite/build-sysext.sh | 25 ++++++++++++++++ packaging/debian/build-deb.sh | 10 +++++++ packaging/nix/nixos-module.nix | 26 +++++++++++++++- packaging/rpm/punktfunk.spec | 10 ++++++- scripts/steamdeck/install.sh | 19 ++++++++++++ 8 files changed, 145 insertions(+), 3 deletions(-) diff --git a/docs-site/content/docs/configuration.md b/docs-site/content/docs/configuration.md index e1f00f01..72b4dbd5 100644 --- a/docs-site/content/docs/configuration.md +++ b/docs-site/content/docs/configuration.md @@ -240,7 +240,7 @@ notes for context. | `PUNKTFUNK_NVENC_SPLIT_ARBITRATE` | `1` | Opt-in: let the host change its split-encode decision **live**, mid-session, as the pixel rate moves, instead of only choosing once at session start. Currently wired on the Linux direct-NVENC path. Only interesting alongside `PUNKTFUNK_SPLIT_ENCODE=auto` at very high pixel rates. | | `PUNKTFUNK_GPU_PRIORITY_CLASS` | `off` · `normal` · `high` · `realtime` · `auto` | **(Windows)** GPU scheduling priority for capture/encode under a GPU-saturating game. Default `auto` (starts `high`, upgrades to `realtime` when it's safe — e.g. HAGS off); `high` pins the static pre-gate behaviour; `realtime` is the strongest lever but can freeze NVENC on some setups. | | `PUNKTFUNK_IDD_DEPTH` | `N` (default `2`) | **(Windows)** IDD-push pipeline depth. `1` cuts latency once GPU priority is raised; higher smooths a contended GPU. | -| `PYROWAVE_QUEUE_PRIORITY` | `realtime` *(default)* · `high` · `off` | [PyroWave](/docs/pyrowave) sessions only. PyroWave encodes on the same GPU shader cores a game uses, so a demanding game can starve it and the frame rate drops. This asks the driver to schedule the encode ahead of the game. `realtime` tries the strongest class and falls back to `high`; `high` asks only for the middle one; `off` disables the request. A driver that refuses simply encodes at normal priority — it can never stop a session starting. **Today most Linux hosts are refused every class**, because the drivers require the `CAP_SYS_NICE` capability and the packaged host does not carry it yet — so this setting currently has no effect on them and the host says so once, at session start. Set `off` to silence that, or if you see the desktop stutter while streaming. | +| `PYROWAVE_QUEUE_PRIORITY` | `realtime` *(default)* · `high` · `off` | [PyroWave](/docs/pyrowave) sessions only. PyroWave encodes on the same GPU shader cores a game uses, so a demanding game can starve it and the frame rate drops. This asks the driver to schedule the encode ahead of the game. `realtime` tries the strongest class and falls back to `high`; `high` asks only for the middle one; `off` disables the request. A driver that refuses simply encodes at normal priority — it can never stop a session starting. The packages grant the host the `CAP_SYS_NICE` capability this needs; on a host built or installed by hand it will be refused, and the host says so once at session start. Set `off` if you see the desktop stutter while streaming. | ## Diagnostics diff --git a/docs-site/content/docs/running-as-a-service.md b/docs-site/content/docs/running-as-a-service.md index 750b6857..f510e8fd 100644 --- a/docs-site/content/docs/running-as-a-service.md +++ b/docs-site/content/docs/running-as-a-service.md @@ -205,6 +205,36 @@ the host. If the host answers, it's up. If not, check `journalctl --user -u punktfunk-host` on the host — on a Windows host, run `punktfunk-host service status` from an elevated prompt on the machine itself. +## GPU scheduling priority + +The Linux packages give the host binary one Linux capability, `CAP_SYS_NICE`, and it is worth +knowing why it is there and how to take it away. + +The [PyroWave](/docs/pyrowave) codec encodes on the same GPU shader cores your game is using, so a +demanding game can crowd it out and the stream's frame rate drops with it. The fix is to ask the +driver to schedule the encode ahead of the game, and every driver we tested gates that request on +this capability: without it the request is simply refused and nothing changes. The other codecs use +a separate video engine on the GPU and are unaffected either way. + +`CAP_SYS_NICE` lets a process raise its own scheduling priority. It grants no access to files, +the network or other users' processes, and it is **not** the same as running as root — the host +still runs as you, under your user session. + +To check, or to take it away: + +```sh +getcap /usr/bin/punktfunk-host # shows cap_sys_nice=ep when granted +sudo setcap -r /usr/bin/punktfunk-host # remove it; streaming still works +``` + +Removing it costs you nothing unless you stream PyroWave, and you can also just set +`PYROWAVE_QUEUE_PRIORITY=off` to stop the host asking. Note that a package update replaces the +binary and re-applies the capability. + +Two side effects, if you are debugging the host: a binary carrying a capability is treated as +security-sensitive by the dynamic loader, so `LD_LIBRARY_PATH` and `LD_PRELOAD` are ignored for it, +and it does not write core dumps by default. + ## Stopping and removing After a Linux package update the user service keeps running the old binary until it's restarted, and diff --git a/packaging/arch/punktfunk-host.install b/packaging/arch/punktfunk-host.install index 07af7c24..ec20beb3 100644 --- a/packaging/arch/punktfunk-host.install +++ b/packaging/arch/punktfunk-host.install @@ -12,9 +12,33 @@ _ensure_punktfunk_group() { getent group punktfunk >/dev/null 2>&1 || groupadd --system punktfunk 2>/dev/null || true } +# CAP_SYS_NICE on the host binary — the GPU-scheduling grant. +# +# WHY: PyroWave encodes on the GPU's shader cores, so a GPU-bound game starves it (measured: the +# encode dispatch goes from ~2 ms to 15-18 ms at 95 % game load). The fix is an elevated +# global-priority Vulkan queue, which the driver gates on CAP_SYS_NICE — measured 2026-08-08 on an +# RTX 5070 Ti: WITHOUT the capability every priority class is refused, WITH it the encoder is +# granted REALTIME on the first attempt. RADV is the same. Without this line the knob exists and +# does nothing. Same capability, same mechanism, as our gamescope package sets on its own binary. +# +# NARROW: CAP_SYS_NICE only permits raising scheduling priority (nice/ioprio/affinity/RT class). It +# grants no filesystem, network or user-switching privilege, and it is NOT setuid. +# +# TWO CONSEQUENCES worth knowing before you debug something odd on this host: +# * a file capability makes the process AT_SECURE, so the dynamic loader IGNORES LD_LIBRARY_PATH +# and LD_PRELOAD for it. A library-path shim that used to work will silently stop. +# * core dumps are suppressed for capability-carrying binaries by default (fs.suid_dumpable). +# +# Never fails the install: a box without libcap, or a filesystem that cannot store capabilities +# (some overlay/NFS setups), just runs at default priority exactly as before. +_grant_sched_capability() { + setcap 'cap_sys_nice=ep' usr/bin/punktfunk-host 2>/dev/null || true +} + post_install() { _ensure_update_group _ensure_punktfunk_group + _grant_sched_capability udevadm control --reload-rules 2>/dev/null || true udevadm trigger --subsystem-match=misc 2>/dev/null || true # Apply the UDP socket-buffer tuning now (also auto-applied at boot by systemd-sysctl). @@ -67,6 +91,8 @@ MSG post_upgrade() { _ensure_update_group + # A replaced binary is a NEW inode — file capabilities do not survive the upgrade, so re-grant. + _grant_sched_capability udevadm control --reload-rules 2>/dev/null || true sysctl -p /usr/lib/sysctl.d/99-punktfunk-net.conf >/dev/null 2>&1 || true _warn_stale_firewall_ports diff --git a/packaging/bazzite/build-sysext.sh b/packaging/bazzite/build-sysext.sh index c88acd62..32c2861b 100644 --- a/packaging/bazzite/build-sysext.sh +++ b/packaging/bazzite/build-sysext.sh @@ -130,6 +130,31 @@ SYSEXT_VERSION_ID=$PF_VR EXTENSION_RELOAD_MANAGER=1 EOF +# CAP_SYS_NICE on the host binary — the GPU-scheduling grant. PyroWave encodes on the GPU shader +# cores a game saturates, and the driver gates the elevated global-priority Vulkan queue that fixes +# it on this capability (measured 2026-08-08 on an RTX 5070 Ti: refused without it, granted REALTIME +# with it; RADV the same). Narrow — scheduling priority only, no filesystem/network privilege, not +# setuid. +# +# It has to be applied HERE, not in the merge hook: a merged sysext's /usr is a read-only squashfs, +# so nothing can setcap it afterwards. And it cannot ride in from the RPM either — the spec declares +# it with %caps, but rpm stores capabilities in its own header and `rpm2cpio | cpio` carries only +# the payload, so the staged file arrives with no capability at all. mksquashfs DOES record +# security.capability (only security.selinux is excluded below), so a setcap on the staging tree is +# what ends up in the image. +# +# Needs CAP_SETFCAP, i.e. root (or fakeroot) — a plain-user CI build cannot do it. That is not fatal: +# the image just ships as it does today and the encode runs at default GPU priority, so warn and +# carry on rather than fail a release build over a performance lever. +if [ -f "$STAGE/usr/bin/punktfunk-host" ]; then + if setcap 'cap_sys_nice=ep' "$STAGE/usr/bin/punktfunk-host" 2>/dev/null; then + echo "granted CAP_SYS_NICE to usr/bin/punktfunk-host (GPU-priority lever active)" + else + echo "WARNING: could not setcap CAP_SYS_NICE (need root/CAP_SETFCAP) — the image will ship" >&2 + echo " without it and PyroWave will encode at default GPU priority." >&2 + fi +fi + # SELinux labels as pseudo-xattrs (see header). matchpathcon resolves each target path against # the targeted policy's file_contexts; <> means "no specific entry" — skip those (the # handful of matches all resolve to real contexts for our payload). diff --git a/packaging/debian/build-deb.sh b/packaging/debian/build-deb.sh index 8ebdaac5..0485d930 100755 --- a/packaging/debian/build-deb.sh +++ b/packaging/debian/build-deb.sh @@ -294,6 +294,16 @@ if [ "$1" = "configure" ]; then # primitive that must not ride on the group users are told to join for gamepads # (security-review 2026-08-05 M-4). getent group punktfunk >/dev/null 2>&1 || addgroup --system punktfunk 2>/dev/null || true + # CAP_SYS_NICE — the GPU-scheduling grant. PyroWave encodes on the shader cores a game + # saturates, and the driver gates the elevated global-priority Vulkan queue that fixes it on + # this capability: measured 2026-08-08 on an RTX 5070 Ti, WITHOUT it every priority class is + # refused and WITH it the encoder is granted REALTIME first try (RADV behaves the same). + # Without this line the knob exists and does nothing. Narrow: it permits raising scheduling + # priority only — no filesystem, network or user-switching privilege, and no setuid. Note a + # capability-carrying binary is AT_SECURE, so the loader ignores LD_LIBRARY_PATH/LD_PRELOAD for + # it and core dumps are suppressed by default. Best-effort: a box without libcap, or a + # filesystem that cannot store capabilities, just runs at default priority as before. + setcap 'cap_sys_nice=ep' /usr/bin/punktfunk-host 2>/dev/null || true # Pick up the /dev/uinput rule without a reboot (best-effort, no-op in containers). udevadm control --reload-rules 2>/dev/null || true udevadm trigger --subsystem-match=misc 2>/dev/null || true diff --git a/packaging/nix/nixos-module.nix b/packaging/nix/nixos-module.nix index fe684878..08d05d96 100644 --- a/packaging/nix/nixos-module.nix +++ b/packaging/nix/nixos-module.nix @@ -340,6 +340,26 @@ in allowedUDPPorts = nativeUDP ++ optionals cfg.host.gamestream gamestreamUDP; }; + # CAP_SYS_NICE — the GPU-scheduling grant. PyroWave encodes on the GPU shader cores a game + # saturates; the elevated global-priority Vulkan queue that fixes it is gated on this + # capability (measured 2026-08-08, RTX 5070 Ti: without it EVERY priority class is refused, + # with it the encoder gets REALTIME on the first attempt; RADV behaves the same). + # + # NixOS cannot `setcap` a store path — it is read-only and shared — so this goes through + # `security.wrappers`, which builds a small setcap'd wrapper in /run/wrappers/bin. The unit's + # ExecStart points at the wrapper below; everything else about the host is unchanged. + # + # Narrow: CAP_SYS_NICE permits raising scheduling priority only — no filesystem, network or + # user-switching privilege, and the wrapper is capability-based, NOT setuid. Two side effects + # to know: the wrapped binary is AT_SECURE (the loader ignores LD_LIBRARY_PATH/LD_PRELOAD for + # it) and core dumps are suppressed by default. + security.wrappers.punktfunk-host = { + source = "${cfg.host.package}/bin/punktfunk-host"; + capabilities = "cap_sys_nice=ep"; + owner = "root"; + group = "root"; + }; + systemd.user.services.punktfunk-host = { description = "punktfunk GameStream + punktfunk/1 streaming host"; documentation = [ "https://git.unom.io/unom/punktfunk" ]; @@ -358,8 +378,12 @@ in # PUNKTFUNK_GAMESCOPE_BIN so an operator's own override of that env still wins. ++ optional cfg.host.gamescopeHdr cfg.host.gamescopePackage; serviceConfig = { + # Through the wrapper (see `security.wrappers.punktfunk-host` above), NOT the store path + # directly — the store path carries no capability and the GPU-priority lever would be + # inert. `config.security.wrapperDir` rather than a hard-coded /run/wrappers/bin so an + # operator who has moved it is still correct. ExecStart = - "${cfg.host.package}/bin/punktfunk-host serve" + optionalString cfg.host.gamestream " --gamestream"; + "${config.security.wrapperDir}/punktfunk-host serve" + optionalString cfg.host.gamestream " --gamestream"; Restart = "on-failure"; RestartSec = 2; EnvironmentFile = diff --git a/packaging/rpm/punktfunk.spec b/packaging/rpm/punktfunk.spec index d4d3eea9..77361408 100644 --- a/packaging/rpm/punktfunk.spec +++ b/packaging/rpm/punktfunk.spec @@ -477,7 +477,15 @@ install -Dm0644 scripts/punktfunk-scripting.service %{buildroot}%{_userunitdir}/ %files %license LICENSE-MIT LICENSE-APACHE THIRD-PARTY-NOTICES.txt %doc README.md packaging/README.md -%{_bindir}/punktfunk-host +# CAP_SYS_NICE — the GPU-scheduling grant, declared the RPM-native way so rpm applies it at +# install, restores it on upgrade, and VERIFIES it (a plain %post setcap does none of those). +# PyroWave encodes on the shader cores a game saturates; the elevated global-priority Vulkan queue +# that fixes it is gated on this capability. Measured 2026-08-08 on an RTX 5070 Ti: without it +# every priority class is refused, with it the encoder gets REALTIME first try (RADV the same). +# Narrow — scheduling priority only, no filesystem/network/user-switching privilege, not setuid. +# Consequences: the binary becomes AT_SECURE, so the loader ignores LD_LIBRARY_PATH/LD_PRELOAD for +# it, and core dumps are suppressed by default. +%caps(cap_sys_nice=ep) %{_bindir}/punktfunk-host %{_bindir}/punktfunk-tray %{_udevrulesdir}/60-punktfunk.rules %dir %{_libexecdir}/punktfunk diff --git a/scripts/steamdeck/install.sh b/scripts/steamdeck/install.sh index 622d972c..f94d460b 100755 --- a/scripts/steamdeck/install.sh +++ b/scripts/steamdeck/install.sh @@ -293,6 +293,25 @@ if [ "$SUDO_OK" = 1 ]; then NEED_RELOGIN=1 warn "added $USER to the 'input' group (applies on next login)" fi + # CAP_SYS_NICE on the host binary — the GPU-scheduling grant, and the Deck is the box that + # needs it most: a Van Gogh APU shares one small GPU between the game and PyroWave's encode + # dispatch. The driver gates the elevated global-priority Vulkan queue on this capability + # (measured 2026-08-08 on an RTX 5070 Ti: refused without it, granted REALTIME with it; RADV + # behaves the same), so without this the knob exists and does nothing. + # + # The binary lives under $HOME, not /usr — so unlike the /etc drop-ins above this survives a + # SteamOS A/B update on its own and needs no atomic-keep entry. It DOES need re-applying after + # every rebuild, because a fresh binary is a new inode; re-running this installer does that. + # + # Narrow (scheduling priority only, no filesystem/network privilege, not setuid) and + # best-effort — a failure just means the encode runs at default priority as it does today. + if [ -x "$BIN" ]; then + if sudo setcap 'cap_sys_nice=ep' "$BIN" 2>/dev/null; then + ok "granted CAP_SYS_NICE (PyroWave encode can outrank a GPU-bound game)" + else + warn "could not grant CAP_SYS_NICE to $BIN — PyroWave encode stays at default GPU priority" + fi + fi # SteamOS A/B updates rebuild /etc and DROP everything not on Valve's keep list — verified # live: an OS update stripped the udev rule + vhci autoload + UDP sysctl (gamepads silently # degrade to Xbox 360, buffers back to 208 KB). The sanctioned fix is a preserve drop-in in