diff --git a/docs-site/content/docs/configuration.md b/docs-site/content/docs/configuration.md index e1f00f01..72b4dbd5 100644 --- a/docs-site/content/docs/configuration.md +++ b/docs-site/content/docs/configuration.md @@ -240,7 +240,7 @@ notes for context. | `PUNKTFUNK_NVENC_SPLIT_ARBITRATE` | `1` | Opt-in: let the host change its split-encode decision **live**, mid-session, as the pixel rate moves, instead of only choosing once at session start. Currently wired on the Linux direct-NVENC path. Only interesting alongside `PUNKTFUNK_SPLIT_ENCODE=auto` at very high pixel rates. | | `PUNKTFUNK_GPU_PRIORITY_CLASS` | `off` · `normal` · `high` · `realtime` · `auto` | **(Windows)** GPU scheduling priority for capture/encode under a GPU-saturating game. Default `auto` (starts `high`, upgrades to `realtime` when it's safe — e.g. HAGS off); `high` pins the static pre-gate behaviour; `realtime` is the strongest lever but can freeze NVENC on some setups. | | `PUNKTFUNK_IDD_DEPTH` | `N` (default `2`) | **(Windows)** IDD-push pipeline depth. `1` cuts latency once GPU priority is raised; higher smooths a contended GPU. | -| `PYROWAVE_QUEUE_PRIORITY` | `realtime` *(default)* · `high` · `off` | [PyroWave](/docs/pyrowave) sessions only. PyroWave encodes on the same GPU shader cores a game uses, so a demanding game can starve it and the frame rate drops. This asks the driver to schedule the encode ahead of the game. `realtime` tries the strongest class and falls back to `high`; `high` asks only for the middle one; `off` disables the request. A driver that refuses simply encodes at normal priority — it can never stop a session starting. **Today most Linux hosts are refused every class**, because the drivers require the `CAP_SYS_NICE` capability and the packaged host does not carry it yet — so this setting currently has no effect on them and the host says so once, at session start. Set `off` to silence that, or if you see the desktop stutter while streaming. | +| `PYROWAVE_QUEUE_PRIORITY` | `realtime` *(default)* · `high` · `off` | [PyroWave](/docs/pyrowave) sessions only. PyroWave encodes on the same GPU shader cores a game uses, so a demanding game can starve it and the frame rate drops. This asks the driver to schedule the encode ahead of the game. `realtime` tries the strongest class and falls back to `high`; `high` asks only for the middle one; `off` disables the request. A driver that refuses simply encodes at normal priority — it can never stop a session starting. The packages grant the host the `CAP_SYS_NICE` capability this needs; on a host built or installed by hand it will be refused, and the host says so once at session start. Set `off` if you see the desktop stutter while streaming. | ## Diagnostics diff --git a/docs-site/content/docs/running-as-a-service.md b/docs-site/content/docs/running-as-a-service.md index 750b6857..f510e8fd 100644 --- a/docs-site/content/docs/running-as-a-service.md +++ b/docs-site/content/docs/running-as-a-service.md @@ -205,6 +205,36 @@ the host. If the host answers, it's up. If not, check `journalctl --user -u punktfunk-host` on the host — on a Windows host, run `punktfunk-host service status` from an elevated prompt on the machine itself. +## GPU scheduling priority + +The Linux packages give the host binary one Linux capability, `CAP_SYS_NICE`, and it is worth +knowing why it is there and how to take it away. + +The [PyroWave](/docs/pyrowave) codec encodes on the same GPU shader cores your game is using, so a +demanding game can crowd it out and the stream's frame rate drops with it. The fix is to ask the +driver to schedule the encode ahead of the game, and every driver we tested gates that request on +this capability: without it the request is simply refused and nothing changes. The other codecs use +a separate video engine on the GPU and are unaffected either way. + +`CAP_SYS_NICE` lets a process raise its own scheduling priority. It grants no access to files, +the network or other users' processes, and it is **not** the same as running as root — the host +still runs as you, under your user session. + +To check, or to take it away: + +```sh +getcap /usr/bin/punktfunk-host # shows cap_sys_nice=ep when granted +sudo setcap -r /usr/bin/punktfunk-host # remove it; streaming still works +``` + +Removing it costs you nothing unless you stream PyroWave, and you can also just set +`PYROWAVE_QUEUE_PRIORITY=off` to stop the host asking. Note that a package update replaces the +binary and re-applies the capability. + +Two side effects, if you are debugging the host: a binary carrying a capability is treated as +security-sensitive by the dynamic loader, so `LD_LIBRARY_PATH` and `LD_PRELOAD` are ignored for it, +and it does not write core dumps by default. + ## Stopping and removing After a Linux package update the user service keeps running the old binary until it's restarted, and diff --git a/packaging/arch/punktfunk-host.install b/packaging/arch/punktfunk-host.install index 07af7c24..ec20beb3 100644 --- a/packaging/arch/punktfunk-host.install +++ b/packaging/arch/punktfunk-host.install @@ -12,9 +12,33 @@ _ensure_punktfunk_group() { getent group punktfunk >/dev/null 2>&1 || groupadd --system punktfunk 2>/dev/null || true } +# CAP_SYS_NICE on the host binary — the GPU-scheduling grant. +# +# WHY: PyroWave encodes on the GPU's shader cores, so a GPU-bound game starves it (measured: the +# encode dispatch goes from ~2 ms to 15-18 ms at 95 % game load). The fix is an elevated +# global-priority Vulkan queue, which the driver gates on CAP_SYS_NICE — measured 2026-08-08 on an +# RTX 5070 Ti: WITHOUT the capability every priority class is refused, WITH it the encoder is +# granted REALTIME on the first attempt. RADV is the same. Without this line the knob exists and +# does nothing. Same capability, same mechanism, as our gamescope package sets on its own binary. +# +# NARROW: CAP_SYS_NICE only permits raising scheduling priority (nice/ioprio/affinity/RT class). It +# grants no filesystem, network or user-switching privilege, and it is NOT setuid. +# +# TWO CONSEQUENCES worth knowing before you debug something odd on this host: +# * a file capability makes the process AT_SECURE, so the dynamic loader IGNORES LD_LIBRARY_PATH +# and LD_PRELOAD for it. A library-path shim that used to work will silently stop. +# * core dumps are suppressed for capability-carrying binaries by default (fs.suid_dumpable). +# +# Never fails the install: a box without libcap, or a filesystem that cannot store capabilities +# (some overlay/NFS setups), just runs at default priority exactly as before. +_grant_sched_capability() { + setcap 'cap_sys_nice=ep' usr/bin/punktfunk-host 2>/dev/null || true +} + post_install() { _ensure_update_group _ensure_punktfunk_group + _grant_sched_capability udevadm control --reload-rules 2>/dev/null || true udevadm trigger --subsystem-match=misc 2>/dev/null || true # Apply the UDP socket-buffer tuning now (also auto-applied at boot by systemd-sysctl). @@ -67,6 +91,8 @@ MSG post_upgrade() { _ensure_update_group + # A replaced binary is a NEW inode — file capabilities do not survive the upgrade, so re-grant. + _grant_sched_capability udevadm control --reload-rules 2>/dev/null || true sysctl -p /usr/lib/sysctl.d/99-punktfunk-net.conf >/dev/null 2>&1 || true _warn_stale_firewall_ports diff --git a/packaging/bazzite/build-sysext.sh b/packaging/bazzite/build-sysext.sh index c88acd62..32c2861b 100644 --- a/packaging/bazzite/build-sysext.sh +++ b/packaging/bazzite/build-sysext.sh @@ -130,6 +130,31 @@ SYSEXT_VERSION_ID=$PF_VR EXTENSION_RELOAD_MANAGER=1 EOF +# CAP_SYS_NICE on the host binary — the GPU-scheduling grant. PyroWave encodes on the GPU shader +# cores a game saturates, and the driver gates the elevated global-priority Vulkan queue that fixes +# it on this capability (measured 2026-08-08 on an RTX 5070 Ti: refused without it, granted REALTIME +# with it; RADV the same). Narrow — scheduling priority only, no filesystem/network privilege, not +# setuid. +# +# It has to be applied HERE, not in the merge hook: a merged sysext's /usr is a read-only squashfs, +# so nothing can setcap it afterwards. And it cannot ride in from the RPM either — the spec declares +# it with %caps, but rpm stores capabilities in its own header and `rpm2cpio | cpio` carries only +# the payload, so the staged file arrives with no capability at all. mksquashfs DOES record +# security.capability (only security.selinux is excluded below), so a setcap on the staging tree is +# what ends up in the image. +# +# Needs CAP_SETFCAP, i.e. root (or fakeroot) — a plain-user CI build cannot do it. That is not fatal: +# the image just ships as it does today and the encode runs at default GPU priority, so warn and +# carry on rather than fail a release build over a performance lever. +if [ -f "$STAGE/usr/bin/punktfunk-host" ]; then + if setcap 'cap_sys_nice=ep' "$STAGE/usr/bin/punktfunk-host" 2>/dev/null; then + echo "granted CAP_SYS_NICE to usr/bin/punktfunk-host (GPU-priority lever active)" + else + echo "WARNING: could not setcap CAP_SYS_NICE (need root/CAP_SETFCAP) — the image will ship" >&2 + echo " without it and PyroWave will encode at default GPU priority." >&2 + fi +fi + # SELinux labels as pseudo-xattrs (see header). matchpathcon resolves each target path against # the targeted policy's file_contexts; <> means "no specific entry" — skip those (the # handful of matches all resolve to real contexts for our payload). diff --git a/packaging/debian/build-deb.sh b/packaging/debian/build-deb.sh index 8ebdaac5..0485d930 100755 --- a/packaging/debian/build-deb.sh +++ b/packaging/debian/build-deb.sh @@ -294,6 +294,16 @@ if [ "$1" = "configure" ]; then # primitive that must not ride on the group users are told to join for gamepads # (security-review 2026-08-05 M-4). getent group punktfunk >/dev/null 2>&1 || addgroup --system punktfunk 2>/dev/null || true + # CAP_SYS_NICE — the GPU-scheduling grant. PyroWave encodes on the shader cores a game + # saturates, and the driver gates the elevated global-priority Vulkan queue that fixes it on + # this capability: measured 2026-08-08 on an RTX 5070 Ti, WITHOUT it every priority class is + # refused and WITH it the encoder is granted REALTIME first try (RADV behaves the same). + # Without this line the knob exists and does nothing. Narrow: it permits raising scheduling + # priority only — no filesystem, network or user-switching privilege, and no setuid. Note a + # capability-carrying binary is AT_SECURE, so the loader ignores LD_LIBRARY_PATH/LD_PRELOAD for + # it and core dumps are suppressed by default. Best-effort: a box without libcap, or a + # filesystem that cannot store capabilities, just runs at default priority as before. + setcap 'cap_sys_nice=ep' /usr/bin/punktfunk-host 2>/dev/null || true # Pick up the /dev/uinput rule without a reboot (best-effort, no-op in containers). udevadm control --reload-rules 2>/dev/null || true udevadm trigger --subsystem-match=misc 2>/dev/null || true diff --git a/packaging/nix/nixos-module.nix b/packaging/nix/nixos-module.nix index fe684878..08d05d96 100644 --- a/packaging/nix/nixos-module.nix +++ b/packaging/nix/nixos-module.nix @@ -340,6 +340,26 @@ in allowedUDPPorts = nativeUDP ++ optionals cfg.host.gamestream gamestreamUDP; }; + # CAP_SYS_NICE — the GPU-scheduling grant. PyroWave encodes on the GPU shader cores a game + # saturates; the elevated global-priority Vulkan queue that fixes it is gated on this + # capability (measured 2026-08-08, RTX 5070 Ti: without it EVERY priority class is refused, + # with it the encoder gets REALTIME on the first attempt; RADV behaves the same). + # + # NixOS cannot `setcap` a store path — it is read-only and shared — so this goes through + # `security.wrappers`, which builds a small setcap'd wrapper in /run/wrappers/bin. The unit's + # ExecStart points at the wrapper below; everything else about the host is unchanged. + # + # Narrow: CAP_SYS_NICE permits raising scheduling priority only — no filesystem, network or + # user-switching privilege, and the wrapper is capability-based, NOT setuid. Two side effects + # to know: the wrapped binary is AT_SECURE (the loader ignores LD_LIBRARY_PATH/LD_PRELOAD for + # it) and core dumps are suppressed by default. + security.wrappers.punktfunk-host = { + source = "${cfg.host.package}/bin/punktfunk-host"; + capabilities = "cap_sys_nice=ep"; + owner = "root"; + group = "root"; + }; + systemd.user.services.punktfunk-host = { description = "punktfunk GameStream + punktfunk/1 streaming host"; documentation = [ "https://git.unom.io/unom/punktfunk" ]; @@ -358,8 +378,12 @@ in # PUNKTFUNK_GAMESCOPE_BIN so an operator's own override of that env still wins. ++ optional cfg.host.gamescopeHdr cfg.host.gamescopePackage; serviceConfig = { + # Through the wrapper (see `security.wrappers.punktfunk-host` above), NOT the store path + # directly — the store path carries no capability and the GPU-priority lever would be + # inert. `config.security.wrapperDir` rather than a hard-coded /run/wrappers/bin so an + # operator who has moved it is still correct. ExecStart = - "${cfg.host.package}/bin/punktfunk-host serve" + optionalString cfg.host.gamestream " --gamestream"; + "${config.security.wrapperDir}/punktfunk-host serve" + optionalString cfg.host.gamestream " --gamestream"; Restart = "on-failure"; RestartSec = 2; EnvironmentFile = diff --git a/packaging/rpm/punktfunk.spec b/packaging/rpm/punktfunk.spec index d4d3eea9..77361408 100644 --- a/packaging/rpm/punktfunk.spec +++ b/packaging/rpm/punktfunk.spec @@ -477,7 +477,15 @@ install -Dm0644 scripts/punktfunk-scripting.service %{buildroot}%{_userunitdir}/ %files %license LICENSE-MIT LICENSE-APACHE THIRD-PARTY-NOTICES.txt %doc README.md packaging/README.md -%{_bindir}/punktfunk-host +# CAP_SYS_NICE — the GPU-scheduling grant, declared the RPM-native way so rpm applies it at +# install, restores it on upgrade, and VERIFIES it (a plain %post setcap does none of those). +# PyroWave encodes on the shader cores a game saturates; the elevated global-priority Vulkan queue +# that fixes it is gated on this capability. Measured 2026-08-08 on an RTX 5070 Ti: without it +# every priority class is refused, with it the encoder gets REALTIME first try (RADV the same). +# Narrow — scheduling priority only, no filesystem/network/user-switching privilege, not setuid. +# Consequences: the binary becomes AT_SECURE, so the loader ignores LD_LIBRARY_PATH/LD_PRELOAD for +# it, and core dumps are suppressed by default. +%caps(cap_sys_nice=ep) %{_bindir}/punktfunk-host %{_bindir}/punktfunk-tray %{_udevrulesdir}/60-punktfunk.rules %dir %{_libexecdir}/punktfunk diff --git a/scripts/steamdeck/install.sh b/scripts/steamdeck/install.sh index 622d972c..f94d460b 100755 --- a/scripts/steamdeck/install.sh +++ b/scripts/steamdeck/install.sh @@ -293,6 +293,25 @@ if [ "$SUDO_OK" = 1 ]; then NEED_RELOGIN=1 warn "added $USER to the 'input' group (applies on next login)" fi + # CAP_SYS_NICE on the host binary — the GPU-scheduling grant, and the Deck is the box that + # needs it most: a Van Gogh APU shares one small GPU between the game and PyroWave's encode + # dispatch. The driver gates the elevated global-priority Vulkan queue on this capability + # (measured 2026-08-08 on an RTX 5070 Ti: refused without it, granted REALTIME with it; RADV + # behaves the same), so without this the knob exists and does nothing. + # + # The binary lives under $HOME, not /usr — so unlike the /etc drop-ins above this survives a + # SteamOS A/B update on its own and needs no atomic-keep entry. It DOES need re-applying after + # every rebuild, because a fresh binary is a new inode; re-running this installer does that. + # + # Narrow (scheduling priority only, no filesystem/network privilege, not setuid) and + # best-effort — a failure just means the encode runs at default priority as it does today. + if [ -x "$BIN" ]; then + if sudo setcap 'cap_sys_nice=ep' "$BIN" 2>/dev/null; then + ok "granted CAP_SYS_NICE (PyroWave encode can outrank a GPU-bound game)" + else + warn "could not grant CAP_SYS_NICE to $BIN — PyroWave encode stays at default GPU priority" + fi + fi # SteamOS A/B updates rebuild /etc and DROP everything not on Valve's keep list — verified # live: an OS update stripped the udev rule + vhci autoload + UDP sysctl (gamepads silently # degrade to Xbox 360, buffers back to 208 KB). The sanctioned fix is a preserve drop-in in