feat(client): PyroWave decode backend on the presenter's device (Phase 2b, part 1)
ci / web (push) Successful in 55s
ci / docs-site (push) Successful in 1m2s
ci / rust (push) Failing after 4m1s
decky / build-publish (push) Successful in 18s
arch / build-publish (push) Failing after 4m45s
docker / build-push (., web/Dockerfile, punktfunk-web) (push) Successful in 9s
docker / build-push (--build-arg FEDORA_VERSION=44, ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora44-rpm) (push) Successful in 9s
docker / build-push (ci, ci/fedora-rpm.Dockerfile, punktfunk-fedora-rpm) (push) Successful in 8s
docker / build-push (ci, ci/rust-ci.Dockerfile, punktfunk-rust-ci) (push) Successful in 8s
docker / build-push (docs-site, docs-site/Dockerfile, punktfunk-docs) (push) Successful in 7s
deb / build-publish (push) Failing after 4m46s
ci / bench (push) Successful in 5m12s
rpm / build-publish (44, fedora-44, punktfunk-fedora44-rpm) (push) Failing after 3m47s
rpm / build-publish (43, bazzite, punktfunk-fedora-rpm) (push) Failing after 4m39s
windows-msix / package (arm64, C:\Users\Public\ffmpeg-arm64, --no-default-features, aarch64-pc-windows-msvc, C:\t-a64) (push) Successful in 3m41s
flatpak / build-publish (push) Successful in 6m20s
android / android (push) Successful in 13m47s
docker / deploy-docs (push) Successful in 28s
windows-msix / package (x64, C:\Users\Public\ffmpeg, , x86_64-pc-windows-msvc, C:\t) (push) Successful in 3m59s
windows / build (aarch64-pc-windows-msvc) (push) Successful in 4m55s
windows / build (x86_64-pc-windows-msvc) (push) Successful in 6m50s
apple / swift (push) Successful in 4m45s
apple / screenshots (push) Successful in 20m33s

The presenter's device creation now probes + enables the PyroWave
compute feature set alongside the Vulkan Video probe (shaderInt16,
storageBuffer8BitAccess, subgroup size control — gated on support,
harmless when unused) and exports the facts through VulkanDecodeDevice
(pyrowave_decode capability + feature bools + apiVersion + the queue-
family shape).

pf-client-core (feature `pyrowave`, Linux): video_pyrowave.rs — the
decoder runs pyrowave compute on the PRESENTER's own VkDevice, zero
interop (plan §4.5): pinned content-equivalent create-info
reconstruction satisfies pyrowave 0.4.0's lifetime rule without
refactoring the presenter's creation; queue access rides the existing
device-wide QueueLock (the FFmpeg/Skia contract); decode records into
our command buffer, fence-synchronous (sub-ms), into a 4-deep ring of
3xR8 plane sets (decode REQUIRES storage usage + identity swizzles, so
the encoder's RG8 trick doesn't apply). Backend::PyroWave +
DecodedImage::PyroWave + Decoder::new_pyrowave + decodable_codecs_for
(advertisement gated on the device probe) wired through the decode
dispatch; no demote ladder (nothing else decodes it — fallback is
session renegotiation, plan §4.6).

Still to come for a live session: the presenter's planar-CSC render
path for the new variant, pump/shell opt-in (preferred_codec) wiring,
and the on-glass .21 run.

Validated on .21: pf-client-core + pf-presenter compile with and
without the feature, clippy clean, 26 client-core tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-15 01:33:38 +02:00
parent 49ba1cd11b
commit 575975687c
6 changed files with 684 additions and 4 deletions
+74
View File
@@ -68,6 +68,11 @@ pub enum DecodedImage {
/// (Intel's Windows driver foremost). See `crate::video_d3d11`.
#[cfg(windows)]
D3d11(crate::video_d3d11::D3d11Frame),
/// PyroWave planar output: three R8 plane views on the presenter's own device,
/// decode already fence-complete, GENERAL layout — the presenter's planar CSC
/// samples them directly (BT.709 limited, the codec's fixed colour contract).
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
PyroWave(crate::video_pyrowave::PyroWavePlanarFrame),
}
/// One Vulkan-decoded frame. The image lives on the presenter's own VkDevice (the
@@ -183,6 +188,8 @@ impl DecodedImage {
DecodedImage::VkFrame(f) => f.keyframe,
#[cfg(windows)]
DecodedImage::D3d11(f) => f.keyframe,
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
DecodedImage::PyroWave(f) => f.keyframe,
}
}
@@ -197,6 +204,8 @@ impl DecodedImage {
DecodedImage::VkFrame(f) => (f.width, f.height),
#[cfg(windows)]
DecodedImage::D3d11(f) => (f.width, f.height),
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
DecodedImage::PyroWave(f) => (f.width, f.height),
}
}
}
@@ -312,6 +321,10 @@ enum Backend {
Vaapi(VaapiDecoder),
#[cfg(windows)]
D3d11va(crate::video_d3d11::D3d11vaDecoder),
/// PyroWave (wired-LAN wavelet codec): pyrowave compute on the presenter's device,
/// no FFmpeg involvement. No demotion rung — there is no other decoder for it.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
PyroWave(crate::video_pyrowave::PyroWaveDecoder),
Software(SoftwareDecoder),
}
@@ -360,6 +373,21 @@ pub fn decodable_codecs() -> u8 {
bits
}
/// [`decodable_codecs`] plus the PyroWave bit when the presenter's device passed the
/// compute-feature probe. Advertisement-only: `resolve_codec` never auto-picks PyroWave —
/// the session must also name it `preferred_codec` (plan §3), which the client does only
/// under its explicit opt-in.
pub fn decodable_codecs_for(vk: Option<&VulkanDecodeDevice>) -> u8 {
let bits = decodable_codecs();
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
if vk.map(|v| v.pyrowave_decode).unwrap_or(false) {
return bits | punktfunk_core::quic::CODEC_PYROWAVE;
}
#[cfg(not(all(target_os = "linux", feature = "pyrowave")))]
let _ = vk;
bits
}
/// libavcodec logs reference-frame recovery to the process stderr very verbosely
/// (`First slice in a frame missing`, `Could not find ref with POC …`, `Error
/// constructing the frame RPS`) — normal chatter while the decoder waits for a keyframe
@@ -539,6 +567,21 @@ impl Decoder {
/// Drain the "please ask the host for an IDR" flag — the pump calls this each iteration
/// (throttled) so a demoted/erroring decoder can resynchronize under the infinite GOP.
/// Open a PyroWave decoder for a `CODEC_PYROWAVE` session (plan §4.5): pyrowave
/// compute on the presenter's device, no FFmpeg. `codec_id` is irrelevant (kept as
/// HEVC so an — impossible — demotion path stays well-formed).
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
pub fn new_pyrowave(vk: &VulkanDecodeDevice, width: u32, height: u32) -> Result<Decoder> {
Ok(Decoder {
backend: Backend::PyroWave(crate::video_pyrowave::PyroWaveDecoder::new(
vk, width, height,
)?),
codec_id: ffmpeg::codec::Id::HEVC,
vaapi_fails: 0,
want_keyframe: false,
})
}
pub fn take_keyframe_request(&mut self) -> bool {
std::mem::take(&mut self.want_keyframe)
}
@@ -572,6 +615,11 @@ impl Decoder {
Backend::Vaapi(v) => v.decode(au).map(|f| f.map(DecodedImage::Dmabuf)),
#[cfg(windows)]
Backend::D3d11va(d) => d.decode(au).map(|f| f.map(DecodedImage::D3d11)),
// No demote ladder below PyroWave (nothing else decodes it): propagate the
// error; the pump surfaces it and the session falls back to HEVC by
// renegotiation (plan §4.6), not by decoder swap.
#[cfg(all(target_os = "linux", feature = "pyrowave"))]
Backend::PyroWave(p) => return Ok(p.decode(au)?.map(DecodedImage::PyroWave)),
Backend::Software(s) => return Ok(s.decode(au)?.map(DecodedImage::Cpu)),
};
match result {
@@ -1077,6 +1125,24 @@ pub struct VulkanDecodeDevice {
/// features). The bundle now exists even without it — Windows D3D11 interop rides the
/// same struct — so consumers gate the FFmpeg-Vulkan decoder on THIS, not on `Some`.
pub video_decode: bool,
/// PyroWave decode (the wired-LAN wavelet codec) is usable: Vulkan 1.3 + the compute
/// features its kernels need were present AND enabled at device creation
/// (`shaderInt16`, `storageBuffer8BitAccess`, subgroup size control). Gates the
/// `CODEC_PYROWAVE` advertisement and the pyrowave decoder backend.
pub pyrowave_decode: bool,
/// The feature facts + creation shape the pyrowave decoder's pinned create-info
/// reconstruction mirrors (pyrowave 0.4.0 requires the instance/device create infos —
/// content-accurate, kept alive — to share our VkDevice).
pub f_shader_int16: bool,
pub f_storage_buffer8: bool,
pub f_subgroup_size_control: bool,
pub f_compute_full_subgroups: bool,
pub f_shader_float16: bool,
/// `VkPhysicalDeviceProperties::apiVersion` of the presenter's device.
pub api_version: u32,
/// The queue families the device was created with (one `VkDeviceQueueCreateInfo` each,
/// one queue per family, priority 1.0) — mirrored by the reconstruction.
pub queue_families: Vec<u32>,
/// The presenter enabled `VK_KHR_external_memory_win32` + `VK_KHR_win32_keyed_mutex`:
/// D3D11 shared-texture frames can reach the screen. Always `false` off Windows.
pub d3d11_import: bool,
@@ -1598,6 +1664,14 @@ mod tests {
f_sampler_ycbcr: true,
f_timeline_semaphore: true,
f_synchronization2: true,
f_shader_int16: false,
f_storage_buffer8: false,
f_subgroup_size_control: false,
f_compute_full_subgroups: false,
f_shader_float16: false,
api_version: 0,
queue_families: Vec::new(),
pyrowave_decode: false,
video_decode: true,
d3d11_import: false,
adapter_luid: None,