fix(pyrowave-sys): vendor patch — size the encoder's payload_data staging for 4:4:4

Upstream sizes the raw quantized-payload worst-case buffer at aligned_width*aligned_height*2 bytes: real headroom for 4:2:0's 1.5 samples/px, but half of what 4:4:4's 3 samples/px can produce. Busy 4:4:4 content overruns the buffer on the GPU and corrupts the adjacent meta/bucket suballocations — nondeterministic corrupt bitstreams and encoder crashes at ANY target bitrate (smooth content never trips it, which is why 4:2:0 and simple 4:4:4 both look fine). Found by the Phase-0 measurement matrix for design/pyrowave-444-hdr.md; fix validated alone via upstream's own devel tools on the RTX 5070 Ti: deterministic byte-identical outputs across runs, 1080p + 4K, 8- and 16-bit, PSNR at the expected operating points. Patch lives in crates/pyrowave-sys/patches/ and vendor-pyrowave.sh now re-applies patches on re-vendor; PUNKTFUNK-VENDOR.txt records it. Upstream report to follow. No wire/ABI change; 4:2:0 sizing unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 12:16:56 +02:00
parent 94533bb071
commit 97cf15e3b7
4 changed files with 50 additions and 2 deletions
@@ -0,0 +1,20 @@
+diff --git a/crates/pyrowave-sys/vendor/pyrowave/pyrowave_encoder.cpp b/crates/pyrowave-sys/vendor/pyrowave/pyrowave_encoder.cpp
+index f5ac6dcc..ad4e9746 100644
+--- a/crates/pyrowave-sys/vendor/pyrowave/pyrowave_encoder.cpp
+++ b/crates/pyrowave-sys/vendor/pyrowave/pyrowave_encoder.cpp
+@@ -187,8 +187,13 @@ void Encoder::Impl::init_block_meta()
+ 	meta_buffer = device->create_buffer(info);
+ 	device->set_name(*meta_buffer, "meta-buffer");
+ 
+-	// Worst case estimate.
+-	info.size = aligned_width * aligned_height * 2;
+	// Worst case estimate. PUNKTFUNK PATCH (patches/0001-payload-data-444-sizing.patch):
+	// 4:4:4 carries 3 samples per pixel vs 4:2:0's 1.5 — the same per-sample headroom
+	// needs twice the bytes, or busy 4:4:4 content overruns this buffer on the GPU and
+	// corrupts the adjacent meta/bucket allocations (nondeterministic bad bitstreams and
+	// crashes at ANY target bitrate). Validated on RTX 5070 Ti @1080p/4K, 8/16-bit.
+	info.size = VkDeviceSize(aligned_width) * aligned_height *
+	            (chroma == ChromaSubsampling::Chroma444 ? 4 : 2);
+ 	payload_data = device->create_buffer(info);
+ 	device->set_name(*payload_data, "payload-data");
+ 
@@ -8,3 +8,10 @@ vulkan-headers:  015e25c3c91b70eb1a754d36fb14c4ba6ad9b0b9
 Tree is pruned to what the pyrowave-sys standalone build needs
 (see the rm -rf list in the script). All parts are MIT-licensed
 (pyrowave, Granite) or Apache-2.0/MIT (volk, Vulkan-Headers).
+
+Local patches (crates/pyrowave-sys/patches/, re-applied on re-vendor):
+  0001-payload-data-444-sizing.patch — encoder payload_data worst-case buffer
+    was sized for 4:2:0's 1.5 samples/px; busy 4:4:4 (3 samples/px) overran it
+    on the GPU → nondeterministic corrupt bitstreams/crashes at any bitrate.
+    Found + validated 2026-07-18 (RTX 5070 Ti, 1080p/4K, 8/16-bit); to be
+    reported upstream.
@@ -187,8 +187,13 @@ void Encoder::Impl::init_block_meta()
 	meta_buffer = device->create_buffer(info);
 	device->set_name(*meta_buffer, "meta-buffer");

-	// Worst case estimate.
-	info.size = aligned_width * aligned_height * 2;
+	// Worst case estimate. PUNKTFUNK PATCH (patches/0001-payload-data-444-sizing.patch):
+	// 4:4:4 carries 3 samples per pixel vs 4:2:0's 1.5 — the same per-sample headroom
+	// needs twice the bytes, or busy 4:4:4 content overruns this buffer on the GPU and
+	// corrupts the adjacent meta/bucket allocations (nondeterministic bad bitstreams and
+	// crashes at ANY target bitrate). Validated on RTX 5070 Ti @1080p/4K, 8/16-bit.
+	info.size = VkDeviceSize(aligned_width) * aligned_height *
+	            (chroma == ChromaSubsampling::Chroma444 ? 4 : 2);
 	payload_data = device->create_buffer(info);
 	device->set_name(*payload_data, "payload-data");

@@ -55,6 +55,15 @@ mkdir -p "$(dirname "$DEST")"
 rm -rf "$DEST"
 cp -a "$WORK/pyrowave" "$DEST"

+# Local patches on top of the pin (crates/pyrowave-sys/patches/*.patch, applied
+# in order). Each patch documents its upstream status; drop it when a vendor
+# bump includes the fix.
+for p in "$REPO_ROOT"/crates/pyrowave-sys/patches/*.patch; do
+  [ -e "$p" ] || continue
+  git -C "$REPO_ROOT" apply "$p"
+  echo "applied $(basename "$p")"
+done
+
 cat > "$DEST/PUNKTFUNK-VENDOR.txt" <<EOF
 Vendored by scripts/vendor-pyrowave.sh — do not edit by hand.

@@ -66,6 +75,13 @@ vulkan-headers:  $VKHDR_COMMIT
 Tree is pruned to what the pyrowave-sys standalone build needs
 (see the rm -rf list in the script). All parts are MIT-licensed
 (pyrowave, Granite) or Apache-2.0/MIT (volk, Vulkan-Headers).
+
+Local patches (crates/pyrowave-sys/patches/, re-applied on re-vendor):
+  0001-payload-data-444-sizing.patch — encoder payload_data worst-case buffer
+    was sized for 4:2:0's 1.5 samples/px; busy 4:4:4 (3 samples/px) overran it
+    on the GPU → nondeterministic corrupt bitstreams/crashes at any bitrate.
+    Found + validated 2026-07-18 (RTX 5070 Ti, 1080p/4K, 8/16-bit); to be
+    reported upstream.
 EOF

 echo "Vendored pyrowave@${PYROWAVE_COMMIT:0:12} (Granite ${GRANITE_COMMIT:0:12}) into $DEST"