fix(packaging/bazzite): the feed publisher signed a redirect page, not the manifest

Every Bazzite install on the stable channel has been refusing the feed:

  !! the feed's SHA256SUMS is NOT signed by packages@unom.io (AF245C506F4E4763).

The client was right and the feed was wrong. The registry answers a file GET
with a 303 See Other pointing at presigned object storage, and `curl -f` does
not treat a 3xx as an error — so the publisher's two un-`-L`'d manifest reads
"succeeded" holding the redirect's HTML body, `<a href="…">See Other</a>.`, and
handed that to callers as the manifest.

That broke both of them:

  * --seal signed the HTML page. Its presigned URL is regenerated per request
    and expires 300s later, so the published .asc covers bytes that exist
    nowhere and can never verify. Stable feeds only publish on a tag, so they
    are only ever sealed — f43 and f44 were re-broken at 19:54Z on 2026-07-29
    by the seal step of a canary run, and every canary push re-broke them.
    f43-canary survived by accident: on the publish path the same polluted
    bytes get both signed and uploaded, so it is at least self-consistent.

  * the publish merge read took the page for the previous manifest, and
    `grep -v " $FNAME$"` kept it — so each publish prepended a stale redirect
    page and dropped every prior image line. f43 runs KEEP=0 (keep all) and
    holds exactly one line, with 0.20.0 and 0.19.2 still in the registry and
    no longer listed. This has been quietly eating feed history since long
    before signing existed; nobody noticed because the client's latest() only
    matches ^punktfunk-.*-x86-64\.raw$, so an HTML line is invisible to it.

Both reads now go through one read_manifest that follows redirects AND keeps
only well-formed "<sha256>  <filename>" lines, so a manifest is never again
whatever the transport happened to return. --seal re-publishes a manifest that
normalizing changed, since the signature has to cover the bytes a client
downloads — that is what repairs the live feeds — and refuses outright when a
manifest lists no images, rather than sealing an empty feed that would read as
"up to date" to `punktfunk-sysext update`. The publish path now logs its
carry-over count; silence there is what let the history loss run for months.

Verified end to end against a fake registry that 303s to a fresh URL per
request, with a throwaway key and the real scripts: HEAD reproduces the
injected page, the lost image line and the client's VERIFY-FAIL; the fix keeps
both images and verifies; and the new --seal over the broken state normalizes
the manifest and turns it back to VERIFY-OK.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

packaging/bazzite/publish-sysext-feed.sh

@@ -17,7 +17,9 @@
 #   <feed>   e.g. f43, f43-canary, f44 (Fedora major x channel)
 #   KEEP     newest images to keep in the feed; 0/unset-for-stable = keep all
 #   --seal   re-sign a feed's EXISTING manifest without publishing an image. For feeds published
-#            before signing existed, and after a key rotation. Idempotent.
+#            before signing existed, and after a key rotation. Idempotent. Also re-publishes the
+#            manifest if normalizing it changed anything, because the signature has to cover the
+#            bytes a client downloads.
 # Env: REGISTRY (git.unom.io), OWNER (unom), TOKEN (write:package PAT), CURL_USER (login name),
 #      RPM_GPG_PRIVATE_KEY (armored private key; absent => unsigned, fatal on a v* tag)
 set -euo pipefail
@@ -40,6 +42,28 @@ HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
 SUMS="$WORK/SHA256SUMS"
 SIG="$WORK/SHA256SUMS.asc"
+FETCHED="$WORK/SHA256SUMS.fetched"   # exactly what the registry served, before normalization
+
+# read_manifest -> the feed's current manifest, normalized into $SUMS and kept verbatim in
+# $FETCHED. Returns non-zero iff the feed has no manifest at all.
+#
+# -L is not optional here. The registry answers a file GET with a 303 See Other pointing at
+# presigned object storage, and `curl -f` does NOT treat a 3xx as an error — so without -L the call
+# "succeeds" and hands back the redirect's HTML body ('<a href="…">See Other</a>.'). Both callers
+# then took that page for the manifest: every publish prepended a stale redirect page and dropped
+# every prior image line, and --seal signed a page whose presigned URL expired 300 seconds later —
+# a signature over bytes that exist nowhere. Clients fetch WITH -L, so they checked the real
+# manifest against that signature and refused the feed, which from a Bazzite box is indistinguishable
+# from someone having tampered with it.
+#
+# The line filter is the second layer, and the one that does not depend on getting curl's flags
+# right: whatever the transport hands back, only well-formed "<sha256>  <filename>" lines are ever
+# signed or re-published. It also scrubs a feed that already carries an injected page.
+read_manifest() {
+  : > "$SUMS"; : > "$FETCHED"
+  curl -fsSL "${AUTH[@]}" -o "$FETCHED" "$BASE/SHA256SUMS" || return 1
+  grep -E '^[0-9a-f]{64}  [^ ]+$' "$FETCHED" > "$SUMS" || :
+}
 
 # sign_manifest — detached-sign $SUMS into $SIG with RPM_GPG_PRIVATE_KEY. Prints nothing and
 # returns 1 if no key is available; the caller decides whether that is survivable.
@@ -88,14 +112,29 @@ require_signature() {
 
 # --seal: re-sign whatever manifest the feed already has, no image, no pruning.
 if [ "$SEAL" = 1 ]; then
-  curl -fsS "${AUTH[@]}" -o "$SUMS" "$BASE/SHA256SUMS" \
-    || { echo "no SHA256SUMS at $BASE — nothing to seal" >&2; exit 1; }
-  sign_manifest || require_signature
-  if [ -f "$SIG" ]; then
-    curl -fsS -o /dev/null "${AUTH[@]}" -X DELETE "$BASE/SHA256SUMS.asc" || true
-    curl -fsS -o /dev/null "${AUTH[@]}" --upload-file "$SIG" "$BASE/SHA256SUMS.asc"
-    echo "sealed $BASE ($(wc -l <"$SUMS") image(s))"
+  read_manifest || { echo "no SHA256SUMS at $BASE — nothing to seal" >&2; exit 1; }
+  # An empty result means the manifest was ALL junk. Signing that would hand clients a feed that
+  # verifies and offers no images, which reads as "up to date" to `punktfunk-sysext update`.
+  if [ ! -s "$SUMS" ]; then
+    echo "$BASE/SHA256SUMS lists no images — refusing to seal it (feed needs a republish)" >&2
+    exit 1
   fi
+  if ! sign_manifest; then
+    require_signature   # non-release: warn and leave the live feed exactly as it was
+    exit 0
+  fi
+  # The signature must cover the bytes a client actually downloads, so a manifest that normalizing
+  # changed gets re-published with it — otherwise the .asc would describe a file the registry does
+  # not have, which is the very failure this is repairing. Manifest first, signature second, and
+  # neither when the stored copy was already clean.
+  if ! cmp -s "$SUMS" "$FETCHED"; then
+    echo "normalizing $BASE/SHA256SUMS: $(grep -c '' <"$FETCHED") line(s) served, $(grep -c '' <"$SUMS") kept"
+    curl -fsS -o /dev/null "${AUTH[@]}" -X DELETE "$BASE/SHA256SUMS" || true
+    curl -fsS -o /dev/null "${AUTH[@]}" --upload-file "$SUMS" "$BASE/SHA256SUMS"
+  fi
+  curl -fsS -o /dev/null "${AUTH[@]}" -X DELETE "$BASE/SHA256SUMS.asc" || true
+  curl -fsS -o /dev/null "${AUTH[@]}" --upload-file "$SIG" "$BASE/SHA256SUMS.asc"
+  echo "sealed $BASE ($(grep -c '' <"$SUMS") image(s))"
   exit 0
 fi
 
@@ -103,7 +142,14 @@ FNAME="$(basename "$RAW")"
 SHA="$(sha256sum "$RAW" | cut -d' ' -f1)"
 
 # Merge into the existing manifest: drop any prior line for this filename, append ours.
-curl -fsS "${AUTH[@]}" "$BASE/SHA256SUMS" 2>/dev/null | grep -v " $FNAME\$" > "$SUMS" || true
+if read_manifest; then
+  sed -i "\| $FNAME\$|d" "$SUMS"
+  # Said out loud on purpose. A manifest that silently shrinks is how a feed loses its rollback
+  # history, and that went unnoticed precisely because nothing ever reported the carry-over.
+  echo "carrying forward $(grep -c '' <"$SUMS") image(s) from the existing manifest"
+else
+  echo "no manifest at $BASE yet — starting a new feed"
+fi
 printf '%s  %s\n' "$SHA" "$FNAME" >> "$SUMS"
 
 # Prune: keep only the newest $KEEP images (by version sort) in manifest + registry.