From 3213c1b61c33ceb3b8d2c7b9e63170bbe3b9ba44 Mon Sep 17 00:00:00 2001 From: enricobuehler Date: Mon, 20 Jul 2026 08:33:34 +0200 Subject: [PATCH] ci(flatpak): resolve over TCP so the flathub bootstrap stops failing on tag pushes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The flathub remote-add has now burned its entire retry budget and failed the job on three consecutive tag builds — v0.15.0, the v0.15.0 re-point, and v0.16.0 — each one needing a manual re-run to land the bundle. The cause was already root-caused here on 2026-07-11 (see the Tooling step's comment): home-runner-1's Docker embedded resolver at 127.0.0.11 drops UDP lookups while the shared multi-org fleet is saturated. Nothing retransmits a dropped datagram, so the lookup simply times out. The fix then was to widen retry.sh to 10 attempts (~9 min), which comfortably outlasts a main push's ~8-workflow fan-out — but a TAG push starts 13 workflows at once, and that burst now outlives the whole budget. Retrying harder is chasing the symptom; the transport is the problem. `options use-vc` moves resolution to TCP, where the kernel retransmits and a query cannot be silently lost under load. Same resolver and search path — only the transport changes — so internal names still resolve exactly as before. Deliberately no additional nameservers: a public resolver in the list could answer an internal name (git.unom.io) with NXDOMAIN. retry.sh stays as the backstop for real upstream blips. Applied non-fatally, because Docker bind-mounts /etc/resolv.conf and may present it read-only: a DNS tuning that cannot be applied must not be the thing that fails a release build. If the write is refused we warn and stay on UDP, i.e. exactly today's behaviour. Scoped to the flatpak workflow, where the failure is actually evidenced, rather than applied fleet-wide on suspicion. Co-Authored-By: Claude Opus 4.8 --- .gitea/workflows/flatpak.yml | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/.gitea/workflows/flatpak.yml b/.gitea/workflows/flatpak.yml index d25879c6..3605b585 100644 --- a/.gitea/workflows/flatpak.yml +++ b/.gitea/workflows/flatpak.yml @@ -73,8 +73,34 @@ jobs: # sufficient — the Tooling step's dnf install pulls a systemd package upgrade whose RPM # trigger re-runs authselect and regenerates this file, undoing the fix. It's reapplied # there, right before the first `flatpak` network call. - - name: Fix container DNS (drop nss-resolve — no systemd-resolved in CI) - run: sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf + - name: Fix container DNS (drop nss-resolve, resolve over TCP) + run: | + sed -i 's/resolve \[!UNAVAIL=return\] //' /etc/nsswitch.conf + # Resolve over TCP instead of UDP. The documented root cause of the flathub + # bootstrap failures (investigated 2026-07-11, see the Tooling step) is this box's + # Docker embedded resolver at 127.0.0.11 DROPPING UDP lookups while the shared + # runner fleet is saturated — a datagram nobody retransmits, so the lookup just + # times out. The answer then was to widen retry.sh's budget to 10 attempts (~9 min), + # which is enough to outlast a main push's ~8-workflow fan-out but NOT a TAG push's + # 13: v0.15.0 (twice) and v0.16.0 each burned all 10 attempts and failed the job, + # each needing a manual re-run. + # + # `use-vc` makes glibc use TCP, where the kernel retransmits and the query cannot be + # silently lost under load. Same resolver, same search path — only the transport + # changes, so internal names (git.unom.io) resolve exactly as before; deliberately + # NO extra nameservers, which would risk answering an internal name from a public + # resolver. Docker's embedded DNS serves TCP on 127.0.0.11:53 as well as UDP. + # retry.sh stays as the backstop for genuine upstream blips. + # + # Non-fatal: Docker bind-mounts /etc/resolv.conf and can present it read-only, and a + # DNS tuning that cannot be applied must not be what fails the release build — that + # would trade an occasional re-run for a hard stop. Falling back to UDP just restores + # today's behaviour, which retry.sh already covers. + if ! grep -q '^options .*use-vc' /etc/resolv.conf 2>/dev/null; then + echo 'options use-vc timeout:3 attempts:3' >> /etc/resolv.conf \ + || echo "::warning::could not set use-vc (read-only resolv.conf?); staying on UDP" + fi + cat /etc/resolv.conf || true # fedora:43 has no node, but actions/checkout (a JS action) needs it. A plain `run:` step # executes via the container shell (no node needed), so install node BEFORE checkout.