merge: bring current main into the audio-substrate branch
ci / bun-nix (pull_request) Successful in 47s
ci / web (pull_request) Successful in 1m19s
apple / swift (pull_request) Successful in 1m27s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m27s
ci / rust-arm64 (pull_request) Successful in 4m25s
android / android (pull_request) Successful in 5m59s
ci / rust (pull_request) Successful in 7m19s
ci / bun-nix (pull_request) Successful in 47s
ci / web (pull_request) Successful in 1m19s
apple / swift (pull_request) Successful in 1m27s
apple / screenshots (pull_request) Skipped
ci / docs-site (pull_request) Successful in 1m27s
ci / rust-arm64 (pull_request) Successful in 4m25s
android / android (pull_request) Successful in 5m59s
ci / rust (pull_request) Successful in 7m19s
Two conflicts, both unions of independent removals/fixes: main fixed the same three install.rs SAFETY comments this branch fixed (main's phrasing kept), and the runner provisioning drops BOTH env lines — main removed PF_FFVK_VULKAN_INCLUDE (pf-ffvk is gone since the FFmpeg replacement), this branch removed VBCABLE_DIR (the retirement).
This commit is contained in:
+27
-10
@@ -1,11 +1,11 @@
|
||||
#!/usr/bin/env bash
|
||||
# CI runner disk hygiene — invoked by docker-prune.service (every 30 min). Lives in a real script
|
||||
# CI runner disk hygiene — invoked by docker-prune.service (every 2 min). Lives in a real script
|
||||
# rather than inline ExecStart= lines because systemd does its OWN $-expansion on ExecStart and
|
||||
# empties shell vars / $(...) before /bin/sh sees them (silently breaking the logic under `|| true`).
|
||||
#
|
||||
# See docker-prune.service for the full why. The headline: the act_runner cache server's blob store
|
||||
# lives INSIDE the long-running runner container's writable layer, where `docker prune` can't reach
|
||||
# it — left alone it grows to tens of GB and fills the disk on its own.
|
||||
# See docker-prune.service for the full why. Sibling: docker-reclaim.sh (hourly) handles what
|
||||
# act_runner *leaks* — per-job volumes, stale networks, old build cache. This one handles what
|
||||
# CI legitimately *produces* and then abandons: per-SHA app tags and the layers they pin.
|
||||
set -u
|
||||
export PATH=/usr/bin:/bin:/usr/local/bin:$PATH
|
||||
|
||||
@@ -23,11 +23,26 @@ MIN_FREE_GB=${MIN_FREE_GB:-60} # ...or this little is left, whichever t
|
||||
# 2026-07-29: zero burst clears fired in six hours
|
||||
# while deb still died of ENOSPC between polls.
|
||||
|
||||
# 1) Routine: trim aged images / build cache / stopped containers. sha-<commit> tags aren't
|
||||
# dangling, so -a is required. until=2h, not 6h: on a busy day every image is younger than six
|
||||
# hours, so the filter matched nothing and a run reclaimed 0B while `docker system df` was
|
||||
# reporting 20+ GB reclaimable. Two hours still protects a re-run of the push being worked on.
|
||||
docker image prune -af --filter until=2h || true
|
||||
# 1) Routine: retire aged per-SHA app tags, then sweep what untagging released.
|
||||
# ⚠ NEVER `docker image prune -a` on this tick. `until=` filters on image CREATION time, so a
|
||||
# CI *base* image (built days ago) that merely has no container this instant counts as "aged" —
|
||||
# including one a job JUST PULLED whose container does not exist yet. Measured 2026-08-07:
|
||||
# this tick ran 07:36:09–:29 and a rust job's `docker create` failed at 07:36:29 with
|
||||
# "No such image: …punktfunk-rust-ci:latest" — three sampled failures that morning, each
|
||||
# coinciding with a prune run to the second — and every idle base image was re-pulled within
|
||||
# minutes (4–7 GB each), churning the LAN registry for nothing.
|
||||
# The only tag debris this host actually accretes is the per-SHA app tags (web/docs — their
|
||||
# creation time IS the local build time, so a 2h age gate is exact), and a dangling-only prune
|
||||
# cannot touch a tagged image, so neither step can race a starting job.
|
||||
now=$(date +%s)
|
||||
docker images --format '{{.Repository}}:{{.Tag}}' 2>/dev/null | grep ':sha-' | while read -r ref; do
|
||||
created=$(docker image inspect -f '{{.Created}}' "$ref" 2>/dev/null) || continue
|
||||
cts=$(date -d "$created" +%s 2>/dev/null) || continue
|
||||
if [ $((now - cts)) -ge 7200 ]; then
|
||||
docker rmi "$ref" >/dev/null 2>&1 || true
|
||||
fi
|
||||
done
|
||||
docker image prune -f || true
|
||||
docker builder prune -af --filter until=2h || true
|
||||
docker buildx prune -af --filter until=2h || true
|
||||
docker container prune -f --filter until=2h || true
|
||||
@@ -44,7 +59,9 @@ docker network prune -f --filter until=2h || true
|
||||
# what matters is absolute headroom for three concurrent target/ dirs, not a ratio — and the
|
||||
# ratio moves whenever the disk is resized (it went 123 G -> 175 G on 2026-07-29) while the
|
||||
# headroom three jobs need does not. In-use images are protected by the daemon, so a burst clear
|
||||
# cannot pull the rug from a live job.
|
||||
# cannot pull the rug from a live job — but the blanket `-a` prune below CAN race an image that
|
||||
# is pulled-but-not-yet-created (the section 1 lesson). That narrow window is accepted HERE
|
||||
# only: when the alternative is every concurrent job dying of ENOSPC, one job re-pulling loses.
|
||||
PCT=$(df --output=pcent / | tr -dc '0-9')
|
||||
FREE_GB=$(df --output=avail -BG / | tr -dc '0-9')
|
||||
# Two flat tests into a flag rather than one multi-line `{ …; } || { …; }` condition: the brace-group
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
# Hourly reclaim of Docker resources act_runner LEAKS (per-job volumes, stale networks, old build
|
||||
# cache). Sibling of docker-prune.service, which handles what CI legitimately produces and then
|
||||
# abandons; the split matters because this one must stay conservative enough to run while jobs are
|
||||
# live (dangling-only volumes, age-gated networks) — see docker-reclaim.sh for the full why.
|
||||
#
|
||||
# Install: see the header of docker-reclaim.sh (note the installed unit name is
|
||||
# ci-docker-reclaim.service — existing fleet hosts already run it under that name).
|
||||
|
||||
[Unit]
|
||||
Description=Reclaim disk leaked by Gitea act_runner (per-job volumes, networks, stale build cache)
|
||||
Documentation=https://git.unom.io/unom/punktfunk
|
||||
After=docker.service
|
||||
Requires=docker.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/sbin/ci-docker-reclaim.sh
|
||||
# Never let maintenance starve a running build.
|
||||
Nice=10
|
||||
IOSchedulingClass=idle
|
||||
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env bash
|
||||
# Reclaim the disk that Gitea act_runner leaks on this host.
|
||||
#
|
||||
# Why this exists: act_runner creates a per-job network and a pair of named volumes, and leaks both
|
||||
# when a job is killed or the runner restarts. By 2026-07-25 that had accumulated 252 unused volumes
|
||||
# (11.7 GB) and 94 stale networks — some dating to task 5626 while current tasks were ~25233 — and
|
||||
# concurrent builds then exhausted the disk, failing CI with "No space left on device" at both the
|
||||
# cargo and the Docker/overlayfs layer. The stale networks are also what once broke the docs deploy
|
||||
# by exhausting Docker's default address pool and swallowing the DMZ 192.168.50.0/24 range.
|
||||
#
|
||||
# This ran on home-runner-1 only, hand-installed; home-runner-2 went without it and by 2026-08-07
|
||||
# had re-accumulated 176 leaked volumes (~60 GB) + 22 GB build cache and spent two days failing
|
||||
# jobs at ENOSPC. Hence checked in: BOTH runner hosts install it, from here.
|
||||
#
|
||||
# Install on a runner host (root):
|
||||
# install -m755 scripts/ci/docker-reclaim.sh /usr/local/sbin/ci-docker-reclaim.sh
|
||||
# install -m644 scripts/ci/docker-reclaim.service /etc/systemd/system/ci-docker-reclaim.service
|
||||
# install -m644 scripts/ci/docker-reclaim.timer /etc/systemd/system/ci-docker-reclaim.timer
|
||||
# systemctl daemon-reload && systemctl enable --now ci-docker-reclaim.timer
|
||||
#
|
||||
# Deliberately NOT `docker volume prune -a`: that would also delete any intentional named volume
|
||||
# that merely has no container attached at the moment the timer fires — e.g. the `docker-mirror`
|
||||
# pull-through registry cache or the runner cache during a restart — silently destroying it. Only
|
||||
# volumes act_runner named are removed here.
|
||||
#
|
||||
# Also deliberately NOT pruning images: on this host the per-SHA CI tags share all their layers with
|
||||
# `:latest`, so removing them reclaims nothing while forcing re-pulls. `docker system df`'s
|
||||
# "RECLAIMABLE" column counts shared layers once per image and overstates the win badly.
|
||||
# (docker-prune.sh owns tag retirement — age-gated and never `image prune -a`, see its header.)
|
||||
set -uo pipefail
|
||||
|
||||
log() { echo "ci-docker-reclaim: $*"; }
|
||||
|
||||
before_avail=$(df --output=avail -BM / | tail -1 | tr -dc '0-9')
|
||||
|
||||
# 1. Leaked per-job volumes — dangling AND named by act_runner. In-use volumes are never listed as
|
||||
# dangling, so a running job's volumes cannot be hit.
|
||||
mapfile -t stale_vols < <(docker volume ls -qf dangling=true 2>/dev/null | grep '^GITEA-ACTIONS-TASK-' || true)
|
||||
if ((${#stale_vols[@]})); then
|
||||
printf '%s\n' "${stale_vols[@]}" | xargs -r docker volume rm >/dev/null 2>&1
|
||||
log "removed ${#stale_vols[@]} leaked act_runner volumes"
|
||||
else
|
||||
log "no leaked act_runner volumes"
|
||||
fi
|
||||
|
||||
# 2. Unused networks older than 2h — never touches a live job's network (it is in use), and the age
|
||||
# filter keeps a just-created one safe against a race with a starting job.
|
||||
net_out=$(docker network prune -f --filter until=2h 2>&1 | grep -c '^GITEA-ACTIONS' || true)
|
||||
log "removed ${net_out:-0} stale job networks"
|
||||
|
||||
# 3. Build cache older than 48h. Recent cache is what makes builds fast, so it is kept.
|
||||
cache_freed=$(docker builder prune -f --filter until=48h 2>&1 | awk '/^Total:/ {print $2}')
|
||||
log "build cache freed: ${cache_freed:-0B}"
|
||||
|
||||
after_avail=$(df --output=avail -BM / | tail -1 | tr -dc '0-9')
|
||||
log "avail ${before_avail}M -> ${after_avail}M (reclaimed $((after_avail - before_avail))M)"
|
||||
df -h / | tail -1 | sed 's/^/ci-docker-reclaim: /'
|
||||
@@ -0,0 +1,16 @@
|
||||
# Hourly is the right cadence for LEAKS: they only accrue when jobs die abnormally, and the
|
||||
# per-tick docker-prune.timer (every 2 min) already carries the burst guard for genuine
|
||||
# disk-pressure emergencies. Install: see the header of docker-reclaim.sh.
|
||||
|
||||
[Unit]
|
||||
Description=Hourly reclaim of act_runner-leaked Docker disk
|
||||
|
||||
[Timer]
|
||||
OnCalendar=hourly
|
||||
# Catch up after a reboot rather than waiting for the next slot.
|
||||
Persistent=true
|
||||
# Spread it off the hour so it does not collide with scheduled CI.
|
||||
RandomizedDelaySec=300
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -1,5 +1,5 @@
|
||||
# Idempotent pre-flight for punktfunk's Windows CI dependencies: WDK + cargo-wdk (driver builds),
|
||||
# FFmpeg x64/ARM64 trees, Inno Setup, and the aarch64-pc-windows-msvc rustup target. Run at the
|
||||
# the x64 FFmpeg tree (host amf-qsv only), Inno Setup, and the aarch64-pc-windows-msvc rustup target. Run at the
|
||||
# start of every Windows CI job so ANY runner - freshly built from unom/infra's windows-runner/
|
||||
# template, rebuilt, or a new one added later - self-provisions on first real use, instead of
|
||||
# needing a human to remember to dispatch a separate provisioning workflow first (and instead of
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Layers punktfunk-specific tooling onto the shared unom Windows CI runner: per-arch FFmpeg
|
||||
# (host + client native builds), Inno Setup (the host installer), and the aarch64-pc-windows-msvc
|
||||
# Layers punktfunk-specific tooling onto the shared unom Windows CI runner: FFmpeg (the HOST's
|
||||
# amf-qsv encode leg, x64 only), Inno Setup (the host installer), and the aarch64-pc-windows-msvc
|
||||
# rustup target (windows-msix.yml's ARM64 leg). The runner itself - act_runner, Node, rustup,
|
||||
# VS Build Tools/NASM/CMake/LLVM - is provisioned generically by unom/infra
|
||||
# (windows-runner/windows-runner.pkr.hcl + proxmox/windows-runner's Terraform clone); this script
|
||||
@@ -26,14 +26,19 @@ if (Test-Path $rustup) {
|
||||
Write-Warning "rustup not found at $rustup - has unom/infra's setup-gitea-runner-base.ps1 run on this box yet?"
|
||||
}
|
||||
|
||||
# --- FFmpeg shared trees for the host (amf-qsv encode) + clients (decode). BtbN **lgpl-shared**
|
||||
# --- FFmpeg shared tree for the HOST's amf-qsv encode leg (windows-host.yml). BtbN **lgpl-shared**
|
||||
# builds: the AMD/Intel AMF + Intel QSV encoders, swscale, and the HEVC decoder are all present in
|
||||
# the LGPL build, and punktfunk never calls the GPL-only encoders (x264/x265 - software encode is
|
||||
# the separate BSD-2 openh264 crate; NVENC is the direct NVIDIA SDK). lgpl-shared keeps the
|
||||
# bundled DLLs LGPL-2.1+ (dynamic linking satisfies the relink duty) rather than GPL, so the
|
||||
# shipped installer/MSIX stay consistent with punktfunk's MIT OR Apache-2.0 posture.
|
||||
# MIGRATION: a runner previously provisioned with the old *gpl-shared* trees must be
|
||||
# re-provisioned - delete C:\Users\Public\ffmpeg and C:\Users\Public\ffmpeg-arm64, then re-run.
|
||||
# ⚠ The CLIENT no longer links FFmpeg at all (M10, design/client-native-decode.md §6): it decodes
|
||||
# with pf-vkdecode / pf-dxvadec / openh264 + rav1d. windows.yml and windows-msix.yml set no
|
||||
# FFMPEG_DIR and the MSIX bundles no libav* DLLs, so only the x64 tree is fetched now - the ARM64
|
||||
# one existed solely for the ARM64 client leg. Delete a stale C:\Users\Public\ffmpeg-arm64 by
|
||||
# hand; this script does not remove what it no longer installs.
|
||||
# MIGRATION: a runner previously provisioned with the old *gpl-shared* tree must be
|
||||
# re-provisioned - delete C:\Users\Public\ffmpeg, then re-run.
|
||||
# These DLLs are bundled verbatim into the code-signed host installer/MSIX, so the download is
|
||||
# SHA-256-pinned (like VB-CABLE below): BtbN's `latest` tag is a ROLLING release whose assets are
|
||||
# re-uploaded over time, so an unverified fetch would let a hijacked/MITM'd upstream asset land
|
||||
@@ -42,7 +47,7 @@ if (Test-Path $rustup) {
|
||||
# that is intentional: re-download, re-verify the new archive, and update the two pins here.
|
||||
# Refresh a pin: (Get-FileHash .\ffmpeg-<tag>.zip -Algorithm SHA256).Hash
|
||||
function Get-BtbnFfmpeg {
|
||||
param([string]$Dir, [string]$ZipTag, [string]$Sha) # ZipTag: 'win64' (x64) or 'winarm64' (ARM64 cross tree)
|
||||
param([string]$Dir, [string]$ZipTag, [string]$Sha) # ZipTag: 'win64' (x64); BtbN also publishes 'winarm64'
|
||||
if (Test-Path (Join-Path $Dir 'lib\avcodec.lib')) { info "FFmpeg ($ZipTag) already present at $Dir"; return }
|
||||
info "fetching FFmpeg ($ZipTag, BtbN lgpl-shared, SHA-256 pinned)"
|
||||
$url = "https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-n7.1-latest-$ZipTag-lgpl-shared-7.1.zip"
|
||||
@@ -60,27 +65,13 @@ function Get-BtbnFfmpeg {
|
||||
Move-Item -Path $inner.FullName -Destination $Dir
|
||||
Remove-Item -Force $zip; Remove-Item -Recurse -Force $tmp -ErrorAction SilentlyContinue
|
||||
}
|
||||
Get-BtbnFfmpeg -Dir "C:\Users\Public\ffmpeg" -ZipTag 'win64' -Sha '89F3469706E5D53AEA5CF34AEE63E62CE746E6159D7AEE473D330B02A47558E6'
|
||||
Get-BtbnFfmpeg -Dir "C:\Users\Public\ffmpeg-arm64" -ZipTag 'winarm64' -Sha 'D96B4CE08CEBDCC6AD0E3934A3F962915E440EEFB9D73831AFEA4D80E35129A5'
|
||||
Get-BtbnFfmpeg -Dir "C:\Users\Public\ffmpeg" -ZipTag 'win64' -Sha '89F3469706E5D53AEA5CF34AEE63E62CE746E6159D7AEE473D330B02A47558E6'
|
||||
|
||||
# --- Vulkan-Headers (pf-ffvk's bindgen: libavutil/hwcontext_vulkan.h includes <vulkan/vulkan.h>,
|
||||
# and Windows has no system copy). Headers only - the loader (vulkan-1.dll) is a GPU-driver
|
||||
# component and is never linked at build time, so the full Vulkan SDK is deliberately NOT
|
||||
# required. Pinned Khronos tag; bump deliberately alongside FFmpeg/driver expectations. ---
|
||||
$vkHdrDir = "C:\Users\Public\vulkan-headers"
|
||||
$vkHdrTag = "v1.4.309"
|
||||
if (-not (Test-Path (Join-Path $vkHdrDir 'include\vulkan\vulkan.h'))) {
|
||||
info "fetching Vulkan-Headers $vkHdrTag"
|
||||
$url = "https://github.com/KhronosGroup/Vulkan-Headers/archive/refs/tags/$vkHdrTag.zip"
|
||||
$zip = "$vkHdrDir.zip"; $tmp = "$vkHdrDir-extract"
|
||||
Invoke-WebRequest -Uri $url -OutFile $zip -UseBasicParsing
|
||||
if (Test-Path $tmp) { Remove-Item -Recurse -Force $tmp }
|
||||
Expand-Archive -Path $zip -DestinationPath $tmp -Force # one top-level Vulkan-Headers-<ver> folder
|
||||
$inner = Get-ChildItem $tmp -Directory | Select-Object -First 1
|
||||
if (Test-Path $vkHdrDir) { Remove-Item -Recurse -Force $vkHdrDir }
|
||||
Move-Item -Path $inner.FullName -Destination $vkHdrDir
|
||||
Remove-Item -Force $zip; Remove-Item -Recurse -Force $tmp -ErrorAction SilentlyContinue
|
||||
} else { info "Vulkan-Headers already present at $vkHdrDir" }
|
||||
# --- No Vulkan-Headers here any more: they existed only for pf-ffvk's bindgen over
|
||||
# libavutil/hwcontext_vulkan.h, and that crate is gone (M10). Nothing punktfunk builds on Windows
|
||||
# needs Vulkan headers at compile time - ash generates its own bindings and dlopens vulkan-1.dll,
|
||||
# which is a GPU-driver component. A stale C:\Users\Public\vulkan-headers is harmless; delete it
|
||||
# by hand if you want the disk back. ---
|
||||
|
||||
# --- Inno Setup (ISCC.exe) for the host installer build (windows-host.yml). pack-host-installer.ps1
|
||||
# locates it at its fixed Program Files path, so it need not be on PATH - just present. The .iss
|
||||
@@ -101,13 +92,14 @@ if (-not (Test-Path $isccPath) -or ($innoVer -and [version]$innoVer -lt [version
|
||||
|
||||
# --- Drop punktfunk's env vars into the generic runner's daemon wrapper extension point (see
|
||||
# unom/infra's scripts/setup-gitea-runner-base.ps1) so the act_runner daemon - and therefore every
|
||||
# job it runs - sees FFMPEG_DIR without unom/infra needing to know punktfunk exists. ---
|
||||
# job it runs - sees FFMPEG_DIR without unom/infra needing to know punktfunk exists.
|
||||
# FFMPEG_DIR + the PATH prepend are the HOST's (windows-host.yml amf-qsv: import libs at link time,
|
||||
# the DLLs at test time). The client workflows ignore both - they link no libav*. ---
|
||||
$projectEnv = "C:\Users\Public\act-runner\project-env.ps1"
|
||||
@'
|
||||
$env:FFMPEG_DIR = "C:\Users\Public\ffmpeg"
|
||||
$env:PF_FFVK_VULKAN_INCLUDE = "C:\Users\Public\vulkan-headers\include"
|
||||
$env:PATH = "C:\Users\Public\ffmpeg\bin;" + $env:PATH
|
||||
'@ | Set-Content -Encoding UTF8 $projectEnv
|
||||
info "wrote $projectEnv (FFMPEG_DIR, PF_FFVK_VULKAN_INCLUDE) - restart the gitea-act-runner scheduled task to pick it up"
|
||||
info "wrote $projectEnv (FFMPEG_DIR) - restart the gitea-act-runner scheduled task to pick it up"
|
||||
|
||||
info "punktfunk extras provisioned OK."
|
||||
|
||||
@@ -12,7 +12,16 @@ Apache/Unicode/etc.) crates linked into shipped punktfunk artifacts. `cargo abou
|
||||
about.toml) produces an equivalent, network-augmented result in CI; this is the dependency-free
|
||||
fallback that also runs locally and is committed as a baseline.
|
||||
|
||||
By default it covers the WHOLE workspace, which is what the root file must be (the host and
|
||||
the desktop clients ship out of it). `--packages <name>[,<name>…]` restricts it to the transitive
|
||||
dependency closure of the named workspace members instead — the Apple and Android clients link
|
||||
exactly one Rust crate each (`punktfunk-core`, and the JNI bridge over it), so a workspace-wide
|
||||
copy attributed them things they do not contain: FFmpeg, the NVENC SDK, GTK, windows-rs. Listing a
|
||||
dependency that is not there is not a licence violation, but it is a false statement in a file
|
||||
whose entire job is to be true.
|
||||
|
||||
Usage: python3 scripts/gen-third-party-notices.py [--out THIRD-PARTY-NOTICES.txt]
|
||||
[--packages punktfunk-core,…]
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
@@ -82,21 +91,71 @@ VENDORED_TREES = [
|
||||
]
|
||||
|
||||
|
||||
def closure(meta, roots):
|
||||
"""Package ids reachable from `roots` through `cargo metadata`'s resolve graph.
|
||||
|
||||
Deliberately the WHOLE resolve graph, not a per-target one: `cargo metadata` resolves
|
||||
every `cfg()`-gated dependency of every member, so this OVER-approximates (an
|
||||
`cfg(windows)`-only crate is reachable from a root even on a Linux build). Over-listing an
|
||||
attribution is the safe direction; under-listing one is the failure this file exists to
|
||||
prevent. What it does NOT do is pull in crates reachable only from OTHER workspace members,
|
||||
which is the whole point.
|
||||
"""
|
||||
by_name = {}
|
||||
for p in meta["packages"]:
|
||||
by_name.setdefault(p["name"], p["id"])
|
||||
nodes = {n["id"]: n for n in meta.get("resolve", {}).get("nodes", [])}
|
||||
seen, stack = set(), []
|
||||
for r in roots:
|
||||
pid = by_name.get(r)
|
||||
if pid is None:
|
||||
raise SystemExit(f"--packages: no package named {r!r} in this workspace")
|
||||
stack.append(pid)
|
||||
while stack:
|
||||
pid = stack.pop()
|
||||
if pid in seen:
|
||||
continue
|
||||
seen.add(pid)
|
||||
stack.extend(nodes.get(pid, {}).get("dependencies", []))
|
||||
return seen
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--out", default="THIRD-PARTY-NOTICES.txt")
|
||||
ap.add_argument("--manifest", default="Cargo.toml")
|
||||
ap.add_argument(
|
||||
"--packages",
|
||||
default="",
|
||||
help="comma-separated workspace member names; restrict the notices to their transitive "
|
||||
"dependency closure instead of the whole workspace",
|
||||
)
|
||||
args = ap.parse_args()
|
||||
|
||||
# `--all-features` is what makes `--packages` a GUARANTEE rather than a coincidence. Without
|
||||
# it, cargo resolves the workspace with default features and unifies them across members, so a
|
||||
# scoped closure can pick up a crate only because some OTHER member turned the feature on —
|
||||
# and, worse, can MISS one when no member does. punktfunk-core's `quic` is exactly that case:
|
||||
# it is not a default feature, and quinn/opus/rustls reach the Apple file today only through
|
||||
# the workspace-wide union. Resolving every feature over-approximates instead, which is the
|
||||
# safe direction for an attribution file: listing a crate that is not linked is untidy,
|
||||
# omitting one that is is the failure this file exists to prevent.
|
||||
meta = json.loads(subprocess.check_output(
|
||||
["cargo", "metadata", "--format-version", "1", "--offline", "--manifest-path", args.manifest],
|
||||
["cargo", "metadata", "--format-version", "1", "--offline", "--all-features",
|
||||
"--manifest-path", args.manifest],
|
||||
text=True))
|
||||
ws_members = set(meta.get("workspace_members", []))
|
||||
|
||||
keep = None
|
||||
if args.packages.strip():
|
||||
keep = closure(meta, [n.strip() for n in args.packages.split(",") if n.strip()])
|
||||
|
||||
pkgs = []
|
||||
for p in meta["packages"]:
|
||||
if p["id"] in ws_members:
|
||||
continue # first-party (covered by the root LICENSE-MIT / LICENSE-APACHE)
|
||||
if keep is not None and p["id"] not in keep:
|
||||
continue
|
||||
pkgs.append(p)
|
||||
pkgs.sort(key=lambda p: (p["name"].lower(), p["version"]))
|
||||
|
||||
@@ -142,6 +201,9 @@ def main():
|
||||
w("below. Each is distributed under its own permissive license; the full license texts")
|
||||
w("follow the manifest. This file is generated by scripts/gen-third-party-notices.py")
|
||||
w("(or `cargo about`, see about.toml) — do not edit by hand.")
|
||||
if keep is not None:
|
||||
w("")
|
||||
w(f"Scope: the Rust crates linked by {args.packages} — not the whole punktfunk workspace.")
|
||||
w("")
|
||||
w(f"Total third-party crates: {len(pkgs)}")
|
||||
w("")
|
||||
|
||||
@@ -20,16 +20,39 @@ else
|
||||
fi
|
||||
echo "==> wrote $OUT" >&2
|
||||
|
||||
# Keep the per-client in-tree copies in sync (the GUI apps bundle these as resources/assets and
|
||||
# show them on their Acknowledgements / Open-source-licenses screen). The Linux/Windows Rust clients
|
||||
# embed the root file directly via include_str!, so they need no copy.
|
||||
# Regenerate the per-client in-tree copies. EVERY client has one now, because every client SHOWS
|
||||
# it: the mobile apps bundle theirs as a resource/asset for their Acknowledgements screen, and the
|
||||
# two desktop shells `include_str!` theirs onto their Licenses page (the MSIX and the client .deb
|
||||
# ship the file as well).
|
||||
#
|
||||
# These are GENERATED, not copied. They used to be the workspace-wide file, which attributed to
|
||||
# every client every crate anything in this repo links: FFmpeg, the NVENC SDK, GTK4, windows-rs.
|
||||
# The Apple app links ONE Rust crate (punktfunk-core, through PunktfunkCore.xcframework — see
|
||||
# scripts/build-xcframework.sh) and Android links the JNI bridge over it; everything else in those
|
||||
# apps is Swift/Kotlin and platform frameworks.
|
||||
#
|
||||
# M10 — the client's FFmpeg excision — is what turned the same untidiness on the DESKTOP copies
|
||||
# into a false statement a user can see: the shells print an `ffmpeg-next 8.1.0 — WTFPL` line and
|
||||
# the full FFmpeg licence text three screens under a card saying no FFmpeg is bundled. So they are
|
||||
# scoped too, each to the binaries its package actually installs — the shell, the session streamer,
|
||||
# the headless CLI, and on Linux the update helper (`pf-update` ships as pf-update-client).
|
||||
#
|
||||
# The ROOT file stays workspace-wide on purpose: the HOST ships out of it, and the host does still
|
||||
# link FFmpeg.
|
||||
#
|
||||
# Only the offline generator can scope a file (cargo-about renders the whole workspace), so these
|
||||
# always go through it — the root file above still prefers cargo-about when installed.
|
||||
if [ "$OUT" = "THIRD-PARTY-NOTICES.txt" ]; then
|
||||
for dest in \
|
||||
clients/apple/Sources/PunktfunkKit/Resources/THIRD-PARTY-NOTICES.txt \
|
||||
clients/android/app/src/main/assets/THIRD-PARTY-NOTICES.txt; do
|
||||
if [ -d "$(dirname "$dest")" ]; then
|
||||
cp "$OUT" "$dest"
|
||||
echo "==> synced $dest" >&2
|
||||
fi
|
||||
done
|
||||
# <in-tree path> <workspace members whose closure it must state>
|
||||
while read -r dest packages; do
|
||||
[ -n "$dest" ] || continue
|
||||
[ -d "$(dirname "$dest")" ] || continue
|
||||
python3 scripts/gen-third-party-notices.py --packages "$packages" --out "$dest"
|
||||
echo "==> generated $dest ($packages closure)" >&2
|
||||
done <<'CLIENTS'
|
||||
clients/apple/Sources/PunktfunkKit/Resources/THIRD-PARTY-NOTICES.txt punktfunk-core
|
||||
clients/android/app/src/main/assets/THIRD-PARTY-NOTICES.txt punktfunk-client-android
|
||||
clients/linux/THIRD-PARTY-NOTICES.txt punktfunk-client-linux,punktfunk-client-session,punktfunk-cli,pf-update
|
||||
clients/windows/THIRD-PARTY-NOTICES.txt punktfunk-client-windows,punktfunk-client-session,punktfunk-cli
|
||||
CLIENTS
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user