unom/punktfunk
5
0
Fork 0
Code Issues 2 Pull Requests 2 Actions 2 Packages Projects Releases 39 Wiki Activity

feat(store): plugin store host module — signed catalogs, tiered trust, install jobs

Browse Source 
The index is the verification gate: a catalog entry pins one exact version
plus that version's tarball integrity hash, so 'verified on every release'
is a property of the data rather than a promise about process. Nothing can
express 'track latest' for a catalogued plugin.

- store/index.rs   signed index parse + ed25519 verify (ring), validate-and-drop
                   per entry, semver minHost/advisory ranges
- store/sources.rs built-in unom source (compiled-in URL + two key slots for
                   rotation) + operator sources in plugin-sources.json
- store/catalog.rs https fetch with size/timeout/redirect caps, signature before
                   parse, last-good disk cache (stale-but-usable when offline)
- store/jobs.rs    single-flight install/uninstall: registry-integrity preflight
                   against the pin, spawn the runner CLI with live log capture,
                   post-install version check with rollback, provenance record,
                   runner restart (discovery is startup-only)
- store/manifest.rs install provenance; absence means CLI-installed
- mgmt/store.rs    12 routes under /api/v1/store, denied to the plugin token
                   (a plugin that can install plugins is an escalation primitive)

Also generalizes runner discovery and listInstalled from @punktfunk/plugin-*
to ANY scope's plugin-*: catalog entries must be scoped so the scope can map
to that entry's registry, so a third-party plugin necessarily arrives under
its own scope and would otherwise install but never run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
enricobuehler
2026-07-20 20:48:02 +02:00
co-authored by Claude Fable 5
parent 96e19986bc
commit 45c3b96907
19 changed files with 3578 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/punktfunk-host/src/store/catalog.rs
+255
View File
@@ -0,0 +1,255 @@
//! Catalog **fetch + cache**: pulling a source's signed index over HTTPS and keeping the last good
//! copy on disk (design `plugin-store.md` §4.1).
//!
//! Two properties matter more than freshness here:
//!
//! - **Fail closed on signatures, fail soft on the network.** A source with a pinned key whose
//! document doesn't verify is an *error* — we keep serving the last good copy and say so, and we
//! never fall back to "well, unsigned then". But a box that simply can't reach the internet
//! (LAN-only, common for a streaming host) keeps a working store: the cached shelf still browses,
//! and installs off it still pin and integrity-check, because the pin travelled with the entry.
//! - **Bounded everything.** Size cap, timeout, redirect cap, https-only, no credentials ever
//! attached. A source URL is operator config, not request-time input, so there is no lever here
//! for a browser to aim the host at an arbitrary address.
use super::index::{Index, MAX_INDEX_BYTES};
use super::sources::Source;
use std::io::Read;
use std::path::{Path, PathBuf};
use std::time::Duration;
/// Wall-clock budget for one index (or signature) fetch.
const FETCH_TIMEOUT: Duration = Duration::from_secs(15);
/// What a refresh attempt produced.
pub(crate) enum Fetched {
/// A new, verified, parsed document.
Fresh {
index: Box<Index>,
etag: Option<String>,
},
/// The source says our cached copy is still current (HTTP 304).
NotModified,
/// Nothing usable arrived. The caller keeps whatever it had and marks the source stale.
Failed(String),
}
/// Fetch, verify and parse a source's index.
///
/// Blocking (`ureq`) — callers run this on a blocking thread, never on the async runtime.
pub(crate) fn fetch(source: &Source, etag: Option<&str>) -> Fetched {
if !source.url.starts_with("https://") {
return Fetched::Failed("source url must be https".into());
}
let agent = ureq::AgentBuilder::new()
.timeout(FETCH_TIMEOUT)
// A signed document doesn't need many hops to reach us; a redirect chain is a good way to
// waste a host's time.
.redirects(3)
.user_agent(&format!("punktfunk-host/{}", super::index::host_version()))
.build();
let mut req = agent.get(&source.url);
if let Some(tag) = etag {
req = req.set("If-None-Match", tag);
}
let resp = match req.call() {
Ok(r) => r,
// 304 is the happy "nothing changed" path, not a failure.
Err(ureq::Error::Status(304, _)) => return Fetched::NotModified,
Err(ureq::Error::Status(code, _)) => {
return Fetched::Failed(format!("index fetch returned HTTP {code}"))
}
Err(e) => return Fetched::Failed(format!("index fetch failed: {e}")),
};
let new_etag = resp.header("etag").map(str::to_string);
let body = match read_capped(resp) {
Ok(b) => b,
Err(e) => return Fetched::Failed(e),
};
// Signature FIRST — nothing below may look at a field before this passes (design §6.3).
let keys = source.keys();
if !keys.is_empty() {
let sig = match agent.get(&source.sig_url()).call() {
Ok(r) => match read_capped(r) {
Ok(b) => b,
Err(e) => return Fetched::Failed(format!("signature: {e}")),
},
Err(e) => {
return Fetched::Failed(format!(
"this source is signed but its signature could not be fetched: {e}"
))
}
};
let sig_text = match String::from_utf8(sig) {
Ok(s) => s,
Err(_) => return Fetched::Failed("signature file is not text".into()),
};
if let Err(e) = super::index::verify_signature(&body, &sig_text, &keys) {
return Fetched::Failed(format!("signature check failed: {e}"));
}
}
match Index::parse(&body) {
Ok(index) => Fetched::Fresh {
index: Box::new(index),
etag: new_etag,
},
Err(e) => Fetched::Failed(format!("{e:#}")),
}
}
/// Read a response body, refusing anything past the cap without buffering it.
fn read_capped(resp: ureq::Response) -> Result<Vec<u8>, String> {
let mut buf = Vec::new();
resp.into_reader()
.take((MAX_INDEX_BYTES + 1) as u64)
.read_to_end(&mut buf)
.map_err(|e| format!("reading the response body failed: {e}"))?;
if buf.len() > MAX_INDEX_BYTES {
return Err(format!("response exceeds the {MAX_INDEX_BYTES}-byte cap"));
}
Ok(buf)
}
// ---------------------------------------------------------------- disk cache
/// `<config_dir>/store-cache` — the last good copy of every source's index, so a host that boots
/// without a network still has a browsable store.
pub(crate) fn cache_dir() -> PathBuf {
pf_paths::config_dir().join("store-cache")
}
fn body_path(dir: &Path, source: &str) -> PathBuf {
dir.join(format!("{source}.json"))
}
fn meta_path(dir: &Path, source: &str) -> PathBuf {
dir.join(format!("{source}.meta.json"))
}
#[derive(serde::Serialize, serde::Deserialize, Default)]
pub(crate) struct CacheMeta {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub etag: Option<String>,
/// Unix seconds of the fetch that produced the cached body.
#[serde(default)]
pub fetched_at: u64,
}
/// The cached index for a source, if one is on disk and still parses. Re-validated on read: a
/// cache file is just as untrusted as the wire (it lives in a directory an admin can write).
///
/// NOTE the signature is **not** re-checked here — it was checked when the bytes were accepted,
/// and the cache directory is inside the host's own private config tree.
pub(crate) fn read_cache(dir: &Path, source: &str) -> Option<(Index, CacheMeta)> {
let body = std::fs::read(body_path(dir, source)).ok()?;
let index = Index::parse(&body).ok()?;
let meta = std::fs::read(meta_path(dir, source))
.ok()
.and_then(|b| serde_json::from_slice::<CacheMeta>(&b).ok())
.unwrap_or_default();
Some((index, meta))
}
/// Persist a freshly verified index as the new last-good copy.
pub(crate) fn write_cache(dir: &Path, source: &str, index: &Index, meta: &CacheMeta) {
if let Err(e) = pf_paths::create_private_dir(dir) {
tracing::warn!("could not create the store cache dir: {e}");
return;
}
let write = |path: PathBuf, bytes: Vec<u8>| {
let tmp = path.with_extension("tmp");
if std::fs::write(&tmp, bytes).is_ok() {
let _ = std::fs::rename(&tmp, &path);
}
};
match serde_json::to_vec_pretty(index) {
Ok(b) => write(body_path(dir, source), b),
Err(e) => tracing::warn!("could not serialize the catalog cache: {e}"),
}
if let Ok(b) = serde_json::to_vec_pretty(meta) {
write(meta_path(dir, source), b);
}
}
/// Drop a removed source's cache files so a later re-add can't be served a stale shelf.
pub(crate) fn drop_cache(dir: &Path, source: &str) {
let _ = std::fs::remove_file(body_path(dir, source));
let _ = std::fs::remove_file(meta_path(dir, source));
}
pub(crate) fn unix_now() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
#[cfg(test)]
mod tests {
use super::*;
fn sample_index() -> Index {
Index::parse(
br#"{"schema":1,"name":"t","plugins":[{"id":"a","pkg":"@p/plugin-a",
"registry":"https://r.example/","title":"A","version":"1.0.0",
"integrity":"sha512-AAAA"}]}"#,
)
.unwrap()
}
#[test]
fn cache_round_trips() {
let dir = tempfile::tempdir().unwrap();
let idx = sample_index();
let meta = CacheMeta {
etag: Some("\"abc\"".into()),
fetched_at: 1_700_000_000,
};
write_cache(dir.path(), "unom", &idx, &meta);
let (back, back_meta) = read_cache(dir.path(), "unom").expect("cache should be readable");
assert_eq!(back.plugins.len(), 1);
assert_eq!(back.plugins[0].id, "a");
assert_eq!(back_meta.etag.as_deref(), Some("\"abc\""));
assert_eq!(back_meta.fetched_at, 1_700_000_000);
}
#[test]
fn missing_or_corrupt_cache_reads_as_none() {
let dir = tempfile::tempdir().unwrap();
assert!(read_cache(dir.path(), "nope").is_none());
std::fs::write(dir.path().join("bad.json"), b"{ not json").unwrap();
assert!(read_cache(dir.path(), "bad").is_none());
}
#[test]
fn cache_is_revalidated_on_read() {
// A tampered cache file (schema bumped by hand) must not be served.
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("x.json"), br#"{"schema":99,"plugins":[]}"#).unwrap();
assert!(read_cache(dir.path(), "x").is_none());
}
#[test]
fn drop_cache_removes_both_files() {
let dir = tempfile::tempdir().unwrap();
write_cache(dir.path(), "s", &sample_index(), &CacheMeta::default());
assert!(read_cache(dir.path(), "s").is_some());
drop_cache(dir.path(), "s");
assert!(read_cache(dir.path(), "s").is_none());
assert!(!meta_path(dir.path(), "s").exists());
}
#[test]
fn non_https_source_never_reaches_the_network() {
let s = Source {
name: "x".into(),
url: "http://example.org/i.json".into(),
public_key: None,
};
assert!(matches!(fetch(&s, None), Fetched::Failed(_)));
}
}
crates/punktfunk-host/src/store/index.rs
+570
View File
@@ -0,0 +1,570 @@
//! The plugin-store **index**: the catalog document a source serves, and the ed25519 signature
//! that makes it trustworthy (design `plugin-store.md` §3.2).
//!
//! The index is the verification gate. Each entry pins **one exact version plus that version's
//! tarball integrity hash** — so "verified on every release" is a property of the data, not a
//! promise about process: upstream can publish a newer version, but nothing in this host will
//! offer it until a re-reviewed entry lands in a signed index. There is deliberately no way to
//! express "track latest" for a catalogued plugin.
//!
//! Two rules keep parsing safe:
//! 1. **Signature before parse.** [`verify_signature`] runs over the exact bytes; only then does
//! anything here look at a field. A source with a pinned key whose signature fails is an
//! error, never a fallback to "unsigned".
//! 2. **Validate every field, drop what fails.** A malformed *entry* is dropped with a warning
//! (one bad row shouldn't blind the operator to the rest of the shelf); a malformed
//! *document* — unknown schema, non-JSON, oversized — is rejected whole.
use anyhow::{bail, Context, Result};
use serde::{Deserialize, Serialize};
/// The only index schema this host understands. A future breaking change bumps this and old
/// hosts report the source as unreadable rather than guessing at unknown semantics.
pub(crate) const SCHEMA: u32 = 1;
/// Hard cap on a fetched index body. Generous for a text catalog (the seed index is ~2 KB) and
/// small enough that a hostile or broken source can't exhaust memory.
pub(crate) const MAX_INDEX_BYTES: usize = 5 * 1024 * 1024;
/// Caps on how much of a (validly signed) index we'll keep. A curated shelf is small; these exist
/// so a compromised signing key can't turn the console into an unbounded list.
const MAX_PLUGINS: usize = 500;
const MAX_ADVISORIES: usize = 200;
// ---------------------------------------------------------------- the document
/// A source's catalog document, as served (and signed).
#[derive(Debug, Clone, Deserialize, Serialize)]
pub(crate) struct Index {
/// Document schema — must equal [`SCHEMA`].
pub schema: u32,
/// Human-readable name of the catalog ("unom official"). Display only.
#[serde(default)]
pub name: String,
/// RFC-3339 build timestamp. Display only — freshness is decided by our own fetch clock,
/// never by a field the source controls.
#[serde(default)]
pub generated: String,
/// The shelf.
#[serde(default)]
pub plugins: Vec<Entry>,
/// Revocations/advisories, checked against **installed** packages as well as catalog rows.
#[serde(default)]
pub security: Vec<Advisory>,
}
/// One catalogued plugin: exactly one installable version, pinned by integrity hash.
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct Entry {
/// The plugin's `definePlugin` id (`[a-z][a-z0-9-]*`) — also how a running plugin appears in
/// the lease registry, so the console can tell "catalogued" from "running".
pub id: String,
/// npm package name. **Must be scoped** (`@scope/name`): the scope is what maps the package
/// to [`Entry::registry`] in bun's `bunfig.toml`, so an unscoped name would be ambiguous
/// about where it installs from (design D8).
pub pkg: String,
/// The npm registry this package installs from. `https://` only.
pub registry: String,
pub title: String,
#[serde(default)]
pub description: String,
/// Lucide icon name for the console card (`[a-z0-9-]{1,48}`), matched against the console's
/// curated set; anything unknown falls back to a generic puzzle piece.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub icon: Option<String>,
#[serde(default)]
pub author: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub homepage: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub license: Option<String>,
/// **The** installable version — exact, never a range. One version per entry: the index is a
/// curated shelf, not a version archive (older releases live in the index repo's git history).
pub version: String,
/// npm integrity of that version's tarball (`sha512-<base64>`). Checked against the registry's
/// own advertised integrity before any install runs — a republish under the same version with
/// different content cannot pass.
pub integrity: String,
/// Present when a human reviewed this exact tarball. Only meaningful in the built-in source's
/// index; a third-party source can write it, which is why it never grants the "Verified" badge
/// (design D6).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub verification: Option<Verification>,
/// Minimum host version this plugin supports (semver). Incompatible rows still list, greyed.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub min_host: Option<String>,
/// Host platforms this plugin works on (`linux`/`windows`/`macos`). Empty ⇒ all.
#[serde(default)]
pub platforms: Vec<String>,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct Verification {
/// ISO date the review landed. Display only.
pub reviewed_at: String,
}
/// A revocation: "this package at these versions is known-bad". Checked against what's *installed*,
/// not just what's listed — the whole point is to reach plugins already on the box.
#[derive(Debug, Clone, Deserialize, Serialize)]
pub(crate) struct Advisory {
pub pkg: String,
/// A semver requirement (`<0.3.2`, `>=1.0.0, <1.2.0`, `*`). Unparseable ⇒ the advisory is
/// dropped rather than applied to everything.
pub versions: String,
pub reason: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub url: Option<String>,
}
// ---------------------------------------------------------------- signature
/// An ed25519 public key pinned by a source record, spelled `ed25519:<base64 of the 32 raw bytes>`.
#[derive(Debug, Clone)]
pub(crate) struct PublicKey(Vec<u8>);
impl PublicKey {
pub(crate) fn parse(s: &str) -> Result<Self> {
use base64::Engine as _;
let b64 = s
.strip_prefix("ed25519:")
.context("public key must be spelled `ed25519:<base64>`")?;
let raw = base64::engine::general_purpose::STANDARD
.decode(b64.trim())
.context("public key is not valid base64")?;
if raw.len() != 32 {
bail!("ed25519 public key must be 32 bytes, got {}", raw.len());
}
Ok(Self(raw))
}
}
/// Verify a detached ed25519 signature over the **exact** index bytes against any of the pinned
/// keys (two slots, so a key rotation is "sign with the new one, ship a host that trusts both,
/// retire the old" rather than a flag day).
///
/// `sig_text` is the `.sig` file's contents: base64, whitespace-tolerant.
pub(crate) fn verify_signature(bytes: &[u8], sig_text: &str, keys: &[PublicKey]) -> Result<()> {
use base64::Engine as _;
if keys.is_empty() {
bail!("no public key pinned for this source");
}
let sig = base64::engine::general_purpose::STANDARD
.decode(sig_text.trim())
.context("signature file is not valid base64")?;
for key in keys {
let pk = ring::signature::UnparsedPublicKey::new(&ring::signature::ED25519, &key.0);
if pk.verify(bytes, &sig).is_ok() {
return Ok(());
}
}
bail!("index signature does not verify against any pinned key")
}
// ---------------------------------------------------------------- parse + validate
impl Index {
/// Parse and validate an index document. Rejects the whole document on a structural problem;
/// drops individual entries/advisories that fail validation (with a warning) so one bad row
/// can't hide the rest of the catalog.
pub(crate) fn parse(bytes: &[u8]) -> Result<Index> {
if bytes.len() > MAX_INDEX_BYTES {
bail!("index is larger than the {MAX_INDEX_BYTES}-byte cap");
}
let mut idx: Index = serde_json::from_slice(bytes).context("index is not valid JSON")?;
if idx.schema != SCHEMA {
bail!(
"unsupported index schema {} (this host understands {SCHEMA})",
idx.schema
);
}
idx.name = sanitize(&idx.name, 64);
idx.generated = sanitize(&idx.generated, 40);
let mut seen: Vec<String> = Vec::new();
idx.plugins.truncate(MAX_PLUGINS);
idx.plugins.retain_mut(|e| match e.validate() {
Err(why) => {
tracing::warn!(pkg = %e.pkg, "dropping catalog entry: {why}");
false
}
Ok(()) => {
// Duplicate ids would make "install this one" ambiguous; first wins.
if seen.iter().any(|s| s == &e.id) {
tracing::warn!(id = %e.id, "dropping duplicate catalog entry");
false
} else {
seen.push(e.id.clone());
true
}
}
});
idx.security.truncate(MAX_ADVISORIES);
idx.security.retain_mut(|a| match a.validate() {
Err(why) => {
tracing::warn!(pkg = %a.pkg, "dropping advisory: {why}");
false
}
Ok(()) => true,
});
Ok(idx)
}
}
impl Entry {
fn validate(&mut self) -> Result<()> {
if !valid_plugin_id(&self.id) {
bail!("id must be kebab-case `[a-z][a-z0-9-]*`, ≤64");
}
if !valid_scoped_pkg(&self.pkg) {
bail!("pkg must be a scoped npm name (`@scope/name`)");
}
if !is_https(&self.registry) {
bail!("registry must be an https:// URL");
}
self.title = sanitize(&self.title, 64);
if self.title.is_empty() {
bail!("title must not be empty");
}
self.description = sanitize(&self.description, 280);
self.author = sanitize(&self.author, 64);
self.version = sanitize(&self.version, 32);
if semver::Version::parse(&self.version).is_err() {
bail!("version must be exact semver (`1.2.3`), not a range");
}
if !valid_integrity(&self.integrity) {
bail!("integrity must look like `sha512-<base64>`");
}
if let Some(icon) = &self.icon {
if !valid_icon(icon) {
self.icon = None; // cosmetic — drop it rather than the whole entry
}
}
if let Some(h) = &self.homepage {
if !is_https(h) || h.len() > 200 {
self.homepage = None;
}
}
if let Some(l) = &self.license {
self.license = Some(sanitize(l, 64)).filter(|s| !s.is_empty());
}
if let Some(v) = &self.verification {
if v.reviewed_at.len() > 32 {
self.verification = None;
}
}
if let Some(m) = &self.min_host {
if semver::Version::parse(m).is_err() {
self.min_host = None; // unusable constraint ⇒ no constraint
}
}
self.platforms
.retain(|p| matches!(p.as_str(), "linux" | "windows" | "macos"));
self.platforms.truncate(4);
Ok(())
}
/// Is this entry installable on the running host? Returns the operator-facing reason when not.
pub(crate) fn incompatible_reason(&self) -> Option<String> {
if !self.platforms.is_empty() && !self.platforms.iter().any(|p| p == HOST_PLATFORM) {
return Some(format!("requires {}", self.platforms.join(" or ")));
}
if let Some(min) = &self.min_host {
let (Ok(min), Ok(host)) = (
semver::Version::parse(min),
semver::Version::parse(host_version()),
) else {
return None;
};
if host < min {
return Some(format!("needs punktfunk {min} or newer"));
}
}
None
}
}
impl Advisory {
fn validate(&mut self) -> Result<()> {
if self.pkg.trim().is_empty() || self.pkg.len() > 214 {
bail!("advisory pkg is empty or too long");
}
semver::VersionReq::parse(&self.versions)
.context("advisory `versions` is not a semver requirement")?;
self.reason = sanitize(&self.reason, 280);
if let Some(u) = &self.url {
if !is_https(u) || u.len() > 200 {
self.url = None;
}
}
Ok(())
}
/// Does this advisory cover `pkg@version`?
pub(crate) fn matches(&self, pkg: &str, version: &str) -> bool {
if self.pkg != pkg {
return false;
}
let (Ok(req), Ok(v)) = (
semver::VersionReq::parse(&self.versions),
semver::Version::parse(version),
) else {
return false;
};
req.matches(&v)
}
}
// ---------------------------------------------------------------- small helpers
/// This host's platform token, as used in [`Entry::platforms`].
pub(crate) const HOST_PLATFORM: &str = if cfg!(target_os = "windows") {
"windows"
} else if cfg!(target_os = "macos") {
"macos"
} else {
"linux"
};
/// The running host's version — the left-hand side of every `minHost` comparison.
pub(crate) fn host_version() -> &'static str {
env!("CARGO_PKG_VERSION")
}
/// Strip control characters (a catalog string ends up in a log line and in the console UI) and
/// clamp to `max` characters.
fn sanitize(s: &str, max: usize) -> String {
s.chars()
.filter(|c| !c.is_control())
.take(max)
.collect::<String>()
.trim()
.to_string()
}
pub(crate) fn valid_plugin_id(id: &str) -> bool {
!id.is_empty()
&& id.len() <= 64
&& id.as_bytes()[0].is_ascii_lowercase()
&& id
.bytes()
.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-')
}
/// `@scope/name` with npm's safe alphabet. Deliberately stricter than npm: no URL-ish characters
/// can survive into a `bun add` argument or a `bunfig.toml` scope key.
pub(crate) fn valid_scoped_pkg(pkg: &str) -> bool {
let Some(rest) = pkg.strip_prefix('@') else {
return false;
};
if pkg.len() > 214 {
return false;
}
let Some((scope, name)) = rest.split_once('/') else {
return false;
};
let ok = |s: &str| {
!s.is_empty()
&& s.bytes().all(|b| {
b.is_ascii_lowercase() || b.is_ascii_digit() || matches!(b, b'-' | b'_' | b'.')
})
};
ok(scope) && ok(name)
}
/// The `@scope` half of a scoped package name — the key `bunfig.toml` maps to a registry.
pub(crate) fn scope_of(pkg: &str) -> Option<String> {
let rest = pkg.strip_prefix('@')?;
let (scope, _) = rest.split_once('/')?;
Some(format!("@{scope}"))
}
fn valid_icon(icon: &str) -> bool {
(1..=48).contains(&icon.len())
&& icon
.bytes()
.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-')
}
/// npm's Subresource-Integrity spelling: `<alg>-<base64>`.
fn valid_integrity(s: &str) -> bool {
if s.len() > 200 {
return false;
}
let Some((alg, b64)) = s.split_once('-') else {
return false;
};
matches!(alg, "sha512" | "sha384" | "sha256" | "sha1")
&& !b64.is_empty()
&& b64
.bytes()
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'+' | b'/' | b'='))
}
fn is_https(url: &str) -> bool {
url.starts_with("https://") && url.len() > "https://".len()
}
#[cfg(test)]
mod tests {
use super::*;
fn doc(entry_json: &str) -> Vec<u8> {
format!(r#"{{"schema":1,"name":"t","plugins":[{entry_json}]}}"#).into_bytes()
}
const GOOD: &str = r#"{"id":"rom-manager","pkg":"@punktfunk/plugin-rom-manager",
"registry":"https://git.unom.io/api/packages/unom/npm/","title":"ROM Manager",
"description":"d","author":"unom","version":"0.2.1","integrity":"sha512-AAAA",
"verification":{"reviewedAt":"2026-07-19"},"platforms":["linux","windows"]}"#;
#[test]
fn parses_a_good_entry() {
let idx = Index::parse(&doc(GOOD)).unwrap();
assert_eq!(idx.plugins.len(), 1);
let e = &idx.plugins[0];
assert_eq!(e.pkg, "@punktfunk/plugin-rom-manager");
assert_eq!(e.version, "0.2.1");
assert!(e.verification.is_some());
}
#[test]
fn rejects_unknown_schema_and_bad_json() {
assert!(Index::parse(br#"{"schema":2,"plugins":[]}"#).is_err());
assert!(Index::parse(b"not json").is_err());
}
#[test]
fn drops_invalid_entries_but_keeps_the_rest() {
let bad_unscoped = GOOD.replace("@punktfunk/plugin-rom-manager", "punktfunk-plugin-x");
let body = format!(r#"{{"schema":1,"plugins":[{bad_unscoped},{GOOD}]}}"#);
let idx = Index::parse(body.as_bytes()).unwrap();
assert_eq!(idx.plugins.len(), 1, "unscoped pkg must be dropped (D8)");
assert_eq!(idx.plugins[0].id, "rom-manager");
}
#[test]
fn rejects_version_ranges_and_http_registries() {
assert!(Index::parse(&doc(
&GOOD.replace(r#""version":"0.2.1""#, r#""version":"^0.2.1""#)
))
.unwrap()
.plugins
.is_empty());
assert!(Index::parse(&doc(
&GOOD.replace("https://git.unom.io", "http://git.unom.io")
))
.unwrap()
.plugins
.is_empty());
}
#[test]
fn drops_duplicate_ids() {
let body = format!(r#"{{"schema":1,"plugins":[{GOOD},{GOOD}]}}"#);
assert_eq!(Index::parse(body.as_bytes()).unwrap().plugins.len(), 1);
}
#[test]
fn sanitizes_control_characters_in_display_fields() {
let e = GOOD.replace("ROM Manager", "RO\\u0007M");
let idx = Index::parse(&doc(&e)).unwrap();
assert_eq!(idx.plugins[0].title, "ROM");
}
#[test]
fn advisory_matching_is_semver_ranged() {
let mut a = Advisory {
pkg: "@x/y".into(),
versions: "<0.3.2".into(),
reason: "bad".into(),
url: None,
};
a.validate().unwrap();
assert!(a.matches("@x/y", "0.3.1"));
assert!(!a.matches("@x/y", "0.3.2"));
assert!(!a.matches("@other/z", "0.3.1"));
}
#[test]
fn advisory_with_unparseable_range_is_dropped() {
let body = r#"{"schema":1,"plugins":[],"security":[
{"pkg":"@x/y","versions":"not a range","reason":"r"}]}"#;
assert!(Index::parse(body.as_bytes()).unwrap().security.is_empty());
}
#[test]
fn scope_extraction() {
assert_eq!(scope_of("@punktfunk/plugin-x").unwrap(), "@punktfunk");
assert_eq!(scope_of("@a/b").unwrap(), "@a");
assert!(scope_of("punktfunk-plugin-x").is_none());
}
#[test]
fn integrity_shape() {
assert!(valid_integrity("sha512-abcABC123+/="));
assert!(!valid_integrity("md5-abc"));
assert!(!valid_integrity("sha512-"));
assert!(!valid_integrity("sha512"));
}
#[test]
fn signature_verifies_only_over_exact_bytes_and_pinned_keys() {
use ring::signature::KeyPair as _;
let rng = ring::rand::SystemRandom::new();
let pkcs8 = ring::signature::Ed25519KeyPair::generate_pkcs8(&rng).unwrap();
let kp = ring::signature::Ed25519KeyPair::from_pkcs8(pkcs8.as_ref()).unwrap();
let key = {
use base64::Engine as _;
PublicKey::parse(&format!(
"ed25519:{}",
base64::engine::general_purpose::STANDARD.encode(kp.public_key().as_ref())
))
.unwrap()
};
let body = br#"{"schema":1,"plugins":[]}"#;
let sig = {
use base64::Engine as _;
base64::engine::general_purpose::STANDARD.encode(kp.sign(body).as_ref())
};
assert!(verify_signature(body, &sig, std::slice::from_ref(&key)).is_ok());
// whitespace in the .sig file is tolerated
assert!(verify_signature(body, &format!("{sig}\n"), std::slice::from_ref(&key)).is_ok());
// one byte of tampering fails
assert!(verify_signature(
br#"{"schema":1,"plugins":[ ]}"#,
&sig,
std::slice::from_ref(&key)
)
.is_err());
// a different key fails
let other = ring::signature::Ed25519KeyPair::from_pkcs8(
ring::signature::Ed25519KeyPair::generate_pkcs8(&rng)
.unwrap()
.as_ref(),
)
.unwrap();
let other_key = {
use base64::Engine as _;
PublicKey::parse(&format!(
"ed25519:{}",
base64::engine::general_purpose::STANDARD.encode(other.public_key().as_ref())
))
.unwrap()
};
assert!(verify_signature(body, &sig, &[other_key]).is_err());
// no pinned key at all fails closed
assert!(verify_signature(body, &sig, &[]).is_err());
}
#[test]
fn public_key_parsing_rejects_junk() {
assert!(PublicKey::parse("nope").is_err());
assert!(PublicKey::parse("ed25519:!!!").is_err());
assert!(PublicKey::parse("ed25519:AAAA").is_err()); // wrong length
}
}
crates/punktfunk-host/src/store/jobs.rs
+687
View File
@@ -0,0 +1,687 @@
//! Install / uninstall **jobs** (design `plugin-store.md` §4.3).
//!
//! A package operation takes tens of seconds, so the API hands back a job id immediately (202) and
//! the console polls. One at a time: `bun add` and `bun remove` share a lockfile and a
//! `node_modules` tree, so a second concurrent op is a corruption bug waiting to happen — a
//! request that arrives while a job runs gets 409, not a queue.
//!
//! The pipeline, and why each step is there:
//!
//! ```text
//! resolve → what exactly are we installing (catalog entry ⇒ pkg@version+integrity, or a raw spec)
//! verify → the registry's advertised integrity for that version MUST equal the index pin
//! install → spawn the runner CLI (`bun add`), streaming its output into the job log
//! check → the version on disk is the version we pinned, else roll back
//! record → provenance manifest, so the tier stays visible forever
//! restart → the runner rediscovers units only at startup, so activation is a restart
//! ```
//!
//! **verify** is the step that makes "verified" mean something. A reviewed entry pins a tarball
//! hash; if the registry now advertises a different hash for that same version, someone republished
//! it after review and the install is refused before any code is fetched, let alone run. Tier-3
//! (raw spec) installs skip it — there is no pin to check against, and that asymmetry *is* the
//! tier.
use super::index::{scope_of, Entry};
use super::manifest::{self, Record, Tier};
use anyhow::{bail, Context, Result};
use std::collections::VecDeque;
use std::io::{BufRead, BufReader, Read};
use std::process::{Command, Stdio};
use std::sync::Mutex;
use std::time::{Duration, Instant};
/// How long a package op may run before we kill it. `bun add` over a cold cache on a slow link is
/// the worst case; five minutes is far past it.
const JOB_TIMEOUT: Duration = Duration::from_secs(300);
/// How many finished jobs we keep for the console to read back.
const JOB_HISTORY: usize = 20;
/// Lines of runner output retained per job (the console shows a tail).
const LOG_LINES: usize = 200;
// ---------------------------------------------------------------- job records
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, utoipa::ToSchema)]
#[serde(rename_all = "lowercase")]
pub(crate) enum State {
Running,
Done,
Failed,
}
/// A job as the console sees it. Field names are snake_case like the rest of the management API
/// (the *file* formats — index, sources, manifest — follow npm's camelCase instead).
#[derive(Debug, Clone, serde::Serialize, utoipa::ToSchema)]
pub(crate) struct Job {
pub id: String,
/// `install` or `uninstall`.
pub kind: String,
/// What the operator asked for — a package name, or the raw spec they typed.
pub target: String,
pub state: State,
/// Coarse step name, for a progress line the operator can read.
pub phase: String,
/// Tail of the runner's combined stdout/stderr.
pub log: Vec<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
pub started_at: u64,
#[serde(skip_serializing_if = "Option::is_none")]
pub finished_at: Option<u64>,
}
struct Jobs {
jobs: VecDeque<Job>,
counter: u64,
}
fn jobs() -> &'static Mutex<Jobs> {
static JOBS: std::sync::OnceLock<Mutex<Jobs>> = std::sync::OnceLock::new();
JOBS.get_or_init(|| {
Mutex::new(Jobs {
jobs: VecDeque::new(),
counter: 0,
})
})
}
fn lock() -> std::sync::MutexGuard<'static, Jobs> {
jobs().lock().unwrap_or_else(|e| e.into_inner())
}
/// Start tracking a job, refusing if one is already in flight (single-flight, §4.3).
fn begin(kind: &str, target: &str) -> Result<String> {
let mut g = lock();
if let Some(active) = g.jobs.iter().find(|j| j.state == State::Running) {
bail!(
"another plugin operation is already running ({} {})",
active.kind,
active.target
);
}
g.counter += 1;
let id = format!("job-{}-{}", super::catalog::unix_now(), g.counter);
let job = Job {
id: id.clone(),
kind: kind.to_string(),
target: target.to_string(),
state: State::Running,
phase: "queued".into(),
log: Vec::new(),
error: None,
started_at: super::catalog::unix_now(),
finished_at: None,
};
g.jobs.push_back(job);
while g.jobs.len() > JOB_HISTORY {
g.jobs.pop_front();
}
Ok(id)
}
fn update(id: &str, f: impl FnOnce(&mut Job)) {
let mut g = lock();
if let Some(job) = g.jobs.iter_mut().find(|j| j.id == id) {
f(job);
}
}
fn set_phase(id: &str, phase: &str) {
tracing::info!(job = %id, phase, "plugin store job");
update(id, |j| j.phase = phase.to_string());
}
fn log_line(id: &str, line: String) {
update(id, |j| {
if j.log.len() >= LOG_LINES {
j.log.remove(0);
}
j.log.push(line);
});
}
fn finish(id: &str, result: Result<()>) {
update(id, |j| {
j.finished_at = Some(super::catalog::unix_now());
match &result {
Ok(()) => {
j.state = State::Done;
j.phase = "done".into();
}
Err(e) => {
j.state = State::Failed;
j.error = Some(format!("{e:#}"));
}
}
});
match result {
Ok(()) => tracing::info!(job = %id, "plugin store job finished"),
Err(e) => tracing::warn!(job = %id, "plugin store job failed: {e:#}"),
}
}
/// One job by id.
pub(crate) fn get(id: &str) -> Option<Job> {
lock().jobs.iter().find(|j| j.id == id).cloned()
}
/// Recent jobs, newest last.
pub(crate) fn list() -> Vec<Job> {
lock().jobs.iter().cloned().collect()
}
/// Is a job in flight? (The console disables install buttons on this.)
pub(crate) fn busy() -> bool {
lock().jobs.iter().any(|j| j.state == State::Running)
}
// ---------------------------------------------------------------- install plans
/// A fully resolved install: everything the executor needs, with the trust decision already made.
pub(crate) struct Plan {
/// Package name without a version.
pub pkg: Option<String>,
/// The argument handed to `bun add` — `pkg@version` for a catalogued entry, the raw spec
/// otherwise.
pub spec: String,
pub version: Option<String>,
/// `(scope, registry_url)` to map in the plugins dir's `bunfig.toml`.
pub registry: Option<(String, String)>,
pub integrity: Option<String>,
pub tier: Tier,
pub source: Option<String>,
pub entry_id: Option<String>,
}
impl Plan {
/// From a catalog entry: exact version, pinned integrity, scope→registry mapping.
pub(crate) fn from_entry(entry: &Entry, source: &str, verified: bool) -> Result<Plan> {
let scope = scope_of(&entry.pkg).context("catalog entry package must be scoped")?;
Ok(Plan {
pkg: Some(entry.pkg.clone()),
spec: format!("{}@{}", entry.pkg, entry.version),
version: Some(entry.version.clone()),
registry: Some((scope, entry.registry.clone())),
integrity: Some(entry.integrity.clone()),
tier: if verified {
Tier::Verified
} else {
Tier::External
},
source: Some(source.to_string()),
entry_id: Some(entry.id.clone()),
})
}
/// From a raw package spec typed into the danger dialog. No pin, no review, no shelf.
pub(crate) fn from_spec(spec: &str) -> Result<Plan> {
let spec = validate_spec(spec)?;
Ok(Plan {
pkg: parse_spec_pkg(&spec),
spec,
version: None,
registry: None,
integrity: None,
tier: Tier::Unverified,
source: None,
entry_id: None,
})
}
}
/// Accept only specs we're willing to hand to `bun add`.
///
/// This is not shell quoting (we always exec with an argument vector, never a shell) — it is about
/// what the *package manager* will do with the string. Two real hazards: a leading `-` turns the
/// operand into a flag, and `file:` specs would let a console session install code from anywhere on
/// the host's filesystem, which is a different (and larger) capability than "install a package".
/// Local paths remain available through the CLI, which is where local development belongs.
fn validate_spec(spec: &str) -> Result<String> {
let s = spec.trim();
if s.is_empty() {
bail!("empty package spec");
}
if s.len() > 400 {
bail!("package spec is too long");
}
if s.starts_with('-') {
bail!("a package spec cannot start with '-'");
}
if s.chars().any(|c| c.is_whitespace() || c.is_control()) {
bail!("a package spec cannot contain whitespace or control characters");
}
let lower = s.to_ascii_lowercase();
for bad in ["file:", "link:", "portal:"] {
if lower.starts_with(bad) {
bail!("`{bad}` specs are not installable from the console — use the `punktfunk-host plugins add` CLI");
}
}
// URL-ish specs: only https and git+https. (http:// and git:// are unauthenticated transports.)
if lower.contains("://")
&& !(lower.starts_with("https://") || lower.starts_with("git+https://"))
{
bail!("only https:// and git+https:// URLs are accepted");
}
Ok(s.to_string())
}
/// Best-effort package name from an npm-style spec (`@scope/name`, `@scope/name@1.2.3`, `name@1`).
/// `None` for URL/git specs — those are identified after the fact by diffing `node_modules`.
fn parse_spec_pkg(spec: &str) -> Option<String> {
if spec.contains("://") {
return None;
}
if let Some(rest) = spec.strip_prefix('@') {
// Scoped: the version separator is the '@' AFTER the scope's slash.
let (scope_and_name, _) = match rest.split_once('/') {
Some((scope, tail)) => match tail.split_once('@') {
Some((name, ver)) => (format!("@{scope}/{name}"), Some(ver)),
None => (format!("@{scope}/{tail}"), None),
},
None => return None,
};
return Some(scope_and_name);
}
Some(
spec.split_once('@')
.map(|(n, _)| n)
.unwrap_or(spec)
.to_string(),
)
}
// ---------------------------------------------------------------- execution
/// Kick off an install. Returns the job id; the work runs on a detached thread.
pub(crate) fn spawn_install(plan: Plan) -> Result<String> {
let target = plan.pkg.clone().unwrap_or_else(|| plan.spec.clone());
let id = begin("install", &target)?;
let job_id = id.clone();
std::thread::Builder::new()
.name("pf-store-install".into())
.spawn(move || {
let result = run_install(&job_id, plan);
finish(&job_id, result);
crate::events::emit(crate::events::EventKind::StoreChanged);
})
.context("spawn the plugin-install worker")?;
Ok(id)
}
/// Kick off an uninstall.
pub(crate) fn spawn_uninstall(pkg: String) -> Result<String> {
if super::valid_installed_pkg(&pkg).is_err() {
bail!("not an installable plugin package name");
}
let id = begin("uninstall", &pkg)?;
let job_id = id.clone();
std::thread::Builder::new()
.name("pf-store-uninstall".into())
.spawn(move || {
let result = run_uninstall(&job_id, &pkg);
finish(&job_id, result);
crate::events::emit(crate::events::EventKind::StoreChanged);
})
.context("spawn the plugin-uninstall worker")?;
Ok(id)
}
fn run_install(id: &str, plan: Plan) -> Result<()> {
let dir = super::plugins_dir();
// ---- verify: the pin must still describe what the registry serves ------------------------
if let (Some(integrity), Some(pkg), Some(version), Some((_, registry))) = (
plan.integrity.as_deref(),
plan.pkg.as_deref(),
plan.version.as_deref(),
plan.registry.as_ref(),
) {
set_phase(id, "verifying");
let advertised = registry_integrity(registry, pkg, version)
.with_context(|| format!("check {pkg}@{version} against {registry}"))?;
if advertised != integrity {
bail!(
"integrity mismatch for {pkg}@{version}: the catalog pins {integrity} but the \
registry now serves {advertised}. This version was republished after it was \
reviewed — refusing to install."
);
}
log_line(id, format!("integrity ok: {pkg}@{version}"));
}
// ---- install ------------------------------------------------------------------------------
set_phase(id, "installing");
let before = super::installed_packages(&dir);
let mut args = vec!["add".to_string(), plan.spec.clone()];
if plan.version.is_some() {
// Pin the dependency range too, so a later `bun install` in this tree can't drift off the
// reviewed version.
args.push("--exact".into());
}
if let Some((scope, url)) = &plan.registry {
args.push("--registry".into());
args.push(format!("{scope}={url}"));
}
if !plan.spec.starts_with("@punktfunk/") {
// The runner CLI's supply-chain gate refuses anything resolving off Punktfunk's own
// registry unless told otherwise. Here the operator already made that decision — either by
// adding the source, or in the danger dialog — so pass the flag rather than making the gate
// unable to express "yes, deliberately".
args.push("--allow-public-registry".into());
}
args.push("--plugins".into());
args.push(dir.to_string_lossy().into_owned());
run_runner(id, &args)?;
// ---- check: is what landed what we asked for? ---------------------------------------------
set_phase(id, "checking");
let after = super::installed_packages(&dir);
let added: Vec<_> = after
.iter()
.filter(|p| !before.iter().any(|b| b.pkg == p.pkg))
.collect();
// For a catalogued install we know the package name; for a URL/git spec we learn it here.
let pkg = plan
.pkg
.clone()
.or_else(|| added.first().map(|p| p.pkg.clone()))
.context(
"the install finished but no new plugin package appeared — is this package a \
punktfunk plugin? (it must be named `@scope/plugin-*` or `punktfunk-plugin-*`)",
)?;
let installed = after
.iter()
.find(|p| p.pkg == pkg)
.with_context(|| format!("{pkg} is not present after install"))?;
if let (Some(want), Some(got)) = (plan.version.as_deref(), installed.version.as_deref()) {
if want != got {
// Roll back rather than leave an unreviewed version installed under a verified badge.
log_line(id, format!("rolling back: expected {want}, found {got}"));
set_phase(id, "rolling back");
let _ = run_runner(
id,
&[
"remove".to_string(),
pkg.clone(),
"--plugins".to_string(),
dir.to_string_lossy().into_owned(),
],
);
bail!("installed {pkg}@{got} but the catalog pinned {want} — rolled back");
}
}
// ---- record + activate ---------------------------------------------------------------------
set_phase(id, "recording");
manifest::record(
&dir,
&pkg,
Record {
tier: plan.tier,
source: plan.source.clone(),
entry_id: plan.entry_id.clone(),
version: installed.version.clone().or(plan.version.clone()),
spec: (plan.tier == Tier::Unverified).then(|| plan.spec.clone()),
installed_at: Some(manifest::now_stamp()),
},
)
.context("record install provenance")?;
restart_runner(id);
Ok(())
}
fn run_uninstall(id: &str, pkg: &str) -> Result<()> {
let dir = super::plugins_dir();
set_phase(id, "removing");
run_runner(
id,
&[
"remove".to_string(),
pkg.to_string(),
"--plugins".to_string(),
dir.to_string_lossy().into_owned(),
],
)?;
set_phase(id, "recording");
manifest::forget(&dir, pkg).context("update install provenance")?;
restart_runner(id);
Ok(())
}
/// Restart the runner so it rediscovers units. Best-effort and non-fatal: the package IS installed
/// at this point, so a restart failure must not present as an install failure — it presents as
/// "installed, runner needs a nudge", which the console says out loud.
fn restart_runner(id: &str) {
set_phase(id, "restarting runner");
match crate::plugins::restart_runtime() {
Ok(true) => log_line(id, "plugin runner restarted".into()),
Ok(false) => log_line(
id,
"the plugin runner is not enabled — enable it to start this plugin".into(),
),
Err(e) => log_line(id, format!("could not restart the plugin runner: {e:#}")),
}
}
/// Run the bun runner CLI with `args`, streaming both streams into the job log.
fn run_runner(id: &str, args: &[String]) -> Result<()> {
let (program, prefix) = crate::plugins::runner_command()?;
tracing::info!(job = %id, program = %program.display(), ?args, "spawning the plugin runner");
let mut child = Command::new(&program)
.args(&prefix)
.args(args)
// No stdin: `bun add` must never sit waiting on a prompt inside a service.
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.with_context(|| format!("run the plugin runner ({})", program.display()))?;
// Pump both streams concurrently: bun writes progress to one and errors to the other, and a
// full pipe on either would block the child.
let mut pumps = Vec::new();
if let Some(out) = child.stdout.take() {
let job = id.to_string();
pumps.push(std::thread::spawn(move || pump(&job, out)));
}
if let Some(err) = child.stderr.take() {
let job = id.to_string();
pumps.push(std::thread::spawn(move || pump(&job, err)));
}
let deadline = Instant::now() + JOB_TIMEOUT;
let status = loop {
match child.try_wait().context("wait for the plugin runner")? {
Some(status) => break status,
None if Instant::now() >= deadline => {
let _ = child.kill();
let _ = child.wait();
bail!(
"the plugin runner timed out after {}s",
JOB_TIMEOUT.as_secs()
);
}
None => std::thread::sleep(Duration::from_millis(150)),
}
};
for p in pumps {
let _ = p.join();
}
if !status.success() {
bail!(
"the plugin runner exited with status {} — see the job log",
status.code().unwrap_or(-1)
);
}
Ok(())
}
/// Copy one child stream into the job log, line by line.
fn pump(job: &str, stream: impl std::io::Read) {
for line in BufReader::new(stream).lines().map_while(Result::ok) {
let line = line.trim_end().to_string();
if !line.is_empty() {
log_line(job, line);
}
}
}
// ---------------------------------------------------------------- registry preflight
/// The integrity hash the registry itself advertises for `pkg@version`.
///
/// This is the check that gives the pin teeth. npm clients already refuse a tarball whose bytes
/// don't match the registry's advertised hash, so the remaining attack is a *republish* — same
/// version, new content, new hash. Comparing the registry's current hash against the reviewed one
/// catches exactly that, before anything is downloaded.
fn registry_integrity(registry: &str, pkg: &str, version: &str) -> Result<String> {
let base = registry.trim_end_matches('/');
// npm registry convention: the scope separator is percent-encoded in the packument path.
let url = format!("{base}/{}", pkg.replace('/', "%2f"));
let agent = ureq::AgentBuilder::new()
.timeout(Duration::from_secs(20))
.redirects(3)
.user_agent(&format!("punktfunk-host/{}", super::index::host_version()))
.build();
let resp = agent
.get(&url)
.set("Accept", "application/json")
.call()
.map_err(|e| match e {
ureq::Error::Status(404, _) => {
anyhow::anyhow!("the registry does not know this package")
}
other => anyhow::anyhow!("registry request failed: {other}"),
})?;
let mut body = Vec::new();
resp.into_reader()
.take(16 * 1024 * 1024)
.read_to_end(&mut body)
.context("read the registry response")?;
let doc: serde_json::Value =
serde_json::from_slice(&body).context("registry returned invalid JSON")?;
let dist = doc
.get("versions")
.and_then(|v| v.get(version))
.and_then(|v| v.get("dist"))
.with_context(|| format!("the registry has no version {version} of this package"))?;
dist.get("integrity")
.and_then(|i| i.as_str())
.map(str::to_string)
.context("the registry did not advertise an integrity hash for this version")
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn spec_validation_refuses_the_dangerous_shapes() {
assert!(validate_spec("@scope/plugin-x").is_ok());
assert!(validate_spec("@scope/plugin-x@1.2.3").is_ok());
assert!(validate_spec("punktfunk-plugin-x").is_ok());
assert!(validate_spec("https://e.org/p.tgz").is_ok());
assert!(validate_spec("git+https://e.org/p.git").is_ok());
assert!(validate_spec("").is_err());
assert!(validate_spec(" ").is_err());
// a leading dash would be parsed by bun as a flag, not an operand
assert!(validate_spec("--production").is_err());
assert!(validate_spec("a b").is_err());
// local-path specs are CLI-only: installing from anywhere on the filesystem is a bigger
// capability than installing a package
assert!(validate_spec("file:/etc/passwd").is_err());
assert!(validate_spec("link:../evil").is_err());
// unauthenticated transports
assert!(validate_spec("http://e.org/p.tgz").is_err());
assert!(validate_spec("git://e.org/p.git").is_err());
}
#[test]
fn spec_package_name_parsing() {
assert_eq!(parse_spec_pkg("@a/b").as_deref(), Some("@a/b"));
assert_eq!(parse_spec_pkg("@a/b@1.2.3").as_deref(), Some("@a/b"));
assert_eq!(parse_spec_pkg("plain").as_deref(), Some("plain"));
assert_eq!(parse_spec_pkg("plain@2").as_deref(), Some("plain"));
// URL specs are identified by diffing node_modules after the install, not by parsing
assert_eq!(parse_spec_pkg("https://e.org/p.tgz"), None);
}
#[test]
fn plan_from_entry_pins_version_registry_and_integrity() {
let idx = super::super::index::Index::parse(
br#"{"schema":1,"plugins":[{"id":"rom-manager","pkg":"@punktfunk/plugin-rom-manager",
"registry":"https://git.unom.io/api/packages/unom/npm/","title":"ROM",
"version":"0.3.0","integrity":"sha512-AAAA"}]}"#,
)
.unwrap();
let plan = Plan::from_entry(&idx.plugins[0], "unom", true).unwrap();
assert_eq!(plan.spec, "@punktfunk/plugin-rom-manager@0.3.0");
assert_eq!(plan.version.as_deref(), Some("0.3.0"));
assert_eq!(plan.tier, Tier::Verified);
assert_eq!(
plan.registry.as_ref().unwrap().0,
"@punktfunk",
"the scope drives the bunfig registry mapping"
);
assert_eq!(plan.integrity.as_deref(), Some("sha512-AAAA"));
// The same entry from an operator-added source is External, never Verified (D6).
let ext = Plan::from_entry(&idx.plugins[0], "retro-hub", false).unwrap();
assert_eq!(ext.tier, Tier::External);
assert_eq!(ext.source.as_deref(), Some("retro-hub"));
}
#[test]
fn plan_from_spec_is_unverified_and_unpinned() {
let plan = Plan::from_spec(" @someone/punktfunk-plugin-x ").unwrap();
assert_eq!(plan.tier, Tier::Unverified);
assert_eq!(plan.spec, "@someone/punktfunk-plugin-x");
assert!(plan.version.is_none());
assert!(
plan.integrity.is_none(),
"nothing to check a raw spec against"
);
assert!(plan.registry.is_none());
}
/// `begin` is process-global and single-flight, so tests that create jobs must not overlap.
fn exclusive() -> std::sync::MutexGuard<'static, ()> {
static M: Mutex<()> = Mutex::new(());
M.lock().unwrap_or_else(|e| e.into_inner())
}
#[test]
fn single_flight_refuses_a_second_job() {
let _g = exclusive();
let first = begin("install", "@a/b").expect("first job starts");
assert!(begin("install", "@c/d").is_err(), "second must be refused");
assert!(busy());
finish(&first, Ok(()));
assert!(!busy());
let second = begin("install", "@c/d").expect("a finished job frees the slot");
finish(&second, Ok(()));
}
#[test]
fn job_log_is_bounded_and_tails() {
let _g = exclusive();
let id = begin("install", "@log/test").unwrap();
for i in 0..(LOG_LINES + 50) {
log_line(&id, format!("line {i}"));
}
let job = get(&id).unwrap();
assert_eq!(job.log.len(), LOG_LINES);
assert_eq!(job.log.last().unwrap(), &format!("line {}", LOG_LINES + 49));
finish(&id, Err(anyhow::anyhow!("boom")));
let job = get(&id).unwrap();
assert_eq!(job.state, State::Failed);
assert!(job.error.unwrap().contains("boom"));
assert!(job.finished_at.is_some());
}
}
crates/punktfunk-host/src/store/manifest.rs
+246
View File
@@ -0,0 +1,246 @@
//! The **provenance manifest**: how a plugin got onto this box, remembered for as long as it is
//! installed (design `plugin-store.md` §4.5).
//!
//! `node_modules` knows *what* is installed; it has no idea whether the operator picked a reviewed
//! entry off the official shelf or pasted a tarball URL into the danger dialog. That distinction is
//! exactly what the console has to keep showing — an unverified plugin must stay visibly
//! unverified in the Installed list and in the header above its own UI, long after the install
//! dialog is forgotten.
//!
//! `<plugins_dir>/install-manifest.json`, written by the host (which is the only thing that ever
//! performs a *store* install). **Absence is meaningful**: a package with no manifest entry was
//! installed by the CLI, and is reported as [`Tier::Cli`] rather than being silently promoted.
use anyhow::{Context, Result};
use serde::{Deserialize, Serialize};
use std::collections::BTreeMap;
/// How a plugin came to be installed. Ordered by how much review stands behind it.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub(crate) enum Tier {
/// From the built-in source's index: unom reviewed this exact tarball.
Verified,
/// From an operator-added source's index: pinned and integrity-checked, but curated by
/// somebody else. Attribution, never the badge (D6).
External,
/// Installed from a raw package spec through the danger dialog. Nobody reviewed it.
Unverified,
/// No manifest entry — `punktfunk-host plugins add` put it there.
Cli,
}
impl Tier {
pub(crate) fn as_str(self) -> &'static str {
match self {
Tier::Verified => "verified",
Tier::External => "external",
Tier::Unverified => "unverified",
Tier::Cli => "cli",
}
}
}
/// One remembered install.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct Record {
pub tier: Tier,
/// The catalog source's slug, when the install came off a shelf.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub source: Option<String>,
/// The catalog entry id, when there was one — lets the console re-associate an installed
/// package with its shelf row even if the package name is unusual.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub entry_id: Option<String>,
/// The version we installed (as pinned). The *live* version always comes from disk; this is
/// what we asked for.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub version: Option<String>,
/// The raw spec, for [`Tier::Unverified`] installs — the only record of what the operator
/// actually typed.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub spec: Option<String>,
/// RFC-3339 install time (wall clock — display only).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub installed_at: Option<String>,
}
#[derive(Serialize, Deserialize)]
struct ManifestFile {
schema: u32,
/// Keyed by npm package name. `BTreeMap` so the file has a stable, diffable order.
#[serde(default)]
plugins: BTreeMap<String, Record>,
}
impl Default for ManifestFile {
fn default() -> Self {
Self {
schema: 1,
plugins: BTreeMap::new(),
}
}
}
fn manifest_path(plugins_dir: &std::path::Path) -> std::path::PathBuf {
plugins_dir.join("install-manifest.json")
}
/// Read the manifest. A missing or corrupt file reads as empty — every package then reports as
/// CLI-installed, which is the honest answer when we have no provenance to show.
pub(crate) fn load(plugins_dir: &std::path::Path) -> BTreeMap<String, Record> {
let Ok(bytes) = std::fs::read(manifest_path(plugins_dir)) else {
return BTreeMap::new();
};
match serde_json::from_slice::<ManifestFile>(&bytes) {
Ok(f) if f.schema == 1 => f.plugins,
Ok(f) => {
tracing::warn!(
schema = f.schema,
"unknown install-manifest schema — ignoring"
);
BTreeMap::new()
}
Err(e) => {
tracing::warn!("install-manifest.json is unreadable ({e}) — treating as empty");
BTreeMap::new()
}
}
}
/// Record (or replace) one package's provenance.
pub(crate) fn record(plugins_dir: &std::path::Path, pkg: &str, rec: Record) -> Result<()> {
let mut plugins = load(plugins_dir);
plugins.insert(pkg.to_string(), rec);
write(plugins_dir, plugins)
}
/// Forget a package (on uninstall). Silent if it wasn't recorded.
pub(crate) fn forget(plugins_dir: &std::path::Path, pkg: &str) -> Result<()> {
let mut plugins = load(plugins_dir);
if plugins.remove(pkg).is_none() {
return Ok(());
}
write(plugins_dir, plugins)
}
fn write(plugins_dir: &std::path::Path, plugins: BTreeMap<String, Record>) -> Result<()> {
std::fs::create_dir_all(plugins_dir)
.with_context(|| format!("create {}", plugins_dir.display()))?;
let json = serde_json::to_string_pretty(&ManifestFile { schema: 1, plugins })
.context("serialize install-manifest.json")?;
let path = manifest_path(plugins_dir);
let tmp = path.with_extension("json.tmp");
std::fs::write(&tmp, format!("{json}\n"))
.with_context(|| format!("write {}", tmp.display()))?;
std::fs::rename(&tmp, &path).with_context(|| format!("replace {}", path.display()))?;
Ok(())
}
/// Wall-clock RFC-3339-ish stamp for [`Record::installed_at`]. Display only — nothing compares
/// these, so a clock jump costs a cosmetic oddity, never a decision.
pub(crate) fn now_stamp() -> String {
let secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0);
// Civil-date arithmetic from a Unix timestamp (days since epoch → y/m/d), so this needs no
// date crate. UTC, second resolution.
let (days, rem) = ((secs / 86_400) as i64, secs % 86_400);
let (h, mi, s) = (rem / 3600, (rem % 3600) / 60, rem % 60);
let z = days + 719_468;
let era = z.div_euclid(146_097);
let doe = z.rem_euclid(146_097);
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365;
let y = yoe + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let d = doy - (153 * mp + 2) / 5 + 1;
let m = if mp < 10 { mp + 3 } else { mp - 9 };
let y = if m <= 2 { y + 1 } else { y };
format!("{y:04}-{m:02}-{d:02}T{h:02}:{mi:02}:{s:02}Z")
}
#[cfg(test)]
mod tests {
use super::*;
fn rec(tier: Tier) -> Record {
Record {
tier,
source: Some("unom".into()),
entry_id: Some("rom-manager".into()),
version: Some("0.2.1".into()),
spec: None,
installed_at: Some(now_stamp()),
}
}
#[test]
fn round_trips_and_absence_means_cli() {
let dir = tempfile::tempdir().unwrap();
assert!(load(dir.path()).is_empty(), "no file ⇒ nothing recorded");
record(
dir.path(),
"@punktfunk/plugin-rom-manager",
rec(Tier::Verified),
)
.unwrap();
let m = load(dir.path());
let r = m.get("@punktfunk/plugin-rom-manager").unwrap();
assert_eq!(r.tier, Tier::Verified);
assert_eq!(r.version.as_deref(), Some("0.2.1"));
// A package we never recorded has no entry — the caller reports it as CLI-installed.
assert!(m.get("punktfunk-plugin-other").is_none());
}
#[test]
fn tier_survives_a_reinstall_at_a_new_tier() {
let dir = tempfile::tempdir().unwrap();
record(dir.path(), "@x/y", rec(Tier::Verified)).unwrap();
record(dir.path(), "@x/y", rec(Tier::Unverified)).unwrap();
assert_eq!(load(dir.path()).get("@x/y").unwrap().tier, Tier::Unverified);
}
#[test]
fn forget_removes_only_the_named_package() {
let dir = tempfile::tempdir().unwrap();
record(dir.path(), "@x/y", rec(Tier::Verified)).unwrap();
record(dir.path(), "@x/z", rec(Tier::External)).unwrap();
forget(dir.path(), "@x/y").unwrap();
let m = load(dir.path());
assert!(m.get("@x/y").is_none());
assert!(m.get("@x/z").is_some());
forget(dir.path(), "@not/here").unwrap(); // no-op, no error
}
#[test]
fn corrupt_manifest_reads_as_empty_not_an_error() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("install-manifest.json"), b"{ not json").unwrap();
assert!(load(dir.path()).is_empty());
}
#[test]
fn tier_json_spelling_is_stable() {
// The console keys its badges off these strings.
assert_eq!(
serde_json::to_string(&Tier::Unverified).unwrap(),
"\"unverified\""
);
assert_eq!(Tier::Verified.as_str(), "verified");
}
#[test]
fn stamp_is_rfc3339_shaped() {
let s = now_stamp();
assert_eq!(s.len(), 20, "{s}");
assert!(s.ends_with('Z'));
assert_eq!(&s[4..5], "-");
assert_eq!(&s[10..11], "T");
// A known epoch value, to catch the civil-date arithmetic drifting.
assert!(s > "2026-01-01T00:00:00Z".to_string(), "{s}");
}
}
crates/punktfunk-host/src/store/sources.rs
+286
View File
@@ -0,0 +1,286 @@
//! Catalog **sources** — where the store's shelves come from (design `plugin-store.md` §3.1).
//!
//! One source = one URL serving a signed index. The **official** source is compiled in (URL plus
//! two pinned key slots) and cannot be edited or removed; operator-added sources live in
//! `<config_dir>/plugin-sources.json`.
//!
//! Adding a source is itself a trust decision, made once: a source's entries become installable
//! on this host. That is why they carry attribution and warning styling in the console but never
//! the "Verified" badge — verification is unom reviewing a specific tarball, and it is not
//! transferable to a third-party curator (D6).
use super::index::PublicKey;
use anyhow::{bail, Context, Result};
use serde::{Deserialize, Serialize};
/// The built-in source's slug. Reserved: an operator source may not take this name.
pub(crate) const OFFICIAL_NAME: &str = "unom";
/// The built-in source's index URL.
pub(crate) const OFFICIAL_URL: &str = "https://plugins.punktfunk.unom.io/v1/index.json";
/// Pinned signing keys for the built-in source. **Two slots** so a key rotation is "sign with the
/// new key, ship a host that trusts both, retire the old one" instead of a flag day where old
/// hosts lose the catalog. An empty slot is ignored.
pub(crate) const OFFICIAL_KEYS: [&str; 2] = [
"ed25519:n/KgBQSSDZqvyGHa8PkHWHZtV1zRXLk0BdQUi4/BD/w=",
"", // rotation slot
];
/// How many operator sources we'll hold. A guard rail, not a design limit.
const MAX_SOURCES: usize = 32;
/// A catalog source as stored in `plugin-sources.json`. camelCase like the index document it
/// points at (the management API's own view of a source is snake_case — see `mgmt::store`).
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct Source {
/// Slug (`[a-z][a-z0-9-]*`, ≤32) — the cache-file name and the console's attribution chip.
pub name: String,
/// `https://` URL of the index document. The `.sig` is fetched from `<url>.sig`.
pub url: String,
/// Pinned ed25519 key. A source without one is accepted but marked **unsigned**, and its
/// entries inherit that marker — we can still show you the shelf, we just can't promise the
/// shelf wasn't rewritten in transit.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub public_key: Option<String>,
}
impl Source {
/// The compiled-in official source.
pub(crate) fn official() -> Source {
Source {
name: OFFICIAL_NAME.to_string(),
url: OFFICIAL_URL.to_string(),
// Held in [`OFFICIAL_KEYS`], not here: the built-in keys are code, not config, so a
// config file can never downgrade the official source to unsigned.
public_key: None,
}
}
pub(crate) fn is_official(&self) -> bool {
self.name == OFFICIAL_NAME
}
/// The keys this source's index must verify against. Empty ⇒ unsigned source (accepted, but
/// flagged); for the official source this is always the compiled-in pair.
pub(crate) fn keys(&self) -> Vec<PublicKey> {
if self.is_official() {
return OFFICIAL_KEYS
.iter()
.filter(|k| !k.is_empty())
.filter_map(|k| PublicKey::parse(k).ok())
.collect();
}
self.public_key
.as_deref()
.and_then(|k| PublicKey::parse(k).ok())
.into_iter()
.collect()
}
/// Does this source's index carry a signature we check? Drives the console's "unsigned" marker.
pub(crate) fn is_signed(&self) -> bool {
!self.keys().is_empty()
}
/// Where the detached signature lives. Kept a pure function of the index URL so a source
/// record can never point the signature fetch somewhere else than the document it signs.
pub(crate) fn sig_url(&self) -> String {
format!("{}.sig", self.url)
}
fn validate(&self) -> Result<()> {
if !valid_source_name(&self.name) {
bail!("source name must be kebab-case `[a-z][a-z0-9-]*`, ≤32 characters");
}
if !self.url.starts_with("https://") || self.url.len() > 500 {
bail!("source url must be an https:// URL (≤500 characters)");
}
if let Some(k) = &self.public_key {
PublicKey::parse(k).context("source publicKey")?;
}
Ok(())
}
}
pub(crate) fn valid_source_name(name: &str) -> bool {
!name.is_empty()
&& name.len() <= 32
&& name.as_bytes()[0].is_ascii_lowercase()
&& name
.bytes()
.all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-')
}
// ---------------------------------------------------------------- persistence
#[derive(Serialize, Deserialize, Default)]
struct SourcesFile {
#[serde(default = "one")]
schema: u32,
#[serde(default)]
sources: Vec<Source>,
}
fn one() -> u32 {
1
}
fn sources_path() -> std::path::PathBuf {
pf_paths::config_dir().join("plugin-sources.json")
}
/// Every source this host reads, official first. The official source is prepended here rather
/// than stored, so it survives a hand-edited (or deleted) config file.
pub(crate) fn load() -> Vec<Source> {
let mut out = vec![Source::official()];
let path = sources_path();
let Ok(bytes) = std::fs::read(&path) else {
return out; // no operator sources yet — the common case
};
match serde_json::from_slice::<SourcesFile>(&bytes) {
Ok(file) => {
for s in file.sources.into_iter().take(MAX_SOURCES) {
// Defense in depth: a hand-edited file can't smuggle in a source that shadows the
// official one or carries an unusable URL/key.
if s.is_official() || s.validate().is_err() {
tracing::warn!(name = %s.name, "ignoring invalid entry in plugin-sources.json");
continue;
}
if !out.iter().any(|e| e.name == s.name) {
out.push(s);
}
}
}
Err(e) => tracing::warn!(
"plugin-sources.json is unreadable ({e}) — using the official source only"
),
}
out
}
/// Add or replace an operator source. Refuses the reserved official name and invalid records.
pub(crate) fn put(source: Source) -> Result<()> {
source.validate()?;
if source.is_official() {
bail!("`{OFFICIAL_NAME}` is the built-in source and cannot be redefined");
}
let mut list: Vec<Source> = load().into_iter().filter(|s| !s.is_official()).collect();
if list.len() >= MAX_SOURCES && !list.iter().any(|s| s.name == source.name) {
bail!("too many plugin sources (max {MAX_SOURCES})");
}
list.retain(|s| s.name != source.name);
list.push(source);
save(list)
}
/// Remove an operator source. Returns `false` if there was nothing to remove.
pub(crate) fn remove(name: &str) -> Result<bool> {
if name == OFFICIAL_NAME {
bail!("the built-in `{OFFICIAL_NAME}` source cannot be removed");
}
let list: Vec<Source> = load().into_iter().filter(|s| !s.is_official()).collect();
if !list.iter().any(|s| s.name == name) {
return Ok(false); // nothing to remove — don't rewrite the file
}
save(list.into_iter().filter(|s| s.name != name).collect())?;
Ok(true)
}
fn save(list: Vec<Source>) -> Result<()> {
let dir = pf_paths::config_dir();
pf_paths::create_private_dir(&dir).context("create the punktfunk config dir")?;
let file = SourcesFile {
schema: 1,
sources: list,
};
let json = serde_json::to_string_pretty(&file).context("serialize plugin-sources.json")?;
let path = sources_path();
// Write-then-rename: a crash mid-write must not leave a half-file that makes the host forget
// the operator's sources on next boot.
let tmp = path.with_extension("json.tmp");
std::fs::write(&tmp, format!("{json}\n"))
.with_context(|| format!("write {}", tmp.display()))?;
std::fs::rename(&tmp, &path).with_context(|| format!("replace {}", path.display()))?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn official_source_is_signed_by_compiled_in_keys() {
let s = Source::official();
assert!(s.is_official());
assert!(s.is_signed(), "the built-in source must carry a pinned key");
assert_eq!(s.sig_url(), format!("{OFFICIAL_URL}.sig"));
// A config file cannot downgrade it: `keys()` ignores the record's own field entirely.
let mut forged = Source::official();
forged.public_key = Some("ed25519:AAAA".into());
assert_eq!(forged.keys().len(), 1);
}
#[test]
fn compiled_in_keys_parse() {
for k in OFFICIAL_KEYS.iter().filter(|k| !k.is_empty()) {
PublicKey::parse(k).expect("compiled-in official key must parse");
}
}
#[test]
fn unsigned_third_party_source_is_flagged_not_rejected() {
let s = Source {
name: "retro-hub".into(),
url: "https://example.org/index.json".into(),
public_key: None,
};
s.validate().unwrap();
assert!(!s.is_signed());
}
#[test]
fn validation_rejects_bad_names_urls_and_keys() {
let base = Source {
name: "ok".into(),
url: "https://e.org/i.json".into(),
public_key: None,
};
assert!(base.validate().is_ok());
assert!(Source {
name: "Bad".into(),
..base.clone()
}
.validate()
.is_err());
assert!(Source {
name: "9bad".into(),
..base.clone()
}
.validate()
.is_err());
assert!(Source {
url: "http://e.org/i.json".into(),
..base.clone()
}
.validate()
.is_err());
assert!(Source {
public_key: Some("ed25519:nope".into()),
..base.clone()
}
.validate()
.is_err());
}
#[test]
fn sig_url_is_derived_not_configurable() {
let s = Source {
name: "x".into(),
url: "https://e.org/v1/index.json".into(),
public_key: None,
};
assert_eq!(s.sig_url(), "https://e.org/v1/index.json.sig");
}
}