Skip to main content
Status: accepted · 2026-08-21 · proven live on the standing station

​
Context

The machinery was built outside-in: subscribers pull a channel of frozen, evidence-carrying releases. But our own staging → prod promotion still ran on hand deploys — helm upgrade plus kubectl set image forks, documented drift tables that themselves went stale, and rollbacks that silently stripped out-of-band hotfixes (see the vexa-platform handover docs, 2026-07). The vendor of a delivery product was not its own first consumer.

​
Decision

Run a vexa-internal channel with the same machinery, inverted in one respect: entries are published before production runs, not after.
  1. A release enters the channel as a candidate: same signed entry, same frozen digest set, but by definition no delivery receipt and no soak numbers — those facts do not exist yet. The schema enforces exactly this asymmetry: candidates are exempt from receipt/soak/per-image-prod-receipt strictness, every other publication mode stays fully strict.
  2. Stations (staging, then prod) consume the candidate, validate against their own contract, and sign a station-verdict attestation beside the entry. The entry is never republished; evidence accumulates next to it at <channel>/attestations:<kind>.<release>.
  3. A downstream contract names what must already be signed: require_attestations: [{kind: station-verdict, station: vexa-staging}]. The verifier pulls each required attestation from the channel, verifies its signature against the pinned key, matches its subject digests against the entry’s images, and checks release, station, and verdict.
  4. The published entry is then built from the accumulated evidence — delivery receipt, soak, hardening, and the station verdicts all ride in the bundle. What a customer receives is the record of what our own chain already consumed and signed.

​
Amendment, 2026-08-31 — a second carrier, for the shape this channel actually publishes

Steps 2–3 above describe the accumulated carrier: an in-toto statement pushed to <channel>/attestations:<kind>.<release> after the entry is sealed. That carrier is structurally unavailable to an estate entry, and vexa-internal publishes estate entries. spec/station-verdict-attestation.schema.json pins predicate.release to ^v[0-9]+\.[0-9]+\.[0-9]+$, so 0.12.23-estate-20260829-seq7 cannot be minted into one — a boundary that is deliberate and pinned by kit/verify/tests/test_verdict_out.sh check 4b. The consequence went unnoticed: internal-prod.json requires {kind: station-verdict, station: vexa-staging}, so against an estate entry that clause could only ever refuse. Not dormant — unsatisfiable. It also predates the proof half. The values_proven block landed on 2026-08-31, and the accumulated statement has nothing to bind it to: it carries a release string and a subject digest set, so a station’s ELIGIBLE and the block the entry ships are merely adjacent. So a second carrier, for the same clause. The verdict (spec/station-verdict.schema.json) is rendered at DEPART by publisher/vexa_station_verdict.py, signed, and carried inside the entry as evidence of kind station_verdict. It binds to the candidate commit and to the sha256 of the entry’s own values_proven block. vexa-verify.sh §6c reads it where it exists, falls back to the accumulated carrier where it does not, and says on the transcript which one adjudicated. The embedded verdict never falls back after a failure — a strong check with a weaker escape route is an optional check. Both carriers stay, because they answer different chain shapes: accumulated where the entry is sealed before the station runs, embedded where the station runs before the entry is minted. The second is the estate’s shape, and the reason the clause had never once been satisfied.

​
What this eliminates — and what it does not

Proven the hard way during adoption: renaming the ApplicationSet’s element env tore down and recreated the whole staging workload, and the station rebuilt itself from the channel alone — 12 pods, zero hand commands. That is the property being bought: the running state is a pure function of (channel entry × pinned position × per-station values). The drift class that bit vexa-platform — hand forks persisting invisibly, rollbacks stripping undeclared state, stale drift bookkeeping, staging validating different bytes than prod runs — becomes impossible rather than discouraged. Not eliminated, by design honesty: per-station values divergence (inspectable, not forbidden), data-plane state (database schema, secrets contents, volumes), and shared-cluster infrastructure. Those are preflight and migration-evidence territory, not GitOps territory.

​
Consequences

  • The emergency-hotfix path changes shape: a hand-built image cannot be set on a station (admission refuses unsigned images). The fast lane is break-glass — a signed, recorded bypass in the entry itself that the next entry must supersede — instead of a silent fork.
  • Element env names in the ApplicationSet are identity. Renaming one is a declared teardown of that station’s apps. Rename only with the recreate in mind.
  • One key currently signs channel and stations alike; per-station keys are a known follow-up alongside the key ceremony.
  • The standing station (LKE vexa-channel-station, ex mvp0-node) hosts the channel registry and both internal stations until the channel moves to GHCR.