Skip to main content
Date: 2026-09-06. Channel: a private subscriber channel, referred to here as <subscriber>-stable. Registry: a Harbor v2.15.2 proxy-cache at <harbor-host>, upstream channel.vexa.ai. Cluster: a dedicated single-node kind cluster, Kubernetes v1.30.0, on its own kubeconfig. Container work ran on an internal build host, never on a laptop.

​
The rung, stated first because everything below is conditioned on it

Rung 4 of 4, and it is NOT OpenShift. The host’s AMD-V is disabled and latched in firmware (MSR_VM_CR = 0x18 → SVMDIS=1 under SVM_LOCK=1), on bare metal, MSR write-locked until reset. One firmware bit gates three of the four rungs. So: no oc, no SCC objects, no OLM, no Routes, no internal registry, no GitOps operator. This run proves the transport, the kit and the gate behaviour. It proves nothing about SCC, and it does not supersede the 2026-08-21 MicroShift 4.18 rehearsal, which remains the strongest SCC evidence held. PROFILE_TESTED on the openshift profile stays no. The tenant is namespace-scoped by ServiceAccount kubeconfig — every application act below was performed as that tenant, and every claim about what it may not do is from attempting the verb, never from kubectl auth can-i.

​
What was proven

​
The transport works, end to end

The subscription resolved chart 0.12.35 through the proxy-cache and rendered 31 resources. ${REGISTRY}/vexa/channel/${CHANNEL} composed onto the Harbor project path exactly as the proxy-cache expects, so no kit change was needed to put a pull-through in front of the channel. --registry-insecure was required: Argo CD has no per-repository CA-bundle knob for OCI repositories, so a corporate-CA registry needs the repo secret to opt out of verification while Kyverno gets the CA as a trust bundle. That asymmetry is already documented in install.sh; this run confirms it is not optional.

​
Gate 1 — the PreSync verify gate: ELIGIBLE

A private channel entry, pulled through a customer-shaped pull-through registry, signature-verified offline against a key delivered by a separate route. That is the whole thesis, and it holds.

​
Gate 2 — Kyverno signature admission: DENY, because the installer was never told where the signatures are

All six Vexa Deployments were denied:
install.sh warns in its own comments that this message is byte-identical whether the image is unsigned, unfetchable, or merely looked for in the wrong place. This run’s first reading of it — unsigned — was wrong, and the correction is the finding. The images are signed. Every image digest of entry 4 answers 200 at the channel’s own signature repository, channel.vexa.ai/vexa/channel/<subscriber>-stable/signatures, at the conventional sha256-<digest>.sig path, carrying a genuine cosign object: a vnd.dev.cosign.simplesigning.v1+json layer, a vnd.dev.sigstore.bundle.v0.3+json layer, and the dev.cosignproject.cosign/signature annotation — the exact layout the signature-layout receipt proved this Kyverno ADMITS. Signature reads on the channel are credential-free by design (RUNBOOK § 5.2), so anyone holding the entry can re-run this check. This run probed only the second row and concluded from it. The second row is the layout working as designed — the signatures live at the channel, which is where the entry says they live — not evidence of an unsigned image. What failed is the install, not the packet. This run did not pass --signature-repository, so install.sh:262-267 deleted the repository line out of the admission policy and Kyverno looked for signatures beside each image on Docker Hub, where by design there are none. That is the failure tested § The from-docs run, 2026-08-27 already documents, reproduced here — and it is defect 7 below. Withdrawn, in full: “the images are genuinely unsigned”, “there is nowhere a signature exists to point it at”, and “on this entry the kit’s own default admission policy is unsatisfiable”. The policy is satisfiable and this entry satisfies it. Nothing follows from this run about the publish crank, and no advice to run with admission off follows from it either.

​
Seven defects in the kit

​
1 · install.sh has no adoption path, and running it against an existing Argo is destructive

There is no --skip-argo, no --adopt, no --argocd-existing; steps 2 and 3 are unconditional except under --dry-run. The openshift profile’s own comment says “When the operator (or a house Argo) is present, run install.sh against it rather than letting step 2 install a second Argo” — but the script offers no mechanism to do that. Measured with kubectl apply --server-side --dry-run=server as the tenant against upstream argo-cd/v3.5.1/manifests/install.yaml:
  • 49 objects would be written, including deployment.apps/argocd-dex-server and statefulset.apps/argocd-application-controller — re-adding the hard-coded runAsUser: 1001 (dex) and runAsUser: 999 (redis) that a restricted-range policy then rejects;
  • 9 objects Forbidden — 3 CRDs, 3 ClusterRoles, 3 ClusterRoleBindings.
So a real run half-lands: it breaks the working Argo, is refused on the cluster-scoped nine, and set -e stops the script — leaving no subscription and an Argo that no longer admits. Step 2 was therefore never executed here; steps 4/5/6 were rendered by install.sh --dry-run and applied by hand, tenant-first. Suggested shape: --argocd-existing / --kyverno-existing that assert the version and namespace and skip the install, rather than a flag that silences the failure.

​
2 · The preflight crashes as a tenant instead of failing

take_snapshot performs an unwrapped kubectl get nodes:
Every other read in that same function is inside try/except RuntimeError — namespace, limitranges, resourcequotas, networkpolicies, and notably storageclasses, which is also cluster-scoped. So someone anticipated cluster-scoped refusals and missed this one. A namespace-scoped tenant gets a Python traceback, and install.sh then prints preflight FAILED, which is a lie: nothing was checked. This lands squarely on the OpenShift path, whose whole premise is project, not cluster. --namespace-scoped on vexa_validate.py does not cover it. The workaround — the one a platform team would be forced into — is --dump-snapshot as an admin, then --snapshot air-gapped. Real verdict once run that way: FAIL on P4, everything else PASS. P4’s failure is the dynamic bot pod carrying no securityContext — correct for SCC, refused by PSA restricted. Worth being precise: it is a static-analysis result and the live cluster contradicts it, because a mutating webhook runs before PSA validation. (kit/report’s own README already handles namespace scope correctly, reporting cluster facts as UNKNOWN rather than guessing. The preflight should borrow that posture.)

​
3 · install.sh never passes the manifests to the preflight — so its best checks run on nothing

No --manifests. So objects = [], and P2 (resources fit the LimitRange) and P3 (quota headroom) evaluate only the synthetic bot profile — never the delivered chart. The cost was immediate. The tenant’s LimitRange had a container memory max of 2560Mi, the recorded value, sized for the bot’s 2Gi /dev/shm. The chart’s own postgres asks a 4Gi limit:
P2’s own anchor is “a customer LimitRange squeezed undeclared bots to 64Mi” — and the same class of failure happened anyway, at sync time, because the check ran against nothing. This is the finding to carry forward first: the preflight is the right tool, invoked so that its two most valuable checks are inert.

​
4 · WaitForFirstConsumer deadlocks the sync

The chart’s standalone agent-workspaces PVC plus a WaitForFirstConsumer StorageClass produce a circular wait: Argo counts a Pending PVC as un-Healthy and will not start the wave containing the Deployments that would consume it, so the PVC waits for a consumer that waits for the PVC. The sync sits at “waiting for healthy state of /PersistentVolumeClaim/…” indefinitely. The one provider with PROFILE_TESTED=yes binds Immediate, which is why this has never been seen — and many OpenShift default CSI classes are WaitForFirstConsumer. Cleared here by pre-binding a PV.

​
5 · Two objects share one name, and only a single-namespace tenant notices

install.sh names both of these vexa-channel-registry:
  • the Argo repository Secret in $ARGOCD_NS (Opaque);
  • the kubelet pull Secret in each workload namespace (dockerconfigjson).
They never collide only because ARGOCD_NAMESPACE defaults to argocd while the workloads live elsewhere. On one project — the shape the openshift profile exists for — they are the same object. The second create fails with type: Invalid value: "kubernetes.io/dockerconfigjson": field is immutable, and every ServiceAccount is left with its imagePullSecrets pointing at an Opaque Argo secret the kubelet cannot use. That is exactly the ImagePullBackOff the thirty-line comment at install.sh:348 was written to prevent, reintroduced by a namespace choice.

​
6 · kit/smoke raises where it should report — and that blocks the receipt

Against a half-converged estate, smoke produced no verdict and no receipt:
kit/smoke/README.md promises that --non-interactive “skips or bounds the human phase but never fakes it: no admitted bot → honest FAIL” — but that guard covers S3 only. S1 and S2 have none. The consequence is the sharpest thing this run found. vexa_validate.py therefore wrote a station-report.yaml with no smoke_receipt section, and the ingest gate correctly refused it:
RUNBOOK § 3 says failure reports matter more than success ones, and § 3.2 says an un-ingested report is an un-represented customer. So today: the path that breaks is the failure-reporting path. A subscriber whose install goes wrong cannot file the receipt that would represent them — precisely when we most need it.

​
7 · install.sh demands a flag for something it already knows, and degrades silently without it

--signature-repository defaults to “alongside each image” (install.sh:39) — the one layout this channel does not use. When the flag is absent, lines 262-267 neither warn nor refuse; they delete the ${SIGNATURE_REPOSITORY} line from the rendered policy, and the admission rule that results denies every Vexa workload with a message that reads exactly like an unsigned image. That is Gate 2 above, and it cost this run its whole second half. The script holds both halves of the correct value already. It composes --registry and --channel into ${REGISTRY}/vexa/channel/${CHANNEL} for the Argo repository (install.sh:298,315); the signature repository is that same reference plus /signatures — which is precisely what the onboarding mail and the install page ask the subscriber to retype by hand. Suggested shape: derive SIG_REPO from --registry + --channel by default, keep the flag as the override for a subscriber who mirrors the channel elsewhere, and make no signature repository an explicit opt-out rather than a silent deletion. This is the third time this defect has been paid for: once on 2026-08-27 (recorded on tested), once here, and once again in the reading of this run’s own DENY. The 2026-08-27 fix put the flag in the documented command — which repairs the copy-paste path and leaves every other path open. Defects 2 and 6 are one class: the kit raises where it should report, on the two paths a struggling subscriber hits first. Defect 7 is the mirror of it — the kit silently proceeds where it should refuse — and it is the one that produced a false finding rather than a slow one.

​
What did not happen, said plainly

  • The HOME / minio-init mkdir /.mc blocker was not reached. Admission stopped every application Deployment before a pod existed, and the rendered set contains no minio-init hook at all at chart 0.12.35. Not reproduced, not refuted — unreachable in this run.
  • Postgres initialised and served under an injected arbitrary UID (uid=1000920000 gid=0(root); “database system is ready to accept connections”), contradicting the audit row that says the official image cannot. Marked unconfirmed — this was Kyverno emulating SCC, not SCC — and offered as a candidate correction to re-check, not a fact to carry forward.
The emulated SCC did hold on real product pods rather than only on probes: all three workloads that reached Running were mutated to runAsUser 1000920000, fsGroup 1000920000, runAsNonRoot true, seccompProfile RuntimeDefault, seLinux s0:c26,c5.

​
Final state

​
One-time platform-team asks, as measured

Each row is a verb the tenant attempted and was refused; see openshift-parity.mdx § How the station lands there.