← Ledger

title: Security incident — tenancy invariant bypassed on rocky.erid.tech (missing ROCKY_SESSION_SECRET) date: 2026-07-02 status: Accepted phase: sec-inc-2026-07-02 (production tenancy invariant restoration) predecessor: docs/decisions/2026-07-01-design-tokens-package.md spec: docs/north-star/intent.md (§ tenancy invariant) plan: (this ADR is the plan) severity: P0

Security incident — production tenancy bypass

Summary

Production rocky.erid.tech served gated routes (/vault, /relay, /agents, /workspace, plus every /api/* under the same prefixes) as an unauthenticated caller with a synthesized admin session. All three legs of the tenancy invariant (Airlock session, HATCH audit, Polar entitlement) were absent for the affected requests. The production smoke suite (added 2026-07-01) did not catch this because every assertion was < 500 — a 200-with-forged-admin is indistinguishable from a legitimate 200 at that granularity.

Detected 2026-07-02 while reviewing smoke output (35 KB Vault body returned to a cookie-less request).

Root cause

console/src/middleware.ts:22-23:

const AUTH_DISABLED =
  process.env.AUTH_DISABLED?.trim() === "true" || !SESSION_SECRET;

When ROCKY_SESSION_SECRET is unset (SESSION_SECRET === ""), AUTH_DISABLED short-circuits to true and the middleware forwards every request as:

{ id: "dev", email: "dev@devarno.cloud", role: "admin" }

This local-dev bypass is load-bearing for npm run dev (no airlock round-trip), but a production Vercel deploy missing the env var inherits it silently. The Vercel project for rocky-console was scaffolded 2026-05-08 and ROCKY_SESSION_SECRET was never populated in the Production environment.

Impact

Blast radius bounded by: (a) domain is not publicly indexed / advertised; (b) /api/vault reads decrypt-with-KMS on demand, not on route hit — an anonymous GET returns metadata, not plaintext.

Remediation (sequence)

  1. Set ROCKY_SESSION_SECRET in Vercel Production (Rocky-console project, all three environments). Rotate against the local .env.example seed if any development instances share the value.
  2. Redeploy to activate the new env. Verify curl -sI https://rocky.erid.tech/vault returns 307 (redirect to airlock.devarno.cloud), not 200.
  3. Invert the smoke suite for gated routes: assert 3xx, not < 500. Landing (/) is currently unprefixed → defaults to operator in getMinRole — it too must redirect. The current /agents renders projection page test asserts 200; it must move to the authed smoke tier (P3) or be replaced with a redirect assertion.
  4. HATCH lookback: query agent-audit.json for any agent.invoked between the Vercel project scaffold date (2026-05-08) and the fix timestamp; flag any entry attributed to dev@devarno.cloud from a non-localhost origin. If any producer fired for anonymous callers, escalate as a real breach event.
  5. Middleware hardening (follow-up): refuse to boot when NODE_ENV === "production" and SESSION_SECRET === "". Fail-closed at process startup beats silent bypass at request time. Filed as separate slice; not in scope of this ADR.

Locked decisions

1. Fail-closed in production, keep dev bypass for npm run dev

Do not delete the AUTH_DISABLED branch. Local dev needs it — every contributor would need a real signing secret to run npm run dev, which we've refused since Phase 2. Gate the branch on NODE_ENV !== "production" in the follow-up middleware hardening. Missing secret in production is a deploy-time error, not a request-time bypass.

2. Smoke suite must assert presence of the redirect, not absence of 5xx

The current suite proved "no server error" — a strictly weaker claim than "gate is closed." Preventive assertion inverts the polarity for every gated prefix: unauth GET must be >= 300 && < 400 and location must include the airlock issuer host. A 200 for /vault is now an incident-severity smoke failure.

3. Audit log integrity — do not rewrite

agent-audit.json entries attributed to the synthesized identity stay in the log. Rewriting audit would break MR-7 (append-only ledger). The ADR itself is the pointer that flags the affected window for future readers of the audit.

References