Skip to content

Operator Console Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: The service provider’s operator signs in to a private console (password + TOTP) and sees whether the service is healthy and whether each organization’s data is intact and compliant, as counts and pass/fail only. They can then run a small set of audited actions (create an organization, re-provision, verify the audit chain, retry a job, suspend or reactivate). The console never shows record contents, never edits or deletes a record, and never lets an operator grant themselves access to an organization’s data.

Architecture: A fourth long-running binary, cmd/operator, bound to a private address. It connects as a new role, erp_operator, which has no privilege on any business table. It reads operator-owned tables in a new ops schema directly. Cross-organization aggregates come only through reviewed SECURITY DEFINER functions owned by erp_ops_owner (NOLOGIN, BYPASSRLS), and those functions return aggregates only. Expensive checks and every action that touches an organization run in cmd/worker, inside that organization’s own tenant transaction. The console enqueues them and reads back the outcome. Pages are server-rendered with Go html/template from internal/operator: no second frontend build, not in api/openapi.yaml.

Tech Stack: Go 1.26 · net/http (stdlib mux, not gin: no public router to share) · html/template + embed · pgx v5 · goose · River · testify · testcontainers (Postgres 17) · existing identity/password, identity/totp, kernel/audit · Prometheus metrics already built (ops-groundwork)

Spec: docs/superpowers/specs/2026-09-25-operator-console-design.md (§ numbers below refer to it) Progress: docs/superpowers/specs/2026-09-25-operator-console-progress.md (P1–P15, I1–I15)


Phase Kind What Gate
0 Decisions The five Stage 0 questions (below) Blocks B4 onward, not B1–B3
B1 Backend Migration range, drift tests widened to the ops schema none
B2 Backend erp_operator role, ops schema, operator account / session / hash-chained audit none
B3 Backend cmd/operator: config, private listener, CLI, sign-in, sessions, CSRF, route guard none
B4 Backend V2 health and V8 releases (no business data at all) none
B5 Backend Cross-organization views V1, V3, V4 (+ nightly reconciliation, R30), V5, V7, V6 Decision D1 (§3.3)
B6 Backend Action queue and A1–A4; A5 suspend (organization.status) D1, D2 (suspend scope)
F1 Frontend Console layout, sign-in and TOTP pages, navigation, error pages B3
F2 Frontend Pages for V1–V8 B4, B5
F3 Frontend Action pages: confirm step, show-once invitation link, result B6
F4 Frontend (web/) ERP side: suspended-org sign-in message; operator actions visible in the org’s audit trail B6
S5 — Support access (A7): its own design spec first (§3.7). Not planned here after F4

“Backend first” for a server-rendered app means this: each B phase ships its routes with a bare template, just a <pre> of the view model, so the phase is verifiable end to end and the route guard test sees real routes. The F phases replace those templates with the designed pages. No handler logic moves in an F phase.


Phase 0: Decisions (owner’s call, before B4)

Section titled “Phase 0: Decisions (owner’s call, before B4)”

Each has a recommended default. B1–B3 do not depend on any of them.

# Decision Recommended Blocks
D1 Accept the §3.3 exception: erp_ops_owner (NOLOGIN, BYPASSRLS) owns ops functions that read across organizations and return aggregates only. Accepted 2026-09-28 (still to be shown to the tax agent / auditor) Accept, with the drift tests in B1/B5 B5, B6
D2 What suspension blocks: sign-in and writes only, or also the org’s scheduled jobs. Decided 2026-09-28: sign-in and writes only Sign-in and writes. Jobs keep running, so the retention clock and reconciliation continue B6 (A5)
D3 Private network path to the console (I2): Tailscale / WireGuard / SSH tunnel Tailscale (research next). The code only needs a private bind address Deploy only
D4 V1 shows the organization name, or the org id only with the name one click away and audited (§5 Q3). Decided 2026-09-28: show the name Show the name. It is a business name, and sole proprietors are rare in this customer base B5 (V1)
D5 Where actions run: the console enqueues and the worker executes (Ruling O6), rather than the console holding erp_app credentials Enqueue (O6) B6

Every task’s requirements implicitly include this section.

  • Migrations 00230–00249 only (design §3.8, still free on main). Task B1.1 registers {230, 249, "Operator console"} in internal/kernel/db/migrationranges_test.go in the same commit as 00230.
  • The console never reads a business table. erp_operator holds USAGE on ops and privileges on the ops.operator_* tables only; everything else is EXECUTE on ops functions. No role the console logs in as has BYPASSRLS. Test: B2.1.
  • ops functions return aggregates only: uuid, bigint, integer, boolean, timestamptz, date, text only for the org name and for fixed vocabulary codes (check keys, job kinds, states). Never an email, a person’s name, an amount or a record value. Test: B5.1.
  • Every function in ops has SET search_path = pg_catalog, public, pg_temp, schema-qualifies every relation, and every SECURITY DEFINER one is on the allowlist and REVOKE EXECUTE … FROM PUBLIC. The existing tests only scan public, so B1.2 widens them before any ops function exists.
  • Every console route except GET /signin, POST /signin, GET /healthz and static assets goes through operator.RequireSession, and every state-changing route through operator.RequireCSRF. Test: B3.5 (the console’s own “every route is authorized”, §6).
  • Every operator action writes ops.operator_audit (hash-chained). One that affects an organization (A1, A2, A5) also writes that organization’s audit_log through kernel/audit.Write, in the worker’s tenant transaction.
  • Nothing is deleted. No console route issues DELETE on any table. Suspension flips a status string (active / suspended, a vocabulary string, never an integer).
  • Private bind. OPERATOR_LISTEN_ADDR refuses an all-interfaces address, reusing config.metricsAddr’s rule (B3.1). The console is never published through the public reverse proxy.
  • No new secrets mechanism. TOTP secrets are sealed with identity/totp.Seal and TOTP_ENCRYPTION_KEY, which must be set when the console runs. Passwords use identity/password.Hash / Verify.
  • Env drift: every new variable is in .env.example (TestEveryEnvVarIsInExample), read with a literal os.Getenv("NAME").
  • Presence before absence (AGENTS.md). Every guard gets a Sabotage step: break it, run it, read the red message, restore.
  • Tests: testdb.Start by default; testdb.StartIsolated for tests that create roles or log in as a role. Iterate on touched packages; one full go test ./... at the end of each phase.
  • Git commits end with: Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Where the design is silent or cannot be built as written. Format: Ruling: <decision> — <why>.

  • O1 Ruling: cmd/operator uses stdlib net/http with Go 1.22 method patterns, not gin. The console shares no middleware with the public API, and an independent mux means no app.Router change can ever route to it (§3.2).
  • O2 Ruling: Operator sessions are an opaque 32-byte random id in an HttpOnly; Secure; SameSite=Strict cookie, stored as its SHA-256 in ops.operator_session, with a 30-minute idle and a 12-hour absolute expiry. The id is rotated on sign-in. This replaces the design’s “same rotating refresh design as session_family” — a server-rendered app on one origin has no refresh token to rotate, and a hashed server-side session is the simpler equivalent. Sign-out revokes the row.
  • O3 Ruling: TOTP is required for every operator; there is no password-only mode. The account is created by the CLI (operator create-account --email), which prints the password once and the otpauth:// URI once. The account is inactive until the first successful TOTP sign-in (proves enrolment). Recovery codes: none in v1. A second operator reactivates (§3.4).
  • O4 Ruling: Sign-in throttling counts signin_failed rows in ops.operator_audit for the same email in the last 15 minutes. Five or more → refuse without checking the password. No new throttle table, and the audit log is already the record.
  • O5 Ruling: erp_operator gets SELECT, INSERT, UPDATE on ops.operator_account and ops.operator_session, SELECT, INSERT on ops.operator_audit (no UPDATE/DELETE: append-only), and SELECT on public.goose_db_version (V8; not business data). Nothing else in public.
  • O6 Ruling (D5): Actions are rows in ops.operator_action (id, kind, org_id NULL, params jsonb, requested_by, requested_at, state, result_code, finished_at, secret_result bytea NULL). The console inserts them through ops.request_action(kind, org_id, params). A worker periodic job, ops_action_dispatch (every 15 s, queue maintenance), claims them FOR UPDATE SKIP LOCKED and runs each in the target organization’s tenant transaction. The console never holds erp_app or owner credentials.
  • O7 Ruling: A1’s invitation link is a credential (§2.2). The worker seals it with totp.Seal into secret_result. The console shows it once, then clears it through ops.take_secret(action_id), which returns it and sets the column to NULL in one statement.
  • O8 Ruling: ops.check_result(org_id, check_key, passed, detail_code, ran_at) is written by the worker as erp_app (INSERT only) inside the org’s tenant transaction. It has org_id, forced RLS and org_isolation like any tenant table, so it passes the existing tenant drift tests unchanged. The console reads it only through ops.check_summary().
  • O9 Ruling: The nightly reconciliation (Plan 9 Ruling R30) is ops_reconcile_all, a River periodic job (24 h, RunOnStart: false). It gets org ids from ops.org_ids() (D1), then runs reconcile.Default().Run per org in db.InTx with that org set. The same job with one org id is A3’s “verify now”.
  • O10 Ruling: Suspension (A5, D2) is enforced where liveness is already checked: identity/liveness already asks EXISTS (SELECT 1 FROM organization …), so it gains AND status = 'active', and the Redis cache entry for the org’s sessions is invalidated. Sign-in refuses with the coded error org_suspended. Writes are refused by the same liveness check on every authenticated request, so no per-handler change is needed. The refusal is a 403 with a translated sentence (F4).
  • O11 Ruling: V2 is /v1/ready probed from the console process (api address from OPERATOR_API_URL, private), plus a link to Grafana (OPERATOR_GRAFANA_URL). The console draws no charts (§3.6).
  • O12 Ruling: V6 backups reads the file the backup script writes (I14) at OPERATOR_BACKUP_STATUS_FILE (one JSON object: last_success_at, last_restore_test_at, wal_lag_seconds). A missing file shows “unknown”, never green.
  • O13 Ruling: Frontend is plain HTML and one embedded stylesheet. The only script is the 8-line static/copy.js behind the copy buttons (owner share block after create-org). CSP default-src 'self'; script-src 'self'; frame-ancestors 'none'. Three languages are not required for the console (operator-facing, one operator team). The ERP-side strings in F4 are en/ms/zh like every web/ string.

cmd/operator/
main.go serve (default) | create-account | reactivate
wiring_test.go links what it must, nothing from internal/modules/*/internal
internal/operator/
config.go OPERATOR_* env, private-bind rule
server.go mux, middleware chain, template rendering
session.go O2 cookie sessions
signin.go password + TOTP + O4 throttle
csrf.go per-session token, double-submit
audit.go ops.operator_audit hash chain (shape of kernel/audit)
views.go V1–V8 view models (B4, B5)
actions.go A1–A5 request + result (B6)
routes_test.go every route authorized (§6)
templates/*.html bare in B phases, designed in F phases
static/console.css F1
internal/platform/opsjobs/ (worker side)
dispatch.go O6 action dispatcher
reconcile.go O9 nightly + on-demand checks
db/migrations/00230_…00242_*.sql

internal/operator is a top-level app package like internal/app (§3.8). It owns no business tables and imports no module’s internal/.


Files: internal/kernel/db/migrationranges_test.go

  • Add {230, 249, "Operator console"}. It lands in the same commit as 00230 (B2.1), so the range test never sees an empty reservation.

Task B1.2: Widen the function drift tests to the ops schema

Section titled “Task B1.2: Widen the function drift tests to the ops schema”

Files: internal/kernel/db/securitydefiner_test.go, internal/kernel/db/searchpath_test.go

  • Change n.nspname = 'public' to n.nspname IN ('public', 'ops') in both, and key the allowlist by schema.name.
  • Presence: a throwaway migration in the test creates ops.probe() without SET search_path. Assert the walker reports it, then drop it. This proves the widened query reaches ops before any real ops function relies on it.
  • Sabotage: revert the IN to 'public'; the probe assertion goes red. Restore.

Task B1.3: Table-inventory guard for ops (built with B2, once ops tables existed)

Section titled “Task B1.3: Table-inventory guard for ops (built with B2, once ops tables existed)”

Files: internal/kernel/db/opsschema_test.go (new)

  • Every table in schema ops is either on opsOperatorTables (no org_id, each with a written reason: §6 “operator tables are the only non-org_id exception”) or has org_id + forced RLS + org_isolation (like check_result, O8).
  • Presence: the walker finds ≥ 1 ops table (after B2.1). Absence: none outside those two shapes.
  • Sabotage (as built): a stray ops.stray table without org_id, a DELETE grant and an UPDATE grant on the audit log each go red. check_result does not exist until B5.

Verify B1: go test ./internal/kernel/db/ -count=1 green; each sabotage seen red once.

Phase B2: Roles, ops schema, operator identity tables

Section titled “Phase B2: Roles, ops schema, operator identity tables”
  • CREATE SCHEMA ops (owned by the migration owner for now; D1 later hands functions to erp_ops_owner).
  • CREATE ROLE erp_operator LOGIN PASSWORD 'erp_operator' guarded like erp_app in 00001 (dev password; production sets its own, RUNBOOK gains a line). GRANT USAGE ON SCHEMA ops. REVOKE ALL ON SCHEMA public FROM erp_operator except SELECT on goose_db_version (O5).
  • Test (StartIsolated): presence — as the owner, SELECT count(*) FROM organization returns rows for a seeded org; absence — connected as erp_operator, the same query is permission denied (§6 row 1). Repeat for audit_log, journal_entry, expense_claim, river_job.
  • Sabotage: GRANT SELECT ON organization TO erp_operator in the test → red.

Task B2.2: 00231_operator_account_session.sql

Section titled “Task B2.2: 00231_operator_account_session.sql”
  • ops.operator_account(id uuid pk, email text unique (lower), password_hash text, totp_sealed bytea, active boolean default false, created_at, deactivated_at).
  • ops.operator_session(id_hash bytea pk, operator_id fk, created_at, last_seen_at, expires_at, revoked_at).
  • Grants per O5. Named constraints (TestEveryConstraintIsNamed).

Task B2.3: 00232_operator_audit.sql + internal/operator/audit.go

Section titled “Task B2.3: 00232_operator_audit.sql + internal/operator/audit.go”
  • ops.operator_audit(seq bigserial, at, operator_id NULL, event text, org_id NULL, detail jsonb, prev_hash, row_hash), REVOKE UPDATE, DELETE from everyone, and the same chain computation as audit_log (00007). event is a vocabulary string: signin_ok, signin_failed, signout, action_requested, action_finished, account_created, account_reactivated.
  • operator.WriteAudit(ctx, tx, clock, entry) and operator.VerifyAudit(ctx, tx), copying kernel/audit’s approach. The chain is serialized by advisory lock class 3 (registered in kernel/audit). ops.operator_audit also has an email column for a failed sign-in with no matching account (the O4 throttle key).
  • Tests: presence — three rows verify clean; absence — altering row 2’s detail as the owner makes VerifyAudit return the break at seq 2. erp_operator gets permission denied on UPDATE.
  • Retention: as built, retention_test.go stays public-only. opsschema_test.go holds the rule for ops instead: no role may DELETE from any ops table, and erp_operator cannot UPDATE ops.operator_audit.

Verify B2: as erp_operator in psql, \dt ops.* works and SELECT * FROM organization is denied.

Files: internal/operator/config.go, .env.example

  • OPERATOR_DATABASE_URL (required), OPERATOR_LISTEN_ADDR (required to serve; config.PrivateAddr, exported from the metrics code), and TOTP_ENCRYPTION_KEY (now required for the console; read by the new config.TOTPKeyFromEnv, which config.Load also uses). OPERATOR_API_URL, OPERATOR_GRAFANA_URL and OPERATOR_BACKUP_STATUS_FILE arrive with the phases that read them (B4, B5), because the env drift test fails on an example entry nothing reads.
  • As built: operator.CheckRole refuses, at startup, any role that is a superuser, has BYPASSRLS, or can SELECT from organization. It checks the privilege, not the role’s name, so a renamed owner is refused too.

Task B3.2: CLI create-account and reset-account

Section titled “Task B3.2: CLI create-account and reset-account”
  • Prints the generated password and the otpauth:// URI once (O3). Writes account_created to ops.operator_audit.
  • As built: reset-account --email --by <operator email>. --by must be a different, active operator. It issues a new password and TOTP secret, makes the account inactive until the new authenticator is used, and revokes its sessions. There is no separate deactivate command yet.

Files: session.go, signin.go

  • O2 cookie sessions (__Host-operator_session), O4 throttle, O3 TOTP via totp.Open + totp.Validate. As built: one form takes email, password and code together (not two pages), so there is no half-signed-in state to protect. A failed sign-in always records the typed email, which is the throttle key. An unknown email still costs one argon2 verification, so the timing does not reveal which emails exist.
  • Tests, each presence-first: right password + right code → session; right password + wrong code → refused and signin_failed audited; five failures → the sixth is refused even with correct credentials; an expired or revoked session → redirect to /signin.
  • The CSRF token is derived from the session token (SHA-256), so nothing is stored. It sits in a hidden field and is compared in constant time on every non-GET. Headers per O13, plus Cache-Control: no-store.

Task B3.5: Route guard (§6 “every console route is authorized”)

Section titled “Task B3.5: Route guard (§6 “every console route is authorized”)”

Files: internal/operator/routes_test.go

  • The server exposes its route table ([]Route{Method, Pattern, Public bool}), and Public is allowed only for the four named routes.
  • Test: presence — the table has ≥ 5 routes, and an unauthenticated GET to a protected route returns 303 to /signin; absence — no protected route answers 200 without a session, and no POST succeeds without CSRF.
  • Sabotage: register one route without RequireSession → red.
  • serve binds OPERATOR_LISTEN_ADDR, stamps version (Makefile LDFLAGS gains bin/operator), and serves /healthz.
  • As built: the assertion lives in internal/app/binaries_test.go, which already runs go list -deps per binary: cmd/operator links no internal/modules package.

Verify B3: create an account from the CLI, sign in with a phone authenticator, and see signin_ok in ops.operator_audit. Wrong TOTP is refused.

Phase B4: V2 health and V8 releases (no business data)

Section titled “Phase B4: V2 health and V8 releases (no business data)”
  • The console’s own version, the api’s version from OPERATOR_API_URL/v1/health, the goose version from goose_db_version, and pending migrations. As built: the migrations are embedded by a new package at db/ (db.Migrations); a Go file inside db/migrations would upset goose and the migration-name test. “Pending” means not applied, not “newer than the applied version”: cmd/migrate applies out of order (WithAllowMissing), and a rolled-back version is listed too. operator.Releases, /releases.
  • Probe /v1/health and /v1/ready with a 2 s timeout, no redirects followed, plus the console’s own database ping; link to Grafana (O11). New optional settings OPERATOR_API_URL (no path) and OPERATOR_GRAFANA_URL. Server.Health, /health.

Verify B4 (as run): with the real api stopped, V2 shows both api probes failing and the database passing; an extra migration file in a test FS and a rolled-back row in goose_db_version are both listed as pending (views_test.go). This proves the plumbing end to end with zero business data (§4 Stage 2).

Phase B5: Cross-organization views (D1 accepted 2026-09-28)

Section titled “Phase B5: Cross-organization views (D1 accepted 2026-09-28)”

Task B5.1: 00233_ops_owner.sql + the aggregate-only guard

Section titled “Task B5.1: 00233_ops_owner.sql + the aggregate-only guard”
  • CREATE ROLE erp_ops_owner NOLOGIN BYPASSRLS. Every ops function is OWNER TO erp_ops_owner, SECURITY DEFINER, pinned search_path, REVOKE EXECUTE FROM PUBLIC, GRANT EXECUTE TO erp_operator, and is on the allowlist with its reason.
  • ops.org_ids() RETURNS SETOF uuid, EXECUTE granted to erp_app too (the worker, O9).
  • Guard test (§6 row 2): presence — the walker finds ≥ 1 ops function; absence — no result column’s type is outside the allowed set, and a text column is allowed only when named name or ending _key / _code / kind / state.
  • Sabotage: add a function returning email text → red.

Task B5.2: V1 organizations: 00234, ops.org_overview()

Section titled “Task B5.2: V1 organizations: 00234, ops.org_overview()”
  • Returns org_id, name, status, created_at, member_count, last_activity_at, storage_bytes. Storage comes from the documents table’s recorded sizes (never object keys).
  • Test: two orgs; each count is correct; the result contains no column outside the B5.1 rule.

Task B5.3: V4 integrity: 00235_check_result.sql, nightly job (O8, O9)

Section titled “Task B5.3: V4 integrity: 00235_check_result.sql, nightly job (O8, O9)”
  • ops.check_result (tenant table). ops.check_summary() returns per org and check: latest passed, detail_code, ran_at.
  • internal/platform/opsjobs/reconcile.go: the ops_reconcile_all periodic job plus a one-org variant, registered in cmd/worker/exports.go.
  • Test: presence — a fixture org with a deliberately unbalanced trial balance shows passed = false for trial_balance; absence — after the fixing entry, the next run shows true (§4 Stage 3 Verify). This closes roadmap R30.
  • Counts by kind, state plus “stuck” (running > 30 min). The org comes from args->>'org_id' where the job carries one. Prometheus already has the time series (erp_river_jobs); this is the per-org table.

Task B5.5: V5 compliance: ops.compliance_flags()

Section titled “Task B5.5: V5 compliance: ops.compliance_flags()”
  • Per org: periods open past their end date; count of claim lines missing supplier TIN or e-Invoice UUID. Counts only.

Task B5.6: V7 security: ops.security_summary()

Section titled “Task B5.6: V7 security: ops.security_summary()”
  • Per org: failed and throttled sign-ins in the last 24 h and 7 days; count of role-permission changes, with org.roles.manage changes separate. Certificate expiry comes from the blackbox exporter via Grafana, not here.
  • Reads OPERATOR_BACKUP_STATUS_FILE (O12). Missing or stale over 26 h → red.

Verify B5: as erp_operator, each function returns rows, and SELECT * FROM organization is still denied.

As built (B5):

  • Migrations 00233 (role, org_ids), 00234 (org_overview), 00235 (check_result, check_summary), 00236 (job_summary), 00237 (compliance_flags), 00238 (security_summary).
  • Every ops function is RETURNS TABLE, so the guard can read its column names. The guard also checks the owner (erp_ops_owner) and who may EXECUTE (only erp_operator; org_ids only erp_app). Sabotaged three ways: an email column, a wrong owner, and erp_app EXECUTE.
  • ops.check_result stores finding_count, never finding text. A run that errors records one check_key = 'all' row with error_code = 'run_failed'. The job lives in internal/platform/opsjobs (ops_reconcile), 24 h, not on start. There is no pruning: about 10 rows per org per day, and the ops guard forbids DELETE. Add a retention decision if it ever matters.
  • V5 counts non-withdrawn receipt lines with no e-Invoice UUID, foreign suppliers excepted; relaxed receipts count. V7 has no throttled-attempt count: login_attempt has no organization.
  • Pages share one bare table.html: /orgs, /jobs, /integrity, /compliance, /security, /backups. An organization with no check result shows NOT YET CHECKED (never a pass). A missing, unset or malformed backup file shows UNKNOWN.
  • TestNoConsolePageShowsPersonalData renders every GET page for two onboarded orgs: their names appear, and their owners’ emails and legal names appear on no page (sabotage: an email in a page note went red).

Phase B6: Actions (D1, D2 decided 2026-09-28)

Section titled “Phase B6: Actions (D1, D2 decided 2026-09-28)”

Task B6.1: 00236_operator_action.sql + dispatcher (O6, O7)

Section titled “Task B6.1: 00236_operator_action.sql + dispatcher (O6, O7)”
  • ops.operator_action, ops.request_action(), ops.action_status(), ops.take_secret().
  • opsjobs.Dispatch periodic job: claim, run, record result_code. On a handler error the row gets failed + code, never the error text (it could carry record values).
  • Calls onboarding.CreateOrganization, the same function cmd/orgadmin create-org calls, so nothing moves. The invitation link is handled per O7. Writes the org’s audit_log (operator_created_org).

Task B6.3: A2 re-run provisioning, A3 verify now, A4 retry job

Section titled “Task B6.3: A2 re-run provisioning, A3 verify now, A4 retry job”
  • A2 calls provisioning.DefaultRegistry().Run in the org’s transaction.
  • A3 enqueues O9 for one org.
  • A4 retries only jobs River marks retryable or discarded, via river.Client.JobRetry.

Task B6.4: A5 suspend / reactivate: 00237_organization_status.sql (D2, O10)

Section titled “Task B6.4: A5 suspend / reactivate: 00237_organization_status.sql (D2, O10)”
  • organization.status text NOT NULL DEFAULT 'active' with CONSTRAINT organization_status_known CHECK (status IN ('active','suspended')), and a constants.yaml vocabulary entry + make generate.
  • Liveness gains status = 'active'. The session cache for the org is invalidated on change. Sign-in refuses org_suspended.
  • Test: presence — a member signs in and posts a claim; absence — after suspension, sign-in is refused, an existing token’s next request is refused, and a row count of every retention table for the org is unchanged (nothing deleted). Reactivate → sign-in works. The org’s audit_log shows both events (§4 Stage 4 Verify).

Verify B6: every action appears in ops.operator_audit and, for A1/A2/A5, in the org’s own audit trail.


As built (B6):

  • Migrations 00239 (ops.operator_action, claim_actions, finish_action, failed_jobs) and 00240 (organization.status, and org_overview gains status).
  • The console inserts actions directly, with column-level INSERT and SELECT on its own table; its only UPDATE is clearing secret_result. The worker reaches the table only through claim_actions and finish_action (both erp_app-only; the ops guard now keeps a workerFunctions list that is exempt from the console’s type rule). finish_action writes action_finished into the operator chain in SQL. TestActionsRunInTheWorkerAndAreAudited verifies Go and SQL rows as one chain (sabotaged: a changed SQL payload went red).
  • The dispatcher runs on its own queue operator (jobs.QueueOperator), so the nightly reconciliation cannot hold actions up. Results are codes (ok, run_failed, invalid_input, misconfigured, owner_member_required, company_profile_required, job_not_found, job_not_failed), never error text.
  • A2 passes no Owner (the console never sees members): a run that must still seat one finishes owner_member_required, and the CLI migrate provision --owner-member covers it.
  • A4 lists ids from ops.failed_jobs() (retryable or discarded only) and retries through River’s JobRetry; any other state finishes job_not_failed.
  • A1: params (owner email and name) is cleared by finish_action. The pages show the org name only (tested). The sealed link is revealed by a POST with CSRF, once, to the requester only (sabotaged: dropping the requester check went red).
  • A5 enforcement: member.MembershipsOnTx returns active organizations only (so login, refresh, the switcher and replay handling skip suspended ones); liveness refuses a suspended org (OrgSuspended, JSON z, so an entry cached before the deploy decodes as not suspended); login and refresh answer 403 org-suspended when every organization is suspended (api/openapi.yaml updated). organization.status is a CHECK-constrained string, not a constants.yaml entry (that file holds permissions only). Tests in internal/modules/identity/suspend_test.go; sabotaging the liveness check or the membership filter went red.
  • An action left running by a worker that died is closed by the next dispatch after 30 minutes (three dispatcher timeouts) as failed, abandoned, through ops.abandon_stale_actions (00241), which calls finish_action so the chain records it. Never re-run: its organization transaction may have committed, and a second create_org would make a second organization. TestAnActionWhoseWorkerDiedIsAbandoned (sabotaged with no threshold and with a 1000-minute one: both red).
  • Local check: the real console queued verify_now, but a still-running pre-B6 dev worker held River leadership, so it stayed pending. Restart pnpm start to pick up the new worker.
  • templates/layout.html: top bar (environment name, operator email, sign-out), left navigation (Organizations, Health, Jobs, Integrity, Compliance, Backups, Security, Releases, Actions).
  • Sign-in page: one form with email, password and the 6-digit code (B3 as built). The refusal message doesn’t reveal which factor was wrong.
  • static/console.css: one file, system font, status colours reusing the ERP’s tokens (2026-09-26-status-colours-design.md: green pass, red fail, amber unknown), readable at phone width.
  • 403 / 404 / 500 pages. Nothing echoes request input.

As built:

  • Each page template defines title and content (and optionally head); server.go parses layout.html plus one page per template set. Signed out, the layout hides the email and navigation.
  • The environment name is OPERATOR_ENV_NAME (optional; production gets a red badge). The stylesheet is the one extra public route, GET /static/console.css; CSP is unchanged (script-src 'none', no inline style).
  • Unknown addresses get the designed 404 (checked in ServeHTTP before the mux). TestErrorPagesEchoNothing sends a probe in the path, query and CSRF field; sabotaged by appending the query to the message, it went red.
  • --muted and the dark-theme link colour are darker/lighter than the ERP’s --fg-muted/--primary, which fall under 4.5:1 as text.
  • Local run: .claude/launch.json has an operator entry. The dev stack leaves TOTP_ENCRYPTION_KEY empty, so it keeps a local key in .dev-stack/operator-totp-key; OPERATOR_CMD=create-account OPERATOR_ARGS="--email …" runs the CLI with the same key. Checked in a browser: light and dark, 375 px wide with no page scroll, sign-in, sign-out.
  • Left for F2: “NOT YET CHECKED” still renders red (a Cell has only Bad); F2 gives cells a pass/fail/unknown tone.
  • One table per view, sortable by server query parameter (no JavaScript). Every red cell names its detail_code and links to the runbook section, not to data.
  • V1 row → organization page: that org’s V3/V4/V5/V7 rows together, plus the action buttons (F3).
  • “Last updated” on every page, and a “run now” button on V4 (A3).

As built:

  • A Cell has a tone (pass green, fail red, unknown amber), a detail code and a RUNBOOK section. “NOT YET CHECKED”, backup UNKNOWN and a suspended organization are amber; failed checks, failed/stuck jobs, compliance flags, token replays and org.roles.manage changes are red. The codes and what to do about each are RUNBOOK §7. There is no hosted runbook to link to, so a cell names the section (RUNBOOK §7.3) rather than linking. TestEveryDetailCodeHasARunbookEntry checks every code is explained in its section and every built-in check has a code (sabotaged by removing one code from the RUNBOOK: red).
  • ?sort=<column>&dir=desc sorts, numbers as numbers (TestSortIsNumericAndDescendable).
  • /orgs/{id}: the V1, V4, V3, V5 and V7 rows for one organization, filtered by the organization link in each row’s first cell, plus Run checks now, Re-run provisioning and Suspend/Reactivate. These post straight to /actions/org until F3 adds the confirm step. The privacy test now covers it, and asserts the other organization is absent (sabotaged by not filtering: red).
  • Every signed-in page shows “Data as of” (the console’s clock at render). V4 ran-at times remain per row.
  • V2 health is a table with PASS/FAIL/UNKNOWN; V8 releases shows an alert when migrations are pending.
  • Each action is a form, then a confirmation page restating the organization and the effect (“Suspending blocks sign-in and changes. No data is deleted.”), then a pending page that refreshes via <meta http-equiv="refresh"> until the result is in.
  • A1 result page shows the invitation link once, with a warning that it will not be shown again (O7).

As built:

  • No new routes: an action form’s first POST renders confirm.html (the facts, the effect from actionInfo, and the form re-posted with confirm=yes); only the confirmed POST records the action. Suspend’s page and button are red. TestEveryActionAsksForConfirmation covers all four forms (sabotaged by skipping the check: red).
  • The create-org confirmation restates the owner’s email and legal name, so a typo is caught before the invitation exists. They are what the operator just typed; no page shows them once the request is stored (the existing A1 test still holds).
  • The action page names the action, shows pending/running (amber, refreshing), done (green) or failed (red, with the result code explained by resultText; TestEveryResultCodeIsExplained covers every opsjobs.Result*). For a finished create-org: a “Show the invitation link” button for the requester; the link in a box with “Copy this link now. It will not be shown again.”; for another operator, who asked; once taken, that it cannot be shown again (RUNBOOK §6.1).
  • Checked in a browser: Suspend’s confirmation (cancelled, not confirmed) and Run checks now through to its pending page. The done and link states were checked by tests only, because the running dev worker predates B6 and never ran the action; the test action and operator were deleted.
  • The sign-in screen shows the org_suspended refusal as its own translated sentence (en/ms/zh, i18nparity test), following the existing coded-refusal pattern (AP fix I4).
  • The organization’s audit trail shows operator_created_org, operator_provisioned, operator_suspended and operator_reactivated with translated labels, so the customer sees what the provider did (§2.2).
  • Playwright: suspend in a test fixture → sign-in shows the sentence; reactivate → sign-in works. Passed 2026-09-29; red with the suspension removed.

As built:

  • The problem kind is org-suspended (B6 as built), handled next to no-organization in LoginPage.tsx: “Your organization has been suspended by its service provider, so no one can sign in. Nothing has been deleted. Ask your administrator to contact the provider.” A member whose token stops working mid-session is sent to sign-in by the existing expired-session path and sees the sentence when they try.
  • The web app has no audit-log screen: what an organization’s people read is the Activity feed (activity_log). So the dispatcher’s orgAudit now also records an activity event (subject organization, verb = the audit action, an English summary as fallback) in the same transaction. The Activity page shows a translated label for any verbs.<subject>.<verb> key it has, else the server summary as before, and “Service provider” as the actor for operator_* verbs. The Go action tests now require an activity row for every audited operator action (sabotaged by dropping it: red); the Activity page test checks the label (sabotaged by showing the summary: red); i18nparity caught a removed ms key.
  • e2e/org-suspension.spec.ts sets organization.status on the dev stack’s database (docker exec erp-dev-pg psql), since no public API suspends an organization; afterEach reactivates. Run with pnpm -C web e2e org-suspension against a dev stack started after B6.

  • Every §6 drift test exists and was seen failing once.
  • docs/ROADMAP.md “Operator console” ticks Stage 0–4. Progress doc P2–P11 and P14–P15 ticked.
  • docs/DEPLOY_NOTES.md: the erp_operator password, the new OPERATOR_* variables, bin/operator, and the private-network requirement.
  • One full go test ./..., make lint, and pnpm -C web test / e2e green.
  • A6 tenant export: with spec §17.3.
  • A7 support access: its own design spec first (§3.7). It touches authentication.
  • Server setup (I1–I15): Prometheus, Grafana, alerts, backups, private network. Blocked on the host decision.