docs: add durable runtime unblock runbook
This commit is contained in:
@@ -174,8 +174,7 @@ Do not declare DEV-LIVE complete while `runtime.adapter="memory"` or
|
||||
`runtime.durable=false`, even when DB live readiness is connected. A selected
|
||||
Postgres adapter may report `runtime.durableRequested=true`, but it still stays
|
||||
non-durable until the schema, migration ledger, and read readiness gates are
|
||||
proven through the configured adapter. Documenting or testing this contract
|
||||
must not mutate runtime state or PROD.
|
||||
proven through the configured adapter.
|
||||
|
||||
`/health` and `/health/live` expose this split in `readiness.durability`.
|
||||
`dbLiveEvidenceObserved=true` with
|
||||
@@ -185,6 +184,13 @@ separate from runtime persistence. If runtime durability is blocked,
|
||||
layer, and `requiredEvidence` remains the durable adapter schema, migration,
|
||||
and read query contract.
|
||||
|
||||
When the active blocker is `runtime_durable_adapter_query_blocked`, DB
|
||||
connectivity is live but the durable runtime adapter has not completed the
|
||||
required read queries through the runtime store. Future reports must call this a
|
||||
runtime durability blocker, not a DB connectivity blocker, and must keep M3,
|
||||
M4, and M5 full acceptance blocked until the durable runtime postconditions
|
||||
below are true.
|
||||
|
||||
## Runtime Migration Automation
|
||||
|
||||
The repo-owned runtime migration entrypoint is:
|
||||
@@ -229,6 +235,71 @@ Runtime migration reports separate the blockers as:
|
||||
| Migration | The required source migration is recorded in the ledger. |
|
||||
| Readiness | Durable runtime read readiness passed after schema and ledger checks. |
|
||||
|
||||
## Durable Runtime Unblock Runbook
|
||||
|
||||
This runbook is a reusable checklist for a future authorized live
|
||||
mutation/migration. It does not itself authorize a live write. Runners working
|
||||
from source must stop at the read-only rows unless the commander explicitly
|
||||
authorizes the DEV apply command and the preconditions below are already true.
|
||||
|
||||
| Check | Safe for normal runners | Evidence allowed | Not allowed |
|
||||
| --- | --- | --- | --- |
|
||||
| Source migration contract | Yes | `node scripts/dev-runtime-migration.mjs --check` and `node scripts/dev-runtime-migration.mjs --dry-run --write-report` | DB connection, Secret read, DEV/PROD write |
|
||||
| Static runtime boundary contract | Yes | `node scripts/validate-runtime-boundary.mjs`, source docs, manifests, and redacted Secret refs | Live mutation, service restart, Secret value output |
|
||||
| Public health observation | Yes, when command scope is read-only | `/health/live` fields such as `db.liveDbEvidence`, `runtime.adapter`, `runtime.durable`, `runtime.blocker`, `readiness.durability.blockedLayer`, `requiredEvidence`, and redaction/safety flags | Printing DB URLs, passwords, tokens, kubeconfig material, or Secret data |
|
||||
| Live DB read verification | Only with explicit DEV read authorization | `--dry-run --allow-live-db-read --confirm-dev --write-report` output with redacted endpoint and `secretValuesPrinted=false` | Kubernetes Secret reads, writes, migration apply, PROD target |
|
||||
| Live migration/repair apply | No, unless explicitly authorized by commander/operator | `--apply --confirm-dev --confirmed-non-production --write-report` plus post-apply health evidence | PROD apply, Secret value output, service restart, Code Agent POST, hardware write, or acceptance promotion |
|
||||
|
||||
Secret handling rules are the same for every row: verify env names,
|
||||
`secretKeyRef` names/keys, configured provider/model/base URL, redacted endpoint
|
||||
source, readiness booleans, blocker codes, and `secretValuesPrinted=false`.
|
||||
Do not run `kubectl get secret`, do not decode Secret data, and do not paste
|
||||
`HWLAB_CLOUD_DB_URL`, `OPENAI_API_KEY`, bearer tokens, kubeconfig content, or
|
||||
passwords into reports, PRs, issues, logs, screenshots, or chat.
|
||||
|
||||
Before any authorized live migration or repair, all of these preconditions must
|
||||
be true:
|
||||
|
||||
- The commander has explicitly authorized a DEV-only migration/repair for this
|
||||
blocker; no PROD target, deploy, restart, or unrelated smoke is bundled into
|
||||
the same action.
|
||||
- Source validation passes for `internal/db/migrations/0001_cloud_core_skeleton.sql`,
|
||||
the durable runtime tables/columns, and the `hwlab_schema_migrations` ledger
|
||||
row.
|
||||
- The runtime is requesting the Postgres durable adapter, not silently using
|
||||
memory: `runtime.adapter="postgres"` or equivalent selected-adapter evidence,
|
||||
with `runtime.durableRequested=true`.
|
||||
- DB live evidence is already green through the redacted Secret URL host:
|
||||
`db.liveDbEvidence=true` and `db.endpointSource="secret-url-host"`.
|
||||
- The current blocker is recorded exactly, for example
|
||||
`runtime_durable_adapter_query_blocked`, with `blockedLayer` and
|
||||
`requiredEvidence` preserved in the report.
|
||||
- The runner/operator can produce a redacted report that states
|
||||
`secretValuesPrinted=false` and does not require reading Kubernetes Secret
|
||||
resources.
|
||||
|
||||
After an authorized migration/repair, the blocker has moved or cleared only
|
||||
when a new read-only health/report observation proves one of these exact
|
||||
postconditions:
|
||||
|
||||
- Cleared: `runtime.adapter="postgres"`, `runtime.durable=true`,
|
||||
`runtime.ready=true`, `runtime.liveRuntimeEvidence=true`,
|
||||
`readiness.durability.ready=true`, no
|
||||
`runtime_durable_adapter_query_blocked`, and durable auth/schema/migration/read
|
||||
gates are all ready.
|
||||
- Moved: the old `runtime_durable_adapter_query_blocked` code is absent and a
|
||||
different explicit blocker is present, such as auth, schema, migration, or
|
||||
driver readiness; the report must name the new blocker and keep acceptance
|
||||
blocked.
|
||||
- Still blocked: `runtime_durable_adapter_query_blocked` remains present,
|
||||
`runtime.durable=false`, or `runtime.liveRuntimeEvidence=false`; no full
|
||||
M3/M4/M5 acceptance may be claimed.
|
||||
|
||||
No full M3, M4, or M5 acceptance is allowed while runtime durability is blocked.
|
||||
DB live readiness, Code Agent readiness, source migration readiness, and
|
||||
read-only health observations are supporting evidence only; they do not prove
|
||||
durable runtime evidence or the M3 trusted hardware loop.
|
||||
|
||||
## Runtime Substitution Ban
|
||||
|
||||
UniDesk services, provider-gateway, backend-core, microservice proxies, local
|
||||
|
||||
Reference in New Issue
Block a user