pikasTech-HWLAB/docs/dev-deploy-apply.md

HWLAB DEV Deploy Apply

This document records the DEV-only apply/preflight path for pikasTech/HWLAB#33 and the apply-plan/report path for pikasTech/HWLAB#50. It targets only the D601 hwlab-dev namespace and must not be used for PROD.

Scope

• Work from latest origin/main.
• Read only HWLAB repository deploy inputs: deploy/deploy.json, deploy/artifact-catalog.dev.json, and deploy/k8s/dev.
• Validate artifact identity, image commit tags, Kubernetes workload/service shape, DEV health, and cloud-api DB connectivity.
• Emit an operator-readable plan in devDeployApply: plan/apply boundary, expected artifact commit, target namespace, workloads, services, remaining blockers, minimal manual commands, and rollback hints.
• Plan and, in apply mode, execute a scoped replacement for DEV suspended template Jobs whose image changed but whose Kubernetes spec.template is immutable.
• Stop before mutation when artifact publish evidence, registry digests, kubectl, DEV health, or cloud-api DB connectivity are missing.

Commands

Run a preflight that is expected to report blockers on the current skeleton:

node --check scripts/dev-deploy-apply.mjs
node --check scripts/src/dev-deploy-apply.mjs
node scripts/dev-deploy-apply.mjs --dry-run --expect-blocked --write-report
node scripts/validate-dev-gate-report.mjs

Run the existing CLI dry-run surface:

node tools/hwlab-cli/bin/hwlab-cli.mjs test e2e --env dev --mvp --dry-run

Only after CI publish evidence and executor prerequisites exist, a real DEV apply requires explicit non-production confirmation:

node scripts/dev-deploy-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report

The script refuses PROD flags, secret-read flags, heavyweight e2e flags, and force-push flags. It also refuses to apply if the DEV artifact catalog still contains skeleton-only unpublished images.

Report Contract

The machine-readable result is written to reports/dev-gate/dev-deploy-report.json when --write-report is used. The report remains compatible with the #33 DEV gate validator and adds these operator fields:

• devDeployApply.conclusion: blocked while blockers are open, otherwise ready for human approval.
• devDeployApply.applyBoundary: default no-write behavior, required apply flags, DEV-only write scope, no-write scope, and forbidden actions.
• devDeployApply.artifactPlan: expected artifact commit, source commit, deploy/catalog commit IDs, publish/registry status, and unpublished services.
• devDeployApply.target: DEV endpoint and hwlab-dev namespace.
• devDeployApply.workloadPlan and servicePlan: planned Kubernetes workloads, containers, images, service IDs, ports, and replica intent.
• devDeployApply.templateJobReplacementPolicy: the DEV-only allowlist for suspended template Job replacement. The only allowlisted Jobs are hwlab-agent-worker-template and hwlab-cli-template in hwlab-dev.
• devDeployApply.templateJobReplacements: per-Job replacement decisions with namespace, Job name, old image, new image, and result (planned, replaced, not_needed, not_found, or blocked/failure states).
• devDeployApply.manualCommands: dry-run/report commands before human approval; the apply command is present only when the plan is ready.
• devDeployApply.rollbackHint: pre-apply capture commands, deployment rollbacks, suspended job cleanup, and post-rollback health checks.
• devDeployApply.remainingBlockers: open blockers with concrete unblock hints. These do not claim live apply happened.

When conclusion.status is blocked, operators should clear the listed remainingBlockers and rerun:

node scripts/dev-deploy-apply.mjs --dry-run --expect-blocked --write-report
node scripts/validate-dev-gate-report.mjs

When conclusion.status is ready, run the dry-run plan and validator immediately before human approval:

node scripts/dev-deploy-apply.mjs --dry-run --write-report
node scripts/validate-dev-gate-report.mjs

Only after that approval may the DEV-only apply command be used.

Suspended Template Job Replacement

Kubernetes Jobs cannot update spec.template in place. For DEV template Jobs that are intentionally suspended and used as templates, an image tag change would otherwise make a normal kubectl apply -k deploy/k8s/dev non-idempotent.

The apply script handles only the explicit DEV allowlist:

• hwlab-agent-worker-template
• hwlab-cli-template

For each allowlisted Job, the script reads the live Job in hwlab-dev and compares its container image with the desired manifest image. If the live Job exists, is suspended, and the image differs, apply mode deletes that Job before running the normal kustomize apply so Kubernetes recreates it from the desired manifest. Dry-run mode records the same decision as planned and runs server-side kustomize dry-run without mutating the cluster. If that dry-run reports only the expected immutable spec.template errors for the planned allowlisted Jobs, the report records those errors as expected replacement evidence instead of treating them as blockers.

The policy refuses any other Job name, any non-hwlab-dev namespace, and any live Job that is not suspended. It does not change Deployment, Service, ConfigMap, Secret, or PROD behavior.

Current Result

The current DEV apply contract is published for artifact commit cb35ada. deploy/artifact-catalog.dev.json, deploy/deploy.json, and the DEV workloads point at 127.0.0.1:5000/hwlab/*:cb35ada images with sha256 digest evidence.

The D601 runner can read and apply the hwlab-dev namespace. A real DEV apply may replace only the allowlisted suspended template Jobs above; the normal Deployment, Service, ConfigMap, and health-contract apply path remains unchanged.

hwlab-cloud-api has a DEV DB env contract placeholder: HWLAB_CLOUD_DB_URL must come from Secret reference hwlab-cloud-api-dev-db/database-url, and HWLAB_CLOUD_DB_SSL_MODE=require. The local health gate reports DB runtime config without printing the connection string.

No PROD resource, secret value, UniDesk runtime substitute, browser e2e, heavyweight e2e, or force push is part of this flow.