pikasTech-HWLAB/docs/reference/deployment-publish.md

# HWLAB Deployment And Publish Reference

This reference records the stable DEV publish and rollout rules. It does not
authorize PROD, service restarts outside the documented DEV path, secret reads,
or direct pushes to `main`.

It is also the long-term reference for
[pikasTech/HWLAB#116](https://github.com/pikasTech/HWLAB/issues/116): service
deployment regularization must progress from stable docs, to controlled
CLI/script entrypoints, to UniDesk CI/CD plus imageized delivery.

## D601 Native k3s Emergency Guard

HWLAB DEV deploy and publish paths must target D601 native k3s only. Docker
Desktop Kubernetes on D601 has been disabled and cleaned; it must not be
re-enabled or used as a second orchestrator. The cross-repo incident, evidence,
and governance plan are tracked in
[pikasTech/unidesk#138](https://github.com/pikasTech/unidesk/issues/138), and
the 2026-05-23 commander recovery context is in
[pikasTech/unidesk#118](https://github.com/pikasTech/unidesk/issues/118).

Every DEV CD status, apply, rollback, smoke, and manual Kubernetes command must
use `KUBECONFIG=/etc/rancher/k3s/k3s.yaml` and verify node `d601` before any
mutation or acceptance. Bare `kubectl`, `docker-desktop` context,
`desktop-control-plane`, or `127.0.0.1:11700` are wrong-control-plane signals,
not HWLAB DEV-LIVE evidence. A second `hwlab-dev` control plane, including a
Docker Desktop cluster that happens to contain similarly named resources, is a
blocker and cannot be used as deploy, rollout, smoke, or acceptance proof.

## Workspaces

| Purpose | Path |
| --- | --- |
| Runner and commander worktree | `/workspace/hwlab` |
| D601 publish/build worktree | `/home/ubuntu/workspace/hwlab` |

Use `/home/ubuntu/workspace/hwlab` for D601 build and rollout operations.
Do not treat `/home/ubuntu/hwlab` or other runner worktrees as the publishing
truth.

## Regularization Stages

| Stage | Current HWLAB state | Required next behavior |
| --- | --- | --- |
| 1. Long-term reference | This file records DEV deployment, Cloud Web publish, API/edge/health, frp, D601 registry, k3s rollout, rollback, and verification rules. `AGENTS.md` indexes it. | Keep process issues as sources only; update this reference when a deployment rule changes. |
| 2. CLI or controlled scripts | `scripts/dev-cd-apply.mjs` is the only formal DEV CD side-effect entrypoint for publish, desired-state refresh, apply, and live verify. Other controlled scripts still cover contract planning, dry-run planning, preflight, single-step internals, k3s read-only visibility, and Cloud Web checks. Rollback is documented as commands/report hints but is not yet one unified CLI command. | Converge check, plan, rollback, status, and verify into HWLAB CLI while preserving the single transaction lock for side effects backed by `deploy/deploy.json`. |
| 3. UniDesk CI/CD plus images | DEV services are imageized and published to the D601 registry. D601 rollout still includes manual bridge steps for Cloud Web and frp/k3s execution. | Move build, publish, desired-state refresh, rollout, post-deploy validation, and report upload into standard UniDesk CI/CD jobs without replacing HWLAB runtime. |

## DEV CD Transaction

正式 DEV CD 写路径只有一个入口：

```sh
node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report
```

该命令是唯一允许一次性发布 DEV artifact、从发布报告刷新 desired
state、apply 到 `hwlab-dev`、并复验公开 `16666/16667` 的正式写路径。
事务开始时读取 `deploy/deploy.json`，记录 manifest hash 和 target commit。
在获取 `Lease/hwlab-dev/hwlab-dev-cd-lock` 或运行任何 publish、runtime Job、
apply、rollout 类副作用前，脚本必须先完成 apply-only preflight：强制
`KUBECONFIG=/etc/rancher/k3s/k3s.yaml`，验证节点包含 `d601`，拒绝
`docker-desktop`、`desktop-control-plane`、`127.0.0.1:11700`、裸
`kubectl` 指向第二套 `hwlab-dev` 控制面或其他不可信控制面信号，并且只验证
必要 SecretRef 的存在性和 key 名。通过后才获取 Lease、串行执行内部步骤、
按需写入 `reports/dev-gate/dev-cd-apply.json`，最后释放 Lease。

apply preflight 检查的 SecretRef 固定为：

- `hwlab-cloud-api-dev-db/database-url`
- `hwlab-cloud-api-dev-db-admin/admin-url`
- `hwlab-code-agent-provider/openai-api-key`

preflight 和报告只能记录 Secret 名、key 名、存在性、key-name 观察结果和脱敏
命令摘要；不得读取或打印 Secret value。阻塞时输出结构化 blocker，包含
`scope`、`reason`、`impact`、`safeNextAction` 和 `retryable`，默认 stdout 只
给有界摘要，完整 preflight 证据写入事务报告。

`deploy/deploy.json` 的 `commitId` 是运行时 promotion commit。正常
latest-main 发布时它可以等于当前 target ref；artifact merge commit 或后续
控制面提交存在时，它也可以指向一个已经发布并在 catalog/workloads 中完全收敛
的早期 promotion commit。`scripts/dev-cd-apply.mjs` 必须先用
`deploy-desired-state-plan --promotion-commit <commit>` 验证该 desired-state
已经发布且内部一致；验证通过后，`--apply` 消费这个 promotion commit 的镜像
执行 apply/verify，不重新 publish 或刷新当前 HEAD 的 artifact。这样 host
commander 可以始终从最新 `origin/main` 控制入口执行正式 DEV CD，而不需要
checkout promotion parent 或手工拼装发布环境。

状态面和事务报告会显式展示这条边界：`target.controlRef` 是最新
`origin/main` 控制入口，`target.promotionCommit` 是从 `deploy/deploy.json`
解析出的真实 promotion commit，`target.artifactBoundary` 记录最近一次触碰
`deploy/deploy.json`、`deploy/artifact-catalog.dev.json`、
`deploy/k8s/base/workloads.yaml`、`reports/dev-gate/dev-artifacts.json` 的
first-parent artifact merge commit，以及该 merge commit 的 promotion parent
和 artifact/report parent。`artifactCatalog`、`artifactReport`、`deployJson`
分别记录 hash、commit、digest 计数和 publish source。若 deploy/catalog/report
或 artifact merge parent 不能同时指向同一个 promotion commit，状态降级为
`degraded`，真实 `--apply` 在获取 Lease 前以 artifact-boundary blocker 停止。
这保证 `deploy/deploy.json` 仍是唯一发布控制源；Lease 只串行化事务，不保存第
二套 desired-state。

Lease 只是发布平面的互斥锁，不是 desired-state 来源。Lease annotations
至少记录 `promotionCommit`、`deployJsonHash`、`ownerTaskId`、
`transactionId`、`phase`、`startedAt`、`updatedAt`、`ttlSeconds`、
`liveBefore` 和 `targetNamespace`。`deploy/deploy.json`、
`deploy/artifact-catalog.dev.json`、`deploy/k8s/base/workloads.yaml` 仍是唯
一权威部署输入。

如果另一个事务持有 Lease，`--apply` 必须在 publish/apply 之前停止，并
输出结构化 `deploy-lock-held`，包含 holder、promotion commit、phase、
target namespace 和 retry seconds。过期锁不能静默覆盖；stale lock 只能
在确认 holder 已失活后显式使用 `--break-stale-lock --confirm-dev`，并把
previous holder 保留在审计 annotations/report 字段中。未过期 Lease 没有
抢占入口，普通 status 输出也不得建议强制抢锁。

`scripts/dev-cd-apply.mjs` 同时提供只读状态面。无参数、`--status`、或
`--dry-run` 都返回紧凑 JSON，覆盖 target ref/commit、
`deploy/deploy.json` commit/hash、当前 Lease 状态摘要、以及公开
`16666/16667` live health identity 摘要。状态面还包含 `liveDelta`，用于对比
当前公开 live commit 与 `target.promotionCommit`；这只是 rollout 口径，不会在
status/dry-run 中执行 rollout。这些模式 host commander 和 runner 都可以执行，
但必须保持只读：不获取或替换 Lease，不 publish artifact，不执行
`kubectl apply/rollout`，不写 report，不修改 `deploy.json`/catalog，也不读取或
打印 Secret value。kubectl 或 kubeconfig
不可用时，状态面应返回 `lock.status=unavailable` 和脱敏原因，而不是把
只读观测失败升级成写路径 blocker。

## `deploy.json` DEV CD 路径

当前已经验证可用的过渡路径是：master server 只作为受控指挥入口，通过
UniDesk CLI 进入 D601；D601 上使用干净 HWLAB 工作区运行本仓库
`scripts/dev-cd-apply.mjs`；发布目标由 `deploy/deploy.json` 和同 commit
的 artifact catalog / k8s workloads 决定；真实写入只落到 D601 原生 k3s
`hwlab-dev` namespace。复盘来源见
[pikasTech/HWLAB#61](https://github.com/pikasTech/HWLAB/issues/61#issuecomment-4525401375)，
长期化跟踪见 [pikasTech/HWLAB#340](https://github.com/pikasTech/HWLAB/issues/340)，
单事务入口和发布锁见 [pikasTech/HWLAB#274](https://github.com/pikasTech/HWLAB/issues/274)，
D601 控制面归一见 [pikasTech/unidesk#138](https://github.com/pikasTech/unidesk/issues/138)。

稳定调用合同如下；外层 master CLI 或 UniDesk wrapper 只能包裹这些
repo-owned 入口，不能绕过它们直接拼接 publish、apply、rollout 或 smoke：

| 目的 | 稳定入口 | 副作用边界 |
| --- | --- | --- |
| 读取 DEV CD 状态 | `node scripts/dev-cd-apply.mjs --status` | 不写入、不获取 Lease、不读取 Secret value。 |
| 事务 dry-run preflight | `node scripts/dev-cd-apply.mjs --dry-run` | 不 publish、不 apply、不 rollout。 |
| 检查 desired-state 收敛 | `node scripts/deploy-desired-state-plan.mjs --promotion-commit <sha> --check` | 只读 source/report。 |
| 执行 DEV CD 事务 | `node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report` | 仅 DEV，在 `Lease/hwlab-dev/hwlab-dev-cd-lock` 下 publish/apply/verify。 |
| 读取 D601 k3s 证据 | `node scripts/d601-k3s-readonly-observability.mjs` | 只读可见性报告；不 rollout、不输出 Secret value。 |

使用 master-side wrapper 时，稳定形态是：

```sh
cd /root/unidesk
bun scripts/cli.ts ssh D601 argv bash -lc 'cd <clean-hwlab-cd-worktree> && KUBECONFIG=/etc/rancher/k3s/k3s.yaml node scripts/dev-cd-apply.mjs --status'
bun scripts/cli.ts ssh D601 argv bash -lc 'cd <clean-hwlab-cd-worktree> && KUBECONFIG=/etc/rancher/k3s/k3s.yaml node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report'
```

`<clean-hwlab-cd-worktree>` 必须指向干净工作区。该工作区可以是固定 clean
mirror，也可以是一次性 `git clone --shared --no-checkout` 生成的 ephemeral
worktree；它不能是 `/home/ubuntu/hwlab` 这类 runner/历史任务集合目录。进入
apply 前必须证明：

- `git status --short` 为空；
- `.git/FETCH_HEAD` 和 `.git/worktrees` 没有阻止普通部署用户读取或创建
  干净工作区的权限问题；
- `node scripts/dev-cd-apply.mjs --status` 显示 `promotionSource=deploy-json`；
- `deploy/deploy.json` 的 commit、`deploy/artifact-catalog.dev.json` 和
  `deploy/k8s/base/workloads.yaml` 收敛到同一个 promotion commit / image tag；
- `KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl get nodes` 能看到节点 `d601`；
- `kubectl config current-context`、cluster server 或节点列表没有
  `docker-desktop`、`desktop-control-plane`、`127.0.0.1:11700` 或第二套
  `hwlab-dev` 控制面证据；
- `Lease/hwlab-dev-cd-lock` 未被未过期事务持有，或者 stale lock 已由受控
  `--break-stale-lock --confirm-dev` 审计处理。

desired-state 来源关系固定如下：

| 来源 | 权威范围 |
| --- | --- |
| `deploy/deploy.json` | DEV environment、namespace、endpoint、service set、promotion commit 和事务消费的 image tag。 |
| `deploy/artifact-catalog.dev.json` | 同一 promotion commit 的不可变 artifact digest/tag 证据。 |
| `deploy/k8s/base/workloads.yaml` | Kubernetes workload desired state，必须镜像 `deploy.json` 和 artifact catalog 后才能 apply。 |
| `reports/dev-gate/*.json` | 只作为证据快照；不能覆盖 repo desired state。 |
| `Lease/hwlab-dev/hwlab-dev-cd-lock` | 只作为事务互斥锁和审计状态；永远不是 desired-state 来源。 |

只改文档的 commit 如果没有修改 `deploy/deploy.json`、
`deploy/artifact-catalog.dev.json`、`deploy/k8s/base/workloads.yaml`、service
source 或 runtime manifest，就不会产生新的 DEV image 或 promotion commit。
这类 commit 的 artifact build/publish 结论是 not applicable，除非另一个
artifact 任务明确更新 desired state。

## Runner 和 Host 边界

Deployment/CD 工作按以下边界分工：

| 角色 | 可以执行 | 没有明确 rollout 授权时不得执行 |
| --- | --- | --- |
| Code Queue runner | 准备分支和 PR；在任务授权且检查通过后更新或自合并 docs/code/artifact PR；通过 repo-owned 路径 build/publish artifact；在被分配时刷新 artifact report 或 desired-state 文件；运行只读 `status`/`dry-run`/preflight 检查。 | 竞争 DEV CD Lease、执行 DEV apply、rollout、声称 `16666/16667` live verification、收口 `#7` 或 `#242`、修改 PROD、打印 Secret value、手工修 live resource。 |
| Host commander | 在需要指挥官 review 时审阅并合并最终 PR；执行单一 DEV deploy apply 事务；执行 rollout 和 `16666/16667` live verification；收口 `#7`/`#242` 状态。 | 绕过 repo-owned CD 入口、使用第二控制面、用 UniDesk runtime 替代 HWLAB runtime、只凭报告而不做一手复验就当作 live evidence。 |

Runner 自合并权限只在任务边界内生效。它不授予 DEV apply、rollout、
live health verification 或看板/status 收口权限。Runner 发布 artifact 时，
输出必须包含 tag、digest 和 report 证据，并把 desired-state convergence 与
live rollout evidence 分开。

## SecretRef Preflight

apply 前还必须把必要 SecretRef 作为 preflight 条件处理，而不是等待
runtime Job 进入 `CreateContainerConfigError`：

- `hwlab-cloud-api-dev-db/database-url`；
- `hwlab-cloud-api-dev-db-admin/admin-url`；
- `hwlab-code-agent-provider/openai-api-key`。

这些检查只能验证 Secret 名称、key 和 env 注入边界，不得打印 Secret value。
如果 `hwlab-cloud-api-dev-db-admin/admin-url` 缺失，短期 runbook 是由受控
DEV-only secret reconcile 从 `unidesk-dev/postgres-dev` runtime source 生成
并写入 `hwlab-dev`；长期应迁移到 declarative secret、sealed/external secret
或统一 secret controller。

SecretRef preflight report 必须使用布尔值和脱敏 metadata：

| 检查项 | 必要证据 | 缺失时 blocker |
| --- | --- | --- |
| Source desired-state reference | `deploy/deploy.json` 和 `deploy/k8s/base/workloads.yaml` 声明预期 `secretRef` / `secretKeyRef` name 和 key。 | `contract_blocker`，scope 指向所属 manifest 或 service。 |
| Live D601 Secret presence | 在 `hwlab-dev` namespace 中，repo-owned preflight 或 controller 证明必要 Secret resource 和 key 存在。 | `runtime_blocker`，scope 指向缺失的 SecretRef。 |
| k3s access for the check | 检查使用 `/etc/rancher/k3s/k3s.yaml`，解析到节点 `d601`，没有命中 Docker Desktop 或第二控制面。 | `environment_blocker` 或 `observability_blocker`，scope 指向 D601 k3s visibility。 |
| Redaction | Report 字段声明 `secretValuesPrinted=false`；日志不包含 decoded data、DSN、token、password、kubeconfig material 或 base64 Secret payload。 | `safety_blocker`；在 publish/apply 前停止。 |

preflight 可以报告 Secret name、key name、namespace、presence boolean、
owning workload、missing scope 和 repair runbook。它不得粘贴
`HWLAB_CLOUD_DB_URL`, `HWLAB_CLOUD_DB_ADMIN_URL`, `OPENAI_API_KEY`, bearer
tokens, passwords, kubeconfig contents, or Secret `.data` values into reports,
issues, PRs, logs, screenshots, or chat.

## Blocker 分类

DEV CD report 必须保持 blocker 分类稳定，使 runner 和 host 不读完整事务日志
也能路由工作：

| Type | 含义 |
| --- | --- |
| `safety_blocker` | 请求动作会违反 DEV-only、non-secret、non-PROD、lock 或控制面安全规则。 |
| `environment_blocker` | 必要 local/D601 tool、kubeconfig、k3s node、registry、workspace 或 permission 不可用。 |
| `contract_blocker` | Source manifest、desired-state 文件、artifact catalog、report schema 或 promotion provenance 不一致。 |
| `runtime_blocker` | DEV runtime 前置条件或后置条件缺失，包括 SecretRef presence、DB readiness、durable runtime、rollout 或 workload image convergence。 |
| `agent_blocker` | Code Agent provider/model/egress/SecretRef readiness 不完整。 |
| `network_blocker` | public 或 internal endpoint 不可达，或返回无效 health response。 |
| `observability_blocker` | runner 无法观测足够只读证据来证明状态，但底层 runtime 可能仍然健康。 |

每个 blocker 必须包含 `type`、`scope`、`status` 和单行 `summary`。可行时还
应包含 `unblockHint` 或 `nextTask`。Secret 相关 blocker 必须声明没有打印
value。

成功收口必须同时具备以下证据：

- `scripts/dev-cd-apply.mjs --status` 返回 `status=pass`；
- `Lease/hwlab-dev-cd-lock` 为 `phase=released`、`releaseStatus=pass`；
- `16666/health/live` 的 Cloud Web revision 等于 `deploy.json` promotion commit；
- `16667/health/live` 的 Cloud API commit、DB readiness、runtime durability 均 ready；
- `KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev get deploy,statefulset`
  显示 persistent workload image tag 收敛到同一 promotion commit；
- 输出保持 bounded summary，完整 report 写入 `reports/dev-gate/dev-cd-apply.json`
  或 CLI dump 文件，避免把长 JSON 直接塞进指挥上下文。

正常 `--apply` stdout 默认也是紧凑摘要；完整事务记录继续通过
`--write-report` 写入既有 report 路径，只有显式 `--full-output` 时才打印
完整 JSON。DEV apply、rollout 和 live verification 仍由 host commander
统一执行；runner 默认只使用 status/dry-run 观测面，不竞争 DEV CD lock。

旧的 side-effect 命令已经降为事务内部步骤。直接手动执行
`scripts/dev-artifact-publish.mjs --publish` 或
`scripts/dev-deploy-apply.mjs --apply` 且没有 `HWLAB_CD_TRANSACTION_ID` 时，
会以 `cd-transaction-required` 拒绝；preflight、build、dry-run、check 和
read-only 模式仍可用。

## Artifact Publish

DEV artifacts publish to the D601 local/internal registry:

```text
127.0.0.1:5000/hwlab/*
```

The single-step publish command is a transaction-internal side-effect step:

```sh
node scripts/dev-artifact-publish.mjs --publish
```

It only runs in side-effect mode when `HWLAB_CD_TRANSACTION_ID` is set by
`scripts/dev-cd-apply.mjs`. Use `node scripts/dev-cd-apply.mjs --apply
--confirm-dev --confirmed-non-production --write-report` for normal DEV CD.

For Cloud Web source freshness before the transaction:

```sh
node web/hwlab-cloud-web/scripts/check.mjs
node web/hwlab-cloud-web/scripts/build.mjs
node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report
```

Cloud Web must build `web/hwlab-cloud-web/dist` before publish because the
runtime wrapper serves `dist` before source files. A correct source commit with
stale `dist` can still publish an old UI. The build/check pair is part of the
publish contract, but with separate responsibilities: `check.mjs` validates
source and does not require ignored `dist/` in a clean checkout. `build.mjs`
regenerates `dist` and verifies every runtime file and route alias matches
source. `scripts/dev-artifact-publish.mjs` runs that build/freshness gate and
records the `distFreshness` result before it builds the image.

After a successful multi-service publish, refresh the artifact catalog from the
publish report:

```sh
node scripts/refresh-artifact-catalog.mjs --target-ref origin/main --publish-report reports/dev-gate/dev-artifacts.json
node scripts/deploy-desired-state-plan.mjs --target-ref origin/main --check
```

If publish is blocked, keep the report blocked rather than inventing digests:

```sh
node scripts/refresh-artifact-catalog.mjs --target-ref origin/main --blocked
node scripts/deploy-desired-state-plan.mjs --target-ref origin/main --check
```

The desired-state files are a single deployment truth: `deploy/deploy.json`,
`deploy/artifact-catalog.dev.json`, and `deploy/k8s/base/workloads.yaml` must
carry the same promotion commit and immutable image tag before they are used as
apply input. A `--target-ref` desired-state plan may report a uniform older
state as `promotion_pending`; PR review and CI/CD promotion gates must use
`--target-ref origin/main --check` or `--promotion-commit <sha> --check` so
stale or partially refreshed commit, image, workload, and mirror-env fields are
blocked before any DEV apply.
Reports under `reports/` are evidence snapshots only and cannot override this
desired-state contract.

## Latest-Main Artifact/Runtime Guard

Before any controlled DEV apply for latest `origin/main`, run the read-only
artifact/runtime guard:

```sh
node scripts/artifact-runtime-readiness-guard.mjs --target-ref origin/main --check --no-report
```

When current readiness is expected to be blocked, use:

```sh
node scripts/artifact-runtime-readiness-guard.mjs --target-ref origin/main --check --no-report --expect-blocked
```

The guard compares target main, `reports/dev-gate/dev-artifacts.json`,
`deploy/artifact-catalog.dev.json`, the desired-state planner, Cloud API
`/health/live`, and Cloud Web `:16666/health/live`. It is report-only and must
not be used as an apply command. A local source check, static workbench smoke,
or stale report snapshot cannot claim latest-main deployment unless the guard's
runtime and artifact identity checks also pass.

## API, Edge, Health, And FRP

The public route is fixed:

| Surface | Current entry |
| --- | --- |
| Cloud Web | `http://74.48.78.17:16666/` |
| Cloud Web live health | `http://74.48.78.17:16666/health/live` |
| API/edge health | `http://74.48.78.17:16667/health` |
| API/live health | `http://74.48.78.17:16667/health/live` |

FRP maps master public `16666/16667` to D601 `hwlab-dev` services. Use
`deploy/deploy.json` as the source for endpoint and FRP drift checks:

```sh
node scripts/deploy-contract-plan.mjs --check
```

Do not reintroduce public `6666/6667` as active DEV endpoints. Internal k3s
services may still use internal `6667`.

## Cloud Web Manual DEV Rollout

This section is retained as historical operational baseline for what the
transaction reproduces. New DEV side effects should use
`scripts/dev-cd-apply.mjs`; do not bypass the Lease with ad hoc publish,
`kubectl set image`, or restart commands.

The DEV CD transaction owns the durable runtime support sequence before final
live postflight:

```sh
node scripts/dev-runtime-provisioning.mjs --apply --confirm-dev --confirmed-non-production --write-report
node scripts/dev-runtime-migration.mjs --apply --confirm-dev --confirmed-non-production --write-report
node scripts/dev-runtime-postflight.mjs --live --confirm-dev --confirmed-non-production --target api --write-report
```

These commands are called by `scripts/dev-cd-apply.mjs` under the transaction
Lease. Runners without explicit rollout authorization may run the source-only
`--check` variants and prepare PRs/artifacts, but must leave DEV apply,
rollout, live verification, and lock ownership to the host commander.

The previous verified manual path for `hwlab-cloud-web` was:

```sh
cd /home/ubuntu/workspace/hwlab
git pull --ff-only origin main
node web/hwlab-cloud-web/scripts/build.mjs
node scripts/dev-artifact-publish.mjs --publish --services hwlab-cloud-web
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev set image deployment/hwlab-cloud-web hwlab-cloud-web=127.0.0.1:5000/hwlab/hwlab-cloud-web:<tag>
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout status deployment/hwlab-cloud-web --timeout=180s
```

That historical sequence is no longer the normal side-effect path; direct
`--publish` is rejected unless it is called by the transaction.

The rollout must use D601 native k3s:

```sh
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev get deploy hwlab-cloud-web -o wide
```

The rollout report must tie the live deployment back to artifact identity:
source commit, image tag, image reference, registry digest or `not_published`,
and Kubernetes rollout revision. `scripts/dev-deploy-apply.mjs` reports this
under `devDeployApply.cloudWebRollout` when read-only Deployment access exists;
without that access, it reports the blocker and the exact read/rollout commands
instead of claiming a live rollout.

Public verification must check the browser entry on `16666`, not legacy
ports:

```sh
curl -fsS http://74.48.78.17:16666/health/live
curl -fsS http://74.48.78.17:16666/ | sed -n '1,80p'
curl -fsS http://74.48.78.17:16666/styles.css | rg 'overflow: hidden|100dvh'
curl -fsS http://74.48.78.17:16666/help.md | sed -n '1,40p'
curl -fsS http://74.48.78.17:16667/health/live
```

Cloud Workbench acceptance also expects Chinese default UI, no outer page
scroll, and an internal Markdown help surface. The frontend contract is
[cloud-workbench.md](cloud-workbench.md).

## Apply Boundary

`scripts/dev-deploy-apply.mjs` is the DEV-only apply/report backend. A real
apply requires the transaction environment created by `scripts/dev-cd-apply.mjs`
plus explicit non-production flags:

```sh
node scripts/dev-deploy-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report
```

Direct manual `--apply` calls without `HWLAB_CD_TRANSACTION_ID` return
`cd-transaction-required`. The apply path must stop before mutation when
artifact evidence, registry digests, kube access, DEV health, or required
runtime readiness are missing. It must not read secret values or target PROD.

Suspended template Jobs are immutable in `spec.template`. Only the explicit
DEV allowlist may be replaced by the apply script:

- `hwlab-agent-worker-template`
- `hwlab-cli-template`

## Rollback Boundary

Rollback remains DEV-only and must be explicit. Capture current image state
before changing a workload:

```sh
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev get deploy hwlab-cloud-web -o wide
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout history deployment/hwlab-cloud-web
```

For a Deployment rolled by `set image`, use Kubernetes rollout undo only for
the same DEV Deployment:

```sh
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout undo deployment/hwlab-cloud-web
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout status deployment/hwlab-cloud-web --timeout=180s
```

After rollback, repeat public verification on `16666` and `16667`. Do not use
rollback to change PROD, read secrets, restart unrelated services, or conceal
an artifact/report mismatch.

## Controlled Entrypoint Design

The #116 target is a single controlled deployment surface. Until that lands,
use the repo-owned scripts below instead of ad hoc shell fragments.

| Action | Current controlled entrypoint | Gap to close |
| --- | --- | --- |
| Check deployment contract and FRP/endpoint drift | `node scripts/deploy-contract-plan.mjs --check` | Fold into `hwlab deploy check --env dev`. |
| Plan desired image/workload state | `node scripts/deploy-desired-state-plan.mjs --plan --pretty` and `node scripts/deploy-desired-state-plan.mjs --promotion-commit <sha> --check` | Fold into `hwlab deploy plan --env dev --file deploy/deploy.json` with a promotion commit gate. |
| Observe DEV CD status | `node scripts/dev-cd-apply.mjs`, `node scripts/dev-cd-apply.mjs --status`, or `node scripts/dev-cd-apply.mjs --dry-run` | Keep read-only for host and runners; no Lease mutation, artifact publish, apply, rollout, report write, or Secret read. |
| Preflight artifact publish | `node scripts/dev-artifact-publish.mjs --preflight` | Keep as backend for `hwlab artifact publish --preflight`. |
| Publish/apply/verify DEV | `node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report` | Keep as the single side-effect transaction and run from repo-owned DEV CI/CD. |
| Publish DEV images | `node scripts/dev-artifact-publish.mjs --publish` inside `HWLAB_CD_TRANSACTION_ID` | Internal step only; direct side-effect calls are rejected. |
| Refresh catalog/desired state | `node scripts/refresh-artifact-catalog.mjs --target-ref origin/main --publish-report reports/dev-gate/dev-artifacts.json` | Internal transaction step after publish; read/write desired-state only from repo files. |
| Apply DEV rollout | `node scripts/dev-deploy-apply.mjs --apply --confirm-dev --confirmed-non-production --write-report` inside `HWLAB_CD_TRANSACTION_ID` | Internal transaction step; direct side-effect calls are rejected. |
| Roll back DEV Deployment | `KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout undo deployment/<name>` | Add `hwlab deploy rollback --env dev --service <id> --to-revision <n>`. |
| Verify status | `curl` checks plus `node scripts/d601-k3s-readonly-observability.mjs` | Add `hwlab deploy verify --env dev --write-report`. |

## Automation Direction

The stable direction from
[pikasTech/HWLAB#61](https://github.com/pikasTech/HWLAB/issues/61) and
[pikasTech/HWLAB#116](https://github.com/pikasTech/HWLAB/issues/116) is to
converge manual operations into CLI plus `deploy/deploy.json`, then run that
surface from UniDesk CI/CD:

- render and verify frps/frpc from the deploy manifest;
- publish images and write digest reports;
- refresh catalog/workload desired state from the publish report;
- apply DEV with immutable Job replacement policy;
- verify `16666` Cloud Web and `16667/health/live` API;
- invalidate or regenerate active reports after endpoint or artifact changes.

UniDesk CI/CD is infrastructure only. It may schedule build/publish/deploy
jobs and store artifacts, but it must not replace `hwlab-cloud-api`,
`hwlab-cloud-web`, `hwlab-router`, `hwlab-patch-panel`, simulators, gateways,
or agent services as the runtime being validated.

## Stable Sources

- [docs/dev-artifact-publish.md](../dev-artifact-publish.md): artifact publish
  command and report contract.
- [docs/dev-deploy-apply.md](../dev-deploy-apply.md): apply/report and Cloud
  Web manual rollout details.
- [web/hwlab-cloud-web/scripts/dist-contract.mjs](../../web/hwlab-cloud-web/scripts/dist-contract.mjs): Cloud Web `dist` freshness contract used by build/check/publish.
- [pikasTech/HWLAB#61](https://github.com/pikasTech/HWLAB/issues/61): manual
  rollout review and automation target.
- [pikasTech/HWLAB#116](https://github.com/pikasTech/HWLAB/issues/116):
  deployment regularization stages and acceptance.
- [pikasTech/HWLAB#340](https://github.com/pikasTech/HWLAB/issues/340):
  `deploy.json` DEV CD path long-termization and master CLI automation.