refactor: remove legacy dev cd scripts
This commit is contained in:
Codex
@@ -14,11 +14,11 @@ HWLAB 是硬件实验室运行面和控制面项目。本文是 agent、指挥
|
||||
- HWLAB #7、用户反馈、长期看板和指挥简报的 GitHub issue 正文写入必须走 UniDesk CLI：`cd /root/unidesk && bun scripts/cli.ts gh ...`；禁止直接用原生 `gh issue edit/create/comment` 写这些 issue。事故和工具补强需求见 [pikasTech/unidesk#142](https://github.com/pikasTech/unidesk/issues/142)。
|
||||
- 在 UniDesk CLI 局部替换、写前备份和写后 hash 验证能力完成前，不要对 #7 做无 guard 的整篇 body replace；必须先保留 before body、确认维护纪律 heading 仍存在，再写入。
|
||||
|
||||
## P0 DEV CD Promotion 顺序
|
||||
## P0 Legacy CD 删除纪律
|
||||
|
||||
- `ci-publish` 只证明镜像已经为 publish report 的 `artifactPublish.sourceCommitId` 构建并推送；它不会改变 rollout 目标。禁止在 `ci-publish` 成功后直接把 `dev-cd-apply` 的旧 desired-state 输出当作新版本上线成功。
|
||||
- 正式 DEV CD 前必须用刚成功的 publish report 刷新三份 repo desired-state：`node scripts/refresh-artifact-catalog.mjs --target-ref <published-sourceCommitId> --publish-report <ci-publish-report.json>`，再执行 `node scripts/deploy-desired-state-plan.mjs --promotion-commit <published-sourceCommitId> --check`，确认 `deploy/deploy.json`、`deploy/artifact-catalog.dev.json` 和 `deploy/k8s/base/workloads.yaml` 全部指向同一个已发布 commit。
|
||||
- 只有刷新后的 desired-state 已提交/推送，并且 CD 报告与 `16666/16667` live health 都观测到该短 commit，才能声明 DEV 上线完成；如果 CD report 里 `commitId` 仍是旧值，必须先修 desired-state，不要重复跑 CD。长期细节见 [docs/reference/deployment-publish.md](docs/reference/deployment-publish.md)。
|
||||
- 旧 D601 `ci-publish`、`dev-cd-apply`、`dev-deploy-apply` 和 `dev-artifact-publish` 脚本入口已经从当前发布面删除；不要恢复这些文件、CLI 子命令或文档入口。
|
||||
- 当前发布只走 G14 k3s Tekton + `G14-gitops` + Argo CD；镜像构建 helper 是 `scripts/g14-artifact-publish.mjs`，desired state 由 `scripts/g14-gitops-render.mjs` 生成，细则见 [docs/reference/g14-gitops-cicd.md](docs/reference/g14-gitops-cicd.md)。
|
||||
- 发现旧脚本、旧文档或旧 `hwlab-cli cicd` 再次进入 G14 Pipeline、AGENTS 或长期参考时，优先删除旧入口并把调用方改到 G14 GitOps 路径，不再做兼容保留。
|
||||
|
||||
## 工作区
|
||||
|
||||
@@ -46,7 +46,7 @@ HWLAB 是硬件实验室运行面和控制面项目。本文是 agent、指挥
|
||||
- 文档治理与 docs-spec 本地权威：[docs/reference/documentation-governance.md](docs/reference/documentation-governance.md)
|
||||
- 架构和 M3 主线：[docs/reference/architecture.md](docs/reference/architecture.md)
|
||||
- DEV 运行态、端口、k3s 和 DB DNS 边界：[docs/reference/dev-runtime-boundary.md](docs/reference/dev-runtime-boundary.md)
|
||||
- 部署正规化、`deploy.json` DEV CD 路径、SecretRef preflight、runner/host 边界和镜像发布：[docs/reference/deployment-publish.md](docs/reference/deployment-publish.md)
|
||||
- G14 GitOps 发布、SecretRef preflight、runner/host 边界和镜像发布：[docs/reference/g14-gitops-cicd.md](docs/reference/g14-gitops-cicd.md)
|
||||
- G14 GitOps CI/CD、Tekton/Argo CD、集群内 registry 和无锁镜像化发布：[docs/reference/g14-gitops-cicd.md](docs/reference/g14-gitops-cicd.md)
|
||||
- Code Agent 对话就绪与真实回复判定：[docs/reference/code-agent-chat-readiness.md](docs/reference/code-agent-chat-readiness.md)
|
||||
- DEV runtime hotfix runbook 与只读审计：[docs/reference/dev-runtime-hotfix-runbook.md](docs/reference/dev-runtime-hotfix-runbook.md)
|
||||
@@ -70,10 +70,10 @@ HWLAB 是硬件实验室运行面和控制面项目。本文是 agent、指挥
|
||||
- Cloud Web 构建：`npm run web:build`
|
||||
- Cloud Web M3 只读护栏：`npm run web:m3-readonly`
|
||||
- Cloud Workbench 布局/遮挡 smoke：`npm run web:layout`；local-build 用 `npm run web:layout:build`；DEV deploy 后用 `npm run web:layout:live`。
|
||||
- G14 artifact build helper：`node scripts/g14-artifact-publish.mjs --publish ...`，只能由 G14 Tekton task 携带 CI artifact identity 调用；人工发布走 G14 poller/GitOps，不走 legacy `ci-publish`。
|
||||
- G14 artifact build helper：`node scripts/g14-artifact-publish.mjs --publish ...`，只能由 G14 Tekton task 携带 CI artifact identity 调用；人工发布走 G14 poller/GitOps，不走 legacy CLI CD。
|
||||
- G14 GitOps 渲染：`npm run g14:gitops:render`；检查已生成 Tekton/Argo CD manifests：`npm run g14:gitops:check`
|
||||
- DEV 依赖 runtime base 构建：`npm run dev-runtime-base:build`
|
||||
- Legacy D601 DEV CD：`dev-cd-apply`/`ci-publish` 仅供迁移事故回放，不作为 G14 DEV/PROD 发布入口。
|
||||
- Legacy D601 DEV CD：旧脚本入口已删除；事故回放只读历史 issue/commit，不恢复旧命令。
|
||||
- CI/CD job 查询：`node tools/hwlab-cli/bin/hwlab-cli.mjs cicd status|logs|report <jobId>`
|
||||
- runner GitHub 可见性预检：`npm run runner:issue-visibility:preflight`
|
||||
- D601 k3s 只读观测：legacy 回溯入口，仅在确认需要 D601 事故复盘时使用；当前 G14 运行面观察使用 UniDesk route `G14:k3s`。
|
||||
|
||||
@@ -1,264 +0,0 @@
|
||||
# HWLAB DEV Artifact Publish
|
||||
|
||||
`scripts/dev-artifact-publish.mjs` is the DEV-only artifact preflight/build
|
||||
backend for D601 local/internal registry artifacts. Publish side effects are
|
||||
normally entered by a CI artifact job identity, or by the legacy
|
||||
`scripts/dev-cd-apply.mjs` transaction path during transition. The artifact
|
||||
script does not deploy workloads, read secrets, enable PROD, or push to
|
||||
GHCR/Docker Hub/other third-party registries.
|
||||
|
||||
长期口径：CI 产物报告不是人工维护的镜像库状态，也不是 CD 放行的本地状态源。
|
||||
它是 CI 自动生成的一次性审计记录。CD 必须从 `deploy/deploy.json` 读取发布控制
|
||||
意图，从 `deploy/artifact-catalog.dev.json` 读取 digest 期望，并直接向 registry
|
||||
manifest 验证目标 tag/digest 是否存在。`/tmp/hwlab-dev-gate/dev-artifacts.json`
|
||||
只能用于排障、刷新 catalog 和审计追溯。
|
||||
|
||||
## Scope
|
||||
|
||||
- Environment: `dev`.
|
||||
- Namespace: `hwlab-dev`.
|
||||
- Default registry prefix: `127.0.0.1:5000/hwlab`.
|
||||
- Base image: resolved by `scripts/preflight-dev-base-image.mjs` from
|
||||
`HWLAB_DEV_BASE_IMAGE`, a local `node:20-*` Docker image, or a local HWLAB
|
||||
base tag such as
|
||||
`127.0.0.1:5000/hwlab/hwlab-dev-base:node20-bookworm-slim`. There is no
|
||||
base-image pull or third-party fallback during publish; the build runs with
|
||||
`--pull=false`.
|
||||
- Node runtime dependencies are installed from the source `package.json` and
|
||||
`package-lock.json` inside the DEV image build with `npm ci --omit=dev
|
||||
--ignore-scripts` when the lockfile is present, unless the selected base
|
||||
image already provides `/opt/hwlab-node-runtime-base/node_modules`. This
|
||||
includes the Postgres `pg` driver required by the durable cloud-api runtime
|
||||
adapter.
|
||||
- Image tag: first seven characters of the Git commit used as build source.
|
||||
- Required image labels:
|
||||
- `hwlab.pikastech.local/repo`
|
||||
- `hwlab.pikastech.local/commit`
|
||||
- `hwlab.pikastech.local/service-id`
|
||||
- `hwlab.pikastech.local/environment=dev`
|
||||
|
||||
## Commands
|
||||
|
||||
Static check:
|
||||
|
||||
```sh
|
||||
node --check scripts/preflight-dev-base-image.mjs
|
||||
node --check scripts/src/dev-base-image-preflight.mjs
|
||||
node --check scripts/src/dev-artifact-services.mjs
|
||||
node --check scripts/src/registry-capabilities.mjs
|
||||
node --check scripts/dev-artifact-publish.mjs
|
||||
node --check scripts/dev-runtime-base-image.mjs
|
||||
node --check scripts/refresh-artifact-catalog.mjs
|
||||
node --test scripts/refresh-artifact-catalog.test.mjs
|
||||
```
|
||||
|
||||
Base image preflight:
|
||||
|
||||
```sh
|
||||
node scripts/preflight-dev-base-image.mjs
|
||||
```
|
||||
|
||||
Preflight and report generation:
|
||||
|
||||
```sh
|
||||
node scripts/dev-artifact-publish.mjs --preflight
|
||||
```
|
||||
|
||||
Build only:
|
||||
|
||||
```sh
|
||||
node scripts/dev-artifact-publish.mjs --build
|
||||
```
|
||||
|
||||
Build only one or more selected services for a CI artifact task:
|
||||
|
||||
```sh
|
||||
node scripts/dev-artifact-publish.mjs --build --services hwlab-cloud-web
|
||||
```
|
||||
|
||||
`--services` narrows the build/publish report to selected service IDs. It is a
|
||||
CI/artifact acceleration surface; the normal DEV rollout still consumes
|
||||
repo-owned desired state through `scripts/dev-cd-apply.mjs`.
|
||||
|
||||
Build and publish to the D601 local/internal registry inside a CI artifact job:
|
||||
|
||||
```sh
|
||||
HWLAB_CI_ARTIFACT_RUN_ID=<run-id> node scripts/dev-artifact-publish.mjs --publish
|
||||
```
|
||||
|
||||
The legacy transaction entry remains available during migration:
|
||||
|
||||
```sh
|
||||
node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
Direct `--publish` without `HWLAB_CI_ARTIFACT_RUN_ID` or
|
||||
`HWLAB_CD_TRANSACTION_ID` is rejected with `cd-transaction-required`; this is a
|
||||
bypass guard, not a second lock.
|
||||
|
||||
Daily Cloud Web source validation does not require ignored `dist/` to exist:
|
||||
|
||||
```sh
|
||||
node web/hwlab-cloud-web/scripts/check.mjs
|
||||
```
|
||||
|
||||
For `hwlab-cloud-web`, the build path refreshes and verifies the static bundle
|
||||
before image build; publish still goes through the transaction command above:
|
||||
|
||||
```sh
|
||||
node web/hwlab-cloud-web/scripts/build.mjs
|
||||
node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
The runtime wrapper serves `/app/web/hwlab-cloud-web/dist` before the source
|
||||
directory. If `dist` is stale, a newly tagged image can still serve old HTML,
|
||||
CSS, and JavaScript. A publish for Cloud Web is not valid until the image has
|
||||
fresh `dist` evidence. `build.mjs` regenerates `dist` and verifies every
|
||||
runtime file and route alias matches source. `dev-artifact-publish.mjs`
|
||||
re-runs that build/freshness gate and records
|
||||
`artifactPublish.services[].distFreshness` for `hwlab-cloud-web` before the
|
||||
Docker build. `check.mjs` remains a clean-checkout source contract and must not
|
||||
fail only because ignored `dist/` is absent.
|
||||
|
||||
Use `--registry-prefix` only for another localhost/private/internal D601
|
||||
registry prefix. The script rejects third-party registry hosts and any prefix
|
||||
that names PROD. Use `--base-image` only to override `HWLAB_DEV_BASE_IMAGE` with
|
||||
an approved local image for that run.
|
||||
|
||||
Docker build output is visible by default so a slow layer is observable from the
|
||||
runner log. Use `--quiet-build` only when a caller already captures the full
|
||||
Docker log elsewhere. The artifact report records per-service
|
||||
`dockerBuildDurationMs`, `cloudWebBuildDurationMs`, and `publishDurationMs`.
|
||||
|
||||
An optional dependency base image can be prepared with:
|
||||
|
||||
```sh
|
||||
node scripts/dev-runtime-base-image.mjs --dry-run
|
||||
node scripts/dev-runtime-base-image.mjs
|
||||
```
|
||||
|
||||
The default tag is
|
||||
`127.0.0.1:5000/hwlab/hwlab-node-runtime-base:deps-<package-lock-hash>`.
|
||||
Selecting that image through `HWLAB_DEV_BASE_IMAGE` lets service builds reuse a
|
||||
preinstalled `node_modules` layer instead of repeating `npm ci` for every
|
||||
service.
|
||||
|
||||
## Report
|
||||
|
||||
The script writes `/tmp/hwlab-dev-gate/dev-artifacts.json`. The file is kept in
|
||||
the existing DEV gate report envelope for compatibility with
|
||||
`scripts/validate-dev-gate-report.mjs`, while the `artifactPublish` object is
|
||||
the HWLAB#35-specific publish record. This report is evidence, not desired
|
||||
state. A later CD rollout must verify catalog images directly against the
|
||||
registry instead of trusting this file.
|
||||
|
||||
Each service record contains:
|
||||
|
||||
- `serviceId`
|
||||
- `image`
|
||||
- `imageTag`
|
||||
- `digest`
|
||||
- `distFreshness` for `hwlab-cloud-web` build/publish runs
|
||||
- `publishEnabled`
|
||||
- `artifactRequired`
|
||||
- `artifactScope`
|
||||
- `notPublishedReason`
|
||||
- `runtimeKind`
|
||||
- `implementationState`
|
||||
- `sourceState`
|
||||
- `entrypoint`
|
||||
|
||||
The report also carries `artifactPublish.serviceInventory` and
|
||||
`artifactPublish.publishPlan`. `serviceInventory` is the resolved frozen service
|
||||
list with enabled/disabled state. `publishPlan` is the machine-readable v2 plan:
|
||||
source commit, image tag, per-service registry target, image reference, digest
|
||||
placeholder or real digest, registry capability evidence, and the reason a
|
||||
service was not published.
|
||||
|
||||
The report also includes `artifactPublish.registryCapabilities`, split into
|
||||
three dimensions:
|
||||
|
||||
- `process-http-access`: runner-process HTTP access to the registry `/v2/`
|
||||
endpoint. This is diagnostic only. A failed
|
||||
`http://127.0.0.1:5000/v2/` fetch from the runner process is `degraded`, not
|
||||
proof that Docker publish failed.
|
||||
- `docker-daemon-push-access`: Docker daemon view of the local/internal
|
||||
registry target. This is the publish-path capability and is the only registry
|
||||
capability that blocks `--publish` before a push attempt.
|
||||
- `k3s-pull-access`: read-only k3s view of `hwlab-dev` image pull state. This
|
||||
is the deploy-path capability and does not prove or disprove Docker daemon
|
||||
push access.
|
||||
|
||||
`digest` is only set to a registry digest after `docker push` succeeds and the
|
||||
push output contains a `sha256:<64 hex>` digest. If the push succeeds but no
|
||||
digest is observable, the service remains `published_unverified_digest`,
|
||||
`digest` stays `not_published`, and the report carries a blocker. The script
|
||||
records blockers instead of claiming a publish when build, push, digest
|
||||
observation, base image, registry, contract, or safety checks fail.
|
||||
|
||||
After a fully successful publish, update the catalog from the report:
|
||||
|
||||
```sh
|
||||
node scripts/refresh-artifact-catalog.mjs --target-ref origin/main --publish-report /tmp/hwlab-dev-gate/dev-artifacts.json
|
||||
```
|
||||
|
||||
If publish remains blocked, refresh only commit/tag identity and keep digests
|
||||
blocked:
|
||||
|
||||
```sh
|
||||
node scripts/refresh-artifact-catalog.mjs --target-ref origin/main --blocked
|
||||
```
|
||||
|
||||
Before any later DEV apply review, run the read-only artifact/runtime guard so
|
||||
artifact evidence is not mistaken for deployed runtime evidence:
|
||||
|
||||
```sh
|
||||
node scripts/artifact-runtime-readiness-guard.mjs --target-ref origin/main --check --no-report --expect-blocked
|
||||
```
|
||||
|
||||
The guard must remain blocked until the target commit, publish report, artifact
|
||||
catalog, desired-state files, Cloud API `/health/live`, and Cloud Web
|
||||
`:16666/health/live` all report the same identity. It does not publish, deploy,
|
||||
restart services, mutate Kubernetes, touch PROD, or read secrets.
|
||||
|
||||
The report also includes `artifactPublish.baseImagePreflight` with the image
|
||||
source, local tag, local image ID, `publishUsable` gate, blockers, and next
|
||||
steps. It also carries `recommendation`, `provision`, and `blockedReport`
|
||||
objects so the blocked report includes the exact blocker, expected image tag,
|
||||
recommended `HWLAB_DEV_BASE_IMAGE` value, and preload commands. When that
|
||||
preflight is blocked, artifact publish is blocked and no build or push is
|
||||
attempted.
|
||||
|
||||
If the D601 builder has no approved Node 20 base image, or the selected DEV
|
||||
registry is not reachable from the Docker daemon publish path, the report must
|
||||
remain `status: "blocked"`. Do not turn a base-image or registry blocker into a
|
||||
fake digest or published state. Do not treat runner-process loopback HTTP
|
||||
failure as a publish failure by itself.
|
||||
|
||||
Committed DEV artifact reports are evidence snapshots and may lag the deploy
|
||||
desired-state files. Use `scripts/deploy-desired-state-plan.mjs` to see whether
|
||||
the report source matches the desired-state commit, then refresh the catalog
|
||||
from a publish report only when that report proves the same target source
|
||||
commit. If a later read-only runner cannot fetch catalog manifests from
|
||||
`127.0.0.1:5000`, treat that as a `#66` registry reachability dimension, not as
|
||||
a missing base-image or missing-publish blocker.
|
||||
|
||||
## Known Implementation States
|
||||
|
||||
- `repo-entrypoint`: the repository has `cmd/<serviceId>/main.mjs`.
|
||||
- `static-web-wrapper`: the cloud web static files are packaged behind a small
|
||||
health/static-file wrapper.
|
||||
- `repo-bundle`: the image packages repository-owned skill artifacts.
|
||||
- `library-only`: the repository has library code but no executable package bin.
|
||||
- `missing-runtime-entrypoint`: the image is only a health placeholder and is
|
||||
not a real service implementation.
|
||||
|
||||
## Known Source States
|
||||
|
||||
- `source-present`: the service remains in the MVP artifact catalog and has a
|
||||
repository-owned source path or entrypoint conclusion.
|
||||
- `intentionally-disabled`: the service is explicitly out of the MVP artifact
|
||||
publish set until it is given source. It must not remain a vague blocker.
|
||||
|
||||
Runtime implementation blockers do not fake a service implementation. They are
|
||||
recorded as the next minimum work needed after artifact publication.
|
||||
@@ -1,347 +0,0 @@
|
||||
# HWLAB DEV Deploy Apply
|
||||
|
||||
This document records the DEV-only apply/preflight backend for
|
||||
`pikasTech/HWLAB#33` and the apply-plan/report path for
|
||||
`pikasTech/HWLAB#50`. Real apply side effects are normally entered through
|
||||
`scripts/dev-cd-apply.mjs`, which sets the transaction environment and
|
||||
serializes publish/apply/verify with the DEV CD Lease lock. The apply backend
|
||||
targets only the D601 `hwlab-dev` namespace and must not be used for PROD.
|
||||
|
||||
## Scope
|
||||
|
||||
- Work from latest `origin/main`.
|
||||
- Read only HWLAB repository deploy inputs: `deploy/deploy.json`,
|
||||
`deploy/artifact-catalog.dev.json`, and `deploy/k8s/dev`.
|
||||
- Use `scripts/deploy-desired-state-plan.mjs` as source/dry-run support to
|
||||
review commit, image, tag, workload image, and env mirror convergence before
|
||||
any apply review.
|
||||
- Validate artifact identity, image commit tags, Kubernetes workload/service
|
||||
shape, DEV health, and cloud-api DB connectivity.
|
||||
- Validate that `hwlab-cloud-api` desired state declares the Code Agent
|
||||
provider env contract: `OPENAI_API_KEY` from
|
||||
`hwlab-code-agent-provider/openai-api-key` and
|
||||
`HWLAB_CODE_AGENT_OPENAI_BASE_URL` through the DEV egress/proxy path.
|
||||
- Emit an operator-readable plan in `devDeployApply`: plan/apply boundary,
|
||||
expected artifact commit, target namespace, workloads, services, remaining
|
||||
blockers, minimal manual commands, and rollback hints.
|
||||
- Plan and, in apply mode, execute a scoped replacement for DEV suspended
|
||||
template Jobs whose image changed but whose Kubernetes `spec.template` is
|
||||
immutable.
|
||||
- Plan and, in apply mode, execute a scoped cleanup for stale DEV simulator
|
||||
Deployments `hwlab-gateway-simu` and `hwlab-box-simu` only when the source
|
||||
manifest contains their indexed StatefulSet replacements. This keeps the M3
|
||||
DEV runtime at exactly two gateway-simu sessions and two box-simu resources.
|
||||
- Stop before mutation when artifact publish evidence, registry digests,
|
||||
`kubectl`, DEV health, cloud-api DB connectivity, or Code Agent provider
|
||||
env preservation are missing.
|
||||
|
||||
## Commands
|
||||
|
||||
Run a preflight that is expected to report blockers on the current skeleton:
|
||||
|
||||
```sh
|
||||
node --check scripts/dev-deploy-apply.mjs
|
||||
node --check scripts/src/dev-deploy-apply.mjs
|
||||
node --check scripts/dev-runtime-migration.mjs
|
||||
node --check scripts/src/dev-runtime-migration.mjs
|
||||
node scripts/deploy-desired-state-plan.mjs --target-ref origin/main --check
|
||||
node scripts/dev-runtime-migration.mjs --check
|
||||
node scripts/artifact-runtime-readiness-guard.mjs --target-ref origin/main --check --no-report --expect-blocked
|
||||
node scripts/dev-deploy-apply.mjs --dry-run --expect-blocked --report /tmp/hwlab-dev-gate/report.json
|
||||
node scripts/validate-dev-gate-report.mjs
|
||||
```
|
||||
|
||||
Run the existing CLI dry-run surface:
|
||||
|
||||
```sh
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs test e2e --env dev --mvp --dry-run
|
||||
```
|
||||
|
||||
Only after CI publish evidence and executor prerequisites exist, a real DEV
|
||||
CD transaction is:
|
||||
|
||||
```sh
|
||||
node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
The underlying apply backend still requires explicit non-production
|
||||
confirmation and a transaction environment:
|
||||
|
||||
```sh
|
||||
node scripts/dev-deploy-apply.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
Direct `--apply` without `HWLAB_CD_TRANSACTION_ID` is rejected with
|
||||
`cd-transaction-required`; this is a bypass guard, not a second lock. The
|
||||
script refuses PROD flags, secret-read flags, heavyweight e2e flags, and
|
||||
force-push flags. It also refuses to apply if the DEV artifact catalog still
|
||||
contains skeleton-only unpublished images.
|
||||
|
||||
`deploy-desired-state-plan.mjs` is not an apply preflight substitute. Pair it
|
||||
with `artifact-runtime-readiness-guard.mjs` before apply review so stale
|
||||
artifact catalogs, stale desired-state, stale API runtime identity, or a stale
|
||||
Cloud Web `:16666` build stay explicit blockers.
|
||||
|
||||
Kubeconfig selection is explicit and safe. The script uses this precedence:
|
||||
|
||||
1. `--kubeconfig PATH`
|
||||
2. `HWLAB_DEV_KUBECONFIG`
|
||||
3. `KUBECONFIG`
|
||||
4. `/etc/rancher/k3s/k3s.yaml`
|
||||
|
||||
The selected path is used for read-only preflight, dry-run planning, and the
|
||||
DEV-only apply path. The report records the kubeconfig source and path, but it
|
||||
does not print kubeconfig contents or Secret values.
|
||||
|
||||
Emergency D601 guard: for normal D601 execution, prefer
|
||||
`/etc/rancher/k3s/k3s.yaml`. Docker Desktop Kubernetes has been disabled and
|
||||
cleaned, but a residual default kubeconfig may still point at
|
||||
`docker-desktop`/`127.0.0.1:11700`; do not pass that path through `KUBECONFIG`
|
||||
or `--kubeconfig`. Any selected kubeconfig whose context/node resolves to
|
||||
`docker-desktop` or `desktop-control-plane` is a blocker, not DEV-LIVE
|
||||
evidence. See [pikasTech/unidesk#138](https://github.com/pikasTech/unidesk/issues/138).
|
||||
|
||||
Runtime DB migration is a separate DEV-only apply boundary. The source-only
|
||||
check is safe for normal PR validation:
|
||||
|
||||
```sh
|
||||
node scripts/dev-runtime-migration.mjs --check
|
||||
node scripts/dev-runtime-migration.mjs --dry-run --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
Read-only live DB verification requires explicit DEV confirmation and an
|
||||
already injected `HWLAB_CLOUD_DB_URL`; the report redacts the value and keeps
|
||||
auth, schema, migration ledger, and readiness blockers separate:
|
||||
|
||||
```sh
|
||||
node scripts/dev-runtime-migration.mjs --dry-run --allow-live-db-read --confirm-dev --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
A real migration write is not part of default deploy validation. It requires a
|
||||
separately authorized DEV operation:
|
||||
|
||||
```sh
|
||||
node scripts/dev-runtime-migration.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
That command may only apply
|
||||
`internal/db/migrations/0001_cloud_core_skeleton.sql` to the DEV Postgres target
|
||||
provided by `HWLAB_CLOUD_DB_URL`. It must not read Kubernetes Secret resources,
|
||||
print the DB URL/password/token, mutate PROD, restart services, or claim
|
||||
M3/M4/M5 acceptance.
|
||||
|
||||
`deploy-desired-state-plan.mjs` is not an apply preflight substitute. It is a
|
||||
read-only source planner and does not prove registry image existence, DEV
|
||||
cluster pullability, DEV apply, service restart, or M3 DEV-LIVE evidence.
|
||||
|
||||
## Report Contract
|
||||
|
||||
The machine-readable result is written to
|
||||
`/tmp/hwlab-dev-gate/dev-deploy-report.json` when `--report /tmp/hwlab-dev-gate/report.json` is used. The
|
||||
report remains compatible with the #33 DEV gate validator and adds these
|
||||
operator fields:
|
||||
|
||||
- `devDeployApply.conclusion`: `blocked` while blockers are open, otherwise
|
||||
`ready` for human approval.
|
||||
- `devDeployApply.applyBoundary`: default no-write behavior, required apply
|
||||
flags, DEV-only write scope, no-write scope, and forbidden actions.
|
||||
- `devDeployApply.artifactPlan`: expected artifact commit, source commit,
|
||||
deploy/catalog commit IDs, publish/registry status, and unpublished services.
|
||||
- `devDeployApply.target`: DEV endpoint and `hwlab-dev` namespace.
|
||||
- `devDeployApply.workloadPlan` and `servicePlan`: planned Kubernetes
|
||||
workloads, containers, images, service IDs, ports, and replica intent.
|
||||
- `devDeployApply.templateJobReplacementPolicy`: the DEV-only allowlist for
|
||||
suspended template Job replacement. The only allowlisted Jobs are
|
||||
`hwlab-agent-worker-template` and `hwlab-cli-template` in `hwlab-dev`.
|
||||
- `devDeployApply.templateJobReplacements`: per-Job replacement decisions with
|
||||
namespace, Job name, old image, new image, and result (`planned`, `replaced`,
|
||||
`not_needed`, `not_found`, or blocked/failure states).
|
||||
- `devDeployApply.legacySimulatorDeploymentCleanupPolicy`: the DEV-only
|
||||
allowlist for stale simulator Deployment cleanup after the M3 simulator
|
||||
runtime moved to indexed StatefulSets.
|
||||
- `devDeployApply.legacySimulatorDeploymentCleanups`: per-Deployment cleanup
|
||||
decisions for `hwlab-gateway-simu` and `hwlab-box-simu`, including old/new
|
||||
images and result (`planned`, `deleted`, `not_found`, or blocked/failure
|
||||
states).
|
||||
- `devDeployApply.cloudWebRollout`: Cloud Web source commit, image tag, image
|
||||
reference, digest, live image, and Kubernetes rollout revision when read-only
|
||||
Deployment access is available. If access is missing, this field records the
|
||||
read/rollout commands and the blocker instead of claiming a live rollout.
|
||||
- `devDeployApply.codeAgentProvider`: no-secret provider validation gate. It
|
||||
records desired-state env/ref readiness plus live `hwlab-cloud-api`
|
||||
Deployment env preservation before apply and, in apply mode, after apply.
|
||||
This gate reads Deployment env metadata only; it must not read Kubernetes
|
||||
Secret data or attempt a provider call.
|
||||
- `devDeployApply.manualCommands`: dry-run/report commands before human
|
||||
approval; the apply command is present only when the plan is ready.
|
||||
- `devDeployApply.rollbackHint`: pre-apply capture commands, deployment
|
||||
rollbacks, suspended job cleanup, and post-rollback health checks.
|
||||
- `devDeployApply.remainingBlockers`: open blockers with concrete unblock
|
||||
hints. These do not claim live apply happened.
|
||||
|
||||
When `conclusion.status` is `blocked`, operators should clear the listed
|
||||
`remainingBlockers` and rerun:
|
||||
|
||||
```sh
|
||||
node scripts/dev-deploy-apply.mjs --dry-run --expect-blocked --report /tmp/hwlab-dev-gate/report.json
|
||||
node scripts/validate-dev-gate-report.mjs
|
||||
```
|
||||
|
||||
When `conclusion.status` is `ready`, run the dry-run plan and validator
|
||||
immediately before human approval:
|
||||
|
||||
```sh
|
||||
node scripts/dev-deploy-apply.mjs --dry-run --report /tmp/hwlab-dev-gate/report.json
|
||||
node scripts/validate-dev-gate-report.mjs
|
||||
```
|
||||
|
||||
Only after that approval may the DEV CD transaction command be used.
|
||||
|
||||
## Cloud Web Manual DEV Rollout
|
||||
|
||||
This is the currently verified manual CD path for `hwlab-cloud-web` on D601.
|
||||
It is the operational baseline that future CLI automation should reproduce from
|
||||
`deploy/deploy.json`; do not rediscover ports or kube contexts during an
|
||||
incident.
|
||||
|
||||
Start from the D601 deployment copy, not a runner worktree:
|
||||
|
||||
```sh
|
||||
cd /home/ubuntu/workspace/hwlab
|
||||
git pull --ff-only origin main
|
||||
```
|
||||
|
||||
Validate source in a clean checkout, then refresh the static bundle before
|
||||
artifact publish. Cloud Web images serve `web/hwlab-cloud-web/dist` first, so
|
||||
stale ignored build output can keep an old English or old layout bundle alive
|
||||
even after the source commit is correct. `check.mjs` does not require ignored
|
||||
`dist/`; `build.mjs` and `dev-artifact-publish.mjs` are the build-first
|
||||
freshness gates.
|
||||
|
||||
```sh
|
||||
node web/hwlab-cloud-web/scripts/check.mjs
|
||||
node web/hwlab-cloud-web/scripts/build.mjs
|
||||
node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
Probe the image locally before touching k3s:
|
||||
|
||||
```sh
|
||||
docker run --rm -d --name hwlab-cloud-web-probe -p 127.0.0.1:18088:8080 127.0.0.1:5000/hwlab/hwlab-cloud-web:<tag>
|
||||
sleep 1
|
||||
curl -fsS http://127.0.0.1:18088/health/live
|
||||
curl -fsS http://127.0.0.1:18088/ | sed -n '1,40p'
|
||||
docker rm -f hwlab-cloud-web-probe
|
||||
```
|
||||
|
||||
Use the D601 native k3s kubeconfig explicitly:
|
||||
|
||||
```sh
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev set image deployment/hwlab-cloud-web hwlab-cloud-web=127.0.0.1:5000/hwlab/hwlab-cloud-web:<tag>
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout status deployment/hwlab-cloud-web --timeout=180s
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev get deploy hwlab-cloud-web -o wide
|
||||
```
|
||||
|
||||
Do not use the default kube context as the source of truth. On D601 it may be
|
||||
`docker-desktop`, which is not the public DEV runtime behind `16666/16667`.
|
||||
|
||||
Public acceptance must verify all of these:
|
||||
|
||||
```sh
|
||||
curl -fsS http://74.48.78.17:16666/health/live
|
||||
curl -fsS http://74.48.78.17:16666/ | sed -n '1,80p'
|
||||
curl -fsS http://74.48.78.17:16666/styles.css | rg 'overflow: hidden|100dvh'
|
||||
curl -fsS http://74.48.78.17:16666/help.md | sed -n '1,40p'
|
||||
```
|
||||
|
||||
For the Cloud Workbench Chinese/no-outer-scroll rollout, the accepted target
|
||||
state was:
|
||||
|
||||
- `GET /health/live` reports revision
|
||||
`1e8805664970839b72be40c34636b08f6d18b131`;
|
||||
- `GET /` returns `html lang="zh-CN"` and title `HWLAB 云工作台`;
|
||||
- CSS locks the outer shell with `html, body { overflow: hidden; }` and
|
||||
`.workbench-shell { height: 100dvh; overflow: hidden; }`;
|
||||
- `GET /help.md` returns the Chinese Cloud Workbench help document.
|
||||
|
||||
The public path is:
|
||||
|
||||
1. master `hwlab-frps-dev` listens on `16666/16667`;
|
||||
2. D601 `hwlab-frpc` runs in `hwlab-dev` from the `hwlab-frpc-config`
|
||||
ConfigMap;
|
||||
3. `hwlab-dev-cloud-web` maps to
|
||||
`hwlab-cloud-web.hwlab-dev.svc.cluster.local:8080`;
|
||||
4. `hwlab-dev-edge-proxy` maps to
|
||||
`hwlab-edge-proxy.hwlab-dev.svc.cluster.local:6667`.
|
||||
|
||||
Master `hwlab-browser-proxy-dev` is only an auxiliary local proxy on `18087`;
|
||||
it is not the public browser endpoint.
|
||||
|
||||
## Suspended Template Job Replacement
|
||||
|
||||
Kubernetes Jobs cannot update `spec.template` in place. For DEV template Jobs
|
||||
that are intentionally suspended and used as templates, an image tag change
|
||||
would otherwise make a normal `kubectl apply -k deploy/k8s/dev` non-idempotent.
|
||||
|
||||
The apply script handles only the explicit DEV allowlist:
|
||||
|
||||
- `hwlab-agent-worker-template`
|
||||
- `hwlab-cli-template`
|
||||
|
||||
For each allowlisted Job, the script reads the live Job in `hwlab-dev` and
|
||||
compares its container image with the desired manifest image. If the live Job
|
||||
exists, is suspended, and the image differs, apply mode deletes that Job before
|
||||
running the normal kustomize apply so Kubernetes recreates it from the desired
|
||||
manifest. Dry-run mode records the same decision as `planned` and runs
|
||||
server-side kustomize dry-run without mutating the cluster. If that dry-run
|
||||
reports only the expected immutable `spec.template` errors for the planned
|
||||
allowlisted Jobs, the report records those errors as expected replacement
|
||||
evidence instead of treating them as blockers.
|
||||
|
||||
The policy refuses any other Job name, any non-`hwlab-dev` namespace, and any
|
||||
live Job that is not suspended. It does not change Deployment, Service,
|
||||
ConfigMap, Secret, or PROD behavior.
|
||||
|
||||
## Current Result
|
||||
|
||||
The current DEV apply contract is published for artifact commit `cb35ada`.
|
||||
`deploy/artifact-catalog.dev.json`, `deploy/deploy.json`, and the DEV
|
||||
workloads point at `127.0.0.1:5000/hwlab/*:cb35ada` images with sha256 digest
|
||||
evidence.
|
||||
|
||||
On Code Queue runners, the approved mounted kubeconfig path is typically
|
||||
`/var/lib/unidesk/code-queue/kubeconfig`. Export `HWLAB_DEV_KUBECONFIG` or
|
||||
pass `--kubeconfig` explicitly when the default D601 path is not available:
|
||||
|
||||
```sh
|
||||
HWLAB_DEV_KUBECONFIG=/var/lib/unidesk/code-queue/kubeconfig node scripts/dev-deploy-apply.mjs --dry-run --expect-blocked --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
The D601 runner can read and apply the `hwlab-dev` namespace. A real DEV apply
|
||||
may replace only the allowlisted suspended template Jobs above; the normal
|
||||
Deployment, Service, ConfigMap, and health-contract apply path remains
|
||||
unchanged.
|
||||
|
||||
`hwlab-cloud-api` has a DEV DB env contract placeholder:
|
||||
`HWLAB_CLOUD_DB_URL` must come from Secret reference
|
||||
`hwlab-cloud-api-dev-db/database-url`, and
|
||||
`HWLAB_CLOUD_DB_SSL_MODE=disable`. Runtime readiness dials the redacted host
|
||||
parsed from that Secret URL. `cloud-api-db` remains an optional desired alias,
|
||||
not a readiness gate or TCP probe target, until this repo owns its
|
||||
Service/Endpoint rollout contract. The local health gate reports DB runtime
|
||||
config without printing the connection string.
|
||||
|
||||
Durable runtime migration now has a repo-owned path:
|
||||
`scripts/dev-runtime-migration.mjs --check` proves the source migration and
|
||||
ledger contract without DB access; `--dry-run --allow-live-db-read
|
||||
--confirm-dev` verifies live auth/schema/migration/readiness without writes;
|
||||
`--apply --confirm-dev --confirmed-non-production` is the only DEV DB write
|
||||
mode and remains outside default PR validation.
|
||||
|
||||
No PROD resource, secret value, UniDesk runtime substitute, browser e2e,
|
||||
heavyweight e2e, or force push is part of this flow.
|
||||
|
||||
Source plans, dry-run reports, and desired-state convergence checks are M3
|
||||
support only. They must not be labelled DEV-LIVE without a real DEV observation
|
||||
of the `DO1 -> patch-panel -> DI1` loop plus operation, trace, audit, and
|
||||
evidence records.
|
||||
@@ -1,90 +0,0 @@
|
||||
# M2 DEV Deployment Smoke
|
||||
|
||||
`m2-dev-deploy-smoke` is a local dry-run smoke harness for the HWLAB DEV
|
||||
deployment shape. It validates the frozen route phases, service identities,
|
||||
artifact metadata, and safety gates without contacting the real DEV ingress by
|
||||
default.
|
||||
|
||||
Run the dry-run from the repository root:
|
||||
|
||||
```sh
|
||||
node scripts/m2-dev-deploy-smoke.mjs --dry-run
|
||||
```
|
||||
|
||||
## What It Checks
|
||||
|
||||
- Parses `fixtures/dev-deploy-smoke/dev-deploy-smoke.json`.
|
||||
- Verifies the route phases in order:
|
||||
`master-edge-proxy`, `frp-tunnel`, `d601-router`, `cloud-surface`,
|
||||
`agent-runtime`, and `sim-and-patch-panel`.
|
||||
- Validates the frozen HWLAB service IDs for master edge proxy, frp tunnel,
|
||||
D601 `hwlab-dev/hwlab-router`, cloud-api/web, agent, sim, and patch-panel.
|
||||
- Checks `commitId`, `image`, `digest` or `not_applicable`, `tag`,
|
||||
`buildSource`, `deployEnv`, `healthTimestamp`, and failure classification
|
||||
for every recorded service contract.
|
||||
- Confirms the cloud surface bundles `hwlab-cloud-api` and `hwlab-cloud-web`
|
||||
artifacts together under the same route phase.
|
||||
- Marks the fixture as dry-run evidence only. Any DB fields in fixture health
|
||||
records are contract placeholders, not live DB evidence.
|
||||
- Keeps the default mode in dry-run and refuses real requests unless an
|
||||
explicit live gate is supplied.
|
||||
|
||||
## Live Gate
|
||||
|
||||
The live gate is intentionally strict. It is only for a future real DEV smoke
|
||||
observation, not for deployment, and this harness stays fixture-only unless the
|
||||
operator supplies the live flags. The public DEV endpoint split is:
|
||||
|
||||
- browser/frontend: `http://74.48.78.17:16666/`
|
||||
- API/edge health: `http://74.48.78.17:16667/health` and
|
||||
`http://74.48.78.17:16667/health/live`
|
||||
- legacy public `:6667`: deprecated and never accepted as active green evidence
|
||||
|
||||
Required flags:
|
||||
|
||||
```sh
|
||||
node scripts/m2-dev-deploy-smoke.mjs --live --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
Before using `--live`, confirm all of the following:
|
||||
|
||||
- the API/edge target remains `http://74.48.78.17:16667`;
|
||||
- the frontend target remains `http://74.48.78.17:16666`;
|
||||
- the workspace is not a PROD or shared deployment context;
|
||||
- no prohibited action is being attempted;
|
||||
- the request is observational only and does not mutate runtime state.
|
||||
|
||||
The script rejects live execution if any of these are missing:
|
||||
|
||||
- `--confirm-dev`
|
||||
- `--confirmed-non-production`
|
||||
- `--prod`
|
||||
- `--heavyweight-e2e`
|
||||
- `--read-secret`
|
||||
- `--force-push`
|
||||
|
||||
When `--report /tmp/hwlab-dev-gate/report.json` is provided, the script writes
|
||||
`/tmp/hwlab-dev-gate/dev-m2-deploy-smoke-active.json`. The report contains only
|
||||
read-only GET evidence and marks legacy `:6667` observations as ineligible for
|
||||
active green.
|
||||
|
||||
Validate the written report with:
|
||||
|
||||
```sh
|
||||
node scripts/validate-m2-deploy-smoke-active.mjs
|
||||
```
|
||||
|
||||
## Boundary
|
||||
|
||||
This harness does not:
|
||||
|
||||
- deploy to DEV or PROD;
|
||||
- call the real DEV endpoint in dry-run mode;
|
||||
- run heavy e2e coverage;
|
||||
- read secrets or tokens;
|
||||
- replace HWLAB runtime services with UniDesk substitutes;
|
||||
- touch L5 deploy manifests.
|
||||
|
||||
The fixture is a dry-run record only. It exists to prove that the M2 smoke
|
||||
entry point understands the frozen route and artifact contract before any live
|
||||
DEV observation is attempted.
|
||||
@@ -262,5 +262,5 @@ blockers/failures：`viewport`、`selector`、`failureType`、summary 和 artifa
|
||||
chat 同源 readiness 和 provider credential blocker 边界。
|
||||
- [docs/cloud-web-workbench.md](../cloud-web-workbench.md): detailed frontend
|
||||
contract.
|
||||
- [docs/dev-deploy-apply.md](../dev-deploy-apply.md): Cloud Web manual rollout
|
||||
- [g14-gitops-cicd.md](g14-gitops-cicd.md): Cloud Web rollout through G14 Tekton, GitOps and Argo CD
|
||||
path.
|
||||
|
||||
@@ -1,821 +0,0 @@
|
||||
# HWLAB Deployment And Publish Reference
|
||||
|
||||
This reference records the stable DEV publish and rollout rules. It does not
|
||||
authorize PROD, service restarts outside the documented DEV path, secret reads,
|
||||
or direct pushes to `main`.
|
||||
|
||||
It is also the long-term reference for
|
||||
[pikasTech/HWLAB#116](https://github.com/pikasTech/HWLAB/issues/116): service
|
||||
deployment regularization must progress from stable docs, to controlled
|
||||
CLI/script entrypoints, to UniDesk CI/CD plus imageized delivery.
|
||||
|
||||
## D601 Native k3s Emergency Guard
|
||||
|
||||
HWLAB DEV deploy and publish paths must target D601 native k3s only. Docker
|
||||
Desktop Kubernetes on D601 has been disabled and cleaned; it must not be
|
||||
re-enabled or used as a second orchestrator. The cross-repo incident, evidence,
|
||||
and governance plan are tracked in
|
||||
[pikasTech/unidesk#138](https://github.com/pikasTech/unidesk/issues/138), and
|
||||
the 2026-05-23 commander recovery context is in
|
||||
[pikasTech/unidesk#118](https://github.com/pikasTech/unidesk/issues/118).
|
||||
|
||||
Every DEV CD status, apply, rollback, smoke, and manual Kubernetes command must
|
||||
use `KUBECONFIG=/etc/rancher/k3s/k3s.yaml` and verify node `d601` before any
|
||||
mutation or acceptance. Bare `kubectl`, `docker-desktop` context,
|
||||
`desktop-control-plane`, or `127.0.0.1:11700` are wrong-control-plane signals,
|
||||
not HWLAB DEV-LIVE evidence. A second `hwlab-dev` control plane, including a
|
||||
Docker Desktop cluster that happens to contain similarly named resources, is a
|
||||
blocker and cannot be used as deploy, rollout, smoke, or acceptance proof.
|
||||
|
||||
## Workspaces
|
||||
|
||||
| Purpose | Path |
|
||||
| --- | --- |
|
||||
| Runner and commander worktree | `/workspace/hwlab` |
|
||||
| D601 shared source cache | `/home/ubuntu/workspace/hwlab` |
|
||||
| D601 clean CI/CD worktree | `/tmp/hwlab-cicd-<target-short-sha>` |
|
||||
| D601 runtime experiment worktree | `/tmp/hwlab-<purpose>-<target-short-sha>` |
|
||||
|
||||
Git remote is the source of truth. D601 may contain several HWLAB clones or
|
||||
worktrees from earlier reviews, Code Queue runs, runtime experiments, or CI/CD
|
||||
jobs; none of those local directories is authoritative by itself. Before using
|
||||
any D601 repo path for checks, publish, or CD, verify its role, `origin`, HEAD,
|
||||
and cleanliness.
|
||||
|
||||
`/home/ubuntu/workspace/hwlab` is the shared source cache and convenient seed
|
||||
clone. It is allowed to be stale, behind `origin/main`, or dirty with generated
|
||||
`reports/dev-gate/**` files from previous runs. Do not `reset --hard`, delete
|
||||
reports, or overwrite that directory just to make a release command work. If it
|
||||
is not clean and already at the target commit, use it only to `git fetch` and
|
||||
create a detached clean worktree.
|
||||
|
||||
Formal CI publish and DEV CD apply must run from a clean, target-pinned
|
||||
worktree such as `/tmp/hwlab-cicd-<target-short-sha>`. The worktree may be
|
||||
created from the shared cache, but it must prove all of the following before
|
||||
side effects:
|
||||
|
||||
- `git remote get-url origin` points to `git@github.com:pikasTech/HWLAB.git`
|
||||
or the approved GitHub source of truth.
|
||||
- `git fetch origin main` succeeds, and `git rev-parse HEAD` equals the target
|
||||
commit being published or deployed.
|
||||
- `git status --short` is empty.
|
||||
- The `hwlab-cli cicd submit ...` job `cwd` in `state.json` points to the clean
|
||||
worktree, not to a stale runner directory or a dirty shared cache.
|
||||
|
||||
Runtime hotfix and gateway probing may use purpose-specific D601 worktrees under
|
||||
`/tmp/hwlab-<purpose>-<sha>`. Those worktrees are for copying scripts into pods,
|
||||
running probes, and staging temporary patches; they are not release truth and
|
||||
must not be used to infer desired state unless they are also clean and pinned to
|
||||
the target commit.
|
||||
|
||||
Do not treat `/home/ubuntu/hwlab`, historical runner directories, previous PR
|
||||
review worktrees, or pod-local copied trees as publish/build truth. If a path's
|
||||
role is unclear, stop and create a new detached worktree from `origin/main`
|
||||
instead of repairing the unknown directory in place.
|
||||
|
||||
## Master Server Check Boundary
|
||||
|
||||
Master server is only a control, coordination, Git, issue/PR, and short polling
|
||||
entrypoint for HWLAB. It must not run HWLAB validation workloads locally:
|
||||
`npm run check`, `npm run web:check`, Playwright/browser smokes, full test
|
||||
suites, build/preflight jobs, image publishing, or DEV CD checks belong on
|
||||
D601, Code Queue runners, or the formal CI/CD job plane.
|
||||
|
||||
The master-side HWLAB clone may be used for lightweight source reading,
|
||||
`git status`, `git diff`, patch preparation, commits, PR creation, and polling
|
||||
D601/CI/CD status. If a validation command would consume significant CPU,
|
||||
memory, browser resources, Docker, k3s, registry, or long-lived process time,
|
||||
skip it on master and run the equivalent command from a clean D601 worktree or
|
||||
submit it through `hwlab-cli cicd submit`.
|
||||
|
||||
Master wrappers may start short-lived D601 commands and poll their status, but
|
||||
the validation work itself must execute on D601 or CI/CD. A master-local
|
||||
timeout, browser hang, or resource shortage is not DEV-LIVE evidence and must
|
||||
not be turned into a product blocker; stop the local check and move the
|
||||
verification to D601/CI/CD.
|
||||
|
||||
## Regularization Stages
|
||||
|
||||
| Stage | Current HWLAB state | Required next behavior |
|
||||
| --- | --- | --- |
|
||||
| 1. Long-term reference | This file records DEV deployment, Cloud Web publish, API/edge/health, frp, D601 registry, k3s rollout, rollback, and verification rules. `AGENTS.md` indexes it. | Keep process issues as sources only; update this reference when a deployment rule changes. |
|
||||
| 2. CLI or controlled scripts | `hwlab-cli cicd submit/status/logs/report` 是正式 CI/CD 外层控制入口；`scripts/dev-artifact-publish.mjs` 和 `scripts/dev-cd-apply.mjs` 只作为后台 worker 或只读状态脚本使用。Other controlled scripts still cover contract planning, dry-run planning, preflight, single-step internals, k3s read-only visibility, and Cloud Web checks. Rollback is documented as commands/report hints but is not yet one unified CLI command. | Continue converging check, plan, rollback, status, and verify into HWLAB CLI while preserving CI/CD separation and the single CD transaction lock backed by `deploy/deploy.json`. |
|
||||
| 3. UniDesk CI/CD plus images | DEV services are imageized and published to the D601 registry. D601 rollout still includes manual bridge steps for Cloud Web and frp/k3s execution. | Move build, publish, desired-state refresh, rollout, post-deploy validation, and report upload into standard UniDesk CI/CD jobs without replacing HWLAB runtime. |
|
||||
|
||||
## CI/CD CLI 非阻塞合同
|
||||
|
||||
HWLAB CI/CD 正式 CLI 必须遵循 `Skill(cli-spec)`：可能超过短时间的 build、
|
||||
push、apply、rollout、runtime Job、smoke 和 postflight 都不得以前台阻塞命令
|
||||
暴露给指挥官、runner 或 master-side wrapper。正式入口必须使用
|
||||
Fire-and-Forget 模式：
|
||||
|
||||
```sh
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd submit --kind ci-publish --concurrency 4
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd submit --kind dev-cd-apply --confirm-dev --confirmed-non-production --concurrency 4
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd status <jobId>
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd logs <jobId> --tail-bytes 8000
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd report <jobId>
|
||||
```
|
||||
|
||||
`submit` 只创建 `.state/hwlab-cicd/jobs/<jobId>/`，写入 `spec.json`、
|
||||
`state.json`、`stdout.log`、`stderr.log` 和 `report.json` 路径，然后启动后台
|
||||
worker 并立即返回有界 JSON。`status` 仅读 `state.json`，`logs` 默认只返回
|
||||
尾部摘要，`report` 默认只返回报告摘要；完整日志和完整报告通过返回的文件路径
|
||||
渐进查看。任何正式 CLI 调用都不得因为 Docker build、registry push、
|
||||
`kubectl wait --timeout` 或 rollout status 在前台长时间占用上下文。
|
||||
|
||||
runner pod 内执行 CD 时，`127.0.0.1:5000` 指向 pod 自身，不等于 D601 host 的
|
||||
loopback registry；但 image reference 必须继续保持 `127.0.0.1:5000/...`，因为
|
||||
k3s/containerd 拉镜像和 CI push 使用的是 host 视角。CD registry manifest 校验
|
||||
默认先走直接 HTTP；如果 pod loopback 被拒绝，则用 Docker host-network helper
|
||||
（`docker run --rm --pull=never --network host`）读取同一个 registry manifest。
|
||||
正式命令仍必须通过非阻塞入口提交；如需显式 registry HTTP 入口，可在提交时加
|
||||
`--registry-manifest-base-url <url>`，该参数只改变 manifest 校验 URL，不改变
|
||||
`deploy.json` 或 workload 里的镜像引用。
|
||||
|
||||
`scripts/dev-artifact-publish.mjs`、`scripts/dev-cd-apply.mjs` 和其他内部脚本仍
|
||||
保留为 worker/backend：它们可以在后台 job 内执行长流程，但不能作为指挥面常规
|
||||
入口直接运行。临时手动排障可以直接调用底层脚本，但排障结论必须回填到正式
|
||||
CLI、长期参考文档或 issue，避免把一次性命令固化为新流程。
|
||||
|
||||
## D601 Code Queue runner 实测调试法
|
||||
|
||||
HWLAB CI/CD 的关键验收必须从真正使用者 runner 的容器或 pod 内复测。master
|
||||
server、指挥官本地 clone、D601 host 普通目录和历史 runner 目录只能作为控制面
|
||||
或辅助观察面；它们不能替代当前 Code Queue 执行面看到的 worktree、网络、
|
||||
kubeconfig、registry、Docker socket、Secret/ConfigMap mount 和 CLI 行为。
|
||||
|
||||
实地调试顺序固定为：
|
||||
|
||||
1. 先从调度器、任务、pod 和进程信息确认当前承担生产调度的 Code Queue runner
|
||||
或 scheduler 容器，避免误入已经废弃、空闲或历史测试容器。
|
||||
2. 通过受控维护入口进入该容器，在容器内建立干净 HWLAB worktree，并快进到
|
||||
`origin/main`；不得直接使用 `/home/ubuntu/hwlab` 等历史任务集合目录作为
|
||||
发布真相。
|
||||
如果 worktree 是从另一个本地目录 clone 出来的，必须先确认 `origin` 指向
|
||||
`git@github.com:pikasTech/HWLAB.git` 或等价 GitHub source of truth，再
|
||||
`git fetch origin main`。DEV CD 的 target-ref guard 会比较 HEAD 与
|
||||
`origin/main`；本地 clone 的 stale `origin/main` 会触发
|
||||
`target-ref-head` blocker，不能通过跳过 guard 处理。
|
||||
3. 在同一个容器内执行 `node tools/hwlab-cli/bin/hwlab-cli.mjs cicd submit ...`，
|
||||
用 `status`、`logs`、`report` 查询后台 job，而不是前台直接跑长脚本。
|
||||
4. 至少完成一次 CI artifact publish 和一次 DEV CD apply，再用公开
|
||||
`16666/16667` health 核对 live commit、service readiness 和 blocker 字段。
|
||||
5. 调试结论必须写回 issue、正式 CLI、preflight 或长期参考文档；不能把一次性
|
||||
容器命令当作新的常规发布流程。
|
||||
|
||||
runner 内最小复测命令形态如下：
|
||||
|
||||
```sh
|
||||
cd <clean-hwlab-worktree>
|
||||
git fetch origin main
|
||||
git checkout main
|
||||
git pull --ff-only origin main
|
||||
git status --short
|
||||
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml \
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd submit --kind ci-publish --concurrency 4
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd status <ci-job-id>
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd report <ci-job-id>
|
||||
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml \
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd submit --kind dev-cd-apply --confirm-dev --confirmed-non-production --concurrency 4
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd status <cd-job-id>
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd report <cd-job-id>
|
||||
```
|
||||
|
||||
一次 runner 实测只有同时满足以下条件，才可判断 CI/CD 链路通畅：
|
||||
|
||||
- `ci-publish` 和 `dev-cd-apply` job 都进入终态 `succeeded`；
|
||||
- job report 中 `blockers` 为空，耗时、服务数、镜像 tag/digest 和 target commit
|
||||
有界展示；
|
||||
- `Lease/hwlab-dev/hwlab-dev-cd-lock` 在事务结束后释放或显示为无活动 holder；
|
||||
- 公开 `16666` 和 `16667` health 的 commit 与 `deploy/deploy.json` promotion
|
||||
commit 一致；
|
||||
- runner 容器内没有残留 `hwlab-cli`、`dev-cd-apply` 或旧发布 worker 进程继续
|
||||
持有写路径。
|
||||
|
||||
本次 CI/CD 疏通暴露出的稳定故障类别也必须长期前置：
|
||||
|
||||
- `127.0.0.1:5000` 在 runner pod 内、Docker host-network helper 内和 k3s
|
||||
containerd 内含义不同。镜像引用仍使用 `127.0.0.1:5000/...` 作为 D601 host
|
||||
registry identity；registry manifest 校验按运行视角选择直接 HTTP 或 Docker
|
||||
host-network helper，不得为了 pod 视角改写 release identity。
|
||||
- Codex stdio 所需 `auth.json` Secret 和 `config.toml` ConfigMap 是发布前置
|
||||
条件。preflight 只检查对象和 key 名，不读取 secret value；ConfigMap 属于
|
||||
repo desired-state，Secret 属于外部环境。
|
||||
- 已经恢复的 live 服务不能替代正式 CD 报告。若第一次 CD 因缺少 mount、
|
||||
registry 视角或权限失败，即使手动恢复了 live，也必须再次从 runner 内走
|
||||
非阻塞 DEV CD job，产出通过报告。
|
||||
- CI 产物事实以 registry manifest 和 artifact catalog 为准；临时
|
||||
`/tmp/hwlab-dev-gate/*.json` 只用于审计和耗时排障，不得成为 CD 放行硬依赖。
|
||||
- `kubectl apply -k` 不会自动移除由运行面 `kubectl patch`、临时 ConfigMap
|
||||
热修或其他 field manager 写入、且不属于 desired manifest / last-applied 的
|
||||
Deployment `volumes`、`volumeMounts` 或 template annotations。正式 CD 不能把
|
||||
“镜像 tag 已更新”误判为“热修覆盖已撤销”。DEV CD apply 必须在 apply 前比对
|
||||
desired Deployment 与 live Deployment：若 live 存在 desired 不包含、且明确带
|
||||
`hotfix`/`override` 语义的额外覆盖字段，应在 `hwlab-dev` 内删除该 desired
|
||||
Deployment，再由同一次 `kubectl apply -k deploy/k8s/dev` 从源码 desired-state
|
||||
重新创建；报告中记录 cleanup 计划和实际动作。不要把这种 cleanup 做成某个服务
|
||||
或某个 skill 的特例。
|
||||
- Kubernetes service links 会为同 namespace Service 自动注入形如
|
||||
`HWLAB_CLOUD_API_PORT=tcp://...` 的环境变量。应用入口不得直接把这类 env 当作
|
||||
数字端口解析；desired-state 应显式声明服务自身监听端口
|
||||
`HWLAB_CLOUD_API_PORT=6667`，入口代码也必须只接受合法数值端口，否则回落到
|
||||
内置默认值。
|
||||
|
||||
## DEV CD Transaction
|
||||
|
||||
正式 DEV CD 写路径只有一个外层入口：
|
||||
|
||||
```sh
|
||||
node tools/hwlab-cli/bin/hwlab-cli.mjs cicd submit --kind dev-cd-apply --confirm-dev --confirmed-non-production --concurrency 4
|
||||
```
|
||||
|
||||
该命令后台调用 `scripts/dev-cd-apply.mjs`，是唯一允许按 `deploy.json` 对 `hwlab-dev` 执行 desired-state apply、并复验
|
||||
公开 `16666/16667` 的正式 CD 写路径。镜像发布属于 CI artifact 职责；过渡期
|
||||
latest-main 兼容路径仍可由该命令调用 artifact publish，但正式 rollout 默认应消费
|
||||
已经发布并经过 registry manifest 验证的 artifact。
|
||||
事务开始时读取 `deploy/deploy.json`，记录 manifest hash 和 target commit。
|
||||
在获取 `Lease/hwlab-dev/hwlab-dev-cd-lock` 或运行任何 publish、runtime Job、
|
||||
apply、rollout 类副作用前，脚本必须先完成 apply-only preflight：强制
|
||||
`KUBECONFIG=/etc/rancher/k3s/k3s.yaml`，验证节点包含 `d601`，拒绝
|
||||
`docker-desktop`、`desktop-control-plane`、`127.0.0.1:11700`、裸
|
||||
`kubectl` 指向第二套 `hwlab-dev` 控制面或其他不可信控制面信号，并且只验证
|
||||
必要 SecretRef 的存在性和 key 名。通过后才获取 Lease、串行执行内部步骤、
|
||||
按需写入 `/tmp/hwlab-dev-gate/dev-cd-apply.json`，最后释放 Lease。
|
||||
|
||||
apply preflight 检查的 SecretRef 固定为：
|
||||
|
||||
- `hwlab-cloud-api-dev-db/database-url`
|
||||
- `hwlab-cloud-api-dev-db-admin/admin-url`
|
||||
- `hwlab-code-agent-provider/openai-api-key`
|
||||
- `hwlab-code-agent-codex-auth/auth.json`
|
||||
|
||||
Code Agent Codex stdio 还必须在 preflight 阶段验证非 secret ConfigMap
|
||||
`hwlab-code-agent-codex-config/config.toml` 的存在性和 key 名；该 ConfigMap 由
|
||||
`deploy/k8s/base/code-agent-codex-config.yaml` 纳入 DEV desired-state。Codex
|
||||
auth 仍然只能作为外部 Secret 管理，禁止把 `auth.json` 内容写进仓库或报告。
|
||||
|
||||
preflight 和报告只能记录 Secret 名、key 名、存在性、key-name 观察结果和脱敏
|
||||
命令摘要；不得读取或打印 Secret value。阻塞时输出结构化 blocker，包含
|
||||
`scope`、`reason`、`impact`、`safeNextAction` 和 `retryable`，默认 stdout 只
|
||||
给有界摘要，完整 preflight 证据写入事务报告。
|
||||
|
||||
`deploy/deploy.json` 的 `commitId` 是运行时 promotion commit。正常
|
||||
latest-main 发布时它可以等于当前 target ref；artifact merge commit 或后续
|
||||
控制面提交存在时，它也可以指向一个已经发布并在 catalog/workloads 中完全收敛
|
||||
的早期 promotion commit。`scripts/dev-cd-apply.mjs` 必须先用
|
||||
`deploy-desired-state-plan --promotion-commit <commit>` 验证该 desired-state
|
||||
已经发布且内部一致；验证通过后，`--apply` 消费这个 promotion commit 的镜像
|
||||
执行 apply/verify，不重新 publish 或刷新当前 HEAD 的 artifact。这样 host
|
||||
commander 可以始终从最新 `origin/main` 控制入口执行正式 DEV CD，而不需要
|
||||
checkout promotion parent 或手工拼装发布环境。
|
||||
|
||||
状态面和事务报告会显式展示这条边界：`target.controlRef` 是最新
|
||||
`origin/main` 控制入口，`target.promotionCommit` 是从 `deploy/deploy.json`
|
||||
解析出的真实 promotion commit，`target.artifactBoundary` 记录最近一次触碰
|
||||
`deploy/deploy.json`、`deploy/artifact-catalog.dev.json`、
|
||||
`deploy/k8s/base/workloads.yaml` 的 first-parent desired-state commit，以及该
|
||||
commit 的 promotion parent。`artifactCatalog`、`artifactReport`、`deployJson`
|
||||
分别记录 hash、commit、digest 计数和 publish source，但只有
|
||||
`deploy/deploy.json`、artifact catalog、workload manifest 和 registry manifest
|
||||
共同决定 CD 是否可执行；`/tmp/hwlab-dev-gate/dev-artifacts.json` 只是 CI 审计快照。
|
||||
若 deploy/catalog/workloads 或 desired-state parent 不能同时指向同一个
|
||||
promotion commit，状态降级为 `degraded`，真实 `--apply` 在获取 Lease 前以
|
||||
artifact-boundary blocker 停止。缺失或过期的 CI 产物报告只能产生诊断提示，不得
|
||||
阻塞已经由 desired-state 和 registry 证明一致的 CD rollout。这保证
|
||||
`deploy/deploy.json` 仍是唯一发布控制源；Lease 只串行化事务，不保存第二套
|
||||
desired-state。
|
||||
|
||||
Lease 只是发布平面的互斥锁，不是 desired-state 来源。Lease annotations
|
||||
至少记录 `promotionCommit`、`deployJsonHash`、`ownerTaskId`、
|
||||
`transactionId`、`phase`、`startedAt`、`updatedAt`、`ttlSeconds`、
|
||||
`liveBefore` 和 `targetNamespace`。`deploy/deploy.json`、
|
||||
`deploy/artifact-catalog.dev.json`、`deploy/k8s/base/workloads.yaml` 仍是唯
|
||||
一权威部署输入。
|
||||
|
||||
如果另一个事务持有 Lease，`--apply` 必须在 publish/apply 之前停止，并
|
||||
输出结构化 `deploy-lock-held`，包含 holder、promotion commit、phase、
|
||||
target namespace 和 retry seconds。过期锁不能静默覆盖；stale lock 只能
|
||||
在确认 holder 已失活后显式使用 `--break-stale-lock --confirm-dev`，并把
|
||||
previous holder 保留在审计 annotations/report 字段中。未过期 Lease 没有
|
||||
抢占入口，普通 status 输出也不得建议强制抢锁。
|
||||
|
||||
`scripts/dev-cd-apply.mjs` 同时提供只读状态面。无参数、`--status`、或
|
||||
`--dry-run` 都返回紧凑 JSON，覆盖 target ref/commit、
|
||||
`deploy/deploy.json` commit/hash、当前 Lease 状态摘要、以及公开
|
||||
`16666/16667` live health identity 摘要。状态面还包含 `liveDelta`，用于对比
|
||||
当前公开 live commit 与 `target.promotionCommit`；这只是 rollout 口径，不会在
|
||||
status/dry-run 中执行 rollout。这些模式 host commander 和 runner 都可以执行，
|
||||
但必须保持只读：不获取或替换 Lease，不 publish artifact，不执行
|
||||
`kubectl apply/rollout`，不写 report，不修改 `deploy.json`/catalog，也不读取或
|
||||
打印 Secret value。kubectl 或 kubeconfig
|
||||
不可用时，状态面应返回 `lock.status=unavailable` 和脱敏原因，而不是把
|
||||
只读观测失败升级成写路径 blocker。
|
||||
|
||||
## `deploy.json` DEV CD 路径
|
||||
|
||||
当前已经验证可用的过渡路径是：master server 只作为受控指挥入口，通过
|
||||
UniDesk CLI 进入 D601；D601 上使用干净 HWLAB 工作区运行本仓库
|
||||
`scripts/dev-cd-apply.mjs`；发布目标由 `deploy/deploy.json` 和同 commit
|
||||
的 artifact catalog / k8s workloads 决定；真实写入只落到 D601 原生 k3s
|
||||
`hwlab-dev` namespace。复盘来源见
|
||||
[pikasTech/HWLAB#61](https://github.com/pikasTech/HWLAB/issues/61#issuecomment-4525401375)，
|
||||
长期化跟踪见 [pikasTech/HWLAB#340](https://github.com/pikasTech/HWLAB/issues/340)，
|
||||
单事务入口和发布锁见 [pikasTech/HWLAB#274](https://github.com/pikasTech/HWLAB/issues/274)，
|
||||
D601 控制面归一见 [pikasTech/unidesk#138](https://github.com/pikasTech/unidesk/issues/138)。
|
||||
|
||||
稳定调用合同如下；外层 master CLI 或 UniDesk wrapper 只能包裹这些
|
||||
repo-owned 入口，不能绕过它们直接拼接 publish、apply、rollout 或 smoke：
|
||||
|
||||
| 目的 | 稳定入口 | 副作用边界 |
|
||||
| --- | --- | --- |
|
||||
| 读取 DEV CD 状态 | `node scripts/dev-cd-apply.mjs --status` | 不写入、不获取 Lease、不读取 Secret value。 |
|
||||
| 事务 dry-run preflight | `node scripts/dev-cd-apply.mjs --dry-run` | 不 publish、不 apply、不 rollout。 |
|
||||
| 检查 desired-state 收敛 | `node scripts/deploy-desired-state-plan.mjs --promotion-commit <sha> --check` | 只读 source/report。 |
|
||||
| 提交 CI artifact publish | `node tools/hwlab-cli/bin/hwlab-cli.mjs cicd submit --kind ci-publish --concurrency 4` | 立刻返回 jobId；后台 build/push，报告写入 `.state/hwlab-cicd/jobs/<jobId>/report.json`。 |
|
||||
| 提交 DEV CD 事务 | `node tools/hwlab-cli/bin/hwlab-cli.mjs cicd submit --kind dev-cd-apply --confirm-dev --confirmed-non-production --concurrency 4` | 立刻返回 jobId；后台在 `Lease/hwlab-dev/hwlab-dev-cd-lock` 下 apply/verify；legacy latest-main publish 仅作过渡兼容。 |
|
||||
| 查询 CI/CD job | `node tools/hwlab-cli/bin/hwlab-cli.mjs cicd status|logs|report <jobId>` | 只读 `.state/hwlab-cicd/jobs/<jobId>/`；日志和报告默认有界摘要。 |
|
||||
| 读取 D601 k3s 证据 | `node scripts/d601-k3s-readonly-observability.mjs` | 只读可见性报告；不 rollout、不输出 Secret value。 |
|
||||
|
||||
使用 master-side wrapper 时，稳定形态是：
|
||||
|
||||
```sh
|
||||
cd /root/unidesk
|
||||
bun scripts/cli.ts ssh D601 argv bash -lc 'cd <clean-hwlab-cd-worktree> && KUBECONFIG=/etc/rancher/k3s/k3s.yaml node scripts/dev-cd-apply.mjs --status'
|
||||
bun scripts/cli.ts ssh D601 argv bash -lc 'cd <clean-hwlab-cd-worktree> && KUBECONFIG=/etc/rancher/k3s/k3s.yaml node tools/hwlab-cli/bin/hwlab-cli.mjs cicd submit --kind dev-cd-apply --confirm-dev --confirmed-non-production --concurrency 4'
|
||||
```
|
||||
|
||||
`<clean-hwlab-cd-worktree>` 必须指向干净工作区。该工作区可以是固定 clean
|
||||
mirror，也可以是一次性 `git clone --shared --no-checkout` 生成的 ephemeral
|
||||
worktree；它不能是 `/home/ubuntu/hwlab` 这类 runner/历史任务集合目录。进入
|
||||
apply 前必须证明：
|
||||
|
||||
- `git status --short` 为空；
|
||||
- `.git/FETCH_HEAD` 和 `.git/worktrees` 没有阻止普通部署用户读取或创建
|
||||
干净工作区的权限问题；
|
||||
- `node scripts/dev-cd-apply.mjs --status` 显示 `promotionSource=deploy-json`；
|
||||
- `deploy/deploy.json` 的 commit、`deploy/artifact-catalog.dev.json` 和
|
||||
`deploy/k8s/base/workloads.yaml` 收敛到同一个 promotion commit / image tag；
|
||||
- `KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl get nodes` 能看到节点 `d601`；
|
||||
- `kubectl config current-context`、cluster server 或节点列表没有
|
||||
`docker-desktop`、`desktop-control-plane`、`127.0.0.1:11700` 或第二套
|
||||
`hwlab-dev` 控制面证据；
|
||||
- `Lease/hwlab-dev-cd-lock` 未被未过期事务持有，或者 stale lock 已由受控
|
||||
`--break-stale-lock --confirm-dev` 审计处理。
|
||||
|
||||
desired-state 来源关系固定如下：
|
||||
|
||||
| 来源 | 权威范围 |
|
||||
| --- | --- |
|
||||
| `deploy/deploy.json` | DEV environment、namespace、endpoint、service set、promotion commit 和事务消费的 image tag。 |
|
||||
| `deploy/artifact-catalog.dev.json` | 同一 promotion commit 的不可变 artifact digest/tag 期望；CD 必须直接向 registry 校验 tag/digest 是否真实存在。 |
|
||||
| `deploy/k8s/base/workloads.yaml` | Kubernetes workload desired state，必须镜像 `deploy.json` 和 artifact catalog 后才能 apply。 |
|
||||
| Registry manifest | 镜像事实源；回答目标 tag/digest 是否存在，不能决定哪一组服务应该上线。 |
|
||||
| `/tmp/hwlab-dev-gate/*.json` | 只作为 CI/CD 审计、排障和耗时统计快照；不能覆盖 repo desired state，也不能成为 CD 放行硬依赖。 |
|
||||
| `Lease/hwlab-dev/hwlab-dev-cd-lock` | 只作为事务互斥锁和审计状态；永远不是 desired-state 来源。 |
|
||||
|
||||
只改文档的 commit 如果没有修改 `deploy/deploy.json`、
|
||||
`deploy/artifact-catalog.dev.json`、`deploy/k8s/base/workloads.yaml`、service
|
||||
source 或 runtime manifest，就不会产生新的 DEV image 或 promotion commit。
|
||||
这类 commit 的 artifact build/publish 结论是 not applicable，除非另一个
|
||||
artifact 任务明确更新 desired state。
|
||||
|
||||
## Runner 和 Host 边界
|
||||
|
||||
Deployment/CD 工作按以下边界分工：
|
||||
|
||||
| 角色 | 可以执行 | 没有明确 rollout 授权时不得执行 |
|
||||
| --- | --- | --- |
|
||||
| Code Queue runner | 准备分支和 PR；在任务明确授权且检查通过后更新或自合并 docs/code/artifact PR；通过 repo-owned 路径 build/publish artifact；在被分配时刷新 artifact report 或 desired-state 文件；运行只读 `status`/`dry-run`/preflight 检查。 | 未授权自合并、竞争 DEV CD Lease、执行 DEV apply、rollout、声称 `16666/16667` live verification、收口 `#7` 或 `#242`、修改 PROD、打印 Secret value、手工修 live resource。 |
|
||||
| Host commander | 在需要指挥官 review 时审阅并合并最终 PR；执行单一 DEV deploy apply 事务；执行 rollout 和 `16666/16667` live verification；收口 `#7`/`#242` 状态。 | 绕过 repo-owned CD 入口、使用第二控制面、用 UniDesk runtime 替代 HWLAB runtime、只凭报告而不做一手复验就当作 live evidence。 |
|
||||
|
||||
Runner 自合并权限只在单个任务和单个 PR 的显式授权边界内生效。它不授予
|
||||
DEV apply、rollout、live health verification 或看板/status 收口权限。Runner
|
||||
发布 artifact 时，输出必须包含 tag、digest 和 report 证据，并把
|
||||
desired-state convergence 与 live rollout evidence 分开。
|
||||
|
||||
## CI/CD 职责拆分
|
||||
|
||||
长期稳定规则是 CI 负责构建和发布镜像，CD 只消费已经构建好的镜像和
|
||||
repo-owned desired state。CD 入口不得临场重新解释 source、手工拼装 build
|
||||
env、重新构建镜像或把构建失败伪装成 rollout blocker。
|
||||
|
||||
CI/CD 三源分工固定如下：
|
||||
|
||||
| 来源 | 职责 | 不能承担 |
|
||||
| --- | --- | --- |
|
||||
| `deploy/deploy.json` | 唯一发布控制源，声明 promotion commit、service image tag 和 DEV 目标。 | 证明镜像真实存在。 |
|
||||
| Registry manifest | 镜像事实源，证明目标 tag/digest 可被 registry 读取，并与 catalog 记录一致。 | 决定哪一批服务应该作为一个发布集合上线。 |
|
||||
| CI 产物报告 | 自动生成的一次性审计记录，包含构建、推送、digest、耗时和日志摘要。 | 作为手工维护状态表，或成为 CD rollout 的必要本地文件。 |
|
||||
|
||||
`deploy/artifact-catalog.dev.json` 是 `deploy.json` 与 registry 之间的 repo-owned
|
||||
digest 期望表；`deploy/k8s/base/workloads.yaml` 是最终 Kubernetes desired state。
|
||||
CD 在 apply 前必须根据 catalog 的 image tag/digest 直接读取 registry manifest，
|
||||
确认镜像事实存在且 digest 匹配。过期、缺失或路径漂移的
|
||||
`/tmp/hwlab-dev-gate/dev-artifacts.json` 只能帮助定位 CI 问题，不得推翻
|
||||
`deploy.json + catalog + registry` 已经证明一致的发布结论。
|
||||
|
||||
过渡期如果 `scripts/dev-cd-apply.mjs` 发现 `deploy/deploy.json` 已经指向一个
|
||||
通过 desired-state check 的 promotion commit，应走 `promotionSource=deploy-json`
|
||||
路径，只执行 desired-state 校验、runtime 前置、apply、postflight 和 live verify，
|
||||
不得重新 publish 当前 HEAD。未来 CI 常态化后，latest-main target-ref publish
|
||||
路径应降级为受限兼容路径，正式 DEV rollout 默认走已发布 artifact 的
|
||||
deploy-json promotion。
|
||||
|
||||
CI/artifact 层可以使用以下加速面：
|
||||
|
||||
- `scripts/dev-artifact-publish.mjs --build --services <id,...>`：只构建受影响服务，避免 web-only 变更重建所有服务；
|
||||
- `scripts/dev-artifact-publish.mjs --publish --services <id,...>`：只在 artifact 任务边界内发布选定服务并记录 digest；
|
||||
- `scripts/dev-artifact-publish.mjs --publish --concurrency <n>`：并行构建/推送服务镜像，默认 4，最高 8；
|
||||
- `scripts/dev-runtime-base-image.mjs`：按 `package.json` / `package-lock.json`
|
||||
hash 构建 `hwlab-node-runtime-base:deps-<hash>`，让后续服务镜像复用依赖层。
|
||||
|
||||
CD 层不得把上述 CI/artifact 加速面当作 rollout 授权。只有当 desired-state
|
||||
文件已经收敛、registry manifest 直接验证通过，CD 才能拿 Lease 执行 DEV apply。
|
||||
CD apply 内部可以用 `--rollout-concurrency <n>` 或 `HWLAB_DEV_CD_CONCURRENCY`
|
||||
并行执行 registry manifest 验证、DEV allowlist 清理和 post-apply rollout status；
|
||||
并发只加速同一事务内的独立检查，不创建第二个 CD 控制源。
|
||||
|
||||
## Git Repo Owner 权限治理
|
||||
|
||||
GitHub repo owner/admin 权限是治理权限，不是日常构建、合并、发布和清理权限。
|
||||
长期目标是把 owner 从常规交付链路中解耦，避免 owner 账号或 owner-owned
|
||||
工作区成为单点阻塞，也避免 root/owner 文件污染让 runner、CI 或 CD 无法继续。
|
||||
|
||||
稳定分工如下：
|
||||
|
||||
| 权限域 | 可以做什么 | 不应承担什么 |
|
||||
| --- | --- | --- |
|
||||
| Repo owner/admin | 维护 branch protection、required checks、runner/App/token 授权、紧急权限恢复和审计规则。 | 日常手工 build、长期持有脏 worktree、替 runner 修权限污染、绕过 PR 合并主线。 |
|
||||
| CI build identity | 从受保护主线或 PR commit 构建镜像，写 CI artifact、digest report 和受控 PR/commit。 | DEV apply、live rollout、看板收口、读取 Secret value。 |
|
||||
| CD deploy identity | 读取已合并 desired-state 和 registry artifact，获取 DEV Lease，执行 apply/verify。 | 构建镜像、修改 source、改写 `deploy.json` promotion commit。 |
|
||||
| Code Queue runner | 在任务分支完成代码/文档/desired-state PR，运行只读 status/preflight 和授权的 artifact 任务。 | 占用 owner 权限、写 main、污染共享发布 worktree、承担长期 CD lock。 |
|
||||
|
||||
所有自动化应使用可审计、可轮换的 GitHub App 或 fine-grained token；token 只授予
|
||||
所需 repo 和最小 scope。日常主线合并通过 branch protection 和 required CI
|
||||
结果完成，owner/admin 只处理策略变更、紧急解锁和权限恢复。至少保留两个维护者
|
||||
具备 owner/admin 恢复能力，避免单人离线造成 repo 或部署链路停摆。
|
||||
|
||||
共享发布目录只允许作为缓存或 clean mirror，不允许成为 source of truth。任何
|
||||
CI/CD 写操作都应在 ephemeral worktree 或明确 owner 的 clean worktree 中执行；
|
||||
进入写路径前必须检查 `git status --short`、`.git` 可读写权限、工作区 owner/group
|
||||
和 expected remote commit。发现 root-owned 文件、不可快进、未知本地修改或权限
|
||||
不一致时，应结构化报 `environment_blocker`，由权限治理流程修复；不得直接在污染
|
||||
工作区继续 build、apply 或 push。
|
||||
|
||||
## SecretRef Preflight
|
||||
|
||||
apply 前还必须把必要 SecretRef 作为 preflight 条件处理，而不是等待
|
||||
runtime Job 进入 `CreateContainerConfigError`：
|
||||
|
||||
- `hwlab-cloud-api-dev-db/database-url`；
|
||||
- `hwlab-cloud-api-dev-db-admin/admin-url`；
|
||||
- `hwlab-code-agent-provider/openai-api-key`。
|
||||
|
||||
这些检查只能验证 Secret 名称、key 和 env 注入边界，不得打印 Secret value。
|
||||
如果 `hwlab-cloud-api-dev-db-admin/admin-url` 缺失，短期 runbook 是由受控
|
||||
DEV-only secret reconcile 从 `unidesk-dev/postgres-dev` runtime source 生成
|
||||
并写入 `hwlab-dev`；长期应迁移到 declarative secret、sealed/external secret
|
||||
或统一 secret controller。
|
||||
|
||||
SecretRef preflight report 必须使用布尔值和脱敏 metadata：
|
||||
|
||||
| 检查项 | 必要证据 | 缺失时 blocker |
|
||||
| --- | --- | --- |
|
||||
| Source desired-state reference | `deploy/deploy.json` 和 `deploy/k8s/base/workloads.yaml` 声明预期 `secretRef` / `secretKeyRef` name 和 key。 | `contract_blocker`，scope 指向所属 manifest 或 service。 |
|
||||
| Live D601 Secret presence | 在 `hwlab-dev` namespace 中，repo-owned preflight 或 controller 证明必要 Secret resource 和 key 存在。 | `runtime_blocker`，scope 指向缺失的 SecretRef。 |
|
||||
| k3s access for the check | 检查使用 `/etc/rancher/k3s/k3s.yaml`，解析到节点 `d601`，没有命中 Docker Desktop 或第二控制面。 | `environment_blocker` 或 `observability_blocker`，scope 指向 D601 k3s visibility。 |
|
||||
| Redaction | Report 字段声明 `secretValuesPrinted=false`；日志不包含 decoded data、DSN、token、password、kubeconfig material 或 base64 Secret payload。 | `safety_blocker`；在 publish/apply 前停止。 |
|
||||
|
||||
preflight 可以报告 Secret name、key name、namespace、presence boolean、
|
||||
owning workload、missing scope 和 repair runbook。它不得粘贴
|
||||
`HWLAB_CLOUD_DB_URL`, `HWLAB_CLOUD_DB_ADMIN_URL`, `OPENAI_API_KEY`, bearer
|
||||
tokens, passwords, kubeconfig contents, or Secret `.data` values into reports,
|
||||
issues, PRs, logs, screenshots, or chat.
|
||||
|
||||
## Blocker 分类
|
||||
|
||||
DEV CD report 必须保持 blocker 分类稳定，使 runner 和 host 不读完整事务日志
|
||||
也能路由工作：
|
||||
|
||||
| Type | 含义 |
|
||||
| --- | --- |
|
||||
| `safety_blocker` | 请求动作会违反 DEV-only、non-secret、non-PROD、lock 或控制面安全规则。 |
|
||||
| `environment_blocker` | 必要 local/D601 tool、kubeconfig、k3s node、registry、workspace 或 permission 不可用。 |
|
||||
| `contract_blocker` | Source manifest、desired-state 文件、artifact catalog、report schema 或 promotion provenance 不一致。 |
|
||||
| `runtime_blocker` | DEV runtime 前置条件或后置条件缺失，包括 SecretRef presence、DB readiness、durable runtime、rollout 或 workload image convergence。 |
|
||||
| `agent_blocker` | Code Agent provider/model/egress/SecretRef readiness 不完整。 |
|
||||
| `network_blocker` | public 或 internal endpoint 不可达，或返回无效 health response。 |
|
||||
| `observability_blocker` | runner 无法观测足够只读证据来证明状态，但底层 runtime 可能仍然健康。 |
|
||||
|
||||
每个 blocker 必须包含 `type`、`scope`、`status` 和单行 `summary`。可行时还
|
||||
应包含 `unblockHint` 或 `nextTask`。Secret 相关 blocker 必须声明没有打印
|
||||
value。
|
||||
|
||||
成功收口必须同时具备以下证据：
|
||||
|
||||
- `scripts/dev-cd-apply.mjs --status` 返回 `status=pass`；
|
||||
- `Lease/hwlab-dev-cd-lock` 为 `phase=released`、`releaseStatus=pass`；
|
||||
- `16666/health/live` 的 Cloud Web revision 等于 `deploy.json` promotion commit；
|
||||
- `16667/health/live` 的 Cloud API commit、DB readiness、runtime durability 均 ready；
|
||||
- `KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev get deploy,statefulset`
|
||||
显示 persistent workload image tag 收敛到同一 promotion commit；
|
||||
- 输出保持 bounded summary，完整 report 写入 `/tmp/hwlab-dev-gate/dev-cd-apply.json`
|
||||
或 CLI dump 文件，避免把长 JSON 直接塞进指挥上下文。
|
||||
|
||||
正常 `--apply` stdout 默认也是紧凑摘要；完整事务记录继续通过
|
||||
`--report /tmp/hwlab-dev-gate/report.json` 写入既有 report 路径，只有显式 `--full-output` 时才打印
|
||||
完整 JSON。DEV apply、rollout 和 live verification 仍由 host commander
|
||||
统一执行；runner 默认只使用 status/dry-run 观测面，不竞争 DEV CD lock。
|
||||
|
||||
旧的 deploy side-effect 命令已经降为事务内部步骤。直接手动执行
|
||||
`scripts/dev-deploy-apply.mjs --apply` 且没有 `HWLAB_CD_TRANSACTION_ID` 时，会以
|
||||
`cd-transaction-required` 拒绝。artifact publish 是 CI 职责，正式入口需要
|
||||
`HWLAB_CI_ARTIFACT_RUN_ID`；过渡期也接受 `HWLAB_CD_TRANSACTION_ID`。preflight、
|
||||
build、dry-run、check 和 read-only 模式仍可用。
|
||||
|
||||
## Artifact Publish
|
||||
|
||||
DEV artifacts publish to the D601 local/internal registry:
|
||||
|
||||
```text
|
||||
127.0.0.1:5000/hwlab/*
|
||||
```
|
||||
|
||||
The single-step publish command is a CI artifact side-effect step:
|
||||
|
||||
```sh
|
||||
HWLAB_CI_ARTIFACT_RUN_ID=<run-id> node scripts/dev-artifact-publish.mjs --publish
|
||||
```
|
||||
|
||||
It only runs in side-effect mode when `HWLAB_CI_ARTIFACT_RUN_ID` is set by the
|
||||
CI artifact job, or when the legacy `HWLAB_CD_TRANSACTION_ID` is set by
|
||||
`scripts/dev-cd-apply.mjs` during transition. Normal DEV CD must consume
|
||||
`deploy.json` and registry-verified catalog state; it should not rebuild images.
|
||||
|
||||
For Cloud Web source freshness before the transaction:
|
||||
|
||||
```sh
|
||||
node web/hwlab-cloud-web/scripts/check.mjs
|
||||
node web/hwlab-cloud-web/scripts/build.mjs
|
||||
node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
Cloud Web must build `web/hwlab-cloud-web/dist` before publish because the
|
||||
runtime wrapper serves `dist` before source files. A correct source commit with
|
||||
stale `dist` can still publish an old UI. The build/check pair is part of the
|
||||
publish contract, but with separate responsibilities: `check.mjs` validates
|
||||
source and does not require ignored `dist/` in a clean checkout. `build.mjs`
|
||||
regenerates `dist` and verifies every runtime file and route alias matches
|
||||
source. `scripts/dev-artifact-publish.mjs` runs that build/freshness gate and
|
||||
records the `distFreshness` result before it builds the image.
|
||||
|
||||
After a successful multi-service publish, refresh the artifact catalog from the
|
||||
publish report:
|
||||
|
||||
```sh
|
||||
node scripts/refresh-artifact-catalog.mjs --target-ref <published-source-commit> --publish-report /tmp/hwlab-dev-gate/dev-artifacts.json
|
||||
node scripts/deploy-desired-state-plan.mjs --promotion-commit <published-source-commit> --check
|
||||
```
|
||||
|
||||
`ci-publish` only proves that images for its `sourceCommitId` were built and
|
||||
pushed; it does not change the rollout target by itself. The refresh target
|
||||
must be the already published `sourceCommitId` from the CI publish report. A
|
||||
later documentation, catalog, or desired-state commit may record the promotion,
|
||||
but it is not automatically an image source and must not be passed as
|
||||
`--target-ref` unless a matching publish report proves images for that exact
|
||||
commit.
|
||||
|
||||
Before any formal `dev-cd-apply`, operators must refresh
|
||||
`deploy/artifact-catalog.dev.json`, `deploy/k8s/base/workloads.yaml`, and
|
||||
`deploy/deploy.json`, commit/push that promotion, and verify the desired-state
|
||||
plan points at the intended `sourceCommitId`. If `deploy/deploy.json` still
|
||||
names an older commit, CD will correctly keep deploying that older promotion
|
||||
even when a newer publish report exists. Do not treat the newest CI publish
|
||||
report, or a later bookkeeping commit, as implicit CD desired state.
|
||||
|
||||
If publish is blocked, keep the report blocked rather than inventing digests:
|
||||
|
||||
```sh
|
||||
node scripts/refresh-artifact-catalog.mjs --target-ref origin/main --blocked
|
||||
node scripts/deploy-desired-state-plan.mjs --target-ref origin/main --check
|
||||
```
|
||||
|
||||
The desired-state files are a single deployment truth: `deploy/deploy.json`,
|
||||
`deploy/artifact-catalog.dev.json`, and `deploy/k8s/base/workloads.yaml` must
|
||||
carry the same promotion commit and immutable image tag before they are used as
|
||||
apply input. A `--target-ref` desired-state plan may report a uniform older
|
||||
state as `promotion_pending`; PR review and CI/CD promotion gates must use
|
||||
`--target-ref origin/main --check` or `--promotion-commit <sha> --check` so
|
||||
stale or partially refreshed commit, image, workload, and mirror-env fields are
|
||||
blocked before any DEV apply.
|
||||
Reports under `repository report directory ` are evidence snapshots only and cannot override this
|
||||
desired-state contract.
|
||||
|
||||
## Latest-Main Artifact/Runtime Guard
|
||||
|
||||
Before any controlled DEV apply for latest `origin/main`, run the read-only
|
||||
artifact/runtime guard:
|
||||
|
||||
```sh
|
||||
node scripts/artifact-runtime-readiness-guard.mjs --target-ref origin/main --check --no-report
|
||||
```
|
||||
|
||||
When current readiness is expected to be blocked, use:
|
||||
|
||||
```sh
|
||||
node scripts/artifact-runtime-readiness-guard.mjs --target-ref origin/main --check --no-report --expect-blocked
|
||||
```
|
||||
|
||||
The guard compares target main, `/tmp/hwlab-dev-gate/dev-artifacts.json`,
|
||||
`deploy/artifact-catalog.dev.json`, the desired-state planner, Cloud API
|
||||
`/health/live`, and Cloud Web `:16666/health/live`. It is report-only and must
|
||||
not be used as an apply command. A local source check, static workbench smoke,
|
||||
or stale report snapshot cannot claim latest-main deployment unless the guard's
|
||||
runtime and artifact identity checks also pass.
|
||||
|
||||
## API, Edge, Health, And FRP
|
||||
|
||||
The public route is fixed:
|
||||
|
||||
| Surface | Current entry |
|
||||
| --- | --- |
|
||||
| Cloud Web | `http://74.48.78.17:16666/` |
|
||||
| Cloud Web live health | `http://74.48.78.17:16666/health/live` |
|
||||
| API/edge health | `http://74.48.78.17:16667/health` |
|
||||
| API/live health | `http://74.48.78.17:16667/health/live` |
|
||||
|
||||
FRP maps master public `16666/16667` to D601 `hwlab-dev` services. Use
|
||||
`deploy/deploy.json` as the source for endpoint and FRP drift checks:
|
||||
|
||||
```sh
|
||||
node scripts/deploy-contract-plan.mjs --check
|
||||
```
|
||||
|
||||
Do not reintroduce public `6666/6667` as active DEV endpoints. Internal k3s
|
||||
services may still use internal `6667`.
|
||||
|
||||
## Cloud Web Manual DEV Rollout
|
||||
|
||||
This section is retained as historical operational baseline for what the
|
||||
transaction reproduces. New DEV side effects should use
|
||||
`scripts/dev-cd-apply.mjs`; do not bypass the Lease with ad hoc publish,
|
||||
`kubectl set image`, or restart commands.
|
||||
|
||||
The DEV CD transaction owns the durable runtime support sequence before final
|
||||
live postflight:
|
||||
|
||||
```sh
|
||||
node cmd/hwlab-cloud-api/provision.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
node cmd/hwlab-cloud-api/migrate.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
node scripts/dev-runtime-postflight.mjs --live --confirm-dev --confirmed-non-production --target api --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
These commands are called by `scripts/dev-cd-apply.mjs` under the transaction
|
||||
Lease. Provisioning and migration run from the current `hwlab-cloud-api` image
|
||||
as stable image-internal operator entrypoints; they receive DB input only
|
||||
through SecretRef-backed env and print only redacted structured status. Runners
|
||||
without explicit rollout authorization may run the source-only `--check`
|
||||
variants and prepare PRs/artifacts, but must leave DEV apply, rollout, live
|
||||
verification, and lock ownership to the host commander.
|
||||
|
||||
The previous verified manual path for `hwlab-cloud-web` was:
|
||||
|
||||
```sh
|
||||
cd /home/ubuntu/workspace/hwlab
|
||||
git pull --ff-only origin main
|
||||
node web/hwlab-cloud-web/scripts/build.mjs
|
||||
node scripts/dev-artifact-publish.mjs --publish --services hwlab-cloud-web
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev set image deployment/hwlab-cloud-web hwlab-cloud-web=127.0.0.1:5000/hwlab/hwlab-cloud-web:<tag>
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout status deployment/hwlab-cloud-web --timeout=180s
|
||||
```
|
||||
|
||||
That historical sequence is no longer the normal side-effect path; direct
|
||||
`--publish` is rejected unless it is called by the transaction.
|
||||
|
||||
The rollout must use D601 native k3s:
|
||||
|
||||
```sh
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev get deploy hwlab-cloud-web -o wide
|
||||
```
|
||||
|
||||
The rollout report must tie the live deployment back to artifact identity:
|
||||
source commit, image tag, image reference, registry digest or `not_published`,
|
||||
and Kubernetes rollout revision. `scripts/dev-deploy-apply.mjs` reports this
|
||||
under `devDeployApply.cloudWebRollout` when read-only Deployment access exists;
|
||||
without that access, it reports the blocker and the exact read/rollout commands
|
||||
instead of claiming a live rollout.
|
||||
|
||||
Public verification must check the browser entry on `16666`, not legacy
|
||||
ports:
|
||||
|
||||
```sh
|
||||
curl -fsS http://74.48.78.17:16666/health/live
|
||||
curl -fsS http://74.48.78.17:16666/ | sed -n '1,80p'
|
||||
curl -fsS http://74.48.78.17:16666/styles.css | rg 'overflow: hidden|100dvh'
|
||||
curl -fsS http://74.48.78.17:16666/help.md | sed -n '1,40p'
|
||||
curl -fsS http://74.48.78.17:16667/health/live
|
||||
```
|
||||
|
||||
Cloud Workbench acceptance also expects Chinese default UI, no outer page
|
||||
scroll, and an internal Markdown help surface. The frontend contract is
|
||||
[cloud-workbench.md](cloud-workbench.md).
|
||||
|
||||
## Apply Boundary
|
||||
|
||||
`scripts/dev-deploy-apply.mjs` is the DEV-only apply/report backend. A real
|
||||
apply requires the transaction environment created by `scripts/dev-cd-apply.mjs`
|
||||
plus explicit non-production flags:
|
||||
|
||||
```sh
|
||||
node scripts/dev-deploy-apply.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json
|
||||
```
|
||||
|
||||
Direct manual `--apply` calls without `HWLAB_CD_TRANSACTION_ID` return
|
||||
`cd-transaction-required`. The apply path must stop before mutation when
|
||||
artifact evidence, registry digests, kube access, DEV health, or required
|
||||
runtime readiness are missing. It must not read secret values or target PROD.
|
||||
|
||||
Suspended template Jobs are immutable in `spec.template`. Only the explicit
|
||||
DEV allowlist may be replaced by the apply script:
|
||||
|
||||
- `hwlab-agent-worker-template`
|
||||
- `hwlab-cli-template`
|
||||
|
||||
## Rollback Boundary
|
||||
|
||||
Rollback remains DEV-only and must be explicit. Capture current image state
|
||||
before changing a workload:
|
||||
|
||||
```sh
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev get deploy hwlab-cloud-web -o wide
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout history deployment/hwlab-cloud-web
|
||||
```
|
||||
|
||||
For a Deployment rolled by `set image`, use Kubernetes rollout undo only for
|
||||
the same DEV Deployment:
|
||||
|
||||
```sh
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout undo deployment/hwlab-cloud-web
|
||||
KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout status deployment/hwlab-cloud-web --timeout=180s
|
||||
```
|
||||
|
||||
After rollback, repeat public verification on `16666` and `16667`. Do not use
|
||||
rollback to change PROD, read secrets, restart unrelated services, or conceal
|
||||
an artifact/report mismatch.
|
||||
|
||||
## Controlled Entrypoint Design
|
||||
|
||||
The #116 target is a single controlled deployment surface. Until that lands,
|
||||
use the repo-owned scripts below instead of ad hoc shell fragments.
|
||||
|
||||
| Action | Current controlled entrypoint | Gap to close |
|
||||
| --- | --- | --- |
|
||||
| Check deployment contract and FRP/endpoint drift | `node scripts/deploy-contract-plan.mjs --check` | Fold into `hwlab deploy check --env dev`. |
|
||||
| Plan desired image/workload state | `node scripts/deploy-desired-state-plan.mjs --plan --pretty` and `node scripts/deploy-desired-state-plan.mjs --promotion-commit <sha> --check` | Fold into `hwlab deploy plan --env dev --file deploy/deploy.json` with a promotion commit gate. |
|
||||
| Observe DEV CD status | `node scripts/dev-cd-apply.mjs`, `node scripts/dev-cd-apply.mjs --status`, or `node scripts/dev-cd-apply.mjs --dry-run` | Keep read-only for host and runners; no Lease mutation, artifact publish, apply, rollout, report write, or Secret read. |
|
||||
| Preflight artifact publish | `node scripts/dev-artifact-publish.mjs --preflight` | Keep as backend for `hwlab artifact publish --preflight`. |
|
||||
| Prepare dependency runtime base | `node scripts/dev-runtime-base-image.mjs --dry-run` and `node scripts/dev-runtime-base-image.mjs` | Move into a CI base-image job keyed by dependency hash; CD only consumes the resulting image tag. |
|
||||
| Publish/apply/verify DEV | `node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json` | Keep as the single side-effect transaction and run from repo-owned DEV CI/CD. |
|
||||
| Publish DEV images | `HWLAB_CI_ARTIFACT_RUN_ID=<run-id> node scripts/dev-artifact-publish.mjs --publish` | CI artifact step only; legacy CD transaction env remains transitional. |
|
||||
| Refresh catalog/desired state | `node scripts/refresh-artifact-catalog.mjs --target-ref origin/main --publish-report /tmp/hwlab-dev-gate/dev-artifacts.json` | CI promotion step after publish; report is input evidence for catalog refresh, not CD rollout state. |
|
||||
| Apply DEV rollout | `node scripts/dev-deploy-apply.mjs --apply --confirm-dev --confirmed-non-production --report /tmp/hwlab-dev-gate/report.json` inside `HWLAB_CD_TRANSACTION_ID` | Internal transaction step; direct side-effect calls are rejected. |
|
||||
| Roll back DEV Deployment | `KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl -n hwlab-dev rollout undo deployment/<name>` | Add `hwlab deploy rollback --env dev --service <id> --to-revision <n>`. |
|
||||
| Verify status | `curl` checks plus `node scripts/d601-k3s-readonly-observability.mjs` | Add `hwlab deploy verify --env dev --report /tmp/hwlab-dev-gate/report.json`. |
|
||||
|
||||
## Automation Direction
|
||||
|
||||
The stable direction from
|
||||
[pikasTech/HWLAB#61](https://github.com/pikasTech/HWLAB/issues/61) and
|
||||
[pikasTech/HWLAB#116](https://github.com/pikasTech/HWLAB/issues/116) is to
|
||||
converge manual operations into CLI plus `deploy/deploy.json`, then run that
|
||||
surface from UniDesk CI/CD:
|
||||
|
||||
- render and verify frps/frpc from the deploy manifest;
|
||||
- publish images and write digest reports as CI audit evidence;
|
||||
- refresh catalog/workload desired state from a verified CI publish report;
|
||||
- apply DEV with immutable Job replacement policy;
|
||||
- verify catalog tag/digest directly against registry manifests before apply;
|
||||
- verify `16666` Cloud Web and `16667/health/live` API;
|
||||
- invalidate or regenerate active reports after endpoint or artifact changes.
|
||||
|
||||
UniDesk CI/CD is infrastructure only. It may schedule build/publish/deploy
|
||||
jobs and store artifacts, but it must not replace `hwlab-cloud-api`,
|
||||
`hwlab-cloud-web`, `hwlab-router`, `hwlab-patch-panel`, simulators, gateways,
|
||||
or agent services as the runtime being validated.
|
||||
|
||||
## Stable Sources
|
||||
|
||||
- [docs/dev-artifact-publish.md](../dev-artifact-publish.md): artifact publish
|
||||
command and report contract.
|
||||
- [docs/dev-deploy-apply.md](../dev-deploy-apply.md): apply/report and Cloud
|
||||
Web manual rollout details.
|
||||
- [web/hwlab-cloud-web/scripts/dist-contract.mjs](../../web/hwlab-cloud-web/scripts/dist-contract.mjs): Cloud Web `dist` freshness contract used by build/check/publish.
|
||||
- [pikasTech/HWLAB#61](https://github.com/pikasTech/HWLAB/issues/61): manual
|
||||
rollout review and automation target.
|
||||
- [pikasTech/HWLAB#116](https://github.com/pikasTech/HWLAB/issues/116):
|
||||
deployment regularization stages and acceptance.
|
||||
- [pikasTech/HWLAB#340](https://github.com/pikasTech/HWLAB/issues/340):
|
||||
`deploy.json` DEV CD path long-termization and master CLI automation.
|
||||
@@ -22,7 +22,7 @@ export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
|
||||
kubectl get nodes -o jsonpath='{.items[*].metadata.name}'
|
||||
```
|
||||
|
||||
结果必须包含 `d601`。出现 `docker-desktop`、`desktop-control-plane`、`127.0.0.1:11700` 或第二套 `hwlab-dev` 控制面时，停止 hotfix、回滚和验收判断。此边界与 [dev-runtime-boundary.md](dev-runtime-boundary.md) 和 [deployment-publish.md](deployment-publish.md) 保持一致。
|
||||
结果必须包含 `d601`。出现 `docker-desktop`、`desktop-control-plane`、`127.0.0.1:11700` 或第二套 `hwlab-dev` 控制面时，停止 hotfix、回滚和验收判断。此边界与 [dev-runtime-boundary.md](dev-runtime-boundary.md) 和 [g14-gitops-cicd.md](g14-gitops-cicd.md) 保持一致。
|
||||
|
||||
## kubectl 热修命令形状
|
||||
|
||||
|
||||
@@ -1,307 +0,0 @@
|
||||
{
|
||||
"smokeId": "m2-dev-deploy-smoke",
|
||||
"issue": "pikasTech/HWLAB#23",
|
||||
"environment": "dev",
|
||||
"endpoint": "http://74.48.78.17:16667",
|
||||
"frontendEndpoint": "http://74.48.78.17:16666",
|
||||
"legacyPublicEndpoint": {
|
||||
"endpoint": "http://74.48.78.17:6667",
|
||||
"status": "deprecated",
|
||||
"activeGreenEligible": false
|
||||
},
|
||||
"defaultMode": "dry-run",
|
||||
"safetyGates": {
|
||||
"dryRunOnly": true,
|
||||
"realRequestsRequire": [
|
||||
"explicit --live flag",
|
||||
"confirmed DEV endpoint",
|
||||
"confirmed non-production workspace",
|
||||
"no forbidden actions"
|
||||
],
|
||||
"forbiddenActions": [
|
||||
"real-dev-deploy",
|
||||
"prod-deploy",
|
||||
"heavyweight-e2e",
|
||||
"secret-read",
|
||||
"force-push",
|
||||
"unidesk-runtime-substitution"
|
||||
]
|
||||
},
|
||||
"serviceContracts": [
|
||||
{
|
||||
"routePhase": 1,
|
||||
"phase": "master-edge-proxy",
|
||||
"serviceId": "hwlab-edge-proxy",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-edge-proxy",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:40b2218b7a1ede3d215eaf99c56caf85cd61318da4e4348122e4e2da4c072cd8",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "ok",
|
||||
"route": "public-dev-ingress",
|
||||
"forwardTarget": "frp",
|
||||
"artifactIdentity": "master edge proxy"
|
||||
},
|
||||
"failureClassification": "network_blocker"
|
||||
},
|
||||
{
|
||||
"routePhase": 2,
|
||||
"phase": "frp-tunnel",
|
||||
"serviceId": "hwlab-tunnel-client",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-tunnel-client",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:53bdcb1f16d8a761d836e09e9bb0fe37d3f726b85769be8658d1e2702ff3d448",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "connected",
|
||||
"route": "master-edge-proxy",
|
||||
"forwardTarget": "D601 hwlab-dev/hwlab-router",
|
||||
"artifactIdentity": "frp tunnel"
|
||||
},
|
||||
"failureClassification": "network_blocker"
|
||||
},
|
||||
{
|
||||
"routePhase": 3,
|
||||
"phase": "d601-router",
|
||||
"serviceId": "hwlab-router",
|
||||
"namespace": "hwlab-dev",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-router",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:be7e87979bbe78fc040d52d64b264bb8494f873c14b455dbf14681e2b4fd3dfb",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "live",
|
||||
"route": "hwlab-dev/hwlab-router",
|
||||
"forwardTarget": "cloud-api/web",
|
||||
"artifactIdentity": "D601 router"
|
||||
},
|
||||
"failureClassification": "runtime_blocker"
|
||||
},
|
||||
{
|
||||
"routePhase": 4,
|
||||
"phase": "cloud-surface",
|
||||
"serviceId": "hwlab-cloud-api",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-cloud-api",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:a147c7dbb6df1d30b0ac5e747f2f1fc42d455a55410b82578573b9b3ae2d6074",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "ok",
|
||||
"live": true,
|
||||
"route": "cloud-api/web",
|
||||
"db": {
|
||||
"fixtureOnly": true,
|
||||
"liveDbEvidence": false,
|
||||
"requiredEnv": [
|
||||
{
|
||||
"name": "HWLAB_CLOUD_DB_URL",
|
||||
"present": false,
|
||||
"redacted": true
|
||||
},
|
||||
{
|
||||
"name": "HWLAB_CLOUD_DB_SSL_MODE",
|
||||
"present": false,
|
||||
"redacted": false
|
||||
}
|
||||
],
|
||||
"missingEnv": [
|
||||
"HWLAB_CLOUD_DB_URL",
|
||||
"HWLAB_CLOUD_DB_SSL_MODE"
|
||||
],
|
||||
"summary": "Fixture records the DB env contract shape only; it is not live DB evidence."
|
||||
},
|
||||
"artifacts": [
|
||||
{
|
||||
"serviceId": "hwlab-cloud-api",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-cloud-api",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:a147c7dbb6df1d30b0ac5e747f2f1fc42d455a55410b82578573b9b3ae2d6074"
|
||||
,
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z"
|
||||
},
|
||||
{
|
||||
"serviceId": "hwlab-cloud-web",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-cloud-web",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:4a546582e6318b77530b9141dfd764c36b974442d6c7916a1294c54d0b02a14e",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z"
|
||||
}
|
||||
]
|
||||
},
|
||||
"failureClassification": "runtime_blocker"
|
||||
},
|
||||
{
|
||||
"routePhase": 5,
|
||||
"phase": "agent-runtime",
|
||||
"serviceId": "hwlab-agent-mgr",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-agent-mgr",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:cf2dab7d0aa49ef6e77dd8e1bacebd588bf2dbd65e86a4b8c2dc23cd248c026b",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "live",
|
||||
"trace": "session-scoped",
|
||||
"cleanup": "no worker leak"
|
||||
},
|
||||
"failureClassification": "agent_blocker"
|
||||
},
|
||||
{
|
||||
"routePhase": 6,
|
||||
"phase": "agent-runtime",
|
||||
"serviceId": "hwlab-agent-worker",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-agent-worker",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:be2941f18376e8e39e0aa27857bb2db4300e861f2c759a7bc447e96e1f0e8bf1",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "scoped",
|
||||
"projectId": "proj_mvp-l6",
|
||||
"trace": "session-scoped",
|
||||
"cleanup": "worker session closed"
|
||||
},
|
||||
"failureClassification": "agent_blocker"
|
||||
},
|
||||
{
|
||||
"routePhase": 7,
|
||||
"phase": "agent-runtime",
|
||||
"serviceId": "hwlab-agent-skills",
|
||||
"commitId": "5583e4b",
|
||||
"image": "skills/hwlab-agent-runtime",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "not_applicable",
|
||||
"digestReason": "local dry-run fixture for repo-local skills bundle",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "traceable",
|
||||
"bundle": "repo-local skills artifact",
|
||||
"cleanup": "explicit commitId required"
|
||||
},
|
||||
"failureClassification": "agent_blocker"
|
||||
},
|
||||
{
|
||||
"routePhase": 8,
|
||||
"phase": "sim-and-patch-panel",
|
||||
"serviceId": "hwlab-gateway",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-gateway",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:dcf57c4a0fab873ab571c1f6314d352d38853ff2cf125356a05a06ae5cfdb444",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "live",
|
||||
"route": "hardware boundary",
|
||||
"artifactIdentity": "gateway boundary"
|
||||
},
|
||||
"failureClassification": "runtime_blocker"
|
||||
},
|
||||
{
|
||||
"routePhase": 9,
|
||||
"phase": "sim-and-patch-panel",
|
||||
"serviceId": "hwlab-gateway-simu",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-gateway-simu",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:0bc140f52733255c937d08cea4d51639c83a71df8ac7a78bb107f60401bc392e",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "live",
|
||||
"route": "simulator",
|
||||
"artifactIdentity": "gateway simulator",
|
||||
"observedAt": "2026-05-21T00:00:00.000Z"
|
||||
},
|
||||
"failureClassification": "runtime_blocker"
|
||||
},
|
||||
{
|
||||
"routePhase": 10,
|
||||
"phase": "sim-and-patch-panel",
|
||||
"serviceId": "hwlab-box-simu",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-box-simu",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:0506757987fb7955ec0089fd9c0d895c33eb714a940fc7b8d58153368f96e8c8",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "live",
|
||||
"route": "patch-panel-owned",
|
||||
"artifactIdentity": "box simulator"
|
||||
},
|
||||
"failureClassification": "runtime_blocker"
|
||||
},
|
||||
{
|
||||
"routePhase": 11,
|
||||
"phase": "sim-and-patch-panel",
|
||||
"serviceId": "hwlab-patch-panel",
|
||||
"commitId": "5583e4b",
|
||||
"image": "ghcr.io/pikastech/hwlab-patch-panel",
|
||||
"tag": "m2-dev-deploy-smoke-5583e4b",
|
||||
"digest": "sha256:bd3045c3235353e2437fbb11308ffe6f1ea3e52ca1e481dc80d4317a4b1600d1",
|
||||
"buildSource": "git+https://github.com/pikasTech/HWLAB.git#origin/main:scripts/m2-dev-deploy-smoke.mjs",
|
||||
"deployEnv": "dev",
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z",
|
||||
"health": {
|
||||
"status": "active",
|
||||
"route": "patch-panel-owned",
|
||||
"artifactIdentity": "patch panel",
|
||||
"observedAt": "2026-05-21T00:00:00.000Z"
|
||||
},
|
||||
"failureClassification": "runtime_blocker"
|
||||
}
|
||||
],
|
||||
"observations": {
|
||||
"serviceIds": [
|
||||
"hwlab-edge-proxy",
|
||||
"hwlab-tunnel-client",
|
||||
"hwlab-router",
|
||||
"hwlab-cloud-api",
|
||||
"hwlab-cloud-web",
|
||||
"hwlab-agent-mgr",
|
||||
"hwlab-agent-worker",
|
||||
"hwlab-agent-skills",
|
||||
"hwlab-gateway",
|
||||
"hwlab-gateway-simu",
|
||||
"hwlab-box-simu",
|
||||
"hwlab-patch-panel"
|
||||
],
|
||||
"routePhases": [
|
||||
"master-edge-proxy",
|
||||
"frp-tunnel",
|
||||
"d601-router",
|
||||
"cloud-surface",
|
||||
"agent-runtime",
|
||||
"sim-and-patch-panel"
|
||||
],
|
||||
"healthTimestamp": "2026-05-21T00:00:00.000Z"
|
||||
}
|
||||
}
|
||||
+1
-1
File diff suppressed because one or more lines are too long
@@ -793,7 +793,7 @@ function dockerfile(baseImage, port) {
|
||||
"ENV HWLAB_CODE_AGENT_SKILLS_DIRS=/app/skills:/root/.agents/skills:/home/ubuntu/.agents/skills",
|
||||
"COPY package.json ./package.json",
|
||||
"COPY package-lock.json* ./",
|
||||
"RUN if [ -d /opt/hwlab-node-runtime-base/node_modules ]; then cp -a /opt/hwlab-node-runtime-base/node_modules ./node_modules; elif [ -f package-lock.json ]; then npm ci --omit=dev --ignore-scripts; else npm install --omit=dev --ignore-scripts; fi",
|
||||
"RUN if [ -d /opt/hwlab-node-runtime-base/node_modules ]; then cp -a /opt/hwlab-node-runtime-base/node_modules ./node_modules; elif [ -f package-lock.json ]; then npm ci --omit=dev --include=optional --ignore-scripts; else npm install --omit=dev --include=optional --ignore-scripts; fi && codex_version=\"$(node -p \"require('./node_modules/@openai/codex/package.json').version\")\" && case \"$(uname -m)\" in x86_64|amd64) test -e node_modules/@openai/codex-linux-x64 || npm install --omit=dev --include=optional --ignore-scripts \"@openai/codex-linux-x64@npm:@openai/codex@${codex_version}-linux-x64\" ;; aarch64|arm64) test -e node_modules/@openai/codex-linux-arm64 || npm install --omit=dev --include=optional --ignore-scripts \"@openai/codex-linux-arm64@npm:@openai/codex@${codex_version}-linux-arm64\" ;; esac",
|
||||
"COPY internal ./internal",
|
||||
"COPY cmd ./cmd",
|
||||
"COPY scripts ./scripts",
|
||||
@@ -1331,7 +1331,7 @@ function createReport({ args, repo, commitId, shortCommit, mode, services, artif
|
||||
"docs/dev-acceptance-matrix.md",
|
||||
"docs/m0-contract-audit.md",
|
||||
"docs/artifact-catalog.md",
|
||||
"docs/dev-artifact-publish.md",
|
||||
"docs/reference/g14-gitops-cicd.md",
|
||||
"docs/dev-base-image-preflight.md"
|
||||
],
|
||||
summary: "DEV artifact publish evidence for pikasTech/HWLAB#35, stored in the DEV gate report envelope."
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
import { formatDevCdApplyFailure, runDevCdApply } from "./src/dev-cd-apply.mjs";
|
||||
|
||||
try {
|
||||
process.exitCode = await runDevCdApply(process.argv.slice(2));
|
||||
} catch (error) {
|
||||
process.stdout.write(`${JSON.stringify(formatDevCdApplyFailure(error), null, 2)}\n`);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
@@ -1,9 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
import { formatFailure, runDevDeployApply } from "./src/dev-deploy-apply.mjs";
|
||||
|
||||
try {
|
||||
process.exitCode = await runDevDeployApply(process.argv.slice(2));
|
||||
} catch (error) {
|
||||
process.stdout.write(`${JSON.stringify(formatFailure(error), null, 2)}\n`);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
@@ -1,475 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
import assert from "node:assert/strict";
|
||||
import { execFile } from "node:child_process";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { promisify } from "node:util";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
import { activeReportLifecycle } from "../internal/dev-report-lifecycle.mjs";
|
||||
import {
|
||||
DEV_ENDPOINT,
|
||||
DEV_FRONTEND_ENDPOINT,
|
||||
ENVIRONMENT_DEV,
|
||||
SERVICE_IDS
|
||||
} from "../internal/protocol/index.mjs";
|
||||
import { ensureNotRepoReportsPath, tempReportPath } from "./src/report-paths.mjs";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const fixturePath = path.join(repoRoot, "fixtures/dev-deploy-smoke/dev-deploy-smoke.json");
|
||||
const defaultActiveReportPath = tempReportPath("dev-m2-deploy-smoke-active.json");
|
||||
const legacyPublicEndpoint = "http://74.48.78.17:6667";
|
||||
const timestampPattern = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/;
|
||||
const digestPattern = /^(sha256:[a-f0-9]{64}|not_applicable)$/;
|
||||
const tagPattern = /^m2-dev-deploy-smoke-[a-f0-9]{7}$/;
|
||||
const allowedFailureClassifications = new Set([
|
||||
"network_blocker",
|
||||
"runtime_blocker",
|
||||
"agent_blocker",
|
||||
"observability_blocker",
|
||||
"safety_blocker"
|
||||
]);
|
||||
|
||||
function parseArgs(argv) {
|
||||
const flags = new Set();
|
||||
const options = {
|
||||
reportPath: defaultActiveReportPath,
|
||||
writeReport: false,
|
||||
timeoutMs: 5000
|
||||
};
|
||||
const rest = [];
|
||||
for (let index = 0; index < argv.length; index += 1) {
|
||||
const arg = argv[index];
|
||||
if (arg === "--report") {
|
||||
flags.add(arg);
|
||||
index += 1;
|
||||
assert.ok(argv[index], "--report requires a path");
|
||||
options.reportPath = argv[index];
|
||||
options.writeReport = true;
|
||||
} else if (arg === "--timeout-ms") {
|
||||
flags.add(arg);
|
||||
index += 1;
|
||||
options.timeoutMs = Number.parseInt(argv[index], 10);
|
||||
assert.ok(Number.isInteger(options.timeoutMs) && options.timeoutMs > 0, "--timeout-ms must be positive");
|
||||
} else if (arg.startsWith("--")) {
|
||||
flags.add(arg);
|
||||
} else {
|
||||
rest.push(arg);
|
||||
}
|
||||
}
|
||||
return { flags, options, rest };
|
||||
}
|
||||
|
||||
function loadFixture() {
|
||||
return readFile(fixturePath, "utf8").then((raw) => JSON.parse(raw));
|
||||
}
|
||||
|
||||
function assertTimestamp(value, context) {
|
||||
assert.equal(typeof value, "string", `${context} must be a string`);
|
||||
assert.match(value, timestampPattern, `${context} must be an RFC 3339 UTC timestamp`);
|
||||
}
|
||||
|
||||
function assertServiceContract(entry) {
|
||||
assert.ok(SERVICE_IDS.includes(entry.serviceId), `unknown serviceId ${entry.serviceId}`);
|
||||
assert.equal(entry.deployEnv, ENVIRONMENT_DEV, `${entry.serviceId} deployEnv must be dev`);
|
||||
assert.ok(!Object.hasOwn(entry, "endpoint"), `${entry.serviceId} service contract must not carry endpoint`);
|
||||
assert.equal(typeof entry.commitId, "string", `${entry.serviceId} commitId must be a string`);
|
||||
assert.match(entry.commitId, /^[a-f0-9]{7}$/u, `${entry.serviceId} commitId must be a short sha`);
|
||||
assert.equal(typeof entry.image, "string", `${entry.serviceId} image must be a string`);
|
||||
assert.ok(entry.image.length > 0, `${entry.serviceId} image must not be empty`);
|
||||
assert.equal(typeof entry.tag, "string", `${entry.serviceId} tag must be a string`);
|
||||
assert.match(entry.tag, tagPattern, `${entry.serviceId} tag must encode the smoke commit`);
|
||||
assert.equal(typeof entry.digest, "string", `${entry.serviceId} digest must be a string`);
|
||||
assert.match(entry.digest, digestPattern, `${entry.serviceId} digest must be sha256 or not_applicable`);
|
||||
if (entry.digest === "not_applicable") {
|
||||
assert.equal(typeof entry.digestReason, "string", `${entry.serviceId} digestReason required when digest is not_applicable`);
|
||||
assert.ok(entry.digestReason.length > 0, `${entry.serviceId} digestReason must not be empty`);
|
||||
} else {
|
||||
assert.equal(entry.digestReason, undefined, `${entry.serviceId} digestReason must be absent when digest is present`);
|
||||
}
|
||||
assert.equal(typeof entry.buildSource, "string", `${entry.serviceId} buildSource must be a string`);
|
||||
assert.ok(entry.buildSource.includes("origin/main"), `${entry.serviceId} buildSource must point at origin/main`);
|
||||
assertTimestamp(entry.healthTimestamp, `${entry.serviceId} healthTimestamp`);
|
||||
assert.equal(typeof entry.routePhase, "number", `${entry.serviceId} routePhase must be numeric`);
|
||||
assert.ok(entry.routePhase > 0, `${entry.serviceId} routePhase must be positive`);
|
||||
assert.ok(entry.phase.length > 0, `${entry.serviceId} phase must not be empty`);
|
||||
assert.equal(typeof entry.health, "object", `${entry.serviceId} health must be an object`);
|
||||
assert.ok(entry.health && !Array.isArray(entry.health), `${entry.serviceId} health must be a plain object`);
|
||||
assert.ok(typeof entry.health.status === "string", `${entry.serviceId} health.status must be a string`);
|
||||
assert.equal(typeof entry.failureClassification, "string", `${entry.serviceId} failureClassification must be a string`);
|
||||
assert.ok(entry.failureClassification.endsWith("_blocker"), `${entry.serviceId} failureClassification must end in _blocker`);
|
||||
assert.ok(
|
||||
allowedFailureClassifications.has(entry.failureClassification),
|
||||
`${entry.serviceId} failureClassification must be one of the frozen blocker classes`
|
||||
);
|
||||
}
|
||||
|
||||
function assertRoutePhaseOrder(entries) {
|
||||
const phases = entries.map((entry) => entry.routePhase);
|
||||
assert.deepEqual(phases, [...phases].sort((a, b) => a - b), "route phases must be sorted");
|
||||
assert.equal(new Set(phases).size, phases.length, "route phases must be unique");
|
||||
}
|
||||
|
||||
function assertAllServiceIds(entries) {
|
||||
const serviceIds = entries.map((entry) => entry.serviceId);
|
||||
assert.equal(new Set(serviceIds).size, serviceIds.length, "service ids must be unique in fixture");
|
||||
for (const serviceId of serviceIds) {
|
||||
assert.ok(SERVICE_IDS.includes(serviceId), `fixture uses unknown serviceId ${serviceId}`);
|
||||
}
|
||||
}
|
||||
|
||||
function assertCloudSurface(entry) {
|
||||
const healthArtifacts = entry.health.artifacts ?? [];
|
||||
assert.equal(entry.serviceId, "hwlab-cloud-api", "cloud surface entry must start at cloud-api");
|
||||
assert.equal(entry.health.db?.fixtureOnly, true, "cloud surface DB note must be fixture-only");
|
||||
assert.equal(entry.health.db?.liveDbEvidence, false, "cloud surface fixture must not claim live DB evidence");
|
||||
assert.deepEqual(
|
||||
entry.health.db?.missingEnv,
|
||||
["HWLAB_CLOUD_DB_URL", "HWLAB_CLOUD_DB_SSL_MODE"],
|
||||
"cloud surface fixture must name missing DB env without values"
|
||||
);
|
||||
assert.equal(healthArtifacts.length, 2, "cloud surface must include cloud-api and cloud-web artifacts");
|
||||
assert.deepEqual(
|
||||
healthArtifacts.map((artifact) => artifact.serviceId),
|
||||
["hwlab-cloud-api", "hwlab-cloud-web"],
|
||||
"cloud surface artifacts must list api/web"
|
||||
);
|
||||
for (const artifact of healthArtifacts) {
|
||||
assert.ok(SERVICE_IDS.includes(artifact.serviceId), `cloud surface artifact ${artifact.serviceId} must be known`);
|
||||
assert.equal(typeof artifact.image, "string", `cloud surface artifact ${artifact.serviceId} image must exist`);
|
||||
assert.equal(typeof artifact.tag, "string", `cloud surface artifact ${artifact.serviceId} tag must exist`);
|
||||
assert.match(artifact.tag, tagPattern, `cloud surface artifact ${artifact.serviceId} tag must encode the smoke commit`);
|
||||
assert.match(artifact.digest, /^sha256:[a-f0-9]{64}$/u, `cloud surface artifact ${artifact.serviceId} digest must be sha256`);
|
||||
assert.equal(typeof artifact.commitId, "string", `cloud surface artifact ${artifact.serviceId} commitId must exist`);
|
||||
assert.equal(typeof artifact.buildSource, "string", `cloud surface artifact ${artifact.serviceId} buildSource must exist`);
|
||||
assert.equal(artifact.deployEnv, ENVIRONMENT_DEV, `cloud surface artifact ${artifact.serviceId} deployEnv must be dev`);
|
||||
assertTimestamp(artifact.healthTimestamp ?? entry.healthTimestamp, `cloud surface artifact ${artifact.serviceId} healthTimestamp`);
|
||||
}
|
||||
}
|
||||
|
||||
function assertFixture(fixture) {
|
||||
assert.equal(fixture.smokeId, "m2-dev-deploy-smoke");
|
||||
assert.equal(fixture.issue, "pikasTech/HWLAB#23");
|
||||
assert.equal(fixture.environment, ENVIRONMENT_DEV);
|
||||
assert.equal(fixture.endpoint, DEV_ENDPOINT);
|
||||
assert.equal(fixture.frontendEndpoint, DEV_FRONTEND_ENDPOINT);
|
||||
assert.equal(fixture.legacyPublicEndpoint?.endpoint, legacyPublicEndpoint);
|
||||
assert.equal(fixture.legacyPublicEndpoint?.activeGreenEligible, false);
|
||||
assert.equal(fixture.defaultMode, "dry-run");
|
||||
assert.equal(fixture.safetyGates.dryRunOnly, true, "dry-run must be the default");
|
||||
assert.ok(Array.isArray(fixture.safetyGates.realRequestsRequire), "real request gates must exist");
|
||||
assert.ok(fixture.safetyGates.realRequestsRequire.includes("explicit --live flag"));
|
||||
|
||||
const entries = fixture.serviceContracts;
|
||||
assert.ok(Array.isArray(entries) && entries.length >= 8, "serviceContracts must contain route observations");
|
||||
assertAllServiceIds(entries);
|
||||
assertRoutePhaseOrder(entries);
|
||||
|
||||
for (const entry of entries) {
|
||||
assertServiceContract(entry);
|
||||
}
|
||||
|
||||
const cloudSurface = entries.find((entry) => entry.phase === "cloud-surface");
|
||||
assert.ok(cloudSurface, "cloud-surface entry must exist");
|
||||
assertCloudSurface(cloudSurface);
|
||||
|
||||
const agentSkills = entries.find((entry) => entry.serviceId === "hwlab-agent-skills");
|
||||
assert.equal(agentSkills.digest, "not_applicable");
|
||||
assert.equal(agentSkills.digestReason, "local dry-run fixture for repo-local skills bundle");
|
||||
|
||||
const observations = fixture.observations;
|
||||
const expectedObservationIds = [];
|
||||
for (const entry of entries) {
|
||||
expectedObservationIds.push(entry.serviceId);
|
||||
if (entry.serviceId === "hwlab-cloud-api") {
|
||||
expectedObservationIds.push("hwlab-cloud-web");
|
||||
}
|
||||
}
|
||||
assert.deepEqual(observations.serviceIds, expectedObservationIds, "observation service ids must track validated entries");
|
||||
assert.deepEqual(
|
||||
observations.routePhases,
|
||||
["master-edge-proxy", "frp-tunnel", "d601-router", "cloud-surface", "agent-runtime", "sim-and-patch-panel"],
|
||||
"route phase labels must stay frozen"
|
||||
);
|
||||
assertTimestamp(observations.healthTimestamp, "observations.healthTimestamp");
|
||||
}
|
||||
|
||||
function redactedHeaders(headers) {
|
||||
return Object.fromEntries(
|
||||
["content-type", "content-length", "cache-control", "date"].flatMap((name) => {
|
||||
const value = headers.get(name);
|
||||
return value ? [[name, value]] : [];
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
async function probeHttp({ id, url, expect }) {
|
||||
const startedAt = Date.now();
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), expect.timeoutMs);
|
||||
try {
|
||||
const response = await fetch(url, { signal: controller.signal });
|
||||
const body = await response.text();
|
||||
const contentType = response.headers.get("content-type") ?? "";
|
||||
let json = null;
|
||||
if (contentType.includes("application/json")) {
|
||||
try {
|
||||
json = JSON.parse(body);
|
||||
} catch {
|
||||
json = null;
|
||||
}
|
||||
}
|
||||
return {
|
||||
id,
|
||||
url,
|
||||
ok: response.ok,
|
||||
status: response.status,
|
||||
statusText: response.statusText,
|
||||
durationMs: Date.now() - startedAt,
|
||||
contentType,
|
||||
headers: redactedHeaders(response.headers),
|
||||
json,
|
||||
title: body.match(/<title>([^<]+)<\/title>/iu)?.[1] ?? null,
|
||||
bodyPreview: body.slice(0, 512)
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
id,
|
||||
url,
|
||||
ok: false,
|
||||
status: null,
|
||||
statusText: null,
|
||||
durationMs: Date.now() - startedAt,
|
||||
contentType: null,
|
||||
headers: {},
|
||||
json: null,
|
||||
title: null,
|
||||
bodyPreview: "",
|
||||
error: error?.name === "AbortError" ? `timeout after ${expect.timeoutMs}ms` : String(error?.message ?? error)
|
||||
};
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
function acceptedProbe(probe) {
|
||||
if (!probe.ok) return false;
|
||||
if (probe.id === "frontend-root") {
|
||||
return probe.contentType?.includes("text/html") && probe.title === "HWLAB DEV MVP Gate";
|
||||
}
|
||||
return Boolean(
|
||||
probe.json &&
|
||||
SERVICE_IDS.includes(probe.json.serviceId) &&
|
||||
probe.json.environment === ENVIRONMENT_DEV &&
|
||||
(probe.json.endpoint === undefined || probe.json.endpoint === DEV_ENDPOINT)
|
||||
);
|
||||
}
|
||||
|
||||
function probeEvidenceLine(probe) {
|
||||
const accepted = acceptedProbe(probe) ? "yes" : "no";
|
||||
const identity = probe.json?.serviceId ?? probe.title ?? "unknown";
|
||||
return `${probe.id}: HTTP ${probe.status ?? "none"} accepted=${accepted} identity=${identity} url=${probe.url}`;
|
||||
}
|
||||
|
||||
async function gitCommitId() {
|
||||
try {
|
||||
const result = await execFileAsync("git", ["rev-parse", "--short=12", "HEAD"], {
|
||||
cwd: repoRoot,
|
||||
timeout: 5000
|
||||
});
|
||||
return result.stdout.trim();
|
||||
} catch {
|
||||
return "unknown";
|
||||
}
|
||||
}
|
||||
|
||||
function buildActiveReport({ fixture, probes, generatedAt, commitId }) {
|
||||
const passed = probes.every(acceptedProbe);
|
||||
const status = passed ? "pass" : "blocked";
|
||||
const commands = [
|
||||
"curl -fsS --max-time 10 http://74.48.78.17:16667/health",
|
||||
"curl -fsS --max-time 10 http://74.48.78.17:16667/health/live",
|
||||
"curl -fsS --max-time 10 http://74.48.78.17:16666/"
|
||||
];
|
||||
|
||||
return {
|
||||
$schema: "https://hwlab.pikastech.local/schemas/dev-gate-report.schema.json",
|
||||
$id: "https://hwlab.pikastech.local/dev-gate/dev-m2-deploy-smoke-active.json",
|
||||
reportVersion: "v1",
|
||||
issue: "pikasTech/HWLAB#23",
|
||||
taskId: "m2-dev-deploy-smoke",
|
||||
commitId,
|
||||
acceptanceLevel: "dev_m2_deploy_smoke",
|
||||
devOnly: true,
|
||||
prodDisabled: true,
|
||||
reportLifecycle: activeReportLifecycle(
|
||||
"Active M2 read-only public endpoint smoke; route reachability is not M5 DEV-LIVE acceptance."
|
||||
),
|
||||
status,
|
||||
generatedAt,
|
||||
endpoint: DEV_ENDPOINT,
|
||||
frontendEndpoint: DEV_FRONTEND_ENDPOINT,
|
||||
sourceContract: {
|
||||
status: "pass",
|
||||
documents: [
|
||||
"docs/m2-dev-deploy-smoke.md",
|
||||
"docs/dev-acceptance-matrix.md",
|
||||
"fixtures/dev-deploy-smoke/dev-deploy-smoke.json"
|
||||
],
|
||||
summary: "M2 DEV deploy smoke is rebaselined to frontend :16666 and API/edge :16667."
|
||||
},
|
||||
validationCommands: [
|
||||
"node --check scripts/m2-dev-deploy-smoke.mjs",
|
||||
"node scripts/m2-dev-deploy-smoke.mjs --dry-run",
|
||||
`node scripts/m2-dev-deploy-smoke.mjs --live --confirm-dev --confirmed-non-production --report ${defaultActiveReportPath}`,
|
||||
"node --check scripts/validate-dev-gate-report.mjs",
|
||||
"node scripts/validate-dev-gate-report.mjs"
|
||||
],
|
||||
localSmoke: {
|
||||
status: "pass",
|
||||
commands: ["node scripts/m2-dev-deploy-smoke.mjs --dry-run"],
|
||||
evidence: [
|
||||
`[m2-smoke] validated ${fixture.serviceContracts.length} service contracts`,
|
||||
`[m2-smoke] mode=dry-run endpoint=${DEV_ENDPOINT}`,
|
||||
"[m2-smoke] no real DEV/PROD request was made"
|
||||
],
|
||||
summary: "The fixture smoke validates the M2 route and artifact contract without contacting DEV."
|
||||
},
|
||||
dryRun: {
|
||||
status: "pass",
|
||||
commands: ["node scripts/m2-dev-deploy-smoke.mjs --dry-run"],
|
||||
evidence: [
|
||||
`fixture endpoint=${DEV_ENDPOINT}`,
|
||||
`fixture frontendEndpoint=${DEV_FRONTEND_ENDPOINT}`,
|
||||
"legacy :6667 public evidence is marked ineligible for active green"
|
||||
],
|
||||
summary: "Dry-run evidence is contract-only and pinned to the current DEV public endpoints."
|
||||
},
|
||||
devPreconditions: {
|
||||
status,
|
||||
requirements: [
|
||||
"GET http://74.48.78.17:16667/health returns HWLAB DEV JSON",
|
||||
"GET http://74.48.78.17:16667/health/live returns HWLAB DEV JSON",
|
||||
"GET http://74.48.78.17:16666/ returns the HWLAB DEV frontend",
|
||||
"No PROD, secret read, runtime restart, deployment, or heavyweight e2e action is performed"
|
||||
],
|
||||
commands,
|
||||
evidence: probes.map(probeEvidenceLine),
|
||||
summary: passed
|
||||
? "Read-only DEV public endpoint probes passed on :16666/:16667."
|
||||
: "One or more read-only DEV public endpoint probes failed; no active green is claimed."
|
||||
},
|
||||
blockers: passed
|
||||
? []
|
||||
: [
|
||||
{
|
||||
type: "network_blocker",
|
||||
scope: "m2-public-endpoints",
|
||||
status: "open",
|
||||
summary: "One or more M2 DEV public endpoint probes failed on :16666/:16667."
|
||||
}
|
||||
],
|
||||
runtimeSmoke: {
|
||||
mode: "live-read-only",
|
||||
status,
|
||||
generatedAt,
|
||||
apiEndpoint: DEV_ENDPOINT,
|
||||
frontendEndpoint: DEV_FRONTEND_ENDPOINT,
|
||||
safety: {
|
||||
environment: ENVIRONMENT_DEV,
|
||||
prodTouched: false,
|
||||
secretsRead: false,
|
||||
restarts: false,
|
||||
deployAttempted: false,
|
||||
heavyE2E: false,
|
||||
unideskRuntimeSubstitute: false
|
||||
},
|
||||
probes,
|
||||
legacyPublicEndpoint: {
|
||||
endpoint: legacyPublicEndpoint,
|
||||
status: "deprecated",
|
||||
activeGreenEligible: false,
|
||||
summary: "Legacy public :6667 observations are retained only as historical blockers and are not accepted as active green evidence."
|
||||
}
|
||||
},
|
||||
notes: "M2 active report uses read-only GET probes only; PROD, service restarts, deployments, and script-driven apply flows were not touched."
|
||||
};
|
||||
}
|
||||
|
||||
async function writeActiveReport(report, reportPath) {
|
||||
const absoluteReportPath = ensureNotRepoReportsPath(repoRoot, reportPath, "--report");
|
||||
await mkdir(path.dirname(absoluteReportPath), { recursive: true });
|
||||
await writeFile(absoluteReportPath, `${JSON.stringify(report, null, 2)}\n`);
|
||||
}
|
||||
|
||||
async function runLiveSmoke(fixture, options) {
|
||||
const probes = await Promise.all([
|
||||
probeHttp({
|
||||
id: "api-health",
|
||||
url: new URL("/health", DEV_ENDPOINT).href,
|
||||
expect: { timeoutMs: options.timeoutMs }
|
||||
}),
|
||||
probeHttp({
|
||||
id: "api-live",
|
||||
url: new URL("/health/live", DEV_ENDPOINT).href,
|
||||
expect: { timeoutMs: options.timeoutMs }
|
||||
}),
|
||||
probeHttp({
|
||||
id: "frontend-root",
|
||||
url: new URL("/", DEV_FRONTEND_ENDPOINT).href,
|
||||
expect: { timeoutMs: options.timeoutMs }
|
||||
})
|
||||
]);
|
||||
|
||||
const report = buildActiveReport({
|
||||
fixture,
|
||||
probes,
|
||||
generatedAt: new Date().toISOString(),
|
||||
commitId: await gitCommitId()
|
||||
});
|
||||
return report;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const { flags, options } = parseArgs(process.argv.slice(2));
|
||||
const isLive = flags.has("--live") || flags.has("--allow-live");
|
||||
|
||||
if (isLive) {
|
||||
assert.ok(!flags.has("--dry-run"), "do not combine --live with --dry-run");
|
||||
assert.ok(flags.has("--confirm-dev"), "live mode requires --confirm-dev");
|
||||
assert.ok(flags.has("--confirmed-non-production"), "live mode requires --confirmed-non-production");
|
||||
assert.ok(!flags.has("--prod"), "PROD requests are forbidden");
|
||||
assert.ok(!flags.has("--heavyweight-e2e"), "heavyweight e2e is forbidden");
|
||||
assert.ok(!flags.has("--read-secret"), "secret reads are forbidden");
|
||||
assert.ok(!flags.has("--force-push"), "force push is forbidden");
|
||||
}
|
||||
|
||||
const fixture = await loadFixture();
|
||||
assertFixture(fixture);
|
||||
assert.equal(fixture.endpoint, DEV_ENDPOINT, "fixture must stay pinned to the frozen DEV endpoint");
|
||||
|
||||
process.stdout.write(`[m2-smoke] validated ${fixture.serviceContracts.length} service contracts\n`);
|
||||
process.stdout.write(`[m2-smoke] mode=${isLive ? "live" : "dry-run"} endpoint=${fixture.endpoint}\n`);
|
||||
if (isLive) {
|
||||
const report = await runLiveSmoke(fixture, options);
|
||||
if (options.writeReport) {
|
||||
await writeActiveReport(report, options.reportPath);
|
||||
process.stdout.write(`[m2-smoke] wrote ${path.relative(repoRoot, options.reportPath)}\n`);
|
||||
}
|
||||
process.stdout.write(`[m2-smoke] active-report=${report.status} probes=${report.runtimeSmoke.probes.length}\n`);
|
||||
process.stdout.write("[m2-smoke] no DEV/PROD mutation was made\n");
|
||||
if (report.status !== "pass") {
|
||||
process.exitCode = 2;
|
||||
}
|
||||
} else {
|
||||
process.stdout.write("[m2-smoke] no real DEV/PROD request was made\n");
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
await main();
|
||||
} catch (error) {
|
||||
process.stderr.write(`[m2-smoke] ${error instanceof Error ? error.message : String(error)}\n`);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
@@ -31,7 +31,7 @@ test("refresh accepts an absolute publish report path", async () => {
|
||||
const reportPath = path.join(tempDir, "dev-artifacts.json");
|
||||
await writeFile(reportPath, `${JSON.stringify({
|
||||
reportVersion: "v1",
|
||||
taskId: "dev-artifact-publish",
|
||||
taskId: "g14-artifact-publish",
|
||||
commitId: shortCommitId,
|
||||
artifactPublish: {
|
||||
status: "published",
|
||||
|
||||
@@ -260,9 +260,7 @@ export function buildArtifactRuntimeReadinessReport({
|
||||
sourceContract: {
|
||||
status: "pass",
|
||||
documents: [
|
||||
"docs/dev-artifact-publish.md",
|
||||
"docs/dev-deploy-apply.md",
|
||||
"docs/reference/deployment-publish.md"
|
||||
"docs/reference/g14-gitops-cicd.md"
|
||||
],
|
||||
summary: "Target commit, artifact publish/catalog identity, desired-state convergence, API runtime identity, and Cloud Web served runtime identity are separate gates."
|
||||
},
|
||||
@@ -1172,7 +1170,7 @@ function nextCommands({ canPublish, canApply, blockedReasons }) {
|
||||
return [
|
||||
"node web/hwlab-cloud-web/scripts/build.mjs",
|
||||
"node web/hwlab-cloud-web/scripts/check.mjs",
|
||||
"node scripts/dev-artifact-publish.mjs --preflight --no-report",
|
||||
"node scripts/g14-artifact-publish.mjs --preflight --no-report",
|
||||
"node scripts/refresh-artifact-catalog.mjs --target-ref origin/main --blocked --no-write",
|
||||
guardCommand
|
||||
];
|
||||
@@ -1181,7 +1179,7 @@ function nextCommands({ canPublish, canApply, blockedReasons }) {
|
||||
return [
|
||||
"node scripts/deploy-desired-state-plan.mjs --target-ref origin/main --pretty",
|
||||
"node scripts/deploy-desired-state-plan.mjs --promotion-commit <origin-main-sha> --check",
|
||||
`node scripts/dev-deploy-apply.mjs --dry-run --report ${tempReportPath("dev-deploy-report.json")}`,
|
||||
"node scripts/g14-gitops-render.mjs --check",
|
||||
guardCommand
|
||||
];
|
||||
}
|
||||
@@ -1194,7 +1192,7 @@ function nextCommands({ canPublish, canApply, blockedReasons }) {
|
||||
];
|
||||
}
|
||||
return [
|
||||
`node scripts/dev-deploy-apply.mjs --dry-run --report ${tempReportPath("dev-deploy-report.json")}`,
|
||||
"node scripts/g14-gitops-render.mjs --check",
|
||||
"node scripts/artifact-runtime-readiness-guard.mjs --target-ref origin/main --check --live --no-report"
|
||||
];
|
||||
}
|
||||
|
||||
@@ -67,7 +67,7 @@ function provisionPlan({ exactBlocker = null, envValue = recommendedNode20Image,
|
||||
`${envVarName}=${recommendedNode20Image} node scripts/preflight-dev-base-image.mjs`
|
||||
],
|
||||
artifactPublishDryRun: [
|
||||
`${envVarName}=${envValue} node scripts/dev-artifact-publish.mjs --preflight --no-report`
|
||||
`${envVarName}=${envValue} node scripts/g14-artifact-publish.mjs --preflight --no-report`
|
||||
]
|
||||
}
|
||||
};
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,122 +0,0 @@
|
||||
import { tempReportPath } from "./report-paths.mjs";
|
||||
|
||||
const transactionEnvNames = [
|
||||
"HWLAB_CD_TRANSACTION_ID",
|
||||
"HWLAB_CD_TRANSACTION_OWNER",
|
||||
"HWLAB_CD_LOCK_NAME"
|
||||
];
|
||||
|
||||
const ciArtifactIdentityEnvNames = [
|
||||
"HWLAB_CI_ARTIFACT_RUN_ID",
|
||||
"GITHUB_RUN_ID",
|
||||
"CI_PIPELINE_ID",
|
||||
"BUILDKITE_BUILD_ID"
|
||||
];
|
||||
|
||||
const artifactPublishIdentityEnvNames = [
|
||||
...ciArtifactIdentityEnvNames,
|
||||
...transactionEnvNames
|
||||
];
|
||||
|
||||
export function hasDevCdTransactionEnv(env = process.env) {
|
||||
return typeof env.HWLAB_CD_TRANSACTION_ID === "string" && env.HWLAB_CD_TRANSACTION_ID.trim().length > 0;
|
||||
}
|
||||
|
||||
export function hasCiArtifactPublishEnv(env = process.env) {
|
||||
return ciArtifactIdentityEnvNames.some((name) => typeof env[name] === "string" && env[name].trim().length > 0);
|
||||
}
|
||||
|
||||
export function hasDevCiArtifactEnv(env = process.env) {
|
||||
return hasCiArtifactPublishEnv(env);
|
||||
}
|
||||
|
||||
export function hasDevArtifactSideEffectEnv(env = process.env) {
|
||||
return hasDevCdTransactionEnv(env) || hasCiArtifactPublishEnv(env);
|
||||
}
|
||||
|
||||
export function devArtifactProducer(env = process.env) {
|
||||
if (hasCiArtifactPublishEnv(env)) {
|
||||
const idName = ciArtifactIdentityEnvNames.find((name) => typeof env[name] === "string" && env[name].trim().length > 0);
|
||||
return {
|
||||
kind: "ci-artifact",
|
||||
runId: env[idName],
|
||||
owner: env.HWLAB_CI_ARTIFACT_RUN_OWNER ?? idName ?? null
|
||||
};
|
||||
}
|
||||
if (hasDevCdTransactionEnv(env)) {
|
||||
return {
|
||||
kind: "cd-transaction-legacy",
|
||||
runId: env.HWLAB_CD_TRANSACTION_ID,
|
||||
owner: env.HWLAB_CD_TRANSACTION_OWNER ?? null
|
||||
};
|
||||
}
|
||||
return {
|
||||
kind: "missing",
|
||||
runId: null,
|
||||
owner: null
|
||||
};
|
||||
}
|
||||
|
||||
export function devCdTransactionGuardFailure({
|
||||
script,
|
||||
mode,
|
||||
requiredEnv = "HWLAB_CD_TRANSACTION_ID"
|
||||
}) {
|
||||
return {
|
||||
ok: false,
|
||||
status: "blocked",
|
||||
error: "cd-transaction-required",
|
||||
code: "cd-transaction-required",
|
||||
script,
|
||||
mode,
|
||||
requiredEnv,
|
||||
acceptedEnv: transactionEnvNames,
|
||||
entrypoint: `node scripts/dev-cd-apply.mjs --apply --confirm-dev --confirmed-non-production --report ${tempReportPath("dev-cd-apply.json")}`,
|
||||
summary: `${script} ${mode} is a DEV CD side-effect step and must run inside scripts/dev-cd-apply.mjs.`,
|
||||
devOnly: true,
|
||||
prodTouched: false,
|
||||
mutationAttempted: false
|
||||
};
|
||||
}
|
||||
|
||||
export function devArtifactPublishIdentityGuardFailure({
|
||||
script,
|
||||
mode,
|
||||
requiredEnv = "HWLAB_CI_ARTIFACT_RUN_ID or HWLAB_CD_TRANSACTION_ID"
|
||||
}) {
|
||||
return {
|
||||
ok: false,
|
||||
status: "blocked",
|
||||
error: "artifact-publish-identity-required",
|
||||
code: "artifact-publish-identity-required",
|
||||
script,
|
||||
mode,
|
||||
requiredEnv,
|
||||
acceptedEnv: artifactPublishIdentityEnvNames,
|
||||
entrypoint: `HWLAB_CI_ARTIFACT_RUN_ID=<run-id> node scripts/artifact-publish.mjs --publish --report ${tempReportPath("dev-artifacts.json")}`,
|
||||
summary: `${script} ${mode} is an artifact publish side-effect step and must run under CI artifact identity.`,
|
||||
devOnly: true,
|
||||
prodTouched: false,
|
||||
cdLockRequired: false,
|
||||
rolloutAttempted: false,
|
||||
mutationAttempted: false
|
||||
};
|
||||
}
|
||||
|
||||
export function devArtifactSideEffectGuardFailure(options) {
|
||||
return devArtifactPublishIdentityGuardFailure(options);
|
||||
}
|
||||
|
||||
export function requireDevCdTransactionForSideEffect({ env = process.env, script, mode }) {
|
||||
if (hasDevCdTransactionEnv(env)) return null;
|
||||
return devCdTransactionGuardFailure({ script, mode });
|
||||
}
|
||||
|
||||
export function requireDevArtifactPublishIdentity({ env = process.env, script, mode }) {
|
||||
if (hasCiArtifactPublishEnv(env) || hasDevCdTransactionEnv(env)) return null;
|
||||
return devArtifactPublishIdentityGuardFailure({ script, mode });
|
||||
}
|
||||
|
||||
export function requireDevArtifactSideEffectEnv(options) {
|
||||
return requireDevArtifactPublishIdentity(options);
|
||||
}
|
||||
@@ -488,7 +488,7 @@ function runStaticSmoke() {
|
||||
const blockers = [];
|
||||
const files = readStaticFiles();
|
||||
const source = Object.values(files).join("\n");
|
||||
const artifactPublisher = readText("scripts/dev-artifact-publish.mjs");
|
||||
const artifactPublisher = readText("scripts/artifact-publish.mjs");
|
||||
const buildScript = readText("web/hwlab-cloud-web/scripts/build.mjs");
|
||||
const distContractScript = readText("web/hwlab-cloud-web/scripts/dist-contract.mjs");
|
||||
|
||||
@@ -1745,7 +1745,7 @@ function readStaticFiles() {
|
||||
panel: readText("web/hwlab-cloud-web/workbench-hardware-panel.mjs"),
|
||||
codeAgentM3Evidence: readText("web/hwlab-cloud-web/code-agent-m3-evidence.mjs"),
|
||||
help: readText("web/hwlab-cloud-web/help.md"),
|
||||
artifactPublisher: readText("scripts/dev-artifact-publish.mjs")
|
||||
artifactPublisher: readText("scripts/artifact-publish.mjs")
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,648 +0,0 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
|
||||
import {
|
||||
buildKubectlCommandPrefix,
|
||||
compareRuntimeIdentityEnv,
|
||||
decideDevLegacySimulatorDeploymentCleanup,
|
||||
decideDevTemplateJobReplacement,
|
||||
decideDevUnmanagedHotfixDeploymentCleanup,
|
||||
formatKubeconfigAccessFailure,
|
||||
inspectCodeAgentProviderDesiredState,
|
||||
inspectCodeAgentProviderLiveDeployment,
|
||||
parseArgs,
|
||||
redactSensitiveText,
|
||||
registryManifestUrlForImage,
|
||||
resolveApplySourceCommit,
|
||||
resolveDevKubeconfigSelection,
|
||||
verifyCatalogRegistryManifest
|
||||
} from "./dev-deploy-apply.mjs";
|
||||
import {
|
||||
devArtifactProducer,
|
||||
devArtifactPublishIdentityGuardFailure,
|
||||
devCdTransactionGuardFailure,
|
||||
hasCiArtifactPublishEnv,
|
||||
hasDevCdTransactionEnv,
|
||||
requireDevArtifactPublishIdentity,
|
||||
requireDevCdTransactionForSideEffect
|
||||
} from "./dev-cd-transaction-guard.mjs";
|
||||
|
||||
const codeAgentBaseUrl = "http://172.26.26.227:17680/v1/responses";
|
||||
|
||||
function codeAgentDeploy(env = {}) {
|
||||
return {
|
||||
services: [
|
||||
{
|
||||
serviceId: "hwlab-cloud-api",
|
||||
env: {
|
||||
HWLAB_CODE_AGENT_PROVIDER: "codex-stdio",
|
||||
HWLAB_CODE_AGENT_MODEL: "gpt-5.5",
|
||||
HWLAB_CODE_AGENT_OPENAI_BASE_URL: codeAgentBaseUrl,
|
||||
CODEX_HOME: "/codex-home",
|
||||
NO_PROXY: "hyueapi.com,.hyueapi.com,hyui.com,.hyui.com,127.0.0.1,localhost,::1,api.minimaxi.com,.minimaxi.com",
|
||||
no_proxy: "hyueapi.com,.hyueapi.com,hyui.com,.hyui.com,127.0.0.1,localhost,::1,api.minimaxi.com,.minimaxi.com",
|
||||
OPENAI_API_KEY: "secretRef:hwlab-code-agent-provider/openai-api-key",
|
||||
...env
|
||||
}
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
|
||||
function codeAgentWorkloads(envOverrides = []) {
|
||||
return {
|
||||
kind: "List",
|
||||
items: [
|
||||
{
|
||||
kind: "Deployment",
|
||||
metadata: {
|
||||
name: "hwlab-cloud-api",
|
||||
namespace: "hwlab-dev",
|
||||
labels: {
|
||||
"hwlab.pikastech.local/service-id": "hwlab-cloud-api"
|
||||
}
|
||||
},
|
||||
spec: {
|
||||
template: {
|
||||
spec: {
|
||||
containers: [
|
||||
{
|
||||
name: "hwlab-cloud-api",
|
||||
env: [
|
||||
{ name: "HWLAB_CODE_AGENT_PROVIDER", value: "codex-stdio" },
|
||||
{ name: "HWLAB_CODE_AGENT_MODEL", value: "gpt-5.5" },
|
||||
{ name: "HWLAB_CODE_AGENT_OPENAI_BASE_URL", value: codeAgentBaseUrl },
|
||||
{ name: "CODEX_HOME", value: "/codex-home" },
|
||||
{ name: "NO_PROXY", value: "hyueapi.com,.hyueapi.com,hyui.com,.hyui.com,127.0.0.1,localhost,::1,api.minimaxi.com,.minimaxi.com" },
|
||||
{ name: "no_proxy", value: "hyueapi.com,.hyueapi.com,hyui.com,.hyui.com,127.0.0.1,localhost,::1,api.minimaxi.com,.minimaxi.com" },
|
||||
{
|
||||
name: "OPENAI_API_KEY",
|
||||
valueFrom: {
|
||||
secretKeyRef: {
|
||||
name: "hwlab-code-agent-provider",
|
||||
key: "openai-api-key"
|
||||
}
|
||||
}
|
||||
},
|
||||
...envOverrides
|
||||
],
|
||||
volumeMounts: [
|
||||
{ name: "hwlab-code-agent-codex-home", mountPath: "/codex-home" },
|
||||
{ name: "hwlab-code-agent-codex-config", mountPath: "/codex-home/config.toml", subPath: "config.toml", readOnly: true },
|
||||
{ name: "hwlab-code-agent-codex-auth", mountPath: "/codex-home/auth.json", subPath: "auth.json", readOnly: true }
|
||||
]
|
||||
}
|
||||
],
|
||||
volumes: [
|
||||
{ name: "hwlab-code-agent-codex-home", emptyDir: {} },
|
||||
{ name: "hwlab-code-agent-codex-config", configMap: { name: "hwlab-code-agent-codex-config", items: [{ key: "config.toml", path: "config.toml" }] } },
|
||||
{ name: "hwlab-code-agent-codex-auth", secret: { secretName: "hwlab-code-agent-codex-auth", items: [{ key: "auth.json", path: "auth.json" }] } }
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
};
|
||||
}
|
||||
|
||||
test("allowlisted suspended template Job image change plans replacement", () => {
|
||||
const decision = decideDevTemplateJobReplacement({
|
||||
jobName: "hwlab-agent-worker-template",
|
||||
jobNamespace: "hwlab-dev",
|
||||
desiredSuspended: true,
|
||||
liveExists: true,
|
||||
liveSuspended: true,
|
||||
oldImages: [
|
||||
{
|
||||
name: "hwlab-agent-worker",
|
||||
serviceId: "hwlab-agent-worker",
|
||||
image: "127.0.0.1:5000/hwlab/hwlab-agent-worker:old1234"
|
||||
}
|
||||
],
|
||||
newImages: [
|
||||
{
|
||||
name: "hwlab-agent-worker",
|
||||
serviceId: "hwlab-agent-worker",
|
||||
image: "127.0.0.1:5000/hwlab/hwlab-agent-worker:new5678"
|
||||
}
|
||||
]
|
||||
});
|
||||
|
||||
assert.equal(decision.replace, true);
|
||||
assert.equal(decision.result, "planned");
|
||||
assert.equal(decision.namespace, "hwlab-dev");
|
||||
assert.equal(decision.jobName, "hwlab-agent-worker-template");
|
||||
assert.equal(decision.oldImage, "127.0.0.1:5000/hwlab/hwlab-agent-worker:old1234");
|
||||
assert.equal(decision.newImage, "127.0.0.1:5000/hwlab/hwlab-agent-worker:new5678");
|
||||
});
|
||||
|
||||
test("DEV CD transaction guard rejects direct legacy side-effect calls", () => {
|
||||
assert.equal(hasDevCdTransactionEnv({}), false);
|
||||
assert.equal(hasDevCdTransactionEnv({ HWLAB_CD_TRANSACTION_ID: "tx-123" }), true);
|
||||
|
||||
const failure = requireDevCdTransactionForSideEffect({
|
||||
env: {},
|
||||
script: "scripts/dev-deploy-apply.mjs",
|
||||
mode: "--apply"
|
||||
});
|
||||
assert.deepEqual(failure, devCdTransactionGuardFailure({
|
||||
script: "scripts/dev-deploy-apply.mjs",
|
||||
mode: "--apply"
|
||||
}));
|
||||
assert.equal(failure.error, "cd-transaction-required");
|
||||
assert.equal(failure.mutationAttempted, false);
|
||||
assert.match(failure.entrypoint, /scripts\/dev-cd-apply\.mjs/u);
|
||||
|
||||
assert.equal(
|
||||
requireDevCdTransactionForSideEffect({
|
||||
env: { HWLAB_CD_TRANSACTION_ID: "tx-123" },
|
||||
script: "scripts/dev-deploy-apply.mjs",
|
||||
mode: "--apply"
|
||||
}),
|
||||
null
|
||||
);
|
||||
});
|
||||
|
||||
test("DEV artifact publish guard allows CI identity without CD transaction", () => {
|
||||
const ciEnv = {
|
||||
HWLAB_CI_ARTIFACT_RUN_ID: "ci-artifact-123",
|
||||
HWLAB_CI_ARTIFACT_RUN_OWNER: "tekton"
|
||||
};
|
||||
assert.equal(hasCiArtifactPublishEnv({}), false);
|
||||
assert.equal(hasCiArtifactPublishEnv(ciEnv), true);
|
||||
assert.equal(requireDevArtifactPublishIdentity({
|
||||
env: ciEnv,
|
||||
script: "scripts/dev-artifact-publish.mjs",
|
||||
mode: "--publish"
|
||||
}), null);
|
||||
assert.deepEqual(devArtifactProducer(ciEnv), {
|
||||
kind: "ci-artifact",
|
||||
runId: "ci-artifact-123",
|
||||
owner: "tekton"
|
||||
});
|
||||
|
||||
const failure = requireDevArtifactPublishIdentity({
|
||||
env: {},
|
||||
script: "scripts/dev-artifact-publish.mjs",
|
||||
mode: "--publish"
|
||||
});
|
||||
assert.deepEqual(failure, devArtifactPublishIdentityGuardFailure({
|
||||
script: "scripts/dev-artifact-publish.mjs",
|
||||
mode: "--publish"
|
||||
}));
|
||||
assert.equal(failure.error, "artifact-publish-identity-required");
|
||||
assert.equal(failure.cdLockRequired, false);
|
||||
assert.equal(failure.rolloutAttempted, false);
|
||||
assert.equal(failure.mutationAttempted, false);
|
||||
assert.match(failure.entrypoint, /HWLAB_CI_ARTIFACT_RUN_ID=.*dev-artifact-publish\.mjs/u);
|
||||
|
||||
assert.equal(
|
||||
requireDevArtifactPublishIdentity({
|
||||
env: { HWLAB_CI_ARTIFACT_RUN_ID: "ci-123" },
|
||||
script: "scripts/dev-artifact-publish.mjs",
|
||||
mode: "--publish"
|
||||
}),
|
||||
null
|
||||
);
|
||||
assert.equal(
|
||||
requireDevArtifactPublishIdentity({
|
||||
env: { HWLAB_CD_TRANSACTION_ID: "tx-123" },
|
||||
script: "scripts/dev-artifact-publish.mjs",
|
||||
mode: "--publish"
|
||||
}),
|
||||
null
|
||||
);
|
||||
});
|
||||
|
||||
test("matching allowlisted suspended template Job image does not replace", () => {
|
||||
const image = "127.0.0.1:5000/hwlab/hwlab-cli:73b379f";
|
||||
const decision = decideDevTemplateJobReplacement({
|
||||
jobName: "hwlab-cli-template",
|
||||
jobNamespace: "hwlab-dev",
|
||||
desiredSuspended: true,
|
||||
liveExists: true,
|
||||
liveSuspended: true,
|
||||
oldImages: [{ name: "hwlab-cli", serviceId: "hwlab-cli", image }],
|
||||
newImages: [{ name: "hwlab-cli", serviceId: "hwlab-cli", image }]
|
||||
});
|
||||
|
||||
assert.equal(decision.replace, false);
|
||||
assert.equal(decision.result, "not_needed");
|
||||
});
|
||||
|
||||
test("replacement policy refuses non-allowlisted or active Jobs", () => {
|
||||
const ignored = decideDevTemplateJobReplacement({
|
||||
jobName: "some-other-job",
|
||||
jobNamespace: "hwlab-dev",
|
||||
desiredSuspended: true,
|
||||
liveExists: true,
|
||||
liveSuspended: true,
|
||||
oldImages: [{ name: "worker", serviceId: "worker", image: "image:old" }],
|
||||
newImages: [{ name: "worker", serviceId: "worker", image: "image:new" }]
|
||||
});
|
||||
assert.equal(ignored.replace, false);
|
||||
assert.equal(ignored.result, "ignored");
|
||||
|
||||
const active = decideDevTemplateJobReplacement({
|
||||
jobName: "hwlab-cli-template",
|
||||
jobNamespace: "hwlab-dev",
|
||||
desiredSuspended: true,
|
||||
liveExists: true,
|
||||
liveSuspended: false,
|
||||
oldImages: [{ name: "hwlab-cli", serviceId: "hwlab-cli", image: "image:old" }],
|
||||
newImages: [{ name: "hwlab-cli", serviceId: "hwlab-cli", image: "image:new" }]
|
||||
});
|
||||
assert.equal(active.replace, false);
|
||||
assert.equal(active.result, "blocked");
|
||||
});
|
||||
|
||||
test("allowlisted stale simulator Deployment plans cleanup only with StatefulSet replacement", () => {
|
||||
const decision = decideDevLegacySimulatorDeploymentCleanup({
|
||||
deploymentName: "hwlab-box-simu",
|
||||
deploymentNamespace: "hwlab-dev",
|
||||
desiredReplacementExists: true,
|
||||
desiredReplacementKind: "StatefulSet",
|
||||
liveExists: true,
|
||||
oldImages: [
|
||||
{
|
||||
name: "hwlab-box-simu",
|
||||
serviceId: "hwlab-box-simu",
|
||||
image: "127.0.0.1:5000/hwlab/hwlab-box-simu:old1234"
|
||||
}
|
||||
],
|
||||
newImages: [
|
||||
{
|
||||
name: "hwlab-box-simu",
|
||||
serviceId: "hwlab-box-simu",
|
||||
image: "127.0.0.1:5000/hwlab/hwlab-box-simu:new5678"
|
||||
}
|
||||
]
|
||||
});
|
||||
|
||||
assert.equal(decision.cleanup, true);
|
||||
assert.equal(decision.result, "planned");
|
||||
assert.equal(decision.namespace, "hwlab-dev");
|
||||
assert.equal(decision.deploymentName, "hwlab-box-simu");
|
||||
assert.equal(decision.serviceId, "hwlab-box-simu");
|
||||
assert.equal(decision.desiredReplacementKind, "StatefulSet");
|
||||
assert.equal(decision.oldImage, "127.0.0.1:5000/hwlab/hwlab-box-simu:old1234");
|
||||
assert.equal(decision.newImage, "127.0.0.1:5000/hwlab/hwlab-box-simu:new5678");
|
||||
});
|
||||
|
||||
test("legacy simulator cleanup refuses non-DEV or missing replacement cleanup", () => {
|
||||
const wrongNamespace = decideDevLegacySimulatorDeploymentCleanup({
|
||||
deploymentName: "hwlab-gateway-simu",
|
||||
deploymentNamespace: "hwlab-prod",
|
||||
desiredReplacementExists: true,
|
||||
desiredReplacementKind: "StatefulSet",
|
||||
liveExists: true
|
||||
});
|
||||
assert.equal(wrongNamespace.cleanup, false);
|
||||
assert.equal(wrongNamespace.result, "blocked");
|
||||
|
||||
const missingReplacement = decideDevLegacySimulatorDeploymentCleanup({
|
||||
deploymentName: "hwlab-gateway-simu",
|
||||
deploymentNamespace: "hwlab-dev",
|
||||
desiredReplacementExists: false,
|
||||
desiredReplacementKind: null,
|
||||
liveExists: true
|
||||
});
|
||||
assert.equal(missingReplacement.cleanup, false);
|
||||
assert.equal(missingReplacement.result, "blocked");
|
||||
|
||||
const ignored = decideDevLegacySimulatorDeploymentCleanup({
|
||||
deploymentName: "hwlab-cloud-api",
|
||||
deploymentNamespace: "hwlab-dev",
|
||||
desiredReplacementExists: true,
|
||||
desiredReplacementKind: "StatefulSet",
|
||||
liveExists: true
|
||||
});
|
||||
assert.equal(ignored.cleanup, false);
|
||||
assert.equal(ignored.result, "ignored");
|
||||
});
|
||||
|
||||
test("unmanaged hotfix Deployment cleanup plans recreation when live has extra hotfix mounts", () => {
|
||||
const desired = codeAgentWorkloads().items[0];
|
||||
const live = structuredClone(desired);
|
||||
live.spec.template.metadata = {
|
||||
annotations: {
|
||||
"hwlab.pikastech.local/runtime-hotfix": "2026-05-24"
|
||||
}
|
||||
};
|
||||
live.spec.template.spec.volumes.unshift({
|
||||
name: "hwlab-code-agent-hotfix-20260524",
|
||||
configMap: { name: "hwlab-code-agent-hotfix-20260524" }
|
||||
});
|
||||
live.spec.template.spec.containers[0].volumeMounts.unshift({
|
||||
name: "hwlab-code-agent-hotfix-20260524",
|
||||
mountPath: "/app/internal/cloud/server.mjs",
|
||||
subPath: "server.mjs",
|
||||
readOnly: true
|
||||
});
|
||||
|
||||
const decision = decideDevUnmanagedHotfixDeploymentCleanup({
|
||||
deploymentName: "hwlab-cloud-api",
|
||||
deploymentNamespace: "hwlab-dev",
|
||||
desiredDeployment: desired,
|
||||
liveDeployment: live
|
||||
});
|
||||
|
||||
assert.equal(decision.cleanup, true);
|
||||
assert.equal(decision.result, "planned");
|
||||
assert.equal(decision.unmanagedVolumes[0].configMap, "hwlab-code-agent-hotfix-20260524");
|
||||
assert.equal(decision.unmanagedVolumeMounts[0].mountPath, "/app/internal/cloud/server.mjs");
|
||||
assert.equal(decision.unmanagedTemplateAnnotations[0].key, "hwlab.pikastech.local/runtime-hotfix");
|
||||
});
|
||||
|
||||
test("unmanaged hotfix Deployment cleanup ignores fields already present in desired source", () => {
|
||||
const desired = codeAgentWorkloads().items[0];
|
||||
desired.spec.template.spec.volumes.unshift({
|
||||
name: "hwlab-code-agent-hotfix-20260524",
|
||||
configMap: { name: "hwlab-code-agent-hotfix-20260524" }
|
||||
});
|
||||
desired.spec.template.spec.containers[0].volumeMounts.unshift({
|
||||
name: "hwlab-code-agent-hotfix-20260524",
|
||||
mountPath: "/app/internal/cloud/server.mjs",
|
||||
subPath: "server.mjs",
|
||||
readOnly: true
|
||||
});
|
||||
const live = structuredClone(desired);
|
||||
|
||||
const decision = decideDevUnmanagedHotfixDeploymentCleanup({
|
||||
deploymentName: "hwlab-cloud-api",
|
||||
deploymentNamespace: "hwlab-dev",
|
||||
desiredDeployment: desired,
|
||||
liveDeployment: live
|
||||
});
|
||||
|
||||
assert.equal(decision.cleanup, false);
|
||||
assert.equal(decision.result, "not_needed");
|
||||
});
|
||||
|
||||
test("apply source commit follows deploy/catalog artifact identity", () => {
|
||||
assert.equal(
|
||||
resolveApplySourceCommit({ commitId: "73b379f" }, { commitId: "73b379f" }, "cb35ada68606"),
|
||||
"73b379f"
|
||||
);
|
||||
assert.equal(
|
||||
resolveApplySourceCommit({ commitId: "73b379f" }, { commitId: "deadbee" }, "cb35ada68606"),
|
||||
"cb35ada68606"
|
||||
);
|
||||
});
|
||||
|
||||
test("runtime identity env comparator accepts matching commit image and tag", () => {
|
||||
const comparison = compareRuntimeIdentityEnv(
|
||||
{
|
||||
HWLAB_COMMIT_ID: "7de6edd",
|
||||
HWLAB_IMAGE: "127.0.0.1:5000/hwlab/hwlab-cloud-web:7de6edd",
|
||||
HWLAB_IMAGE_TAG: "7de6edd"
|
||||
},
|
||||
{
|
||||
HWLAB_COMMIT_ID: "7de6edd2c41fb50dcb007353780338be8832b27e",
|
||||
HWLAB_IMAGE: "127.0.0.1:5000/hwlab/hwlab-cloud-web:7de6edd",
|
||||
HWLAB_IMAGE_TAG: "7de6edd"
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(comparison.status, "match");
|
||||
assert.equal(comparison.matchesDesired, true);
|
||||
assert.deepEqual(comparison.drift, []);
|
||||
});
|
||||
|
||||
test("runtime identity env comparator detects stale cloud-web env drift", () => {
|
||||
const comparison = compareRuntimeIdentityEnv(
|
||||
{
|
||||
HWLAB_COMMIT_ID: "c7de474",
|
||||
HWLAB_IMAGE: "127.0.0.1:5000/hwlab/hwlab-cloud-web:c7de474",
|
||||
HWLAB_IMAGE_TAG: "c7de474"
|
||||
},
|
||||
{
|
||||
HWLAB_COMMIT_ID: "7de6edd",
|
||||
HWLAB_IMAGE: "127.0.0.1:5000/hwlab/hwlab-cloud-web:7de6edd",
|
||||
HWLAB_IMAGE_TAG: "7de6edd"
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(comparison.status, "drift");
|
||||
assert.equal(comparison.matchesDesired, false);
|
||||
assert.deepEqual(
|
||||
comparison.drift.map((field) => field.name),
|
||||
["HWLAB_COMMIT_ID", "HWLAB_IMAGE", "HWLAB_IMAGE_TAG"]
|
||||
);
|
||||
});
|
||||
|
||||
test("Code Agent provider desired state preserves Secret ref and egress base URL", () => {
|
||||
const inspection = inspectCodeAgentProviderDesiredState(codeAgentDeploy(), codeAgentWorkloads());
|
||||
|
||||
assert.equal(inspection.ready, true);
|
||||
assert.equal(inspection.status, "pass");
|
||||
assert.equal(inspection.provider, "codex-stdio");
|
||||
assert.equal(inspection.runtimeProvider, "openai-responses");
|
||||
assert.equal(inspection.backend, "hwlab-cloud-api/codex-app-server-stdio");
|
||||
assert.equal(inspection.secretRef.present, true);
|
||||
assert.equal(inspection.secretMaterialRead, false);
|
||||
assert.equal(inspection.egress.deployMatchesDevProxy, true);
|
||||
assert.equal(inspection.egress.workloadMatchesDevProxy, true);
|
||||
assert.equal(inspection.codexHome.present, true);
|
||||
assert.equal(inspection.codexHome.configMapPresent, true);
|
||||
assert.equal(inspection.codexHome.authSecretPresent, true);
|
||||
});
|
||||
|
||||
test("Code Agent provider desired state blocks direct public OpenAI base URL", () => {
|
||||
const inspection = inspectCodeAgentProviderDesiredState(
|
||||
codeAgentDeploy({ HWLAB_CODE_AGENT_OPENAI_BASE_URL: "https://api.openai.com/v1/responses" }),
|
||||
codeAgentWorkloads([
|
||||
{ name: "HWLAB_CODE_AGENT_OPENAI_BASE_URL", value: "https://api.openai.com/v1/responses" }
|
||||
])
|
||||
);
|
||||
|
||||
assert.equal(inspection.ready, false);
|
||||
assert.equal(inspection.status, "blocked");
|
||||
assert.equal(inspection.egress.directPublicOpenAi, true);
|
||||
assert.ok(inspection.missingEgressContract.some((entry) => entry.includes("api.openai.com")));
|
||||
});
|
||||
|
||||
test("Code Agent live Deployment inspection reads only env metadata", () => {
|
||||
const deployment = codeAgentWorkloads().items[0];
|
||||
const inspection = inspectCodeAgentProviderLiveDeployment(deployment);
|
||||
|
||||
assert.equal(inspection.ready, true);
|
||||
assert.equal(inspection.secretRef.present, true);
|
||||
assert.equal(inspection.secretValuesRead, false);
|
||||
assert.equal(inspection.kubernetesSecretDataRead, false);
|
||||
assert.equal(inspection.validationScope, "deployment-env-and-secretKeyRef-metadata-only");
|
||||
});
|
||||
|
||||
test("DEV kubeconfig selection prefers flag then HWLAB_DEV_KUBECONFIG then KUBECONFIG then D601 default", () => {
|
||||
assert.deepEqual(
|
||||
resolveDevKubeconfigSelection({
|
||||
flagValue: "/tmp/from-flag",
|
||||
env: {
|
||||
HWLAB_DEV_KUBECONFIG: "/tmp/from-hwlab",
|
||||
KUBECONFIG: "/tmp/from-kubeconfig"
|
||||
}
|
||||
}),
|
||||
{
|
||||
kubeconfig: "/tmp/from-flag",
|
||||
source: "flag:--kubeconfig",
|
||||
paths: ["/tmp/from-flag"]
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
resolveDevKubeconfigSelection({
|
||||
env: {
|
||||
HWLAB_DEV_KUBECONFIG: "/tmp/from-hwlab",
|
||||
KUBECONFIG: "/tmp/from-kubeconfig"
|
||||
}
|
||||
}),
|
||||
{
|
||||
kubeconfig: "/tmp/from-hwlab",
|
||||
source: "env:HWLAB_DEV_KUBECONFIG",
|
||||
paths: ["/tmp/from-hwlab"]
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
resolveDevKubeconfigSelection({
|
||||
env: {
|
||||
KUBECONFIG: "/tmp/from-kubeconfig"
|
||||
}
|
||||
}),
|
||||
{
|
||||
kubeconfig: "/tmp/from-kubeconfig",
|
||||
source: "env:KUBECONFIG",
|
||||
paths: ["/tmp/from-kubeconfig"]
|
||||
}
|
||||
);
|
||||
|
||||
assert.deepEqual(
|
||||
resolveDevKubeconfigSelection({ env: {} }),
|
||||
{
|
||||
kubeconfig: "/etc/rancher/k3s/k3s.yaml",
|
||||
source: "default:d601-k3s",
|
||||
paths: ["/etc/rancher/k3s/k3s.yaml"]
|
||||
}
|
||||
);
|
||||
});
|
||||
|
||||
test("DEV kubeconfig selection keeps multi-path KUBECONFIG readable checks explicit", () => {
|
||||
const delimiter = process.platform === "win32" ? ";" : ":";
|
||||
const selection = resolveDevKubeconfigSelection({
|
||||
env: {
|
||||
KUBECONFIG: [`/tmp/one`, `/tmp/two`].join(delimiter)
|
||||
}
|
||||
});
|
||||
|
||||
assert.equal(selection.kubeconfig, [`/tmp/one`, `/tmp/two`].join(delimiter));
|
||||
assert.equal(selection.source, "env:KUBECONFIG");
|
||||
assert.deepEqual(selection.paths, ["/tmp/one", "/tmp/two"]);
|
||||
});
|
||||
|
||||
test("kubectl command prefix quotes kubeconfig paths without exposing contents", () => {
|
||||
assert.equal(
|
||||
buildKubectlCommandPrefix("/var/lib/unidesk/code-queue/kubeconfig"),
|
||||
"KUBECONFIG=/var/lib/unidesk/code-queue/kubeconfig kubectl"
|
||||
);
|
||||
assert.equal(
|
||||
buildKubectlCommandPrefix("/tmp/path with space/kubeconfig"),
|
||||
"KUBECONFIG='/tmp/path with space/kubeconfig' kubectl"
|
||||
);
|
||||
});
|
||||
|
||||
test("kubeconfig access failure names source and path without secret material", () => {
|
||||
const message = formatKubeconfigAccessFailure(
|
||||
{
|
||||
source: "env:HWLAB_DEV_KUBECONFIG"
|
||||
},
|
||||
"/missing/kubeconfig"
|
||||
);
|
||||
|
||||
assert.match(message, /env:HWLAB_DEV_KUBECONFIG/u);
|
||||
assert.match(message, /\/missing\/kubeconfig/u);
|
||||
assert.match(message, /Secret values are never printed/u);
|
||||
assert.equal(message.includes("client-key-data:"), false);
|
||||
});
|
||||
|
||||
test("parseArgs reports a missing kubeconfig path as a blocking error", () => {
|
||||
const args = parseArgs(["--apply", "--kubeconfig"]);
|
||||
assert.equal(args.apply, true);
|
||||
assert.equal(args.kubeconfigSpecified, true);
|
||||
assert.deepEqual(args.errors, ["--kubeconfig requires a non-empty path value"]);
|
||||
});
|
||||
|
||||
test("registry manifest URL can be redirected without changing deploy image identity", () => {
|
||||
const parsed = registryManifestUrlForImage("127.0.0.1:5000/hwlab/hwlab-cloud-api:abc1234", {
|
||||
registryManifestBaseUrl: "http://hwlab-registry-proxy.hwlab-dev.svc.cluster.local:5000/"
|
||||
});
|
||||
|
||||
assert.equal(parsed.registry, "127.0.0.1:5000");
|
||||
assert.equal(parsed.repository, "hwlab/hwlab-cloud-api");
|
||||
assert.equal(parsed.url, "http://hwlab-registry-proxy.hwlab-dev.svc.cluster.local:5000/v2/hwlab/hwlab-cloud-api/manifests/abc1234");
|
||||
assert.equal(parsed.registryAccess.source, "flag:--registry-manifest-base-url");
|
||||
});
|
||||
|
||||
test("registry manifest verification falls back to docker host-network helper when pod loopback is blocked", async () => {
|
||||
const expectedDigest = "sha256:2cc48ca2bd0e260d7ea6df1e1a791df6615a87da3d89f4b11103735a1e2d9b63";
|
||||
const blockers = [];
|
||||
const commands = [];
|
||||
const result = await verifyCatalogRegistryManifest(
|
||||
{
|
||||
serviceId: "hwlab-cloud-api",
|
||||
image: "127.0.0.1:5000/hwlab/hwlab-cloud-api:fad9fc9",
|
||||
imageTag: "fad9fc9",
|
||||
digest: expectedDigest
|
||||
},
|
||||
blockers,
|
||||
{
|
||||
env: {
|
||||
HWLAB_DEV_REGISTRY_HOSTNET_HELPER_IMAGE: "helper:local"
|
||||
},
|
||||
httpRequestText: async () => {
|
||||
throw new Error("connect ECONNREFUSED 127.0.0.1:5000");
|
||||
},
|
||||
runCommand: async (command, args) => {
|
||||
commands.push([command, args]);
|
||||
return {
|
||||
ok: true,
|
||||
code: 0,
|
||||
stdout: JSON.stringify({
|
||||
statusCode: 200,
|
||||
headers: { "docker-content-digest": expectedDigest }
|
||||
}),
|
||||
stderr: ""
|
||||
};
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.status, "pass");
|
||||
assert.equal(result.registryAccess.mode, "docker-host-network");
|
||||
assert.equal(result.registryAccess.helperImage, "helper:local");
|
||||
assert.match(result.registryAccess.directError, /ECONNREFUSED/u);
|
||||
assert.deepEqual(blockers, []);
|
||||
assert.equal(commands[0][0], "docker");
|
||||
assert.deepEqual(commands[0][1].slice(0, 6), ["run", "--rm", "--pull=never", "--network", "host", "helper:local"]);
|
||||
});
|
||||
|
||||
test("redaction removes common token and password material from kubectl output", () => {
|
||||
const output = redactSensitiveText(
|
||||
[
|
||||
"Authorization: Bearer abc.def-123",
|
||||
"token: abcdef123456",
|
||||
"password=plain-secret",
|
||||
"client-key-data: LS0tPRIVATE",
|
||||
"postgresql://hwlab:db-secret@example.invalid:5432/hwlab"
|
||||
].join("\n")
|
||||
);
|
||||
|
||||
assert.equal(output.includes("abc.def-123"), false);
|
||||
assert.equal(output.includes("plain-secret"), false);
|
||||
assert.equal(output.includes("LS0tPRIVATE"), false);
|
||||
assert.equal(output.includes("db-secret"), false);
|
||||
assert.match(output, /Bearer <redacted>/u);
|
||||
assert.match(output, /password=<redacted>/u);
|
||||
assert.match(output, /client-key-data: <redacted>/u);
|
||||
assert.match(output, /postgresql:\/\/hwlab:<redacted>@example.invalid/u);
|
||||
});
|
||||
@@ -58,7 +58,7 @@ const artifactBuildInputFiles = new Set([
|
||||
"package.json",
|
||||
"package-lock.json",
|
||||
"npm-shrinkwrap.json",
|
||||
"scripts/dev-artifact-publish.mjs",
|
||||
"scripts/g14-artifact-publish.mjs",
|
||||
"scripts/src/dev-artifact-services.mjs"
|
||||
]);
|
||||
|
||||
@@ -654,10 +654,10 @@ async function validateLocalContracts(reporter, contracts, optionalReports, targ
|
||||
|
||||
function validateArtifactPublishReport(reporter, artifactReport, artifactIdentity, targetShortCommit, targetCommit, targetRef) {
|
||||
if (!artifactReport) {
|
||||
reporter.check("dev-artifact-publish-report", "registry", "blocked", "No DEV artifact publish report is present.");
|
||||
reporter.check("g14-artifact-publish-report", "registry", "blocked", "No G14 artifact publish report is present.");
|
||||
reporter.block({
|
||||
type: "runtime_blocker",
|
||||
scope: "dev-artifact-publish",
|
||||
scope: "g14-artifact-publish",
|
||||
summary: `${defaultArtifactReportPath} is missing, so the preflight has no publish evidence for HWLAB runtime artifacts.`,
|
||||
nextTask: `Run the DEV artifact publish workflow and keep its temporary JSON at ${defaultArtifactReportPath}; do not commit report output.`
|
||||
});
|
||||
@@ -701,7 +701,7 @@ function validateArtifactPublishReport(reporter, artifactReport, artifactIdentit
|
||||
const summary = sourceMatchesTarget
|
||||
? "DEV artifact publish report covers all required enabled service IDs with immutable digests for the target commit, and records disabled service reasons."
|
||||
: `DEV artifact publish report covers artifact source ${artifactIdentity.artifactSource.shortCommitId}; target ${targetRef} ${targetShortCommit} has no artifact build input changes since that source.`;
|
||||
reporter.check("dev-artifact-publish-report", "registry", "pass", summary, [artifactIdentity?.targetCoverage].filter(Boolean));
|
||||
reporter.check("g14-artifact-publish-report", "registry", "pass", summary, [artifactIdentity?.targetCoverage].filter(Boolean));
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -711,14 +711,14 @@ function validateArtifactPublishReport(reporter, artifactReport, artifactIdentit
|
||||
? `source states resolved: ${sourcePresentCount} source-present, ${intentionallyDisabledCount} intentionally-disabled`
|
||||
: "source states are not fully resolved";
|
||||
reporter.check(
|
||||
"dev-artifact-publish-report",
|
||||
"g14-artifact-publish-report",
|
||||
"registry",
|
||||
"blocked",
|
||||
`DEV artifact publish report status is ${artifactPublish?.status ?? "missing"} with ${publishedCount}/${requiredServices.length || serviceCount} required services published for source ${artifactPublish?.sourceCommitId ?? artifactReport.commitId ?? "unknown"}; ${sourceSummary}.`
|
||||
);
|
||||
reporter.block({
|
||||
type: "runtime_blocker",
|
||||
scope: "dev-artifact-publish",
|
||||
scope: "g14-artifact-publish",
|
||||
summary: `${defaultArtifactReportPath} does not prove all required HWLAB service artifacts for ${targetRef} ${targetShortCommit}; current status is ${artifactPublish?.status ?? "missing"} with ${publishedCount}/${requiredServices.length || serviceCount} required services published and ${sourceSummary}.`,
|
||||
nextTask: "Complete DEV artifact publishing for every enabled required HWLAB service at the artifact source commit, or prove the target commit has no artifact build input changes; keep disabled services marked not_published with reasons."
|
||||
});
|
||||
|
||||
@@ -230,7 +230,7 @@ export async function runM3IoControlSourceChecks({ repoRoot: root = repoRoot } =
|
||||
readRepoText(root, "internal/cloud/m3-io-control.mjs"),
|
||||
readRepoText(root, "web/hwlab-cloud-web/app.mjs"),
|
||||
readRepoText(root, "web/hwlab-cloud-web/index.html"),
|
||||
readRepoText(root, "scripts/dev-artifact-publish.mjs"),
|
||||
readRepoText(root, "scripts/artifact-publish.mjs"),
|
||||
readRepoText(root, "internal/dev-entrypoint/cloud-web-routes.mjs")
|
||||
]);
|
||||
|
||||
|
||||
@@ -236,7 +236,7 @@ function assertDurableRuntimeRunbook(value) {
|
||||
}
|
||||
}
|
||||
|
||||
function assertCloudApiImageRuntimeEntrypoints({ packageJson, ciJson, artifactPublisher, devCdApplySource }) {
|
||||
function assertCloudApiImageRuntimeEntrypoints({ packageJson, ciJson, artifactPublisher, gitopsRenderSource }) {
|
||||
const label = "cloud-api image runtime maintenance entrypoints";
|
||||
for (const expected of [
|
||||
"COPY internal ./internal",
|
||||
@@ -250,14 +250,16 @@ function assertCloudApiImageRuntimeEntrypoints({ packageJson, ciJson, artifactPu
|
||||
"cmd/hwlab-cloud-api/provision.mjs",
|
||||
"cmd/hwlab-cloud-api/migrate.mjs"
|
||||
]) {
|
||||
assertIncludes(devCdApplySource, expected, `${label} DEV CD job command`);
|
||||
assertIncludes(packageJson.scripts?.check, `node --check ${expected}`, `${label} package check`);
|
||||
assertIncludes(packageJson.scripts?.check, `node ${expected} --check`, `${label} package runtime check`);
|
||||
}
|
||||
for (const expected of ["scripts/g14-artifact-publish.mjs", "runtime-dev", "runtime-prod"]) {
|
||||
assertIncludes(gitopsRenderSource, expected, `${label} G14 GitOps render source`);
|
||||
}
|
||||
assert.equal(
|
||||
ciJson.commands?.some((command) => command.run === "npm run check"),
|
||||
ciJson.commands?.some((command) => command.id === "g14-contract-check"),
|
||||
true,
|
||||
`${label} CI must run npm run check`
|
||||
`${label} CI must run the G14 contract check`
|
||||
);
|
||||
}
|
||||
|
||||
@@ -441,8 +443,8 @@ assertDurableRuntimeRunbook(await readText("docs/reference/dev-runtime-boundary.
|
||||
assertCloudApiImageRuntimeEntrypoints({
|
||||
packageJson: await readJSON("package.json"),
|
||||
ciJson: await readJSON("CI.json"),
|
||||
artifactPublisher: await readText("scripts/dev-artifact-publish.mjs"),
|
||||
devCdApplySource: await readText("scripts/src/dev-cd-apply.mjs")
|
||||
artifactPublisher: await readText("scripts/artifact-publish.mjs"),
|
||||
gitopsRenderSource: await readText("scripts/g14-gitops-render.mjs")
|
||||
});
|
||||
|
||||
console.log(
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
import { runCicdWorker } from "./cicd-jobs.mjs";
|
||||
|
||||
const jobId = process.argv[2];
|
||||
if (!jobId) {
|
||||
process.stderr.write(`${JSON.stringify({ ok: false, code: "missing-job-id", message: "usage: node tools/hwlab-cli/lib/cicd-job-worker.mjs <jobId>" }, null, 2)}\n`);
|
||||
process.exitCode = 1;
|
||||
} else {
|
||||
process.exitCode = await runCicdWorker({ repoRoot: process.cwd(), jobId });
|
||||
}
|
||||
@@ -1,398 +0,0 @@
|
||||
import { spawn } from "node:child_process";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { createWriteStream } from "node:fs";
|
||||
import { mkdir, open, readdir, readFile, stat, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
const stateRoot = ".state/hwlab-cicd/jobs";
|
||||
const knownKinds = new Set(["ci-publish", "dev-cd-apply"]);
|
||||
|
||||
function isoNow(now) {
|
||||
return (now ? now() : new Date()).toISOString();
|
||||
}
|
||||
|
||||
function compactTimestamp(value) {
|
||||
return value.replace(/[-:.]/gu, "").slice(0, 15);
|
||||
}
|
||||
|
||||
function makeJobId(kind, now) {
|
||||
return `${kind}-${compactTimestamp(isoNow(now))}-${randomBytes(3).toString("hex")}`;
|
||||
}
|
||||
|
||||
function jobDir(repoRoot, jobId) {
|
||||
return path.join(repoRoot, stateRoot, jobId);
|
||||
}
|
||||
|
||||
function jobPaths(repoRoot, jobId) {
|
||||
const dir = jobDir(repoRoot, jobId);
|
||||
return {
|
||||
dir,
|
||||
specPath: path.join(dir, "spec.json"),
|
||||
statePath: path.join(dir, "state.json"),
|
||||
stdoutPath: path.join(dir, "stdout.log"),
|
||||
stderrPath: path.join(dir, "stderr.log"),
|
||||
reportPath: path.join(dir, "report.json")
|
||||
};
|
||||
}
|
||||
|
||||
async function readJson(filePath) {
|
||||
return JSON.parse(await readFile(filePath, "utf8"));
|
||||
}
|
||||
|
||||
async function writeJsonFile(filePath, value) {
|
||||
await writeFile(filePath, `${JSON.stringify(value, null, 2)}\n`, "utf8");
|
||||
}
|
||||
|
||||
function result(exitCode, payload, stream = "stdout") {
|
||||
return { exitCode, payload, stream };
|
||||
}
|
||||
|
||||
function error(code, message, details = {}) {
|
||||
return { ok: false, code, message, ...details };
|
||||
}
|
||||
|
||||
function followup(jobId) {
|
||||
return {
|
||||
status: `hwlab-cli cicd status ${jobId}`,
|
||||
logs: `hwlab-cli cicd logs ${jobId}`,
|
||||
report: `hwlab-cli cicd report ${jobId}`
|
||||
};
|
||||
}
|
||||
|
||||
function optionValue(options, name, fallback = null) {
|
||||
return options.get(name) ?? fallback;
|
||||
}
|
||||
|
||||
function positiveInteger(value, fallback) {
|
||||
if (value == null) return fallback;
|
||||
const parsed = Number.parseInt(value, 10);
|
||||
return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback;
|
||||
}
|
||||
|
||||
function buildJobSpec({ kind, flags, options, repoRoot, env, now }) {
|
||||
if (!knownKinds.has(kind)) {
|
||||
return { error: error("unsupported-cicd-kind", "Use --kind ci-publish or --kind dev-cd-apply.", { kind }) };
|
||||
}
|
||||
|
||||
const jobId = makeJobId(kind, now);
|
||||
const paths = jobPaths(repoRoot, jobId);
|
||||
const envPatch = {};
|
||||
const args = [];
|
||||
|
||||
if (kind === "ci-publish") {
|
||||
args.push("scripts/dev-artifact-publish.mjs", "--publish", "--report", paths.reportPath, "--quiet-build");
|
||||
const services = optionValue(options, "--services");
|
||||
if (services) args.push("--services", services);
|
||||
const concurrency = positiveInteger(optionValue(options, "--concurrency"), 4);
|
||||
args.push("--concurrency", String(concurrency));
|
||||
envPatch.HWLAB_CI_ARTIFACT_RUN_ID = optionValue(options, "--owner-task-id", jobId);
|
||||
envPatch.HWLAB_CI_ARTIFACT_RUN_OWNER = optionValue(options, "--owner", "hwlab-cli");
|
||||
} else {
|
||||
if (!flags.has("--confirm-dev") || !flags.has("--confirmed-non-production")) {
|
||||
return {
|
||||
error: error("confirmation-required", "DEV CD apply requires --confirm-dev --confirmed-non-production.", {
|
||||
requiredFlags: ["--confirm-dev", "--confirmed-non-production"]
|
||||
})
|
||||
};
|
||||
}
|
||||
args.push(
|
||||
"scripts/dev-cd-apply.mjs",
|
||||
"--apply",
|
||||
"--confirm-dev",
|
||||
"--confirmed-non-production",
|
||||
"--owner-task-id",
|
||||
optionValue(options, "--owner-task-id", jobId),
|
||||
"--report",
|
||||
paths.reportPath
|
||||
);
|
||||
if (flags.has("--break-stale-lock")) args.push("--break-stale-lock");
|
||||
const concurrency = optionValue(options, "--concurrency");
|
||||
if (concurrency) envPatch.HWLAB_DEV_CD_CONCURRENCY = String(positiveInteger(concurrency, 4));
|
||||
const registryManifestBaseUrl = optionValue(options, "--registry-manifest-base-url");
|
||||
if (registryManifestBaseUrl) args.push("--registry-manifest-base-url", registryManifestBaseUrl);
|
||||
}
|
||||
|
||||
if (optionValue(options, "--target-ref")) args.push("--target-ref", optionValue(options, "--target-ref"));
|
||||
if (optionValue(options, "--kubeconfig")) args.push("--kubeconfig", optionValue(options, "--kubeconfig"));
|
||||
|
||||
return {
|
||||
spec: {
|
||||
jobId,
|
||||
kind,
|
||||
cwd: repoRoot,
|
||||
node: process.execPath,
|
||||
command: process.execPath,
|
||||
args,
|
||||
envPatch,
|
||||
createdAt: isoNow(now),
|
||||
host: os.hostname(),
|
||||
paths
|
||||
},
|
||||
paths,
|
||||
env
|
||||
};
|
||||
}
|
||||
|
||||
async function defaultSpawnDetached({ repoRoot, jobId, env }) {
|
||||
const workerPath = path.join(repoRoot, "tools/hwlab-cli/lib/cicd-job-worker.mjs");
|
||||
const child = spawn(process.execPath, [workerPath, jobId], {
|
||||
cwd: repoRoot,
|
||||
env,
|
||||
detached: true,
|
||||
stdio: "ignore"
|
||||
});
|
||||
child.unref();
|
||||
return { pid: child.pid };
|
||||
}
|
||||
|
||||
async function submitJob(ctx) {
|
||||
const kind = optionValue(ctx.options, "--kind");
|
||||
const built = buildJobSpec({ ...ctx, kind });
|
||||
if (built.error) return result(1, built.error, "stderr");
|
||||
|
||||
await mkdir(built.paths.dir, { recursive: true });
|
||||
await writeJsonFile(built.paths.specPath, built.spec);
|
||||
const initialState = {
|
||||
ok: true,
|
||||
jobId: built.spec.jobId,
|
||||
kind: built.spec.kind,
|
||||
status: "submitted",
|
||||
nonBlocking: true,
|
||||
createdAt: built.spec.createdAt,
|
||||
updatedAt: built.spec.createdAt,
|
||||
cwd: built.spec.cwd,
|
||||
command: [path.basename(built.spec.command), ...built.spec.args].join(" "),
|
||||
statePath: built.paths.statePath,
|
||||
stdoutPath: built.paths.stdoutPath,
|
||||
stderrPath: built.paths.stderrPath,
|
||||
reportPath: built.paths.reportPath
|
||||
};
|
||||
await writeJsonFile(built.paths.statePath, initialState);
|
||||
|
||||
const launcher = ctx.spawnDetached ?? defaultSpawnDetached;
|
||||
const launched = await launcher({ repoRoot: ctx.repoRoot, jobId: built.spec.jobId, env: ctx.env, spec: built.spec });
|
||||
const submitted = {
|
||||
...initialState,
|
||||
pid: launched?.pid ?? null,
|
||||
followup: followup(built.spec.jobId)
|
||||
};
|
||||
await writeJsonFile(built.paths.statePath, submitted);
|
||||
return result(0, submitted);
|
||||
}
|
||||
|
||||
async function appendLog(filePath, line) {
|
||||
await writeFile(filePath, `${line}\n`, { flag: "a" });
|
||||
}
|
||||
|
||||
function reportSummary(report) {
|
||||
if (report?.artifactPublish) {
|
||||
return {
|
||||
kind: "ci-publish",
|
||||
status: report.artifactPublish.status ?? report.status ?? null,
|
||||
sourceCommitId: report.sourceCommitId ?? report.commitId ?? null,
|
||||
serviceCount: report.artifactPublish.serviceCount ?? null,
|
||||
requiredServiceCount: report.artifactPublish.requiredServiceCount ?? null,
|
||||
publishedCount: report.artifactPublish.publishedCount ?? null,
|
||||
timings: report.artifactPublish.timings ?? null,
|
||||
blockers: report.blockers ?? []
|
||||
};
|
||||
}
|
||||
if (report?.devCdApply) {
|
||||
return {
|
||||
kind: "dev-cd-apply",
|
||||
status: report.status ?? null,
|
||||
commitId: report.commitId ?? null,
|
||||
target: report.target ?? null,
|
||||
transaction: report.transaction
|
||||
? {
|
||||
transactionId: report.transaction.transactionId ?? null,
|
||||
ownerTaskId: report.transaction.ownerTaskId ?? null,
|
||||
phases: report.transaction.phases ?? [],
|
||||
release: report.transaction.release ?? null
|
||||
}
|
||||
: null,
|
||||
steps: report.devCdApply.steps?.map((step) => ({ id: step.id, status: step.status, reportPath: step.reportPath })) ?? [],
|
||||
blockers: report.blockers ?? []
|
||||
};
|
||||
}
|
||||
return {
|
||||
kind: "generic",
|
||||
status: report?.status ?? report?.ok ?? null,
|
||||
keys: report && typeof report === "object" ? Object.keys(report).slice(0, 20) : []
|
||||
};
|
||||
}
|
||||
|
||||
async function runJob(ctx) {
|
||||
const jobId = ctx.rest[0];
|
||||
if (!jobId) return result(1, error("missing-job-id", "usage: hwlab-cli cicd run-job <jobId>"), "stderr");
|
||||
const paths = jobPaths(ctx.repoRoot, jobId);
|
||||
const spec = await readJson(paths.specPath);
|
||||
const startedAt = isoNow(ctx.now);
|
||||
await writeJsonFile(paths.statePath, {
|
||||
ok: true,
|
||||
jobId,
|
||||
kind: spec.kind,
|
||||
status: "running",
|
||||
nonBlocking: true,
|
||||
createdAt: spec.createdAt,
|
||||
startedAt,
|
||||
updatedAt: startedAt,
|
||||
cwd: spec.cwd,
|
||||
pid: process.pid,
|
||||
command: [path.basename(spec.command), ...spec.args].join(" "),
|
||||
statePath: paths.statePath,
|
||||
stdoutPath: paths.stdoutPath,
|
||||
stderrPath: paths.stderrPath,
|
||||
reportPath: paths.reportPath
|
||||
});
|
||||
await appendLog(paths.stdoutPath, `[hwlab-cli] job ${jobId} started at ${startedAt}`);
|
||||
|
||||
const child = spawn(spec.command, spec.args, {
|
||||
cwd: spec.cwd,
|
||||
env: { ...process.env, ...spec.envPatch },
|
||||
stdio: ["ignore", "pipe", "pipe"]
|
||||
});
|
||||
const stdoutStream = createWriteStream(paths.stdoutPath, { flags: "a" });
|
||||
const stderrStream = createWriteStream(paths.stderrPath, { flags: "a" });
|
||||
child.stdout.pipe(stdoutStream);
|
||||
child.stderr.pipe(stderrStream);
|
||||
const exitCode = await new Promise((resolve) => {
|
||||
child.on("close", resolve);
|
||||
});
|
||||
stdoutStream.end();
|
||||
stderrStream.end();
|
||||
|
||||
const finishedAt = isoNow(ctx.now);
|
||||
const state = await readJson(paths.statePath);
|
||||
let summary = null;
|
||||
try {
|
||||
summary = reportSummary(await readJson(paths.reportPath));
|
||||
} catch {
|
||||
summary = null;
|
||||
}
|
||||
const finalState = {
|
||||
...state,
|
||||
status: exitCode === 0 ? "succeeded" : "failed",
|
||||
ok: exitCode === 0,
|
||||
exitCode,
|
||||
updatedAt: finishedAt,
|
||||
finishedAt,
|
||||
durationMs: Date.parse(finishedAt) - Date.parse(startedAt),
|
||||
reportSummary: summary,
|
||||
followup: followup(jobId)
|
||||
};
|
||||
await writeJsonFile(paths.statePath, finalState);
|
||||
return result(exitCode === 0 ? 0 : 2, finalState, exitCode === 0 ? "stdout" : "stderr");
|
||||
}
|
||||
|
||||
async function readState(repoRoot, jobId) {
|
||||
const paths = jobPaths(repoRoot, jobId);
|
||||
const state = await readJson(paths.statePath);
|
||||
return { paths, state };
|
||||
}
|
||||
|
||||
async function statusJob(ctx) {
|
||||
const jobId = ctx.rest[0];
|
||||
if (!jobId) return result(1, error("missing-job-id", "usage: hwlab-cli cicd status <jobId>"), "stderr");
|
||||
try {
|
||||
const { state } = await readState(ctx.repoRoot, jobId);
|
||||
return result(0, { ...state, followup: followup(jobId) });
|
||||
} catch (err) {
|
||||
return result(2, error("job-not-found", `No CI/CD job state found for ${jobId}.`, { detail: err.message }), "stderr");
|
||||
}
|
||||
}
|
||||
|
||||
async function tailFile(filePath, bytes) {
|
||||
try {
|
||||
const info = await stat(filePath);
|
||||
const length = Math.min(info.size, bytes);
|
||||
const handle = await open(filePath, "r");
|
||||
try {
|
||||
const buffer = Buffer.alloc(length);
|
||||
await handle.read(buffer, 0, length, info.size - length);
|
||||
return { path: filePath, totalBytes: info.size, returnedBytes: length, truncated: info.size > length, text: buffer.toString("utf8") };
|
||||
} finally {
|
||||
await handle.close();
|
||||
}
|
||||
} catch {
|
||||
return { path: filePath, totalBytes: 0, returnedBytes: 0, truncated: false, text: "" };
|
||||
}
|
||||
}
|
||||
|
||||
async function logsJob(ctx) {
|
||||
const jobId = ctx.rest[0];
|
||||
if (!jobId) return result(1, error("missing-job-id", "usage: hwlab-cli cicd logs <jobId>"), "stderr");
|
||||
const bytes = Math.min(positiveInteger(optionValue(ctx.options, "--tail-bytes"), 8000), 20000);
|
||||
try {
|
||||
const { paths, state } = await readState(ctx.repoRoot, jobId);
|
||||
return result(0, {
|
||||
ok: true,
|
||||
jobId,
|
||||
status: state.status,
|
||||
tailBytes: bytes,
|
||||
stdout: await tailFile(paths.stdoutPath, bytes),
|
||||
stderr: await tailFile(paths.stderrPath, bytes),
|
||||
followup: followup(jobId)
|
||||
});
|
||||
} catch (err) {
|
||||
return result(2, error("job-not-found", `No CI/CD job state found for ${jobId}.`, { detail: err.message }), "stderr");
|
||||
}
|
||||
}
|
||||
|
||||
async function reportJob(ctx) {
|
||||
const jobId = ctx.rest[0];
|
||||
if (!jobId) return result(1, error("missing-job-id", "usage: hwlab-cli cicd report <jobId>"), "stderr");
|
||||
try {
|
||||
const { paths, state } = await readState(ctx.repoRoot, jobId);
|
||||
const report = await readJson(paths.reportPath);
|
||||
return result(0, {
|
||||
ok: true,
|
||||
jobId,
|
||||
status: state.status,
|
||||
reportPath: paths.reportPath,
|
||||
summary: reportSummary(report),
|
||||
report: ctx.flags.has("--full") ? report : undefined,
|
||||
followup: followup(jobId)
|
||||
});
|
||||
} catch (err) {
|
||||
return result(2, error("report-not-ready", `No report is available for ${jobId}.`, { detail: err.message }), "stderr");
|
||||
}
|
||||
}
|
||||
|
||||
async function listJobs(ctx) {
|
||||
const root = path.join(ctx.repoRoot, stateRoot);
|
||||
const limit = Math.min(positiveInteger(optionValue(ctx.options, "--limit"), 20), 100);
|
||||
try {
|
||||
const names = (await readdir(root)).sort().reverse().slice(0, limit);
|
||||
const jobs = [];
|
||||
for (const name of names) {
|
||||
try {
|
||||
jobs.push((await readState(ctx.repoRoot, name)).state);
|
||||
} catch {
|
||||
// Ignore incomplete job directories; logs/report still remain on disk.
|
||||
}
|
||||
}
|
||||
return result(0, { ok: true, command: "hwlab-cli cicd jobs", count: jobs.length, jobs });
|
||||
} catch {
|
||||
return result(0, { ok: true, command: "hwlab-cli cicd jobs", count: 0, jobs: [] });
|
||||
}
|
||||
}
|
||||
|
||||
export async function runCicdCommand(ctx) {
|
||||
if (ctx.subcommand === "submit") return submitJob(ctx);
|
||||
if (ctx.subcommand === "status") return statusJob(ctx);
|
||||
if (ctx.subcommand === "logs") return logsJob(ctx);
|
||||
if (ctx.subcommand === "report") return reportJob(ctx);
|
||||
if (ctx.subcommand === "jobs") return listJobs(ctx);
|
||||
return result(1, error("usage", "usage: hwlab-cli cicd submit --kind <ci-publish|dev-cd-apply> | status|logs|report <jobId> | jobs"), "stderr");
|
||||
}
|
||||
|
||||
export async function runCicdWorker({ repoRoot = process.cwd(), jobId, now }) {
|
||||
const outcome = await runJob({
|
||||
repoRoot,
|
||||
rest: [jobId],
|
||||
now
|
||||
});
|
||||
return outcome.exitCode;
|
||||
}
|
||||
+10
-19
@@ -1,6 +1,5 @@
|
||||
import { DEV_ENDPOINT, loadMvpGateSummary } from "../../../internal/mvp-gate/summary.mjs";
|
||||
import { runM3IoSkillCommand } from "../../../skills/hwlab-agent-runtime/scripts/src/m3-io-skill-client.mjs";
|
||||
import { runCicdCommand } from "./cicd-jobs.mjs";
|
||||
|
||||
function formatJson(value) {
|
||||
return JSON.stringify(value, null, 2);
|
||||
@@ -174,12 +173,7 @@ function writeHelp(stdout) {
|
||||
"m3 status --api-base-url URL",
|
||||
"m3 io --action do.write --value true --approved --api-base-url URL",
|
||||
"m3 io --action di.read --api-base-url URL",
|
||||
"cicd submit --kind ci-publish --concurrency 4",
|
||||
"cicd submit --kind dev-cd-apply --confirm-dev --confirmed-non-production --concurrency 4",
|
||||
"cicd status JOB_ID",
|
||||
"cicd logs JOB_ID --tail-bytes 8000",
|
||||
"cicd report JOB_ID",
|
||||
"cicd jobs --limit 20",
|
||||
"G14 CI/CD is managed by native Tekton + GitOps; hwlab-cli cicd is removed",
|
||||
"test e2e --env dev --mvp",
|
||||
"test e2e --env dev --mvp --dry-run",
|
||||
"test e2e --env dev --mvp --live --confirm-dev --confirmed-non-production"
|
||||
@@ -202,18 +196,15 @@ export async function runCli(argv, io) {
|
||||
const { flags, options } = parseArgs(rest);
|
||||
|
||||
if (command === "cicd") {
|
||||
const outcome = await runCicdCommand({
|
||||
subcommand,
|
||||
rest,
|
||||
flags,
|
||||
options,
|
||||
repoRoot,
|
||||
env,
|
||||
now: io.now,
|
||||
spawnDetached: io.spawnDetached
|
||||
});
|
||||
writeJson(outcome.stream === "stderr" ? stderr : stdout, outcome.payload);
|
||||
return outcome.exitCode;
|
||||
writeJson(stderr, makeError("legacy-cicd-removed", "hwlab-cli cicd was removed from the active G14 release path. Use G14 k3s Tekton/GitOps via tran G14:k3s and scripts/g14-gitops-render.mjs.", {
|
||||
subcommand: subcommand ?? null,
|
||||
replacement: {
|
||||
observePipelineRuns: "tran G14:k3s kubectl get pipelineruns -n hwlab-ci",
|
||||
observeArgo: "tran G14:k3s kubectl -n argocd get application hwlab-g14-dev hwlab-g14-prod",
|
||||
renderSource: "node scripts/g14-gitops-render.mjs --check"
|
||||
}
|
||||
}));
|
||||
return 2;
|
||||
}
|
||||
|
||||
if (command === "m3" && ["io", "status"].includes(subcommand)) {
|
||||
|
||||
@@ -1,7 +1,4 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdtemp, readFile, writeFile } from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import test from "node:test";
|
||||
|
||||
import {
|
||||
@@ -34,80 +31,14 @@ async function captureCli(args, options = {}) {
|
||||
return { exitCode, stdout, stderr };
|
||||
}
|
||||
|
||||
test("repo hwlab-cli cicd submit is non-blocking and records follow-up commands", async () => {
|
||||
const cwd = await mkdtemp(path.join(os.tmpdir(), "hwlab-cli-cicd-"));
|
||||
const launched = [];
|
||||
const result = await captureCli(
|
||||
["cicd", "submit", "--kind", "ci-publish", "--services", "hwlab-cloud-web", "--concurrency", "2"],
|
||||
{
|
||||
cwd,
|
||||
spawnDetached: async (job) => {
|
||||
launched.push(job);
|
||||
return { pid: 123456 };
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.exitCode, 0, result.stderr);
|
||||
const body = JSON.parse(result.stdout);
|
||||
assert.equal(body.ok, true);
|
||||
assert.equal(body.nonBlocking, true);
|
||||
assert.equal(body.kind, "ci-publish");
|
||||
assert.match(body.jobId, /^ci-publish-/u);
|
||||
assert.equal(body.pid, 123456);
|
||||
assert.equal(body.followup.status, `hwlab-cli cicd status ${body.jobId}`);
|
||||
assert.equal(launched.length, 1);
|
||||
|
||||
const spec = JSON.parse(await readFile(path.join(cwd, ".state", "hwlab-cicd", "jobs", body.jobId, "spec.json"), "utf8"));
|
||||
assert.deepEqual(spec.args.slice(0, 4), ["scripts/dev-artifact-publish.mjs", "--publish", "--report", body.reportPath]);
|
||||
assert.ok(spec.args.includes("--quiet-build"));
|
||||
assert.equal(spec.envPatch.HWLAB_CI_ARTIFACT_RUN_ID, body.jobId);
|
||||
});
|
||||
|
||||
test("repo hwlab-cli cicd status/logs/report disclose bounded job state", async () => {
|
||||
const cwd = await mkdtemp(path.join(os.tmpdir(), "hwlab-cli-cicd-"));
|
||||
const submit = await captureCli(["cicd", "submit", "--kind", "ci-publish"], {
|
||||
cwd,
|
||||
spawnDetached: async () => ({ pid: 123457 })
|
||||
});
|
||||
const submitted = JSON.parse(submit.stdout);
|
||||
const jobDir = path.join(cwd, ".state", "hwlab-cicd", "jobs", submitted.jobId);
|
||||
await writeFile(path.join(jobDir, "stdout.log"), "build line 1\nbuild line 2\n", "utf8");
|
||||
await writeFile(path.join(jobDir, "stderr.log"), "warning line\n", "utf8");
|
||||
await writeFile(
|
||||
path.join(jobDir, "report.json"),
|
||||
`${JSON.stringify({
|
||||
sourceCommitId: "abc123",
|
||||
artifactPublish: {
|
||||
status: "published",
|
||||
serviceCount: 1,
|
||||
requiredServiceCount: 1,
|
||||
publishedCount: 1,
|
||||
timings: { durationMs: 42 }
|
||||
},
|
||||
blockers: []
|
||||
})}\n`,
|
||||
"utf8"
|
||||
);
|
||||
|
||||
const status = await captureCli(["cicd", "status", submitted.jobId], { cwd });
|
||||
assert.equal(status.exitCode, 0, status.stderr);
|
||||
assert.equal(JSON.parse(status.stdout).status, "submitted");
|
||||
|
||||
const logs = await captureCli(["cicd", "logs", submitted.jobId, "--tail-bytes", "12"], { cwd });
|
||||
assert.equal(logs.exitCode, 0, logs.stderr);
|
||||
const logBody = JSON.parse(logs.stdout);
|
||||
assert.equal(logBody.stdout.returnedBytes, 12);
|
||||
assert.equal(logBody.stdout.truncated, true);
|
||||
assert.ok(logBody.stdout.text.includes("line 2"));
|
||||
|
||||
const report = await captureCli(["cicd", "report", submitted.jobId], { cwd });
|
||||
assert.equal(report.exitCode, 0, report.stderr);
|
||||
const reportBody = JSON.parse(report.stdout);
|
||||
assert.equal(reportBody.summary.kind, "ci-publish");
|
||||
assert.equal(reportBody.summary.status, "published");
|
||||
assert.equal(reportBody.summary.publishedCount, 1);
|
||||
assert.equal(Object.hasOwn(reportBody, "report"), false);
|
||||
test("repo hwlab-cli cicd is removed from the active G14 release path", async () => {
|
||||
const result = await captureCli(["cicd", "submit", "--kind", "ci-publish"]);
|
||||
assert.equal(result.exitCode, 2);
|
||||
const body = JSON.parse(result.stderr);
|
||||
assert.equal(body.ok, false);
|
||||
assert.equal(body.code, "legacy-cicd-removed");
|
||||
assert.match(body.message, /G14 release path/u);
|
||||
assert.match(body.replacement.observePipelineRuns, /tran G14:k3s/u);
|
||||
});
|
||||
|
||||
test("repo hwlab-cli m3 status uses Skill CLI and calls only HWLAB API /v1/m3/status", async () => {
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
"hwlab-cli": "./bin/hwlab-cli.mjs"
|
||||
},
|
||||
"scripts": {
|
||||
"check": "node --check bin/hwlab-cli.mjs && node --check lib/cli.mjs && node --check lib/cicd-jobs.mjs && node --check lib/cicd-job-worker.mjs && node --test lib/cli.test.mjs && node ../../scripts/l6-cli-web-smoke.mjs",
|
||||
"check": "node --check bin/hwlab-cli.mjs && node --check lib/cli.mjs && node --test lib/cli.test.mjs && node ../../scripts/l6-cli-web-smoke.mjs",
|
||||
"health": "node bin/hwlab-cli.mjs health"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -56,7 +56,7 @@ const helpMarkdown = fs.readFileSync(path.resolve(rootDir, "help.md"), "utf8");
|
||||
const buildScript = fs.readFileSync(path.resolve(rootDir, "scripts/build.mjs"), "utf8");
|
||||
const distContractScript = fs.readFileSync(path.resolve(rootDir, "scripts/dist-contract.mjs"), "utf8");
|
||||
const markedLicense = fs.readFileSync(path.resolve(rootDir, "third_party/marked/LICENSE"), "utf8");
|
||||
const artifactPublisher = fs.readFileSync(path.resolve(repoRoot, "scripts/dev-artifact-publish.mjs"), "utf8");
|
||||
const artifactPublisher = fs.readFileSync(path.resolve(repoRoot, "scripts/artifact-publish.mjs"), "utf8");
|
||||
const cloudWebRoutes = fs.readFileSync(path.resolve(repoRoot, "internal/dev-entrypoint/cloud-web-routes.mjs"), "utf8");
|
||||
const cloudWebProxy = fs.readFileSync(path.resolve(repoRoot, "internal/dev-entrypoint/cloud-web-proxy.mjs"), "utf8");
|
||||
const cloudApiServer = fs.readFileSync(path.resolve(repoRoot, "internal/cloud/server.mjs"), "utf8");
|
||||
|
||||
@@ -28,7 +28,7 @@ export function runM3ControlPanelGuard() {
|
||||
const html = readText("web/hwlab-cloud-web/index.html");
|
||||
const app = readText("web/hwlab-cloud-web/app.mjs");
|
||||
const styles = readText("web/hwlab-cloud-web/styles.css");
|
||||
const artifactPublisher = readText("scripts/dev-artifact-publish.mjs");
|
||||
const artifactPublisher = readText("scripts/artifact-publish.mjs");
|
||||
const cloudWebRouteSource = readText("internal/dev-entrypoint/cloud-web-routes.mjs");
|
||||
const browserSource = `${html}\n${app}\n${styles}`;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user