diff --git a/docs/reference/spec-v02-cicd.md b/docs/reference/spec-v02-cicd.md index 20f80a7b..0f0edf39 100644 --- a/docs/reference/spec-v02-cicd.md +++ b/docs/reference/spec-v02-cicd.md @@ -10,7 +10,7 @@ ## 内部架构 -CI/CD 内部由 UniDesk 手动触发入口、PipelineRun、component planner、BuildKit publish、GitOps promotion、Argo sync 和公网验收构成。source branch 只保存源码和人写配置；`v0.2-gitops` branch 保存 catalog 和 rendered runtime desired state；live runtime 是最终通过证据。 +CI/CD 内部由 UniDesk 手动触发入口、PipelineRun、component planner、BuildKit publish、GitOps promotion、Argo sync 和公网验收构成。`v0.2` 不设置 branch poller、control-plane reconciler 或其他 CronJob；source branch 只保存源码和人写配置；`v0.2-gitops` branch 保存 catalog 和 rendered runtime desired state；live runtime 是最终通过证据。 ## API 接口说明 @@ -42,6 +42,7 @@ CI/CD 内部由 UniDesk 手动触发入口、PipelineRun、component planner、B | Runtime namespace | `hwlab-v02` | | Tekton Pipeline | `hwlab-ci/hwlab-v02-ci-image-publish` | | Manual trigger | UniDesk CLI `bun scripts/cli.ts hwlab g14 control-plane trigger-current --lane v02 --confirm` | +| Scheduler/CronJob | 不创建、不保留 v02 CronJob；发布只由手动 CLI 创建 commit-pinned PipelineRun | | Tekton ServiceAccount | `hwlab-ci/hwlab-v02-tekton-runner` | | PipelineRun prefix | `hwlab-v02-ci-poll-` | | Argo CD AppProject | `argocd/hwlab-v02` | @@ -104,6 +105,8 @@ CI/CD 内部由 UniDesk 手动触发入口、PipelineRun、component planner、B `v0.2` 可以复用 G14 的 registry、proxy、BuildKit、工具镜像和脚本库；不得复用 `hwlab-g14-ci-image-publish`、`hwlab-g14-branch-poller`、`hwlab-g14-control-plane-reconciler`、`G14-gitops` runtime path 或 DEV/PROD Argo Application 作为 `v0.2` 发布入口。 +`v0.2` 不设自动轮询发布。历史上若存在 `hwlab-v02-branch-poller`、`hwlab-v02-control-plane-reconciler` 或等价 CronJob，均视为迁移残留，应由 UniDesk control-plane apply 清理；后续发布、重跑、暂停和恢复都通过手动 CLI 触发或停止创建新的 PipelineRun 完成，不新增替代 CronJob。 + ## Env 容器复用与三变量启动 `v0.2` code-only fast lane 的 runtime desired state 由三类输入组成：可复用 env image digest、自动推导的 code boot metadata、以及既有 service runtime config。开发者仍按当前 DEV/OPS 流程提交 `v0.2` source commit 和维护 `deploy/deploy.json`；发布入口不得要求人工填写 repo、commitId 或 boot script 路径。 @@ -158,6 +161,7 @@ registry 与 git mirror/cache 分属不同基础设施边界。registry 保持 - source branch publish 后不得出现 `deploy/artifact-catalog.v02.json` 或 `deploy/gitops/g14/runtime-v02/**` 变更。 - GitOps promotion 的 changed paths 只能落在 `deploy/artifact-catalog.v02.json` 与 `deploy/gitops/g14/runtime-v02/**` 及必要的 v02 Argo/GitOps 元数据。 - 公网验收只能使用 `19666/19667`。 +- `v0.2` 不得创建或依赖 CronJob；`hwlab-v02-branch-poller`、`hwlab-v02-control-plane-reconciler` 或同类调度器出现时应清理，而不是接入发布链路。 - 旧 DEV/D601/main gate、fallback、legacy mode 和双路径兼容不得进入 `v0.2` 调用链。 - env 容器复用 fast lane 中，`HWLAB_BOOT_REPO`、`HWLAB_BOOT_COMMIT` 和 `HWLAB_BOOT_SH` 必须由 CI/CD 自动推导并写入 GitOps desired state，不得作为人工发布参数或 runtime 临时 patch。 - git mirror/cache 必须来自独立 `devops-infra` 只读服务；`hwlab-v02` runtime namespace 不部署 mirror、不持有 GitHub deploy key、不在 mirror miss 时直连 GitHub fallback。 @@ -175,13 +179,13 @@ registry 与 git mirror/cache 分属不同基础设施边界。registry 保持 - DEV `17666/17667` 与 PROD `18666/18667` FRP 入口。 - D601 legacy 回溯路径和旧运行面边界。 -如果 `v0.2` 接入失败，回滚或暂停只能作用于 `hwlab-v02` lane：停止触发新的 v02 PipelineRun、暂停或删除 `hwlab-g14-v02`、回滚 `v0.2-gitops` runtime path、关闭 `hwlab-v02-frpc` 或清理 `hwlab-v02` namespace 资源；不得重启、删除或回滚 DEV/PROD 运行面。 +如果 `v0.2` 接入失败，回滚或暂停只能作用于 `hwlab-v02` lane：停止手动触发新的 v02 PipelineRun、暂停或删除 `hwlab-g14-v02`、回滚 `v0.2-gitops` runtime path、关闭 `hwlab-v02-frpc` 或清理 `hwlab-v02` namespace 资源；不得通过新增 CronJob 维持重试，也不得重启、删除或回滚 DEV/PROD 运行面。 ## 验收标准 `v0.2` CI/CD 通过必须同时满足： -- `hwlab-ci` 中存在 `hwlab-v02-ci-image-publish` 和 `hwlab-v02-tekton-runner`；`hwlab-v02-branch-poller` 与 `hwlab-v02-control-plane-reconciler` 不再作为 v02 标准对象，若历史残留应由 UniDesk control-plane apply 清理。 +- `hwlab-ci` 中存在 `hwlab-v02-ci-image-publish` 和 `hwlab-v02-tekton-runner`；不存在 v02 CronJob。`hwlab-v02-branch-poller` 与 `hwlab-v02-control-plane-reconciler` 不再作为 v02 标准对象，若历史残留应由 UniDesk control-plane apply 清理。 - 最新 `v0.2` source commit 对应的 PipelineRun 完成，且 promotion 写入 `v0.2-gitops`。 - `v0.2-gitops` 中存在 `deploy/artifact-catalog.v02.json` 与 `deploy/gitops/g14/runtime-v02/**`。 - `argocd/hwlab-g14-v02` 指向 `v0.2-gitops:deploy/gitops/g14/runtime-v02`，sync revision 与目标 GitOps revision 对齐。 @@ -227,14 +231,14 @@ GitOps branch 已更新、source branch render 通过、PipelineRun 名称存在 | 规格项 | 状态 | 说明 | | --- | --- | --- | | v02 独立 source/GitOps/runtime lane | 已实现 | `v0.2`、`v0.2-gitops`、`hwlab-v02` 和 `runtime-v02` 已固定。 | -| 手动 CLI trigger/pipeline/promotion | 已实现 | 通过 UniDesk `trigger-current` 创建 commit-pinned PipelineRun，并由 `hwlab-v02-ci-image-publish` 与 GitOps promotion 管理。 | +| 手动 CLI trigger/pipeline/promotion | 已实现 | 通过 UniDesk `trigger-current` 创建 commit-pinned PipelineRun；v02 不设 CronJob，由 `hwlab-v02-ci-image-publish` 与 GitOps promotion 管理。 | | v02 runtime readiness fail-closed | 已实现 | `gitops-promote` 推送后触发 Argo hard refresh；`runtime-ready` 读取 workload Pod template source commit，timeout、observer RBAC 不足或未就绪会让 PipelineRun 失败。 | | v02 裁撤服务不进入发布面 | 已实现 | v02 Tekton build service set、artifact catalog 和 runtime render 只包含保留服务；v02 Argo 开启 prune 清理旧 live 对象；裁撤服务仍可保留在 DEV legacy 源码中。 | | Argo v02 Application | 已实现 | `hwlab-g14-v02` 指向 v02 GitOps path 和 namespace。 | | FRP `19666/19667` 入口 | 已实现 | 由 `hwlab-v02-frpc` 与 master frps allowlist 共同提供。 | | SecretRef 独立与 provider 验收 | 已实现/持续约束 | SecretRef 已独立；验收必须做真实短连接聊天。 | -| env 容器复用三变量启动 | 规划中 | 由 CI/CD 自动推导 `HWLAB_BOOT_REPO`、`HWLAB_BOOT_COMMIT`、`HWLAB_BOOT_SH`，code-only rollout 复用 env image digest，只更新代码身份。 | -| `devops-infra` git mirror/cache 加速 | 规划中 | 独立基础设施集群承载只读 mirror/cache；runtime namespace 不持有 GitHub deploy key，registry 保持 G14 `hwlab-ci/hwlab-registry`。 | +| env 容器复用三变量启动 | 已实现/持续约束 | device-pod fast lane 已由 CI/CD 自动推导 `HWLAB_BOOT_REPO`、`HWLAB_BOOT_COMMIT`、`HWLAB_BOOT_SH`；code-only rollout 复用 env image digest，只更新代码身份。 | +| `devops-infra` git mirror/cache 加速 | 已实现/持续约束 | runtime checkout 读路径使用独立基础设施集群只读 mirror/cache；runtime namespace 不持有 GitHub deploy key，registry 保持 G14 `hwlab-ci/hwlab-registry`。 | | 自动 registry GC | 未实现 | 初期不启用自动 GC，后续需 lane/profile 保护集。 | ## 平行 lane 运维边界 diff --git a/scripts/refresh-artifact-catalog.mjs b/scripts/refresh-artifact-catalog.mjs index 78b05488..4f2122d4 100644 --- a/scripts/refresh-artifact-catalog.mjs +++ b/scripts/refresh-artifact-catalog.mjs @@ -383,6 +383,7 @@ function publishRecordsFromReport(report, target, serviceIds = SERVICE_IDS) { imageTag: image.tag, buildCreatedAt: service.buildCreatedAt ?? null, buildSource: service.buildSource ?? null, + buildBackend: service.buildBackend ?? null, digest: service.digest, repositoryDigest: service.repositoryDigest ?? `${image.repository}@${service.digest}`, componentCommitId: service.componentCommitId ?? null, @@ -455,12 +456,15 @@ function refreshDocuments({ deploy, catalog, target, publishRecords, serviceInve const image = publishRecord?.image ?? publishRecord?.environmentImage ?? fallbackImage; const imageTag = publishRecord?.imageTag ?? parseTaggedImage(image, `${serviceId} catalog image`).tag; const serviceCommitId = shortCommitForRecord(publishRecord, target); + const sourceCommitId = revisionForRecord(publishRecord, target); catalogService.commitId = serviceCommitId; + catalogService.sourceCommitId = sourceCommitId; catalogService.image = image; catalogService.imageTag = imageTag; catalogService.buildCreatedAt = publishRecord?.buildCreatedAt ?? null; catalogService.buildSource = publishRecord?.buildSource ?? null; + catalogService.buildBackend = publishRecord ? publishRecord.buildBackend ?? null : catalogService.buildBackend ?? null; catalogService.digest = publishRecord?.digest ?? "not_published"; catalogService.componentCommitId = publishRecord?.componentCommitId ?? catalogService.componentCommitId ?? null; catalogService.componentInputHash = publishRecord?.componentInputHash ?? catalogService.componentInputHash ?? null; @@ -496,6 +500,8 @@ function refreshDocuments({ deploy, catalog, target, publishRecords, serviceInve image, imageTag, buildCreatedAt: catalogService.buildCreatedAt, + sourceCommitId: catalogService.sourceCommitId, + buildBackend: catalogService.buildBackend, digest: catalogService.digest, componentCommitId: catalogService.componentCommitId, componentInputHash: catalogService.componentInputHash, @@ -565,6 +571,10 @@ async function main() { await writeJson(args.catalogPath, catalog); } + const publishedRecords = [...(publishRecords?.records.values() ?? [])]; + const publishedCount = publishedRecords.filter((record) => record.status === "published").length; + const reusedCount = publishedRecords.filter((record) => record.status === "reused").length; + console.log(JSON.stringify({ status: publishRecords ? "published" : "blocked", targetRef: target.ref, @@ -577,7 +587,8 @@ async function main() { deployTruthPolicy: "deploy.json is human-authored runtime config; artifact promotion must not rewrite it", ciPublished: Boolean(publishRecords), registryVerified: Boolean(publishRecords), - publishedCount: publishRecords?.records.size ?? 0, + publishedCount, + reusedCount, requiredServiceCount: serviceInventory.requiredServiceCount, disabledServiceCount: serviceInventory.disabledServiceCount, notPublishedCount: services.filter((service) => service.digest === "not_published").length, diff --git a/scripts/refresh-artifact-catalog.test.mjs b/scripts/refresh-artifact-catalog.test.mjs index c5769f36..6a7b610f 100644 --- a/scripts/refresh-artifact-catalog.test.mjs +++ b/scripts/refresh-artifact-catalog.test.mjs @@ -229,6 +229,8 @@ test("refresh keeps reused service image tags from the publish report", async () }); const payload = JSON.parse(result.stdout); assert.equal(payload.status, "published"); + assert.equal(payload.publishedCount, 1); + assert.equal(payload.reusedCount, SERVICE_IDS.length - 1); assert.equal(payload.services[0].imageTag, shortCommitId); assert.equal(payload.services[1].imageTag, reusedTag); assert.equal(payload.services[1].publishState, "reused"); @@ -351,3 +353,134 @@ test("v02 refresh accepts env-reuse device-pod image tag from environment hash", assert.equal(devicePod.bootCommit, commitId); assert.equal(devicePod.bootSh, "deploy/runtime/boot/hwlab-device-pod.sh"); }); + +test("v02 refresh reports code-only env-reuse records as reused", async () => { + const commitId = await git(["rev-parse", "HEAD"]); + const previousCommitId = "1234567890abcdef1234567890abcdef12345678"; + const environmentInputHash = "3e03261df109abfc804a4d776b6e8d59d5c71d84268adecf97ee2a420f2747c9"; + const codeInputHash = "dbc39fd148d0cb27551400eccb2f7027a614de3ce3c9a0054c407c26478a92d1"; + const envTag = `env-${environmentInputHash.slice(0, 12)}`; + const tempDir = await mkdtemp(path.join(os.tmpdir(), "hwlab-refresh-v02-code-only-reuse-")); + const reportPath = path.join(tempDir, "v02-artifacts.json"); + await writeFile(reportPath, `${JSON.stringify({ + reportVersion: "v1", + taskId: "v02-artifact-publish", + commitId, + artifactPublish: { + status: "published", + mode: "publish", + sourceCommitId: commitId, + registryPrefix: "127.0.0.1:5000/hwlab", + serviceCount: V02_SERVICE_IDS.length, + requiredServiceCount: V02_SERVICE_IDS.length, + disabledServiceCount: 0, + publishedCount: 0, + reusedCount: V02_SERVICE_IDS.length, + serviceInventory: { + version: "v2", + serviceCount: V02_SERVICE_IDS.length, + requiredServiceCount: V02_SERVICE_IDS.length, + disabledServiceCount: 0, + requiredServiceIds: V02_SERVICE_IDS, + disabledServiceIds: [], + services: V02_SERVICE_IDS.map((serviceId) => ({ + serviceId, + publishEnabled: true, + artifactRequired: true, + artifactScope: "required", + runtimeKind: serviceId === "hwlab-device-pod" ? "env-reuse" : "node-command", + implementationState: "repo-entrypoint", + sourceState: "source-present", + entrypoint: `cmd/${serviceId}/main.ts`, + disabledReason: null + })) + }, + publishPlan: { + version: "v2", + services: V02_SERVICE_IDS.map((serviceId, index) => ({ + serviceId, + required: true, + status: "reused", + digest: digestFor(index), + image: serviceId === "hwlab-device-pod" + ? `127.0.0.1:5000/hwlab/hwlab-device-pod-env:${envTag}` + : `127.0.0.1:5000/hwlab/${serviceId}:${previousCommitId}`, + imageTag: serviceId === "hwlab-device-pod" ? envTag : previousCommitId + })) + }, + services: V02_SERVICE_IDS.map((serviceId, index) => { + if (serviceId !== "hwlab-device-pod") { + return { + serviceId, + status: "reused", + artifactRequired: true, + commitId: previousCommitId, + sourceCommitId: previousCommitId, + image: `127.0.0.1:5000/hwlab/${serviceId}:${previousCommitId}`, + imageTag: previousCommitId, + digest: digestFor(index), + repositoryDigest: `127.0.0.1:5000/hwlab/${serviceId}@${digestFor(index)}`, + buildBackend: "reused-catalog", + reusedFrom: `catalog:${previousCommitId}` + }; + } + return { + serviceId, + status: "reused", + artifactRequired: true, + commitId: envTag, + sourceCommitId: commitId, + image: `127.0.0.1:5000/hwlab/hwlab-device-pod-env:${envTag}`, + imageTag: envTag, + digest: digestFor(index), + repositoryDigest: `127.0.0.1:5000/hwlab/hwlab-device-pod-env@${digestFor(index)}`, + buildBackend: "reused-env-catalog", + runtimeMode: "env-reuse-git-mirror-checkout", + envReuse: true, + envChanged: false, + codeChanged: true, + environmentImage: `127.0.0.1:5000/hwlab/hwlab-device-pod-env:${envTag}`, + environmentDigest: digestFor(index), + environmentInputHash, + codeInputHash, + bootRepo: "git@github.com:pikasTech/HWLAB.git", + bootCommit: commitId, + bootSh: "deploy/runtime/boot/hwlab-device-pod.sh", + reusedFrom: `env-catalog:${envTag}` + }; + }) + } + }, null, 2)}\n`); + + const result = await execFileAsync(process.execPath, [ + "scripts/refresh-artifact-catalog.mjs", + "--lane", + "v02", + "--image-tag-mode", + "full", + "--target-ref", + "HEAD", + "--catalog-path", + path.join(tempDir, "artifact-catalog.v02.json"), + "--publish-report", + reportPath, + "--no-write" + ], { + cwd: repoRoot, + timeout: 15000, + maxBuffer: 10 * 1024 * 1024 + }); + const payload = JSON.parse(result.stdout); + const devicePod = payload.services.find((service) => service.serviceId === "hwlab-device-pod"); + assert.equal(payload.status, "published"); + assert.equal(payload.publishedCount, 0); + assert.equal(payload.reusedCount, V02_SERVICE_IDS.length); + assert.equal(devicePod.publishState, "reused"); + assert.equal(devicePod.buildBackend, "reused-env-catalog"); + assert.equal(devicePod.sourceCommitId, commitId); + assert.equal(devicePod.imageTag, envTag); + assert.equal(devicePod.runtimeMode, "env-reuse-git-mirror-checkout"); + assert.equal(devicePod.environmentInputHash, environmentInputHash); + assert.equal(devicePod.codeInputHash, codeInputHash); + assert.equal(devicePod.bootCommit, commitId); +});