Merge pull request #594 from pikasTech/fix/v02-publish-reuse-counts

fix: report v02 env reuse publishes accurately
2026-05-30 02:05:25 +08:00
parent cde83efa07 72d80b35a4
commit d1457bda4c
3 changed files with 155 additions and 7 deletions
@@ -10,7 +10,7 @@

 ## 内部架构

-CI/CD 内部由 UniDesk 手动触发入口、PipelineRun、component planner、BuildKit publish、GitOps promotion、Argo sync 和公网验收构成。source branch 只保存源码和人写配置；`v0.2-gitops` branch 保存 catalog 和 rendered runtime desired state；live runtime 是最终通过证据。
+CI/CD 内部由 UniDesk 手动触发入口、PipelineRun、component planner、BuildKit publish、GitOps promotion、Argo sync 和公网验收构成。`v0.2` 不设置 branch poller、control-plane reconciler 或其他 CronJob；source branch 只保存源码和人写配置；`v0.2-gitops` branch 保存 catalog 和 rendered runtime desired state；live runtime 是最终通过证据。

 ## API 接口说明

@@ -42,6 +42,7 @@ CI/CD 内部由 UniDesk 手动触发入口、PipelineRun、component planner、B
 | Runtime namespace | `hwlab-v02` |
 | Tekton Pipeline | `hwlab-ci/hwlab-v02-ci-image-publish` |
 | Manual trigger | UniDesk CLI `bun scripts/cli.ts hwlab g14 control-plane trigger-current --lane v02 --confirm` |
+| Scheduler/CronJob | 不创建、不保留 v02 CronJob；发布只由手动 CLI 创建 commit-pinned PipelineRun |
 | Tekton ServiceAccount | `hwlab-ci/hwlab-v02-tekton-runner` |
 | PipelineRun prefix | `hwlab-v02-ci-poll-<short12>` |
 | Argo CD AppProject | `argocd/hwlab-v02` |
@@ -104,6 +105,8 @@ CI/CD 内部由 UniDesk 手动触发入口、PipelineRun、component planner、B

 `v0.2` 可以复用 G14 的 registry、proxy、BuildKit、工具镜像和脚本库；不得复用 `hwlab-g14-ci-image-publish`、`hwlab-g14-branch-poller`、`hwlab-g14-control-plane-reconciler`、`G14-gitops` runtime path 或 DEV/PROD Argo Application 作为 `v0.2` 发布入口。

+`v0.2` 不设自动轮询发布。历史上若存在 `hwlab-v02-branch-poller`、`hwlab-v02-control-plane-reconciler` 或等价 CronJob，均视为迁移残留，应由 UniDesk control-plane apply 清理；后续发布、重跑、暂停和恢复都通过手动 CLI 触发或停止创建新的 PipelineRun 完成，不新增替代 CronJob。
+
 ## Env 容器复用与三变量启动

 `v0.2` code-only fast lane 的 runtime desired state 由三类输入组成：可复用 env image digest、自动推导的 code boot metadata、以及既有 service runtime config。开发者仍按当前 DEV/OPS 流程提交 `v0.2` source commit 和维护 `deploy/deploy.json`；发布入口不得要求人工填写 repo、commitId 或 boot script 路径。
@@ -158,6 +161,7 @@ registry 与 git mirror/cache 分属不同基础设施边界。registry 保持
 - source branch publish 后不得出现 `deploy/artifact-catalog.v02.json` 或 `deploy/gitops/g14/runtime-v02/**` 变更。
 - GitOps promotion 的 changed paths 只能落在 `deploy/artifact-catalog.v02.json` 与 `deploy/gitops/g14/runtime-v02/**` 及必要的 v02 Argo/GitOps 元数据。
 - 公网验收只能使用 `19666/19667`。
+- `v0.2` 不得创建或依赖 CronJob；`hwlab-v02-branch-poller`、`hwlab-v02-control-plane-reconciler` 或同类调度器出现时应清理，而不是接入发布链路。
 - 旧 DEV/D601/main gate、fallback、legacy mode 和双路径兼容不得进入 `v0.2` 调用链。
 - env 容器复用 fast lane 中，`HWLAB_BOOT_REPO`、`HWLAB_BOOT_COMMIT` 和 `HWLAB_BOOT_SH` 必须由 CI/CD 自动推导并写入 GitOps desired state，不得作为人工发布参数或 runtime 临时 patch。
 - git mirror/cache 必须来自独立 `devops-infra` 只读服务；`hwlab-v02` runtime namespace 不部署 mirror、不持有 GitHub deploy key、不在 mirror miss 时直连 GitHub fallback。
@@ -175,13 +179,13 @@ registry 与 git mirror/cache 分属不同基础设施边界。registry 保持
 - DEV `17666/17667` 与 PROD `18666/18667` FRP 入口。
 - D601 legacy 回溯路径和旧运行面边界。

-如果 `v0.2` 接入失败，回滚或暂停只能作用于 `hwlab-v02` lane：停止触发新的 v02 PipelineRun、暂停或删除 `hwlab-g14-v02`、回滚 `v0.2-gitops` runtime path、关闭 `hwlab-v02-frpc` 或清理 `hwlab-v02` namespace 资源；不得重启、删除或回滚 DEV/PROD 运行面。
+如果 `v0.2` 接入失败，回滚或暂停只能作用于 `hwlab-v02` lane：停止手动触发新的 v02 PipelineRun、暂停或删除 `hwlab-g14-v02`、回滚 `v0.2-gitops` runtime path、关闭 `hwlab-v02-frpc` 或清理 `hwlab-v02` namespace 资源；不得通过新增 CronJob 维持重试，也不得重启、删除或回滚 DEV/PROD 运行面。

 ## 验收标准

 `v0.2` CI/CD 通过必须同时满足：

- `hwlab-ci` 中存在 `hwlab-v02-ci-image-publish` 和 `hwlab-v02-tekton-runner`；`hwlab-v02-branch-poller` 与 `hwlab-v02-control-plane-reconciler` 不再作为 v02 标准对象，若历史残留应由 UniDesk control-plane apply 清理。
+- `hwlab-ci` 中存在 `hwlab-v02-ci-image-publish` 和 `hwlab-v02-tekton-runner`；不存在 v02 CronJob。`hwlab-v02-branch-poller` 与 `hwlab-v02-control-plane-reconciler` 不再作为 v02 标准对象，若历史残留应由 UniDesk control-plane apply 清理。
 - 最新 `v0.2` source commit 对应的 PipelineRun 完成，且 promotion 写入 `v0.2-gitops`。
 - `v0.2-gitops` 中存在 `deploy/artifact-catalog.v02.json` 与 `deploy/gitops/g14/runtime-v02/**`。
 - `argocd/hwlab-g14-v02` 指向 `v0.2-gitops:deploy/gitops/g14/runtime-v02`，sync revision 与目标 GitOps revision 对齐。
@@ -227,14 +231,14 @@ GitOps branch 已更新、source branch render 通过、PipelineRun 名称存在
 | 规格项 | 状态 | 说明 |
 | --- | --- | --- |
 | v02 独立 source/GitOps/runtime lane | 已实现 | `v0.2`、`v0.2-gitops`、`hwlab-v02` 和 `runtime-v02` 已固定。 |
-| 手动 CLI trigger/pipeline/promotion | 已实现 | 通过 UniDesk `trigger-current` 创建 commit-pinned PipelineRun，并由 `hwlab-v02-ci-image-publish` 与 GitOps promotion 管理。 |
+| 手动 CLI trigger/pipeline/promotion | 已实现 | 通过 UniDesk `trigger-current` 创建 commit-pinned PipelineRun；v02 不设 CronJob，由 `hwlab-v02-ci-image-publish` 与 GitOps promotion 管理。 |
 | v02 runtime readiness fail-closed | 已实现 | `gitops-promote` 推送后触发 Argo hard refresh；`runtime-ready` 读取 workload Pod template source commit，timeout、observer RBAC 不足或未就绪会让 PipelineRun 失败。 |
 | v02 裁撤服务不进入发布面 | 已实现 | v02 Tekton build service set、artifact catalog 和 runtime render 只包含保留服务；v02 Argo 开启 prune 清理旧 live 对象；裁撤服务仍可保留在 DEV legacy 源码中。 |
 | Argo v02 Application | 已实现 | `hwlab-g14-v02` 指向 v02 GitOps path 和 namespace。 |
 | FRP `19666/19667` 入口 | 已实现 | 由 `hwlab-v02-frpc` 与 master frps allowlist 共同提供。 |
 | SecretRef 独立与 provider 验收 | 已实现/持续约束 | SecretRef 已独立；验收必须做真实短连接聊天。 |
-| env 容器复用三变量启动 | 规划中 | 由 CI/CD 自动推导 `HWLAB_BOOT_REPO`、`HWLAB_BOOT_COMMIT`、`HWLAB_BOOT_SH`，code-only rollout 复用 env image digest，只更新代码身份。 |
-| `devops-infra` git mirror/cache 加速 | 规划中 | 独立基础设施集群承载只读 mirror/cache；runtime namespace 不持有 GitHub deploy key，registry 保持 G14 `hwlab-ci/hwlab-registry`。 |
+| env 容器复用三变量启动 | 已实现/持续约束 | device-pod fast lane 已由 CI/CD 自动推导 `HWLAB_BOOT_REPO`、`HWLAB_BOOT_COMMIT`、`HWLAB_BOOT_SH`；code-only rollout 复用 env image digest，只更新代码身份。 |
+| `devops-infra` git mirror/cache 加速 | 已实现/持续约束 | runtime checkout 读路径使用独立基础设施集群只读 mirror/cache；runtime namespace 不持有 GitHub deploy key，registry 保持 G14 `hwlab-ci/hwlab-registry`。 |
 | 自动 registry GC | 未实现 | 初期不启用自动 GC，后续需 lane/profile 保护集。 |

 ## 平行 lane 运维边界
@@ -383,6 +383,7 @@ function publishRecordsFromReport(report, target, serviceIds = SERVICE_IDS) {
      imageTag: image.tag,
      buildCreatedAt: service.buildCreatedAt ?? null,
      buildSource: service.buildSource ?? null,
+      buildBackend: service.buildBackend ?? null,
      digest: service.digest,
      repositoryDigest: service.repositoryDigest ?? `${image.repository}@${service.digest}`,
      componentCommitId: service.componentCommitId ?? null,
@@ -455,12 +456,15 @@ function refreshDocuments({ deploy, catalog, target, publishRecords, serviceInve
    const image = publishRecord?.image ?? publishRecord?.environmentImage ?? fallbackImage;
    const imageTag = publishRecord?.imageTag ?? parseTaggedImage(image, `${serviceId} catalog image`).tag;
    const serviceCommitId = shortCommitForRecord(publishRecord, target);
+    const sourceCommitId = revisionForRecord(publishRecord, target);

    catalogService.commitId = serviceCommitId;
+    catalogService.sourceCommitId = sourceCommitId;
    catalogService.image = image;
    catalogService.imageTag = imageTag;
    catalogService.buildCreatedAt = publishRecord?.buildCreatedAt ?? null;
    catalogService.buildSource = publishRecord?.buildSource ?? null;
+    catalogService.buildBackend = publishRecord ? publishRecord.buildBackend ?? null : catalogService.buildBackend ?? null;
    catalogService.digest = publishRecord?.digest ?? "not_published";
    catalogService.componentCommitId = publishRecord?.componentCommitId ?? catalogService.componentCommitId ?? null;
    catalogService.componentInputHash = publishRecord?.componentInputHash ?? catalogService.componentInputHash ?? null;
@@ -496,6 +500,8 @@ function refreshDocuments({ deploy, catalog, target, publishRecords, serviceInve
      image,
      imageTag,
      buildCreatedAt: catalogService.buildCreatedAt,
+      sourceCommitId: catalogService.sourceCommitId,
+      buildBackend: catalogService.buildBackend,
      digest: catalogService.digest,
      componentCommitId: catalogService.componentCommitId,
      componentInputHash: catalogService.componentInputHash,
@@ -565,6 +571,10 @@ async function main() {
    await writeJson(args.catalogPath, catalog);
  }

+  const publishedRecords = [...(publishRecords?.records.values() ?? [])];
+  const publishedCount = publishedRecords.filter((record) => record.status === "published").length;
+  const reusedCount = publishedRecords.filter((record) => record.status === "reused").length;
+
  console.log(JSON.stringify({
    status: publishRecords ? "published" : "blocked",
    targetRef: target.ref,
@@ -577,7 +587,8 @@ async function main() {
    deployTruthPolicy: "deploy.json is human-authored runtime config; artifact promotion must not rewrite it",
    ciPublished: Boolean(publishRecords),
    registryVerified: Boolean(publishRecords),
-    publishedCount: publishRecords?.records.size ?? 0,
+    publishedCount,
+    reusedCount,
    requiredServiceCount: serviceInventory.requiredServiceCount,
    disabledServiceCount: serviceInventory.disabledServiceCount,
    notPublishedCount: services.filter((service) => service.digest === "not_published").length,
@@ -229,6 +229,8 @@ test("refresh keeps reused service image tags from the publish report", async ()
  });
  const payload = JSON.parse(result.stdout);
  assert.equal(payload.status, "published");
+  assert.equal(payload.publishedCount, 1);
+  assert.equal(payload.reusedCount, SERVICE_IDS.length - 1);
  assert.equal(payload.services[0].imageTag, shortCommitId);
  assert.equal(payload.services[1].imageTag, reusedTag);
  assert.equal(payload.services[1].publishState, "reused");
@@ -351,3 +353,134 @@ test("v02 refresh accepts env-reuse device-pod image tag from environment hash",
  assert.equal(devicePod.bootCommit, commitId);
  assert.equal(devicePod.bootSh, "deploy/runtime/boot/hwlab-device-pod.sh");
 });
+
+test("v02 refresh reports code-only env-reuse records as reused", async () => {
+  const commitId = await git(["rev-parse", "HEAD"]);
+  const previousCommitId = "1234567890abcdef1234567890abcdef12345678";
+  const environmentInputHash = "3e03261df109abfc804a4d776b6e8d59d5c71d84268adecf97ee2a420f2747c9";
+  const codeInputHash = "dbc39fd148d0cb27551400eccb2f7027a614de3ce3c9a0054c407c26478a92d1";
+  const envTag = `env-${environmentInputHash.slice(0, 12)}`;
+  const tempDir = await mkdtemp(path.join(os.tmpdir(), "hwlab-refresh-v02-code-only-reuse-"));
+  const reportPath = path.join(tempDir, "v02-artifacts.json");
+  await writeFile(reportPath, `${JSON.stringify({
+    reportVersion: "v1",
+    taskId: "v02-artifact-publish",
+    commitId,
+    artifactPublish: {
+      status: "published",
+      mode: "publish",
+      sourceCommitId: commitId,
+      registryPrefix: "127.0.0.1:5000/hwlab",
+      serviceCount: V02_SERVICE_IDS.length,
+      requiredServiceCount: V02_SERVICE_IDS.length,
+      disabledServiceCount: 0,
+      publishedCount: 0,
+      reusedCount: V02_SERVICE_IDS.length,
+      serviceInventory: {
+        version: "v2",
+        serviceCount: V02_SERVICE_IDS.length,
+        requiredServiceCount: V02_SERVICE_IDS.length,
+        disabledServiceCount: 0,
+        requiredServiceIds: V02_SERVICE_IDS,
+        disabledServiceIds: [],
+        services: V02_SERVICE_IDS.map((serviceId) => ({
+          serviceId,
+          publishEnabled: true,
+          artifactRequired: true,
+          artifactScope: "required",
+          runtimeKind: serviceId === "hwlab-device-pod" ? "env-reuse" : "node-command",
+          implementationState: "repo-entrypoint",
+          sourceState: "source-present",
+          entrypoint: `cmd/${serviceId}/main.ts`,
+          disabledReason: null
+        }))
+      },
+      publishPlan: {
+        version: "v2",
+        services: V02_SERVICE_IDS.map((serviceId, index) => ({
+          serviceId,
+          required: true,
+          status: "reused",
+          digest: digestFor(index),
+          image: serviceId === "hwlab-device-pod"
+            ? `127.0.0.1:5000/hwlab/hwlab-device-pod-env:${envTag}`
+            : `127.0.0.1:5000/hwlab/${serviceId}:${previousCommitId}`,
+          imageTag: serviceId === "hwlab-device-pod" ? envTag : previousCommitId
+        }))
+      },
+      services: V02_SERVICE_IDS.map((serviceId, index) => {
+        if (serviceId !== "hwlab-device-pod") {
+          return {
+            serviceId,
+            status: "reused",
+            artifactRequired: true,
+            commitId: previousCommitId,
+            sourceCommitId: previousCommitId,
+            image: `127.0.0.1:5000/hwlab/${serviceId}:${previousCommitId}`,
+            imageTag: previousCommitId,
+            digest: digestFor(index),
+            repositoryDigest: `127.0.0.1:5000/hwlab/${serviceId}@${digestFor(index)}`,
+            buildBackend: "reused-catalog",
+            reusedFrom: `catalog:${previousCommitId}`
+          };
+        }
+        return {
+          serviceId,
+          status: "reused",
+          artifactRequired: true,
+          commitId: envTag,
+          sourceCommitId: commitId,
+          image: `127.0.0.1:5000/hwlab/hwlab-device-pod-env:${envTag}`,
+          imageTag: envTag,
+          digest: digestFor(index),
+          repositoryDigest: `127.0.0.1:5000/hwlab/hwlab-device-pod-env@${digestFor(index)}`,
+          buildBackend: "reused-env-catalog",
+          runtimeMode: "env-reuse-git-mirror-checkout",
+          envReuse: true,
+          envChanged: false,
+          codeChanged: true,
+          environmentImage: `127.0.0.1:5000/hwlab/hwlab-device-pod-env:${envTag}`,
+          environmentDigest: digestFor(index),
+          environmentInputHash,
+          codeInputHash,
+          bootRepo: "git@github.com:pikasTech/HWLAB.git",
+          bootCommit: commitId,
+          bootSh: "deploy/runtime/boot/hwlab-device-pod.sh",
+          reusedFrom: `env-catalog:${envTag}`
+        };
+      })
+    }
+  }, null, 2)}\n`);
+
+  const result = await execFileAsync(process.execPath, [
+    "scripts/refresh-artifact-catalog.mjs",
+    "--lane",
+    "v02",
+    "--image-tag-mode",
+    "full",
+    "--target-ref",
+    "HEAD",
+    "--catalog-path",
+    path.join(tempDir, "artifact-catalog.v02.json"),
+    "--publish-report",
+    reportPath,
+    "--no-write"
+  ], {
+    cwd: repoRoot,
+    timeout: 15000,
+    maxBuffer: 10 * 1024 * 1024
+  });
+  const payload = JSON.parse(result.stdout);
+  const devicePod = payload.services.find((service) => service.serviceId === "hwlab-device-pod");
+  assert.equal(payload.status, "published");
+  assert.equal(payload.publishedCount, 0);
+  assert.equal(payload.reusedCount, V02_SERVICE_IDS.length);
+  assert.equal(devicePod.publishState, "reused");
+  assert.equal(devicePod.buildBackend, "reused-env-catalog");
+  assert.equal(devicePod.sourceCommitId, commitId);
+  assert.equal(devicePod.imageTag, envTag);
+  assert.equal(devicePod.runtimeMode, "env-reuse-git-mirror-checkout");
+  assert.equal(devicePod.environmentInputHash, environmentInputHash);
+  assert.equal(devicePod.codeInputHash, codeInputHash);
+  assert.equal(devicePod.bootCommit, commitId);
+});