114 lines
10 KiB
Markdown
114 lines
10 KiB
Markdown
# Env Reuse
|
||
|
||
Env reuse is the CI/CD optimization that decides whether a pipeline can reuse an already prepared environment artifact or dependency set instead of rebuilding the same environment from scratch. It is not a single implementation. In UniDesk today it has two concrete shapes:
|
||
|
||
- AgentRun manager image reuse: compare source-declared identity files, reuse an existing image artifact when the environment identity is unchanged, and skip the image build stage.
|
||
- Web sentinel dependency reuse: link a node-local dependency directory into the build workspace and rely on BuildKit state for image build acceleration.
|
||
|
||
Do not collapse these into one meaning. A status value of `reused`, `hit`, `skipped`, or `cache=hit` must be interpreted in the context of the consumer that produced it.
|
||
|
||
## Source Of Truth
|
||
|
||
The reusable CI/CD contract is declared in `gitops/reuse.ymal` inside the source repository snapshot consumed by CI. UniDesk parses it through `scripts/src/cicd-reuse-config.ts`; helper script parity exists in `scripts/native/cicd/reuse-config-summary.mjs` and branch-follower gate code.
|
||
|
||
The relevant service fields are:
|
||
|
||
- `runtimeReuse`: source/runtime identity comparison. It can declare `codeIdentity.paths` and `envIdentity.paths`.
|
||
- `envReuse`: environment identity and dependency reuse declaration. It can declare `enabled`, `mode`, `nodeDepsPath`, `envIdentityFiles`, and `buildArgs`.
|
||
|
||
All paths in `gitops/reuse.ymal` are relative to the source tree, except runtime dependency locations such as a node-local `nodeDepsPath` where the owning runtime YAML explicitly declares the path. Do not invent default path lists in code or docs. If a service requires env reuse, use `requiredReuseServiceError(..., "envReuse")` style validation so missing declarations are reported before a pipeline silently falls back.
|
||
|
||
## AgentRun
|
||
|
||
AgentRun manager env reuse is identity based. `scripts/src/cicd-agentrun-reuse.ts` reads the `agentrun-mgr` or `manager` service from `gitops/reuse.ymal` and computes a reuse decision against the current snapshot and its parent commit:
|
||
|
||
- `sourceIdentity` hashes configured code identity paths.
|
||
- `envIdentity` hashes configured env identity paths plus `envReuse.envIdentityFiles`.
|
||
- `runtimeReuse` hits only when both source and env identity hit.
|
||
- `envReuse` hits when env identity hits.
|
||
- Either hit can set `skipImageBuild=true`.
|
||
|
||
When the build is skipped, AgentRun consumes the existing artifact catalog entry from the GitOps branch instead of creating a new image. The artifact must still provide image, digest, and env identity for `agentrun-mgr`; otherwise reuse is invalid and the pipeline must build or fail visibly. Passing evidence is `IMAGE_STATUS=reused`, a stable `ENV_ID`, a digest, and a GitOps commit from `platform-infra pipelines-as-code status|history`.
|
||
|
||
AgentRun Tekton also has an in-pipeline image probe path in `scripts/src/agentrun-manifests.ts`: checkout computes an env identity from configured files, probes the local registry for an image tagged by that identity, writes `status=reused` and `skip-build` on hit, otherwise runs BuildKit and writes `status=built`.
|
||
|
||
## Web Sentinel
|
||
|
||
Web sentinel env reuse is dependency reuse, not whole-image reuse. `scripts/src/hwlab-node-web-sentinel-cicd.ts` overlays `gitops/reuse.ymal` service ids `web-probe-sentinel` or `monitor-web` onto the sentinel `monitorWeb.envReuse` config. The actual publish job in `scripts/src/hwlab-node-web-sentinel-cicd-jobs.ts`:
|
||
|
||
- checks whether the YAML-selected `nodeDepsPath` exists on the target node/build workspace;
|
||
- removes workspace `node_modules`;
|
||
- symlinks packages from the reusable dependency directory into workspace `node_modules`;
|
||
- emits `sentinel-publish-env-reuse` with `dependencyReuse=hit|miss`, entry counts, and linked count;
|
||
- then verifies the monitor web assets and builds the image through the configured BuildKit path.
|
||
|
||
For sentinel, `ENV_REUSE=hit` means the dependency directory was present and linked. It does not mean the image build was skipped. The image may still be published in the same run; separate BuildKit evidence such as `cache=hit` describes layer/cache behavior.
|
||
|
||
## Gitea PaC Observation
|
||
|
||
Gitea/Pipelines-as-Code is the current operator-facing observation surface for migrated JD01 consumers. The canonical commands are:
|
||
|
||
```bash
|
||
bun scripts/cli.ts platform-infra pipelines-as-code status --target JD01 [--consumer <id>]
|
||
bun scripts/cli.ts platform-infra pipelines-as-code history --target JD01 [--consumer <id>] [--limit 10]
|
||
bun scripts/cli.ts platform-infra pipelines-as-code history --target JD01 --id <pipelinerun>
|
||
```
|
||
|
||
`status` reports latest PipelineRun, TaskRun durations, image status, env identity, digest, GitOps revision, Argo health, and runtime image. `history` reads live Gitea Repository CR plus Tekton PipelineRun/TaskRun objects on the target node, aggregates there, and returns Beijing-time display according to `config/platform-infra/pipelines-as-code.yaml`.
|
||
|
||
The history/status env reuse display is log-derived:
|
||
|
||
- JSON records with `envReuse`, `envReuseStatus`, `buildSkippedCount`, `serviceReusedCount`, or equivalent fields are parsed.
|
||
- Sentinel's compact table headed `ENV_REUSE NODE_DEPS ...` is parsed as human-readable fallback.
|
||
- Missing env reuse text is not proof that no reuse happened; first check the consumer type, the relevant TaskRun logs, and whether the pipeline emits the expected fields.
|
||
|
||
## 自动交付预算告警
|
||
|
||
- 自动交付端到端预算的唯一真相是 `config/platform-infra/pipelines-as-code.yaml#deliveryTiming.endToEndBudgetSeconds`。
|
||
- `delivery-timing` 只有在 PR `mergedAt`、PipelineRun 时间、Argo、runtime、health 和 provenance 证据完整时才判定预算。
|
||
- 证据不完整时状态固定为 `partial`,`overBudget=null`,不得提前输出超预算告警。
|
||
- 严格超过预算时输出 `pac-automatic-delivery-over-budget` typed warning:
|
||
- `blocking=false`;
|
||
- `mutation=false`;
|
||
- 不改变 PipelineRun、artifact、GitOps、Argo、runtime 或 health 的成功终态。
|
||
- 告警只指向现有只读 `status`、`history` 和 `delivery-timing` 入口,并提示检查:
|
||
- env identity;
|
||
- 依赖缓存;
|
||
- BuildKit/cache;
|
||
- stage timing。
|
||
- `status/history` 复用同一预算摘要;只有流水线阶段耗时而缺少完整端到端证据时仍显示 `partial`,不得把 PipelineRun 完成等同于 runtime closeout 完成。
|
||
|
||
## Interpreting Results
|
||
|
||
Use these interpretations:
|
||
|
||
- AgentRun `IMAGE_STATUS=reused`: image build was skipped or registry artifact reused for the same env identity.
|
||
- AgentRun `IMAGE_STATUS=built`: env identity did not hit or registry/artifact probe missed; a new image was built.
|
||
- Sentinel `ENV_REUSE=hit`: node dependency directory was present and linked before verify/build.
|
||
- Sentinel `cache=hit`: BuildKit cache evidence, separate from dependency reuse.
|
||
- HWLAB v0.3 `skipped,skip=<n>` in PaC history: service-level artifact planning skipped builds for already reusable services; inspect the detail id before interpreting it as AgentRun-style image reuse.
|
||
- HWLAB v0.3 `env-reuse-gitea-checkout`: a `codeIdentity` change with unchanged `envIdentity` should mark the service as affected/rolled out while still skipping image build. Evidence is in `plan-artifacts` and `collect-artifacts`: `affectedServices` includes the service, `buildServices=[]`, the artifact has `ciAffected=true`, `ciReason` such as `code-input-changed`, a new `componentCommitId`/`codeInputHash`, and `publishState=reused` / `buildBackend=reused-env-catalog`. This is correct env reuse consumption, not a missed rebuild.
|
||
- HWLAB v0.3 shared TS runtime env groups build one representative environment image and let compatible TS services consume that same artifact with different runtime checkout/entrypoint metadata. A semantic env identity change, such as dependency maps, `engines`, `packageManager`, `overrides`, `resolutions`, `type`, or env launcher inputs, should schedule the representative build service; consumer services should keep shared artifact provenance and skip individual image builds.
|
||
- HWLAB v0.3 non-reuse image build is expected when `envIdentity` changes, such as shared dependency or env launcher inputs. Then `buildServices` should include services whose environment identity changed and build TaskRuns should be present.
|
||
- HWLAB v0.3 treats `package.json` env identity semantically. Arbitrary top-level metadata fields do not count; meaningful env inputs include dependency maps, `overrides`, `resolutions`, `engines`, `packageManager`, and `type`. Use one of those fields when deliberately testing non-reuse behavior.
|
||
|
||
Do not use one consumer's evidence vocabulary to judge another consumer. In particular, do not expect every PaC consumer to expose an `ENV_ID`, and do not treat a blank `ENV_REUSE` cell as a failed env reuse without checking the consumer implementation.
|
||
|
||
## Common Pitfalls
|
||
|
||
- `status` and `history` must use the same PipelineRun matching contract: name prefix or Tekton pipeline label. Requiring a PaC repository label can hide valid HWLAB PipelineRuns because not every generated PipelineRun carries that label.
|
||
- PaC history has no UniDesk-owned history database. If a row is missing, inspect live target-side reads and read errors before concluding the pipeline never ran.
|
||
- Env reuse is a CI/build concern. The k8s runtime boundary still starts when Kubernetes pulls the already built or reused image; runtime must remain Docker-free.
|
||
- Do not preserve a second trigger path to "test" env reuse. Use the configured source authority and trigger path for that consumer, then observe through `status`/`history`.
|
||
- Do not add contract tests or hard-code business thresholds for reuse. Validate shape, required declarations, target-side PipelineRun evidence, artifact digest, GitOps revision, Argo health, runtime provenance, and the formal public HTTPS `/health` probe when the consumer has a public runtime. Do not use web-probe/Playwright as env reuse or CI/CD closeout evidence.
|
||
|
||
## Debug Checklist
|
||
|
||
1. Confirm the consumer and repository in `config/platform-infra/pipelines-as-code.yaml`.
|
||
2. Run `platform-infra pipelines-as-code history --target JD01 --consumer <id> --limit 10`.
|
||
3. If the latest row is ambiguous, run `history --id <pipelinerun>` and inspect task counts, longest task, and env reuse source.
|
||
4. Run `platform-infra pipelines-as-code status --target JD01 --consumer <id>` for current runtime/GitOps alignment.
|
||
5. If status and history disagree, fix the status/history query first; do not infer env reuse behavior from an empty table.
|
||
6. For AgentRun, inspect image status, env identity, digest, and artifact catalog provenance.
|
||
7. For sentinel, inspect dependency reuse and BuildKit cache separately.
|