feat: add m5 mvp e2e dry run
This commit is contained in:
HWLAB Code Queue
@@ -0,0 +1,88 @@
|
||||
# M5 MVP E2E Dry-Run Orchestration
|
||||
|
||||
M5 is the static orchestration gate before the first real DEV MVP e2e run. It
|
||||
does not deploy, does not call DEV, does not run browser e2e, and does not read
|
||||
secrets. The only executable gate in this milestone is:
|
||||
|
||||
```sh
|
||||
node scripts/m5-mvp-e2e-dry-run.mjs
|
||||
```
|
||||
|
||||
The script reads the frozen protocol contracts, the DEV acceptance checklist,
|
||||
`fixtures/mvp/runtime.json`, and `fixtures/mvp/m5-e2e/dry-run-plan.json`. It
|
||||
validates the dependency graph and checks that each step has resolvable inputs
|
||||
and outputs.
|
||||
|
||||
## Dry-Run Flow
|
||||
|
||||
| Order | Step | Purpose | Required Inputs | Required Outputs |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 1 | DEV health | Confirm the frozen DEV endpoint and recorded edge/frp/D601/cloud health contracts. | DEV endpoint, ingress, edge route, frp, router, cloud API/web health records | Route, tunnel, namespace, cloud service identities |
|
||||
| 2 | Gateway/box status | Confirm gateway, simulator, resources, and capabilities are present. | Gateway sessions, box resources, capabilities | Gateway session, resource IDs, capability IDs |
|
||||
| 3 | Wiring | Confirm the active wiring config is patch-panel owned. | Wiring config, patch-panel status | Wiring config ID, patch-panel status ID |
|
||||
| 4 | Direct hardware call | Validate the direct JSON-RPC hardware operation envelope and field flow. | RPC request/response, resource, capability, patch-panel status | Operation ID, trace ID |
|
||||
| 5 | Audit | Confirm mutating setup and direct operation audit records exist. | Project, gateway, wiring, operation audit records | Direct operation audit ID |
|
||||
| 6 | Agent session | Confirm agent manager, worker, and skills contracts are linked. | Agent/worker sessions, agent health records | Agent session ID, worker session ID |
|
||||
| 7 | Agent hardware call | Validate the agent worker JSON-RPC hardware operation envelope and field flow. | RPC request/response, agent, worker, resource, capability, patch-panel status | Operation ID, trace ID |
|
||||
| 8 | Evidence | Confirm both hardware operations emit trace and evidence records. | Trace events, evidence records | Evidence IDs |
|
||||
| 9 | Cleanup | Confirm worker and gateway transient cleanup are recorded. | Cleanup records, cleanup audit | Cleanup IDs |
|
||||
|
||||
## M0-M4 Dependency Graph
|
||||
|
||||
M5 closes over the earlier MVP contracts as contract-only prerequisites:
|
||||
|
||||
| Source | Provides | Consumed By M5 |
|
||||
| --- | --- | --- |
|
||||
| M0 contract audit | Frozen service IDs, protocol schemas, MVP e2e sequence contract | DEV health, hardware operation envelopes, evidence contract |
|
||||
| M1 local smoke | Static parse boundary and dry-run safety posture | DEV health and cleanup safety checks |
|
||||
| M2 cloud core | Project record, JSON-RPC envelope, audit surface | Audit, direct hardware call, agent hardware call |
|
||||
| M3 hardware loop | Gateway session, box resources/capabilities, wiring config, patch-panel status | Gateway/box status, wiring, direct hardware call, agent hardware call |
|
||||
| M4 agent runtime | Agent session, worker session, trace events, evidence, cleanup | Agent session, agent hardware call, evidence, cleanup |
|
||||
|
||||
The graph must remain acyclic. Each M5 step may only depend on earlier steps
|
||||
except the agent hardware call, which also depends on the already-established
|
||||
wiring step.
|
||||
|
||||
## Blocker Classes
|
||||
|
||||
M5 keeps the blocker classes from the DEV acceptance matrix:
|
||||
|
||||
| Class | Meaning |
|
||||
| --- | --- |
|
||||
| `contract_blocker` | Static contract, schema, checklist, or documentation cannot be parsed or contradicts frozen names. |
|
||||
| `environment_blocker` | DEV endpoint, environment, namespace, or PROD boundary is wrong. |
|
||||
| `network_blocker` | Master edge, frp, D601 route, or public DEV ingress cannot be observed or routes incorrectly. |
|
||||
| `runtime_blocker` | HWLAB cloud, router, gateway, simulator, box simulator, or patch-panel runtime contract fails. |
|
||||
| `agent_blocker` | Agent manager, worker, skills, trace, audit, or cleanup contract fails. |
|
||||
| `observability_blocker` | Artifact identity, commit, image/tag/digest, build source, deploy env, or health timestamp is missing. |
|
||||
| `safety_blocker` | Any prohibited action was attempted: real deployment, PROD target, browser/heavyweight e2e, secret read, force push, or UniDesk runtime substitution. |
|
||||
|
||||
## Real DEV E2E Gate
|
||||
|
||||
Do not switch to the real command until all of these are true:
|
||||
|
||||
1. `node scripts/m5-mvp-e2e-dry-run.mjs` passes on `origin/main`.
|
||||
2. `docs/dev-acceptance-checklist.json` and `docs/dev-acceptance-matrix.md`
|
||||
still freeze DEV at `http://74.48.78.17:6667`.
|
||||
3. Every accepted artifact has `serviceId`, `commitId`, `image`, `tag`,
|
||||
`digest` or `not_applicable` reason, `buildSource`, `deployEnv`, and
|
||||
`healthTimestamp`.
|
||||
4. DEV ingress, master edge proxy, frp, and D601 `hwlab-dev/hwlab-router` have
|
||||
current route observations.
|
||||
5. Cloud API/web, gateway, gateway simulator, box simulator, patch panel, agent
|
||||
manager, worker, and skills are all observed in DEV with HWLAB service
|
||||
identities.
|
||||
6. Gateway and box control remain patch-panel owned; no direct box bypass path
|
||||
is accepted.
|
||||
7. Agent worker scope, trace, audit, evidence, and cleanup are observable and
|
||||
tied to one DEV project.
|
||||
8. No blocker class is open and a human has explicitly approved a real DEV
|
||||
smoke.
|
||||
|
||||
Only after those gates are satisfied should the runner switch from dry-run to:
|
||||
|
||||
```sh
|
||||
hwlab-cli test e2e --env dev --mvp
|
||||
```
|
||||
|
||||
That command is intentionally not executed by M5.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,6 @@
|
||||
M5 MVP E2E dry-run evidence
|
||||
operationId: op_m5-agent-0001
|
||||
traceId: trc_m5-agent-0001
|
||||
mode: dry-run
|
||||
source: fixtures/mvp/m5-e2e/dry-run-plan.json
|
||||
No DEV or PROD request was made.
|
||||
@@ -0,0 +1,6 @@
|
||||
M5 MVP E2E dry-run evidence
|
||||
operationId: op_m5-direct-0001
|
||||
traceId: trc_m5-direct-0001
|
||||
mode: dry-run
|
||||
source: fixtures/mvp/m5-e2e/dry-run-plan.json
|
||||
No DEV or PROD request was made.
|
||||
@@ -0,0 +1,632 @@
|
||||
#!/usr/bin/env node
|
||||
import assert from "node:assert/strict";
|
||||
import crypto from "node:crypto";
|
||||
import { readFile, stat } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const planPath = "fixtures/mvp/m5-e2e/dry-run-plan.json";
|
||||
const checklistPath = "docs/dev-acceptance-checklist.json";
|
||||
const matrixPath = "docs/dev-acceptance-matrix.md";
|
||||
const runtimePath = "fixtures/mvp/runtime.json";
|
||||
const contractPath = "protocol/mvp-e2e-contract.md";
|
||||
const commonSchemaPath = "protocol/schemas/common.json";
|
||||
const DEV_ENDPOINT = "http://74.48.78.17:6667";
|
||||
const ENVIRONMENT_DEV = "dev";
|
||||
const JSON_RPC_VERSION = "2.0";
|
||||
let serviceIds = [];
|
||||
let serviceIdSet = new Set();
|
||||
|
||||
const requiredArtifactFields = [
|
||||
"serviceId",
|
||||
"commitId",
|
||||
"image",
|
||||
"tag",
|
||||
"digest",
|
||||
"buildSource",
|
||||
"deployEnv",
|
||||
"healthTimestamp"
|
||||
];
|
||||
|
||||
const requiredStepSequence = [
|
||||
"step-dev-health",
|
||||
"step-gateway-box-status",
|
||||
"step-wiring",
|
||||
"step-direct-hardware-call",
|
||||
"step-audit",
|
||||
"step-agent-session",
|
||||
"step-agent-hardware-call",
|
||||
"step-evidence",
|
||||
"step-cleanup"
|
||||
];
|
||||
|
||||
const requiredTraceStates = ["accepted", "dispatched", "executed", "completed"];
|
||||
|
||||
function readJSON(relativePath) {
|
||||
return readFile(path.join(repoRoot, relativePath), "utf8").then((raw) => JSON.parse(raw));
|
||||
}
|
||||
|
||||
function byKey(items, key) {
|
||||
const map = new Map();
|
||||
for (const item of items) {
|
||||
assert.ok(item && typeof item === "object", `${key} item must be an object`);
|
||||
assert.ok(item[key], `${key} item missing ${key}`);
|
||||
assert.equal(map.has(item[key]), false, `${key} ${item[key]} must be unique`);
|
||||
map.set(item[key], item);
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
function requireDevObject(label, object, environmentField = "environment") {
|
||||
assert.equal(object[environmentField], ENVIRONMENT_DEV, `${label} ${environmentField}`);
|
||||
}
|
||||
|
||||
function requireTimestamp(label, value) {
|
||||
assert.equal(typeof value, "string", `${label} timestamp must be a string`);
|
||||
assert.equal(Number.isNaN(Date.parse(value)), false, `${label} timestamp must parse`);
|
||||
}
|
||||
|
||||
function requireServiceId(label, serviceId) {
|
||||
assert.equal(serviceIdSet.has(serviceId), true, `${label} unknown serviceId ${serviceId}`);
|
||||
}
|
||||
|
||||
function assertArrayIncludesAll(label, actual, expected) {
|
||||
assert.ok(Array.isArray(actual), `${label} must be an array`);
|
||||
for (const item of expected) {
|
||||
assert.equal(actual.includes(item), true, `${label} missing ${item}`);
|
||||
}
|
||||
}
|
||||
|
||||
function assertNoProdTarget(value, trail = "plan") {
|
||||
if (typeof value === "string") {
|
||||
assert.equal(value.includes(":6666"), false, `${trail} must not target PROD port`);
|
||||
assert.equal(/\bprod\b/i.test(value), false, `${trail} must not target PROD`);
|
||||
return;
|
||||
}
|
||||
|
||||
if (Array.isArray(value)) {
|
||||
value.forEach((item, index) => assertNoProdTarget(item, `${trail}[${index}]`));
|
||||
return;
|
||||
}
|
||||
|
||||
if (value && typeof value === "object") {
|
||||
for (const [key, nested] of Object.entries(value)) {
|
||||
if (
|
||||
key === "prohibitedActions" ||
|
||||
key === "failureCriteria" ||
|
||||
key === "failRule" ||
|
||||
key === "digest" ||
|
||||
key === "sha256"
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
assertNoProdTarget(nested, `${trail}.${key}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function assertObjectHasOnlyDryRunSafety(plan) {
|
||||
assert.equal(plan.mode, "dry-run", "plan mode");
|
||||
assert.equal(plan.safety.allowNetwork, false, "network must be disabled");
|
||||
assert.equal(plan.safety.allowDeploy, false, "deploy must be disabled");
|
||||
assert.equal(plan.safety.allowBrowserE2E, false, "browser e2e must be disabled");
|
||||
assert.equal(plan.safety.allowSecrets, false, "secret reads must be disabled");
|
||||
assertArrayIncludesAll("prohibited actions", plan.safety.prohibitedActions, [
|
||||
"real-dev-deploy",
|
||||
"prod-deploy",
|
||||
"prod-smoke",
|
||||
"browser-e2e",
|
||||
"heavyweight-e2e",
|
||||
"secret-or-token-read",
|
||||
"force-push",
|
||||
"unidesk-runtime-substitution"
|
||||
]);
|
||||
}
|
||||
|
||||
function validateMeta(meta) {
|
||||
assert.ok(meta && typeof meta === "object" && !Array.isArray(meta), "meta must be an object");
|
||||
assert.ok(meta.traceId, "meta.traceId is required");
|
||||
requireServiceId("meta.serviceId", meta.serviceId);
|
||||
assert.equal(meta.environment, ENVIRONMENT_DEV, "meta.environment");
|
||||
}
|
||||
|
||||
function validateRequest(envelope) {
|
||||
assert.ok(envelope && typeof envelope === "object" && !Array.isArray(envelope), "request must be an object");
|
||||
assert.equal(envelope.jsonrpc, JSON_RPC_VERSION, "request jsonrpc");
|
||||
assert.notEqual(envelope.id, undefined, "request id");
|
||||
assert.match(envelope.method, /^[a-z][a-z0-9]*(\.[a-z][a-z0-9_]*)+$/, "request method");
|
||||
if (Object.hasOwn(envelope, "params")) {
|
||||
assert.ok(
|
||||
envelope.params && typeof envelope.params === "object" && !Array.isArray(envelope.params),
|
||||
"request params must be an object"
|
||||
);
|
||||
}
|
||||
validateMeta(envelope.meta);
|
||||
}
|
||||
|
||||
function validateResponse(envelope) {
|
||||
assert.ok(envelope && typeof envelope === "object" && !Array.isArray(envelope), "response must be an object");
|
||||
assert.equal(envelope.jsonrpc, JSON_RPC_VERSION, "response jsonrpc");
|
||||
assert.notEqual(envelope.id, undefined, "response id");
|
||||
const hasResult = Object.hasOwn(envelope, "result");
|
||||
const hasError = Object.hasOwn(envelope, "error");
|
||||
assert.notEqual(hasResult, hasError, "response must contain exactly one of result or error");
|
||||
if (hasError) {
|
||||
assert.ok(envelope.error && typeof envelope.error === "object", "response error must be an object");
|
||||
assert.equal(Number.isInteger(envelope.error.code), true, "response error code");
|
||||
assert.ok(envelope.error.message, "response error message");
|
||||
}
|
||||
validateMeta(envelope.meta);
|
||||
}
|
||||
|
||||
function assertEndpointFreeze(plan, checklist, runtime, contract) {
|
||||
assert.equal(plan.environment, ENVIRONMENT_DEV, "plan environment");
|
||||
assert.equal(plan.endpoint, DEV_ENDPOINT, "plan endpoint");
|
||||
assert.equal(checklist.environment, ENVIRONMENT_DEV, "checklist environment");
|
||||
assert.equal(checklist.endpoint, DEV_ENDPOINT, "checklist endpoint");
|
||||
assert.equal(runtime.endpoints.dev, DEV_ENDPOINT, "runtime fixture endpoint");
|
||||
assert.equal(contract.includes(DEV_ENDPOINT), true, "MVP contract must include DEV endpoint");
|
||||
assertNoProdTarget(plan);
|
||||
}
|
||||
|
||||
function assertArtifacts(plan, checklist) {
|
||||
assert.deepEqual(checklist.artifactObservabilityFields, requiredArtifactFields);
|
||||
const artifacts = byKey(plan.artifacts, "serviceId");
|
||||
const requiredServiceIds = new Set([
|
||||
"hwlab-edge-proxy",
|
||||
"hwlab-tunnel-client",
|
||||
"hwlab-router",
|
||||
"hwlab-cloud-api",
|
||||
"hwlab-cloud-web",
|
||||
"hwlab-gateway",
|
||||
"hwlab-gateway-simu",
|
||||
"hwlab-box-simu",
|
||||
"hwlab-patch-panel",
|
||||
"hwlab-agent-mgr",
|
||||
"hwlab-agent-worker",
|
||||
"hwlab-agent-skills",
|
||||
"hwlab-cli"
|
||||
]);
|
||||
|
||||
for (const serviceId of requiredServiceIds) {
|
||||
assert.equal(artifacts.has(serviceId), true, `artifact missing ${serviceId}`);
|
||||
}
|
||||
|
||||
for (const artifact of artifacts.values()) {
|
||||
requireServiceId(`artifact ${artifact.serviceId}`, artifact.serviceId);
|
||||
for (const field of requiredArtifactFields) {
|
||||
assert.ok(artifact[field], `artifact ${artifact.serviceId} missing ${field}`);
|
||||
}
|
||||
assert.equal(artifact.deployEnv, ENVIRONMENT_DEV, `artifact ${artifact.serviceId} deployEnv`);
|
||||
requireTimestamp(`artifact ${artifact.serviceId}.healthTimestamp`, artifact.healthTimestamp);
|
||||
if (artifact.digest === "not_applicable") {
|
||||
assert.ok(
|
||||
artifact.digestNotApplicableReason,
|
||||
`artifact ${artifact.serviceId} must explain not_applicable digest`
|
||||
);
|
||||
} else {
|
||||
assert.match(artifact.digest, /^sha256:[a-f0-9]{64}$/, `artifact ${artifact.serviceId} digest`);
|
||||
}
|
||||
}
|
||||
|
||||
return artifacts;
|
||||
}
|
||||
|
||||
function assertHealth(plan, checklist, artifacts) {
|
||||
const healthById = byKey(plan.health, "healthId");
|
||||
const healthByComponent = byKey(plan.health, "component");
|
||||
const healthByService = new Map();
|
||||
for (const health of plan.health) {
|
||||
if (!healthByService.has(health.serviceId)) {
|
||||
healthByService.set(health.serviceId, health);
|
||||
}
|
||||
}
|
||||
|
||||
for (const contract of checklist.healthContracts) {
|
||||
const health = healthByComponent.get(contract.component);
|
||||
assert.ok(health, `health missing component ${contract.component}`);
|
||||
assert.equal(health.serviceId, contract.serviceId, `health ${contract.component} serviceId`);
|
||||
assert.equal(artifacts.has(health.serviceId), true, `health ${contract.component} missing artifact`);
|
||||
assert.equal(health.status, "healthy", `health ${contract.component} status`);
|
||||
assert.equal(health.deployEnv, ENVIRONMENT_DEV, `health ${contract.component} deployEnv`);
|
||||
assert.equal(health.dryRun, true, `health ${contract.component} dryRun`);
|
||||
requireTimestamp(`health ${contract.component}`, health.healthTimestamp);
|
||||
}
|
||||
|
||||
assert.equal(healthByService.get("hwlab-edge-proxy").endpoint, `${DEV_ENDPOINT}/health`);
|
||||
assert.equal(healthByService.get("hwlab-cloud-web").output.directHardwareCall, false);
|
||||
assert.equal(healthByService.get("hwlab-gateway").output.patchPanelRequired, true);
|
||||
assert.equal(healthByService.get("hwlab-box-simu").output.patchPanelOnlyPropagation, true);
|
||||
assert.equal(healthByService.get("hwlab-agent-mgr").output.workerLeak, false);
|
||||
|
||||
return { byId: healthById, byService: healthByService, count: plan.health.length };
|
||||
}
|
||||
|
||||
function assertAcceptanceSteps(plan, checklist) {
|
||||
const checklistStepIds = new Set(checklist.smokeSteps.map((step) => step.id));
|
||||
assert.deepEqual(
|
||||
checklist.smokeSteps.map((step) => step.order),
|
||||
checklist.smokeSteps.map((_, index) => index + 1),
|
||||
"checklist smoke steps must be ordered"
|
||||
);
|
||||
|
||||
const referencedStepIds = new Set();
|
||||
for (const step of plan.steps) {
|
||||
for (const acceptanceStepId of step.acceptanceStepIds) {
|
||||
assert.equal(
|
||||
checklistStepIds.has(acceptanceStepId),
|
||||
true,
|
||||
`${step.id} references unknown acceptance step ${acceptanceStepId}`
|
||||
);
|
||||
referencedStepIds.add(acceptanceStepId);
|
||||
}
|
||||
}
|
||||
|
||||
assertArrayIncludesAll("dry-run acceptance coverage", [...referencedStepIds], [
|
||||
"static-contract-parse",
|
||||
"endpoint-freeze",
|
||||
"dev-ingress-health",
|
||||
"edge-route",
|
||||
"frp-tunnel",
|
||||
"d601-router",
|
||||
"cloud-surface",
|
||||
"gateway-sim-patch-panel",
|
||||
"agent-runtime",
|
||||
"artifact-observability"
|
||||
]);
|
||||
}
|
||||
|
||||
function assertStepGraph(plan) {
|
||||
const steps = plan.steps.slice().sort((a, b) => a.order - b.order);
|
||||
assert.deepEqual(
|
||||
steps.map((step) => step.id),
|
||||
requiredStepSequence,
|
||||
"M5 sequence must cover required dry-run flow"
|
||||
);
|
||||
assert.deepEqual(plan.expectedSequence, requiredStepSequence, "expectedSequence must match required sequence");
|
||||
|
||||
const seen = new Set();
|
||||
for (const step of steps) {
|
||||
assert.equal(seen.has(step.id), false, `duplicate step ${step.id}`);
|
||||
assert.ok(Array.isArray(step.inputs) && step.inputs.length > 0, `${step.id} inputs`);
|
||||
assert.ok(Array.isArray(step.outputs) && step.outputs.length > 0, `${step.id} outputs`);
|
||||
for (const dependency of step.requires) {
|
||||
assert.equal(seen.has(dependency), true, `${step.id} requires ${dependency} before it is produced`);
|
||||
}
|
||||
seen.add(step.id);
|
||||
}
|
||||
}
|
||||
|
||||
function assertProjectGatewayAndBoxes(plan) {
|
||||
requireDevObject("project", plan.project);
|
||||
assert.equal(plan.project.status, "active", "project status");
|
||||
|
||||
const gatewaySessions = byKey(plan.gatewaySessions, "gatewaySessionId");
|
||||
assert.equal(gatewaySessions.has("gws_m5-0001"), true, "real gateway boundary session is required");
|
||||
assert.equal(gatewaySessions.has("gws_m5-simu-0001"), true, "gateway simulator session is required");
|
||||
|
||||
for (const session of gatewaySessions.values()) {
|
||||
assert.equal(session.projectId, plan.project.projectId, `gateway ${session.gatewaySessionId} project`);
|
||||
assert.ok(["hwlab-gateway", "hwlab-gateway-simu"].includes(session.serviceId), "gateway serviceId");
|
||||
assert.equal(session.status, "connected", `gateway ${session.gatewaySessionId} status`);
|
||||
requireDevObject(`gateway ${session.gatewaySessionId}`, session);
|
||||
}
|
||||
|
||||
const resources = byKey(plan.boxResources, "resourceId");
|
||||
const capabilities = byKey(plan.boxCapabilities, "capabilityId");
|
||||
|
||||
for (const resource of resources.values()) {
|
||||
assert.equal(resource.projectId, plan.project.projectId, `resource ${resource.resourceId} project`);
|
||||
assert.equal(gatewaySessions.has(resource.gatewaySessionId), true, `resource ${resource.resourceId} gateway`);
|
||||
assert.equal(resource.state, "available", `resource ${resource.resourceId} state`);
|
||||
requireDevObject(`resource ${resource.resourceId}`, resource);
|
||||
}
|
||||
|
||||
for (const capability of capabilities.values()) {
|
||||
assert.equal(capability.projectId, plan.project.projectId, `capability ${capability.capabilityId} project`);
|
||||
assert.equal(resources.has(capability.resourceId), true, `capability ${capability.capabilityId} resource`);
|
||||
assert.match(capability.name, /^[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)+$/);
|
||||
}
|
||||
|
||||
return { gatewaySessions, resources, capabilities };
|
||||
}
|
||||
|
||||
function assertWiring(plan, { gatewaySessions, resources, capabilities }) {
|
||||
const wiring = plan.wiringConfig;
|
||||
assert.equal(wiring.projectId, plan.project.projectId, "wiring project");
|
||||
assert.equal(gatewaySessions.has(wiring.gatewaySessionId), true, "wiring gateway");
|
||||
assert.equal(wiring.status, "active", "wiring status");
|
||||
assert.equal(wiring.constraints.patchPanelOwnsRouting, true, "wiring must be patch-panel-owned");
|
||||
assert.equal(wiring.constraints.directBoxBypassAllowed, false, "wiring must reject direct box bypass");
|
||||
|
||||
for (const connection of wiring.connections) {
|
||||
for (const side of ["from", "to"]) {
|
||||
const endpoint = connection[side];
|
||||
assert.equal(resources.has(endpoint.resourceId), true, `wiring ${side} resource`);
|
||||
assert.equal(capabilities.has(endpoint.capabilityId), true, `wiring ${side} capability`);
|
||||
assert.equal(
|
||||
capabilities.get(endpoint.capabilityId).resourceId,
|
||||
endpoint.resourceId,
|
||||
`wiring ${side} capability must belong to resource`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const patch = plan.patchPanelStatus;
|
||||
assert.equal(patch.projectId, plan.project.projectId, "patch project");
|
||||
assert.equal(patch.gatewaySessionId, wiring.gatewaySessionId, "patch gateway");
|
||||
assert.equal(patch.wiringConfigId, wiring.wiringConfigId, "patch wiring");
|
||||
assert.equal(patch.serviceId, "hwlab-patch-panel", "patch service");
|
||||
assert.equal(patch.state, "active", "patch state");
|
||||
requireDevObject("patch", patch);
|
||||
requireTimestamp("patch observedAt", patch.observedAt);
|
||||
|
||||
const activeConnectionKeys = new Set(
|
||||
patch.activeConnections.map(
|
||||
(connection) =>
|
||||
`${connection.fromResourceId}:${connection.fromPort}->${connection.toResourceId}:${connection.toPort}`
|
||||
)
|
||||
);
|
||||
for (const connection of wiring.connections) {
|
||||
const key = `${connection.from.resourceId}:${connection.from.port}->${connection.to.resourceId}:${connection.to.port}`;
|
||||
assert.equal(activeConnectionKeys.has(key), true, `patch panel missing active connection ${key}`);
|
||||
}
|
||||
}
|
||||
|
||||
function assertRpcAndOperations(plan, { gatewaySessions, resources, capabilities }) {
|
||||
const operations = byKey(plan.operations, "operationId");
|
||||
const rpcByName = byKey(plan.rpc, "name");
|
||||
|
||||
for (const rpc of rpcByName.values()) {
|
||||
validateRequest(rpc.request);
|
||||
validateResponse(rpc.response);
|
||||
assert.equal(rpc.request.meta.environment, ENVIRONMENT_DEV, `${rpc.name} request env`);
|
||||
assert.equal(rpc.response.meta.environment, ENVIRONMENT_DEV, `${rpc.name} response env`);
|
||||
assert.equal(rpc.request.meta.traceId, rpc.response.meta.traceId, `${rpc.name} trace continuity`);
|
||||
assert.equal(rpc.request.params.dryRun, undefined, `${rpc.name} params must not use top-level dryRun`);
|
||||
assert.equal(rpc.request.params.input.dryRun, true, `${rpc.name} input dryRun`);
|
||||
assert.equal(rpc.response.result.operationId, rpc.operationId, `${rpc.name} response operation`);
|
||||
assert.equal(operations.has(rpc.operationId), true, `${rpc.name} operation exists`);
|
||||
}
|
||||
|
||||
assert.equal(rpcByName.has("direct-hardware-request"), true, "direct RPC is required");
|
||||
assert.equal(rpcByName.has("agent-hardware-request"), true, "agent RPC is required");
|
||||
|
||||
for (const operation of operations.values()) {
|
||||
assert.equal(operation.projectId, plan.project.projectId, `operation ${operation.operationId} project`);
|
||||
assert.equal(gatewaySessions.has(operation.gatewaySessionId), true, `operation ${operation.operationId} gateway`);
|
||||
assert.equal(resources.has(operation.resourceId), true, `operation ${operation.operationId} resource`);
|
||||
assert.equal(capabilities.has(operation.capabilityId), true, `operation ${operation.operationId} capability`);
|
||||
assert.equal(operation.status, "succeeded", `operation ${operation.operationId} status`);
|
||||
assert.equal(operation.input.dryRun, true, `operation ${operation.operationId} dryRun`);
|
||||
assert.equal(operation.output.patchPanelStatusId, plan.patchPanelStatus.patchPanelStatusId);
|
||||
requireDevObject(`operation ${operation.operationId}`, operation);
|
||||
}
|
||||
|
||||
const agentOperation = operations.get("op_m5-agent-0001");
|
||||
assert.equal(agentOperation.agentSessionId, plan.agentSession.agentSessionId, "agent operation session");
|
||||
assert.equal(agentOperation.workerSessionId, plan.workerSession.workerSessionId, "agent operation worker");
|
||||
|
||||
return { operations, rpcByName };
|
||||
}
|
||||
|
||||
function assertAgent(plan, { gatewaySessions }) {
|
||||
const { agentSession, workerSession } = plan;
|
||||
assert.equal(agentSession.projectId, plan.project.projectId, "agent project");
|
||||
assert.equal(agentSession.serviceId, "hwlab-agent-mgr", "agent service");
|
||||
assert.equal(agentSession.status, "completed", "agent status");
|
||||
assert.equal(agentSession.metadata.workerSessionId, workerSession.workerSessionId, "agent worker link");
|
||||
requireDevObject("agent", agentSession);
|
||||
|
||||
assert.equal(workerSession.agentSessionId, agentSession.agentSessionId, "worker agent link");
|
||||
assert.equal(workerSession.projectId, plan.project.projectId, "worker project");
|
||||
assert.equal(workerSession.serviceId, "hwlab-agent-worker", "worker service");
|
||||
assert.equal(workerSession.status, "completed", "worker status");
|
||||
assert.equal(gatewaySessions.has(workerSession.gatewaySessionId), true, "worker gateway");
|
||||
assert.equal(workerSession.metadata.scopedProjectId, plan.project.projectId, "worker project scope");
|
||||
requireDevObject("worker", workerSession);
|
||||
}
|
||||
|
||||
function assertTraceAuditEvidenceAndCleanup(plan, { operations }) {
|
||||
const traceEvents = byKey(plan.traceEvents, "traceEventId");
|
||||
const auditEvents = byKey(plan.auditEvents, "auditId");
|
||||
const evidenceRecords = byKey(plan.evidenceRecords, "evidenceId");
|
||||
const cleanupRecords = byKey(plan.cleanup, "cleanupId");
|
||||
|
||||
for (const trace of traceEvents.values()) {
|
||||
assert.equal(trace.projectId, plan.project.projectId, `trace ${trace.traceEventId} project`);
|
||||
requireServiceId(`trace ${trace.traceEventId}`, trace.serviceId);
|
||||
requireDevObject(`trace ${trace.traceEventId}`, trace);
|
||||
requireTimestamp(`trace ${trace.traceEventId}`, trace.occurredAt);
|
||||
if (trace.operationId) {
|
||||
assert.equal(operations.has(trace.operationId), true, `trace ${trace.traceEventId} operation`);
|
||||
}
|
||||
}
|
||||
|
||||
for (const operationId of ["op_m5-direct-0001", "op_m5-agent-0001"]) {
|
||||
const states = plan.traceEvents
|
||||
.filter((trace) => trace.operationId === operationId)
|
||||
.map((trace) => trace.metadata.state);
|
||||
assert.deepEqual(states, requiredTraceStates, `${operationId} trace lifecycle`);
|
||||
}
|
||||
|
||||
for (const audit of auditEvents.values()) {
|
||||
assert.equal(audit.projectId, plan.project.projectId, `audit ${audit.auditId} project`);
|
||||
requireServiceId(`audit ${audit.auditId}`, audit.serviceId);
|
||||
requireDevObject(`audit ${audit.auditId}`, audit);
|
||||
requireTimestamp(`audit ${audit.auditId}`, audit.occurredAt);
|
||||
assert.ok(audit.outcome, `audit ${audit.auditId} outcome`);
|
||||
}
|
||||
|
||||
assert.equal(
|
||||
[...auditEvents.values()].some((audit) => audit.action === "hardware.operation.request"),
|
||||
true,
|
||||
"hardware operation audit required"
|
||||
);
|
||||
assert.equal(auditEvents.has("aud_m5-cleanup"), true, "cleanup audit required");
|
||||
|
||||
for (const evidence of evidenceRecords.values()) {
|
||||
assert.equal(evidence.projectId, plan.project.projectId, `evidence ${evidence.evidenceId} project`);
|
||||
assert.equal(operations.has(evidence.operationId), true, `evidence ${evidence.evidenceId} operation`);
|
||||
requireServiceId(`evidence ${evidence.evidenceId}`, evidence.serviceId);
|
||||
requireDevObject(`evidence ${evidence.evidenceId}`, evidence);
|
||||
requireTimestamp(`evidence ${evidence.evidenceId}`, evidence.createdAt);
|
||||
}
|
||||
|
||||
for (const operationId of operations.keys()) {
|
||||
assert.equal(
|
||||
[...evidenceRecords.values()].some((evidence) => evidence.operationId === operationId),
|
||||
true,
|
||||
`evidence missing for ${operationId}`
|
||||
);
|
||||
}
|
||||
|
||||
for (const cleanup of cleanupRecords.values()) {
|
||||
assert.equal(cleanup.status, "completed", `cleanup ${cleanup.cleanupId} status`);
|
||||
assert.equal(cleanup.environment, ENVIRONMENT_DEV, `cleanup ${cleanup.cleanupId} environment`);
|
||||
requireServiceId(`cleanup ${cleanup.cleanupId}`, cleanup.serviceId);
|
||||
requireTimestamp(`cleanup ${cleanup.cleanupId}`, cleanup.completedAt);
|
||||
}
|
||||
|
||||
assert.equal(cleanupRecords.get("cln_m5-worker").outputs.workspaceReleased, true, "worker cleanup release");
|
||||
assert.equal(cleanupRecords.get("cln_m5-gateway-state").outputs.transientStateCleared, true);
|
||||
|
||||
return { traceEvents, auditEvents, evidenceRecords, cleanupRecords };
|
||||
}
|
||||
|
||||
async function assertEvidenceFiles(evidenceRecords) {
|
||||
for (const evidence of evidenceRecords.values()) {
|
||||
const absolutePath = path.join(repoRoot, evidence.uri);
|
||||
const content = await readFile(absolutePath);
|
||||
const digest = crypto.createHash("sha256").update(content).digest("hex");
|
||||
const stats = await stat(absolutePath);
|
||||
assert.equal(digest, evidence.sha256, `evidence ${evidence.evidenceId} sha256`);
|
||||
assert.equal(stats.size, evidence.sizeBytes, `evidence ${evidence.evidenceId} sizeBytes`);
|
||||
}
|
||||
}
|
||||
|
||||
function assertStepInputsOutputs(plan, indexes) {
|
||||
const resolvers = {
|
||||
health: (id) => indexes.healthIds.has(id),
|
||||
gatewaySession: (id) => indexes.gatewaySessions.has(id),
|
||||
boxResource: (id) => indexes.resources.has(id),
|
||||
resource: (id) => indexes.resources.has(id),
|
||||
capability: (id) => indexes.capabilities.has(id),
|
||||
wiringConfig: (id) => id === plan.wiringConfig.wiringConfigId,
|
||||
patchPanelStatus: (id) => id === plan.patchPanelStatus.patchPanelStatusId,
|
||||
rpc: (id) => indexes.rpcByName.has(id),
|
||||
operation: (id) => indexes.operations.has(id),
|
||||
audit: (id) => indexes.auditEvents.has(id),
|
||||
agentSession: (id) => id === plan.agentSession.agentSessionId,
|
||||
workerSession: (id) => id === plan.workerSession.workerSessionId,
|
||||
trace: (id) => indexes.traceIds.has(id),
|
||||
evidence: (id) => indexes.evidenceRecords.has(id),
|
||||
cleanup: (id) => indexes.cleanupRecords.has(id)
|
||||
};
|
||||
|
||||
for (const step of plan.steps) {
|
||||
for (const input of step.inputs) {
|
||||
if (input === "endpoint") {
|
||||
continue;
|
||||
}
|
||||
const [type, id] = input.split(":");
|
||||
assert.ok(type && id, `${step.id} input ${input} must be typed`);
|
||||
assert.ok(resolvers[type], `${step.id} input ${input} has unknown type ${type}`);
|
||||
assert.equal(resolvers[type](id), true, `${step.id} input ${input} does not resolve`);
|
||||
}
|
||||
}
|
||||
|
||||
const allOutputs = new Set([
|
||||
"route_m5-dev",
|
||||
"tun_m5-d601",
|
||||
"hwlab-dev",
|
||||
"hwlab-cloud-api",
|
||||
"hwlab-cloud-web",
|
||||
...indexes.gatewaySessions.keys(),
|
||||
...indexes.resources.keys(),
|
||||
...indexes.capabilities.keys(),
|
||||
plan.wiringConfig.wiringConfigId,
|
||||
plan.patchPanelStatus.patchPanelStatusId,
|
||||
...indexes.operations.keys(),
|
||||
...indexes.traceIds,
|
||||
...indexes.auditEvents.keys(),
|
||||
plan.agentSession.agentSessionId,
|
||||
plan.workerSession.workerSessionId,
|
||||
...indexes.evidenceRecords.keys(),
|
||||
...indexes.cleanupRecords.keys()
|
||||
]);
|
||||
|
||||
for (const step of plan.steps) {
|
||||
for (const output of step.outputs) {
|
||||
assert.equal(allOutputs.has(output), true, `${step.id} output ${output} does not resolve`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function assertDependencies(plan) {
|
||||
assertArrayIncludesAll(
|
||||
"M0-M4 dependencies",
|
||||
plan.m0ToM4Dependencies.map((dependency) => dependency.id),
|
||||
["m0-contract-audit", "m1-local-smoke", "m2-cloud-core", "m3-hardware-loop", "m4-agent-runtime"]
|
||||
);
|
||||
|
||||
const dependencyStepIds = new Set(plan.steps.map((step) => step.id));
|
||||
for (const dependency of plan.m0ToM4Dependencies) {
|
||||
assert.ok(dependency.provides.length > 0, `${dependency.id} provides`);
|
||||
assert.equal(dependency.status, "contract_only", `${dependency.id} status`);
|
||||
for (const stepId of dependency.requiredFor) {
|
||||
assert.equal(dependencyStepIds.has(stepId), true, `${dependency.id} references unknown ${stepId}`);
|
||||
}
|
||||
}
|
||||
|
||||
assert.equal(plan.realDevGate.command, "hwlab-cli test e2e --env dev --mvp");
|
||||
assert.equal(plan.realDevGate.requiresAllBlockersClear, true);
|
||||
assert.equal(plan.realDevGate.requiresDryRunPass, true);
|
||||
assert.equal(plan.realDevGate.requiresHumanApprovalForRealDev, true);
|
||||
}
|
||||
|
||||
const [plan, checklist, runtime, contract, matrix, commonSchema] = await Promise.all([
|
||||
readJSON(planPath),
|
||||
readJSON(checklistPath),
|
||||
readJSON(runtimePath),
|
||||
readFile(path.join(repoRoot, contractPath), "utf8"),
|
||||
readFile(path.join(repoRoot, matrixPath), "utf8"),
|
||||
readJSON(commonSchemaPath)
|
||||
]);
|
||||
|
||||
serviceIds = commonSchema.$defs.serviceId.enum;
|
||||
serviceIdSet = new Set(serviceIds);
|
||||
|
||||
assert.equal(matrix.includes("HWLAB DEV Acceptance Matrix"), true, "acceptance matrix exists");
|
||||
assertObjectHasOnlyDryRunSafety(plan);
|
||||
assertEndpointFreeze(plan, checklist, runtime, contract);
|
||||
const artifacts = assertArtifacts(plan, checklist);
|
||||
const health = assertHealth(plan, checklist, artifacts);
|
||||
assertAcceptanceSteps(plan, checklist);
|
||||
assertStepGraph(plan);
|
||||
const topology = assertProjectGatewayAndBoxes(plan);
|
||||
assertWiring(plan, topology);
|
||||
assertAgent(plan, topology);
|
||||
const rpcAndOperations = assertRpcAndOperations(plan, topology);
|
||||
const evidenceIndexes = assertTraceAuditEvidenceAndCleanup(plan, rpcAndOperations);
|
||||
await assertEvidenceFiles(evidenceIndexes.evidenceRecords);
|
||||
assertStepInputsOutputs(plan, {
|
||||
...topology,
|
||||
...rpcAndOperations,
|
||||
...evidenceIndexes,
|
||||
healthIds: new Set(health.byId.keys()),
|
||||
traceIds: new Set(plan.traceEvents.map((trace) => trace.traceId))
|
||||
});
|
||||
assertDependencies(plan);
|
||||
|
||||
console.log(
|
||||
[
|
||||
`M5 MVP E2E dry-run passed: ${plan.steps.length} steps`,
|
||||
`${artifacts.size} artifacts`,
|
||||
`${health.count} health contracts`,
|
||||
`${rpcAndOperations.operations.size} hardware operations`,
|
||||
`${evidenceIndexes.evidenceRecords.size} evidence records`,
|
||||
"0 network calls"
|
||||
].join(", ")
|
||||
);
|
||||
Reference in New Issue
Block a user