diff --git a/docs/reference/spec-hwpod-harness.md b/docs/reference/spec-hwpod-harness.md index 12165426..92189499 100644 --- a/docs/reference/spec-hwpod-harness.md +++ b/docs/reference/spec-hwpod-harness.md @@ -233,7 +233,9 @@ bun tools/hwlab-cli/bin/hwlab-cli.ts case run result bun tools/hwlab-cli/bin/hwlab-cli.ts case run logs --tail 8000 ``` -`case run` 在当前版本只负责编排流程和记录原始证据，不做 evidence 自动评价、自动打分或门禁判断。流程是：prepare 先在 subject repo 本地 checkout 下创建 `.worktree/caserun-` 隔离 worktree并写入 run-local spec，然后生成 Code Agent prompt/context，创建或调用 HWLAB Code Agent session，把 isolated worktree、run-local HWPOD spec、任务目标和约束交给 Code Agent；agent 阶段结束或超时后，CaseRun 采集 agent trace/terminal state/commandExecution 摘要、subject worktree 的 `git status` / `git diff --stat` / `git diff --binary`，再按 case 定义执行 runner 后置 `hwpod-cli build` 和 `cmd.run` Keil 异步 job-status 查询，最后把原始证据和可读归档写入 case registry repo 的 `runs///`。固定归档包括 `evidence.json`、`summary.md`、`run.json`、`result.json`、`agent-prompt.md`、`agent-diff.patch`、run-local `.hwlab/hwpod-spec.yaml`、worker stdout/stderr、`artifact-manifest.json`，以及共享 Web/CLI trace renderer 生成的 `agent-messages.json`、`agent-trace.md`、`agent-transcript.md` 和 `final-response.md`。`agent-trace.md` / `agent-transcript.md` 必须包含 rendered trace rows、renderer 标识和 subject diff 区块；`final-response.md` 在存在权威 finalResponse 时保存正文，不存在时显式记录 `finalResponse=null`、terminal/error rows 和缺失原因。下载不作为 compile-only CaseRun 的隐式步骤；后续需要下载时，应作为独立 action 显式加入 case 定义和流程记录。 +`case run` 在当前版本只负责编排流程和记录原始证据，不做 evidence 自动评价、自动打分或门禁判断。流程是：prepare 先在 subject repo 本地 checkout 下创建 `.worktree/caserun-` 隔离 worktree并写入 run-local spec，然后生成 Code Agent prompt/context，创建或调用 HWLAB Code Agent session，把 isolated worktree、run-local HWPOD spec、任务目标和约束交给 Code Agent；agent 阶段结束或超时后，CaseRun 记录 agent identity、terminal result 里已有的终态字段、trace lookup hint、subject worktree 的 `git status` / `git diff --stat` / `git diff --binary`，再按 case 定义执行 runner 后置 `hwpod-cli build` 和 `cmd.run` Keil 异步 job-status 查询，最后把原始证据和可读归档写入 case registry repo 的 `runs///`。固定归档包括 `evidence.json`、`summary.md`、`run.json`、`result.json`、`agent-prompt.md`、`agent-diff.patch`、run-local `.hwlab/hwpod-spec.yaml`、worker stdout/stderr、`artifact-manifest.json`，以及 `agent-messages.json`、`agent-trace.md`、`agent-transcript.md` 和 `final-response.md`。`agent-trace.md` / `agent-transcript.md` 必须包含 trace lookup hint、renderer 标识和 subject diff 区块；如果归档了完整 rendered rows，其语义必须来自共享 Web/CLI trace renderer，不能由 CaseRun 自行发明。`final-response.md` 在 terminal result 存在权威 finalResponse 时保存正文，不存在时显式记录 `finalResponse=null`、terminal/error rows 和缺失原因。下载不作为 compile-only CaseRun 的隐式步骤；后续需要下载时，应作为独立 action 显式加入 case 定义和流程记录。 + +CaseRun 的 trace 处理必须遵循 [spec-v02-code-agent-trace.md](spec-v02-code-agent-trace.md) 的组合优先原则：CaseRun 只负责稳定返回和归档 `traceId`、`sessionId`、`conversationId`、`threadId`、AgentRun 执行线索以及 `traceLookup.commands.*`，完整 trace/result/session 读取继续使用已有 `hwlab-cli client agent trace/result/inspect`。CaseRun 不应在自身内部再实现一套 `/v1/agent/chat/trace/{traceId}` 查询器、row renderer、session 反查或 final response 归因逻辑；如果已有 agent CLI 的 trace 可见性不足，先改进 `client agent trace/result/inspect`，再让 CaseRun 输出新的 ID 或 hint。归档中的 `agent-trace.json` 可以是 `lookupOnly=true` 的 identity manifest，表示“用这些 ID 调既有 CLI 查完整 trace”，不能伪装成完整 trace dump。 长时间 CaseRun 必须优先使用 `case run start/status/result/logs` 短连接形态，而不是用 shell/nohup 包装同步 `case run`。`start` 只负责创建 `.state/hwlab-cli/caserun//`、启动后台 worker 并立即返回 `runId`、PID、stdout/stderr 路径和下一条查询命令；`status`、`result`、`logs` 都是短查询。`run.json` 是阶段状态权威，阶段开始时就要刷新 `status`、`stage`、`traceId/sessionId/conversationId/threadId`、AgentRun/trace 轮询信息、stdout/stderr byte count 和下一条轮询命令；长等待期间不得出现无输出、无 trace、无状态文件更新的黑洞窗口。 diff --git a/docs/reference/spec-v02-code-agent-trace.md b/docs/reference/spec-v02-code-agent-trace.md index 20067302..56a4ec32 100644 --- a/docs/reference/spec-v02-code-agent-trace.md +++ b/docs/reference/spec-v02-code-agent-trace.md @@ -100,6 +100,21 @@ Web、CLI 和 CI 的一致性按“入口同源、API 同 path、renderer 同代 如果 Web、CLI 或 CI 只覆盖了上述链路的一段，不能称为同路径修复。例如只修 Web DOM、只修 CLI formatter、只修 session cache、只修 AgentRun events filter、或只让 CI mock helper 变绿，都可能继续保留另一条 final/trace 路径，导致“修过很多次但现象还在”。Trace 相关变更必须同时对齐 authority、API、renderer 和 component plan 四层中的相关部分。 +## 组合优先原则 + +CaseRun、harness、runner、诊断脚本和其他上层编排工具需要查看 Code Agent trace 时，必须采用“ID + 既有工具组合”的方式，而不是在上层工具里重新实现 trace/result/session 查询和渲染逻辑。上层工具的职责是稳定返回 `traceId`、`sessionId`、`conversationId`、`threadId`、AgentRun `runId/commandId` 线索和可复制的下一条 HWLAB CLI 命令；完整 trace、result、inspect 和 Web renderer 语义继续由 `hwlab-cli client agent trace/result/inspect` 提供。 + +推荐输出合同： + +- `traceLookup.source=hwlab-cli.client.agent`，说明后续查询复用既有 HWLAB Agent CLI。 +- `traceLookup.strategy=id_plus_existing_cli`，说明当前工具只提供 identity 和组合 hint。 +- `traceLookup.commands.result=hwlab-cli client agent result --base-url `。 +- `traceLookup.commands.trace=hwlab-cli client agent trace --base-url --render web`。 +- `traceLookup.commands.inspect=hwlab-cli client agent inspect --trace-id --base-url `。 +- 有 `sessionId` 时可以补 `traceLookup.commands.sessionStatus=hwlab-cli client agent session status --base-url `。 + +这条规则的目的是通过组合复用保持单一 trace authority：CaseRun 或 harness 可以归档 `traceLookup` 和由 result payload 直接可得的终态摘要，但不得为了方便再发明一套 `/trace/{traceId}` fetch、row renderer、session 反查、final response 归因或 AgentRun events 过滤器。需要更多可见性时，先改进 `hwlab-cli client agent trace/result/inspect` 这些已有入口，再让上层工具返回新的 hint 或消费这些入口的稳定字段。 + ## 派生缓存边界 允许存在的派生状态只有两个用途：降低轮询开销和提升 UI 可见性。派生状态必须可被权威路径重建，且不得成为 terminal result 的替代 authority。 diff --git a/tools/hwlab-cli/caserun.test.ts b/tools/hwlab-cli/caserun.test.ts index 73ecbf8a..485b3050 100644 --- a/tools/hwlab-cli/caserun.test.ts +++ b/tools/hwlab-cli/caserun.test.ts @@ -328,22 +328,22 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit assert.equal(result.payload.agent.stageStatus, "completed"); assert.equal(result.payload.agent.traceId.startsWith("trc_case_"), true); assert.equal(result.payload.agent.timeoutMs, 234000); - assert.ok(result.payload.agentTrace.hwpodBuildCommandCount >= 1); - assert.ok(result.payload.evidence.agentTrace.commandCount > 0); - assert.ok(result.payload.evidence.agentTrace.commandCount <= 30); - assert.ok(result.payload.evidence.agentTrace.hwpodCommandCount >= 3); - assert.equal(result.payload.evidence.agentTrace.keilJobCandidates.includes("20260605_203835_798515c0"), true); + assert.equal(result.payload.traceLookup.commands.trace, `hwlab-cli client agent trace ${result.payload.agent.traceId} --base-url http://web.test --render web`); + assert.equal(result.payload.traceLookup.commands.result, `hwlab-cli client agent result ${result.payload.agent.traceId} --base-url http://web.test`); + assert.equal(result.payload.traceLookup.commands.inspect, `hwlab-cli client agent inspect --trace-id ${result.payload.agent.traceId} --base-url http://web.test`); + assert.equal(result.payload.agentTrace.source, "caserun-identity"); + assert.equal(result.payload.agentTrace.lookupOnly, true); + assert.equal(result.payload.agentTrace.hwpodBuildCommandCount, 0); + assert.equal(result.payload.evidence.agentTrace.commandCount, 0); + assert.equal(result.payload.evidence.agentTrace.hwpodCommandCount, 0); + assert.equal(result.payload.evidence.agentTrace.keilJobCandidates.includes("20260605_203835_798515c0"), false); assert.equal(result.payload.evidence.agentTask.timeoutMs, 234000); assert.equal(result.payload.agent.polls <= 3, true); - assert.equal(result.payload.evidence.agentTrace.commands.some((item: any) => /^hwpod-ctl spec validate --spec \.hwlab\/hwpod-spec\.yaml/u.test(item.normalizedCommand ?? "")), true); - assert.equal(result.payload.evidence.agentTrace.commands.some((item: any) => /^hwpod inspect --spec \.hwlab\/hwpod-spec\.yaml/u.test(item.normalizedCommand ?? "")), true); - assert.equal(result.payload.evidence.agentTrace.commands.some((item: any) => /^hwpod build --spec \.hwlab\/hwpod-spec\.yaml/u.test(item.normalizedCommand ?? "")), true); - assert.equal(result.payload.evidence.agentTrace.commands.some((item: any) => /cat .*hwpod-cli\/SKILL\.md/u.test(item.command ?? "") && /hwpod build/u.test(item.bodyPreview ?? "")), true); - assert.equal(result.payload.evidence.agentTrace.commands.some((item: any) => /^hwpod build --spec \.hwlab\/hwpod-spec\.yaml/u.test(item.normalizedCommand ?? "")), true); assert.equal(result.payload.evidence.hwpod.source, "case-run-runner-post-agent-compile-check"); assert.match(result.payload.agentDiff.statusShort, /projects\/01_baseline\/main\.c/u); assert.equal(result.payload.build.autoEvaluation, false); - assert.ok(result.payload.build.agentTraceHwpodCommandCount >= 3); + assert.equal(result.payload.build.traceLookup.commands.trace, result.payload.traceLookup.commands.trace); + assert.equal(result.payload.build.agentTraceHwpodCommandCount, 0); assert.equal(result.payload.build.hwpodExitCode, 0); assert.equal(result.payload.build.hwpodSource, "case-run-runner-post-agent-compile-check"); assert.equal(result.payload.build.keilStatus, "completed"); @@ -374,7 +374,10 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit assert.equal(manifest.trace.traceId, result.payload.agent.traceId); assert.equal(manifest.trace.sessionId, "ses_case_run"); assert.equal(manifest.trace.conversationId, "cnv_case_d601-f103-v2-compile_run-agent-flow"); - assert.equal(manifest.trace.agentRun.runId, "run_agent_trace"); + assert.equal(manifest.trace.source, "caserun-identity"); + assert.equal(manifest.trace.lookupOnly, true); + assert.equal(manifest.trace.lookup.commands.trace, result.payload.traceLookup.commands.trace); + assert.equal(manifest.traceLookup.commands.inspect, result.payload.traceLookup.commands.inspect); assert.equal(manifest.prompt.sha256, result.payload.evidence.agent.promptSha256); assert.equal(manifest.subject.commitId, SUBJECT_COMMIT_ID); assert.equal(manifest.diff.sha256, result.payload.evidence.agentDiff.diffPatchSha256); @@ -385,10 +388,8 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit assert.equal(manifest.readableAgent.finalResponsePath, "final-response.md"); assert.equal(manifest.readableAgent.finalResponse.present, true); assert.equal(manifest.readableAgent.finalResponse.text, "done"); - assert.deepEqual(manifest.agentStage.kinds, ["spec-validate", "inspect", "workspace-edit", "build", "download", "uart-read"]); + assert.deepEqual(manifest.agentStage.kinds, []); assert.equal(manifest.agentStage.autoEvaluation, false); - assert.equal(manifest.agentStage.commands.some((item: any) => item.kind === "download" && /uvoptx file not found/u.test(item.detail)), true); - assert.equal(manifest.agentStage.commands.some((item: any) => item.kind === "uart-read" && /hwpod_uart_monitor_not_active/u.test(item.detail)), true); assert.equal(manifest.decisions.autoEvaluation, false); assert.equal(manifest.decisions.downloadSkipped, undefined); assert.equal(manifest.decisions.runnerPostAgentCompileCheck, "recorded"); @@ -399,10 +400,14 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit assert.equal(manifest.files.some((item: any) => item.path === ".hwlab/hwpod-spec.yaml"), true); const messages = JSON.parse(await readFile(path.join(registryRunDir, "agent-messages.json"), "utf8")); assert.equal(messages.renderer, "tools/src/hwlab-cli/trace-renderer:traceDisplayRows"); + assert.equal(messages.lookupOnly, true); + assert.equal(messages.traceLookup.commands.trace, result.payload.traceLookup.commands.trace); assert.equal(messages.finalResponse.present, true); - assert.match(JSON.stringify(messages.rows), /hwpod build --spec \.hwlab\/hwpod-spec\.yaml/u); + assert.equal(messages.rows.length, 0); const traceMarkdown = await readFile(path.join(registryRunDir, "agent-trace.md"), "utf8"); assert.match(traceMarkdown, /renderer: tools\/src\/hwlab-cli\/trace-renderer:traceDisplayRows/u); + assert.match(traceMarkdown, /traceLookupStrategy: id_plus_existing_cli/u); + assert.match(traceMarkdown, /client agent trace trc_case_/u); assert.match(traceMarkdown, /## Subject Diff/u); assert.match(traceMarkdown, /\+printf\("hello"\);/u); const finalResponse = await readFile(path.join(registryRunDir, "final-response.md"), "utf8"); @@ -410,16 +415,13 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit assert.match(finalResponse, /done/u); assert.equal(result.payload.collect.caseRepoRunDir, registryRunDir); assert.equal(result.payload.collect.artifactManifestPath, path.join(registryRunDir, "artifact-manifest.json")); - assert.equal(result.payload.collect.agentStage.commands.some((item: any) => item.kind === "download"), true); + assert.deepEqual(result.payload.collect.agentStage.commands, []); assert.equal(result.payload.collect.caseRepoFiles.some((item: string) => item.endsWith("artifact-manifest.json")), true); const registryResult = JSON.parse(await readFile(path.join(registryRunDir, "result.json"), "utf8")); assert.equal(registryResult.collect.artifactManifestPath, path.join(registryRunDir, "artifact-manifest.json")); const summaryMarkdown = await readFile(path.join(registryRunDir, "summary.md"), "utf8"); - assert.match(summaryMarkdown, /## Agent HWPOD Raw Steps/u); - assert.match(summaryMarkdown, /download/u); - assert.match(summaryMarkdown, /uvoptx file not found/u); - assert.match(summaryMarkdown, /uart-read/u); - assert.match(summaryMarkdown, /hwpod_uart_monitor_not_active/u); + assert.match(summaryMarkdown, /traceLookupStrategy: id_plus_existing_cli/u); + assert.match(summaryMarkdown, /traceCommand: hwlab-cli client agent trace trc_case_/u); assert.doesNotMatch(summaryMarkdown, /downloadSkipped/u); const prompt = await readFile(path.join(cwd, ".state", "hwlab-cli", "caserun", "run-agent-flow", "agent-prompt.md"), "utf8"); @@ -444,7 +446,7 @@ test("case run orchestrates agent prompt, diff capture, and compile evidence wit assert.match(agentChatRequest.body.workspaceFiles[0].content, /HWLAB-CASE-F103.*caserun-run-agent-flow/u); assert.doesNotMatch(agentChatRequest.body.message, /path: "F:\\Work\\HWLAB-CASE-F103\\\.worktree\\caserun-run-agent-flow"/u); assert.equal(requests.some((item) => item.url === "http://web.test/v1/agent/chat/result/" + result.payload.agent.traceId), true); - assert.equal(requests.some((item) => item.url === "http://web.test/v1/agent/chat/trace/" + result.payload.agent.traceId), true); + assert.equal(requests.some((item) => item.url === "http://web.test/v1/agent/chat/trace/" + result.payload.agent.traceId), false); } finally { await rm(root, { recursive: true, force: true }); } @@ -587,6 +589,7 @@ test("case run status reports latest run stage when async control stage is stale assert.equal(result.exitCode, 0); assert.equal(result.payload.ready, false); assert.equal(result.payload.stage, "agent-diff-collected"); + assert.equal(result.payload.traceLookup.commands.trace, "hwlab-cli client agent trace trc_case --base-url http://web.test --render web"); assert.equal(result.payload.staleControlWarning.code, "stale_control_stage"); } finally { await rm(root, { recursive: true, force: true }); @@ -680,10 +683,6 @@ function caseRunFlowFetch(requests: any[]) { resultPolls += 1; return new Response(JSON.stringify({ ok: true, status: resultPolls > 1 ? "completed" : "running", reply: { content: "done" } }), { status: 200, headers: { "content-type": "application/json" } }); } - if (String(url).startsWith("http://web.test/v1/agent/chat/trace/")) { - const events = caseRunTraceEvents(); - return new Response(JSON.stringify({ ok: true, status: "completed", sourceEventCount: events.length, renderedRowCount: 0, traceSummary: { terminalStatus: "completed", agentRun: { runId: "run_agent_trace", commandId: "cmd_agent_trace" } }, events }), { status: 200, headers: { "content-type": "application/json" } }); - } if (body.ops?.[0]?.args?.argv?.includes("status") && body.ops?.length === 1) { return hwpodResponse([{ exitCode: 0, stdout: body.ops[0].args.argv.includes("--short") ? " M projects/01_baseline/main.c\n" : "" }]); } @@ -700,23 +699,6 @@ function caseRunFlowFetch(requests: any[]) { }; } -function caseRunTraceEvents() { - const events: any[] = [ - { seq: 1, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "cat .agents/skills/hwpod-cli/SKILL.md", message: "use `hwpod build` for compile", exitCode: 0 }, - { seq: 248, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod-ctl spec validate --spec .hwlab/hwpod-spec.yaml'", message: "validated", exitCode: 0 }, - { seq: 323, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod inspect --spec .hwlab/hwpod-spec.yaml'", message: "inspected", exitCode: 0 }, - { seq: 360, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod workspace insert-after --spec .hwlab/hwpod-spec.yaml --path projects/01_baseline/User/main.c --anchor ready --line marker'", message: "inserted marker", exitCode: 0 }, - { seq: 480, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod download --spec .hwlab/hwpod-spec.yaml'", message: "uvoptx file not found; program accepted job 20260605_213000_aaaaaaaa", exitCode: 1 }, - { seq: 520, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod uart read --spec .hwlab/hwpod-spec.yaml --port uart1 --max-bytes 4096'", message: "hwpod_uart_monitor_not_active on COM9", exitCode: 1 } - ]; - for (let index = 0; index < 31; index += 1) { - const seq = 400 + index * 2; - events.push({ seq, label: "item/commandExecution:started", type: "tool_call", toolName: "commandExecution", status: "started", command: "/bin/sh -lc 'hwpod build --spec .hwlab/hwpod-spec.yaml'", message: "commandExecution inProgress: /bin/sh -lc 'hwpod build --spec .hwlab/hwpod-spec.yaml'" }); - events.push({ seq: seq + 1, label: "item/commandExecution:completed", type: "tool_call", toolName: "commandExecution", status: "completed", command: "/bin/sh -lc 'hwpod build --spec .hwlab/hwpod-spec.yaml'", message: index === 0 ? "{\"job_id\":\"20260605_203835_798515c0\"}" : "build completed", exitCode: 0 }); - } - return events; -} - function hwpodResponse(outputs: Array<{ exitCode: number; stdout?: string; stderr?: string }>) { return new Response(JSON.stringify({ body: { results: outputs.map((output) => ({ ok: output.exitCode === 0, output })) } }), { status: 200, headers: { "content-type": "application/json" } }); } diff --git a/tools/src/hwlab-caserun-lib.ts b/tools/src/hwlab-caserun-lib.ts index b2234667..b46529e3 100644 --- a/tools/src/hwlab-caserun-lib.ts +++ b/tools/src/hwlab-caserun-lib.ts @@ -105,6 +105,10 @@ type AgentRunStage = { lastPollAt?: string; lastHttpStatus?: number; nextPollCommand?: string; + traceCommand?: string; + inspectCommand?: string; + sessionCommand?: string; + traceLookup?: Record; sessionResponse: Record | null; error?: Record | null; }; @@ -135,6 +139,9 @@ type AgentTraceStage = { traceId: string; status: string; httpStatus: number; + source?: string; + lookupOnly?: boolean; + lookup?: Record; sourceEventCount?: number; renderedRowCount?: number; commandCount: number; @@ -246,6 +253,7 @@ export async function runCaseRun(context: CaseContext) { prepare: prepared.summary, agent: agentSummary(agentStage.agent), agentTrace: traceSummary(traceStage.trace), + traceLookup: agentTraceLookupForRun(traceStage.run), agentDiff: diffSummary(diffStage.diff), build: built.summary, collect: finalCollect, @@ -358,6 +366,7 @@ export async function statusCaseRun(context: CaseContext) { registryArchive: registryArchive?.summary ?? null, prepare: run.subject?.worktreePath ? prepareSummary(run) : null, agent: run.agent ? agentSummary(run.agent) : null, + traceLookup: run.agent ? agentTraceLookupForRun(run) : null, agentDiff: run.agentDiff ? diffSummary(run.agentDiff) : null, evidencePath: text(run.evidencePath ?? control.resultPath) || null, nextPollCommand: `hwlab-cli case run status ${run.runId} --state-dir ${stateRoot(context)}`, @@ -381,6 +390,8 @@ export async function resultCaseRun(context: CaseContext) { stage: stage.stage, staleControlWarning: stage.warning, ready: false, + agent: run.agent ? agentSummary(run.agent) : null, + traceLookup: run.agent ? agentTraceLookupForRun(run) : null, nextPollCommand: `hwlab-cli case run status ${run.runId} --state-dir ${stateRoot(context)}`, logsCommand: `hwlab-cli case run logs ${run.runId} --state-dir ${stateRoot(context)} --tail 8000` }, "running"); @@ -406,6 +417,7 @@ export async function resultCaseRun(context: CaseContext) { registryArchive: registryArchive?.summary ?? null, summary: evidence ? buildSummary(evidence) : null, agent: run.agent ? agentSummary(run.agent) : null, + traceLookup: run.agent ? agentTraceLookupForRun(run) : null, agentDiff: run.agentDiff ? diffSummary(run.agentDiff) : null, evidence: context.parsed.full === true ? evidence : compactObject(evidence), result: context.parsed.full === true ? result : compactObject(result) @@ -503,6 +515,7 @@ export async function buildCaseRun(context: CaseContext, prepared?: PreparedCase subject: run.subject, agentTask: run.agentTask ? agentTaskSummary(run.agentTask) : null, agent: run.agent ? agentSummary(run.agent) : null, + traceLookup: agentTraceLookupForRun(run), agentTrace: run.agentTrace ? traceSummary(run.agentTrace) : null, agentDiff: run.agentDiff ? diffSummary(run.agentDiff) : null, hwpod: clean({ @@ -543,6 +556,7 @@ export async function collectCaseRun(context: CaseContext, prepared?: PreparedCa caseRepoFiles: [], artifactManifestPath: null, trace: null, + traceLookup: agentTraceLookupForRun(run, evidence), status: evidence.status, autoEvaluation: false }; @@ -684,6 +698,7 @@ async function runAgentTaskStage(context: CaseContext, run: PreparedCaseRun) { } const resultPath = text(accepted.body?.resultUrl) || `/v1/agent/chat/result/${encodeURIComponent(traceId)}`; const resultUrl = resultPath.startsWith("http") ? resultPath : `${baseUrl}${resultPath}`; + const traceLookup = agentTraceLookup({ baseUrl, traceId, sessionId, conversationId, threadId }); const submittedAgent: AgentRunStage = clean({ stageStatus: "submitted", baseUrl, @@ -702,7 +717,11 @@ async function runAgentTaskStage(context: CaseContext, run: PreparedCaseRun) { timeoutMs: effectiveAgentTimeoutMs(context, run.agentTask), pollIntervalMs: effectiveAgentPollIntervalMs(context, run.agentTask), resultUrl, - nextPollCommand: `hwlab-cli client agent result ${traceId} --base-url ${baseUrl}`, + nextPollCommand: traceLookup.commands.result, + traceCommand: traceLookup.commands.trace, + inspectCommand: traceLookup.commands.inspect, + sessionCommand: traceLookup.commands.sessionStatus, + traceLookup, sessionResponse: compactObject(session.body) }); run = await updateRun(context, run, { stage: "agent-running", agent: submittedAgent }); @@ -727,7 +746,11 @@ async function runAgentTaskStage(context: CaseContext, run: PreparedCaseRun) { resultUrl, lastPollAt: result.lastPollAt, lastHttpStatus: result.lastHttpStatus, - nextPollCommand: `hwlab-cli client agent result ${traceId} --base-url ${baseUrl}`, + nextPollCommand: traceLookup.commands.result, + traceCommand: traceLookup.commands.trace, + inspectCommand: traceLookup.commands.inspect, + sessionCommand: traceLookup.commands.sessionStatus, + traceLookup, sessionResponse: compactObject(session.body), error: result.error }); @@ -739,19 +762,13 @@ async function collectAgentTraceEvidence(context: CaseContext, run: PreparedCase const agent = run.agent; const traceId = text(agent?.traceId); if (!agent || !traceId) { - const trace: AgentTraceStage = clean({ traceId, status: "skipped_no_trace_id", httpStatus: 0, commandCount: 0, hwpodCommandCount: 0, hwpodBuildCommandCount: 0, keilJobCandidates: [], commands: [] }); + const trace: AgentTraceStage = clean({ traceId, status: "skipped_no_trace_id", httpStatus: 0, source: "caserun-identity", lookupOnly: true, commandCount: 0, hwpodCommandCount: 0, hwpodBuildCommandCount: 0, keilJobCandidates: [], commands: [] }); const nextRun = await updateRun(context, run, { agentTrace: trace, stage: "agent-trace-collected" }); return { run: nextRun, trace }; } - const traceUrl = `${agent.baseUrl}/v1/agent/chat/trace/${encodeURIComponent(traceId)}`; - const response = await caseFetchJson(context, traceUrl, { method: "GET", headers: caseAgentHeaders(context) }, "agent_trace_fetch"); - if (!response.ok) { - const trace: AgentTraceStage = clean({ traceId, status: "fetch_failed", httpStatus: response.status, commandCount: 0, hwpodCommandCount: 0, hwpodBuildCommandCount: 0, keilJobCandidates: [], commands: [], error: response.error }); - const nextRun = await updateRun(context, run, { agentTrace: trace, stage: "agent-trace-collected" }); - return { run: nextRun, trace }; - } - await writeJson(path.join(run.runDir, "agent-trace.json"), response.body); - const trace = summarizeAgentTrace(traceId, response.status, response.body); + const traceBody = agentTraceIdentityArtifact(run, agent); + await writeJson(path.join(run.runDir, "agent-trace.json"), traceBody); + const trace = summarizeAgentTrace(traceId, 0, traceBody); const nextRun = await updateRun(context, run, { agentTrace: trace, stage: "agent-trace-collected" }); return { run: nextRun, trace }; } @@ -823,6 +840,56 @@ async function submitAgentTask(context: CaseContext, input: { baseUrl: string; p }, "agent_task_submit"); } +function agentTraceLookup(input: { baseUrl: string; traceId: string; sessionId?: string; conversationId?: string; threadId?: string }) { + const traceId = input.traceId; + const base = input.baseUrl; + const commands = clean({ + result: `hwlab-cli client agent result ${traceId} --base-url ${base}`, + trace: `hwlab-cli client agent trace ${traceId} --base-url ${base} --render web`, + traceFull: `hwlab-cli client agent trace ${traceId} --base-url ${base} --render web --full`, + inspect: `hwlab-cli client agent inspect --trace-id ${traceId} --base-url ${base}`, + sessionStatus: input.sessionId ? `hwlab-cli client agent session status ${input.sessionId} --base-url ${base}` : undefined + }); + return clean({ + source: "hwlab-cli.client.agent", + strategy: "id_plus_existing_cli", + note: "CaseRun records trace/session identity and reuses existing HWLAB Web-equivalent agent CLI for trace/result/inspect; it does not implement a separate trace query path.", + baseUrl: base, + traceId, + sessionId: input.sessionId, + conversationId: input.conversationId, + threadId: input.threadId, + commands + }); +} + +function agentTraceIdentityArtifact(run: PreparedCaseRun, agent: AgentRunStage) { + const traceId = text(agent.traceId); + const lookup = agent.traceLookup ?? agentTraceLookup({ baseUrl: agent.baseUrl, traceId, sessionId: agent.sessionId, conversationId: agent.conversationId, threadId: agent.threadId }); + const resultBody = agent.result?.body && typeof agent.result.body === "object" ? agent.result.body : agent.result; + return clean({ + contractVersion: "hwpod-case-run-agent-trace-identity-v1", + source: "caserun-identity", + lookupOnly: true, + status: text(resultBody?.status ?? agent.stageStatus) || "recorded", + traceId, + sessionId: agent.sessionId, + conversationId: agent.conversationId, + threadId: agent.threadId, + providerProfile: agent.providerProfile, + projectId: agent.projectId, + resultStatus: text(resultBody?.status), + finalResponse: resultBody?.finalResponse ?? resultBody?.reply ?? null, + traceSummary: resultBody?.traceSummary ?? null, + terminalEvidence: resultBody?.terminalEvidence ?? null, + agentRun: compactObject(resultBody?.agentRun ?? resultBody?.traceSummary?.agentRun ?? null), + lookup, + hint: "Use lookup.commands.trace or lookup.commands.result to query the full Web-equivalent AgentRun trace with the existing HWLAB CLI.", + runId: run.runId, + caseId: run.caseId + }); +} + async function agentWorkspaceFilesForRun(run: PreparedCaseRun): Promise { const content = await readFile(run.specPath, "utf8"); return [{ path: ".hwlab/hwpod-spec.yaml", content, encoding: "utf8" }]; @@ -980,6 +1047,9 @@ function summarizeAgentTrace(traceId: string, httpStatus: number, body: any): Ag traceId, status: text(traceBody?.status ?? body?.status) || "recorded", httpStatus, + source: text(traceBody?.source ?? body?.source), + lookupOnly: traceBody?.lookupOnly === true || body?.lookupOnly === true ? true : undefined, + lookup: compactObject(traceBody?.lookup ?? body?.lookup ?? null), sourceEventCount: numberOption(traceBody?.sourceEventCount ?? traceBody?.traceSummary?.sourceEventCount ?? body?.sourceEventCount), renderedRowCount: rows.length || numberOption(traceBody?.renderedRowCount), commandCount: commands.length, @@ -987,7 +1057,7 @@ function summarizeAgentTrace(traceId: string, httpStatus: number, body: any): Ag hwpodBuildCommandCount, keilJobCandidates: uniqueStrings(Array.from(searchable.matchAll(/20\d{6}_\d{6}_[0-9a-f]{8}/giu)).map((match) => match[0])), terminalStatus: text(traceBody?.terminalStatus ?? traceBody?.traceSummary?.terminalStatus ?? body?.terminalStatus), - agentRun: compactObject(traceBody?.agentRun ?? traceBody?.traceSummary?.agentRun ?? body?.agentRun ?? null), + agentRun: compactObject(traceBody?.agentRun ?? traceBody?.traceSummary?.agentRun ?? body?.agentRun ?? traceBody?.terminalEvidence?.agentRun ?? null), commands }); } @@ -1164,7 +1234,7 @@ function agentTaskSummary(agentTask: PreparedAgentTask) { } function agentSummary(agent: AgentRunStage) { - return clean({ stageStatus: agent.stageStatus, baseUrl: agent.baseUrl, projectId: agent.projectId, providerProfile: agent.providerProfile, conversationId: agent.conversationId, sessionId: agent.sessionId, threadId: agent.threadId, traceId: agent.traceId, promptPath: agent.promptPath, promptSha256: agent.promptSha256, polls: agent.polls, timedOut: agent.timedOut, timeoutMs: agent.timeoutMs, pollIntervalMs: agent.pollIntervalMs, lastHttpStatus: agent.lastHttpStatus, error: agent.error }); + return clean({ stageStatus: agent.stageStatus, baseUrl: agent.baseUrl, projectId: agent.projectId, providerProfile: agent.providerProfile, conversationId: agent.conversationId, sessionId: agent.sessionId, threadId: agent.threadId, traceId: agent.traceId, promptPath: agent.promptPath, promptSha256: agent.promptSha256, polls: agent.polls, timedOut: agent.timedOut, timeoutMs: agent.timeoutMs, pollIntervalMs: agent.pollIntervalMs, lastHttpStatus: agent.lastHttpStatus, nextPollCommand: agent.nextPollCommand, traceCommand: agent.traceCommand, inspectCommand: agent.inspectCommand, sessionCommand: agent.sessionCommand, traceLookup: agent.traceLookup, error: agent.error }); } function traceSummary(trace: AgentTraceStage) { @@ -1172,6 +1242,9 @@ function traceSummary(trace: AgentTraceStage) { traceId: trace.traceId, status: trace.status, httpStatus: trace.httpStatus, + source: trace.source, + lookupOnly: trace.lookupOnly, + lookup: trace.lookup, sourceEventCount: trace.sourceEventCount, renderedRowCount: trace.renderedRowCount, commandCount: trace.commandCount, @@ -1418,11 +1491,27 @@ function prepareSummary(run: PreparedCaseRun) { } function buildSummary(evidence: any) { - return { status: evidence.status, autoEvaluation: false, agentTraceCommandCount: evidence.agentTrace?.commandCount ?? 0, agentTraceHwpodCommandCount: evidence.agentTrace?.hwpodCommandCount ?? 0, agentStage: agentStageSummary(evidence), hwpodExitCode: evidence.hwpod?.exitCode ?? null, hwpodSource: evidence.hwpod?.source ?? null, jobId: evidence.keilJob?.jobId ?? null, keilStatus: evidence.keilJob?.status ?? null, artifacts: evidence.artifacts ?? [] }; + return { status: evidence.status, autoEvaluation: false, traceLookup: evidence.traceLookup ?? null, agentTraceCommandCount: evidence.agentTrace?.commandCount ?? 0, agentTraceHwpodCommandCount: evidence.agentTrace?.hwpodCommandCount ?? 0, agentStage: agentStageSummary(evidence), hwpodExitCode: evidence.hwpod?.exitCode ?? null, hwpodSource: evidence.hwpod?.source ?? null, jobId: evidence.keilJob?.jobId ?? null, keilStatus: evidence.keilJob?.status ?? null, artifacts: evidence.artifacts ?? [] }; +} + +function agentTraceLookupForRun(run: PreparedCaseRun, evidence?: any) { + const agent = run.agent ?? evidence?.agent; + const traceId = text(agent?.traceId ?? evidence?.agentTrace?.traceId ?? run.agentTrace?.traceId); + if (!traceId) return null; + const baseUrl = text(agent?.baseUrl ?? run.agent?.baseUrl ?? evidence?.agent?.baseUrl) || runtimeWebUrl(run.definition) || DEFAULT_WEB_URL; + return agentTraceLookup({ baseUrl, traceId, sessionId: text(agent?.sessionId), conversationId: text(agent?.conversationId), threadId: text(agent?.threadId) }); +} + +function runtimeWebUrl(definition: Record) { + const runtime = definition.runtime; + if (!runtime || typeof runtime !== "object" || Array.isArray(runtime)) return ""; + return text((runtime as Record).webUrl); } function renderEvidenceSummary(evidence: any) { const agentStage = agentStageSummary(evidence); + const traceLookup = evidence.traceLookup ?? evidence.agent?.traceLookup; + const traceCommands = traceLookup?.commands ?? {}; const stageTable = agentStage.commands.length > 0 ? [ ``, `## Agent HWPOD Raw Steps`, @@ -1431,7 +1520,7 @@ function renderEvidenceSummary(evidence: any) { `|---|---:|---:|---|---|`, ...agentStage.commands.map((command: any) => `| ${command.kind} | ${command.status || ""} | ${command.exitCode ?? ""} | \`${markdownTableText(command.command, 160)}\` | ${markdownTableText(command.detail, 180)} |`) ].join("\n") : ""; - return `# HWPOD CaseRun ${evidence.caseId}\n\n- runId: ${evidence.runId}\n- status: ${evidence.status}\n- autoEvaluation: false\n- compileOnly: ${evidence.compileOnly}\n- subjectRepoLocalPath: ${evidence.subject?.repoLocalPath ?? ""}\n- subjectCommitId: ${evidence.subject?.commitId ?? ""}\n- subjectWorktreePath: ${evidence.subject?.worktreePath ?? ""}\n- agentTraceId: ${evidence.agent?.traceId ?? ""}\n- agentSessionId: ${evidence.agent?.sessionId ?? ""}\n- agentTraceCommandCount: ${evidence.agentTrace?.commandCount ?? 0}\n- agentTraceHwpodCommandCount: ${evidence.agentTrace?.hwpodCommandCount ?? 0}\n- agentTraceHwpodBuildCommandCount: ${evidence.agentTrace?.hwpodBuildCommandCount ?? 0}\n- agentStageCommandCount: ${agentStage.commands.length}\n- agentStageKinds: ${agentStage.kinds.join(", ")}\n- diffPatchPath: ${evidence.agentDiff?.diffPatchPath ?? ""}\n- runnerHwpodSource: ${evidence.hwpod?.source ?? ""}\n- hwpodExitCode: ${evidence.hwpod?.exitCode ?? ""}\n- jobId: ${evidence.keilJob?.jobId ?? ""}\n- runnerPostAgentCompileCheck: recorded\n${stageTable}\n`; + return `# HWPOD CaseRun ${evidence.caseId}\n\n- runId: ${evidence.runId}\n- status: ${evidence.status}\n- autoEvaluation: false\n- compileOnly: ${evidence.compileOnly}\n- subjectRepoLocalPath: ${evidence.subject?.repoLocalPath ?? ""}\n- subjectCommitId: ${evidence.subject?.commitId ?? ""}\n- subjectWorktreePath: ${evidence.subject?.worktreePath ?? ""}\n- agentTraceId: ${evidence.agent?.traceId ?? ""}\n- agentSessionId: ${evidence.agent?.sessionId ?? ""}\n- traceLookupStrategy: ${traceLookup?.strategy ?? ""}\n- traceCommand: ${traceCommands.trace ?? ""}\n- resultCommand: ${traceCommands.result ?? ""}\n- inspectCommand: ${traceCommands.inspect ?? ""}\n- agentTraceCommandCount: ${evidence.agentTrace?.commandCount ?? 0}\n- agentTraceHwpodCommandCount: ${evidence.agentTrace?.hwpodCommandCount ?? 0}\n- agentTraceHwpodBuildCommandCount: ${evidence.agentTrace?.hwpodBuildCommandCount ?? 0}\n- agentStageCommandCount: ${agentStage.commands.length}\n- agentStageKinds: ${agentStage.kinds.join(", ")}\n- diffPatchPath: ${evidence.agentDiff?.diffPatchPath ?? ""}\n- runnerHwpodSource: ${evidence.hwpod?.source ?? ""}\n- hwpodExitCode: ${evidence.hwpod?.exitCode ?? ""}\n- jobId: ${evidence.keilJob?.jobId ?? ""}\n- runnerPostAgentCompileCheck: recorded\n${stageTable}\n`; } function agentStageSummary(evidence: any) { @@ -1513,6 +1602,7 @@ function collectSummaryFromArchive(run: PreparedCaseRun, evidence: any, archive: artifactCount: archive.files.length + 1, skippedFiles: archive.skippedFiles, trace: archive.manifest.trace ?? null, + traceLookup: archive.manifest.traceLookup ?? agentTraceLookupForRun(run, evidence), agentStage: archive.manifest.agentStage ?? agentStageSummary(evidence), status: evidence.status, autoEvaluation: false @@ -1528,10 +1618,14 @@ async function writeReadableAgentArtifacts(caseRepoRunDir: string, run: Prepared const events = arrayOfObjects(traceBody?.events ?? traceBody?.trace?.events ?? traceBody?.sourceEvents); const rows = traceDisplayRows(traceBody ?? {}, events); const finalResponse = agentFinalResponse(run, evidence, traceBody, rows); + const traceLookup = traceBody?.lookup ?? evidence.traceLookup ?? agentTraceLookupForRun(run, evidence); const diffPatch = await readTextIfExists(evidence.agentDiff?.diffPatchPath ?? run.agentDiff?.diffPatchPath ?? path.join(run.runDir, "agent-diff.patch")); const messages = clean({ contractVersion: "hwpod-case-run-agent-messages-v1", renderer: CASE_TRACE_RENDERER, + traceSource: text(traceBody?.source), + lookupOnly: traceBody?.lookupOnly === true ? true : undefined, + traceLookup, traceId: evidence.agent?.traceId ?? run.agent?.traceId ?? traceBody?.traceId ?? run.agentTrace?.traceId, conversationId: evidence.agent?.conversationId ?? run.agent?.conversationId, sessionId: evidence.agent?.sessionId ?? run.agent?.sessionId, @@ -1615,6 +1709,11 @@ function renderAgentTranscript(run: PreparedCaseRun, evidence: any, messages: an `- sessionId: ${messages.sessionId ?? ""}`, `- threadId: ${messages.threadId ?? ""}`, `- renderer: ${CASE_TRACE_RENDERER}`, + `- traceLookupStrategy: ${messages.traceLookup?.strategy ?? ""}`, + `- traceCommand: ${messages.traceLookup?.commands?.trace ?? ""}`, + `- resultCommand: ${messages.traceLookup?.commands?.result ?? ""}`, + `- inspectCommand: ${messages.traceLookup?.commands?.inspect ?? ""}`, + `- lookupOnly: ${messages.lookupOnly === true}`, `- finalResponse: ${messages.finalResponse?.present === true ? "present" : "null"}`, `- autoEvaluation: false`, ``, @@ -1726,6 +1825,7 @@ function caseRunArtifactManifest(context: CaseContext, run: PreparedCaseRun, evi sourceRunDir: run.runDir, caseRepoRunDir: path.join(run.caseRepo, caseRegistryRunRel(run)), trace: caseRunArtifactTrace(run, evidence), + traceLookup: evidence.traceLookup ?? agentTraceLookupForRun(run, evidence), subject: evidence.subject ?? run.subject, agent: clean({ providerProfile: evidence.agent?.providerProfile ?? run.agent?.providerProfile, @@ -1767,6 +1867,9 @@ function caseRunArtifactTrace(run: PreparedCaseRun, evidence: any) { threadId: evidence.agent?.threadId ?? run.agent?.threadId, providerProfile: evidence.agent?.providerProfile ?? run.agent?.providerProfile, projectId: evidence.agent?.projectId ?? run.agent?.projectId, + source: evidence.agentTrace?.source ?? run.agentTrace?.source, + lookupOnly: evidence.agentTrace?.lookupOnly ?? run.agentTrace?.lookupOnly, + lookup: evidence.traceLookup ?? evidence.agentTrace?.lookup ?? run.agentTrace?.lookup ?? agentTraceLookupForRun(run, evidence), status: evidence.agentTrace?.status ?? run.agentTrace?.status, terminalStatus: evidence.agentTrace?.terminalStatus ?? run.agentTrace?.terminalStatus, commandCount: evidence.agentTrace?.commandCount ?? run.agentTrace?.commandCount,